How SAP makes large language models work for its customers
Key Takeaways
⇨ SAP is combining LLMs with company-specific real-time data through a new feature in SAP HANA Cloud, vector engine
⇨ Vector engine turns data, images, sounds and more into a form that AI can process to find patterns or similarities
⇨ Vector engine establishes SAP HANA Cloud as SAP’s default database for its generative AI strategy
SAP has gone all-in on artificial intelligence, headlined by its generative AI offering Joule copilot, which is being added to several of its products.
Another one of SAP’s AI offerings is found within database-as-a-service SAP HANA Cloud, which combines the power of large language models (LLMs) with company-specific, real-time data.
The LLMs and company data are combined by a new SAP HANA Cloud feature, called vector engine, which was made generally available this month.
The LLMs SAP use include GPT-4, Llama2, Falcon-40b and Claude2, and the SAP HANA Cloud vector engine provides them with relevant organisation data and business process context.
In a blog post, SAP CTO Juergen Mueller and SAP HANA head of Database Stefan Baeuerle said while the LLMs offer “amazing” opportunities, they are also limited as they rely on outdated training data.
“Imagine having an LLM as a colleague. This colleague would be very intelligent, able to program, pass exams, or have arguments – but this colleague would not know anything about what happened in the world in the past year, nor have any idea about internal processes of your company or any of your systems,” the post read.
“Even worse, after every conversation you have, this colleague would forget what you just talked about. Working with such a lack of memory would be of limited value.”
“An LLM can only work with the initial training data – all other data must be provided as context.”
How the SAP HANA Cloud vector engine works
The vector engine enhances SAP HANA Cloud’s multi-model engines, enabling it to convert information like numerical and textual data, images, sounds and more into vector data, a form that AI can process to find patterns or similarities.
This also means developers can work with vector data in addition to relational, graph, spatial and JSON data, all within the same database, helping simplify architecture and improve performance.
Organisations can now also apply semantic and similarity search to business processes using documents like contracts, design specifications and service call notes.
SAP says this addition establishes SAP HANA Cloud as its default database for its generative AI strategy, with use cases in SAP BTP and more.
One use case is SAP BTP can provide centralised access to SaaS-based LLMs from multiple vendors as well as host LLMs from open-source models or third parties.
