Oracle Powered RAG - Revisiting our Chatbot

Back in March, we demonstrated a chatbot that uses RAG to allow users to query the content of our blog using natural language. It made use of OpenAI’s API to access their embedding models and the GPT 3.5 turbo Large Language Model (LLM), and used Weaviate as the vector database provider.

Since then, Oracle have released Oracle Database 23 ai, which added the vector datatype to be used alongside all the existing features of an Oracle Database. This means that they can now function as vector databases, the heart of the retrieval system in a RAG application. There are no special steps to setting up a vector database, since vectors are simply stored as the new ‘vector’ datatype, and uploading vectors is as simple as inserting any other record into the schema.

With this piece of the puzzle in place, it’s now possible to construct an end-to-end RAG pipeline powered entirely by Oracle, and we have therefore rebuilt our chatbot to do just that. Our chunks of text data are now embedded using the OCI Generative Services Embedding API with the cohere.embed-english-v3.0 model, uploaded to an Oracle Database 23 ai schema, queried using its integrated vector search functions, and summarised using the cohere.command LLM from OCI Generative AI Services.

By why talk about it when we can show it? Check out this quick video demo of the new model in action. Note that while some query response times have been edited down for viewer quality of life, none of the calls to the model took longer than around 10 seconds to return a response.

By bringing all of this functionality into the Oracle ecosystem, we have been able to reduce the number of APIs and services required to create an end-to-end RAG solution, and made more complete use of our existing Oracle Cloud Infrastructure. For any business already using OCI, this means that the power of RAG can be leveraged alongside existing data workflows, integrating those tasks and capabilities rather than needing to manage another siloed data source.

If you’d like to know more about RAG and how it could help your business, or any of our other data science services, please feel free to contact us.