RAG processing at scale

Summary

Application of generative AI has become more diverse. Ubiquitous chatbot RAG pattern is complemented by other scenarios.

One of the emerged patterns is processing large amounts of data (knowledge base) using multiple requests to GenAI model. For example large volume of legal or scientific documents can be processed in batch mode to extract relevant information as pertains to each document. This information would be stored in a database for further processing.

Approach

In this case for example in Databricks one could use user-defined functions to process data in batch mode. Each user-defined function is a RAG chain.

Complete code for this post is in GitHub 

Note: Other parts of this notebook which may find useful: referencing URL of the notebook workspace and getting Databricks session token