How much does enabling RAG costs for your organization?
So you can know in advance how much your RAG project will cost.
tldr:
👉 Worst case scenario: 2.5$ per GB, or 10,000 pages/$.
👉 Real, average scenario: 1$ per GB, or 25,000 pages/$.
Here is a breakdown of that rule-of-thumb:
Say you have tons of internal data as word documents, Excel sheets, powerpoints…
You want an IA chatbot connected on your data, to speed up some of your processes.
To have a chatbot on your data, your need RAG (Retrieval Augmented Generation), which consist of providing access to your file to the IA.
To do RAG, you need to embed your data in vectors.
To embed your data, we need to read your files, extract the texts and images, chunk them into parts, and transform that into vectors.
The embedding model isn’t free. For example, let’s consider a 1GB file:
- A 1GB Word document has about 12.5 million words.
- A 1GB Excel file contains roughly 1 million rows.
- A 1GB PowerPoint mostly contains images but about 1 million words too.
Let’s take the worst case here, word documents, in order to have a upper bound estimation.
12.5 million words make ~17 million tokens (taking the upper bound here again). We increase this number by 15% because overlapping the data chunks improves accuracy, totaling about 20 million tokens.
Let’s take the most expensive embedding model out there, “text-embedding-3-large” from OpenAI.
It is priced at 0.13$ per million tokens.
Our 1GB of data, which gives us around 20M tokens, will cost 2.6$ to embed.
But remember, this is the most expensive case scenario, as we took the max amount of token for a GB of data, through the most expensive models.
If we use a slightly lesser model like text-embedding-3-small, we go down to 0.4$ for a GB of data.
These embeddings need to be stored. Embeddings are not big in comparison to the data they represent. My rule of thumb for embedding-to-data ratio is 0.05. So a 100GB of data will result in 5GB of vectors. But you have to add this little storage cost to the embedding process. Depending on your storage solution, this could vary.
Considering all factors, a safe estimate for embedding your data is about $1 per GB. This means embedding about 25,000 pages (since 1GB equals about 12.5 million words or approximately 25,000 pages) costs $1.