Why does RAG use embeddings instead of storing raw text for keyword search?

RAG uses embeddings because keyword search over raw text can only match exact or similar words, not meaning. Embeddings convert text into numbers that capture semantic content, so retrieval becomes a search for meaning instead of literal word matches. This makes RAG far more powerful and flexible at finding relevant context for a question.

The book directly addresses this question in the ingestion phase. Storing chunks as raw text in a standard database and using traditional keyword search would technically work, but the performance of the RAG system would be much lower. The reason is that keyword searches only find exact or similar words, not their meaning. Embeddings solve this by converting each chunk of text into a vector of numbers that represent its semantic content, including its intent, topic, and tone. When a user asks a question, it is also embedded into a vector, and the system calculates distances between the question vector and all chunk vectors. Chunks whose embeddings lie close to the question's embedding are considered semantically similar and are selected as context, even if they use completely different words than the question. In contrast, keyword search would miss useful chunks that express the same idea with different vocabulary. This shift from string matching to meaning-based matching is described as a big part of what makes RAG powerful.