Whitepaper · rag · 26 min read
RAG in Production: Patterns and Anti-patterns
Everything we learned shipping retrieval-augmented generation across regulated industries: chunking, hybrid search, reranking, citations, freshness and evaluation.
A practical guide to RAG that survives production: chunking strategies, hybrid search, reranking, citations, freshness, multilingual, evaluation and the ten anti-patterns that ruin most RAG projects.
November 24, 2025 · For AI Engineers - ML Platform Leads - Knowledge Engineers
When RAG is the right answer
RAG is right when the answer must be grounded, the corpus changes, and citation matters. RAG is not right when the corpus is small enough for the context window, the task is reasoning rather than recall, or the cost of the lookup outweighs the value of the answer.
Chunking: the most under-engineered step
Chunking decides retrieval quality more than embedding choice does. Use semantic chunking with overlap; preserve heading and table boundaries; emit a parent-document pointer so the reader can fetch context the chunk lacks. The blog post on parent-document retrieval expands this in detail.
Embeddings - pick boring, swap rarely
Pick a proven embedding model with strong multilingual support and low latency; swap rarely; rebuild indexes when you do. Cosine similarity is fine; do not chase MTEB ranks; benchmark on your corpus.
Hybrid search: dense + BM25
BM25 catches what dense vectors miss (rare terms, codes, numbers); dense catches paraphrases. Combine via reciprocal rank fusion or weighted sum; tune weights per corpus type.
Reranking is the cheap RAG win
A cross-encoder reranker on the top 50 candidates is the highest-use upgrade most RAG systems are missing. Costs little, lifts NDCG@5 noticeably, and reduces hallucination by surfacing better evidence.
Citations and provenance
Every answer cites every source it used; the citation is clickable; the source is rendered with a deep link or anchor. No citation, no answer - even when it costs a token. This is what survives audit and earns user trust.
Freshness and incremental indexing
Schedule incremental ingest; expose freshness in the citation; maintain a TTL per source class. Long-lived caches are tempting and dangerous.
Multilingual
Cross-language retrieval is solved with multilingual embeddings; cross-language generation is solved by prompting in the user language. Test both axes; do not assume English is enough.
Agentic RAG and GraphRAG
Agentic RAG: the model decides what to retrieve next based on what it just read. Useful for multi-hop questions; expensive; needs strict budget. GraphRAG: extract entities and relations into a graph; query the graph for structural questions; combine with vector for hybrid coverage.
Evaluation: metrics that correlate with satisfaction
Recall@k and citation correctness correlate with satisfaction. Generation quality without grounded citations does not. Track both; ship a regression dashboard; gate deploys on it.
Ten anti-patterns and the recovery pattern
For each anti-pattern (no eval, no citations, single retriever, fixed-size chunking, monolingual indexes, no reranker, full-document context spam, stale indexes, unsafe KBs, "vector DB will fix it") we ship a recovery pattern with an SLA.
More papers on the whitepapers index.