engineering · 1 min read
Query Rewriting: The Easiest 10-Point Recall Boost in RAG
Users ask short, ambiguous questions. Query rewriting makes them retrievable. Here is how Bhogar AI ships query rewriting with zero added latency.
BABhogar AI TeamProduct & Engineering
User queries are short, ambiguous and often missing context. The retriever sees "renew?" and has nothing to work with. Query rewriting expands that into a search-friendly form before retrieval.
Why it matters
The trick is doing rewriting cheaply. A full LLM call per query is too expensive at scale; the answer is a small, fast rewriter model invoked in parallel with the first retrieval pass.
How Bhogar AI approaches it
Bhogar AI runs a tiny rewriter model in parallel with the first retrieval call, then fuses the rewritten retrieval into the final answer. Net effect: 10-15 point recall lift with effectively zero added latency.
- Tiny rewriter model invoked in parallel
- HyDE-style hypothetical-document expansion as an option
- Per-KB rewriter on/off toggle
- Telemetry comparing rewritten vs raw recall
- Compatible with hybrid and graph retrieval
What you get
Across customer KBs, query rewriting lifts top-3 recall 8-15 points with no measurable user-perceived latency.