Production RAG Assistant — From Demo to Dependable
Taking a production RAG assistant from demo to dependable.
- •Hardened a production RAG assistant for a Top-5 global CPG client
- •Security assessment: surfaced a live auth bypass and exposed credentials
- •Retrieval & prompt improvements; ReAct agent optimization (~2.9x faster)
- •Went from external assessor to first merged contributor PR
Tap to enlargeIt’s easy to build a RAG assistant that works in a demo. Making one dependable in production is the hard part. Working with a Top-5 global CPG client inside a top-tier services company, I moved a production RAG assistant toward reliability — through security assessment, retrieval and prompt improvements, and architecture optimization — and earned the trust to ship changes into the client codebase myself. The screens below are anonymised illustrations of that work — the client’s own system and data are not shown.
Security First
Reliability starts with security you can prove, not assume
Before improving answers, I pressure-tested the system’s safety. The assessment surfaced critical gaps in a pre-production environment — a live authentication bypass and exposed credentials — escalated with clear, reproducible evidence so they could be closed before reaching production.
Better Retrieval, Faster Reasoning
Fix retrieval and orchestration — where the real failures hide
The real defects lived in retrieval and orchestration, not just prompts. Targeted retrieval and prompt improvements recovered previously-failing cases, and moving the assistant to a ReAct-style agent loop cut latency substantially (~2.9x faster) with no loss of answer quality.
Assessor to Contributor
Forward-deployed: embedded, trusted, accountable for the outcome
The engagement started as an outside assessment and ended with the first contributor pull request merged into the client’s own codebase — a shift from “telling them what’s wrong” to “shipping the fix with them.” That’s the forward-deployed mode: embedded, trusted, and accountable for outcomes.
Interested in building something similar?
Let's Connect