RAG

RAG pairs a generative model with a retrieval layer so answers come from retrieved evidence rather than the model's baked-in training data. I take retrievers seriously: chunking strategy, embedding choice, and the vector store all decide whether responses are grounded or confidently wrong.

The architecture I reach for separates the knowledge pipeline (ingest → chunk → embed → index) from the query path (embed → retrieve → rerank → synthesize). That boundary keeps retraining out of day-to-day updates — refresh the index, not the model.

Beyond simple top-k lookup I care about ranking quality and evals: hybrid search, rerankers, and a small eval harness that measures groundedness before any RAG endpoint ever reaches production.

Chunking is where most RAG systems quietly degrade. I tune chunk size, overlap, and splitting strategy to the document shape — prose, tables, code, contracts each want different boundaries — because retrieval quality is decided upstream of the model answering.

The embedding pipeline is treated as infrastructure: a clear data contract, versioned embedding models, and an index that can be rebuilt deterministically. I pair embeddings with keyword/hybrid retrieval so rare terms and exact identifiers do not get lost in semantic space.

Reranking is the step between raw retrieval and generation. A cross-encoder reranker over the top candidates materially lifts precision, and I wire it as an explicit, optimisable stage rather than trusting the vector store's raw ordering.

Groundedness is not a vibe — it is measured. I build a small eval set of questions with known-good citations and track retrieval hit-rate and answer groundedness as CI-style checks, so changes to chunking or embeddings are verified, not guessed.

The knowledge source matters as much as the pipeline: structured SQL, product APIs, and documents all feed RAG in my setups, and each is retrieved differently. The answer synthesizer is told exactly which sources it may cite and when to stay silent.

I view RAG as one pattern inside a larger AI architecture. It coexists with agentic loops and tool calling — retrieval feeding reasoning, not competing with it — and it is the piece I reach for whenever a model must speak to a user's own, current, private data.

I start RAG by defining what "grounded" must mean for the product. Before any embedding, I write the retrieval contract — what a correct answer must cite, when the model may say "I do not know", and how silence is handled.

Chunking is tuned to the document, not to a default. Prose, tables, contracts, and code each want different boundaries, and I size the chunk and overlap from the retrieval quality the evidence shows rather than a one-size rule.

The embedding pipeline is explicit infrastructure. I version the embedding model, keep a deterministic, rebuildable index, and treat it like a database migration — so retrieval stays reproducible.

I never trust raw top-k. A hybrid retriever plus a reranker over the top candidates is the step that lifts precision, and I tune it against a real eval set before any endpoint ships.

The eval set is the control room. A curated set of questions with known-good citations lets me measure hit-rate and groundedness as the pipeline changes, so a tweak to chunking or embeddings is verified, not guessed.

Sources stay typed. SQL, product APIs, and documents are retrieved differently and cited explicitly, and the synthesizer is told which sources it may use — so answers are grounded in the right place.

Freshness drives the design. Because the index is separable from the model, refreshing data means rebuilding the index, not retraining — and that operation is documented, scheduled, and observable.

The retrieval layer coexists with agents, not instead of them. I plug retrieval into an agentic loop where tool use and reasoning are the point, keeping grounding central rather than one-shot generation.

When a single store no longer serves scale or recall, I weigh a dedicated vector platform honestly against an in-DB index — the choice is workload-driven, not allegiance-driven.