Context Engines Explained: And How to Build the Half That RAG Misses

You Used to Be the Context Layer

Think back to life before AI agents.

You join a new company. You get access to the repo, the docs, Slack, Jira. And then you spend weeks digging. Reading code you didn’t write. Searching wikis that are half out of date. Asking around to find out why something was built the way it was — the incident two years ago, the migration that went sideways, the decision someone made in a thread nobody can find anymore.

In that world, you were the context layer. Your head was the thing that connected the code to the docs to the history.

Now look at how a coding agent starts a task. Exactly the same way. It knows nothing about your codebase or your organization, so it starts grepping around, opening files, and slowly building a picture. Eventually it gives you an answer — or at least sounds confident that it has one.

So What Is a Context Engine?

A context engine takes all that scattered institutional knowledge — code, pull requests, review comments, tickets, docs, chat threads — and turns it into grounded context. That means context that is:

Connected: the PR links to the Slack thread that debated it, which links to the design doc behind it.
Intent-specific: you get what matters for this task, not everything ever written.
Personalized: it knows who’s asking.
Permission-aware: you only see what you’re allowed to see.

Instead of an agent exploring from zero, it gets a tight bundle of relevant context up front — with links it can follow if it needs more.

Why It Matters: Cost and Accuracy

In the talk that inspired this post, Peter from Unblocked showed two demos that make the case better than any definition.

Demo 1: Same task, with and without context. The agent was asked to plan an optimization for an internal component. Without a context engine, it explored the codebase on its own and produced a reasonable-looking plan. With a context engine attached, it first pulled a task-specific bundle: the source, the architecture docs, past PRs, and the Slack discussions about optimizing that exact component.

The difference was concrete. The run with context cost about $1.29 and took ~1.5 minutes. The run without it cost around $2.60 and took ~3 minutes. Roughly half the cost and half the time, with a better-grounded answer.

Demo 2: Debugging a silent regression. A teammate noticed their AI code-review product was surfacing fewer issues than before. He asked the context engine why. It traced the drop to a model version upgrade that changed the reviewer’s behavior. Then he asked it to fix it. It opened a PR whose description referenced the change that caused the regression and the Slack conversation where the team had discussed it.

That’s the point. Not just finding code — connecting the code to the history behind it.

Two Halves of Retrieval

Here’s where it gets interesting for builders.

Most of that scattered context is semantically searchable. Classic RAG is great at content questions:

Find the code that handles X
Find the PR that changed it
Find the Slack thread arguing about it

Wrap that in an agent loop with search tools and it goes a long way.

But a lot of questions aren’t about meaning. They’re about structure:

PRs I merged last week
Who reviews payments changes the most
Open PRs, grouped by repo

Those aren’t searches. They’re queries. The regression demo above actually started with one — a request for a weekly chart of a metric since January. You can’t pull that from a vector store, because a date range and a weekly aggregation aren’t encoded in an embedding.

	Semantic retrieval	Structured retrieval
Answers	”What is this about?"	"Which ones, how many, since when?”
Engine	Vector index	Query engine
Weak at	Time, counts, exact people	Fuzzy meaning

I covered combining both sides in Stop Giving Users Half-Answers. The rest of this post is about building the structured half: turning natural language into a safe database query.

Generating the Query Is the Easy Part

Any decent LLM can write a Mongo filter or a SQL WHERE clause. The hard part is the scaffolding around that call:

Schema: the model needs to know which fields and values actually exist.
Identity: “open PRs by Rashid” means nothing if Rashid is rsh-dev on GitHub.
Validation: the model’s output is untrusted. It could run something dangerous or cross a tenant boundary.
Self-correction: validation and the database will throw errors. The model should get a chance to fix them.

The goal is simple: go from “open PRs by Rashid” to a clean, safe, scoped query. Here’s how to build it, layer by layer.

Building the Structured Query Engine

Step 1: Discover the Schema Automatically

Hand-written schemas drift, and document stores often don’t have one at all. So discover it at runtime: sample a batch of documents and derive the fields and types from what’s actually there.

The tricky part is inferring enums. If a field like state only ever shows open, closed, and merged across hundreds of samples, it’s an enum — and telling the model that stops it from inventing values like "in progress".

from collections import Counter, defaultdict

HIDDEN = {"tenant_id"}  # fields the model must never see

def infer_schema(collection, sample_size=500, enum_max=12):
    docs = list(collection.aggregate([{"$sample": {"size": sample_size}}]))
    types, values = defaultdict(Counter), defaultdict(Counter)

    for doc in docs:
        for field, value in doc.items():
            if field in HIDDEN:
                continue  # never let tenant fields reach the prompt
            types[field][type(value).__name__] += 1
            if isinstance(value, str):
                values[field][value] += 1

    schema = {}
    for field, seen in types.items():
        entry = {"type": seen.most_common(1)[0][0]}
        if values[field] and len(values[field]) <= enum_max:
            entry["enum"] = sorted(values[field])
        schema[field] = entry
    return schema

Two details matter here. First, hidden fields like tenant_id are dropped during discovery, so they never end up in the schema you hand to the model. Second, this version only reads top-level fields. If your documents are nested (say author.login), flatten them into dotted paths while sampling, so the schema contains the same dotted keys the model will write in its filters.

Grab the collection’s indexes at the same time. They tell the model which filters are cheap — and later, whether full-text search is available.

You could throw an LLM at this like Thor’s hammer. You don’t need to. It looks fuzzy, but it’s deterministic work, and plain code handles it well.

Step 2: Resolve Identities

People use names. Systems use usernames. The same person is a different string in GitHub, Slack, and Jira.

Again, start with simple procedure: fuzzy-match the name against known aliases. Only if there’s a genuine tie, pass a short list of candidates to the LLM and let it act as a tiebreaker.

from difflib import SequenceMatcher

def resolve_user(name, directory, threshold=0.8):
    def score(person):
        return max(SequenceMatcher(None, name.lower(), a.lower()).ratio()
                   for a in person["aliases"])

    ranked = sorted(directory, key=score, reverse=True)
    if ranked and score(ranked[0]) >= threshold:
        return ranked[0]["github_login"]
    return llm_tiebreak(name, ranked[:5])  # only for ambiguous cases

Step 3: Synthesize the Query

Now the LLM call — and with the scaffolding in place, it’s the easiest step. Give the model:

the schema you discovered,
the current date and time, so “last week” becomes a real range,
the user who’s asking, so “my PRs” resolves correctly.

SYSTEM_PROMPT = """Turn questions about pull requests into a MongoDB query.
Schema: {schema}
Indexes: {indexes}
Now (UTC): {now}
Asking user: {github_login}
Only use listed fields and exact enum values.
Return JSON: {{"filter": {{...}}, "sort": {{...}}, "limit": <int>}}"""

Because infer_schema already stripped the hidden fields, tenant_id never appears in this prompt. The model can’t filter on a field it has never heard of.

Step 4: Validate Everything

This is the most important step.

Treat the generated query like user input. It might include an operator like $function, which executes raw JavaScript on the database server. It might touch fields you never meant to expose. In a multi-tenant system, it might reach into someone else’s data.

So: allowlist operators, block hidden fields, and inject tenant isolation yourself after validation. Stripping tenant_id from the schema keeps it out of the model’s view; the validator is the second lock, in case the model guesses the field name anyway.

ALLOWED = {"$eq", "$ne", "$in", "$gt", "$gte", "$lt", "$lte",
           "$and", "$or", "$exists", "$text"}
TEXT_ARGS = {"$search", "$language", "$caseSensitive", "$diacriticSensitive"}

class QueryRejected(Exception):
    pass

def validate(node, schema):
    if isinstance(node, dict):
        for key, value in node.items():
            if key == "$text":
                # $search here is $text's own argument, not an operator.
                if not isinstance(value, dict) or set(value) - TEXT_ARGS \
                        or not isinstance(value.get("$search"), str):
                    raise QueryRejected("Malformed $text query")
                continue
            if key.startswith("$") and key not in ALLOWED:
                raise QueryRejected(f"{key} is not allowed")
            if not key.startswith("$") and (key in HIDDEN or key not in schema):
                raise QueryRejected(f"Field '{key}' is unknown or restricted")
            validate(value, schema)
    elif isinstance(node, list):
        for item in node:
            validate(item, schema)

def scope_to_tenant(query_filter, tenant_id):
    return {"$and": [query_filter, {"tenant_id": tenant_id}]}

A note on $search: it isn’t in the allowlist on purpose. In MongoDB it only appears as the argument inside $text, which the validator checks separately. Top-level $search is an Atlas Search aggregation stage, not a find() operator. Operator names differ between databases, so build the allowlist from your own database’s docs, not from a generic list.

The schema check also compares keys exactly. If you flattened nested fields into dotted paths in Step 1, a filter on author.login passes. If you didn’t, it gets rejected.

Ask for “PRs with authentication in the title” at this stage and the model will probably reach for $regex. The validator bounces it. Safe — but now you just have an error.

Step 5: Let the Model Self-Correct

Instead of failing, loop the error back to the LLM and let it try again. Technically, it’s just a loop with a retry limit:

def run_question(question, ctx, max_attempts=3):
    messages = [{"role": "user", "content": question}]
    for _ in range(max_attempts):
        raw = llm(system=build_prompt(ctx), messages=messages)
        try:
            query = json.loads(raw)
            validate(query["filter"], ctx.schema)
            scoped = scope_to_tenant(query["filter"], ctx.tenant_id)
            return list(ctx.collection.find(scoped).limit(min(query.get("limit", 50), 200)))
        except Exception as err:
            messages += [{"role": "assistant", "content": raw},
                         {"role": "user", "content": f"Query failed: {err}. Fix it, JSON only."}]
    raise RuntimeError("No valid query after retries")

Run the authentication question again and it gets there in a couple of attempts. But there’s a catch: with regex off the table, the model “fixes” it with an exact title match. Valid query, zero results — no PR is titled exactly “authentication.”

Step 6: Add Full-Text Search

The missing piece is keyword search inside a structured query. Most databases support it natively:

collection.create_index([("title", "text"), ("body", "text")])

Since schema discovery picks up indexes, the model now knows a text index exists and uses $text instead of regex or an exact match. Run the question one more time and you get the PRs you actually meant.

What You End Up With

Put the six steps together and you have the other half of RAG: a query engine that handles filters, aggregations, time-scoped questions, and corrects itself when it gets something wrong.

Better still, wrap it as a tool for a chat agent. Ask “What auth-related work did we do last week?” and the agent generates the queries, pulls the data, and reasons over it — showing each query along the way.

Here’s where the LLM sits across the whole thing:

Schema discovery: code
Identity resolution: code, LLM only as tiebreaker
Query synthesis: LLM
Validation: code
Self-correction: LLM, driven by code’s errors
Full-text search: a database index

Rule of thumb: procedure first, LLM for the genuinely fuzzy parts, code as the final gate.

Try It Before You Commit

If you want to see the payoff on your own tasks before wiring up any platform, Unblocked also talked about a context engine simulator: it builds a context bundle per task, runs the same task with and without it, and shows you the A/B difference. It’s a cheap way to measure whether better context actually saves your agents time and tokens.

Most teams have built the semantic half. The structured half is where “which ones, how many, since when” lives.

Don’t just embed your context. Make it queryable.

This post is my write-up of a talk by Peter from Unblocked on context engines and structured retrieval. The code here is my own simplified take; Unblocked has open-sourced their document query engine and context engine simulator if you want the full versions.