RAG systems and AI agents · Retrieval grounding · Guardrail architecture
At a glance
Why this problem is hard right now
Nobody asks a training company’s assistant for its opinion. They ask when the next class runs, what it costs, whether a seat is left, and whether the code they were given still works.
Each of those has a correct answer that changes week to week, and a wrong answer that costs a booking or a refund. A general model will answer all four fluently, and some of them wrongly, and the customer has no way to tell those apart. Fluent and wrong is the worst output this system can produce, worse than a refusal, because a refusal sends someone to a human while a confident fabrication closes the conversation.
This is the specific shape of the retrieval problem in commerce. It is not a knowledge task where approximately right is useful. It is a lookup task wearing conversational clothes, and the value of the conversation collapses to zero the moment the underlying facts are stale.
So the interesting engineering was never the dialogue. It sits underneath: where the facts come from, how old they are allowed to be, what the assistant is structurally prevented from saying, and what the system does when the data behind it goes bad.
The situation
The organisation sells scheduled certification training with real dates, real prices and finite seats, across a main site and more than thirty separate landing pages. Prospects arrive at all hours and ask the same narrow set of pre-purchase questions.
The commercial context made a naive assistant actively dangerous. Seat availability is competitive information. Discount codes exist in families, some public and some not. Course events are duplicated across two countries with different pricing. Zero-value orders occur and mean something specific operationally.
An assistant that answered helpfully across all of that, without constraint, would leak commercial information, promise discounts that do not apply, and quote the wrong country’s event. None of those are model quality failures. They are all failures to encode the business.
What organisations in this position usually get wrong
The default approach is to put the rules in the prompt and the catalogue in the context window, then test the happy path and ship it.
That fails in three predictable ways.
First, prompt instructions degrade. A rule stated once in a long system prompt competes with the model’s much stronger prior toward being helpful. Ask the right question in the right way and the assistant will reason its way to disclosing the thing you told it not to disclose, because you gave it an instruction rather than removing the capability.
Second, a catalogue pasted into context is a snapshot that begins ageing immediately. It looks correct in testing because testing happens the day it was pasted.
Third, and most commonly, the system has no concept of its own data being stale. It reports healthy because the service is responding. Nobody built a signal for the state where the application is perfectly available and confidently wrong, which is precisely the state that damages a business.
How we approached it
We grounded answers in operating data rather than in a trained snapshot. A scheduled job pulls current course and schedule records at 23:00 each night and loads them into the production database the assistant reads from. Class answers come from those records. The assistant does not answer from whatever the model absorbed about this catalogue during training, and it does not answer from a document pasted into a prompt three weeks ago.
We wrote the commercial rules as constraints rather than guidance. Seat counts are never revealed. Only one designated coupon family is ever disclosed. Coverage stops at two countries. Zero-value orders route to a human rather than being handled conversationally. Competitor recommendations are off the table.
The distinction that matters: these are not tone preferences the model weighs against helpfulness. They are properties of the system. A model cannot be argued into disclosing something the architecture does not hand it.
We solved a data ambiguity in the rules, not in the prompt. The catalogue carries duplicate events for two countries, distinguished only by date format, slashes for one and hyphens for the other. That is exactly the class of subtlety a model gets quietly wrong at volume: correct in testing, wrong two percent of the time in production, and invisible until a customer books the wrong country’s class. It became an explicit rule in the data layer rather than a line of guidance the model might weight lightly.
We treated stale data as an outage. This is the part worth stealing for any retrieval system.
The health endpoint does not simply report that the service is up. It reports how old the underlying course data is. It warns at 25 hours and fails outright at 30. And if the assistant can see no classes at all, that is a P0 incident, not an empty result set.
The reasoning is that an assistant answering confidently from week-old schedule data does more commercial damage than one that is switched off. If the retrieval layer is stale, the product is broken even though every service is green. So staleness was given a threshold, an alarm and a severity, the same as any other outage.
We kept the sending identity off the brand domain. Assistant-generated email goes out on a dedicated subdomain rather than the primary domain, so that nothing automated can put the main sending reputation at risk. An AI system that can email customers is an AI system that can damage deliverability, and that risk should be contained to a domain you can abandon.
We separated the cloud application from the data worker. The assistant runs as a cloud application. The pipeline that feeds it runs as a separate worker with its own repository. They fail independently, they deploy independently, and the boundary between them is a database rather than a shared codebase.
The call we had to make
The pipeline that keeps the assistant current runs on a local machine rather than in the cloud. It drives a browser session against a legacy administration system that has no API.
That is a single point of failure and we have never dressed it up as anything else. It is recorded as one in the operating notes rather than buried. If that machine stops, the assistant’s course knowledge begins ageing immediately.
The alternative was to wait for an API that does not exist, on a system the client does not control, and ship nothing until it did. We took the view that a working assistant with a known fragile input beats no assistant with a clean architecture diagram.
What made that acceptable was where we put the mitigation. The weak link is the pipeline, so the health model was built specifically to detect and escalate the exact failure that link produces. The system does not assume the pipeline holds. It assumes the pipeline will eventually fail and makes that failure loud, visible and severity-rated within hours.
That is a different engineering posture from hoping, and it is the one we would defend in any architecture review.
What the engagement could not fix
We have no measured conversion, deflection or containment figures for this assistant. None were instrumented, so none are claimed here. What can be evidenced is what was built, what it is structurally prevented from doing, where its facts originate, and how it reports its own failure.
The legacy administration system remains without an API, so the scraping dependency remains. It is mitigated, not removed. Removing it requires a change on a system outside the engagement’s control.
The assistant is also bounded to its remit by design. It is not a general support agent, and questions outside the pre-purchase set are routed to humans rather than attempted. Some organisations would consider that a limitation. We consider it the correct scope for a system whose wrong answers cost money.
What transfers
The chatbot is the least portable thing here. The architecture is the asset.
Three habits carry into any enterprise retrieval or agent deployment where being wrong has a cost.
Ground answers in operating data, not model memory. If the answer changes weekly, the model must never be the source of it. The model’s job is to converse about facts the system hands it.
Express policy as constraints the model cannot argue with. Anything you would be unwilling to see disclosed should be structurally unavailable, not merely discouraged in a prompt. The test is simple: if a determined user could talk the assistant into it, the rule was guidance rather than a guardrail.
Treat data freshness as a health metric with a threshold and an alarm. Most AI monitoring watches latency, errors and cost. Almost none watches whether the retrieved facts are still true. That is the failure mode that reaches customers, and it is invisible on a conventional dashboard.
For a regulated buyer the last one is usually the point at which the conversation changes. Every vendor can describe a RAG pipeline. Very few can show you the alarm that fires when the pipeline silently goes stale, or tell you the severity they assigned to it.
Where it stands
The assistant is live across the main site and the landing page estate, answering from nightly-refreshed course records, inside its commercial guardrails.
The known fragile dependency remains the known fragile dependency, and it is the failure the system is best instrumented to catch.