RAG does not ship itself

Most RAG projects die in a notebook. A retrieval demo looks intelligent on Tuesday. On Thursday an on-call engineer needs a severity, an affected service, and a sentence they can paste into Slack. The notebook has none of that.

We treat the language model as a component inside a service. Typed input, typed output, an abort path, and a trace you can open when it is wrong. Zod (or the equivalent) is not bureaucracy. It is how dashboards and tickets stay intact when the model improvises.

What production actually requires

A retrieval pipeline that never measures recall is a search box with extra steps. LangSmith (or your eval harness) is not optional theatre. You write a dozen golden cases that look like real incidents, you fail the build when they regress, and you ship only when the structured brief is stable.

Traces matter more than prompts. When a summarizer hallucinates a root cause, you need to see which documents were retrieved, which tool was called, and where the schema first broke. Without that, you are arguing with a black box in a war room.

The unfashionable parts

Rate limits, tenant isolation, and idempotent jobs. Streaming that does not leak keys. A mock mode so product can demo without a production secret. None of this is in the launch tweet. All of it is why the SRE team keeps the tab open.

If your vendor cannot point to evals, traces, and a contract for the JSON, you do not have an AI system. You have a demo. We ship the former.