The prototype worked. You pointed it at a few hundred documents, asked questions, and got answers with citations. Everyone in the room agreed this was the thing.
Then it went against the real corpus and the answers got worse in a way nobody could pin down. They still read well. They were still cited. They were just incomplete, and there was no way to tell which ones.
No amount of tuning fixes this. Retrieval behaves this way by design.
Rung one, retrieval
RAG answers a question by finding the passages most similar to it and handing them to a model. It works, and for a large class of questions it is the right tool.
It has one structural property that matters here. It returns the top k results. Not the results that matter, not all the results, the closest k of them.
Top k has no relationship to how many matches exist. Nine relevant passages or nine thousand, you get the same handful back. Nothing in the output tells you which case you were in.
Similarity is not relevance. The passage that phrases things closest to your question is not always the one holding the fact. The one that matters may use different words entirely.
Chunking cuts records apart. A finding split across a paragraph boundary becomes two chunks, each of which looks incomplete and neither of which retrieves well.
You cannot tell what it missed. This is the one that hurts. The output looks the same whether it found everything or a third of it, and there is no signal to check.
For a question with one answer, none of this bites. For a question with two hundred answers, all of it does.
Rung two, extraction
The job most people actually have is not question answering. It is enumeration. Find every instance of a thing across the corpus and put it in a structure.
Every reported variant and the evidence attached to it. Every clause of a given type across ten thousand contracts. Every measurement with its units and conditions. The value of the output depends on it being complete, which means the tool has to be built around completeness rather than around similarity.
That means reading everything rather than retrieving a slice of it. Which sounds expensive, and used to be prohibitive, and is now mostly an engineering problem. The harder part is what you do with what you read.
Extraction at scale is a pipeline, not a query. Documents get classified. The right extractor runs against each type. Every extracted value keeps a pointer to where it came from. Conflicts between sources get surfaced instead of averaged away. Low confidence output goes to a person rather than into the table.
The output is a structured corpus rather than an answer. That is a different asset, and a considerably more useful one.
Rung three, inference
Once the corpus is structured, the questions worth asking change.
Not what does this document say, which retrieval handles. Questions like where do these sources disagree, what has shifted over fifteen years of publication, which of these findings has never been replicated, what is missing from this body of work. No passage contains those answers. They exist across the structure, and you can only ask them once the structure is there.
This is what an inference platform does. Extraction is a component of it, not the point of it. The point is reasoning over a corpus with the completeness and the provenance to trust the result.
The gap between rung one and rung three is where most projects get stuck. They reach for retrieval, get a convincing demo, and then discover the demo does not become the system.
Where this came from
We did not arrive at this position from theory.
Our team built an extraction system against a scientific literature corpus, in a domain where a missed instance is not a ranking problem but a wrong answer with consequences. Retrieval was the obvious starting point and it broke exactly where described above. Completeness had to be designed in, provenance had to survive every stage, and disagreement between sources had to be surfaced rather than resolved silently.
What we learned building it became Lissajous, our inference platform. The extraction pipeline sits inside it as one layer, with the reasoning built on top.
An inference platform for document repositories at scale. A product of Pragmatic AI.
If this sounds like your problem
The signals are consistent. Your RAG prototype impressed everyone and then plateaued. You need every instance, not the best few. Someone is still checking output by hand because nobody trusts the completeness. The corpus keeps growing and the approach is not getting better with it.
The first thing to establish is what complete means for your corpus, and how you would know if you had it. That question is answerable in a conversation, and it usually reframes the project.