If ordinary RAG has let you down, GraphRAG is probably on your shortlist. It is a real improvement for a specific problem. It is worth being clear about which problem, because adding a graph changes what the model reads, not who decides.
What GraphRAG adds
Standard RAG retrieves passages that resemble the question. That struggles when the answer depends on connections: which contracts reference this supplier, who reported to whom, what changed between versions. A knowledge graph stores entities and the relationships between them, so the system can follow those links and gather connected facts, or summarise whole clusters of related material.
GraphRAG is genuinely better at:
- Questions that need several connected facts at once.
- “Big picture” questions across a whole collection.
- Keeping related information together instead of in scattered chunks.
What it leaves unchanged
In most GraphRAG designs, the graph improves retrieval and then a language model writes the answer from what was retrieved. So the generation risks remain: the model can over-read, blend in what it learned in training, or fill gaps fluently. Answers can still differ between runs.
Two further points are easy to miss:
- The graph has to be built. Graphs are often extracted from documents by a language model. Extraction makes mistakes, and an error in the graph is an error the system will repeat with confidence. Ask how extraction is checked.
- The graph has to be maintained. When documents change, the affected parts of the graph must be updated. Rebuilding can be slow and costly for large collections. Ask what an update costs and how long it takes.
How Compile-Time Inference differs
Compile-Time Inference shares GraphRAG’s starting point: do structured work on your information before questions arrive. It differs at question time. For supported tasks, software selects a result from the prepared knowledge, and the language model’s job is to interpret the question and explain that result, with the supporting evidence available to inspect. When support is missing, the intended behaviour is to decline.
The practical differences:
- Deciding step. GraphRAG: the model, reading better context. Compile-Time Inference: selection over prepared records and rules, for supported tasks.
- Repeatability. GraphRAG answers can vary between runs. Compile-Time Inference aims for the same supported decision given the same inputs, records and rules — though wording may vary, and repeatable does not mean correct.
- Scope. GraphRAG will attempt most questions. Compile-Time Inference is scoped to agreed tasks and should decline outside them.
- Shared burden. Both must check what they extract and both must keep prepared knowledge current. Neither makes update cost disappear.
These describe typical designs. A GraphRAG system can add claim checks, refusal and fixed rules, and some do.
Which should you choose?
- Choose GraphRAG when the need is exploration and synthesis across connected material, and a person reviews the result.
- Evaluate Compile-Time Inference when answers must be checkable against a record, decisions must repeat under an agreed policy, and the system must say when it cannot answer.
- Keep plain RAG or search when questions are simple look-ups and the stakes are low. More machinery is not automatically better.
If you have already invested in a knowledge graph, that work is not wasted under either approach: the effort of defining your entities and relationships is the hard part, and it carries over.
Questions to ask any vendor — including us
- At answer time, what decides the result?
- How is extracted knowledge checked, and what is the error rate?
- What happens when the answer is not in the data?
- Will the same question give the same answer tomorrow? Under what conditions?
- What does an update cost, and how long until it is reflected?
- Can I see the measurements, with their scope and failure cases?
Our own answers, with their limits, are in the Compile-Time Inference documentation and on Evidence & evaluation. For the broader comparison, read RAG vs Compile-Time Inference; for diagnosing an existing system, Why RAG still hallucinates.