What it is
Most AI question-answering does its work when the question arrives: find some passages, hand them to a language model, and let the model write an answer. Compile-Time Inference moves much of that work earlier. Your records are organised, linked to their sources and prepared before anyone asks anything.
When a question arrives, software selects a result from that prepared knowledge for the tasks it supports. A language model helps interpret the question and explain the result. The supporting evidence is available to inspect.
Two kinds of task
Evaluations go better when you separate two needs that are often blurred together. They are measured differently.
Swipe sideways to see all 3 columns.
| Task | The question you are asking | What to measure |
|---|---|---|
| Source-backed answers | Can I inspect the evidence behind this answer? | Whether cited sources exist, whether they support the actual claim, and what happens when they are missing. |
| Supported decisions | Does the same situation produce the same choice? | Whether the same interpreted inputs, records and rules select the same decision — and whether that decision is right. |
The video and genomics examples are source-backed answer applications. Decision support is described only for applications where it has been implemented and evaluated; the published studies are on the research page.
Prepare, keep current, answer
1. Prepare
Records are organised together with their sources and relationships. Which sources are in scope, who may see them, and which tasks are supported are agreed at the start of an evaluation.
2. Keep current
Preparation is not a one-off. Records and rules change, and yesterday’s answer may no longer apply. How updates are checked and made available — and how quickly — is part of the agreed scope. Updating has a cost; include it when you compare total cost.
3. Answer
For supported requests, a result is selected from prepared knowledge and explained, with its supporting evidence. When support is missing, the intended behaviour is to decline rather than fill the gap.
What the language model does
- Interprets the question. Everyday language is turned into a request the system can act on. Interpretation can be wrong, so it is worth testing on its own.
- Explains the selected result. The wording of an explanation may vary between runs and between models.
- Does not make the supported decision. For supported decision tasks, selection happens over prepared records and rules rather than being left to the model’s sampling.
An explanation is not a substitute for checking the record it points to. If you change the language model, validate interpretation and explanation again.
Missing and conflicting evidence
Every evaluation should include questions the system should not answer. Test, on your own material:
- Questions whose answer is not in the records.
- Questions where sources disagree, or where a record has been superseded.
- Questions outside the agreed scope.
- Questions that are ambiguous or phrased unusually.
Record how often the system declines correctly, declines when it should have answered, and answers when it should have declined. These rates matter as much as accuracy.
What “repeatable” means
Repeatability applies to supported decisions with the same interpreted inputs, authorised records and rules. Within that boundary, the same situation should select the same decision.
- New or corrected evidence can — and should — change a result.
- Interpreting a question is a separate step from selecting a decision.
- A repeatable decision does not mean identical wording.
- Repeatability does not show that the underlying data is correct, or that past outcomes were fair.
Integration and deployment
Compile-Time Inference is delivered through an assisted evaluation, not self-service sign-up. The integration method, authentication, permissions model and deployment boundary are agreed for your application and documented for you as part of that work. We do not publish a generic public API for it here, because we will not document an interface you cannot yet call.
Topics to raise early: where records and models run, who can access prepared knowledge, how source versions and permissions are reflected in answers, and whether any information reaches an external service. If you need models to run on infrastructure you control, see Helix.
How to evaluate it
Agree a test set and success criteria before you start. Use your own questions, including hard and unanswerable ones, and compare against your current system configured as well as you can configure it.
Swipe sideways to see all 3 columns.
| Measure | What it tells you | What it does not tell you |
|---|---|---|
| A citation resolves | The cited source exists and can be opened. | That the source supports the claim. |
| A source supports the claim | The evidence says what the answer says it does. | That the answer is complete or current. |
| The answer is correct | The result matches your reference answer. | That it will repeat, or that it generalises. |
| The decision repeats | Same inputs, records and rules select the same choice. | That the choice is correct or fair. |
| Coverage | The share of useful questions that get an answer. | Whether the declined questions should have been answered. |
| Refusal quality | Declines happen when support is absent — and only then. | Anything about answered questions. |
| Total cost | Preparation, updates, storage and serving together. | Value — that depends on what errors cost you. |
To see how published results are scoped, visit Evidence & evaluation. To plan an evaluation, get in touch — no confidential files are needed for the first conversation.
What is not published
These pages describe behaviour you can observe and test. They deliberately leave out how knowledge is extracted and prepared, how results are scored and selected, internal data layouts, and infrastructure detail. Customer records and prepared customer knowledge are never published.
Glossary
| Term | Meaning on this site |
|---|---|
| RAG | Retrieval-augmented generation: find relevant passages, then ask a language model to write an answer using them. |
| Prepared knowledge | Your records, organised with their sources and relationships before questions arrive. |
| Supported task | A kind of question or decision that has been scoped, prepared for and evaluated in your application. |
| Decline | An explicit response that there is not enough supporting evidence to answer. |
| Repeatable decision | The same interpreted inputs, authorised records and rules selecting the same choice. |