Why We Built a Corpus That Knows Its Own Answers

Scope: Engineering or methodology note based on a September 17, 2026 snapshot. Source-level behavior is not a public availability or universal compatibility claim. Any scenarios are illustrative, not customer outcomes.

A recovery demonstration is easiest to trust when the evaluator already knows what was deployed. That sounds obvious, but it changes how a test environment should be built. The recovery tool should not get to invent the answer after seeing its own output.

Korpus is our source-owned capability corpus. It gives us controlled applications, declared journeys, and artifact identities against which acquisition and recovery behavior can be investigated. It is distinct from the frozen Ushman corpus work and distinct from the products being evaluated.

Build the question into the fixture

Suppose a tiny application contains two requests with the same URL but different positions in a sequence. A useful fixture does not merely show a page that loads. It declares the sequence and the expected consequences. A replay that returns the first response twice should be distinguishable from one that preserves the intended interaction.

The same approach applies to runtime metadata, nested contexts, or a deliberate refusal case. The fixture should make the relevant behavior observable without handing the recovery engine a private answer file that it would not have in an ordinary capture.

Source integration and live evidence are different

A fixture committed to a repository is not automatically a live-verified deployment. Korpus records those states separately. Its September source snapshot contains both identity-bound live-provider evidence and source-integrated applications whose broader live evidence is not established there.

That distinction matters for product claims. A corpus application demonstrating a worker, storage mechanism, or realtime pattern does not establish KodeCapture support for that capability. It may instead be a pressure test for a future boundary or an example the current product must refuse.

Keep the evaluator independent

A corpus can still become too friendly. Reusing the same examples while tuning a recognizer risks teaching the implementation the evaluation set. Independent holdouts and explicit negative controls are therefore important design requirements, not decoration on a benchmark chart.

The supplied Korpus documents also name remaining validation gaps. We should preserve those disclosures rather than treat a fixture count as a universal quality score. More examples can improve pressure, but counting them does not prove that every relevant behavior was exercised.

Our goal is a useful disagreement: a fixture that can say the tool missed something, even when the tool reports success. That is what makes a corpus valuable to product development. It gives a promising implementation somewhere honest to be wrong before a customer depends on it.