Judge a Context Packet by the Plan It Changes
A proposed evaluation for bounded, cited repository context.
Scope: Archive capabilities are website-described; semantic context and later roadmap capabilities are in development or planned. No delivery date is promised.
A context packet can be beautifully organized and still have no effect on the quality of an engineering plan. That is why KodeAtlas’s proposed next-phase experiment is not simply to build a larger graph of repository facts.
The roadmap points toward one archived TypeScript or JavaScript repository, connected code and historical evidence, and a bounded context packet for a concrete question. The useful test is whether the packet changes the resulting plan for the better.
This is an evaluation design, not a report of completed results or a currently qualified context product.
Start with a specific change
Imagine a synthetic task: replace a caching helper while preserving an interface used by two packages. A useful packet might include the current callers, the tests that exercise invalidation, and a review explaining why one consumer requires an explicit reset.
An unhelpful packet might include hundreds of related files and every historical mention of caching. Both packets contain information. Only one is shaped around the decision the engineer must make.
The question is not whether the model can read more. It is whether it can recognize the relevant constraint and use it without inventing additional obligations.
Compare plans, not just answers
An evaluation could compare plans produced with and without the packet, using the same task and a held-out review rubric. Did the plan include the affected consumer? Did it preserve the documented reset behavior? Did it request evidence for an unresolved relationship instead of treating a guess as fact?
These are proposed measures. They should not become marketing performance claims until the experiment has been conducted and the limitations are available for inspection.
Give uncertainty room in the packet
A bounded packet should have a budget, citations, and explicit omissions. An ambiguous lineage link may be important precisely because it cannot safely be resolved from the available evidence.
The packet should also explain its scope. A context view for an interface change is not a complete repository audit, and a relevant historical discussion is not proof that today’s tests protect the same behavior.
KodeAtlas’s development direction is toward a small, inspectable answer that improves an actual engineering decision. The graph and index are supporting machinery. The quality of the resulting plan is the reason to build them.