Similarity is not truth
Cosine scores rank candidates. They do not verify the facts inside a passage.
Test what your retriever remembers.
Measure source recall and the courage to stop. No answer generation, no hidden reranking, no invented confidence.
-ANSWERABLE RECALL
-UNKNOWN GATE
-CASES PASSED
-CONTEXT REDUCTION
Run the labeled benchmark to create real vectors and source rankings.
Waiting for local model
Retrieval is a testable subsystem.
Recall asks whether known evidence lands on the right source. Abstention asks whether missing evidence stops the pipeline. Both must be measured before fluent generation can hide the failure.
Thresholds are local
The bundled gate is a test fixture, not a universal constant. Calibrate it on representative positives and negatives.
Evaluation precedes scale
A small labeled set reveals wrong-source routing before thousands of files make failures harder to inspect.