EVAL-Kannual evaluation record
United States · Securities Filings Retrieval Laboratory

secrag Evaluation Record

Pursuant to nothing but engineering honesty: retrieval quality, latency and cost, measured on a quote-anchored golden dataset and versioned in git.

Item 1.

Headline result — recall@5, before and after

Same 52 answerable questions, three retrieval configurations. Hybrid fuses pgvector cosine search with Postgres full-text via reciprocal rank fusion; reranking re-orders the hybrid top-30 with a local cross-encoder (ms-marco-MiniLM-L6-v2, CPU).

Item 2.

Recall at k

Item 3.

Latency & cost, per stage

Wall-clock per query on CPU-only hardware — no GPU anywhere in the stack. The retrieval path calls no paid API: embedding and reranking are local models, so retrieval cost per query is $0. Generation cost is metered separately at the /ask endpoint.

Item 4.

Results by question category

Item 6.

Run register

Every run is a committed JSON file; this table is read from those artifacts.