RAGauge
Measure your RAG.
A retrieval-augmented QA system over SEC 10-K filings where the headline isn't the chatbot — it's the evaluation harness. Hand-verified golden set, retrieval and groundedness metrics, an ablation that proves each stage earns its complexity, real cost accounting, and a regression-blocking CI gate.
- 0.58recall@5
voyage-finance-2 - 1.00groundedness
supported-claim rate - 0.00unsupported
claim rate - $0.32per eval run
30-question sweep
- Deterministic retrieval ablation across three embedding strategies — no LLM in the loop, so the number is reproducible.
- A three-model judged sweep picks the generator by data: the cheapest tier won on groundedness and cost simultaneously.
- 10-K filings chosen deliberately as a hard target — adversarial HTML, huge financial tables, dense footnotes. 789 chunks, 76 offline unit tests green.
- Python
- RAG
- Embeddings
- LLM evaluation
- CI gates