Research
Insights on retrieval systems, language models, and building verified intelligence for financial workflows.
RAG evaluation suffers from Diagnostic Collapse: single scalar scores cannot distinguish retrieval failure from grounding failure. Models achieve 31% deflection rate when grounding is insufficient.
In 73-84% of multi-hop errors, correct evidence was already retrieved. The model failed to reason over it. Decomposition instability and error propagation compound with hop depth.
RAG systems exhibit Temporal-Semantic Mismatch where failures arise from multi-layer structural degradation. RAG mitigates, but does not resolve, the dynamic knowledge problem.
RAG assumes models prioritize retrieved evidence over parametric memory. Empirical studies show this fails in high-dynamicity domains like financial markets, where context conflicts with outdated priors.
Citation-based RAG aims to improve groundedness, yet citations do not guarantee claims are supported. This note explains why RAG systems still produce unsupported statements despite retrieving evidence.
Do long-context LLMs replace retrieval? Evidence shows complementary roles in knowledge-intensive AI. Larger context windows help models reason, but retrieval is what makes their output verifiable.
