Neural search over scientific claims
A two-stage retriever: a MiniLM bi-encoder with an exact FAISS index finds candidates in milliseconds, then a cross-encoder rereads each claim–abstract pair to put the evidence first. Every number on this page was measured on the SciFact test set, not estimated.
Where the time goes
Median per-stage latency for one query with the default configuration (embedder on CPU, reranker on Apple MPS, pool of 20 candidates).
Reranking puts the right paper first more often
MRR@10 and nDCG@10 by configuration
Quality against latency
See what reranking changed
Real test claims, chosen automatically as the queries where reranking moved the first relevant paper the most, plus the worst regression. Green rows are papers the SciFact annotators judged relevant.
CPU or GPU depends on the stage
Apple's GPU makes the cross-encoder about three times faster, but makes encoding a single short query slower because dispatch overhead outweighs the arithmetic. The service therefore runs each model on the device that suits it.
Cross-encoder latency by candidate pool (p50)
Benchmark detail
What the measurements say
Run it locally
CPU works anywhere; Apple MPS and CUDA are used when present. The evaluation and benchmark scripts regenerate every number above.
# setup python3 -m venv .venv && source .venv/bin/activate pip install -r requirements.txt python -m scripts.fix_macos_openmp # macOS only: torch and faiss ship separate OpenMP runtimes # build, evaluate, serve python -m scripts.build_index # 5,183 abstracts → 384-d vectors → FAISS python -m scripts.evaluate # 300 test claims, dense vs. reranked python -m scripts.benchmark # per-stage latency, CPU vs. MPS uvicorn app.api:app --port 8000 # demo UI at /, OpenAPI at /docs pytest # 50 tests