p95 went from 2.1s to 9.4s the day i added a reranker, is that expected latency
Retrieval is pgvector, top 40 candidates, then a cross encoder reranking down to 8, running on CPU in the same container. Answer quality went up noticeably people stopped complaining about the wrong section being cited.…