Artificial Intelligence
Local reranking for your RAG with llama.cpp and pgvector, measured
A reranker rereads the passages pgvector returns and reorders them by relevance. With bge-reranker-v2-m3 served by llama.cpp on CPU, reranking 20 candidates raised nDCG@10 from 0.805 to 0.850 on 300 Spanish MIRACL questions; handing it 50 improved nothing and cost 2.7 times more tokens.
