Why the Embedding Model Is the Bottleneck
In a RAG system, the embedding model determines what gets retrieved. Everything downstream — the reranker, the prompt construction, the generation model — operates on the retrieval set that the embedding model selected. If the embedding model misses a relevant document or surfaces an irrelevant one, no amount of prompt engineering or generation-side sophistication will compensate.
Yet this is the component teams spend the least time evaluating. The default pattern: pick OpenAI’s text-embedding-3-large because the team already uses the OpenAI API, index the documents, and move on.
The embedding model is the foundation. The generation model is replaceable. Swapping a generation model is a configuration change. Swapping an embedding model requires re-embedding your entire corpus, rebuilding your vector index, and revalidating retrieval quality. The switching cost is high, which means the initial selection matters more than most teams realize.
The Leaderboard
The Massive Text Embedding Benchmark (MTEB) is the standard reference for comparing embedding models. As of March 2026, Gemini Embedding 001 leads the overall leaderboard at 68.32, with open-source models narrowing the gap to within a few points [S1].
The leaderboard is useful as a starting filter. A model that scores poorly on MTEB retrieval tasks is unlikely to perform well in your RAG pipeline. A model that scores well might.
That distinction matters. MTEB evaluates against standardized academic datasets — MS MARCO, Natural Questions, BEIR — that may bear little resemblance to your production corpus. None contain your company’s documentation, your domain’s terminology, or your users’ query patterns. Rankings also shift frequently. Optimizing for the current top-ranked model is chasing a moving target. Optimizing for a model that performs well on your data is a durable decision.
Why Benchmarks Lie
The most counterintuitive finding in recent embedding research challenges a reasonable assumption: that domain-specific embedding models outperform general-purpose ones on domain-specific tasks.
A clinical embedding study evaluated general-purpose and medical-specific embedding models on clinical retrieval tasks. The BGE general model outperformed the medical-specific models [S2]. The general model, trained on massive diverse corpora, had learned richer representations of language that transferred better to clinical text than the domain-specific models, which had been trained on narrower medical datasets.
This reflects a broader pattern: models trained on more diverse data often build more robust feature spaces, even for specialized tasks. A domain-specific model can overfit to its narrow training corpus, losing the generalization capacity that makes retrieval work across varied query formulations.
The practical implication: do not default to a domain-specific embedding model because your use case is domain-specific. Test general-purpose alternatives on your actual data. Weaviate’s guidance reinforces this — create 50-100 labeled examples from your production data and evaluate candidates against that set [S5]. MTEB scores should inform the candidate list. Your own evaluation set should make the final decision.
Dimensionality Tradeoffs
Embedding dimensionality directly affects three things: retrieval quality, storage cost, and search latency. Higher dimensions capture more semantic nuance. They also consume more memory, require larger indexes, and slow down nearest-neighbor search.
The question is where the quality curve flattens — the point at which additional dimensions add storage cost without meaningful retrieval improvement.
Recent systematic evaluation provides concrete guidance. PCA with 50% dimension retention — cutting a 1024-dimensional embedding to 512 dimensions — loses only 0.7-3.6% retrieval quality depending on the model and task [S3]. For most production use cases, a 1-3% quality reduction is imperceptible in end-user experience but cuts vector storage in half.
The more aggressive optimization is quantization. Float8 quantization achieves 4x storage reduction with less than 0.3% quality loss [S3]. This means a vector database consuming 100GB of storage at float32 precision can be compressed to 25GB at float8 with negligible impact on retrieval quality. For teams running large-scale RAG systems with millions of documents, this is not a minor optimization — it is the difference between a single-node and multi-node vector database deployment.
These techniques are complementary. A 1024-dimensional float32 embedding occupies 4,096 bytes. After 50% PCA reduction and float8 quantization, the same embedding occupies 512 bytes — an 8x reduction with combined quality loss in the low single digits. Treat dimensionality reduction and quantization as default optimization steps, not last-resort measures.
Cost Comparison
Embedding cost is a function of three variables: per-token API pricing, embedding dimensionality (which drives storage), and query volume (which drives both API calls and search operations). Most teams track only the first.
The price-performance spread across providers is wider than most teams realize. Voyage-3-large scored 10.58% higher than OpenAI’s text-embedding-3-large on retrieval benchmarks while costing 54% less per token [S4]. That is a model that is simultaneously better and cheaper than the market default. It is the kind of finding that only surfaces through systematic comparison — and that most teams miss because they default to whichever embedding API ships with their generation model provider.
The cost of not comparing is real. A team embedding 10 million documents at a higher per-token price, with a model that produces lower-quality retrievals, is paying more for worse results. The switching cost is high — re-embedding the full corpus — which is precisely why the comparison should happen before the first production index is built.
Storage cost scales with dimensionality. A corpus of 10 million documents at 1024 dimensions in float32 consumes roughly 40GB. At 3072 dimensions — OpenAI’s text-embedding-3-large default — the same corpus consumes 120GB. Total cost of ownership includes API pricing for indexing and queries, vector storage, and search compute. Optimizing only the API price while ignoring downstream storage implications is an incomplete analysis.
Practical Selection Process
The selection process that minimizes regret follows a structured sequence.
Step 1: Build an evaluation set. Create 50-100 labeled query-document pairs from your actual production data [S5]. Without this, you are selecting based on benchmarks that may not predict your domain performance.
Step 2: Establish a candidate list. Use MTEB rankings to identify the top 5-8 models on retrieval tasks [S1]. Include at least one open-source option, one proprietary API option, and any domain-specific model relevant to your vertical.
Step 3: Embed and evaluate. Embed your evaluation corpus with each candidate. Measure retrieval quality — recall@10, nDCG@10 — and rank by performance on your data, not on MTEB.
Step 4: Apply compression. Test 50% PCA and float8 quantization on the top model against your evaluation set [S3]. If quality holds, use the compressed version.
Step 5: Calculate total cost. For the top 2-3 models, compute full cost: API pricing, vector storage at chosen dimensionality, and search compute. The cheapest model per token is not necessarily the cheapest to operate.
This process takes a few days of engineering time. It produces a decision grounded in your actual data, your actual queries, and your actual cost structure. The alternative — defaulting to a well-known model and discovering its limitations after indexing millions of documents — takes longer to recover from.
The embedding model is the one component of your RAG system that is expensive to change after deployment. Invest the evaluation time upfront. The rest of the pipeline is more forgiving.