MTEB leaderboard:
best embedding models for RAG.

Given a query, find the most relevant documents from a corpus.

Classify text into categories using embeddings as features.

Group semantically similar texts into clusters without labels.

Given a query and candidate documents, reorder by relevance.

Predict the degree of semantic equivalence between sentence pairs.

Determine the relationship between two texts (duplicate, paraphrase, entailment).

Evaluate how well a summary captures the meaning of a source document.

Find translation pairs between two sets of sentences in different languages.

# Install
pip install mteb sentence-transformers

# Run full English benchmark
import mteb
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("Qwen/Qwen3-Embedding-0.6B")

# Run all English tasks (56+ datasets)
evaluation = mteb.MTEB(tasks=mteb.get_tasks(languages=["eng"]))
results = evaluation.run(model, output_folder="results/qwen3-0.6b")

# Or run specific task types
retrieval_tasks = mteb.get_tasks(
    languages=["eng"],
    task_types=["Retrieval"]
)
evaluation = mteb.MTEB(tasks=retrieval_tasks)
results = evaluation.run(model, output_folder="results/retrieval-only")

# Option 1: ceiling benchmark model (KaLM-Gemma3-12B — high memory)
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("tencent/KaLM-Embedding-Gemma3-12B-2511", trust_remote_code=True)
embeddings = model.encode(["What is machine learning?", "ML is a subset of AI."])
print(f"Similarity: {embeddings[0] @ embeddings[1]:.4f}")

# Option 2: practical baseline (Qwen3-0.6B — cheaper to run)
model = SentenceTransformer("Qwen/Qwen3-Embedding-0.6B")
embeddings = model.encode(["Hello world", "Bonjour le monde"])

# Option 3: Production serving with HuggingFace TEI
# docker run --gpus all -p 8080:80 \
#   ghcr.io/huggingface/text-embeddings-inference:latest \
#   --model-id Qwen/Qwen3-Embedding-0.6B

Normalised Discounted Cumulative Gain at rank 10. Measures how well the model ranks relevant documents in the top 10 results, with higher positions weighted more heavily.

NDCG@10 = DCG@10 / IDCG@10
DCG@10 = Σ(rel_i / log₂(i+1))

A score of 1.0 means all relevant documents appear at the top. Most models score 0.4–0.7, reflecting the difficulty of zero-shot retrieval.

Spearman rank correlation between model cosine similarities and human similarity judgments. Measures whether the model's relative ordering of sentence pairs matches human intuition.

ρ = 1 − (6 × Σd_i²) / (n(n²−1))
where d_i = rank difference for pair i

Spearman correlation of 0.80+ indicates strong alignment with human judgment. Top models now exceed 0.81.

Retrieve the right context chunks before generation. Embedding recall is often the first bottleneck in answer accuracy.

Find relevant documents by meaning, not just keyword overlap. Embeddings enable natural-language queries over large corpora.

Identify near-duplicate content, support tickets, or records using cosine similarity between embedding pairs.

Group documents by semantic similarity. Embeddings provide dense features for k-means, HDBSCAN, or topic extraction.

Sentence transformers

Models trained specifically for sentence and paragraph embedding. Run locally with full control.

• Optimised for retrieval, fast inference
• Many specialised variants available

• Fixed context length
• May need domain fine-tuning

LLM embeddings via API

Embedding endpoints from OpenAI, Cohere, Voyage, and others. Zero infrastructure to manage.

• High quality, long context
• No infrastructure to maintain

• Cost per token
• Data leaves your system

Sparse + dense hybrid

Combine BM25 with dense embeddings for better recall. Best of both worlds for production search.

• Handles exact matches well
• More robust for rare terms

• More complex pipeline
• Two indices to maintain

from openai import OpenAI

client = OpenAI()

def get_embedding(text: str, model='text-embedding-3-large'):
    response = client.embeddings.create(input=text, model=model)
    return response.data[0].embedding

documents = [
    'The cat sat on the mat',
    'A dog played in the park',
    'Machine learning is fascinating'
]

embeddings = [get_embedding(doc) for doc in documents]
print(f'Embedding dimension: {len(embeddings[0])}')

from sentence_transformers import SentenceTransformer
import numpy as np

model = SentenceTransformer('BAAI/bge-large-en-v1.5')

documents = [
    'The cat sat on the mat',
    'A dog played in the park',
    'Machine learning is fascinating'
]

embeddings = model.encode(documents, normalize_embeddings=True)

query = 'pets resting at home'
query_embedding = model.encode(query, normalize_embeddings=True)

similarities = np.dot(embeddings, query_embedding)
for doc, sim in zip(documents, similarities):
    print(f'{sim:.3f}: {doc}')