What Are Vector Embeddings? How They Work, Cosine Similarity and a Python Semantic Search Example

A vector embedding is a list of numbers that represents the meaning of a piece of text (or an image, product or user). An embedding model is trained so that similar things get vectors that point in similar directions, which lets a computer measure "how related are these two things?" with simple arithmetic such as cosine similarity. That single trick powers semantic search, retrieval-augmented generation (RAG), recommendations, clustering and deduplication.

Why do we need embeddings?

Computers compare numbers well and words badly. To a string comparison, "reset your password" and "I forgot my login" share no words, so a keyword search says they are unrelated. A person knows they are about the same thing.

Embeddings fix this by mapping each piece of text to a point in a high-dimensional space where distance means difference in meaning. "Reset your password" and "I forgot my login" land close together; "shipping times for international orders" lands far away. Once text is a point, every question about meaning becomes geometry: nearest neighbours, clusters, angles.

The same idea works for anything you can train a model on: images, audio, code, products, users.

What do the numbers in an embedding actually mean?

An embedding for a sentence from a typical model looks like this:

[0.021, -0.113, 0.067, 0.004, ..., -0.059]   # 384 to 3072 numbers

No single number has a human-readable meaning. There is no "dimension 17 = about food". Meaning is spread across the whole vector, and only the relationships between vectors are interpretable: which ones are near each other, and in which direction they differ.

Two useful mental models:

• A location. Each text is a point; related texts are neighbours. Searching is "find the points nearest to my query point".
• A direction. Most models care about where a vector points, not how long it is. That is why cosine similarity (angle) is the usual measure, and why vectors are often normalised to length 1.

Typical sizes: open-source sentence models such as all-MiniLM-L6-v2 produce 384 dimensions; bge-base and similar produce 768; large hosted models produce 1024 to 3072. Bigger is not automatically better; it is more storage per vector and slower comparisons, in exchange for finer distinctions.

How does an embedding model learn?

Modern text embedding models are transformer encoders (BERT-style) trained with contrastive learning:

1. Take millions of pairs that should be close: a question and its answer, a title and its article, two paraphrases, a query and the result someone clicked.
2. Also take texts that should be far apart (random or hard negatives).
3. Adjust the model so that the pair's vectors move together and the negatives move apart, usually by pushing cosine similarity up for positives and down for negatives.

After enough of this, the model's output space has the property we want: semantic closeness equals geometric closeness, even for sentences it has never seen. Older word-level methods (word2vec, GloVe) learned a vector per word from co-occurrence; sentence-transformer models embed a whole passage with context, which is what you want for search and RAG.

How do you measure similarity between embeddings?

In practice: normalise every vector once when you create it, then use the dot product everywhere. It is the same ranking as cosine similarity and a single multiply-add per dimension.

import numpy as np

def cosine(a, b):
    return float(np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b)))

What counts as "similar" depends on the model. For one model 0.8 may be a near-paraphrase and for another 0.6 may be; the absolute number is not comparable across models, only the ranking within one model is.

Semantic search in Python in 20 lines

This runs on a laptop CPU with an open-source model. Install with pip install sentence-transformers numpy.

from sentence_transformers import SentenceTransformer
import numpy as np

model = SentenceTransformer("all-MiniLM-L6-v2")   # 384-dim, small, fast on CPU

docs = [
    "How to reset your password",
    "Refund policy for cancelled orders",
    "Setting up two-factor authentication",
    "Shipping times for international orders",
    "Changing the email address on your account",
]

# Embed once; normalise so the dot product equals cosine similarity.
doc_vecs = model.encode(docs, normalize_embeddings=True)   # shape (5, 384)

def search(query, k=3):
    q = model.encode([query], normalize_embeddings=True)[0]
    scores = doc_vecs @ q                                     # one dot product per doc
    top = np.argsort(-scores)[:k]
    return [(round(float(scores[i]), 2), docs[i]) for i in top]

for s, d in search("I forgot my login"):
    print(s, d)

Output (scores vary slightly by model version):

0.58 How to reset your password
0.40 Changing the email address on your account
0.31 Setting up two-factor authentication

The query shares no words with the top result. That is the whole point of embeddings.

Three details worth noticing: the documents were embedded once and reused; the query was embedded with the same model; and both were normalised so a plain matrix-vector product gives cosine similarity for every document at once.

Where are embeddings used?

Embeddings vs keyword search: use both

Embedding search is not a replacement for keyword search; it is a complement.

Production systems use hybrid search: run both, merge the ranked lists (reciprocal rank fusion is the common method), and optionally pass the top candidates through a reranker, a cross-encoder model that reads query and document together and scores them more accurately than a dot product can.

Do you need a vector database?

Only at scale. A rough guide:

• Up to ~50k vectors: a NumPy array in memory and a dot product. Milliseconds. No infrastructure.
• Up to a few million: an approximate nearest neighbour (ANN) index such as FAISS or HNSW (used by Chroma, Qdrant, Weaviate, pgvector). ANN trades a little recall for queries that stay fast as the collection grows.
• Already running PostgreSQL: the pgvector extension keeps vectors next to your rows, with normal SQL filters and joins.
• Large, multi-tenant, or no ops team: a managed service (Pinecone, Qdrant Cloud, Weaviate Cloud, MongoDB Atlas Vector Search).

Whichever you pick, store the original text and metadata alongside each vector (source, title, date, permissions). You will need them to build the prompt, show citations and filter results.

How to choose an embedding model

• Check a benchmark, then test on your data. The MTEB leaderboard ranks models on retrieval, clustering and classification. Use it to shortlist, then evaluate on 50 to 100 real queries from your own domain; rankings shift between domains.
• Open source vs API. Open models (all-MiniLM, bge, e5, nomic-embed, gte) run locally, cost nothing per call and keep data in-house. Hosted models (OpenAI, Google, Cohere, Voyage) are stronger on many benchmarks and zero-maintenance, but every embedding is a network call and a line on a bill.
• Dimensions and storage. One million 1536-dimension float32 vectors is about 6 GB before any index. Smaller models, int8 quantisation or truncated dimensions cut this a lot.
• Context length. Models truncate input past their limit (often 512 tokens for small models, 8k for larger ones). Anything beyond the limit is silently ignored, which is one reason documents are chunked.
• Language. If your users write in Hindi or a mix of languages, pick a multilingual model and test it; English-only models degrade sharply.
• Query vs document prefixes. Some models (e5, bge, nomic) expect a prefix such as "query: " or "passage: ". Skipping it costs real accuracy.

Mistakes that quietly ruin embedding results

• Mixing models. Vectors from two models, or two versions of one model, are not comparable. Re-embed everything when you switch.
• Not normalising when using the dot product. Long texts get longer vectors in some models and win every search.
• Embedding whole documents. A 30-page PDF squeezed into one vector matches everything a little and nothing well. Chunk into passages of a few hundred tokens.
• Chunks with no context. A chunk that says "It defaults to 30 days" is unsearchable. Prepend the document title or section heading to each chunk before embedding.
• Trusting the score threshold. A fixed cutoff like 0.7 means different things for different models and domains. Calibrate on labelled examples.
• Ignoring metadata filtering. "Documents this user may see, from this year" should be a filter, not something you hope the vector encodes.
• No evaluation set. Without labelled query-to-passage pairs you cannot tell whether a new model, chunk size or prefix helped.

Key takeaway

An embedding is meaning turned into coordinates. Train a model so similar things point the same way, normalise the vectors, and a dot product tells you how related any two items are. Use embeddings for semantic search and retrieval, pair them with keyword search for exact matches, chunk documents sensibly, never mix models, and measure on your own queries. For how retrieval plugs into an LLM, read What is RAG?; for the surrounding ML concepts on one page, download the free AI/ML and GenAI handwritten notes. If you need semantic search or a RAG system built on your own data, see AI engineering services.