What are embeddings in machine learning

Updated August 2026 · 5 minute read

An embedding is a vector, a long list of numbers, that a neural network produces to represent a piece of text, an image, or any other input, positioned so that similar meanings land at nearby points . That turns similarity into geometry: instead of comparing characters, you measure the distance or angle between vectors.

What an embedding is, precisely

An embedding model maps an input to a fixed-length vector of floating point numbers, typically hundreds to thousands of dimensions. The model is trained so that inputs with similar meaning produce vectors that sit close together and inputs with unrelated meaning produce vectors that sit far apart. The vector is the representation; nothing else about the input survives the mapping.

No single dimension means anything you could name. There is no coordinate for payroll or for hardware; meaning lives in where the whole vector sits relative to every other vector the same model produces. Inspecting one coordinate tells you as much as one pixel tells you about a photograph. The geometry is the meaning.

Closeness is measured with cosine similarity, the angle between two vectors, or with a dot product, and many models normalize their output so the two agree. A score near 1 means the model considers the inputs close in meaning; a score near 0 means unrelated.

What keyword matching misses

Take two bank transaction descriptions: ACH DEPOSIT PAYROLL ACME CORP, and Direct deposit, Acme payroll. To a keyword search they barely match; they share a single token. A person reads them as the same event, Acme paying wages into an account.

An embedding model sides with the person. Both descriptions land near each other in embedding space, because the model has learned from enormous amounts of text that an ACH deposit and a direct deposit are the same mechanism, and that casing, punctuation, and word order are surface details. Now add a third description: ACME HARDWARE PURCHASE. It shares a word with both of the others, enough for a keyword system to call it related, yet its vector lands far away. Money leaving an account for goods is a different meaning from wages arriving, and the space encodes that.

That reversal is the whole value. Keyword matching scores surface overlap; embeddings score meaning, so paraphrases, abbreviations, and bank-mangled uppercase strings stop being obstacles to finding things.

What embeddings are used for

One capability, nearest-in-meaning lookup, turns out to power most of the applied AI toolbox.

Semantic search: Embed the query and every document, return the nearest vectors. This finds the payroll pair above even when no words overlap at all.
Retrieval for RAG: Retrieval augmented generation fetches the passages whose embeddings sit nearest the question, then hands them to a language model to answer from. The retrieval half is embeddings.
Clustering: Vectors that bunch together are inputs that mean similar things, so unlabeled transactions group into merchants and spending patterns without hand-written rules.
Classification: Checking the labels of an input's nearest neighbors, or training a small model over the vectors, is often enough to categorize new items.
Deduplication and recommendation: Nearly identical vectors flag the same record written two ways; merely nearby vectors suggest items similar to what someone already chose.

How embeddings are produced, and why the model matters

Early word embeddings such as word2vec and GloVe assigned one fixed vector per word, so bank the institution and bank the riverside shared a single point. Transformer models replaced them with contextual embeddings: the vector for a sentence or document depends on all of its words at once, which is what lets a full transaction description encode an event rather than a bag of tokens.

Embedding models are not generation models. A generation model writes text; an embedding model only ever outputs a vector, and it is usually much smaller and cheaper to run. Pipelines tend to use both, embeddings to find the relevant material and a generation model to write with it, but they are separate tools with separate jobs.

The model choice fixes the space. Vectors from two different models are not comparable, even when their dimension counts happen to match, so every stored vector and every query vector must come from the same model. Switching models means re-embedding the entire corpus, which is worth budgeting for before the corpus gets large.

Clean input is most of the work

Embeddings inherit the quality of the text you feed them. A raw statement PDF embeds as a blur of headers, balances, and page furniture; one clean row per transaction, description and amount intact, embeds as a precise point worth searching. Producing those rows is the part LedgerBox handles: a REST API and an open source TypeScript SDK that extract structured rows from bank statements and similar financial documents, so what reaches your embedding model is data rather than layout.

Questions people ask

What is the difference between an embedding and a vector?

Every embedding is a vector, but the word embedding implies the vector was learned by a model so that distance encodes similarity of meaning. A vector of raw measurements, say temperature and humidity, is not an embedding; nothing trained its geometry to mean anything.

How many dimensions do embeddings have?

Current text embedding models output vectors of hundreds to thousands of dimensions, and the count is a property of the model, not the input. A five word phrase and a full page produce vectors of identical length from the same model.

Are embeddings only for text?

No. Images, audio, code, and product catalogs can all be embedded, and multimodal models place text and images in one shared space, so a text query can retrieve matching images directly.

Can I compare embeddings from two different models?

No. Each model defines its own space, so distances between vectors from different models carry no information even at matching dimension counts. Standardize on one model per collection, and plan to re-embed everything if you ever change it.

What is the difference between an embedding model and an LLM?

One represents, the other generates. You send an embedding model text and get numbers back for comparing things, at a fraction of the cost of generating text, which is why most pipelines pair the two rather than choosing between them.

Where do I store embeddings once I have them?

In memory as a plain array while the collection is small; a flat scan over a few thousand vectors is exact and fast. Past that point they go into a vector database, which is its own subject and its own article.