How to Build a Semantic Search App (Step by Step)

// In plain English

Ordinary search matches words. Type "car" and it finds documents that contain the letters c-a-r. Ask for "automobile" and it shrugs, even though it's the same thing. Semantic search matches meaning instead. It understands that "car", "automobile", and "vehicle" are close, that "refund window" and "how long do I have to return this" are asking the same question, and it ranks results by how related they feel, not by which exact words overlap.

The trick is to turn every piece of text into a list of numbers — an embedding — where similar meanings produce similar numbers. Picture a giant library where books aren't shelved alphabetically but by topic: cookbooks cluster in one corner, war histories in another, poetry in a third. To answer a question you don't read the spine, you walk to the right neighbourhood. A semantic search app is exactly that, except the "neighbourhood" is a point in a few hundred numeric dimensions, and finding nearby points takes milliseconds.

This article is a build guide, not a concept explainer. If you want the theory first, read What Is Semantic Search?. Here we wire up the four moving parts — an embedding model, a vector store, a similarity query, and a ranked results view — into something you can actually run today, in both Python and TypeScript.

// Why it matters

Keyword search has a quiet failure mode: it only works when the user happens to type the same words the author used. Real people don't. They paraphrase, misspell, use synonyms, and ask in full sentences. Every one of those is a missed match for a keyword index, and a missed match is a user who concludes "this product has nothing useful" and leaves.

• Synonyms and phrasing. "Cancel my subscription" should find a doc titled "Ending your plan." Keyword search won't; semantic search will.
• Natural-language questions. Users increasingly ask rather than type keywords. Semantic search is built for "how do I get a refund on a digital purchase", not just the word "refund".
• Messy, unstructured data. Support tickets, PDFs, meeting notes, and wiki pages have no tidy tags. Embeddings give you search over them with zero manual labelling.
• The foundation for AI features. Once you can retrieve by meaning, you're one step from "chat with your docs", recommendations, deduplication, and clustering — they all sit on the same vector machinery.

For a builder, the appeal is the effort-to-payoff ratio. A useful semantic search prototype is a few dozen lines of code and one API call per document. There's no model to train, no labelled data to collect, and no taxonomy to design. That's why it's one of the best first AI projects — small enough to finish in an afternoon, real enough to ship.

// How it works

A semantic search app has two phases. Indexing runs once, ahead of time: you prepare every document so it's searchable. Querying runs on every search: you find the closest matches and show them. The same embedding model is used in both phases — that's what makes the question and the documents comparable.

You can't embed a 100-page manual as one vector — you'd get a single blurry "average meaning" that matches nothing well. So you split each document into short passages (chunking) and embed each passage on its own. A good starting point is roughly 200–500 words per chunk, with a small overlap (say 50 words) so a sentence split across a boundary still appears whole in one chunk. Then you store each vector alongside its original text in a vector database.

At query time you embed the user's search text with the same model, then ask the index: which stored vectors point in the most similar direction? "Similar direction" is measured by cosine similarity — the cosine of the angle between two vectors, a number from -1 (opposite) to 1 (identical). You sort by that score, take the top k (often 5–10), and display those passages. There is no LLM in the loop; the embedding model and a sort do all the work.

Why you normalize vectors

Cosine similarity only cares about direction, not length. If you first normalize every vector to length 1 (divide it by its own magnitude), then cosine similarity becomes a plain dot product — a single fast multiply-and-add. Most embedding APIs return already-normalized vectors, but normalizing yourself is cheap insurance and lets you use the dot product everywhere. The payoff: identical ranking, simpler and faster code.

// Build it: a working app in Python

Here is the whole thing in one file — no vector database, just NumPy, so you can see every step with nothing hidden. Swap get_embeddings for any embedding API (Voyage AI, OpenAI, Cohere, or a local model via sentence-transformers).

import numpy as np

# 1) INDEXING — your corpus, already split into short passages.
docs = [
    "To cancel your subscription, open Settings and choose End plan.",
    "Refunds on physical items are accepted within 30 days of purchase.",
    "Digital goods are non-refundable once they have been downloaded.",
    "Support hours are 9am to 6pm Eastern, Monday through Friday.",
    "Reset your password from the login screen via Forgot password.",
]

def embed(texts):
    # Returns one vector per text. Use any embedding provider here.
    vecs = get_embeddings(texts)            # -> np.ndarray (n, dim)
    # 2) NORMALIZE so cosine similarity == dot product.
    return vecs / np.linalg.norm(vecs, axis=1, keepdims=True)

doc_vecs = embed(docs)                       # embed every passage once.

def search(query, k=3):
    # 3) QUERY — embed the query with the SAME model.
    q = embed([query])[0]
    # 4) SCORE — dot product of normalized vectors = cosine similarity.
    scores = doc_vecs @ q
    # 5) RANK — take the k highest-scoring passages.
    top = np.argsort(scores)[::-1][:k]
    return [(float(scores[i]), docs[i]) for i in top]

for score, text in search("how do I stop being billed?"):
    print(f"{score:.3f}  {text}")

Run it and the query "how do I stop being billed?" returns the cancel your subscription passage first — even though it shares no keywords with the query. That's the entire point of semantic search, demonstrated in one print statement.

The TypeScript version

The same four steps in TypeScript, for a Node or edge backend. (Choosing between the two? See Python vs TypeScript for AI.)

function normalize(v: number[]): number[] {
  const mag = Math.hypot(...v);
  return v.map((x) => x / mag);
}

function dot(a: number[], b: number[]): number {
  return a.reduce((sum, x, i) => sum + x * b[i], 0);
}

// embed(texts) -> Promise<number[][]>, one normalized vector per text.
const docVecs = (await embed(docs)).map(normalize);

async function search(query: string, k = 3) {
  const q = normalize((await embed([query]))[0]);
  return docs
    .map((text, i) => ({ text, score: dot(docVecs[i], q) }))
    .sort((a, b) => b.score - a.score)
    .slice(0, k);
}

// From a script to a real app

The NumPy version scans every vector on every query. That's fine for a few thousand passages and instant to write. Past tens of thousands it gets slow, and you want a real vector index that uses an approximate-nearest-neighbour algorithm (like HNSW) to find close matches without comparing against everything. Here's roughly what changes, and what doesn't.

The crucial thing: the four conceptual steps never change. Embed, store, embed-the-query, rank. A vector database just makes "store" and "rank" fast, durable, and filterable. So your prototype is not throwaway — it's the same program with the storage layer swapped out. Many teams ship pgvector (Postgres with a vector column) precisely because it adds search to a database they already run.

// Common pitfalls

Semantic search is easy to demo and easy to do badly. Almost every disappointing result traces back to one of these — and notably, not to the LLM, because there isn't one.

• Mismatched models. Embedding the documents with one model and the query with another (or a different version) silently destroys relevance. Use one model for both, always.
• Bad chunking. Chunks too big average away the meaning; too small lose context. Start at 200–500 words with light overlap and adjust based on what your top results look like.
• Forgetting to normalize. If your vectors aren't length-1, a raw dot product rewards long vectors, not relevant ones, skewing the ranking. Normalize, or use a cosine-distance index.
• No score threshold. Top-k always returns k results, even when nothing is relevant. Without a minimum score you'll confidently show garbage for out-of-scope queries.
• Ignoring exact matches. Pure semantic search can miss literal strings like error codes, SKUs, or names. The production fix is hybrid search: blend semantic scores with keyword (BM25) scores.

// Going deeper

The basic build — embed, store, search, rank — gets you a genuinely useful app. Each stage has a deeper layer worth knowing once the fundamentals click.

Hybrid search and reranking. The strongest setups don't choose between semantic and keyword search — they run both and merge the scores, then run a reranker (a cross-encoder that reads the query and each candidate together) over the top results to re-order them far more precisely than a vector distance can. Retrieve broadly and cheaply, then rerank narrowly and accurately.

Metadata filters. A vector index can pre-filter before it ranks — "only billing docs", "only this user's files", "only documents newer than 2024." This is faster and more relevant, and it's the standard way to scope search per-tenant in a multi-user product.

Cost and latency. Embedding is one API call per document at index time and one per query at search time, so re-embedding a large corpus on every deploy can get expensive — cache vectors and only re-embed changed documents. Most apps embed once and store forever. For perceived speed, see designing for LLM latency; search itself is fast, but the query embedding round-trip is a network hop you can sometimes hide.

Where to go next. The natural follow-on is to feed your top passages to an LLM and let it write an answer — that's the leap from search to RAG, and a chat-with-your-PDF app is the classic first version. Everything you built here — chunking, embedding, the vector index, top-k retrieval — carries straight over; RAG just adds the generation step on top of the retrieval you already have.