Semantic search vs. keyword search: what actually changes

Semantic search turns your query and your documents into meaning vectors and matches them by proximity. Keyword search (BM25) matches on word frequency and rarity. The difference shows up fast: BM25 won’t connect “car rental” to a document titled “auto hire” — semantic search will. But BM25 nails the part code “TX-4410-B” exactly, where semantic search drifts. So they aren’t rivals: in Elasticsearch, RRF-fused hybrid retrieval is usually the more robust choice across a mixed query load — but measure, don’t assume.

What keyword search actually does

BM25 is Elasticsearch’s default scoring function. It weighs three things: how often a term appears in a document, how rare that term is across the collection, and how long the document is. Its defaults are `k1 = 1.2` and `b = 0.75` — the first controls how quickly repeated terms stop helping, the second how hard length is penalised.

BM25 is fast, cheap, explainable, and a much stronger baseline than people expect. Its weakness sits in one place: matching happens on the token itself. If the query term isn’t in the document, the document scores nothing. Synonyms, rephrased questions, users who don’t know your internal vocabulary — they all break against this wall.

What semantic search does

A language model converts text into a fixed-length array of numbers: an embedding. Texts that mean similar things land close together in that space. The query goes through the same model, and retrieval becomes a nearest-neighbour problem rather than a token-matching one. Similarity is usually cosine, and to keep latency sane across millions of vectors you use an approximate neighbour index such as HNSW — “approximate” is literal here: you trade a slice of recall for speed.

Its weaknesses are equally clear. Rare proper nouns, SKUs, statute article numbers, error codes, exact-phrase lookups: vector similarity happily returns something near the right answer, which is the worst kind of wrong. There’s a cost story too — running a model over every document, chunking long text, and storing the vectors.

Side by side

	Keyword (BM25)	Semantic (vector)
Matches on	The term itself	Meaning proximity
Synonyms / rephrasing	Misses	Handles
SKUs, article numbers, exact quotes	Strong	Weak, drifts
Typo tolerance	Low (partly via fuzzy)	Moderate
Indexing cost	Low	High (model + vector storage)
Query latency	Very low	Low–moderate (with ANN)
Why this result ranked	Explainable	Hard to interpret
Moving to a new domain	Still works	Degrades if the model hasn’t seen it
Language coverage	Depends on the analysis chain	Depends on the model’s languages

Read the table one more time and the point lands: the two columns fail in different places. Wherever one drops, the other is standing. That is the entire argument for hybrid.

Hybrid search and RRF

In hybrid retrieval, both queries run and their result lists get merged. The catch: a BM25 score and a cosine similarity live on different scales, so you can’t just add them. Reciprocal Rank Fusion sidesteps this by throwing the scores away and looking only at rank:

score(d) = Σ  1 / (k + rank(d, query))

Take each document’s position in each list, invert it, sum. Elasticsearch defaults `rank_constant` to 60. Cormack, Clarke and Buettcher published the method in 2009, and it owes its popularity to one property: there’s almost nothing to tune.

With the Elasticsearch 8.14+ retriever syntax, it looks like this:

{
  "retriever": {
    "rrf": {
      "retrievers": [
        { "standard": { "query": { "match": { "content": "car rental" } } } },
        { "knn": { "field": "content_vector", "query_vector": [/* ... */],
                   "k": 50, "num_candidates": 100 } }
      ],
      "rank_window_size": 50,
      "rank_constant": 60
    }
  }
}

On older 8.x installs, RRF is configured through the `rank` block instead.

If you’d rather not wire the vector side by hand, the `semantic_text` field type takes over the mapping, the chunking, and embedding generation at index time.

The Turkish case: suffixes and morphology

Turkish is agglutinative. From the root ev (house) you get evlerimizden, evlerindekiler, and a long tail beyond. To BM25 each surface form is a separate term: search kiralama while the document says kiralamalarımızda, and without stemming there is no match at all.

Now the honest part: this alone is not a reason to switch to semantic search. Elasticsearch’s `turkish` analyzer already handles much of it. The chain is `apostrophe` → `turkish_lowercase` → `turkish_stop` → `turkish_keywords` → `turkish_stemmer`. Two links matter more than people realise. The `apostrophe` filter drops everything after the apostrophe, so Ankara’dan indexes as Ankara. And the lowercase filter, configured with `language: turkish`, gets the dotted/dotless I right: a capital I lowercases to i generically, but to ı in Turkish. Push IŞIK through the generic chain and it lands in the index as işik — a user typing ışık will never reach it.

Semantic search’s real contribution in Turkish isn’t inflection — it’s word choice. No stemmer will link fatura to irsaliye, or an employee’s mazeret to the HR policy’s izin. A vector can put them next to each other. On top of that, the Snowball-based Turkish stemmer is rule-driven: it over-stems some compounds and under-stems irregular forms. A semantic layer papers over those errors; it doesn’t erase them.

The second caveat matters more. Semantic search only works if the model has genuinely seen Turkish. Elastic recommends ELSER for English text; for Turkish content the right call is a multilingual model such as E5, or another embedding model whose Turkish coverage you’ve verified yourself. That single choice will decide more than the rest of your setup combined.

Where to start

Measure first. Build a small evaluation set — 30–50 real queries with their expected results. Without it, nobody can tell you which method is better.
Get BM25 right. The `turkish` analyzer, a synonym list for your house vocabulary, sensible field weights. Score the baseline.
Pick an embedding model with proven Turkish coverage, self-hosted if your scale or data-sovereignty needs point that way.
Add the vector leg, score it separately.
Fuse with RRF, score the same set again. You’ll see the lift — or you won’t, and that’s information too.

How we help

A production-scale example: LEGAPALAS, which we build end to end, vectorizes more than 11 million court decisions and statutes for meaning-based search — users ask in plain language, and results follow legal context rather than keyword overlap.

In our semantic search work we build this chain on the organisation’s own data: the Elasticsearch analysis pipeline, model selection, hybrid ranking, and an evaluation set before anything else. When you want the model running on your own servers, we design the setup that way from the start. If you’d like to talk through your search quality, get in touch or drop a line to [email protected].