Introduction
In a recent engagement, we worked with a customer who already had a similarity search system backed by LanceDB. The foundation was solid: vector search over embedded documents powering retrieval for end user similarity searches.
But their use case wasn’t purely semantic.
Users frequently searched by identifier numbers and exact fields. They valued that LanceDB could support hybrid retrieval, allowing them to directly pinpoint a specific result while also retrieving semantically similar ones.
Sure, the search system worked. But as we enhanced their system, the conversation naturally evolved into:
If we were to design this from scratch, would we choose Keyword Search, or Hybrid Search?
Or, more broadly:
How should enterprises decide between these two architectures?
Rather than answer based on intuition, we developed a structured evaluation framework. Over time, that framework distilled into five measurable criteria we now use to guide this decision.
While this discussion frames the choice in terms of keyword vs. hybrid search, the same evaluation approach generalizes to RAG systems as well, where semantic retrieval (via embeddings) can be combined with keyword-based signals to improve first-result accuracy.
Why This Decision Matters
In enterprise AI systems, generation quality is rarely the root issue. Retrieval quality is.
If the correct document is retrieved, then
- The LLM usually produces a grounded answer
- Hallucination risk drops
- System gains user trust and attracts more users
However, if the wrong document is retrieved:
- The model answers confidently but incorrectly
- Users would need to spend time to self verify, then update prompts again
- User trust is lost
So the question then becomes: Which retrieval architecture produces the highest measurable accuracy within my constraints?
At this stage, the evaluation is primarily about search quality, not the LLM itself.
Whether results are ultimately shown directly to users, or passed into an LLM for summarization, is largely secondary to the core retrieval decision. The goal is to determine which approach — vector search or hybrid search — most reliably retrieves relevant documents.
This aligns well with a practical crawl–walk–run maturity model often seen in enterprise AI adoption:
- Crawl: Organize and make enterprise knowledge accessible
- Documents ingested
- Metadata structured
- Searchable data available
- Walk: Implement effective retrieval
- Keyword Search
- Semantic Search
- Hybrid Search
- Run: Add AI-powered experiences
- RAG pipelines
- Agents
- Automated workflows
This article focuses primarily on the Walk phase, which is selecting the right retrieval architecture, as many enterprise teams are still building reliable search before layering on AI capabilities.
LLMs do introduce an additional consideration: they often require higher precision retrieval so that incorrect context is not amplified during summarization. However, this is typically a refinement on top of the core retrieval decision rather than the starting point.
The Two Architectural Patterns
Before evaluating trade-offs, let’s align on what each retrieval architecture actually looks like in practice.
Vector Search Pattern
┌───────────────┐ ┌────────────────────┐ ┌────────────────────┐
│ User │ ──► │ Embedding Model │ ──► │ Vector Search │
└───────────────┘ └────────────────────┘ └────────────────────┘
▲ │
│ ▼
│ ┌────────────────────┐
└────────────── Response ◄────────────── │ Application Logic │
└────────────────────┘Minimal code sample:
import LanceDB
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("all-MiniLM-L6-v2")
db = LanceDB.connect("./data")
table = db.open_table("documents")
def search(query, k=5):
query_vector = model.encode(query)
results = table.search(query_vector).limit(k).to_pandas()
return results
Hybrid Search Pattern
┌───────────────┐
│ User │◄─────────────────────────────────────────────┐
└───────────────┘ │
│ │
▼ │
┌─────────────────────┐ ┌────────────────────┐ │
│ Semantic Search │ │ Keyword Search │ │
│ (BERT/ada/word2vec) │ │ (BM25) │ │
└─────────────────────┘ └────────────────────┘ │
│ │ │
└──────────────┬──────────┘ │
▼ │
┌────────────────────┐ │
│ Rank Fusion │ │
└────────────────────┘ │
▼ │
┌────────────────────┐ │
│ Top-K Results │ │
└────────────────────┘ │
▼ │
┌────────────────────┐ │
│ Response to User │─────────────────┘
└────────────────────┘
Minimal code sample:
vector_results = vector_search(query, k=20)
keyword_results = bm25_search(query, k=20)
response_limit = 5
fused_results = reciprocal_rank_fusion(
vector_results,
keyword_results
)
return fused_results[:response_limit]
The Five Evaluation Criteria
1. Identifier & Exact-Match Sensitivity
What to measure
Semantic embeddings excel at capturing conceptual similarity. They are less reliable when the query depends on:
- identification numbers
- Error codes (404, 500, etc.)
- SKUs or uuids
- Acronyms with domain-specific meaning
In domains where identifiers carry significant meaning, purely semantic similarity may not consistently surface the intended document.
Hybrid search mitigates this by incorporating keyword-based retrieval (such as BM25), ensuring exact signals are not lost. BM25 is a classical lexical ranking algorithm that scores documents based on term frequency and inverse document frequency (TF-IDF-style statistics). Unlike semantic search, it operates on exact token matches rather than embeddings. It is a traditional lexical ranking algorithm used in search engines to score documents by keyword relevance.
Decision guidance
Does your use case frequently involve searching by exact codes, identifiers, or structured fields? If so, consider hybrid search.
2. Search Quality Metrics
What to measure
Search quality metrics evaluate how effectively the system retrieves relevant documents and ranks them in useful positions.
- Recall@K
- Precision@K
- Mean Reciprocal Rank (MRR)
Why it matters
In enterprise systems, Recall@K is especially important. This ensures the user will get back at least some results even when there are no high precision matches. High recall minimizes false negatives and improves user experience.
Precision@K is important as well. Most users expect the right answer immediately. If the correct document does not appear near the top, the system quickly feels unreliable.
Mean Reciprocal Rank (MRR) is also a useful metric because it captures how early the first correct result appears in the ranking. A higher MRR indicates that users are more likely to find the correct result quickly without scanning multiple results, making it a strong indicator of real-world search efficiency.
Vector-only retrieval typically performs well on natural-language and paraphrased queries. However, when queries include exact identifiers, structured references, or tightly scoped terminology, hybrid retrieval often improves first-result accuracy by combining semantic and keyword signals.
Decision guidance:
If your evaluation dataset shows lower Precision for identifier-heavy or mixed queries under vector-only retrieval, hybrid search is worth considering.
3. Query Refinement Rate
What to measure
The average number of search attempts required by the user before user gets a satisfactory result. This is a practical KPI that often reflects real-world usability better than offline precision metrics alone. The could be done by grouping the number of search calls from a user within a short burst window, as users tend to have search activities clustered before they reach a satisfactory response.
When retrieval misses the correct document on the first attempt, users typically:
- Update their query or prompt
- Add more keywords
- Try an exact identifier
Hybrid retrieval can reduce this friction by capturing both semantic similarity and exact matches in a single pass.
Decision guidance
If users frequently rephrase queries to “force” exact matches, that’s a strong signal that vector-only retrieval may not be sufficient.
4. Latency & SLA Constraints
Hybrid search will introduce additional steps, such as vector retrieval, keyword retrieval, and reranking. Each step adds additional computational cost. Can your system handle the higher latency? If not, consider vector search.
What to measure
- Median retrieval latency
- P95 / P99 latency
- End-to-end response time (including generation)
- Latency under load
If possible, measuring latency broken down by component would give a clear latency flow.
Decision guidance
If your SLA has headroom and accuracy improves meaningfully, modest latency increases are often worth the trade-off. For high-frequency, low-latency systems with tight budgets, measure retrieval overhead carefully before adopting hybrid search.
5. Operational Complexity & Maintainability
Architecture decisions should consider long-term maintainability. Can your new Software Engineers and Data Scientists navigate the existing system?
Vector-only systems typically involve:
- A single index
- One retrieval path
- Fewer tuning parameters
Hybrid systems typically involve:
- Maintaining multiple indexes
- Fusion or reranking logic
- Monitoring dual retrieval signals
- Tuning keyword relevance parameters
The complexity difference may be minor for mature teams, but meaningful for smaller teams or early-stage deployments.
What to measure
- Infrastructure components required
- Index storage overhead
- Operational maintenance effort (manual tuning frequency)
- Monitoring complexity
- Engineering effort to implement and maintain
Decision guidance
If your use case does not show measurable gains from hybrid retrieval, the additional operational overhead may not be justified.
How We Apply This Framework
We treat this as a repeatable evaluation loop, not a one-time architecture decision.
- Build a labeled query dataset that reflects real user behavior.
- Measure search quality metrics such as Precision@K, Recall@K, identifier sensitivity, and latency for each retrieval approach. These metrics provide an objective baseline for comparing retrieval quality.
- Segment results by query type (semantic vs identifier-heavy) to understand where each approach performs best.
- Validate online behavior metrics, such as query refinement rate, using real users. Improvements in Precision@K and Recall@K are expected to reduce refinement rate, but this assumption should be verified with live usage data.
- Compare measurable accuracy gains against latency and operational cost.
If hybrid retrieval meaningfully improves first-result accuracy without violating SLA requirements or exceeding acceptable operational overhead, we adopt it.
If gains are marginal, we recommend the simpler vector-only architecture.
Final Thoughts
The decision between keyword and hybrid search is not philosophical. It is measurable.
By evaluating the five criteria below, teams can make a data-driven architectural choice.
- Identifier sensitivity
- Precision@K
- Query refinement rate
- Latency impact
- Operational complexity
In enterprise AI systems, retrieval strategy strongly influences user trust. Rather than treating architecture as a one-time decision, teams should iterate through experiments and pilot deployments. The retrieval system can then be refined based on both offline (historical) evaluation and online (live) user feedback.
Choose the approach that consistently performs best for your users, and support that decision with measurable results before scaling to production.
Additional Resources
- Retrieval-Augmented Generation (RAG)
- Hybrid search explained
- Evaluation Metrics for Search and Recommendation Systems
- What is BM25?
- precision@k explained
- LanceDB official documents
- Vector database explained
The feature image was generated using gpt-4.0.
