Semantic Search and Vector Search for Business Websites and Apps: Embeddings, Hybrid Search, and Evaluation

The search box is one of the most used features and also one of the most disappointing. A visitor types "how to claim laptop warranty" but the page titled "Electronic product return procedure" does not appear because it does not contain the same words. An employee searches the intranet for "vendor contracts ending next month" and only finds documents with the word "contract" in the title. Traditional keyword search works by matching letters and words, not meaning. Semantic search and vector search try to fix this by matching meaning: two sentences that use different words but mean the same thing are treated as close.

This article explains how semantic search works in language business owners and product managers can follow, then moves to the technical decisions that matter: when the technology is worth using, how to combine it with keyword search, how to protect data quality and privacy, and how to measure whether results are really better. There are no invented figures; what follows is a way of thinking and a checklist.

Summary

• Semantic search matches meaning through embeddings, so it finds relevant results even when the words differ
• Keyword search still wins for product codes, order numbers, proper names, and exact terms; the best solution is usually hybrid
• Search quality depends more on the quality and structure of documents than on the choice of model or vector database
• Access rights must be applied to search results so users do not see documents they must not see
• Measure with a set of test questions and simple metrics, not just subjective impressions
• Start from a small use case with clear value, such as a help center or product catalog

Why keyword search often fails

A classic search engine builds an index of words. When a user types a query, the engine looks for documents containing those words, then ranks them by how often and how distinctively the words appear. This approach is fast, cheap, and predictable. But it has weaknesses that recur:

• **Vocabulary gap**: the user writes "fridge", the document says "refrigerator"; the user writes "send back an item", the document says "product return"
• **Long or question-style queries**: "how do I change my delivery address after the order is placed" is hard to map to one or two key words
• **Typos and language variations**: a mix of Indonesian, English, and conversational abbreviations
• **Context**: the word "apple" can be a fruit, a brand, or a person's name
• **Zero-result queries**: a "not found" page that sends visitors away

A discussion of how visitors search for goods in an online store, including filters and synonyms, is in product search and filters for online stores. Semantic search does not replace all that; it adds the ability to understand intent.

How semantic search works

The concept can be explained in four steps.

1. Turning text into embeddings

A language model converts a piece of text into a list of numbers called an embedding or vector. These numbers are "coordinates" of meaning. Sentences with similar meaning have vectors that sit close together, while those on different topics are far apart. Embedding models are trained specifically for this task and differ from conversational models.

2. Storing vectors in an index

Each document is split into chunks, such as paragraphs or chapter sections, then each is converted into a vector and stored together with metadata such as title, category, language, date, and access rights. The store can be a dedicated vector database or a vector extension on a database you already use. For small to medium document collections, an extension on an existing database is often enough and simpler to operate.

3. Converting the query and finding nearest neighbors

When a user types a query, it is converted into a vector with the same model, and the system looks for the document vectors that are closest. On large collections, nearest-neighbor search is done approximately to be fast, with a small, tunable sacrifice in accuracy.

4. Ranking and showing results

The top results are displayed, often with the relevant text snippet. Many systems add a re-ranking step using a more careful model to pick the best few from dozens of candidates. This step improves quality at the cost of extra time.

Hybrid search: combining keywords and meaning

Semantic search is not a silver bullet. It is weak at exactly what keyword search is strong at: a product code like "SKU-48213", invoice numbers, exact names of people or companies, and rare technical terms. Vectors tend to consider "SKU-48213" similar to other codes, while the user wants the exact one.

A hybrid approach runs both and then combines scores. In practice:

• Run keyword search and vector search in parallel
• Merge the result lists with a simple rank-fusion method
• Give more weight to keywords when the query looks like a code, number, or name
• Apply metadata filters (category, language, date) before or after the search
• Provide filters and sorting for users instead of relying solely on automatic scores

A hybrid approach usually gives more stable results than either method alone. That is why many successful projects do not replace the old search but enrich it.

When semantic search is worth using

Semantic search delivers the most value when users express needs in varied words and your content is free text. Suitable examples:

• **Help centers and knowledge bases**: customers ask in everyday language while articles are written in official terms. More accurate results reduce support tickets. See also [customer support ticket systems and SLAs](/blog/sistem-tiket-dukungan-pelanggan-dan-sla)
• **Product catalogs with rich descriptions**: a search for "a cool office shirt" can find products without the word "cool" at all
• **Internal documents**: policies, contracts, procedures, and meeting notes scattered across many folders
• **Related content recommendations**: blog articles, case studies, or similar products
• **Cross-language search**: some multilingual embedding models let an Indonesian query find English documents
• **Search behind AI assistants**: company-knowledge chatbots use semantic search to retrieve material for answers, as discussed in [AI chatbots based on company knowledge with RAG](/blog/chatbot-ai-berbasis-pengetahuan-perusahaan-dengan-rag)

Conversely, if your catalog is small, well-structured, and users mainly search by name or code, good keyword search with synonyms and filters is often enough. Do not add complexity just because the technology is popular.

Preparing data: the part that matters most

The biggest mistake in search projects is spending time choosing a model while the source documents are a mess. Several things to fix first:

• **Duplicates and outdated versions**: if three versions of the same policy sit in the index, users get conflicting answers. Mark the active version and remove outdated ones
• **Document structure**: clear titles, subtitles, and summaries produce meaningful chunks. Scanned documents without text need character recognition and a quality check
• **Chunk size**: too small loses context, too large mixes many topics. Start from logical sections such as a paragraph with its heading, then test
• **Metadata**: source, date, owner, language, category, and access level. Metadata enables filtering and citation
• **Internal language and terms**: company abbreviations and special terms should be recorded in a synonym list or included in the text

More general data readiness guidance is in company data readiness before starting an AI project and data quality and data governance for mid-sized businesses.

Choosing an embedding model and database

Technical choices should follow needs, not trends. Some considerations:

• **Language**: make sure the model supports your content's language well, including Indonesian and mixed terms. Test with your own queries and documents
• **Dimensions and storage cost**: longer vectors use more space and search time
• **API service or self-hosted model**: a cloud API is quick to adopt but sends data to a third party. A self-hosted model gives more control over data with a higher operational burden. A similar trade-off is discussed in [self-hosted LLMs or cloud APIs](/blog/llm-self-hosted-atau-api-cloud-memilih-model-ai-untuk-data-bisnis)
• **Model updates**: if you change the embedding model, all documents must be re-indexed because vectors from different models cannot be compared. Store the model name and version in the index metadata
• **Scale**: hundreds of thousands of chunks can usually be handled by a general database with vector support; very large scale with high traffic may call for a dedicated service
• **Supporting features**: metadata filtering, hybrid search, and incremental updates matter more than small differences in benchmark scores

Access rights, privacy, and security

Semantic search makes documents easier to find, including documents certain people should not see. The basic principle: access rights must be enforced on every result, not only in the interface.

• Store the access level or role list as metadata and filter results by user identity before displaying them
• Do not index sensitive data that is not needed, such as identity numbers or health data
• Log queries carefully. Queries can contain personal information, so apply retention and access limits to logs
• If documents are sent to a third-party embedding service, check the data storage and usage terms
• Beware of documents hiding injected instructions when search results are fed to an AI assistant. This risk is discussed in [AI application security: prompt injection and data leakage](/blog/keamanan-aplikasi-ai-prompt-injection-dan-kebocoran-data)
• Protect the search endpoint from abuse and bulk harvesting with rate limiting, as in [protecting against bots, scraping, and automated attacks](/blog/perlindungan-bot-scraping-dan-serangan-otomatis-pada-website-bisnis)

Measuring search quality

Without measurement, "feels better" is easily misleading. Build a simple evaluation set:

• **Collect real queries**: take them from search logs, support tickets, or user interviews. Include short, long, misspelled, and previously failing queries
• **Define the correct answers**: for each query, mark one or more documents that should appear. Involve people who understand the content
• **Choose simple metrics**: whether the correct document is among the top three or ten results, and how high its position is
• **Compare variants**: keyword only, vector only, hybrid, with and without re-ranking, different chunk sizes
• **Monitor user behavior**: the rate of zero-result queries, clicks on top results, and how often users rewrite their query after seeing results
• **Review periodically**: content and user habits change, so the evaluation needs repeating

A similar approach to judging AI system output is discussed in evaluating the quality of AI and LLM output for business applications. Do not forget that fast search is part of the experience too; measure latency alongside quality, and see Core Web Vitals for business websites if search is embedded in a web page.

Costs to account for

Semantic search cost consists of several components that should be estimated from the start:

• **Generating embeddings** for the whole document collection, and updating them as documents change
• **Query embeddings** each time a user searches
• **Storage and compute for the vector index**
• **Re-ranking or answer generation** if a language model is used, usually the most expensive component. Control techniques are in [controlling AI costs: tokens and inference](/blog/mengendalikan-biaya-ai-token-dan-inferensi-untuk-aplikasi-bisnis)
• **Labor** for cleaning data, building the evaluation set, and maintaining the index

Ways to reduce cost include caching results of frequent queries, using re-ranking only on top candidates, and choosing an embedding model of reasonable size. For a wider view on choosing the right AI approach, read prompt, RAG, or fine-tuning.

Impact on SEO and user experience

On-site search affects conversion, but it also gives data about what visitors look for. Zero-result queries are a list of content ideas you have not written. Popular queries show the terms your audience uses, which is useful for page titles and descriptions. Note that internal search result pages generally do not need to be indexed by search engines; use appropriate directives so they do not create thousands of thin pages. A discussion of modern search direction and its impact is in AI-based search and the future of SEO for business websites.

A realistic implementation path

A staged approach reduces risk and proves value sooner:

• **First weeks**: pick one case, such as the help center. Collect around a hundred real queries and define the correct answers
• **Basic stage**: tidy the documents, create chunks and metadata, build the index, and measure against the existing keyword search as the baseline
• **Hybrid stage**: combine keyword and vector, add filters and re-ranking where needed
• **Production stage**: apply access rights, monitoring, incremental updates when documents change, and a user feedback loop
• **Expansion**: add other content sources and, where relevant, connect to AI assistants or other systems through standard integrations such as the [Model Context Protocol](/blog/model-context-protocol-mcp-menghubungkan-ai-dengan-sistem-bisnis)

Use staged releases and feature switches so you can return to the old search if there is a problem, as discussed in feature flags and staged releases.

Common mistakes

• Replacing the old search entirely without a baseline, so regressions go undetected
• Indexing all documents without cleaning duplicates and outdated versions
• Ignoring access rights on search results
• Choosing a model only from general rankings without testing on your own language and content
• Not storing the embedding model version, making migration hard
• Assuming semantic search suits every kind of query, including codes and numbers
• Giving users no way out, such as filters, sorting, and a contact-us button when there are no results

Frequently asked questions

Does semantic search require generative AI?

No. Semantic search uses embedding models, which differ from models that compose answers. You can show a normal list of results without generative AI. A generative model is needed only if you want summaries or direct answers.

Does it work for Indonesian and mixed languages?

It can, as long as the embedding model supports the language well. Quality differs between models, so test with your real queries and documents, including abbreviations and conversational language, before deciding.

Do we have to buy a dedicated vector database?

Not necessarily. For small to medium collections, a vector extension on an existing database is often enough. A dedicated service is worth considering when the scale, speed, or features you need go beyond that.

How many documents are needed for it to be worthwhile?

There is no fixed threshold. Value appears when users struggle to find information that actually exists. If you only have dozens of pages with clear structure, good navigation may be more effective.

How do we keep the index up to date?

Design an incremental update flow: when a document is created, changed, or deleted, the related chunks are re-indexed or removed from the index. Also schedule periodic checks to catch missed changes.

What about highly sensitive documents?

Exclude them from the index if they do not need to be searchable, or use a model run in your own environment with strict access control. Involve security and legal teams before indexing personal or confidential data.

Conclusion

Semantic search lets a system understand user intent instead of just matching words. Its value is greatest when combined with keyword search, supported by clean data and good metadata, protected by firm access rights, and evaluated with a real test set. Start with one case whose benefit is clear, measure before and after, then expand carefully.