How to Build Site Search: Chinese Tokenization, Synonyms, and Typo Tolerance

1. The Search Box Nobody Maintains

An industrial parts supplier had 9,400 SKUs on its website. A customer typed "bearing 6204" and got zero results; only "6204bearing" worked. The analytics told the rest of the story: 3,100 on-site searches per month, 41% of them returning nothing, and visitors who searched converted at 4.2 times the rate of those who did not. That company was dumping roughly 1,270 of its highest-intent visits per month onto a blank page. A search box is not decoration — it is the only field on your site where customers tell you, unprompted, what they want and cannot find.

2. When It Fits × When It Does Not

Worth investing in site search when:

• You have more than 300 products, SKUs, or articles — category menus stop scaling
• Users search with unofficial names: nicknames, abbreviations, model variants, typos
• Your GA4 site-search report shows a zero-result rate above 15%
• You run a knowledge base and support answers the same 20 questions daily
• One product has three or more names (local name, international part number, customer code)

Not worth doing yet when:

• You have fewer than 150 records — faceted filters deliver more value per dollar
• Content rarely changes and structure is simple (a 12-page brochure site)
• Your data is messy: specs stuffed into title fields, notes stuffed into spec fields. Clean first, or a search engine just retrieves garbage faster
• Nobody on the team will read search logs — search is a fed system, not a one-off deliverable

3. Alternatives Matrix

For most small and mid-size businesses the sweet spot is Meilisearch: CJK segmentation works immediately, memory needs stay predictable, and synonyms plus ranking rules live in a JSON config instead of a query DSL.

4. The Full Process (Tools and Deliverables)

Stage 1: Search behaviour audit (3–5 days)
Tools: GA4 site-search events, existing database query logs, Notion.
Deliverables: top 200 query list, zero-result query list, first draft of the synonym table. This stage determines 80% of the final quality; skipping it means guessing.

Stage 2: Data model and index design (3–5 days)
Tools: Figma for search UI wireframes, a Notion field-mapping table.
Deliverables: field weighting table (title ×10, model number ×8, specs ×3, description ×1), facet list, ranking rules.

Stage 3: Build and data sync (7–10 days)
Tools: Meilisearch, Laravel Scout or a custom sync worker, Redis Queue.
Deliverables: full import script, incremental sync (indexed within 30 seconds of a product update), retry-on-failure handling.

Stage 4: Front-end search experience (5–8 days)
Tools: InstantSearch.js or custom components, Tailwind.
Deliverables: instant suggestions with a 150ms debounce, keyword highlighting, filters, and a zero-result page that offers alternatives.

Stage 5: Testing and tuning (3–5 days)
Deliverables: a 50-query golden test set (each with its expected top result), a regression script, and the production synonym table.

ScriptWalker's Site Search Implementation service starts at NT$68,000 (audit, index design, sync pipeline, and front-end search components). The full build with multilingual indexes, facets, and A/B ranking experiments starts at NT$150,000.

5. Real Cost Breakdown

• Development: NT$68,000–180,000 depending on record count, field complexity, and multilingual scope
• Search server: NT$800–2,000/month for a 2-core 4GB VPS; a 100,000-product CJK index typically occupies 600MB–1.5GB of RAM
• Data cleanup labour (most underestimated): parts catalogues usually need 16–40 hours to normalise title and spec fields — NT$16,000–40,000
• Synonym maintenance: 1–2 hours per month after launch, NT$12,000–24,000 per year
• Hidden cost 1: peak memory doubles during a full reindex. A 2GB box that runs fine will OOM mid-rebuild; budget for 4GB
• Hidden cost 2: CDN/WAF tuning. High-frequency search API calls are routinely caught by Cloudflare rate limiting — 3–5 hours to configure and test
• Hidden cost 3: backup and restore drills. An index is not a database; you need a script that rebuilds it from the primary DB within 30 minutes

6. What Clients Expect vs What Actually Happens

• Expectation: "Install a search engine and the site gets smart."
  Reality: 70% of search quality comes from data hygiene and the synonym table; 30% from the engine. The same Meilisearch produces wildly different results on clean versus dirty data.
• Expectation: "Build it once and we're done."
  Reality: the first month's zero-result log is the real requirements document. After the day-30 synonym pass, zero-result rates typically drop from over 30% to under 10%.
• Expectation: "Max out fuzzy matching."
  Reality: loose tolerance makes "6204" return 6205 and 6304, and precision collapses. Short model numbers usually need typo tolerance disabled entirely.
• Expectation: "Search speed is a server spec problem."
  Reality: most latency comes from firing a request on every keystroke. A 150ms debounce plus result caching improves perceived speed far more than a CPU upgrade — measure against the web.dev INP threshold of 200ms.

7. Six Traps and How to Avoid Them

• Trap 1: dropping CJK text into a default tokenizer. Compound terms get split mid-word and nonsense fragments match. Fix: pick an engine with native CJK segmentation, or set ngram_token_size=2 if you stay on MySQL.
• Trap 2: no synonym table. The customer's word and your catalogue's word never meet, so the result is permanently zero. Fix: build the table backwards from zero-result logs and update weekly for the first month.
• Trap 3: index drifts from the database. Discontinued products stay findable and customers order stock you do not have. Fix: queue-driven event sync plus a nightly full reconciliation.
• Trap 4: the zero-result page is a dead end. Fix: return near-miss suggestions, popular searches, and a support contact, and log the term into a backlog.
• Trap 5: an exposed, unthrottled search API. Scrapers inflate the bill or take the site down. Fix: search-only keys, per-IP rate limits, and a 60-second front-end result cache.
• Trap 6: weights set by gut feel. Fix: keep a 50-query golden test set, rerun it after every weight change, and judge only by top-3 hit rate so fixing A cannot silently break B.

8. Success Metrics and the 90-Day Roadmap

• Day 30: track zero-result rate (target <15%), search adoption (target >20% of visitors), median response time (target <100ms). Action: add synonyms for the top 100 zero-result terms.
• Day 60: track post-search click-through (target >45%) and top-3 hit rate (target >80%). Action: retune field weights against popular queries and add facets for brand, spec, and stock status.
• Day 90: track search-driven conversion and post-search exit rate. Action: turn the top 20 queries into dedicated landing pages (which also earn SEO traffic), introduce pinned results, and evaluate adding vector-based semantic search.

9. Decision Checklist

• ☐ I have more than 300 searchable records
• ☐ I know how many on-site searches happen each month
• ☐ I know my current zero-result rate
• ☐ My products or articles have nickname/alias problems
• ☐ My title fields are clean, with no specs or notes jammed in
• ☐ I have explicit ranking logic, not just "relevance"
• ☐ I can list five golden queries that must return an exact top result
• ☐ Updates must appear in search within one minute
• ☐ I need filters (price, brand, spec, stock)
• ☐ I need multilingual search
• ☐ I can budget NT$1,000–2,000 per month for a search server
• ☐ Someone will spend an hour a month reading search logs
• ☐ I accept 2–3 months of tuning after launch before it is genuinely good

Eight or more boxes: start now. Four to seven: clean the data and instrument GA4 search events first, then reassess in a month. Fewer than four: fix categories and filters instead.

10. FAQ

Q1: Is MySQL Fulltext really not viable?

Under 20,000 records with low typo tolerance requirements, MySQL with an ngram parser works and costs NT$0. But it has no built-in typo tolerance, synonyms, or runtime ranking control, so teams end up patching with LIKE queries and application logic — and maintenance eventually costs more than adopting Meilisearch would have.

Q2: How much data can Meilisearch handle?

A single node handles several hundred thousand documents reliably; RAM is the binding constraint, roughly 600MB–1.5GB for 100,000 CJK product records. Only past a million documents with complex aggregation needs does Elasticsearch-class infrastructure become necessary.

Q3: Should we jump straight to AI semantic search?

Treat it as phase two. Semantic search handles vague natural-language questions well but is less reliable for exact model numbers and part codes. The pragmatic path is to get keyword search to an 80% top-3 hit rate first, then add hybrid retrieval for long-tail questions.

Q4: How long does a build take from scratch?

Single language, under 100,000 clean records: roughly 3–4 weeks to launch. Add 1–2 weeks if data cleanup is needed. The real tuning window is the 8–12 weeks after go-live.

Q5: Does site search help SEO?

Search result pages themselves are usually set to noindex to avoid generating thin duplicate pages. The genuine SEO value comes from mining search logs for the words customers actually use, then building landing pages and content around them — that is the most valuable use of search data.

11. Next Step

Export your GA4 site-search report and we will run a free Search Health Check: zero-result rate, top 50 queries, and synonym gaps, returned as an actionable fix list with a price range. No payment required up front.

• Email: [email protected]
• Phone: 0916-224-047
• LINE: @ufv9089p