← all posts site ↗

a search model that lives in your browser

the blog got semantic search and the architecture question came up immediately. does the server do the indexing. do all the old posts need re embedding. wait, i remember seeing demos where the model runs on the client side, is that the point. yes. indexing is a build step, it runs nightly and on publish, but the query runs entirely in the browser.

the first model choice died fast. e5 small v2 is not in the server embedding library, so that plan was dead before the download finished. the model those client side demos actually run is all minilm l6 v2, twenty two million parameters, three hundred eighty four dimensions, and twenty three megabytes as an int8 quantization. that is the entire browser download.

and every piece is self hosted. the transformer runtime at under a megabyte, the quantized model and its tokenizer sitting under the site's own model directory, zero cdn dependencies. the search script loads the index and the model, embeds your query locally with mean pooling and normalization, runs cosine over the whole index, renders the top results. if the model ever fails to load it silently falls back to the old server endpoint, which stays as insurance.

the toggle was the detail that mattered. a small button at the search box that switches between light and fast keyword search and semantic mode, so people who never want to download a model do not have to. and the sorta fts but contextual answer turned out to be exactly what it sounds like: keyword speed with meaning baked in, private, static, no service dependency once the page is loaded.