Loading
The word-embedding table is 31,254,528 bytes. A twelve-word sentence needs about 20 KB of it. HTTP Range makes that the only part that travels.
Open the instrumentMost of a small BERT is not the model. It is the dictionary.
bert-mini's transformer blocks — every weight matrix in all four layers — come to about six megabytes. The word-embedding table alone is 31,254,528 bytes, because it holds one 256-number vector for each of 30,522 vocabulary entries.
Ship the whole thing and every visitor downloads 31 MB to look at a nine-word sentence, of which they will use nine rows.
The decisive idea: the embedding table is not a model file, it is a random-access array. Treat it like one, and a sentence costs kilobytes instead of megabytes.
HTTP has had the answer since 1999. A client can ask for a byte interval instead of a whole file, and a compliant server answers 206 Partial Content with only those bytes.
The export is written specifically to make that possible: the embedding table is a flat, row-major binary file where row i starts at exactly i × 1024 bytes. So once the tokenizer has produced the input ids, each one is a byte offset, and the runtime fetches precisely the rows the sentence uses.
The embedding table is 31,254,528 bytes. The scene fetches only the rows its tokens need, over HTTP Range, so the cost scales with your sentence rather than with the model.
A twelve-word sentence is around sixteen tokens. Sixteen kilobytes. The other 99.95% of the table never leaves the server.
The cold load — the part every visitor pays once — is 6.32 MiB: the quantized blocks, the manifest, and the vocabulary. The vocabulary has to arrive whole, because you cannot know which rows you need until you have tokenized, and you cannot tokenize without it.
That ordering has a visible consequence I like more than the alternative. The input limit is twenty-four tokens, and how many tokens your sentence makes is unknowable until the vocabulary is present. So the interface reports a too-long sentence honestly after the fact instead of guessing beforehand. A predicted limit would be wrong for exactly the inputs where it matters — the ones full of unusual words that fragment.
Every "runs locally, private by design" claim on the internet is a promise about code you cannot see. Here it is a property of the deployment, and you can check it from your own browser's network panel.
The Content-Security-Policy is the load-bearing part:
default-src 'self' connect-src 'self' worker-src 'self' blob:
object-src 'none' frame-ancestors 'none' base-uri 'self'connect-src 'self' means the page is not permitted to open a connection to any other origin. Not an inference API, not an analytics endpoint, not a logging service. Not by accident, and not by a future edit either — the browser refuses.
That is a different kind of statement from "we don't send your data." It is: this page cannot send your data, and the enforcement is not mine.
There is deliberately no 'wasm-unsafe-eval' in that list, which the previous post explains. The browser test asserts its absence rather than trusting anyone to remember.
Range requests are not free.
Twenty separate small requests have twenty round trips of latency, where one big download has one. On a fast connection the row fetches disappear into the noise; on a slow high-latency link they are the slowest part of a query. That trade is worth making here because the alternative is 31 MB, but it is a trade.
The server also has to cooperate. A CDN that ignores Range and returns 200 with the whole body would make the scene work perfectly while quietly transferring the entire table on every request — no error, no warning, just thirty megabytes a visitor. That failure is invisible from inside the page, which is why there is a script that checks a real host answers 206 with the exact byte count, rather than assuming.
The interesting part is not Range requests. It is that a constraint I treated as fixed — "using a real model in a browser means downloading a real model" — turned out to be an artifact of packaging.
The transformer needs to be present in full: every block runs on every token. The dictionary does not. It is looked up sparsely, a handful of entries per query, and treating it as a file rather than a table was costing 31 MB to deliver 16 KB of useful information.
Worth asking of anything that feels too heavy to ship: is this a model, or is it an index?
Next: the weight on screen turns out to be a share of something you were never shown.
More to read
Naive int8 held attention parity at 2.97e-2 — thirty times worse than the bar. Where you spend precision is the whole engineering story.
A median 24% of every displayed row had already been deleted before you saw it. In the worst case, 95%. The arithmetic was right the whole time.
Choosing the most confident head picked a positional one every time. It reported “it → was, 0.962” — the answer that head gives for every word in every sentence.