Loading
A median 24% of every displayed row had already been deleted before you saw it. In the worst case, 95%. The arithmetic was right the whole time.
Open the instrumentBERT adds two tokens to every sentence you give it. [CLS] at the front, [SEP] at the end. They are bookkeeping — one marks the start, one marks the end — and neither is a word you wrote.
They are also, in many heads, where most of the attention goes.
Which creates a problem for anything that wants to draw the sentence. You cannot put arcs on [SEP]; a visitor did not type it and has no idea what it is. So you drop those columns and renormalise the rest so the row still sums to 1.
That is what this project did. The arithmetic was correct. The presentation was a lie.
The decisive idea: if you remove part of a distribution and renormalise, every number you display becomes a share of a smaller whole — and it looks identical to a share of the original.
the → scissors 0.431 looked like a strong relationship. In the head it came from, the true weight was 0.021. The displayed value was a share of the 4.8% of that row which survived deletion.
Word rows are renormalised over visible words, so a displayed weight is a share of what remains after [CLS] and [SEP] are removed. Toggle to see the same row before removal.
Toggle between the two views. The bars are the same shape — of course they are, renormalising is a uniform rescale — but one row describes what the model did and the other describes what was left after editing.
Measured across 61 focused rows over nine sentences:
| Median share of a row removed | 24% |
| Rows losing more than half | 19 of 61 |
| Worst case | 95% |
A quarter of a typical row, gone before display. Nothing on screen said so.
The aggregation code had been computing that removed quantity since the day it was written. There was a variable called removedMass, and a unit test asserting it was correct.
Then the worker threw it away and never sent it to the interface.
The honesty machinery was built, tested, and disconnected. That is worse than not having it, because everyone involved — me included — could point at a test proving the number was known.
Knowing a number and showing it are different achievements, and only the second one reaches a reader.
Not a rewrite. A protocol version bump so the removed share travels alongside every matrix, and then the interface has to say it.
Three changes, in order of how much they hurt:
The number is stated. Every focused row now reports what share went to [CLS]/[SEP], and past 50% it says so in words rather than as a figure a reader could skim past.
Sink rows are tagged and dimmed. Punctuation absorbs enormous weight in most heads — heads → . measures 0.983 in one — so punctuation is excluded from choosing the peak that names a row. It is still shown, still ranked, still exactly its true value. It just does not get to define what the row is about.
The panel explains its own header. For Water boils at 100°c. the header read 100°c 0.163 above a list whose top entry was . 0.438. Both true. Nothing explained why the header disagreed with the list, so now it says: this slice is named for the strongest content word, and here is the punctuation weight that beat it.
And the phone got all of it. Every qualification had been hidden md:block or buried in a closed sheet — which is backwards, since the phone visitor is the one who cannot open the matrix and check for themselves.
I tried. It is worse.
An arc drawn to [SEP] is an arc to a token the reader never typed and cannot interpret. It dominates the picture, teaches nothing about their sentence, and makes the tool look broken. The renormalised view is genuinely the more useful one — it answers "among the words I wrote, where did attention go?", which is the question a visitor is asking.
The defect was never the renormalisation. It was renormalising silently. A share of a subset is a fine thing to display, as long as the reader is told which subset, and how much of the original is missing.
That distinction — between a wrong number and a true number that misleads — is the one this whole project turns on. Five defects were fixed in a single day that all had this shape: correct arithmetic, misleading presentation.
There is a published technique that claims to dissolve this problem entirely, by measuring what actually flows rather than what is attended to. Part 9 tests it.
Next: the mathematically correct way to combine all sixteen heads, and why it says almost nothing.
More to read
The word-embedding table is 31,254,528 bytes. A twelve-word sentence needs about 20 KB of it. HTTP Range makes that the only part that travels.
Every rollout row in the sentence sits between 0.935 and 0.971 normalized entropy. That is a uniform distribution wearing a ranking, and it shipped as the default view.
Naive int8 held attention parity at 2.97e-2 — thirty times worse than the bar. Where you spend precision is the whole engineering story.
