Mind Map
Explore real transformer attention without sending your sentence to a server.
The premise
A sentence enters as words. Inside the model, it becomes a field of relationships.
A precomputed bert-mini result opens instantly — the same model, the same four layers and sixteen heads, and the same numbers you get by running a sentence, without the download. Your own sentence then runs through that encoder on your device.
The one idea
Weight becomes light
Choose one word and the interface reads its real attention row. Every arc lands on another word: brighter, thicker paths carry more weight, while the exact value remains visible beside the ranking.
Everything above is one real result, computed from the shipped weights — not an illustration of one. Run your own sentence and the same numbers are recomputed on your device.
Three ways to read it
The same computed attention can be read as a cinematic relationship, an individual head, or the complete matrix. Switching views changes the lens, never the underlying result.
Diagram
RELATIONSHIP
A focused word sends weighted arcs across the sentence, with a full ranking that makes every visual claim inspectable.
Tokens
WORDS → PIECES
Every word beside the pieces the vocabulary forced it into, each with the row it occupies. The model never receives your words — only these numbers.
Matrix
ALL WORD PAIRS
Read every source-to-target weight on one shared scale, then select any row to bring it back into focus.
Under the hood
Real BERT forward pass
A four-layer BERT mini encoder runs directly in JavaScript. The displayed weights come from its actual softmax attention—not an animation fixture.
Mixed precision weights
Most matrices use int8; attention-sensitive query and key weights use int16; biases and normalization remain fp32.
Range-loaded embeddings
The browser requests only the embedding rows needed by your sentence instead of downloading the complete 31.25 MB table.
Dedicated worker
Tokenization, asset validation and inference stay off the main thread so the visual scene remains responsive.
SHA-256 verified assets · WordPiece tokenizer · no backend · no remote model API
By the numbers
How to read it
A diagnostic, not a mind reader
Attention is a diagnostic view, not proof of meaning or reasoning. Head-averaged rollout is close to uniform on a four-layer model, so it is not what you are shown: for the word you focus, the scene picks the most selective head that is not simply reading the neighbouring word, and names it on screen with the weight that justifies it.
Private by design
Inference runs in a dedicated browser worker. The page excludes product analytics and Sentry collection, and the sentence stays on your device.
Put a sentence inside the model
Focus a word, change the head, open the matrix, and inspect the numbers behind every beam—all without sending the sentence away.
Open the interactive sceneOther Projects
0101
Particle Life
Thousands of particles follow simple attraction and repulsion rules. No structure is programmed — complex, life-like behavior emerges on its own.
0202
Creature Evolution
2D creatures made of joints and muscles start helpless. A genetic algorithm mutates them across generations until they discover how to move.