Visualize self-attention as beams of light between tokens and feel how softmax(QKᵀ/√dk) decides which word "it" is looking at.
Maskinöversatt från den engelska originaltexten. Öppna engelska originalet
Seven words float as arcs of light. Each time the viewpoint moves from word to word, where that word is “looking” in the sentence appears as a bundle of light. First watch the flow and confirm that the shape of the light changes from word to word.
Watch the sweep and find the difference between words where the light concentrates and words where it scatters.
