See the residual stream as a glowing river that every layer reads from and writes to, with attention and MLP tributaries adding features.
Được dịch máy từ bản gốc tiếng Anh. Mở bản gốc tiếng Anh
Each layer of a Transformer does not compute on its own — it reads from and writes to one shared flow, the residual stream. Here we watch it as a glowing river running from input to output. Every time the river passes through a layer, a little more information is added and it grows brighter.
Watch the river flow from input to output, brightening as it passes each layer.
attn: gather the context / mlp: finishing before output. The final layer tidies the whole and finishes the representation handed to next-token prediction.
