Tag: attention output projection
-
The Output Projection: How 64 Attention Heads Become One Thought
Work a full output projection by hand on a 6 x 6 grid, then see why W_O routes head findings and why…
Work a full output projection by hand on a 6 x 6 grid, then see why W_O routes head findings and why…