Skip to content
Road to Intelligence
← The Lab

Step through a real forward pass of one GPT-style Transformer block — embeddings, positions, multi-head attention, residuals, the MLP and the output softmax — with the tensor shape at every stage.

Try it · toy model

Transformer Explorer

Know well15 min