Encoder-decoder,
revisited.

An encoder-decoder model has two stacks: the encoder reads the whole input with bidirectional attention (every token sees every other), producing a contextual memory; the decoder writes the output autoregressively, attending both to what it has written (causal self-attention) and to the encoder's memory (cross-attention).

Decoder-only models (GPT, Llama) fold reading and writing into one stack: simpler to scale, and one architecture handles any task framed as continuation. Encoder-decoder (T5, Whisper, translation models) still wins when input and output are different kinds of things — audio→text, document→summary — because the encoder can be bidirectional and the modalities stay cleanly separated.

In the decoder, queries come from the output-so-far while keys and values come from the encoder's output. Translating "the green house" → "la casa verde": when writing verde, the decoder's query attends to the encoder's green. Same QKV math as module 22 — only the source of K and V changes. This exact pattern returns in diffusion models, where image latents cross-attend to text prompts (23).

Pure encoder-decoder LLMs are now rare, but the idea is everywhere: Whisper is enc-dec (audio encoder, text decoder), every text-to-image diffusion model conditions on a text encoder, and many multimodal LLMs bolt vision encoders onto decoder stacks — though the newest omni models increasingly use early fusion, feeding image tokens straight into the decoder. Learn the pattern once, recognize it in every modality.

Inspector vs. Storyteller Transformers process sequences differently based on their architecture. Encoders (like BERT) look at the entire sentence at once in both directions, making them great at analyzing text. Decoders (like GPT) are causally masked, meaning they can only see the past and must generate the future one word at a time, making them master storytellers.