What is the difference between encoder-only, decoder-only, and encoder-decoder transformer architectures?

How to think about it

These are three ways of arranging the same basic transformer machinery: self-attention, which lets tokens weigh information from other tokens. An encoder-only model reads an input and builds rich representations; a decoder-only model predicts a continuation one token at a time; an encoder-decoder model reads one sequence and generates another. BERT, GPT, and T5 are the standard examples.

Why the attention pattern matters

An encoder-only model, such as BERT, uses unrestricted self-attention over its input. Every token can use every other input token, including tokens that appear later in the sentence. The result is a contextual representation for each input position.

For the sentence “The bank raised rates,” the representation of “bank” can use “raised” and “rates” to decide whether the sentence concerns finance or a riverbank. That is useful when the complete input is already available.

Encoder-only models are commonly trained with masked language modelling: some input tokens are hidden, and the model predicts them from the visible context. They are then fine-tuned with a small task-specific layer for classification, named-entity recognition, ranking, or similar tasks.

A decoder-only model, such as GPT, uses causal self-attention. “Causal” means that a position can attend to itself and earlier positions, but not to later positions. A mask enforces this rule. When the model predicts the next token, it cannot quietly look at the answer.

Standard decoder-only transformers do not have a separate encoder and therefore do not use encoder-decoder cross-attention. They read a prompt and its generated continuation as one growing sequence.

An encoder-decoder model, such as T5, has both stacks:

• The encoder reads the complete source sequence with unrestricted self-attention.
• The decoder reads previously generated target tokens with causal self-attention.
• The decoder also uses cross-attention, which lets the current target position retrieve information from the encoder’s source representations.

Cross-attention is the important extra connection. The decoder’s current state effectively asks, “Which parts of the source matter for the next output?” The encoder states provide the searchable memory.

Architecture	What each position can see	Natural output
Encoder-only	Every input token can see every input token	Labels, spans, or representations
Decoder-only	Earlier prompt and output tokens	A continuation
Encoder-decoder	Decoder sees earlier target tokens and all encoded source tokens	A transformed target sequence

Common trap: “Bidirectional” does not mean an encoder sees the future in every situation. It sees all the tokens you provide. That is fine when a complete source sentence is available for translation. It is not fine for next-token generation, because future output tokens do not exist yet.

The training difference

The attention mask determines the learning objective.

BERT-style encoder-only training might turn:

My card was charged [MASK].

into a prediction for the missing word using both sides of the gap. The model learns useful representations of the whole input.

GPT-style decoder-only training uses next-token prediction. Given a prefix such as “My card was,” it learns to predict “charged”; given “My card was charged,” it learns to predict “twice.” The model repeats this across a large text corpus.

There is a subtle point here. Decoder-only generation is sequential, but decoder-only training is highly parallel. During training, the correct target sequence is already present, so the model can calculate predictions for many positions in one pass. The causal mask prevents information leakage between positions. At inference time, the next correct token is unknown, so the model must generate one token, append it, and then generate the next.

Encoder-decoder models are trained for sequence-to-sequence tasks, meaning one sequence is transformed into another. Translation pairs are the obvious example. T5 also uses denoising: parts of an input are corrupted, and the decoder learns to reconstruct the missing text. During training, the correct target tokens are supplied to the decoder; this is called teacher forcing. During inference, they are not, so decoding becomes sequential.

One support ticket through all three

Consider this ticket:

My card was charged twice; please refund one charge.

Assume a tokenizer represents the source as nine tokens. The exact count depends on the tokenizer, but nine makes the information flow easy to see.

With an encoder-only model, all nine tokens attend to one another in a single forward pass. A classification layer can read the pooled representation and return a label such as duplicate_charge. Another task-specific layer could mark “charged twice” as the relevant span. The model is not naturally producing a paragraph; it is producing representations from which a task can make a decision.

With a decoder-only model, the ticket might appear in a prompt:

Classify this ticket and explain the decision: My card was charged twice...

The model predicts the response one token at a time. The first response token can attend to all nine ticket tokens and the instruction. Once it emits the first response token, the next position can attend to those nine prompt tokens plus the first response token. If the final response is eight tokens long, there are eight sequential generation steps.

With an encoder-decoder model, the encoder processes the nine ticket tokens once. The decoder then generates an eight-token response such as “I’m sorry; we’ll refund one charge.” At every target position, the decoder can attend to the earlier response tokens and all nine source representations. Ignoring attention heads and projection costs, the source-to-target cross-attention contains roughly 9 x 8 query-key interactions for those nine source and eight target tokens.

The task determines which information flow is convenient:

• “Is this a duplicate charge?” suits an encoder-only classifier.
• “Write a helpful reply” suits a decoder-only model.
• “Translate this ticket into Spanish” or “summarize this document” suits an encoder-decoder model.

Those are preferences, not prison walls. A decoder-only model can classify a ticket through prompting or a classification head. A decoder-only model can summarize. An encoder-only model can support generation if a separate decoder is added. The native architecture simply makes some jobs easier and more efficient than others.

The senior nuance: architecture is not capability

The beginner’s shortcut is “encoders understand and decoders generate.” It is useful, but incomplete.

For a fixed-label task such as fraud detection, an encoder-only model often makes a better production choice. It processes the input in parallel and returns a result after one forward pass. There is no generation loop, no risk that duplicate_charge becomes This appears to be a duplicate charge, and no need to parse free-form text. Embedding models for semantic search also commonly use encoder-style architectures because the desired output is a vector, not a paragraph.

For open-ended chat, code completion, and tool-oriented interaction, decoder-only models are a natural fit. Everything can be represented as one sequence: system instructions, conversation history, tool results, and the next response. The ecosystem and serving infrastructure for these models are also extensive. The cost is that every generated token adds another decoding step. A key-value cache, usually called a KV cache, stores earlier attention information so the model does not recompute the entire history from scratch, but generation is still limited by the number of output tokens.

Encoder-decoder models are attractive when the boundary between source and target is clear. The encoder can read a long source bidirectionally, and its output can be reused while generating several candidate responses. But there are two stacks to run, plus cross-attention from every target position back to the source. Self-attention costs also grow roughly with the square of sequence length, so long documents can still be expensive. Encoder-decoder is not automatically cheaper or more accurate.

The model’s training data, parameter count, fine-tuning, tokenizer, and serving system often matter more than the label on the architecture. A well-tuned decoder-only classifier can beat a weak encoder-only classifier. Architecture gives the model an information-flow bias; it does not determine the final score by itself.

A failure mode you can spot in production

A common mistake is using a decoder-only model for a fixed-label classifier and asking it to “answer with one of three labels.” At 3 a.m., the first symptom is usually not a crash. It is output drift:

urgent

then:

Urgent

then:

This looks urgent because the customer may be blocked.

The model may understand the ticket perfectly, but exact-match accuracy and downstream parsing fail because generation is a free-form interface. Use a classification head, constrain the output format, and validate the result. If the task is high-volume and latency-sensitive, compare that design against an encoder-only classifier rather than assuming a larger generative model is automatically better.

The opposite mistake is asking BERT to write a fluent customer reply. A masked-language-model head can fill selected gaps, but it was not trained as a left-to-right response generator. The first symptom is often a brittle or repetitive answer rather than a coherent paragraph.

What they’ll ask next

“Does the decoder in an encoder-decoder model use bidirectional attention?”

Not over the target sequence. Its self-attention is causal, so a target position cannot see future target tokens. It can, however, cross-attend to every encoder position in the complete source. That combination prevents target leakage while preserving full access to the source.

“Can a decoder-only model do classification?”

Yes. You can add a classification head to a hidden representation, or prompt it to emit a label. The second approach is flexible but may produce inconsistent formatting and consumes generation time. For a small, fixed label set, an encoder-only model or a decoder with a dedicated head is often easier to operate.

“Why not use an encoder-decoder model for every generation task?”

You can use one for chat or summarization, but decoder-only models make arbitrary continuation simpler: the entire conversation is one sequence, and the model only needs one stack. Encoder-decoder models are especially compelling when there is a stable source-to-target transformation, such as translation or controlled rewriting. The choice depends on output flexibility, source and target lengths, training data, latency, memory, and the serving ecosystem.

Say this in the interview

Encoder-only models build bidirectional representations, decoder-only models predict continuations under a causal mask, and encoder-decoder models combine both to transform a fully read source into an autoregressively generated target; I choose among them based on the task’s information flow and production constraints.