Neural machine translation

Seq2seq with attention is the neural machine translation (NMT) engine — you already have the architecture. But translation as a task comes with its own toolkit: how you turn the model’s probabilities into a sentence, how you measure whether the result is any good, and how you cope with scarce data and an unbounded vocabulary. That toolkit became standard for all text generation, so it’s worth knowing on its own.

Around 2016, NMT swept aside statistical MT (SMT) — the old pipeline of phrase tables and n-gram language models. End-to-end neural models were more fluent and handled long-range word reordering far better, and systems like Google’s GNMT made the switch in production almost overnight.

Decoding: beam search beats greedy

The decoder outputs a probability distribution over the next token at each step. The naive choice is greedy decoding — take the most likely token every step — but that’s locally greedy and globally short-sighted: a high-probability first word can lead into a low-probability dead end. Beam search keeps the top-k partial sequences (“hypotheses”) alive at each step and expands them all, so it can find a sequence with higher total probability that greedy never reaches:

step1 = {"A": 0.6, "B": 0.4}
step2 = {"A": {"x": 0.3, "y": 0.3}, "B": {"x": 0.8, "y": 0.2}}

t1 = max(step1, key=step1.get); t2 = max(step2[t1], key=step2[t1].get)   # greedy: argmax each step
print(f"greedy:    {t1}{t2}  prob={step1[t1] * step2[t1][t2]:.2f}")

beams = sorted(((a + b, step1[a] * step2[a][b]) for a in step1 for b in step2[a]), key=lambda x: -x[1])
print(f"beam best: {beams[0][0]}  prob={beams[0][1]:.2f}   (greedy missed it)")

greedy:    Ax  prob=0.18
beam best: Bx  prob=0.32   (greedy missed it)

Greedy grabbed A (0.6 > 0.4) and got stuck with a weak continuation (0.18 total). Beam search kept B alive and discovered Bx at 0.32 — nearly double. The beam width k trades compute for quality; a width of 4–10 is typical.

Evaluation, data, and vocabulary

Three more pieces complete the NMT toolkit:

• BLEU measures quality by n-gram precision: what fraction of the candidate’s 1-, 2-, 3-, 4-grams appear in one or more reference translations, times a brevity penalty so the model can’t game it with short outputs. It’s automatic and cheap, correlates roughly with human judgment — but it’s shallow (surface n-grams, and a perfectly good paraphrase that shares few n-grams scores low). Still the field’s default.
• Back-translation fixes the data scarcity problem. Parallel sentence pairs are scarce; monolingual text is abundant. So train a reverse model, use it to translate target-language text back into the source, and add those synthetic pairs to training. It reliably boosts quality and is one of NMT’s most effective tricks.
• Subword units (BPE) handle the open vocabulary. Names, numbers, and rare morphological forms would be out-of-vocabulary as whole words; splitting into subwords means any word is representable from known pieces — the same insight as fastText, now for generation.

In one breath

• NMT is seq2seq + attention applied to translation; around 2016 it swept aside statistical MT (phrase tables) by being end-to-end, fluent, and better at long-range reordering.
• Decoding: greedy (argmax each step) is short-sighted; beam search keeps the top-k hypotheses and finds a higher-total-probability sequence (the demo: beam’s Bx 0.32 vs greedy’s Ax 0.18).
• BLEU scores translations by n-gram precision against references + a brevity penalty — automatic but shallow (misses valid paraphrases).
• Back-translation turns abundant monolingual text into synthetic parallel pairs to beat data scarcity; subword/BPE units give an open vocabulary (rare words from known pieces).
• The task is now subsumed by multilingual LLMs, but the toolkit — beam search, subwords, BLEU/ROUGE, back-translation — became standard across all text generation.

Quick check

Next

NMT is the application of seq2seq + attention; its sibling generation task is text summarization (and ROUGE vs BLEU). The subword units come from tokenization & BPE.