Text summarization

Summarization compresses a document to its essence — and there are two fundamentally different ways to do it. Extractive summarization selects the most important sentences from the original and stitches them together. Abstractive summarization generates new sentences that may not appear anywhere in the source, the way a person would paraphrase. The choice between them is a choice between safety and fluency.

Extractive: select the important sentences

Extractive methods score each sentence for importance and keep the top few. The classic scorer is TextRank. It runs PageRank (the algorithm that ranks web pages by who links to them) on a graph where sentences are nodes and edges are their similarity. The intuition is the same as ranking web pages: a sentence is important if it’s similar to many other important sentences — central to the document. A simple version just scores each sentence by its total similarity to the rest:

sentences = ["the cat sat on the mat", "the cat is a pet animal", "stocks fell sharply today"]

def words(s): return set(s.split())
def sim(a, b):
    wa, wb = words(a), words(b)
    return len(wa & wb) / (len(wa | wb) or 1)        # Jaccard overlap: shared words / all distinct words

scores = [sum(sim(s, t) for t in sentences if t != s) for s in sentences]
for sc, s in sorted(zip(scores, sentences), reverse=True):
    print(f"{sc:.3f}  {s!r}")

0.222  'the cat sat on the mat'
0.222  'the cat is a pet animal'
0.000  'stocks fell sharply today'

The two cat sentences reinforce each other (they overlap), so they’re central and rank high; the unrelated finance sentence shares nothing, scores zero, and is dropped. Extractive summaries are safe — every sentence is verbatim from the source, so they can’t fabricate — but they can be choppy and redundant (selected sentences don’t always flow, and may repeat).

Abstractive: generate the summary

Abstractive summarization treats the problem as sequence-to-sequence generation (a model reads one piece of text and writes another). It reads the document and generates a short summary in new words — paraphrasing, fusing ideas across sentences, compressing as a human would. It’s far more fluent and natural, and it’s what modern transformer summarizers and LLMs do. The danger is the flip side of generation. That danger is hallucination: the model can produce fluent sentences asserting facts that aren’t in the source (or contradict it). A bridge design, the pointer-generator, lets the model copy words directly from the source when it should and generate when it should, reducing fabrication.

Evaluating with ROUGE

Summaries are scored with ROUGE, the recall-oriented cousin of BLEU. Where BLEU (for translation) emphasizes precision — are the candidate’s n-grams correct? — ROUGE emphasizes recall: did the candidate cover the n-grams (short runs of words, like pairs) in the reference summary, the human-written one it is scored against? That makes sense for summarization, where the worry is leaving important content out. Like BLEU, it’s surface-level and blind to meaning-preserving paraphrase, so it’s a rough proxy, not ground truth.

The 40-page problem: long documents

A model can only read so much text at once (its context window). For a 40-page report you have two options. If the report fits, give it all to an LLM and ask for a summary that quotes numbers from the text. If it does not fit, use map-reduce: split into sections, summarize each, then summarize the summaries. Each pass loses detail, so after the final summary, run a faithfulness check such as NLI (does each summary sentence follow from the source section?).

In one breath

• Summarization has two paradigms: extractive (select important sentences verbatim) and abstractive (generate new sentences).
• Extractive scores sentences by importance — TextRank runs PageRank on a sentence-similarity graph (central sentences win; the demo drops the unrelated one at score 0). It’s safe (no fabrication) but choppy/redundant.
• Abstractive is seq2seq generation — fluent and human-like, but can hallucinate facts not in the source; pointer-generator copies-or-generates to reduce that.
• ROUGE evaluates summaries by recall (did you cover the reference’s n-grams?), versus BLEU’s precision for translation — both surface-level proxies.
• The trade-off is safe-but-choppy vs fluent-but-unfaithful; LLMs made abstractive the default, but faithfulness remains the open problem, so extractive still wins where fabrication is unacceptable.

Quick check

Next

Abstractive summarization is seq2seq generation, evaluated with ROUGE (the recall cousin of BLEU); extractive ranking uses graph PageRank. The faithfulness problem is hallucination & grounding.