Text classification: CNN & RNN

The most common NLP task is text classification: one label for a whole text — positive vs negative review, spam vs not, which topic. Classical models (Naive Bayes, SVM on TF-IDF) do this well but treat the text as a bag of words: “not good” and “good not” look identical. The neural approaches keep word order by feeding word embeddings through an architecture that respects sequence. Two classic ones: the CNN and the RNN.

CNN for text: learnable n-gram detectors

A 1D convolution slides a small filter across the sequence of word embeddings. A filter spanning 2–3 words is an n-gram detector — it learns to fire on a phrase like “not good” or “highly recommend.” Run many filters and you detect many phrases; then max-pool each filter’s activations over the whole sentence, which answers “did this pattern appear anywhere?” — a position-invariant feature.

import numpy as np

emb = {"the":[0,0], "movie":[0,0], "was":[0,0], "i":[0,0], "not":[1,0], "good":[0,1], "loved":[0,1]}
filt = np.array([[1.5, 0.0], [0.0, 1.5]])      # window-2 filter: negation-then-positive detector

def conv_maxpool(sent):
    v = np.array([emb[w] for w in sent], float)
    acts = [float((v[i:i+2] * filt).sum()) for i in range(len(v) - 1)]   # slide the filter
    return max(acts), [round(a, 1) for a in acts]                         # max-pool

for sent in [["the","movie","was","not","good"], ["i","loved","the","movie"]]:
    mp, acts = conv_maxpool(sent)
    print(f"{sent} -> maxpool={mp:.1f}  per-position acts={acts}")

['the', 'movie', 'was', 'not', 'good'] -> maxpool=3.0  per-position acts=[0.0, 0.0, 0.0, 3.0]
['i', 'loved', 'the', 'movie'] -> maxpool=1.5  per-position acts=[1.5, 0.0, 0.0]

The filter lights up strongest (3.0) exactly at the “not good” window and stays quiet elsewhere; max-pool grabs that 3.0 no matter where in the sentence it occurred. That’s the CNN’s strength: fast, parallel detection of local cue phrases — and for sentiment/topic, the presence of a few key phrases is often most of the signal.

RNN for text: read it in order

A recurrent network takes the opposite approach: read the words one at a time, left to right, updating a hidden state, and use the final state (a summary of the whole sentence) for the label. Because it carries state forward, an RNN/LSTM captures order and long-range dependencies a CNN’s fixed window can miss — negation scope (“I did not think the movie, despite the hype, was good”), or agreement across a long clause. A bidirectional LSTM reads both directions so each position sees its full context. The cost is that it’s sequential — it can’t parallelize across the sentence the way a CNN can.

Choosing, and what came next

The trade-off is clean: CNN is fast, parallel, and excellent at local cue phrases; RNN/LSTM is slower but models order and long-range structure. Both beat bag-of-words by using embeddings plus architecture instead of raw counts — and for many classification tasks the CNN’s speed makes it the pragmatic pick, since a few key phrases carry most of the signal.

In one breath

• Text classification assigns one label to a whole text (sentiment, spam, topic); classical bag-of-words models work but ignore word order.
• CNN for text: a 1D convolution slides filters over word embeddings as n-gram detectors, then max-pools to detect a cue phrase anywhere (the demo: “not good” fires 3.0, max-pooled position-invariantly) — fast and parallel.
• RNN/LSTM for text: reads the sequence in order, carrying a hidden state, capturing order and long-range dependencies (negation scope) a fixed CNN window misses; BiLSTM reads both ways; cost is being sequential.
• Trade-off: CNN = fast/local, RNN = order-aware/long-range; both beat bag-of-words via embeddings + structure.
• The modern default is fine-tuning a transformer (BERT) — but CNN/RNN stay useful for fast, cheap classification.

Quick check

Next

Text classification labels a whole document; its per-token sibling is sequence labeling. The building blocks are CNNs and RNNs/LSTMs applied to word embeddings; the modern successor is BERT.