What is the split-apply-combine model in pandas GroupBy?

How to think about it

This question checks whether you have a mental model of groupby or just memorized method names. Hadley Wickham’s framing — split the frame into groups by key, apply a function to each group independently, combine the pieces back — is the whole thing. The practical payoff is that it tells you which method to reach for, because each one rejoins the data at a different shape. Decide what you want the output to look like, and the method picks itself: one row per group means agg, a new column on the original means transform, dropping whole groups means filter.

The three methods on one frame

Watch the same region grouping run through all three. Notice how each rejoins the data at a different shape: agg returns one row per region, transform returns six rows matching the original, and filter returns a subset of the original rows.

import pandas as pd

df = pd.DataFrame({
    "region": ["East", "East", "West", "West", "West", "North"],
    "rep":    ["Alice", "Bob", "Carol", "Dave", "Eve", "Frank"],
    "sales":  [300, 150, 400, 220, 180, 90],
})

# agg — collapse to one row per group (summary table)
summary = df.groupby("region")["sales"].agg(
    total="sum", avg="mean", deals="count",
).reset_index()
print("agg (one row per group):")
print(summary)
print()

# transform — broadcast the group total back to every original row
df["region_total"]  = df.groupby("region")["sales"].transform("sum")
df["pct_of_region"] = (df["sales"] / df["region_total"] * 100).round(1)
print("transform (same shape as original):")
print(df)
print()

# filter — keep only regions whose total sales exceed 500
big = df.groupby("region").filter(lambda g: g["sales"].sum() > 500)
print("filter (only high-sales regions):")
print(big)

agg (one row per group):
  region  total         avg  deals
0   East    450  225.000000      2
1  North     90   90.000000      1
2   West    800  266.666667      3

transform (same shape as original):
  region    rep  sales  region_total  pct_of_region
0   East  Alice    300           450           66.7
1   East    Bob    150           450           33.3
2   West  Carol    400           800           50.0
3   West   Dave    220           800           27.5
4   West    Eve    180           800           22.5
5  North  Frank     90            90          100.0

filter (only high-sales regions):
  region    rep  sales  region_total  pct_of_region
2   West  Carol    400           800           50.0
3   West   Dave    220           800           27.5
4   West    Eve    180           800           22.5

The shapes tell the whole story. agg gave three rows — one per region — so it answers “summarize.” transform kept all six rows and pushed each region’s total onto every member, so the per-row percentage works without a merge. filter evaluated the sum-over-500 test once per group and kept only West’s three rows, dropping East and North whole. Same split, three different combine steps.

Phase	Method	Output shape	Typical use
Apply	agg	One row per group	Summary tables, dashboards
Apply	transform	Same shape as input	Derived columns (z-scores, ratios, ranks)
Apply	filter	Subset of original rows	Removing rare or outlier groups