Autoregression (AR)

Why autoregression?

Many real-world time series carry momentum or persistence — a warm day tends to be followed by another warm day; a busy hour at a server often follows a busy hour. Autoregression (AR) is the workhorse model that captures exactly this structure. It forms the AR half of the famous ARIMA framework, so understanding it deeply pays dividends throughout forecasting work.

The core idea is disarmingly simple: treat the past values of the series itself as the predictors in an ordinary linear regression.

The AR(p) model

An AR(p) model says that the value at time t is a linear combination of its own p previous values plus a white-noise error term (a random shock with mean zero and constant variance):

y_t = c + phi_1·y_(t-1) + phi_2·y_(t-2) + ... + phi_p·y_(t-p) + e_t

• c — a constant (the intercept, related to the series mean)
• phi_1 … phi_p — the autoregressive coefficients (the weights given to each lagged value)
• lag — a time-shifted copy of the series; y_(t-1) is the series lagged by one period
• e_t — white noise: independent, identically distributed errors with mean zero
• p — the order of the model, i.e. how many past values you look back

To fit an AR model you literally stack lagged columns of the series into a design matrix and run ordinary least squares. That is why the method is called autoregression — you regress the series on itself.

The AR(1) case in depth

The simplest non-trivial case, AR(1), uses only the immediately preceding value:

y_t = c + phi·y_(t-1) + e_t

The single coefficient phi controls everything interesting:

Stationarity requirement: for the series to be stationary (have a stable, finite mean and variance), you need |phi| < 1. When |phi| = 1 you have a random walk; when |phi| > 1 the series explodes.

The PACF connection

In the previous lesson you learned that the Partial Autocorrelation Function (PACF) measures the correlation between y_t and y_(t-k) after removing the influence of all shorter lags. For a true AR(p) process, the PACF has a clean sharp cutoff after lag p — all partial autocorrelations beyond lag p are zero (within sampling noise). This is the primary diagnostic for choosing p: plot the PACF, find where it drops into the confidence band, and that lag is your order.

The AR feedback loop

The diagram below shows AR(1) as a feedback loop: the current output y_t is fed back as input y_(t-1) on the next step, scaled by phi, and combined with fresh noise e_t.

Simulate AR(1): see persistence change

The playground simulates two AR(1) series side by side — one with phi = 0.9 (high persistence) and one with phi = 0.3 (quick mean-reversion) — so you can see the difference directly.

What you should observe:

• The phi = 0.9 series drifts in long, sweeping runs — a positive excursion lasts many steps before the series returns to zero.
• The phi = 0.3 series bounces back toward zero rapidly; consecutive values look nearly independent.
• Both series are stationary because |phi| < 1 in both cases.

Try editing phi to 0.99 and observe how the series starts to look like a slow random walk. Then try a negative value such as -0.7 to see the oscillating mean-reversion pattern.

Fitting AR with statsmodels

In practice you rarely simulate — you fit AR to observed data. The statsmodels library provides AutoReg:

import pandas as pd
from statsmodels.tsa.ar_model import AutoReg

# Assume `series` is a pd.Series of your observed time series
model = AutoReg(series, lags=2)   # AR(2)
result = model.fit()

print(result.summary())           # shows phi_1, phi_2 and standard errors
print(result.params)              # constant, phi_1, phi_2

# One-step-ahead forecasts on the training period
fitted_values = result.fittedvalues

# Out-of-sample forecast for 5 steps ahead
forecast = result.forecast(steps=5)
print(forecast)

Choosing p in practice

1. Plot the PACF of your (stationary) series. Count the lags before the bars drop inside the 95 % confidence band. That count is a good starting value for p.
2. Information criteria (AIC, BIC) let you compare candidate orders rigorously — AutoReg reports both.
3. Start small. AR(1) or AR(2) often captures most of the structure. Higher-order models may overfit.

Summary

• An AR(p) model predicts y_t as a weighted sum of its last p values plus white noise: y_t = c + phi_1·y_(t-1) + ... + phi_p·y_(t-p) + e_t.
• phi controls persistence: values near 1 mean long memory; values near 0 mean rapid mean-reversion.
• Stationarity requires |phi| < 1 for AR(1) (and the analogous condition on all roots for higher orders).
• The PACF cuts off sharply at lag p — use it to identify the order before fitting.
• Next up: Moving Average (MA) models, which blend past errors rather than past values — the complementary half of ARIMA.