Autoregressive Model

Definition

An autoregressive model of order $p$, AR($p$), expresses the current value of a time series as a linear combination of its own $p$ most recent past values plus a random innovation: $$y_t = c + \phi_1 y_{t-1} + \cdots + \phi_p y_{t-p} + \varepsilon_t,\qquad \varepsilon_t \sim \text{WN}(0,\sigma^2).$$ Introduced by Yule (1927) to model Wolfer's sunspot numbers, it treats an observed series as a stochastic difference equation driven by random disturbances (innovations) rather than as a fixed periodic function contaminated by measurement error. It is the univariate building block underlying VARs, ARMA/ARIMA models, and much of time-series econometrics.

Key Ideas

• Disturbances vs. superposed fluctuations (Yule 1927): the defining conceptual move. Superposed fluctuations are noise added to the observations (a fixed sinusoid you see imperfectly); disturbances are shocks that enter the dynamics and are propagated, so amplitude and phase drift and unpredictability grows with the horizon. AR models are built on the second view — a "pendulum hit by random impulses."
• Periodicity from complex roots. The homogeneous part of the AR difference equation, $1-\phi_1 z-\cdots-\phi_p z^p=0$, has characteristic roots that govern the dynamics; complex roots produce damped oscillations. A stationary AR process therefore looks periodic without containing any deterministic periodic term — the cycle is a resonance the innovations keep re-exciting.
• Stationarity condition. The AR($p$) is (covariance-)stationary iff all roots of the characteristic polynomial lie outside the unit circle (equivalently, the companion-matrix eigenvalues lie inside it). A root on the unit circle gives a unit root; the AR(1) boundary $\phi_1=1$ is the random walk.
• Yule-Walker equations. Multiplying the AR recursion by $y_{t-k}$ and taking expectations relates the coefficients $\{\phi_j\}$ to the autocovariances $\{\gamma_k\}$: $\gamma_k=\phi_1\gamma_{k-1}+\cdots+\phi_p\gamma_{k-p}$ for $k>0$. Solving this linear system (from sample autocovariances) gives the classical Yule-Walker estimator — the direct descendant of Yule's least-squares regression of a series on its own lags.
• Relation to ARMA. Adding a moving-average term in the disturbances gives ARMA($p,q$); by Wold's theorem any stationary process has an infinite-order AR (and MA) representation, so finite AR models are parsimonious approximations.

How It Works

Estimate the coefficients by OLS (regress $y_t$ on its lags), Yule-Walker (autocovariance matching), or maximum likelihood / Bayesian methods; select the order $p$ by information criteria (AIC/BIC) or the partial autocorrelation function (which cuts off at lag $p$ for a pure AR). Forecasts iterate the fitted recursion forward, with forecast-error variance accumulating through the propagated innovations and converging (for a stationary process) to the unconditional variance.

Why It Matters

• Foundational. The AR model is the workhorse of time-series analysis and the univariate case of the VAR; unit-root testing, cointegration, ARCH/GARCH mean equations, and state-space models all build on it.
• Cycles without periodic forcing. It explains how business cycles, sunspot cycles, and other quasi-periodic phenomena can arise from a linear system perturbed by shocks — no exogenous oscillator required.
• Estimation care. OLS AR estimates are biased in finite samples (downward for persistent processes), and near-unit-root behavior distorts inference — motivating unit-root theory and bias corrections.

Open Questions

• Distinguishing a stationary AR with roots near unity from a genuine unit root is a low-power problem in finite samples.
• Order selection and structural stability: real series often have time-varying coefficients or regime changes that a fixed AR($p$) misses.
• Spurious inference: regressing one trending series on another (or on lags) can manufacture apparent relationships (spurious regression, itself a Yule concern).

Related

• Vector Autoregression
• Unit Root Inference
• Spurious Regression
• Heterogeneous Autoregressive (HAR) Model
• GARCH and BEKK-GARCH
• Yule (1927) On a Method of Investigating Periodicities in Disturbed Series
• G. Udny Yule