Zou-Hastie (2005) Regularization and Variable Selection via the Elastic Net

Summary

Zou and Hastie introduce the elastic net, a penalized-regression method combining the lasso's $\ell_1$ penalty with a ridge $\ell_2$ penalty. It keeps the lasso's sparsity (exact zeros / automatic variable selection) but fixes two of its weaknesses: the lasso saturates at $n$ selected variables in the $p\gg n$ case, and it arbitrarily picks just one of a group of highly correlated predictors. The $\ell_2$ term removes the $n$ -variable ceiling and induces a grouping effect (correlated predictors enter or leave the model together), and a rescaling correction removes the double-shrinkage bias of the "naive" version. The LARS-EN algorithm computes the whole regularization path efficiently, and on real and simulated data the elastic net often out-predicts the lasso at comparable sparsity.

Key Claims

The elastic-net penalty. Minimize $\|y-X\beta\|^2 + \lambda_1\|\beta\|_1 + \lambda_2\|\beta\|_2^2$ — a convex combination of the lasso ( $\ell_1$ ) and ridge ( $\ell_2$ ) penalties. With $\lambda_2=0$ it is the lasso; with $\lambda_1=0$ it is ridge.
Breaks the lasso's $p\gg n$ ceiling. The lasso selects at most $\min(n,p)$ variables before saturating; adding the strictly-convex $\ell_2$ term lets the elastic net select more than $n$ predictors — critical for genomics-scale problems where relevant variables outnumber observations.
Grouping effect for correlated predictors. The lasso keeps one representative of a correlated group and zeros the rest, which is unstable; the $\ell_2$ penalty makes the coefficients of strongly correlated predictors shrink toward each other, so they tend to be selected (or dropped) as a group — more interpretable when predictors are genuinely co-informative.
Naive vs. corrected elastic net. The "naive" elastic net (solve as an augmented lasso) incurs double shrinkage that hurts prediction; multiplying the naive estimates by $(1+\lambda_2)$ removes the extra bias and defines the elastic-net estimator proper.
Augmented-lasso trick. By stacking $X$ with $\sqrt{\lambda_2}I$ and $y$ with zeros, the elastic-net problem becomes an equivalent lasso on an artificial dataset — so lasso machinery (and its path algorithm) applies directly.
LARS-EN path algorithm. An extension of LARS traces the full elastic-net solution path in essentially the cost of one least-squares fit; tuning is over $(\lambda_1,\lambda_2)$ (or a lasso-vs-ridge mixing parameter) by cross-validation.

Concepts Introduced or Extended

Entities Mentioned

Quotes

"The elastic net often outperforms the lasso, while enjoying a similar sparsity of representation. In addition, the elastic net encourages a grouping effect, where strongly correlated predictors tend to be in or out of the model together."

"The elastic net is particularly useful when the number of predictors ( $p$ ) is much bigger than the number of observations ( $n$ ). By contrast, the lasso is not a very satisfactory variable selection method in the $p\gg n$ case."

My Take

The elastic net is one of those additions that looks trivial — "just add a ridge term to the lasso" — but resolves exactly the two situations where the lasso disappoints in practice, so it became a workhorse the moment glmnet shipped it. The grouping effect is the conceptual heart: the $\ell_1$ corner geometry that gives the lasso its sparsity is also what makes it capriciously drop all-but-one of a correlated cluster, and the strictly-convex $\ell_2$ term is precisely the smoothing that ties correlated coefficients together — you buy stability with a little density. The $p\gg n$ ceiling fix matters wherever signals outnumber samples (genomics, text, finance factor zoos). For the wiki it is the canonical extension of the lasso and a member of the penalized-selection family it anchors — the frequentist cousin of the Bayesian spike-and-slab / horseshoe shrinkage the variable-selection page catalogues, and a reminder that the $\ell_1$ / $\ell_2$ mix is a dial, not a dichotomy.