Zou-Hastie (2005) Regularization and Variable Selection via the Elastic Net

elastic-netlassoridgevariable-selectionshrinkagepenalizationgrouping-effecthigh-dimensionallars

Summary

Zou and Hastie introduce the elastic net, a penalized-regression method combining the lasso's ℓ1\ell_1 penalty with a ridge ℓ2\ell_2 penalty. It keeps the lasso's sparsity (exact zeros / automatic variable selection) but fixes two of its weaknesses: the lasso saturates at nn selected variables in the p≫np\gg n case, and it arbitrarily picks just one of a group of highly correlated predictors. The ℓ2\ell_2 term removes the nn-variable ceiling and induces a grouping effect (correlated predictors enter or leave the model together), and a rescaling correction removes the double-shrinkage bias of the "naive" version. The LARS-EN algorithm computes the whole regularization path efficiently, and on real and simulated data the elastic net often out-predicts the lasso at comparable sparsity.

Key Claims

Concepts Introduced or Extended

Entities Mentioned

Quotes

"The elastic net often outperforms the lasso, while enjoying a similar sparsity of representation. In addition, the elastic net encourages a grouping effect, where strongly correlated predictors tend to be in or out of the model together."

"The elastic net is particularly useful when the number of predictors (pp) is much bigger than the number of observations (nn). By contrast, the lasso is not a very satisfactory variable selection method in the p≫np\gg n case."

My Take

The elastic net is one of those additions that looks trivial — "just add a ridge term to the lasso" — but resolves exactly the two situations where the lasso disappoints in practice, so it became a workhorse the moment glmnet shipped it. The grouping effect is the conceptual heart: the ℓ1\ell_1 corner geometry that gives the lasso its sparsity is also what makes it capriciously drop all-but-one of a correlated cluster, and the strictly-convex ℓ2\ell_2 term is precisely the smoothing that ties correlated coefficients together — you buy stability with a little density. The p≫np\gg n ceiling fix matters wherever signals outnumber samples (genomics, text, finance factor zoos). For the wiki it is the canonical extension of the lasso and a member of the penalized-selection family it anchors — the frequentist cousin of the Bayesian spike-and-slab / horseshoe shrinkage the variable-selection page catalogues, and a reminder that the ℓ1\ell_1/ℓ2\ell_2 mix is a dial, not a dichotomy.