Bühlmann-Kalisch-Meier (2014) High-Dimensional Statistics with a View Toward Applications in Biology

Summary

This review surveys statistical methods for high-dimensional data ($p\gg n$), with particular emphasis on the recent problem of quantifying uncertainty — constructing p-values and confidence intervals, and controlling false positives — after high-dimensional model selection. The core setting is the sparse linear model, estimated by the Lasso; the review then moves to methods that produce valid inference on top of such estimators (sample splitting, stability selection, de-sparsified Lasso), to multiple-testing error control (FWER, FDR), and briefly to graphical modeling and causal inference from observational data. Everything is illustrated with R packages on a genomic riboflavin-production dataset ($p=4088$ genes, $n=71$).

Key Claims

• Sparsity rescues estimation when $p\gg n$. In $Y=X\beta+\varepsilon$ with more covariates than observations, ordinary least squares overfits and is unidentified; assuming only a few $\beta_j\neq0$ makes estimation well-posed. The Lasso (Tibshirani 1996), $\hat\beta=\arg\min_\beta\|Y-X\beta\|_2^2/n+\lambda\|\beta\|_1$, does simultaneous shrinkage and variable selection.
• Selection is not inference. A selected model does not come with valid p-values, because the same data were used to choose and to test; naive post-selection standard errors are anticonservative. The review's central theme is how to recover honest uncertainty statements.
• Sample splitting / multi-splitting. Split the data: select variables on one half, compute p-values on the other. Repeating the split many times and aggregating (multi-sample-splitting, Meinshausen-Meinshausen-Bühlmann 2009) yields reproducible, FWER-controlling p-values.
• Stability selection. Meinshausen-Bühlmann (2010): repeatedly subsample and record how often each variable is selected; keep variables with high selection frequency. This controls the expected number of false positives with weak assumptions and is far more robust than a single Lasso fit.
• De-sparsified (debiased) Lasso. Correcting the Lasso's shrinkage bias yields an asymptotically normal estimator for each coefficient, giving p-values and confidence intervals for individual $\beta_j$ even when $p\gg n$; the scaled Lasso (Sun-Zhang 2012) provides a companion estimate of the noise level $\sigma_\varepsilon$.
• Multiple-testing control. With thousands of hypotheses, control the family-wise error rate (FWER) or the false discovery rate (FDR = expected proportion of false rejections among rejections) rather than per-test error.
• Beyond regression. The same sparsity toolkit extends to graphical models (estimating conditional-independence networks by neighborhood selection) and to causal inference from observational data (the PC algorithm and IDA for bounds on intervention effects).

Concepts Introduced or Extended

• High-Dimensional Inference
• Lasso
• Variable Selection

Entities Mentioned

• Peter Bühlmann
• Markus Kalisch
• Lukas Meier

Quotes

"We review statistical methods for high-dimensional data analysis and pay particular attention to recent developments for assessing uncertainties in terms of controlling false positive statements (type I error) and p-values."

My Take

The review's organizing insight is the gap between selection and inference: the Lasso literature had, by 2014, largely solved point estimation and prediction under sparsity, but a Lasso fit tells you nothing trustworthy about whether a given coefficient is really nonzero. The methods it foregrounds — sample splitting, stability selection, and especially the de-sparsified Lasso — are exactly the bridge to valid p-values that the wiki's Lasso and variable-selection pages stop short of. For a Bayesian-leaning wiki it also usefully marks the frequentist counterpart to spike-and-slab posterior inclusion probabilities and horseshoe shrinkage: both camps are answering "which variables, and how sure are we?" under $p\gg n$, by different routes. The main caveat is that these guarantees lean on sparsity and design (compatibility/beta-min) conditions that are unverifiable in practice — the honest uncertainty is conditional on assumptions that biology rarely lets you check.