Space-Varying Regression Model

bayesianspatial-econometricsmcmcmarkov-random-fieldhierarchical-bayesregressionblocking

Definition

A Space-Varying Regression Model (SVRM) is a Bayesian regression model in which each spatial unit ii has its own coefficient vector βi\beta_i, replacing the global β\beta of a standard linear model. Spatial coherence is enforced by a pairwise difference prior — a multivariate Markov random field (MRF) prior that penalises differences between the coefficient vectors of neighbouring units. The result is a fully Bayesian spatial model in which coefficient heterogeneity is data-driven and regularised by the neighbourhood graph.

Key Ideas

How It Works

Pairwise Difference Prior

Neighbours i∼ji \sim j are defined by shared borders; wij=1w_{ij} = 1 if i∼ji \sim j, 0 otherwise, and wi+=∑jwijw_{i+} = \sum_j w_{ij} is the degree. Define the graph Laplacian WW with Wii=wi+W_{ii} = w_{i+} and Wij=−wijW_{ij} = -w_{ij}. The prior is:

f(β∣κ,η)∝κnr/2∣η∣n/2exp⁡ ⁣{−κ2 β′(W⊗η)β}f(\beta|\kappa,\eta) \propto \kappa^{nr/2}|\eta|^{n/2}\exp\!\left\{-\frac{\kappa}{2}\,\beta'(W \otimes \eta)\beta\right\}

where η\eta is an r×rr \times r precision hyperparameter. The quadratic form equals ∑i∼j(βi−βj)′η(βi−βj)\sum_{i \sim j}(\beta_i - \beta_j)'\eta(\beta_i - \beta_j). The observational precision κ∼Gamma(νκ/2,νκSκ/2)\kappa \sim \text{Gamma}(\nu_\kappa/2, \nu_\kappa S_\kappa/2) and η∼Wishart(ν/2,νS/2)\eta \sim \text{Wishart}(\nu/2, \nu S/2).

Full Conditional of βi\beta_i (Scheme A — Single-Site Gibbs)

(βi∣β∂i,κ,η,y)∼N ⁣(ai,  κ−1Ri)(\beta_i|\beta_{\partial i}, \kappa, \eta, y) \sim N\!\bigl(a_i,\; \kappa^{-1}R_i\bigr) Ri=(xixi′+wi+η)−1,ai=Ri ⁣(xiyi+wi+ηβˉ∂i)R_i = (x_ix_i' + w_{i+}\eta)^{-1}, \qquad a_i = R_i\!\left(x_i y_i + w_{i+}\eta\bar\beta_{\partial i}\right)

where βˉ∂i=wi+−1∑j∼iwijβj\bar\beta_{\partial i} = w_{i+}^{-1}\sum_{j\sim i}w_{ij}\beta_j is the weighted neighbour mean. Each draw requires only an r×rr\times r matrix inversion. Slow mixing because neighbouring βi\beta_i's are strongly correlated.

Block Update (Scheme B)

Jointly draw (β,κ)(\beta, \kappa) from the Normal-Gamma marginal:

(β,κ∣η,y)∼NG(a,R,ν^κ/2,S^κ/2)(\beta, \kappa | \eta, y) \sim \text{NG}(a, R, \hat\nu_\kappa/2, \hat S_\kappa/2)

where R=(X′X+W⊗η)−1R = (X'X + W \otimes \eta)^{-1}, a=RX′ya = RX'y, ν^κ=νκ+n\hat\nu_\kappa = \nu_\kappa + n. The nr×nrnr \times nr precision matrix X′X+W⊗ηX'X + W \otimes \eta is sparse; Rue (2001) sparse Cholesky makes this feasible for n∼102n \sim 10^2–10310^3.

Analytical Marginalisation (Scheme C)

Integrate out (β,κ)(\beta, \kappa) analytically to obtain the marginal π(η∣y)\pi(\eta|y), then draw η\eta via Metropolis-Hastings or sampling-importance-resampling (SIR). This is the preferred scheme when rr is small: it achieves the lowest chain autocorrelation by eliminating the high-dimensional β\beta from the sampler entirely.

MCMC Efficiency Ranking

C >> B >> A in effective sample size per unit time. Scheme A is approximately 20× faster per iteration but has much lower acceptance per effective sample. The gap between schemes is smaller than in state-space models because the observation likelihood is highly informative about each βi\beta_i — single-site updates are less hampered by prior correlation when data overwhelm prior.

Proper Prior Variant

Add a pinning term to break impropriety:

f(β∣κ,η,δ)∝exp⁡ ⁣{−κ2[∑i∼j(βi−βj)′η(βi−βj)+δ∑i(βi−bi)′η(βi−bi)]}f(\beta|\kappa,\eta,\delta) \propto \exp\!\left\{-\frac{\kappa}{2}\left[\sum_{i\sim j}(\beta_i-\beta_j)'\eta(\beta_i-\beta_j) + \delta\sum_i(\beta_i - b_i)'\eta(\beta_i-b_i)\right]\right\}

The full conditional becomes (βi∣…)∼N ⁣(wiβˉ∂i+(1−wi)bi,  (1/(κ(wi++δ)))η−1)(\beta_i|\ldots) \sim N\!\bigl(w_i\bar\beta_{\partial i} + (1-w_i)b_i,\; (1/(\kappa(w_{i+}+\delta)))\eta^{-1}\bigr) where wi=wi+/(wi++δ)w_i = w_{i+}/(w_{i+}+\delta).

Why It Matters

Open Questions

Related