Dynamic Discrete Choice Model

Definition

A dynamic discrete choice (DDC) model describes a forward-looking agent choosing among a finite action set $\mathcal{A}=\{0,1,\ldots,J\}$ in each period to maximize expected discounted payoffs. State $(x_t,\varepsilon_t)$ combines observable variables $x_t$ and i.i.d. per-alternative shocks $\varepsilon_t(a)$; the Bellman equation $V(x,\varepsilon)=\max_{a\in\mathcal{A}}\{v(a,x)+\varepsilon(a)+\beta E[V(x',\varepsilon')\mid a,x]\}$ defines optimal value, and the conditional choice probabilities $P(a\mid x;\theta)$ are equilibrium objects that depend on structural parameter vector $\theta$.

Key Ideas

• Conditional choice probabilities (CCPs): the estimable equilibrium objects; under Type-I extreme value errors $P(a\mid x)=\exp\{v(a,x)\}/\sum_j\exp\{v(j,x)\}$ (Dynamic Programming (DP) conditional logit).
• Hotz-Miller invertibility: $P(\cdot\mid x;\theta)$ is invertible in value function differences $v(a,x)-v(0,x)$; enables CCP estimators that bypass full DP solution at each $\theta$.
• Nested fixed-point (NFXP): Rust (1987) algorithm solves DP at every trial $\theta$ inside the likelihood loop; computationally intensive but asymptotically efficient (partial Maximum Likelihood Estimation, MLE).
• Nested Pseudo-Likelihood (NPL) algorithm: Aguirregabiria-Mira (2002) iterative update $\hat{P}^{k+1}=\Lambda(\hat{P}^k,\hat{\theta}(\hat{P}^k))$; fixed point coincides with partial MLE; better finite-sample bias than Hotz-Miller two-step.
• Simulation-based CCP: Hotz et al. (1994) forward simulation replaces analytical value function inversion for large or continuous state spaces.
• Initial conditions problem: when unobservables are serially correlated, the period-1 unobservable distribution depends on pre-sample history — requires an auxiliary model or simulation to handle.
• Dynamic games / Markov Perfect Equilibrium (MPE): in multi-agent settings the solution concept is MPE; each agent's Bellman equation has other agents' CCPs embedded in the transition.

How It Works

1. Specify payoff functions $u(a,x;\theta)$ and state transition $F(x'\mid a,x;\theta)$.
2. Compute value function $V^\theta(x)$ via DP fixed-point iteration (NFXP) or form value-function differences from inverted CCPs (CCP methods).
3. Build (pseudo-)likelihood $\mathcal{L}(\theta)=\prod_t P(a_t\mid x_t;\theta)$.
4. Maximize via Berndt-Hall-Hall-Hausman (BHHH) method (NFXP), plug-in pseudo-MLE (Hotz-Miller), or NPL iteration.

Why It Matters

DDC models recover preference and technology parameters invariant to counterfactual policy changes, unlike reduced-form estimates. Applications span retirement timing, education investment, occupational choice, firm entry/exit, and technology adoption. CCP methods made DDC estimation feasible for large state spaces, greatly expanding empirical work in labor economics and industrial organization.

Open Questions

• MPE multiplicity makes identification and estimation of dynamic games difficult; equilibrium selection assumptions are typically untestable.
• NPL convergence is not guaranteed in all models; multiple fixed points may exist.
• Incorporating serially correlated unobservables and unobserved heterogeneity simultaneously raises the curse of dimensionality.
• Bayesian MCMC approaches (Imai-Jain-Ching 2009) offer alternative estimation but convergence properties are less well characterized.

Related

• Markov Chain Monte Carlo
• Selection Model
• Victor Aguirregabiria
• Pedro Mira
• Michael Keane
• Aguirregabiria and Mira (2010)