The Black–Litterman Model¶

Risk and Asset Allocation — the from-scratch engine¶

"Give a mean-variance optimiser your best guess of expected returns and it will hand you back a portfolio that is 100% long one asset and 100% short another. Black–Litterman is how the industry made Markowitz usable."

Self-contained; no prior reading of Meucci's Risk and Asset Allocation (Springer 2005, ch. 9) required.

The problem Black–Litterman solves¶

Projects 1–2 showed that plugging sample average returns into a mean-variance optimiser is a disaster: tiny estimation errors in $\mu$ get amplified into gigantic, unstable long/short positions. Practitioners noticed this in the 1980s and mostly gave up on optimisation — until Fischer Black and Robert Litterman (Goldman Sachs, 1990) reframed the problem.

Their insight: don't estimate expected returns from scratch at all. Instead,

  1. start from a neutral prior — the expected returns that would make today's market portfolio optimal (the "equilibrium" or "implied" returns, obtained by running the optimiser backwards); then
  2. nudge that prior toward the investor's specific views ("tech will beat staples", "energy returns 0.2%/week"), by an amount that reflects how confident the investor is in each view.

The result is an expected-return vector that equals equilibrium where you have no opinion and tilts only where you do — producing stable, intuitive portfolios. And the update is exactly a Bayesian normal-normal calculation: the equilibrium is the prior, the views are the data, Black–Litterman is the posterior. Everything from Project 2 applies.

Roadmap¶

  1. Reverse optimisation: the equilibrium prior $\Pi$.
  2. The naive mean-variance disaster (why we need this).
  3. Views: how to state them as $P,Q,\Omega$.
  4. The Black–Litterman posterior (the master formula) as Bayesian updating.
  5. The portfolio: sensible tilts, not chaos.
  6. The confidence dial: from market portfolio to full conviction.
In [1]:
import numpy as np, pandas as pd, matplotlib.pyplot as plt
import blacklitterman as bl

plt.rcParams.update({"figure.figsize": (9,4.5), "axes.grid": True, "grid.alpha": .25,
                     "axes.spines.top": False, "axes.spines.right": False, "font.size": 11})
BLUE, ORANGE, GREEN, RED, GREY, PURP = "#2b6cb0","#dd6b20","#2f855a","#c53030","#718096","#6b46c1"
np.set_printoptions(precision=3, suppress=True)

R = pd.read_csv("sector_etf_weekly.csv", index_col=0, parse_dates=True)
SECTORS = [c for c in R.columns if c != "SPY"]
X = R[SECTORS].values
Sigma = np.cov(X.T)
T, N = X.shape
print("Panel:", N, "sector ETFs,", T, "weeks (weekly log-returns, %)")

Panel: 10 sector ETFs, 310 weeks (weekly log-returns, %)

The real dataset and "the market"¶

Weekly log-returns (%) of the ten SPDR sector ETFs, Jan 2019 – Dec 2024 (312 weeks). Each ETF holds the S&P 500 members of one GICS sector. Black–Litterman needs a market-capitalisation portfolio to reverse-engineer the equilibrium; we use the approximate 2024 S&P 500 sector cap-weights below, and define the market as that cap-weighted combination of the ten sectors — a self-contained stand-in for the broad US equity market.

Ticker Sector ≈cap weight Ticker Sector ≈cap weight
XLK Technology 32% XLI Industrials 8%
XLF Financials 13% XLP Staples 6%
XLV Health Care 12% XLE Energy 4%
XLY Discretionary 10% XLU Utilities 2.5%
XLC Communications 9% XLB Materials 2.3%
In [2]:
# Market-cap weights (approx S&P 500 GICS, 2024) and the implied market portfolio
capw = {"XLK":32,"XLF":13,"XLV":12,"XLY":10,"XLC":9,"XLI":8,"XLP":6,"XLE":4,"XLU":2.5,"XLB":2.3}
w_mkt = np.array([capw[s] for s in SECTORS]); w_mkt = w_mkt / w_mkt.sum()
mkt_ret = X @ w_mkt                                        # the market portfolio's return series
delta = bl.risk_aversion(mkt_ret.mean(), mkt_ret.var())   # risk aversion from its own Sharpe
print("market risk-aversion  delta = %.4f" % delta)
print("market weekly return  mean = %.3f%%   vol = %.3f%%" % (mkt_ret.mean(), mkt_ret.std()))

market risk-aversion  delta = 0.0420
market weekly return  mean = 0.312%   vol = 2.727%

1. Reverse optimisation: the equilibrium prior $\Pi$¶

Standard (forward) optimisation takes expected returns $\mu$ and produces weights $w$. Black–Litterman runs it backwards: it takes the observed market weights $w_{\text{mkt}}$ as given (the market has already "optimised") and asks what expected returns would make these weights optimal? The answer is the implied equilibrium return $$\Pi = \delta\,\Sigma\,w_{\text{mkt}},$$ where $\delta = \mathbb E[r_{\text{mkt}}]/\operatorname{Var}[r_{\text{mkt}}]$ is the market's risk-aversion (its price of risk). These $\Pi$ are our prior mean for expected returns; the prior is $\mu\sim\mathcal N(\Pi,\ \tau\Sigma)$ with $\tau$ a small scalar for the prior's own uncertainty.

A sanity check with real teeth: feeding $\Pi$ back into the forward optimiser must return the market portfolio, $\;w=\tfrac1\delta\Sigma^{-1}\Pi = w_{\text{mkt}}$. It does, exactly — so with no views Black–Litterman holds the market, which is the neutral thing to do.

In [3]:
Pi = bl.implied_returns(delta, Sigma, w_mkt)
w_check = bl.mv_weights(Pi, Sigma, delta)                 # forward-optimise Pi -> should be w_mkt
print("no-views recovers the market portfolio? max|w - w_mkt| = %.2e\n" % np.max(np.abs(w_check - w_mkt)))

order = np.argsort(Pi)
fig, ax = plt.subplots(1, 2, figsize=(12, 4.4))
ax[0].barh(np.array(SECTORS)[order], Pi[order]*52, color=BLUE)   # annualise for readability
ax[0].set_title("Equilibrium implied returns $\\Pi$ (annualised %)"); ax[0].set_xlabel("% / year")
ax[1].barh(np.array(SECTORS)[order], w_mkt[order]*100, color=GREY)
ax[1].set_title("Market-cap weights"); ax[1].set_xlabel("%")
plt.tight_layout(); plt.show()
print("High-beta sectors (Tech, Energy, Discretionary) carry the highest implied returns:")
print("they contribute most to market risk, so equilibrium must reward them more.")

no-views recovers the market portfolio? max|w - w_mkt| = 5.62e-16
No description has been provided for this image 
High-beta sectors (Tech, Energy, Discretionary) carry the highest implied returns:
they contribute most to market risk, so equilibrium must reward them more.

2. Why we need this: the naive mean-variance disaster¶

Before adding views, let us see what happens if we do the "obvious" thing — estimate expected returns by their historical averages and optimise. This is the trap Projects 1–2 warned about; here it is on live data.

In [4]:
mu_hist = X.mean(0)
w_naive = bl.mv_weights(mu_hist, Sigma, delta)

idx = np.arange(N)
plt.figure(figsize=(11,4.6))
plt.bar(idx-0.2, w_naive*100, width=0.4, color=RED,  label="naive MV (historical means)")
plt.bar(idx+0.2, w_mkt*100,  width=0.4, color=GREY, label="market portfolio")
plt.xticks(idx, SECTORS); plt.axhline(0, color="k", lw=.6); plt.ylabel("weight (%)")
plt.title("Naive mean-variance produces extreme, unstable bets"); plt.legend(); plt.show()
print("naive MV: min %.0f%%  max %.0f%%  gross leverage %.0f%%" % (w_naive.min()*100, w_naive.max()*100, np.abs(w_naive).sum()*100))
print("market  : min %.0f%%  max %.0f%%  gross leverage %.0f%%" % (w_mkt.min()*100, w_mkt.max()*100, np.abs(w_mkt).sum()*100))
print("\nThe naive portfolio takes huge long/short positions driven by noise in the return averages.")
print("No investor would hold it. Black-Litterman keeps the sanity of the market and adds views on top.")
No description has been provided for this image 
naive MV: min -109%  max 188%  gross leverage 518%
market  : min 2%  max 32%  gross leverage 100%

The naive portfolio takes huge long/short positions driven by noise in the return averages.
No investor would hold it. Black-Litterman keeps the sanity of the market and adds views on top.

3. Stating views: $P$, $Q$ and $\Omega$¶

A view is a statement about expected returns, written as a linear equation $P\mu = Q + \varepsilon$, $\varepsilon\sim\mathcal N(0,\Omega)$:

  • $P$ (the "pick" matrix, $K\times N$): one row per view, selecting the assets involved.
  • $Q$ ($K$): the expected return each view asserts.
  • $\Omega$ ($K\times K$, diagonal): the uncertainty of each view — small $\Omega$ = high confidence.

Views come in two flavours:

  • Absolute — "Energy will return 0.2%/week": row $= [0,\dots,1,\dots,0]$ on XLE, $Q=0.2$.
  • Relative — "Tech will outperform Staples by 0.6%/week": row $=[+1\text{ on XLK},\,-1\text{ on XLP}]$, $Q=0.6$.

For $\Omega$ we use the standard He–Litterman choice $\Omega=\operatorname{diag}(P(\tau\Sigma)P')$: each view is as uncertain as the prior thinks its portfolio is — no arbitrary numbers needed. We will express two views and, crucially, read them relative to equilibrium — that is what determines the tilt.

In [5]:
tau = 0.05
views = [
    {"assets": ["XLK","XLP"], "weights": [1,-1], "q": 0.60},   # Tech beats Staples by 0.60%/wk
    {"assets": ["XLE"],        "weights": [1],    "q": 0.20},   # Energy returns 0.20%/wk
]
P, Q = bl.view_matrix(SECTORS, views)
Omega = bl.omega_proportional(P, tau, Sigma)

# compare each view to what equilibrium already implies
print("View 1 (relative): Tech - Staples")
print("   view says %.3f%%/wk;  equilibrium implies %.3f%%/wk  -> view is BULLISH on the spread" %
      (0.60, Pi[SECTORS.index("XLK")] - Pi[SECTORS.index("XLP")]))
print("View 2 (absolute): Energy")
print("   view says %.3f%%/wk;  equilibrium implies %.3f%%/wk  -> view is BEARISH on energy" %
      (0.20, Pi[SECTORS.index("XLE")]))
print("\nP =\n", P.astype(int))

View 1 (relative): Tech - Staples
   view says 0.600%/wk;  equilibrium implies 0.186%/wk  -> view is BULLISH on the spread
View 2 (absolute): Energy
   view says 0.200%/wk;  equilibrium implies 0.351%/wk  -> view is BEARISH on energy

P =
 [[ 0  0  0  0  0  1 -1  0  0  0]
 [ 0  0  1  0  0  0  0  0  0  0]]

4. The Black–Litterman posterior (the master formula)¶

Now the Bayesian update. Prior $\mu\sim\mathcal N(\Pi,\tau\Sigma)$; view "likelihood" $P\mu\sim\mathcal N(Q,\Omega)$. Combining them (the standard normal-normal conjugate calculation — the very same algebra as Project 2) gives the posterior $\mu\sim\mathcal N(\mu_{\text{BL}},M)$ with $$ \boxed{\;\mu_{\text{BL}} = \Big[(\tau\Sigma)^{-1} + P'\Omega^{-1}P\Big]^{-1}\Big[(\tau\Sigma)^{-1}\Pi + P'\Omega^{-1}Q\Big]\;} $$ and $M = \big[(\tau\Sigma)^{-1}+P'\Omega^{-1}P\big]^{-1}$. Read it as a precision-weighted average: the posterior mean blends the equilibrium $\Pi$ and the views $Q$, each weighted by how precise it is. Where a view is confident (small $\Omega$) it pulls $\mu_{\text{BL}}$ toward $Q$; where it is vague it barely moves. This is shrinkage of the views toward equilibrium — Project 1's idea, one more time.

In [6]:
res = bl.black_litterman(Pi, Sigma, tau, P, Q, Omega)
mu_bl = res["mu_bl"]

order = np.argsort(Pi)
x = np.arange(N)
plt.figure(figsize=(11,4.8))
plt.plot(x, Pi[order]*52,   "o-", color=GREY, lw=2, label="equilibrium prior $\\Pi$")
plt.plot(x, mu_bl[order]*52, "s-", color=BLUE, lw=2, label="Black-Litterman posterior $\\mu_{BL}$")
for a in ["XLK","XLP","XLE"]:
    i = list(np.array(SECTORS)[order]).index(a)
    plt.annotate(a, (i, mu_bl[order][i]*52), color=RED, fontweight="bold", xytext=(0,8), textcoords="offset points", ha="center")
plt.xticks(x, np.array(SECTORS)[order]); plt.ylabel("expected return (annualised %)")
plt.title("Views tilt the equilibrium prior — mostly where the views point"); plt.legend(); plt.show()
print("Tech is pulled UP and Staples DOWN (view 1); Energy is pulled DOWN (view 2, below equilibrium).")
print("Sectors with no views barely move -- Black-Litterman leaves them at equilibrium.")
No description has been provided for this image 
Tech is pulled UP and Staples DOWN (view 1); Energy is pulled DOWN (view 2, below equilibrium).
Sectors with no views barely move -- Black-Litterman leaves them at equilibrium.

5. The portfolio: sensible tilts, not chaos¶

Finally, optimise with the posterior returns: $w_{\text{BL}} = \tfrac1\delta\Sigma^{-1}\mu_{\text{BL}}$. Because $\mu_{\text{BL}}$ is close to equilibrium except where views bite, the resulting weights are the market portfolio plus a few deliberate tilts — not the deranged long/short book of naive MV.

In [7]:
w_bl = bl.mv_weights(mu_bl, Sigma, delta)
tilt = w_bl - w_mkt

fig, ax = plt.subplots(1, 2, figsize=(13, 4.8))
ax[0].bar(x-0.27, w_mkt*100,   width=0.27, color=GREY, label="market")
ax[0].bar(x,      w_bl*100,    width=0.27, color=BLUE, label="Black-Litterman")
ax[0].bar(x+0.27, w_naive*100, width=0.27, color=RED,  label="naive MV")
ax[0].set_xticks(x); ax[0].set_xticklabels(SECTORS, rotation=90); ax[0].axhline(0,color="k",lw=.6)
ax[0].set_ylabel("weight (%)"); ax[0].set_title("Portfolios compared"); ax[0].legend()
ax[0].set_ylim(-110, 170)
cols = [GREEN if t>0 else RED for t in tilt]
ax[1].bar(x, tilt*100, color=cols)
ax[1].set_xticks(x); ax[1].set_xticklabels(SECTORS, rotation=90); ax[1].axhline(0,color="k",lw=.6)
ax[1].set_ylabel("tilt vs market (%)"); ax[1].set_title("Black-Litterman tilts: only where views point")
plt.tight_layout(); plt.show()
print("gross leverage:  market %.0f%%   Black-Litterman %.0f%%   naive MV %.0f%%"
      % (np.abs(w_mkt).sum()*100, np.abs(w_bl).sum()*100, np.abs(w_naive).sum()*100))
print("BL overweights Tech, underweights Staples and Energy -- exactly the views -- and holds")
print("everything else at market weight. Compare the naive book's %.0f%% gross leverage." % (np.abs(w_naive).sum()*100))
No description has been provided for this image 
gross leverage:  market 100%   Black-Litterman 219%   naive MV 518%
BL overweights Tech, underweights Staples and Energy -- exactly the views -- and holds
everything else at market weight. Compare the naive book's 518% gross leverage.

6. The confidence dial¶

How hard the portfolio tilts depends on how confident the investor is — encoded in $\Omega$. Scaling $\Omega$ by a factor $c$ interpolates between the two extremes: $c\to\infty$ (no confidence) leaves the market portfolio untouched; $c\to0$ (certainty) makes the views bind exactly. Watch the tilt on the viewed sectors grow as confidence rises.

In [8]:
confidences = np.array([20, 5, 1, 0.25, 0.05])          # multiplier on Omega: high -> vague
tracked = ["XLK","XLP","XLE"]
paths = {a: [] for a in tracked}
for c in confidences:
    r = bl.black_litterman(Pi, Sigma, tau, P, Q, c*Omega)
    w = bl.mv_weights(r["mu_bl"], Sigma, delta)
    for a in tracked: paths[a].append((w[SECTORS.index(a)] - w_mkt[SECTORS.index(a)])*100)

xlab = ["vague\n(c=20)","","base\n(c=1)","","confident\n(c=0.05)"]
for a,c_ in zip(tracked, [BLUE,ORANGE,GREEN]):
    plt.plot(range(len(confidences)), paths[a], "o-", color=c_, lw=2, label=a)
plt.axhline(0, color=GREY, lw=.8)
plt.xticks(range(len(confidences)), xlab); plt.ylabel("tilt vs market (%)")
plt.title("From market portfolio to full conviction as view confidence rises")
plt.legend(); plt.show()
print("At low confidence the portfolio IS the market; as confidence rises the tilts on the viewed")
print("sectors grow smoothly. The investor dials exactly how much to bet on each opinion.")
No description has been provided for this image 
At low confidence the portfolio IS the market; as confidence rises the tilts on the viewed
sectors grow smoothly. The investor dials exactly how much to bet on each opinion.

7. Summary and the bridge to robust allocation¶

  • Reverse optimisation turns the market portfolio into a neutral equilibrium prior $\Pi=\delta\Sigma w_{\text{mkt}}$; with no views Black–Litterman holds the market exactly.
  • Naive mean-variance on historical means is unusable — extreme, unstable weights. Black–Litterman fixes the inputs, not the optimiser.
  • Views $P,Q,\Omega$ are combined with the prior by the master formula — a plain Bayesian normal-normal update. The posterior $\mu_{\text{BL}}$ shrinks the views toward equilibrium, and the portfolio is the market plus targeted tilts, with a confidence dial ($\Omega$) controlling their size.
  • This is the same Bayesian machinery as Projects 1–2, now with the prior supplied by the market and the "data" supplied by the investor.

The bridge to Project 4. Black–Litterman still commits to point inputs — a single $\mu_{\text{BL}}$ and $\Sigma$ — and optimises as if they were exact. But we know from Project 2 that they are uncertain (the posterior $M$ quantifies exactly how much). The final project, Robust Bayesian allocation, takes that uncertainty seriously: instead of optimising for one point, it seeks a portfolio that performs well across the whole posterior — the last defence against estimation error.