import numpy as np, pandas as pd, matplotlib.pyplot as plt, warnings
warnings.filterwarnings("ignore")
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import roc_auc_score, mean_squared_error, roc_curve
BLUE="#2b6cb0"; RED="#c53030"; GREEN="#2f855a"; ORANGE="#dd6b20"; GREY="#a0aec0"; PURP="#6b46c1"
d=pd.read_csv("credit_default.csv"); feat=[c for c in d.columns if c!="default"]
h=pd.read_csv("cali_housing.csv")
print(f"Credit default: {d.shape[0]:,} clients x {len(feat)} features;  default rate {100*d['default'].mean():.1f}%  (imbalanced)")
print(f"California housing: {h.shape[0]:,} block groups x {h.shape[1]-1} features;  target median value ${h['MedHouseVal'].mean()*100:.0f}k avg\n")
print("Feature groups (credit):")
print("  credit line : LIMIT_BAL")
print("  demographics: SEX, EDUCATION, MARRIAGE, AGE")
print("  repayment   : PAY_1..PAY_6  (months of delay)")
print("  bill/payment: BILL_AMT1..6, PAY_AMT1..6")
fig,ax=plt.subplots(1,3,figsize=(15,3.8))
g=d.groupby("PAY_1")["default"].agg(["mean","size"]); g=g[g["size"]>=50]
ax[0].bar(g.index,g["mean"]*100,color=BLUE); ax[0].set_xlabel("PAY_1 (months delay, latest)"); ax[0].set_ylabel("default rate (%)")
ax[0].set_title("Default rate rises with recent delay")
q=pd.qcut(d["LIMIT_BAL"],5); gl=d.groupby(q)["default"].mean()*100
ax[1].bar(range(5),gl.values,color=GREEN); ax[1].set_xticks(range(5)); ax[1].set_xticklabels(["Q1\nlow","Q2","Q3","Q4","Q5\nhigh"])
ax[1].set_xlabel("credit-limit quintile"); ax[1].set_ylabel("default rate (%)"); ax[1].set_title("...and falls with the credit limit")
ax[2].hist(d["AGE"],bins=30,color=GREY,edgecolor="white"); ax[2].set_xlabel("age"); ax[2].set_ylabel("clients"); ax[2].set_title("Client age distribution")
plt.tight_layout(); plt.show()
print("Even single features carry clear signal: clients 2+ months behind default far more often, and higher credit limits")
print("(granted to lower-risk clients) default less. The network's job is to combine all 23 into a calibrated risk score.")

Credit default: 30,000 clients x 23 features;  default rate 22.1%  (imbalanced)
California housing: 20,640 block groups x 8 features;  target median value $207k avg

Feature groups (credit):
  credit line : LIMIT_BAL
  demographics: SEX, EDUCATION, MARRIAGE, AGE
  repayment   : PAY_1..PAY_6  (months of delay)
  bill/payment: BILL_AMT1..6, PAY_AMT1..6

Even single features carry clear signal: clients 2+ months behind default far more often, and higher credit limits
(granted to lower-risk clients) default less. The network's job is to combine all 23 into a calibrated risk score.

from mlp import MLP, gradient_check, relu, sigmoid
z=np.linspace(-4,4,200)
fig,ax=plt.subplots(1,3,figsize=(14,3.4))
ax[0].plot(z,relu(z),color=BLUE,lw=2); ax[0].set_title("ReLU (hidden): gradient 1 when active"); ax[0].axhline(0,color="k",lw=.5); ax[0].axvline(0,color="k",lw=.5)
ax[1].plot(z,np.tanh(z),color=GREY,lw=2); ax[1].set_title("tanh (old): saturates -> vanishing gradient"); ax[1].axhline(0,color="k",lw=.5)
ax[2].plot(z,sigmoid(z),color=RED,lw=2); ax[2].set_title("sigmoid (output): z -> probability"); ax[2].axhline(0.5,color="k",lw=.5,ls=":")
plt.tight_layout(); plt.show()
# one concrete forward pass to make the shapes real
d0=pd.read_csv("credit_default.csv"); feat=[c for c in d0.columns if c!="default"]
X=d0[feat].to_numpy(float); y=d0["default"].to_numpy(float)
Xtr,Xte,ytr,yte=train_test_split(X,y,test_size=0.3,random_state=0,stratify=y)
sc=StandardScaler().fit(Xtr); Ztr=sc.transform(Xtr); Zte=sc.transform(Xte)
demo=MLP([Ztr.shape[1],32,16,1],seed=0); phat=demo.forward(Ztr[:1])
print("one client through an untrained net:")
print(f"  input x           shape {Ztr[:1].shape}")
for i,a in enumerate(demo.a[1:],1):
    print(f"  layer {i} activation shape {a.shape}"+("   <- ReLU features" if i<3 else "   <- sigmoid output = P(default)"))
print(f"  predicted P(default) = {phat.ravel()[0]:.3f}  (random, because the net is untrained)")

one client through an untrained net:
  input x           shape (1, 23)
  layer 1 activation shape (1, 32)   <- ReLU features
  layer 2 activation shape (1, 16)   <- ReLU features
  layer 3 activation shape (1, 1)   <- sigmoid output = P(default)
  predicted P(default) = 0.429  (random, because the net is untrained)

for l2 in (0.0, 1e-4, 1e-2):
    chk=MLP([Ztr.shape[1],16,8,1],lr=0.1,l2=l2,seed=0)
    err=gradient_check(chk, Ztr[:200], ytr[:200], n_params=12)
    print(f"gradient check, weight decay {l2:<6} -- max relative error: {err:.2e}")
print("\nOrder 1e-8 to 1e-10 = agreement to machine precision. The hand-written backpropagation computes the true")
print("gradient. This check is not a formality: a single wrong transpose or missing activation derivative shows up")
print("here as ~1e-1. Note it is run at the weight decay actually used for training too, not only at zero -- the")
print("differenced objective has to be the same function backward() differentiates, penalty included, or the check")
print("reports a false failure on correct gradients.")

gradient check, weight decay 0.0    -- max relative error: 1.85e-08
gradient check, weight decay 0.0001 -- max relative error: 7.74e-08
gradient check, weight decay 0.01   -- max relative error: 1.53e-09

Order 1e-8 to 1e-10 = agreement to machine precision. The hand-written backpropagation computes the true
gradient. This check is not a formality: a single wrong transpose or missing activation derivative shows up
here as ~1e-1. Note it is run at the weight decay actually used for training too, not only at zero -- the
differenced objective has to be the same function backward() differentiates, penalty included, or the check
reports a false failure on correct gradients.

net=MLP([Ztr.shape[1],32,16,1],lr=0.3,l2=1e-4,seed=0).fit(Ztr,ytr,epochs=60,batch=256,seed=0)
auc_scratch=roc_auc_score(yte, net.predict_proba(Zte))
from sklearn.linear_model import LogisticRegression
auc_logit=roc_auc_score(yte, LogisticRegression(max_iter=2000).fit(Ztr,ytr).predict_proba(Zte)[:,1])
fig,ax=plt.subplots(1,2,figsize=(12,4))
ax[0].plot(net.loss_,color=BLUE,lw=2); ax[0].set_xlabel("epoch"); ax[0].set_ylabel("training cross-entropy"); ax[0].set_title("Loss falls under SGD + backprop")
fpr,tpr,_=roc_curve(yte,net.predict_proba(Zte)); ax[1].plot(fpr,tpr,color=BLUE,lw=2,label=f"MLP (AUC {auc_scratch:.3f})")
fl,tl,_=roc_curve(yte,LogisticRegression(max_iter=2000).fit(Ztr,ytr).predict_proba(Zte)[:,1]); ax[1].plot(fl,tl,color=GREY,lw=2,label=f"logistic (AUC {auc_logit:.3f})")
ax[1].plot([0,1],[0,1],"k:",lw=1); ax[1].set_xlabel("false positive rate"); ax[1].set_ylabel("true positive rate"); ax[1].set_title("Out-of-sample ROC — the net beats the linear baseline"); ax[1].legend(loc="lower right")
plt.tight_layout(); plt.show()
print(f"from-scratch MLP  OOS AUC {auc_scratch:.4f}   (logistic baseline {auc_logit:.4f})")
print("The gain over logistic is the whole point of hidden layers: learned nonlinear interactions among the 23 features.")

from-scratch MLP  OOS AUC 0.7495   (logistic baseline 0.7145)
The gain over logistic is the whole point of hidden layers: learned nonlinear interactions among the 23 features.

import torch, torch.nn as nn
torch.manual_seed(0)
Xt=torch.tensor(Ztr,dtype=torch.float32); yt=torch.tensor(ytr,dtype=torch.float32).view(-1,1)
model=nn.Sequential(nn.Linear(Ztr.shape[1],32),nn.ReLU(),nn.Linear(32,16),nn.ReLU(),nn.Linear(16,1))
opt=torch.optim.Adam(model.parameters(),lr=1e-3,weight_decay=1e-4); lossf=nn.BCEWithLogitsLoss()
for ep in range(60):
    perm=torch.randperm(len(Xt))
    for s in range(0,len(Xt),256):
        bi=perm[s:s+256]; opt.zero_grad(); lossf(model(Xt[bi]),yt[bi]).backward(); opt.step()
with torch.no_grad(): p=torch.sigmoid(model(torch.tensor(Zte,dtype=torch.float32))).numpy().ravel()
auc_torch=roc_auc_score(yte,p)
f1,f2=feat.index("PAY_1"),feat.index("LIMIT_BAL"); Z2=Ztr[:,[f1,f2]]
torch.manual_seed(0); m2=nn.Sequential(nn.Linear(2,16),nn.ReLU(),nn.Linear(16,8),nn.ReLU(),nn.Linear(8,1))
o2=torch.optim.Adam(m2.parameters(),lr=5e-3); X2=torch.tensor(Z2,dtype=torch.float32)
for ep in range(120):
    o2.zero_grad(); lossf(m2(X2),yt).backward(); o2.step()
gx,gy=np.meshgrid(np.linspace(Z2[:,0].min(),Z2[:,0].max(),150),np.linspace(np.percentile(Z2[:,1],1),np.percentile(Z2[:,1],99),150))
with torch.no_grad(): G=torch.sigmoid(m2(torch.tensor(np.c_[gx.ravel(),gy.ravel()],dtype=torch.float32))).numpy().reshape(gx.shape)
fig,ax=plt.subplots(1,2,figsize=(12.5,4.6))
ax[0].bar(["from-scratch\n(SGD)","PyTorch\n(Adam)","logistic"],[auc_scratch,auc_torch,auc_logit],color=[BLUE,GREEN,GREY])
for i,v in enumerate([auc_scratch,auc_torch,auc_logit]): ax[0].text(i,v+0.002,f"{v:.3f}",ha="center")
ax[0].set_ylim(0.68,0.79); ax[0].set_ylabel("OOS AUC"); ax[0].set_title("Same architecture: hand-coded vs PyTorch autograd")
cf=ax[1].contourf(gx,gy,G,levels=20,cmap="RdBu_r",alpha=.85); plt.colorbar(cf,ax=ax[1],label="P(default)")
ax[1].set_xlabel("PAY_1 (standardized)"); ax[1].set_ylabel("credit limit (standardized)"); ax[1].set_title("MLP decision surface — nonlinear, not a straight line")
plt.tight_layout(); plt.show()
def _torch_mlp(): return nn.Sequential(nn.Linear(Ztr.shape[1],32),nn.ReLU(),nn.Linear(32,16),nn.ReLU(),nn.Linear(16,1))
def _he(m):
    for lay in m:
        if isinstance(lay,nn.Linear):
            with torch.no_grad(): lay.weight.normal_(0.,(2./lay.weight.shape[1])**.5); lay.bias.zero_()
    return m
def _run(m,opt,epochs=60):
    for ep in range(epochs):
        perm=torch.randperm(len(Xt))
        for s in range(0,len(Xt),256):
            bi=perm[s:s+256]; opt.zero_grad(); lossf(m(Xt[bi]),yt[bi]).backward(); opt.step()
    with torch.no_grad(): return roc_auc_score(yte, torch.sigmoid(m(torch.tensor(Zte,dtype=torch.float32))).numpy().ravel())
torch.manual_seed(0); _m=_torch_mlp(); a_sgd=_run(_m,torch.optim.SGD(_m.parameters(),lr=0.3,weight_decay=1e-4))
torch.manual_seed(0); _m=_he(_torch_mlp()); a_sgd_he=_run(_m,torch.optim.SGD(_m.parameters(),lr=0.3,weight_decay=1e-4))
print(f"PyTorch MLP OOS AUC {auc_torch:.4f} vs from-scratch {auc_scratch:.4f}. Where does that {auc_torch-auc_scratch:+.4f} come from?")
print(f"   PyTorch, SGD lr=0.3, He init (our exact setup) : {a_sgd_he:.4f}   vs from-scratch {auc_scratch:.4f}")
print(f"   -> matched on optimiser AND initialisation, autograd and our hand-written backprop agree to {abs(a_sgd_he-auc_scratch):.4f} AUC.")
print(f"      That is the real check on the maths, and it passes. The rest is settings, which decompose as:")
print(f"   PyTorch, SGD lr=0.3, PyTorch default init          : {a_sgd:.4f}")
print(f"   PyTorch, Adam, PyTorch default init                : {auc_torch:.4f}")
print(f"   initialisation (PyTorch default vs He, SGD held fixed): {a_sgd-a_sgd_he:+.4f}")
print(f"   optimiser      (Adam vs SGD, init held fixed)         : {auc_torch-a_sgd:+.4f}")
print("Both matter, and here initialisation matters at least as much as the optimiser -- which is why section 4 lists")
print("it among the mechanics rather than as a detail. What a framework buys is not better gradients; it is autograd")
print("for any architecture, plus well-chosen defaults you would otherwise have to discover yourself.")

PyTorch MLP OOS AUC 0.7653 vs from-scratch 0.7495. Where does that +0.0159 come from?
   PyTorch, SGD lr=0.3, He init (our exact setup) : 0.7484   vs from-scratch 0.7495
   -> matched on optimiser AND initialisation, autograd and our hand-written backprop agree to 0.0011 AUC.
      That is the real check on the maths, and it passes. The rest is settings, which decompose as:
   PyTorch, SGD lr=0.3, PyTorch default init          : 0.7582
   PyTorch, Adam, PyTorch default init                : 0.7653
   initialisation (PyTorch default vs He, SGD held fixed): +0.0098
   optimiser      (Adam vs SGD, init held fixed)         : +0.0071
Both matter, and here initialisation matters at least as much as the optimiser -- which is why section 4 lists
it among the mechanics rather than as a detail. What a framework buys is not better gradients; it is autograd
for any architecture, plus well-chosen defaults you would otherwise have to discover yourself.

from sklearn.ensemble import RandomForestClassifier, RandomForestRegressor
from sklearn.linear_model import LinearRegression
import xgboost as xgb
hf=[c for c in h.columns if c!="MedHouseVal"]; Xh=h[hf].to_numpy(float); yh=h["MedHouseVal"].to_numpy(float)
Xh_tr,Xh_te,yh_tr,yh_te=train_test_split(Xh,yh,test_size=0.3,random_state=0)
sch=StandardScaler().fit(Xh_tr); Rtr=sch.transform(Xh_tr); Rte=sch.transform(Xh_te); ym,ys=yh_tr.mean(),yh_tr.std()
torch.manual_seed(0); reg=nn.Sequential(nn.Linear(Rtr.shape[1],64),nn.ReLU(),nn.Linear(64,32),nn.ReLU(),nn.Linear(32,1))
oR=torch.optim.Adam(reg.parameters(),lr=3e-3,weight_decay=1e-4); mse=nn.MSELoss()
XtR=torch.tensor(Rtr,dtype=torch.float32); ytR=torch.tensor((yh_tr-ym)/ys,dtype=torch.float32).view(-1,1)
for ep in range(80):
    perm=torch.randperm(len(XtR))
    for s in range(0,len(XtR),256):
        bi=perm[s:s+256]; oR.zero_grad(); mse(reg(XtR[bi]),ytR[bi]).backward(); oR.step()
with torch.no_grad(): pr=reg(torch.tensor(Rte,dtype=torch.float32)).numpy().ravel()*ys+ym
mlp_rmse=mean_squared_error(yh_te,pr)**.5
# one seed is not a result: refit across seeds so the comparison with the forest is honest
_rm=[]
for _s in range(5):
    torch.manual_seed(_s)
    _r=nn.Sequential(nn.Linear(Rtr.shape[1],64),nn.ReLU(),nn.Linear(64,32),nn.ReLU(),nn.Linear(32,1))
    _o=torch.optim.Adam(_r.parameters(),lr=3e-3,weight_decay=1e-4)
    for ep in range(80):
        perm=torch.randperm(len(XtR))
        for s in range(0,len(XtR),256):
            bi=perm[s:s+256]; _o.zero_grad(); mse(_r(XtR[bi]),ytR[bi]).backward(); _o.step()
    with torch.no_grad(): _p=_r(torch.tensor(Rte,dtype=torch.float32)).numpy().ravel()*ys+ym
    _rm.append(mean_squared_error(yh_te,_p)**.5)
_rm=np.array(_rm)
cl={"logistic / linear":auc_logit,"neural net (MLP)":auc_torch,
    "random forest":roc_auc_score(yte,RandomForestClassifier(n_estimators=400,min_samples_leaf=5,random_state=0,n_jobs=-1).fit(Xtr,ytr).predict_proba(Xte)[:,1]),
    "XGBoost":roc_auc_score(yte,xgb.XGBClassifier(n_estimators=300,learning_rate=0.05,max_depth=4,verbosity=0,n_jobs=-1).fit(Xtr,ytr).predict_proba(Xte)[:,1])}
rg={"logistic / linear":mean_squared_error(yh_te,LinearRegression().fit(Rtr,yh_tr).predict(Rte))**.5,"neural net (MLP)":mlp_rmse,
    "random forest":mean_squared_error(yh_te,RandomForestRegressor(n_estimators=300,min_samples_leaf=3,random_state=0,n_jobs=-1).fit(Xh_tr,yh_tr).predict(Xh_te))**.5,
    "XGBoost":mean_squared_error(yh_te,xgb.XGBRegressor(n_estimators=300,learning_rate=0.05,max_depth=4,verbosity=0,n_jobs=-1).fit(Xh_tr,yh_tr).predict(Xh_te))**.5}
print(pd.DataFrame({"credit AUC (higher=better)":cl,"California RMSE (lower=better)":rg}).round(3).to_string())
fig,ax=plt.subplots(1,2,figsize=(13.5,4.2)); nm=list(cl); col=[GREY,PURP,GREEN,BLUE]
ax[0].barh(nm,[cl[k] for k in nm],color=col); ax[0].set_xlim(0.5,0.8); ax[0].invert_yaxis(); ax[0].set_title("Credit — test AUC (higher better)")
ax[1].barh(nm,[rg[k] for k in nm],color=col); ax[1].invert_yaxis(); ax[1].set_title("California — test RMSE (lower better)")
plt.tight_layout(); plt.show()
print(f"\nThe California MLP figure above is one seed. Across 5 seeds the RMSE runs {_rm.min():.4f}-{_rm.max():.4f} (mean {_rm.mean():.4f},")
print(f"sd {_rm.std():.4f}), which brackets the random forest's {rg['random forest']:.4f}: on regression the net and the forest are a")
print(f"tie once seed variation is admitted, and quoting a single run either way would be reading noise. Against XGBoost")
print(f"({rg['XGBoost']:.4f}) there is no ambiguity -- the gap is {_rm.mean()-rg['XGBoost']:+.3f}, far outside that spread.")
print("So: the MLP clears the linear baseline comfortably, ties the forest, and loses to boosting -- the honest and")
print("well-replicated result. Neural nets earn their keep on STRUCTURED data (images, sequences, language), the")
print("subject of the next four notebooks.")

                   credit AUC (higher=better)  California RMSE (lower=better)
logistic / linear                       0.715                           0.737
neural net (MLP)                        0.765                           0.518
random forest                           0.775                           0.523
XGBoost                                 0.774                           0.494

The California MLP figure above is one seed. Across 5 seeds the RMSE runs 0.5182-0.5320 (mean 0.5247,
sd 0.0054), which brackets the random forest's 0.5235: on regression the net and the forest are a
tie once seed variation is admitted, and quoting a single run either way would be reading noise. Against XGBoost
(0.4943) there is no ambiguity -- the gap is +0.030, far outside that spread.
So: the MLP clears the linear baseline comfortably, ties the forest, and loses to boosting -- the honest and
well-replicated result. Neural nets earn their keep on STRUCTURED data (images, sequences, language), the
subject of the next four notebooks.

# torch and nn already imported in section 5
# --- classification: P(default) vs credit limit ---
lim=Xtr[:,feat.index("LIMIT_BAL")]/1000.0                      # NT dollars, thousands
qs=np.unique(np.quantile(lim,np.linspace(0,1,16))); bb=np.clip(np.digitize(lim,qs[1:-1]),0,len(qs)-2)
ctr=[lim[bb==k].mean() for k in range(len(qs)-1)]; emp=[ytr[bb==k].mean() for k in range(len(qs)-1)]
grid=np.linspace(lim.min(),np.quantile(lim,0.99),300)
lo=LogisticRegression().fit(lim.reshape(-1,1),ytr); plo=lo.predict_proba(grid.reshape(-1,1))[:,1]
mu_,sd_=lim.mean(),lim.std(); torch.manual_seed(0)
nnc=nn.Sequential(nn.Linear(1,16),nn.ReLU(),nn.Linear(16,8),nn.ReLU(),nn.Linear(8,1))
oc=torch.optim.Adam(nnc.parameters(),lr=5e-3); XL=torch.tensor(((lim-mu_)/sd_).reshape(-1,1),dtype=torch.float32); yL=torch.tensor(ytr,dtype=torch.float32).view(-1,1)
for ep in range(200): oc.zero_grad(); nn.BCEWithLogitsLoss()(nnc(XL),yL).backward(); oc.step()
with torch.no_grad(): pnn=torch.sigmoid(nnc(torch.tensor(((grid-mu_)/sd_).reshape(-1,1),dtype=torch.float32))).numpy().ravel()
# --- regression: value vs median income ---
inc=Xh_tr[:,hf.index("MedInc")]; gi=np.linspace(inc.min(),np.quantile(inc,0.99),300)
lin=LinearRegression().fit(inc.reshape(-1,1),yh_tr); pli=lin.predict(gi.reshape(-1,1))
mI,sI=inc.mean(),inc.std(); torch.manual_seed(0)
nnr1=nn.Sequential(nn.Linear(1,16),nn.ReLU(),nn.Linear(16,8),nn.ReLU(),nn.Linear(8,1))
orr=torch.optim.Adam(nnr1.parameters(),lr=5e-3); XI=torch.tensor(((inc-mI)/sI).reshape(-1,1),dtype=torch.float32); yI=torch.tensor((yh_tr-ym)/ys,dtype=torch.float32).view(-1,1)
for ep in range(250): orr.zero_grad(); nn.MSELoss()(nnr1(XI),yI).backward(); orr.step()
with torch.no_grad(): pri=nnr1(torch.tensor(((gi-mI)/sI).reshape(-1,1),dtype=torch.float32)).numpy().ravel()*ys+ym
fig,ax=plt.subplots(1,2,figsize=(13,4.6))
ax[0].scatter(ctr,emp,color="black",zorder=5,label="empirical proportion")
ax[0].plot(grid,plo,color=GREY,lw=2,label="logistic (S-curve)"); ax[0].plot(grid,pnn,color=BLUE,lw=2,label="neural net")
ax[0].set_xlabel("credit limit (NT dollars, thousands)"); ax[0].set_ylabel("P(default)"); ax[0].set_title("Classification: default rate vs credit limit"); ax[0].legend()
ax[1].scatter(inc,yh_tr,s=4,alpha=.08,color=GREY); ax[1].plot(gi,pli,color=ORANGE,lw=2,label="linear"); ax[1].plot(gi,pri,color=BLUE,lw=2,label="neural net")
ax[1].set_xlabel("median income"); ax[1].set_ylabel("median house value (100k USD)"); ax[1].set_xlim(inc.min(),np.quantile(inc,0.99)); ax[1].set_title("Regression: value vs income"); ax[1].legend()
plt.tight_layout(); plt.show()
print("Classification: the empirical default rate falls with the credit limit; the logistic fit is a smooth monotone")
print("S-curve, while the neural net bends to follow the dots more closely (and can wiggle in sparse regions). Regression:")
print("the linear fit is a straight line through the income-price cloud; the neural net curves to the concave, saturating")
print("relationship the line misses -- the same nonlinearity the hidden layers buy, shown directly against the data.")

Classification: the empirical default rate falls with the credit limit; the logistic fit is a smooth monotone
S-curve, while the neural net bends to follow the dots more closely (and can wiggle in sparse regions). Regression:
the linear fit is a straight line through the income-price cloud; the neural net curves to the concave, saturating
relationship the line misses -- the same nonlinearity the hidden layers buy, shown directly against the data.

block	features	meaning
credit line	`LIMIT_BAL`	the client's credit limit (NT dollars)
demographics	`SEX`, `EDUCATION`, `MARRIAGE`, `AGE`	who the client is
repayment status	`PAY_1`…`PAY_6`	months of delay in each of the last 6 months (−1 = paid on time, 1–8 = months late) — the most informative block
billing / payment	`BILL_AMT1`…`6`, `PAY_AMT1`…`6`	monthly bill amounts and amounts actually paid

Neural Networks I — the multilayer perceptron & backpropagation¶

Built from scratch (manual backprop) → PyTorch, with a full data description and the honest tabular verdict¶

1. The data and the prediction goals¶

Primary task — credit-card default (binary classification)¶

Secondary task — California housing (regression)¶

2. The forward pass — from inputs to a probability¶

3. Backpropagation from scratch — the chain rule, then a gradient check¶

4. Training the network — the mechanics that matter¶

5. The same network in PyTorch — what a framework automates¶

6. The honest verdict on tabular data¶

7. Proportions vs predictions — fitted curves along a feature¶

8. Summary¶