t-SNE & UMAP — Manifold Learning, and What These Plots Cannot Tell You

Visualisation, Not Compression

PCA (principal component analysis) and autoencoders compress data so the codes can be reused downstream. t-SNE (t-distributed stochastic neighbour embedding) and UMAP (uniform manifold approximation and projection) have a narrower goal: render high-dimensional data in two dimensions for a human to look at. Both prioritise local neighbourhood structure — points close in high dimensions stay close in 2-D — which makes groups far more legible than a variance-maximising linear projection does. t-SNE converts pairwise distances into neighbour probabilities in both spaces and minimises the Kullback–Leibler divergence between them — a measure of how far one probability distribution sits from another; UMAP instead connects each point to its nearest neighbours in a graph, weighting each link by how confident it is that the two are genuinely close, then arranges that graph in two dimensions so the strong links stay short. Both are therefore local by construction: they are told to get the neighbourhoods right and are indifferent to what happens between distant regions, which is precisely the property the global-structure test below exploits.

Scoring an embedding without rewarding it for its own objective

Scoring an embedding needs some care, because the obvious measure is the wrong one. Silhouette rewards compact, well-separated geometry in the embedding — which is exactly what these methods optimise for, whether or not the groups are real. Three measures that don't reward the method's own objective: nearest-neighbour accuracy reading a held-out label off the 2-D map, trustworthiness (are embedded neighbours genuine high-dimensional neighbours?), and the share of each point's 15 nearest neighbours in 50-D that survive the projection.

On those, the result is stronger than the silhouette column suggests. PCA keeps only 14% of each point's neighbourhood and carries the label barely better than half the time; t-SNE and UMAP keep three to four times as much and reach 0.766 and 0.750. For comparison, the autoencoder's 2-D latent code on this same data reached 0.673 — better than PCA, still short of methods built for the job.

The Noise Control

The caveat that costs people published claims is that these methods produce convincing clusters in data that has none, and it deserves a demonstration rather than a warning. Embedding 2,000 points drawn from a single isotropic Gaussian — no structure of any kind — returns a picture of well-separated islands. Scored by silhouette, that noise lands above the genuine ten-class structure of Fashion-MNIST.

In 50 dimensions the noise correctly scores near zero: there is nothing to find. Projected to 2-D by either method it scores roughly twice what real class structure does. Every group in those panels is manufactured by the algorithm, and no amount of looking at the picture would reveal it. This is also the reason silhouette cannot referee these embeddings in the first place — it scores the geometry the method imposes, not the fidelity of the map.

Two Claims for UMAP, Measured

Two comparative claims are routinely used to prefer UMAP over t-SNE, and neither survived measurement at this scale. On speed, t-SNE was faster at every size tried — 0.9s against 1.1s at $n=1{,}000$, 8.6s against 13.8s at $n=4{,}000$. UMAP's advantage is a large-$n$ claim, and scikit-learn's Barnes–Hut t-SNE is well optimised in exactly this range. On global structure, ranking pairwise distances against the original 50-D space gives t-SNE +0.695 against UMAP's +0.598 — UMAP preserves the coarse arrangement less well here, not more. The usual comparison is against a t-SNE started from a random layout; initialised from PCA, as it is throughout, t-SNE keeps the arrangement it was handed. PCA beats both at +0.891, which is the point — it is the method that optimises for exactly that.

What does separate them is that UMAP can embed new points and vanilla t-SNE cannot. That matters whenever the map has to be applied to data arriving later, and it is a difference in kind rather than degree.

Mapping the Market by Sector

Applied to assets, the 48 stocks each become a 312-week return series and UMAP places them in the plane. Coloured by sector — assigned from ticker knowledge, never used to fit the map — technology names sit together, financials together, energy apart. The tempting reading is that UMAP discovered the sector taxonomy, and that is the wrong way round: the structure is already in the returns, which is why the correlation matrix in the PCA example has the block structure it does. What a 2-D map can do is carry that structure down to two dimensions without losing it, and that is the checkable claim.

Measured against the raw return space as a ceiling and PCA as the linear alternative, UMAP loses almost nothing and PCA loses noticeably more. The value of the map is real but precisely bounded: it makes visible, at almost no cost in fidelity, structure that already existed in 312 dimensions where nobody could look at it. One caveat on scale — UMAP is embedding only 48 points here, far below the sample sizes it is designed for. It is stable across seeds (0.507 ± 0.010), but a 48-point map is a picture to reason from, not evidence.

Where this sits

The same 48-stock cross-section is compressed linearly in PCA, SVD & Factor Analysis — where the block structure these maps render is visible directly in the correlation matrix — and nonlinearly in Autoencoders, whose 2-D latent code is the natural third comparison to the two embeddings here. Clustering is the same question asked without a picture: the noise control on this page is the visual counterpart of the fat-tailed control there, both establishing what a method reports when there is nothing to report.

Notebook

Downloads

Fashion-MNIST is downloaded by torchvision when the notebook runs. Both embeddings come from library implementations (sklearn.manifold.TSNE, umap-learn), so there is no module to accompany this example — the work here is in the diagnostics rather than the algorithms.

References

• van der Maaten, L. & Hinton, G. (2008). Visualizing data using t-SNE. Journal of Machine Learning Research 9, 2579–2605.
• McInnes, L., Healy, J. & Melville, J. (2018). UMAP: Uniform Manifold Approximation and Projection for dimension reduction. arXiv:1802.03426.
• Wattenberg, M., Viégas, F. & Johnson, I. (2016). How to use t-SNE effectively. Distill. — clusters in data that has none, and why perplexity changes the picture
• Kobak, D. & Linderman, G. C. (2021). Initialization is critical for preserving global data structure in both t-SNE and UMAP. Nature Biotechnology 39, 156–157. — why PCA initialisation closes the global-structure gap
• Venna, J. & Kaski, S. (2006). Local multidimensional scaling. Neural Networks 19(6–7), 889–899. — trustworthiness
• Chari, T. & Pachter, L. (2023). The specious art of single-cell genomics. PLOS Computational Biology 19(8), e1011288. — the cost of over-reading these plots in an applied field