Imagine crumpling a flat piece of paper into a ball. The paper is fundamentally 2-dimensional, but it is now twisted and folded through 3D space. A manifold can be pictured like that crumpled paper. Manifold learning asks whether a simpler structure can explain data represented in many dimensions.

The catch: straight-line distance through 3D space is not necessarily the same as distance along the crumpled paper's surface. Two points might be close if measured straight through the air, but far apart along the folds. Some manifold methods try to recover this surface geometry. Others, including t-SNE, focus more narrowly on preserving local neighborhood affinities.

▶ What is a manifold?
A manifold is a space that is locally similar to ordinary Euclidean space, even when its larger shape is curved. Think of Earth: from far away it is a sphere, but a small neighborhood can look nearly flat. Manifold learning begins with the hypothesis that observed data may vary along fewer effective degrees of freedom than its representation suggests. Whether that hypothesis fits a dataset must be tested.

Try it: Crumple the Paper (3D)

0%

💡 Drag to rotate, scroll to zoom

Straight-line distance (through space):
100 units
Sampled surface-following curve (illustrative):
100 units

This green curve follows one sampled route between the points. It is not a shortest-path calculation and should not be read as an exact geodesic.

The Core Problem: Which Distance Matters?

Euclidean distance measures a straight-line distance in the chosen coordinate space. If a curved wall separates two locations, that line may pass through the wall. Geodesic distance is the length of the shortest valid path along a specified surface or manifold.

Example: two people on opposite sides of a mountain can be close by straight-line distance but far apart along a walkable route. Isomap and related graph methods try to approximate this kind of surface-aware distance. Not every manifold-learning method estimates or preserves geodesics.

▶ What is geodesic distance?
Geodesic distance is the length of a shortest path between two points that remains on a specified manifold. On an ideal sphere, a geodesic follows a great-circle arc. In sampled data, the manifold is usually unknown, so graph shortest paths are estimates whose quality depends on sampling, noise, the metric, and neighborhood construction.

Similarity vs. Closeness

Here is the key possibility: two points can be close under one representation or metric but far apart under another. If the manifold model is appropriate, ambient straight-line distance and distance along the inferred structure can tell different stories.

Rolled-up scroll: Two points on different layers are close if you measure straight through (high-dimensional space), but far apart if you follow the paper (manifold).

Handwritten digits: raw pixel distance can be sensitive to shifts, stroke thickness, and noise. Images that people judge as similar can therefore be separated in pixel space, while images with similar aggregate pixel values need not share the same label.

This is why the choice of representation and metric matters: straight-line distance in the original feature space can be misleading for some datasets. A manifold method is useful only if its structural assumptions match the data and the task.

Manifold-learning algorithms offer ways to model local or surface-aware relationships rather than relying only on raw ambient distance. They do not reveal a uniquely "true" similarity automatically. Results still depend on preprocessing, metric choice, neighborhood size, noise, sampling, and the algorithm's objective.

The Manifold Assumption: A dataset represented with many coordinates may have a lower intrinsic dimension because fewer independent factors generate most of its meaningful variation. A handwritten digit image has 784 pixel coordinates, for example, but valid digits occupy only a restricted portion of that space. The size, smoothness, and even existence of a useful lower-dimensional manifold are empirical assumptions to test, not fixed facts. The same caution applies to text embeddings and other learned representations.

Local Neighborhoods: The Building Block

If a manifold is smooth and sampled densely enough, a small neighborhood can look approximately flat. Graph-based methods can exploit that idea in three steps:

Step 1: Find neighbors. For each data point, find nearby points under a chosen metric, often Euclidean distance after suitable preprocessing. If sampling is sufficiently dense and the metric is appropriate, these local neighbors may also be close on the underlying structure.

Step 2: Build a network. Create a graph: each point is a node, and you draw edges between neighbors. This creates a connected web of points.

Step 3: Use the network to measure distance. Instead of measuring a single straight line between two points, trace a shortest path along graph edges. Under suitable sampling and neighborhood conditions, this graph distance can approximate distance along the manifold. More points can help, but only if noise, shortcuts, disconnected regions, and neighborhood choices are controlled.

▶ What is k-nearest neighbors (k-NN)?
k-nearest neighbors identifies the k closest observations under a chosen metric. For k=5 with Euclidean distance, it selects the five points with the smallest Euclidean distance. Treating them as manifold neighbors is a modeling decision. It can work when sampling is adequate and local geometry is well represented, but it can fail when noise, scaling, density differences, or folds create false connections.

Three Popular Approaches

1. Isomap (Isometric Mapping)
Build a neighborhood graph, estimate manifold distances with shortest paths through that graph, then apply classical multidimensional scaling to find a lower-dimensional representation that preserves those estimated distances as closely as possible. Isomap works best when the neighborhood graph faithfully samples a connected manifold and its geometry can be represented approximately isometrically.

▶ What is multidimensional scaling (MDS)?
MDS takes a dissimilarity matrix and finds a lower-dimensional configuration that represents those dissimilarities according to its objective. Classical MDS has a specific relationship to Euclidean distance geometry. It is like reconstructing a map from distance data, but the reconstruction may be approximate when the supplied distances cannot be represented faithfully in the target dimension.

2. Locally Linear Embedding (LLE)
The idea: approximate each point as a linear combination of its neighbors. This can work when sufficiently small neighborhoods are close to locally linear. First find weights that reconstruct each point from its neighbors. Then find a lower-dimensional representation where the same reconstruction weights remain useful.

The embedding objective preserves local reconstruction relationships rather than all-pairs or graph-shortest-path distances. Whether those relationships represent useful geometry depends on neighborhood size, sampling, noise, and regularization.

3. t-SNE (t-Distributed Stochastic Neighbor Embedding)
Convert pairwise relationships into local affinity distributions, then optimize a low-dimensional visualization whose affinities resemble them. t-SNE is designed primarily to reveal local neighborhood structure. Its optimization is stochastic, so different seeds and settings can produce different layouts. Distances between separated groups, group sizes, empty space, and the overall arrangement should not be interpreted as preserved global geometry or definitive cluster evidence.

▶ What is stochastic neighbor embedding?
Instead of making only a hard neighbor decision, stochastic neighbor embedding represents local similarities with probability distributions. Nearby or similar points receive greater affinity. The embedding then minimizes a mismatch between high-dimensional and low-dimensional affinity distributions. This is useful for visualization, but it does not guarantee preservation of every distance or global relationship.

Why This Matters

Understanding manifold structure changes how you think about similarity. It explains why:

  • Distance can concentrate: For some high-dimensional distributions and metrics, nearest and farthest distances become less distinguishable. This is not a universal rule at a fixed dimension. Scaling, sparsity, representation, metric, and data distribution all matter.
  • Visualization is a lossy projection: A useful plot should be interpreted according to the method's objective. For t-SNE, local neighborhoods are generally more meaningful than global spacing or orientation.
  • Visual clusters need validation: Apparent groups can change with parameters, initialization, density, and sampling. A separated shape in a 2D embedding is not, by itself, proof of a cluster in the original data.
  • Interpolation depends on the representation: A path that stays near a learned data manifold may produce more plausible transitions in some applications, but neither manifold-space nor ambient-space interpolation is guaranteed to be meaningful.

Practical insight: Methods such as k-means optimize Euclidean geometry and can work well when roughly spherical groups under that metric are a good model. Manifold-aware methods encode different assumptions about neighborhoods or geometry. Compare methods with task-relevant validation, stability checks, and out-of-sample evaluation rather than assuming one geometry is always superior.

The Catch: The Manifold Assumption Isn't Always True

Not every dataset is well described by one smooth, well-sampled manifold. Manifold learning can distort structure when its assumptions fail. It is most plausible when:

  • local neighborhoods are meaningful under the chosen representation and metric;
  • fewer effective factors explain variation than the coordinate count suggests;
  • sampling is dense enough relative to intrinsic dimension and curvature; and
  • noise, outliers, boundaries, disconnected components, and density variation are checked.

The takeaway: Manifold learning provides a family of models for testing whether local neighborhoods or lower-dimensional geometry offer a useful view of data. An embedding is evidence to examine, not a revelation of one uniquely true shape.

Living technical reference: See the scikit-learn manifold learning guide for current descriptions, parameters, limitations, and implementation links. Accessed July 30, 2026.