Cluster Analysis

Most individuals can grasp the concept of cluster analysis within ten minutes. The part that takes much more time – and which almost all guides fail to address – is choosing the correct algorithm, preparing for the algorithm, and determining whether the results mean anything at all.

That’s what we cover here.

What Is Cluster Analysis?

Cluster analysis is all about grouping the objects according to their similarities without prior labeling. In other words, you do not tell the algorithm into which groups to divide the objects; it will find out the groups itself based on the degree of similarity among data points.

This “without prior labeling” aspect makes cluster analysis an example of unsupervised learning – a branch of machine learning dealing with finding the structure in the data. It would be a mistake not to mention this because it might sound too complicated until the algorithm shows its magic through plotting the scatter plot, in which points turn into clear clusters.

In technical terms, the aim of the clustering algorithms is to maximize the intercluster similarity while minimizing the intracluster one.

Cluster analysis cannot be mixed up with classification. While the latter is the process of predicting which class to which a new object belongs using the labeled training set, cluster analysis can help you find out whether there are any classes in your dataset in the first place.

How Cluster Analysis Actually Works

However, any machine learning algorithm requires a mechanism to calculate the distance between two data points. Distance metrics provide the mathematical foundation underlying all clustering algorithms.

The Euclidean distance computes the shortest distance between two points as the crow flies. This metric is commonly used when working with numerical data where all dimensions are roughly comparable.

Manhattan distance sums up the absolute differences of each dimension. Imagine navigating through a city on a rectangular grid, where the Manhattan distance is preferred over the Euclidean distance due to obstacles. This metric ignores large differences in a single dimension compared to the others, making it less sensitive to outliers.

Cosine similarity computes the angular difference between two vectors. In reality, this metric is utilized when working with text. Two documents are considered more similar to each other when their vocabulary overlaps, regardless of the difference in length, which may vary up to tenfold.

Gower distance can handle mixed data types within a dataset, containing both numerical and categorical data. Gower distance is recommended when dealing with mixed data types; however, tutorials frequently overlook its utility.

The 6 Main Types of Cluster Analysis Algorithms

Partitioning Methods (K-Means)

k-means splits your dataset into precisely K clusters, where K is something you determine in advance. The technique selects K centroids, assigns every data point to its closest centroid, recalculates the actual centers of each formed cluster, and iterates the process until nothing changes anymore.

The measure it’s trying to minimize is called WCSS (Within-Cluster Sum of Squares). WCSS is essentially the sum of all squared distances from data points to their cluster centroid. Less WCSS, better clusters!

The algorithm was invented by Stuart P. Lloyd in 1957 while working at Bell Labs, allegedly doing the math manually since the computing equipment couldn’t cope yet; his paper was published in IEEE Transactions on Information Theory way later, in 1982. Even so, it’s still the most common algorithm for clustering in industrial applications today.

Best suited for: datasets with numerical data, of medium-to-large sizes and roughly spherical clusters with equal densities. Not good for: anything that doesn’t meet that criterion and especially situations where the number of clusters is unknown.

Hierarchical Methods

Hierarchical clustering forms a tree of clusters, unlike k-means which form flat partitions. There are two possible ways to construct such trees:

Agglomerative method (bottom-up approach): Each data point initially belongs to its own cluster. One finds the closest clusters and merges them. One repeats this process until only one big cluster remains.

Divisive method (top-down approach): All data points belong to one cluster initially. Then one splits this cluster into several smaller ones and continues doing so until each cluster consists of only one element.

The agglomerative method is much more popular because of its efficiency and practicality. The result of the procedure is a dendrogram, i.e., a hierarchical tree that represents which clusters are united at which distances. The advantage of the approach is that one runs an algorithm once and can choose the number of clusters by simply cutting the obtained dendrogram at a certain height.

There are three possible approaches that specify how one should measure distances between clusters for further mergers. The single linkage approach uses the minimal distance between points from different clusters. The complete linkage approach uses the maximal distance between elements. Finally, average linkage takes an average distance between all possible pairs from two clusters.

Best for: exploratory work where you don’t know how many clusters to expect. One run gives you the complete hierarchy to examine.

Density-Based Methods (DBSCAN)

DBSCAN (Density-based Spatial Clustering of Applications with Noise) is a method developed by Martin Ester, Hans-Peter Kriegel, Jörg Sander, and Xiaowei Xu in 1996 at the KDD conference. What did the researchers need to solve? K-means could not discover non-spherical clusters and lacked noise detection.

In DBSCAN, however, for each data point, the question was posed as follows: are there at least minPts other points located at a distance of ε (epsilon)? Then the point belongs to a cluster, and its neighborhood becomes expanded further on. If no, then such a point is noise without a cluster.

K-means does not create arbitrarily shaped clusters and does not mark noisy outliers explicitly. These are the characteristics of the problem of fraud detection since fraudulent actions do not fit into any density pattern and belong to the noise layer in the model.

Best suited for: arbitrary cluster shapes, outlier detection, spatial data. Not good at: clusters with drastically different densities in the dataset (the same ε cannot suit all areas simultaneously).

Grid-Based Methods

The grid approach involves partitioning the data space into grids, calculating the density of each grid, and combining grids of high density to form clusters. Examples of grid approaches include STING and CLIQUE.

This approach is fast, since no pairwise distance calculation is required. However, the downside is that the shape of the clusters formed is limited to being rectangular in nature. If the cluster shape is diagonal or curved in shape, it will be approximated to the nearest block.

In summary, while they appear in almost all classifications of clustering algorithms, they have been overtaken by better approaches.

Model-Based Methods (Gaussian Mixture Models)

Gaussian Mixture Models are built under the premise that your data comes from a mixture of bell-shaped distributions, called Gaussians, with different means and variances. Unlike K-means, which assigns a data point strictly to one cluster only, GMM computes the probability of a data point for each cluster.

This is the core distinction. A point located in-between two clusters will be assigned a 60/40 probability ratio instead of choosing just one of the two. The results will be more truthful in the case of overlapping clusters.

Voice recognition software traditionally employs GMM, treating each individual’s voice as an individual Gaussian distribution. Risk stratification in medicine is another example where, in reality, a person can exist in a state of two risk categories.

Constraint-Based Methods

With constraint-based clustering, we can easily incorporate our knowledge directly. The must-link constraint ensures that two particular points belong to the same cluster while the cannot-link constraint guarantees that two particular points cannot belong to the same cluster.

The application example is healthcare. According to the guidelines in medicine, some patients must be clustered together or should never be included in the same cluster under any circumstances.

How to Choose the Right Clustering Algorithm

The truth is that this is the mistake I have seen practitioners make over and over again, and I do not think most tutorials on algorithms will help much here because while they tell you how an algorithm works, they never tell you why you would use it.

There are four criteria that influence the choice: the nature of your data, the structure of your clusters, the knowledge of the number of clusters you want, and whether there are significant outliers in your data.

Or better yet, the question should be how will the algorithm fail.

K-means breaks down when the data doesn’t form spherical, equally sized clusters, when outliers are present (because they skew the centroid away from the actual center), and when there are categorical variables (since you can’t calculate the mean of something like product type).

DBSCAN breaks down when clusters have vastly different densities. You can’t use one ε to capture both a densely populated urban cluster and a sparsely populated rural cluster — the rural cluster will be considered noise.

Hierarchical clustering breaks at scale. Hierarchical clustering on millions of points will be feasible by 2026, but it’s not the tool of choice when the dataset is very large.

GMM breaks when the clusters are perfectly compact and separable. The probabilistic nature of GMM introduces unnecessary complexity where K-means suffices.

Your Situation	Recommended Start
Numerical data, spherical clusters, large dataset	K-Means
Categorical data only	K-Modes
Mixed numerical + categorical	K-Prototypes (Gower distance)
Irregular cluster shapes or outliers to identify	DBSCAN
Unknown number of clusters, exploratory work	Hierarchical
Overlapping clusters, probabilistic output needed	GMM
Must respect domain constraints on grouping	Constraint-based

How to Determine the Number of Clusters

Two approaches. Neither should be used individually; combine both.

Elbow Method. Execute the algorithm with K values ranging between 1 and 10. Note down WCSS for each iteration. Plot it. WCSS will decrease as K increases since clustering leads to lower total within-cluster distance; however, at some point, the slope will become smaller and result in a bent line. And the bend is what you want as a possible K value.

The catch here is that the elbow may not always appear distinctively. Yes, it seems like a no-brainer, but looking at the smoothed curve won’t give any results in the case of a lack of elbow. That’s why you shouldn’t rely on it only.

Silhouette Score. This measure compares each observation to its cluster and the nearest one by similarity. Its value ranges from -1 (observation definitely doesn’t belong to its cluster) to +1 (it definitely does belong). Average silhouette scores over all observations. The K with the largest average Silhouette Score indicates the greatest cluster integrity.

Execute both. Suppose the Elbow points out that K=4 and the domain of your business requires K=5. You need to write it down and talk to an expert.

Preprocessing Before You Cluster

In a nutshell, preparation of your data is more important than the choice of algorithm itself.

Normalize your data prior to applying K-means. Always. If you have one column that includes annual income ranging from hundreds of thousands and another that includes age in tens, then the income column will weigh too much and every distance measure will be skewed towards income. Normalization or standardization is required for every numerical column before using any centroid-based algorithm.

Remove or handle missing values first. Not all algorithms can work with NaN values and some algorithms that can work technically will treat such a situation as an indication that the corresponding observation doesn’t fit the pattern. Numerical missing values should be substituted with means; categorical ones with modes or a separate class.

Consider reducing the number of dimensions in case you deal with high-dimensional space. For example, with 50 or even more columns, it won’t make sense to calculate any distances since, due to the curse of dimensionality, the distances may appear meaningless. In such cases, PCA will help reduce the number of dimensions.

Three steps, in order: normalize, handle missing values, reduce dimensions if needed. But here’s what happens when you skip this: your clusters will reflect your data’s scaling quirks and gaps rather than its actual structure, and you won’t discover that until a domain expert looks at the output and tells you these segments make no sense… at which point you rerun everything from scratch anyway. 

Running Cluster Analysis in Python (scikit-learn)

Weirdly enough, most tutorials overcomplicate this step. scikit-learn’s sklearn.cluster module handles the practical majority of clustering work, and K-means takes eight lines.

python
from sklearn.cluster import KMeans
from sklearn.preprocessing import StandardScaler

# Normalize first -- always
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

# Run K-means with K=3
kmeans = KMeans(n_clusters=3, random_state=42, n_init=10)
kmeans.fit(X_scaled)

labels = kmeans.labels_    # cluster assignment per point
wcss   = kmeans.inertia_   # WCSS for elbow method

The random_state=42 matters. K-means initializes centroids randomly, so results differ between runs without a fixed seed. Setting it makes your output reproducible.

DBSCAN:

python
from sklearn.cluster import DBSCAN

db = DBSCAN(eps=0.5, min_samples=5)
db.fit(X_scaled)

labels = db.labels_
# Points labeled -1 are noise -- your outliers, explicitly flagged

That -1 noise label is the feature K-means doesn’t have. Right. Those are your outliers, labeled without a separate step.

Cluster Analysis Applications in 2026

Customer segmentation is still the most common use case. K-means is the default starting point; hierarchical clustering is used when teams need multiple granularity levels from one run — broad segments for strategy, detailed sub-segments for campaign targeting.

Fraud detection maps directly to DBSCAN. Fraudulent transactions don’t cluster with normal behavior; they fall in the noise layer. Labeling them is what the algorithm was built for.

Gene expression research uses hierarchical clustering to group genes that behave similarly across experimental conditions. The dendrogram output maps to biological intuition about gene families in a way that flat cluster assignments don’t.

Vector database retrieval — and this is the application that’s grown most in the past two years — clusters embedding vectors from LLM-based systems so retrieval can search a relevant cluster rather than scanning the entire database. If you’re building anything with retrieval-augmented generation, you’re depending on clustering at the infrastructure level whether you realize it or not.

Anomaly detection across domains: network intrusion, medical outliers, manufacturing defects. Normal behavior clusters; anomalies don’t. The pattern holds regardless of the domain.

Common Mistakes in Cluster Analysis

Start with a failure mode, then the principle behind it.

A data team runs K-means on customer purchase data. The clusters look reasonable on a scatter plot. They build three marketing strategies around them. Six months later, their highest-value customers — a small, dense group spending 10x the average — had been absorbed into a larger moderate-spending cluster, because K-means forced equal-sized spheres onto data that wasn’t shaped that way. The small high-value group was never treated as distinct.

That’s kind of the whole point of knowing failure modes before you run your first job.

Not normalizing before K-means. The example above traces partly to this. Normalize first. Every time.

Choosing K without evidence. Picking 3 clusters because it looks right on a plot, or 5 because you have 5 sales regions, produces clusters that reflect your assumption rather than your data’s structure. Use the Elbow Method and the Silhouette Score. If they disagree with your intuition, that tension is worth a conversation.

Using K-means on non-spherical data. Plot two PCA components before choosing an algorithm. If the data forms elongated or irregular shapes, K-means will force it into spheres. DBSCAN or GMM will give you a more honest result.

Running K-means on outlier-heavy data. A single extreme point can pull a centroid away from the true cluster center and corrupt every assignment downstream. Remove confirmed outliers first, or switch to an algorithm that handles them natively.

Accepting the output without validation. Clustering is exploratory. The groups your algorithm finds are patterns consistent with its mathematical assumptions — not facts about the real world. Show the output to someone who knows the domain before you act on it.

Cluster Analysis vs Classification

Both terms come up together enough that the distinction is worth making explicitly.

You’ve likely seen them presented as competing techniques. They’re not; they serve different questions at different points in a project.

Use clustering when you don’t have labels and want to discover whether natural groups exist in your data. Use classification when you have labeled training examples and want to predict labels for new data.

The most common sequence in practice: run clustering on unlabeled data to discover segments, then use those segments as labels to train a classification model. Clustering defines the target; classification predicts it for new observations. Neither replaces the other.

Algorithm Comparison Table

Algorithm	Best For	Handles Noise	Needs K Predefined	Cluster Shape	Scales Well
K-Means	Numerical, spherical, large datasets	No	Yes	Spherical	Yes
K-Medoid (PAM)	Small datasets, noisy numerical data	Partial	Yes	Spherical	No
Hierarchical	Unknown K, exploratory analysis	No	No	Any	No
DBSCAN	Irregular shapes, outlier detection	Yes	No	Arbitrary	Moderate
GMM	Overlapping clusters, probabilistic output	No	Yes	Elliptical	Moderate
Constraint-Based	Domain-rule-constrained grouping	No	Optional	Any	Varies

Frequently Asked Questions