Hierarchical Clustering: Complete Guide With Python Code

Hierarchical clustering has been implemented on your dataset, giving a dendrogram by SciPy. Now you’re facing a tree of branches wondering how and where to cut, whether your choice of linkage method was incorrect, or perhaps the one and only isolated data point ruined it all.

This exact problem – of knowing how to proceed following the execution of the algorithm – is exactly what most tutorials don’t cover at all.

This tutorial will cover not just that, but also explain how hierarchical clustering works from scratch, how to interpret a dendrogram correctly, which linking method should be used with your dataset, and how to tackle the case of a large dataset where the algorithm becomes unmanageable. All of the provided code snippets are well-commented and justified. No artificial examples pretending to be practical ones using make_blobs.

One upfront disclaimer: in my opinion, the limitations part of this guide is more informative than the advantages one. After all, knowing when you shouldn’t use a particular algorithm is also half of the skill – that’s exactly what your competitors won’t tell you.

What Is Hierarchical Clustering?

Hierarchical clustering is an unsupervised machine learning technique that creates hierarchical clusters of data points in a treelike structure known as the dendrogram, and importantly, you do not have to specify the number of clusters ahead of time.

It is important because of one thing. With the k-means approach, you have to make a decision on the number of clusters even before you see the behavior of the data. With hierarchical clustering, however, you perform clustering and then use the dendrogram to determine how to split it. You see what needs to be done instead of making an assumption beforehand.

The downside of the approach is clear, but we will see it later.

How Hierarchical Clustering Works: The Two Approaches

Each hierarchical clustering method is always either growing trees from the bottom or deconstructing them from the top. It seems pedantic, but you care when deciding which to use.

Agglomerative (Bottom-Up) — The One Most People Use

Begin with 150 cities within Europe, and each city is considered a cluster. Choose any two characteristics – mean temperature in January and total rainfall for the year. The agglomerative process involves determining which two cities are closest to one another and then merging the two cities into one cluster. Repeat. Repeat. Until all 150 cities become just one cluster.

This is what the process actually entails.

The five steps are as follows:

1. Each data item is treated as a cluster (150 cities = 150 clusters)
2. Calculate the distance between each pair of clusters
3. Combine the two closest clusters
4. Recalculate the distance matrix with the new cluster
5. Repeat the last two steps till only one cluster is left

A clear point to make is that every merge is permanent. For example, when Helsinki and Oslo are merged together in step 3 due to their proximity in terms of temperature values, the same cities will always be merged together in all the further steps, even if there could be another suitable choice.

That is what makes hierarchical clustering greedy and leads to some unexpected outcomes, since the algorithm doesn’t think in advance and acts on the spot.

Divisive (Top-Down) — The Less Common Approach

The opposite is true for divisive clustering; here you start with all your data grouped into a single cluster and iteratively break that down into smaller and smaller pieces until every point forms its own cluster. This algorithm has an official name and it’s called DIANA (Divisive Analysis Clustering), created in 1990 by Kaufman and Rousseeuw.

However, divisive clustering is very rarely used in practice because there is no ready-made solution in scikit-learn for it. You’d have to write a recursive k-means splitter yourself, which can prove to be computationally expensive. It is theoretically interesting because it gives the algorithm access to the full data distribution from the start, which IBM claims increases the accuracy of large cluster identification; however, in practice we don’t use it.

Reading a Dendrogram — The Skill Nobody Teaches Properly

All competitive articles present a dendrogram. None of them explain how to interpret it.

This is how you should interpret it properly:

The x-axis denotes your data points (clusters). The y-axis denotes the distance at which two clusters were joined — the further down the y-axis, the farther the two objects that were joined were from each other. A horizontal line that joins two branches represents the fact that these branches were joined at the y-axis value where this line is located.

And here’s why the y-axis is of paramount importance. A join at the level of y = 2 represents a very similar data point joining. A join at the level of y = 40 represents a totally different cluster joining just because everything else has been merged already.

How to find the right cut point:

Identify the longest possible vertical line such that you do not have to cross any horizontal line. The vertical gap is where there is maximum increase in the value of merge distance. Now if you draw a horizontal line cutting the middle of that vertical gap, the number of vertical lines it cuts will give you the number of clusters to consider.

For instance: Let us suppose that in your dendrogram the longest continuous vertical line exists between height 8 and 21. You need to draw a horizontal line at height 14. If this line cuts three vertical lines, you are left with three clusters.

Strangely enough, this “Longest Branch Rule” never finds mention in any textbook.

Linkage Methods — Which One Should You Actually Use?

Choice of linkage is the most influential parameter for hierarchical clustering, and almost every paper spends just three sentences discussing this before moving on. This is not right. Different linkages may generate completely different clusters for the same dataset.

Here is how each one works – and when to use it.

Single Linkage (Minimum)

In single linkage, the measure of distance between two clusters is determined by the distance between the two closest points from these two clusters. Therefore, for example, if cluster A has London and cluster B has Paris, and the distance between London and Paris is smaller than any other distance, it will be the measure of distance between these two clusters.

This issue comes down to what is referred to as the chaining effect: a few points can be used as intermediaries between these two clusters, which have to remain separate but get merged because of these points.

In reality, single linkage works best when dealing with data sets with irregular/non-globular distributions such as river networks or chain-like genetic sequences. However, for most tabular data, it is the worst option.

Complete Linkage (Maximum)

Complete linkage computes the maximum distance between any two points in the pair of clusters. This is a stricter criterion; the two clusters merge only if even their furthest apart points are sufficiently close. In effect, the clusters formed will be compact, more like spheres.

However, complete linkage is very sensitive to outliers. A single outlying data point will make the distance between the two clusters increase beyond acceptable limits.

Average Linkage

Average linkage determines the average distance among all point pairs within two clusters. The technical name for average linkage clustering is UPGMA (Unweighted Pair Group Method with Arithmetic Mean).

Average linkage falls somewhere in between single and complete linkage in performance: it is more resistant to chaining than single linkage and less vulnerable to outlier effects than complete linkage. Average linkage is an acceptable option when it is unclear how the data is structured.

Ward’s Method — The Practitioner Default

There is no measurement of distance between points involved in Ward’s algorithm at all. What the algorithm actually does is calculating how much the sum of variance inside the resulting clusters will increase after combining those particular two clusters. And those are combined that produce the smallest increase in this value.

This method was proposed by Joe H. Ward Jr. in 1963 in the Journal of the American Statistical Association. And the idea is brilliant — instead of trying to measure distance between points, one should optimize the compactness of clusters.

My honest advice is to use Ward’s method by default. This way you get the best-balanced and spherical clusters from any real-world dataset, it is the least sensitive to noise, and it is even used by default in scipy and sklearn.

One thing to keep in mind about Ward’s algorithm is that it cannot be used with any other than squared Euclidean distance measure.

Centroid Linkage

Centroid linkage calculates the distance between the centroids of two clusters. This is not popular due to the problem of inversion, which is when a merger occurs at a smaller value than an earlier merger. It should be avoided unless you have a compelling reason for its use.

Comparison Table — Linkage Method Decision Guide

Method	Cluster shape tendency	Outlier sensitivity	Use it when
Single	Elongated, irregular	Very high	Data has chain-like or non-globular structure
Complete	Compact, spherical	Moderate-high	You want tight, balanced clusters; no extreme outliers
Average	Moderate	Moderate	General use; data shape unknown
Ward’s	Compact, spherical	Low	Most real datasets — your default choice
Centroid	Moderate	Moderate	Rarely; risk of dendrogram inversions

Python Implementation — Full Working Code

Here we will be using the Iris data set — 150 instances, 4 variables (sepal length, sepal width, petal length, petal width), 3 classes. It is a very well-known data set, and hence, helps in checking whether the clustering is done right.

Step 1 — Install Libraries and Load Data

import numpy as np

import pandas as pd

from sklearn.datasets import load_iris

from sklearn.preprocessing import StandardScaler

from scipy.cluster.hierarchy import linkage, dendrogram, fcluster, cophenet

from scipy.spatial.distance import pdist

from sklearn.cluster import AgglomerativeClustering

from sklearn.metrics import silhouette_score

import matplotlib.pyplot as plt

Import the data and normalize it. This step is mandatory; otherwise, if you have features with varying magnitudes, such as age in years and salary in dollars, then the larger magnitude feature will dominate the distance calculations.

# Load data

iris = load_iris()

X = iris.data

# Standardize: mean=0, std=1 for each feature

scaler = StandardScaler()

X_scaled = scaler.fit_transform(X)

Step 2 — Build the Linkage Matrix

The linkage matrix is the central data structure. Each row contains information about a single merging process: which clusters were merged, the distance between them, and the number of points in the new cluster.

# Ward's method on Euclidean distances

Z = linkage(X_scaled, method='ward')

# Z has shape (n-1, 4): [cluster_1, cluster_2, distance, count]

print(Z.shape)  # (149, 4)

print(Z[:3])    # First three merges — should be at low distances

The first merge is done at smaller distance values since the points to be merged are the closest ones. Take a look at the distance values that appear in the distance column and note where there is a large gap.

Step 3 — Plot and Read the Dendrogram

plt.figure(figsize=(12, 6))

dendrogram(

    Z,

    truncate_mode='lastp',  # show only the last p merged clusters

    p=20,                   # show last 20 merges

    leaf_rotation=45,

    leaf_font_size=10,

    show_contracted=True

)

plt.title('Iris Dataset — Hierarchical Clustering Dendrogram')

plt.xlabel('Cluster (point count in parentheses)')

plt.ylabel('Distance')

plt.axhline(y=6, color='red', linestyle='--', label='Cut line — 3 clusters')

plt.legend()

plt.tight_layout()

plt.show()

The red horizontal line drawn on the graph at y = 6 intersects three vertical lines, thereby establishing three clusters. This is consistent with the three types of flowers contained in the Iris data set. When analyzing your own data, you would have to slide the line up and down.

Step 4 — Extract Cluster Labels

Two ways to cut the dendrogram:

# Method 1: Cut at a specific distance threshold

labels_by_distance = fcluster(Z, t=6.0, criterion='distance')

# Method 2: Specify exactly how many clusters you want

labels_by_count = fcluster(Z, t=3, criterion='maxclust')

print(np.unique(labels_by_count, return_counts=True))

# (array([1, 2, 3]), array([50, 49, 51]))  — roughly balanced

criterion=’distance’ gives you whatever number of clusters exists at that cut height. criterion=’maxclust’ forces exactly k clusters. For exploratory work, use distance; for when you’ve already decided on k, use maxclust.

Using sklearn’s AgglomerativeClustering

Sklearn’s version is less flexible for dendrogram plotting but integrates cleanly into sklearn pipelines:

# sklearn version — no dendrogram, but pipeline-compatible

agg = AgglomerativeClustering(n_clusters=3, linkage='ward')

sklearn_labels = agg.fit_predict(X_scaled)

When to use which: Use SciPy when you need the dendrogram to make the cluster decision. Use sklearn when you’ve already decided on k and need the clustering step inside a larger pipeline (preprocessing → clustering → evaluation → model).

How to Choose the Right Number of Clusters

This is the question that DataCamp half-answers and IBM barely mentions. There are three methods worth knowing, and you should probably run at least two of them before committing.

Method 1 — The Longest Vertical Line Rule (Dendrogram)

As stated above. Locate the largest number of continuous vertical spaces in your dendrogram. Draw a horizontal line across them. The number of vertical lines crossed by your horizontal line is your k.

This approach is both quick and visual. However, it is subjective, meaning that two different people looking at the same dendrogram can identify different gaps between clusters. This is where verification comes in.

Method 2 — Silhouette Score

The silhouette value measures how well each point belongs to its own cluster as opposed to neighboring clusters. Its range is from -1 to 1, where the higher the value, the better. A value greater than 0.5 usually implies that the clusters make sense, whereas anything less than 0.2 indicates overfitting.

# Test k=2 through k=6, find the best

silhouette_scores = {}

for k in range(2, 7):

    labels = fcluster(Z, t=k, criterion='maxclust')

    score = silhouette_score(X_scaled, labels)

    silhouette_scores[k] = round(score, 3)

    print(f"k={k}: silhouette score = {score:.3f}")

best_k = max(silhouette_scores, key=silhouette_scores.get)

print(f"\nBest k: {best_k} (score: {silhouette_scores[best_k]})")

For the Iris dataset, k=3 typically scores highest — confirming what the dendrogram showed us.

Method 3 — Cophenetic Correlation Coefficient

This one is never discussed anywhere. The cophenetic correlation coefficient determines the accuracy of your dendrogram in capturing the pairwise distance of your data points. Any number above 0.75 indicates that your dendrogram accurately represents your data, while any number below 0.75 indicates that you should consider other clustering algorithms.

from scipy.cluster.hierarchy import cophenet

from scipy.spatial.distance import pdist

c, coph_dists = cophenet(Z, pdist(X_scaled))

print(f"Cophenetic correlation: {c:.3f}")

# Typical result for Iris with Ward's: ~0.88 — excellent

If your cophenetic correlation is low, switching the linkage method often helps more than anything else. Try average linkage and recheck.

The Honest Performance Reality — Complexity and Scale

Yes. This isn’t discussed elsewhere. Let’s get down to brass tacks.

Hierarchical clustering uses O(n²) memory, and O(n² log n) to O(n³) time complexity depending on the choice of linkage. Which means:

• 1,000 observations: 1 million distances to calculate and store. Pretty quick.
• 10,000 observations: 100 million distances. Too slow for most laptops.
• 100,000 observations: 10 billion distances. Not possible unless you find ways around it.

The Ward algorithm itself is O(n²); the optimal single linkage is O(n² log n). But it’s the memory usage – storing that complete distance matrix – that is the problem no matter what linkage you use.

When Hierarchical Clustering Breaks Down

The practical threshold, from working with real datasets, is roughly this:

• Under 5,000 rows: No issues. Run it freely.
• 5,000 to 15,000 rows: Manageable but slow. Expect 10–60 seconds depending on your machine.
• Above 15,000 rows: You’ll start hitting memory limits or multi-minute runtimes. Consider workarounds.
• Above 50,000 rows: Hierarchical clustering is generally the wrong tool. Use BIRCH or k-means instead.

Workarounds for Large Datasets

Three options that actually work:

1. Sub-sampling. Apply hierarchical clustering to a randomly sampled representative 5,000-row subset. Obtain your k value from the dendrogram. Finally apply k-means or nearest-centroid assignment to each remaining point based on its closest cluster centroid. The best of both worlds without the scale issues.

2. BIRCH algorithm. BIRCH (Balanced Iterative Reducing and Clustering using Hierarchies) is a clustering algorithm designed for very large datasets. It first constructs a highly compact CF-tree, thus drastically cutting down memory usage, and then performs hierarchical clustering of this compact tree structure, not the data itself. Implemented in sklearn via sklearn.cluster.Birch.

3. fastcluster library. fastcluster is a Python package implementing various distance-based linkage clustering algorithms (single, complete, average, Ward) in an efficient way (both speed-wise and memory-wise).

# BIRCH for large datasets

from sklearn.cluster import Birch

birch = Birch(n_clusters=3)

birch_labels = birch.fit_predict(X_scaled)

Handling Outliers Before You Cluster

Of all the clustering algorithms that you will apply, hierarchical clustering is certainly the one that is most susceptible to outliers, but unfortunately not enough people pay heed to this fact. Just one data point at an extreme can be seen as an independent little cluster with a huge distance from merging.

Here are three techniques you can follow:

1. Z-score filtering. Remove points where any feature is more than 3 standard deviations from the mean. Quick and simple.

from scipy import stats

# Z-score filter: remove rows where any feature exceeds 3 std devs

z_scores = np.abs(stats.zscore(X_scaled))

mask = (z_scores < 3).all(axis=1)

X_clean = X_scaled[mask]

print(f"Removed {X_scaled.shape[0] - X_clean.shape[0]} outliers")

2. IQR method. More conservative than z-scores for skewed distributions. For each feature, compute Q1 and Q3, set bounds at Q1 – 1.5×IQR and Q3 + 1.5×IQR, and remove anything outside.

3. LOF (Local Outlier Factor). Run LOF first to score each data point by how isolated it is relative to its neighbors. Remove high-LOF points before hierarchical clustering.

from sklearn.neighbors import LocalOutlierFactor

lof = LocalOutlierFactor(n_neighbors=20, contamination=0.05)

outlier_labels = lof.fit_predict(X_scaled)  # -1 = outlier, 1 = normal

X_clean = X_scaled[outlier_labels == 1]

print(f"LOF removed {(outlier_labels == -1).sum()} outliers")

The choice between these depends on your data. Z-scores work well for roughly normal distributions. IQR is better for skewed data. LOF is the most powerful but slowest, and it requires choosing the contamination parameter — an estimate of what fraction of your data you expect to be outliers.

DataCamp has a comparison table. I’ll give you the actual decision logic instead.

Hierarchical Clustering vs. K-Means — The Real Decision Framework

Use hierarchical clustering when:

• You don’t know k in advance and want the data to suggest it
• Your dataset has under ~10,000 rows
• You need nested or hierarchical cluster structure (think: taxonomies, organizational charts, gene families)
• You need to show stakeholders a visual that explains the clustering — a dendrogram is much more intuitive than k-means cluster plots for non-technical audiences

Use k-means when:

• Your dataset is large (10,000+ rows)
• Speed matters — k-means scales far better
• Cluster shapes are roughly spherical and similar in size
• You already have a good estimate of k from domain knowledge

Use DBSCAN when:

• Data has irregular shapes that k-means and hierarchical clustering both handle poorly
• Noise and outlier identification are part of the goal, not just a preprocessing problem
• You genuinely don’t know the number of clusters, and the data may not have well-separated groups at all

The common framing — “hierarchical is better for small data, k-means for big” — is mostly right but misses the real distinction. The deeper question is whether you need to explore your data’s structure (hierarchical) or whether you already know enough to partition it (k-means).

Real-World Use Cases

Customer Segmentation (The Non-Toy Version)

The e-commerce firm has 4,000 customers. The three RFM attributes of each customer are Recency (days since last purchase), Frequency (orders placed in 12 months) and Monetary value (total spend in USD).

Standardizing these attributes, we find that applying Ward’s linkage on Euclidean distances generates a dendrogram where the largest gap occurs between 3 and 4 clusters. Silhouette score proves k=4. Clusters are formed as follows:

• Cluster 1: High frequency, high spending, recently – loyal premium customers
• Cluster 2: Medium frequency, medium spending, recently – engaged customers
• Cluster 3: Low frequency, low spending, long back – customers who need to be won back with campaigns
• Cluster 4: Only one or two orders made, recently – new customers who haven’t decided yet.

Now, here is the difference: If the firm had applied k-means for cluster analysis, they would have got almost the same clusters using k=4. But here the team wouldn’t have known what k to use in advance, which can be very expensive for a business of this size.

Gene Expression Analysis

Hierarchical clustering is the typical approach used by bioinformatics experts in studies involving gene expression. The bioinformaticians obtain information about the expression levels of several thousands of genes in several dozen different conditions or patients and group together those genes that exhibit similar behavior. A heatmap along with the dendrogram (typical of R’s pheatmap package) will help them find which genes co-express, which patient groups have common gene expressions, and which conditions yield similar results. In bioinformatics, the typical approach involves Ward+Euclidean, but cosine is sometimes used.

Text and Document Clustering

However, when doing text document clustering, there is a documented reason why Euclidean distance does not work in such cases: the high dimensional vector that has been produced using TF-IDF will have what is known as the curse of dimensionality, and this means that all documents will be about equally distant from each other.

from sklearn.feature_extraction.text import TfidfVectorizer

from sklearn.metrics.pairwise import cosine_distances

from scipy.cluster.hierarchy import linkage, dendrogram

docs = [

    "machine learning clustering algorithms",

    "neural networks deep learning",

    "hierarchical clustering dendrogram",

    "k-means unsupervised learning",

    "transformer attention mechanism"

]

vectorizer = TfidfVectorizer()

tfidf_matrix = vectorizer.fit_transform(docs).toarray()

# Use cosine distance — NOT Euclidean — for text

dist_matrix = cosine_distances(tfidf_matrix)

Z_text = linkage(dist_matrix, method='average')  # average linkage works well with cosine

Note the linkage method: average instead of Ward’s. Ward’s requires squared Euclidean distance — it doesn’t work correctly with cosine. For text, average linkage is the better fit.

Anomaly Detection Pipeline

This particular use case is not addressed anywhere else. Hierarchical clustering can serve as an initial anomaly detection method: apply the algorithm and then examine the dendrogram for the data points that come together at an extraordinarily large distance from each other. The data point that only merges with the majority group towards the end of the process and stands by itself at the extreme right side of the dendrogram is probably an anomaly.

# Find points that merge last (at highest distance)

last_merges = Z[-10:, :]  # Last 10 merge operations

print("Highest merge distances:", last_merges[:, 2])

# High merge distance = those clusters were far apart = potential anomalies

This isn’t a replacement for dedicated anomaly detection (LOF, Isolation Forest), but it’s a useful secondary signal when you’re already running hierarchical clustering for segmentation.

Advantages and Limitations

What hierarchical clustering genuinely does well:

• No need for a value of k to be specified initially; the algorithm uncovers the structure without forcing it.
• Dendrogram serves as the most explainable output for those who do not specialize in the technical field.
• Compatible with all metrics of distance in contrast to Ward’s only approaches.
• Unfolds the nested structure that cannot be detected by other clustering approaches.
• Deterministic approach; applying the algorithm twice for the identical input will yield the same output (in contrast to the k-means algorithm).

The real limitations — stated plainly:

• O(n²) memory. The pairwise distance matrix doesn’t shrink. This is a hard constraint, not a performance issue you can optimize your way out of.
• Merges are permanent. One bad merge early in the process propagates through everything that follows. The algorithm can’t correct itself.
• Outlier sensitivity. One extreme data point can distort the entire dendrogram. Always filter outliers first.
• Dendrograms become unreadable above ~300–500 data points without truncation. The visual clarity that makes hierarchical clustering compelling disappears at scale.
• Linkage choice heavily affects results. The same data can produce very different clusters depending on which linkage method you pick. This is a feature (flexibility) and a bug (sensitivity to choices you might make arbitrarily).

Frequently Asked Questions

CONCLUSION

Hierarchical clustering is the only algorithm that fits a particular purpose: analysis of unknown data. In cases when you don’t know k, when the structure is as important as cluster labeling, when you have to visually demonstrate how your data is structured — use hierarchical clustering.

However, it isn’t the right algorithm for all occasions, and re-reading the limitations part of the guide may prove useful. O(n²) memory requirement is oblivious to the quality of your hardware. Greedy clustering cannot be reversed. Dendrograms are hard to read when dealing with large amounts of data.

It’s meant to help you gain understanding of your data. And then apply the right algorithm to the right data. The dendrogram you’ve been studying? Cut it where there is the biggest gap. Most likely you got it right from the start.