Confusion Matrix Explained: Read, Build & Interpret It Right

Your model had an accuracy of 99%. You deployed it. After three weeks, your fraud department let you know that it had identified precisely zero frauds.

The figure 99%, however, was not incorrect. In fact, it was perfectly accurate but totally irrelevant since you had 9,900 genuine transactions but 100 frauds in your dataset. All your model learned to do was to label every single transaction as ‘no fraud’. That way, you were right 9,900 times but failed to catch 100 frauds – which meant that your whole anti-fraud budget was wasted.

A confusion matrix would have caught this in about ten seconds.

A confusion matrix is a table that splits your model’s classifications into 4 different types – true positives, true negatives, false positives, and false negatives – to give you an exact picture of where your model fails, rather than just how much.

In this article, I’ll be covering all the metrics you can extract from the confusion matrix, when you should pay attention to each of them, and how you can construct and interpret them in Python.

What Is a Confusion Matrix? (The One-Sentence Version)

The confusion matrix is essentially a table of performance that pits your classifier’s prediction against the actual value that has occurred for each possible case of results.

This is all very technical; let me tell you what it means when the risks are high.

Consider a model developed for detecting potential cases of early-stage lung cancer using chest x-rays. There can be four possible outcomes for each patient’s case: 

• True Positive (TP): Cancer is present. The model correctly flags it.
• True Negative (TN): No cancer. The model correctly clears the patient.
• False Positive (FP): No cancer. The model flags it anyway — the patient goes through unnecessary follow-up testing.
• False Negative (FN): Cancer is present. Model misses it entirely — patient walks out undiagnosed.

This final one is not just a number; it represents a real individual who required treatment but did not receive it.

It is the confusion matrix that allows us to place all four of these outcomes next to each other in tabular format. From there, accuracy ceases to be our desired metric.

The Four Cells of a Confusion Matrix

Every confusion matrix — binary or multiclass — is built from the same four building blocks.

True Positives (TP) mean win. Your algorithm predicted a positive label, and it was correct. You have a lot of TP – your model is detecting important cases.

True Negatives (TN) mean win as well. Your algorithm predicted a negative label, and reality confirmed that it was right. These are cases you have successfully ignored.

False Positives (FP) mean Type I error. Your algorithm warned about some problem that does not exist. If you are working on medical imaging, it leads a healthy person to an intimidating next step of a diagnostic process. If you develop a spam filter, you end up with a regular email in your junk folder. The price varies drastically depending on the field – that’s why we need a framework for this, not just a formula.

False Negatives (FN) mean Type II error. Your algorithm decided there was nothing interesting there, but actually, there was. It is fatal in cancer detection. It is annoying in spam filters. And it is costly, but not lethal in credit card fraud..

The difference between FP and FN is the most important distinction in applied ML, and it is the distinction that most beginner articles treat as an afterthought.

Right.

The matrix in Python, for 1,000 chest scans (100 actual cancer cases, 900 healthy patients), might look like this:

	Predicted: Cancer	Predicted: Healthy
Actual: Cancer	82 (TP)	18 (FN)
Actual: Healthy	31 (FP)	869 (TN)

We’ll use these numbers throughout the rest of this article.

The 5 Metrics You Can Calculate from a Confusion Matrix

Once you’ve got those four numbers, the rest is arithmetic.

Why Accuracy Is the Metric That Lies

Accuracy = (TP + TN) / (TP + TN + FP + FN)

For our radiology example: (82 + 869) / 1,000 = 95.1%

Now that looks like a great number, doesn’t it? Well, not really. We misclassified 18 cases of cancer out of 100. A 18% error rate for a cancer screening algorithm is not something to be proud about.

Let’s consider an even more obvious example: fraud. You have 10,000 transactions. Only 100 of them are actually fraudulent (1%). Your algorithm classifies all the transactions as “not fraudulent”.

Accuracy: 9,900 / 10,000 = 99%.

You made the most expensive coin toss in the history of your company and put a percentage after it just for fun.

It is known as the accuracy paradox, and it is exactly what happens whenever you start talking about accuracy in case of class imbalance.

Precision — The Cost of Crying Wolf

Precision = TP / (TP + FP)

In our radiology example: 82 / (82 + 31) = 72.6%

Precision is: “Of all those who tested positive, how many really were?” 

High precision means whenever you sound the alert, you are most likely correct. With low precision, you will be creating undue anxiety in healthy individuals through further scanning. Precision is low in a spam filter when your users are losing valuable emails.

Recall — The Cost of Missing It

Recall = TP / (TP + FN)

In our radiology example: 82 / (82 + 18) = 82.0%

The question that recall answers is “How many of the positive examples in the dataset did I find?”

Recall is high when you’re not missing any instances. The cost here is that while chasing high recall, your precision ends up being low because you end up capturing more false alarms along with true positives.

This is what we call the precision-recall tradeoff. This is not an issue with your algorithm – it is an inherent characteristic of classification problems with a threshold, and understanding this helps distinguish between people who know what they’re doing and those who just use accuracy_score().

F1-Score — The Balancer

F1 = 2 × (Precision × Recall) / (Precision + Recall)

For our example: 2 × (0.726 × 0.82) / (0.726 + 0.82) = 0.770

The F1 score is the harmonic mean of precision and recall. The F1 score provides a single value, which is affected by the imbalance between the two measures. A classification algorithm with 100% precision but 0% recall would get a perfect F1 score of zero — indicating that it’s completely useless.

F1 scores should be used when both types of errors incur some cost and you need to weigh them. F1 scores shouldn’t be used if the cost of one error is much higher than the cost of the other one. We’ll talk about this case later.

Specificity — The True Negative Rate

Specificity = TN / (TN + FP)

For our example: 869 / (869 + 31) = 86.9%

Specificity indicates how accurately your model detects the negative class; that is, how proficient your model is at filtering non-cancer patients and fraudulent transactions. Although specificity is paid less attention to than precision and recall, it plays an extremely important role in medical diagnosis settings where it is critical to maintain a high level of specificity.

Which Metric Should You Use? (Decision Framework)

This is the truth — it is the question that none of the three articles that come up when searching “confusion matrix” actually answer. Precision and recall are defined. The explanation for F1 is given. But then you are left to determine what to optimize for your specific situation.

Here is the straightforward answer.

Use Case	Worse Error	Optimize For
Cancer / disease screening	False Negative (missing the disease)	Recall — catch everything, accept false alarms
Spam filter	False Positive (blocking real email)	Precision — only flag when confident
Credit card fraud	Both costly, classes wildly imbalanced	F1 or MCC — accuracy will completely mislead you
Content moderation at scale	Depends on policy (err toward removal vs err toward free speech)	Set explicitly by stakeholder, not ML team
Predictive maintenance (factory equipment)	False Negative (missing a failing machine)	Recall — unplanned downtime costs more than extra checks
Loan default prediction	False Positive (wrongly denying credit)	Precision — regulatory and fairness risk

This common approach to the problem is wrong. Almost all tutorials start with introducing metrics, while leaving “how to choose” for the end as an afterthought. In fact, you need to consider which kind of error costs more before you choose your metric – not vice versa. The metric comes as the result of such thinking, not its premise.

I believe that the section on limitations is actually more useful than the one about advantages in almost any machine learning evaluation guidelines.

Beyond F1 — The Metrics Your Articles Never Mention

Actually, this part could be the most valuable thing in this topic.

F1 score receives the most attention, however, it is very biased in terms of ignoring true negatives totally. True Negatives (TN) are not mentioned in the equation at all. In a cancer detection problem with 900 healthy patients and 100 unhealthy people, F1 score doesn’t give any information on 900 healthy patients.

This problem is solved by the Matthews Correlation Coefficient (MCC). Named after biologist B.W. Matthews, who introduced it back in 1975 when studying protein secondary structure prediction, the MCC uses all four elements of the confusion matrix:

In a research paper published in BMC Genomics, 2020, titled “An improved Matthews correlation coefficient formula with false discoveries penalization,” Chicco & Jurman evaluated F1 and MCC on 1,000 binary classification datasets and concluded that MCC yielded a more informative output in most scenarios, especially for imbalanced class distributions. This score ranges from -1 (totally wrong) to +1 (totally correct), while 0 implies the result is totally random.

For our radiology example: MCC = (82×869 − 31×18) / √[(82+31)(82+18)(869+31)(869+18)] ≈ 0.67

It is an informative metric that covers all four cases.

The Cohen’s Kappa metric is also available, usually used in medicine and natural language processing. This metric takes into account chance agreement. It is more common for scientists with a statistical background than a computer science one, but no matter what, this metric is almost never covered in ML blogs.

If you have an imbalanced dataset and you need one reliable score, choose MCC. If the negatives don’t really matter for your task, then use F1.

How to Read a Confusion Matrix — The Sklearn Axis Problem

There is one important thing you need to know before writing even a line of code in Python, since this is what confuses everyone at first.

For instance, in most textbooks and guides, the confusion matrix is represented with the actual values in the columns and predicted values in the rows. scikit-learn represents them the other way round – actual values in the rows and predicted values in the columns.

Both approaches yield the same four numbers, but if you interpret the heatmap without paying attention to the axis labels, you get the wrong precision and recall.

sklearn.metrics.confusion_matrix() returns a matrix where cm[i][j] means “the model predicted class j for actual class i.” Actual is always the row. Predicted is always the column.

The newer ConfusionMatrixDisplay class (added in sklearn 0.22) makes this less confusing by labeling the axes directly. Use it.

python
from sklearn.metrics import ConfusionMatrixDisplay
ConfusionMatrixDisplay.from_predictions(y_test, y_pred, display_labels=["No Cancer", "Cancer"])

But in reality, this axis reversal has been responsible for real-world bugs in production where developers misinterpret the values of recall, use inappropriate thresholds, and deploy a system that acts totally differently from how it was evaluated. Check your axis labels always. Always.

Build a Confusion Matrix in Python (Step-by-Step)

Our dataset will be the Wisconsin Breast Cancer Dataset that is bundled along with scikit-learn. This is a true medical dataset (569 instances, 212 are malignant and 357 are benign) with actual class imbalance.

Step 1: Load the data and split it

python
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split

data = load_breast_cancer()
X, y = data.data, data.target  # 0 = malignant, 1 = benign

# 70% train, 30% test -- stratified to preserve class ratio
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.3, random_state=42, stratify=y
)

stratify=y keeps the malignant/benign split proportional in both sets. Skip this and your test set might have a different class ratio than your training set, making evaluation unreliable.

Step 2: Train a classifier

python
from sklearn.ensemble import RandomForestClassifier

model = RandomForestClassifier(n_estimators=100, random_state=42)
model.fit(X_train, y_train)
y_pred = model.predict(X_test)

Step 3: Generate and plot the confusion matrix

Python
import matplotlib.pyplot as plt
from sklearn.metrics import ConfusionMatrixDisplay

# Modern approach -- auto-labels axes correctly
disp = ConfusionMatrixDisplay.from_predictions(
    y_test, y_pred,
    display_labels=data.target_names,  # ["malignant", "benign"]
    cmap="Blues"
)
plt.title("Confusion Matrix -- Breast Cancer Classifier")
plt.tight_layout()
plt.show()

Step 4: Get the full metric report

python
from sklearn.metrics import classification_report

print(classification_report(y_test, y_pred, target_names=data.target_names))

It calculates the accuracy, recall, f-measure, and support values for all classes, along with the macro average and weighted average of these metrics. The “support” value indicates the number of data points for each class in the test dataset.

Step 5: Calculate MCC and check for imbalance

python
from sklearn.metrics import matthews_corrcoef

mcc = matthews_corrcoef(y_test, y_pred)
print(f"Matthews Correlation Coefficient: {mcc:.3f}")

A result above 0.7 generally indicates a strong classifier on this type of dataset.

Multiclass Confusion Matrix — When You Have More Than Two Classes

Binary classification is simple. Life isn’t so simple.

Consider a classifier trained to classify birds into Robin, Sparrow, or Crow. You have a test set that contains 90 images. The confusion matrix now looks like this:

	Predicted: Robin	Predicted: Sparrow	Predicted: Crow
Actual: Robin	28	2	0
Actual: Sparrow	3	25	2
Actual: Crow	0	1	29

In the case of (28, 25, 29), there are true predictions along the diagonal. All other elements are errors. Sparrows are the most difficult to classify because they confuse not only robins but also crows, whereas crows never confuse robins.

For multiclass problems, precision and recall values are determined individually for per class as in a one-vs-rest manner and then averaged over all classes according to one of three methods:

• Macro average: Plain average across all classes, treating each equally regardless of size. Use this when all classes matter equally to you.
• Weighted average: Average weighted by how many samples belong to each class. Better when class sizes differ significantly.
• Micro average: Aggregates TP, FP, FN globally before computing. In multiclass problems where no single class dominates, micro average equals accuracy.

python
# All three averaging strategies in one call
from sklearn.metrics import classification_report
print(classification_report(y_test, y_pred, average=None))  # Per-class

The Normalized Confusion Matrix — When Raw Counts Mislead

The number of elements in the confusion matrix could deceive you since the categories would be of different sizes. For instance, you have a testing dataset that contains 900 benign cases and 100 malignant cases. Having “87 correct malignant cases” sounds worse than having “870 correct benign cases.” 

Normalizing solves this problem. scikit-learn makes it easy for you to do that:

python
from sklearn.metrics import confusion_matrix
import seaborn as sns

# normalize='true' divides each row by the row sum
# So each cell shows the fraction of that actual class
cm = confusion_matrix(y_test, y_pred, normalize='true')

sns.heatmap(cm, annot=True, fmt='.2f',
            xticklabels=data.target_names,
            yticklabels=data.target_names,
            cmap='Blues')
plt.ylabel('Actual')
plt.xlabel('Predicted')
plt.title('Normalized Confusion Matrix')
plt.show()

Three normalization options exist: normalize=’true’ (divide by row totals — shows recall per class), normalize=’pred’ (divide by column totals — shows precision per class), normalize=’all’ (divide by total samples — shows proportion of all predictions). In most cases, normalize=’true’ is what you want; it makes the diagonal directly readable as per-class recall.

Weirdly enough, this is one of those features that’s been in sklearn for years but barely shows up in tutorials. For any imbalanced dataset, a normalized matrix communicates more than raw counts do.

Confusion Matrix vs ROC Curve — What Each Tells You

The confusion matrix and the ROC curve are related concepts but not interchangeable measures.

A confusion matrix is a snapshot at one threshold. Typically, by default, classifiers use a threshold of 0.5, above which the prediction is considered positive. The confusion matrix you have provided gives an overview of how your classifier performs when using a 0.5 threshold.

ROC Curve is a full sweep– a range-of-threshold measurement where true positive rate (recall) is plotted on the Y-axis and false positive rate is plotted on the X-axis for each possible threshold value between 0 and 1. Thus, each point on the ROC curve is basically a confusion matrix.

Area under this curve – AUC – reflects classifier performance at all possible thresholds in one numerical measure. An AUC score of 1.0 indicates perfect discrimination; 0.5 is no better than random guessing.

In case of an imbalanced dataset, the Precision-Recall Curve might be much more insightful than the ROC Curve since AUC may appear to be great while precision is terrible.

Think of the confusion matrix as where you land at the threshold you ship; think of the ROC curve as the map of everywhere you could land. You need both.

Common Mistakes When Interpreting a Confusion Matrix

Mistake 1: Leading with accuracy on imbalanced data. Discussed earlier, but deserves to be stated again: if your positive class makes up less than 10% of your sample space, accuracy isn’t your metric. Use F1, MCC, or Precision-Recall curve.

Mistake 2: Comparing raw confusion matrix counts across datasets of different sizes. A model that correctly identified 50 instances in a test dataset consisting of 500 items isn’t giving you the same performance level as a model that correctly identified 50 instances in a test dataset of 5,000 items.

Mistake 3: Ignoring the cost asymmetry of FP vs FN. Your confusion matrix will show you the number of occurrences. What it won’t do is tell you which type of error is more expensive to make in your case. You’ll have to provide that insight yourself.

Mistake 4: Misreading sklearn’s axis convention. Described above – actuals in rows, predictions in columns. Do it every time. Not just once when setting up the matrix. Every single time you come across a new matrix.

Frequently Asked Questions About Confusion Matrices

A Final Word on Getting This Right

All claims made in this article have one message underneath them: accuracy ≠ evaluation. The four boxes of the confusion matrix are. And once you get into the habit of considering all four of them and not only the headline figure, you will start to recognize the cases when the failure of the model becomes obvious only after it happens.

To know whether you really understand the confusion matrix or not doesn’t depend on your ability to remember all those formulas. It depends on your ability to look at the matrix and to ask yourself: “Which kind of error is more important here – and does this model take into consideration our priorities?”

If you can do that – congratulations, you are an expert now.