Market Basket Analysis

Osco Drug stores in 1992, when NCR consultants were examining Osco Drug’s data, discovered something quite illogical from common sense: people kept buying both beer and baby diapers on Fridays during evening hours from 25 shops. In most cases, this phenomenon has been attributed to Walmart; but in actuality, this was done by Thomas Blischok, who analyzed 1.2 million market baskets for American Stores’ Osco Drug chain. 

Nobody actually hypothesized or tested anything about it.

The insights Osco gained did not help them to change product positioning, but rather served the purpose of taking around 5,000 SKUs off the shelves, making the store more selected. This one insight generated through the use of the technique that was later termed ‘market basket analysis’ shifted the way retailers treated transactional data.

Market basket analysis is a data mining process that uncovers the products purchased in tandem and quantifies the co-purchase strength. Market basket analysis works with transaction data and can be applied to the data from retail stores, pharmacies, medical insurance companies, e-commerce platforms, and even security systems. Market basket analysis provides output in the form of rules that say IF {A product} THEN {another product} along with quantifiable probability.

The current guide explains how MBA works, which algorithm to use, where to put thresholds that will generate useful, rather than thousands of evident rules and, above all, how to select the useful rules to follow. The selection of rules for implementation is never addressed in any competitor’s guide, although this is the core element of the process.

What Is Market Basket Analysis?

Market Basket Analysis (MBA) is an example of association rule mining, a type of unsupervised learning where the objective is to find out associations between different items in a dataset.

The term “basket” here refers to the physical shopping basket. Each time you shop at a store, it is considered a shopping basket; MBA analyzes thousands or even millions of these shopping baskets and wonders what sets of items show up together more than you might expect through chance alone.

However, it is important to realize that this technique is not limited only to grocery shops. The same method of finding correlations that applies to the case of beer and diapers can apply to prescriptions of antidepressants and sleeping pills, correlations among sequences of certain pages’ visitations and cyber attacks on the network, and even correlations between “added to wish list” event and final purchase in an online store. The concept of the basket is simply a metaphor here.

The rigorous mathematical foundations of MBA were laid by two researchers from IBM Research Almaden, Rakesh Agrawal and Ramakrishnan Srikant, in their paper titled “Fast Algorithms for Mining Association Rules in Large Databases” presented at VLDB ’94

How Market Basket Analysis Works

The Association Rule Explained

Each MBA rule consists of two parts.

Antecedent (also known as IF) refers to the set of item(s) being observed in transactions.

The consequent (THEN) is what data suggests will happen next. An example of an MBA rule is:

IF {laptop} THEN {laptop bag, extended warranty}.

This means that when a customer purchases a laptop, he or she is likely to purchase a bag and warranty. The reason behind such behavior is unknown, because it does not need to be. The MBA rule shows what happens and not why.

What distinguishes MBA rules from ordinary frequencies is that they assess if two or more items co-occur statistically more often than they could occur if customers had no connection to each other. This is where the three basic measures come in.

The Three Core Metrics

=Choose an example that will remain consistent throughout the discussion: 1,000 transactions overall. Laptops occur in 100 transactions. Laptop bags occur in 80 transactions. Laptops and laptop bags occur in 60 transactions simultaneously.

Support measures the frequency with which an itemset occurs in the entire database.

Support (Laptop → Bag) = Transactions including both / Total number of transactions = 60 / 1,000 = 0.06 (6%)

This support value shows that the occurrence is once every 17 transactions. In other words, low support does not necessarily indicate weak rules; they can be quite valuable but rare. High support indicates high frequency.

Confidence calculates conditional probability: what is the chance that somebody who buys a laptop also buys a bag?

Confidence (Laptop → Bag) = Transactions with both ÷ Transactions with laptop = 60 ÷ 100 = 0.60 (60%)

Six out of ten laptop buyers also buy a bag. That’s a strong rule by most standards.

Lift is the metric that actually matters most. It tells you whether the association is stronger than pure chance.

Lift (Laptop → Bag) = Confidence ÷ Support of Bag alone = 0.60 ÷ (80 ÷ 1,000) = 0.60 ÷ 0.08 = 7.5

The lift value of 7.5 indicates that purchasers of laptops are 7.5 times more likely to purchase bags than the average customer entering the store. A lift value greater than 1.0 implies a positive relationship. A lift value of 1.0 is an indication of independent values, where there is no predictability of one over the other. However, a lift value less than 1.0 shows negative dependency.

For business rules to be actionable, the threshold for lift is set at between 1.5 and 2.0.

Beyond Lift: Conviction, Leverage, and Zhang’s Metric

To be fair, this is the point where all tutorials get this algorithmic analysis wrong – and make a grave error in their business decision-making.

The main issue with lift is symmetry – Lift(A→B) = Lift(B→A). A lift of 3.0 between diapers and beer will give the same result as a lift of 3.0 between beer and diapers. While the metric tells you the relationship’s strength, it gives no clue about its direction.

Conviction: The metric measures how far off a rule would have been if the events in question were truly unrelated. A value of conviction over 1.0 shows that the antecedent does predict the consequent, not vice versa.

Zhang’s measure, named after T. Zhang, who described the measure in 2000, varies from -1 to 1. Values below zero suggest an antagonism between the items while zero means independence. Positive values mean true association. As such, Zhang’s measure is more sensitive than lift at discovering weak associations in big and sparse data sets.

Leverage captures the gap between observed association and what would be expected if there was no relationship between them. It is presented in terms of actual transactions instead of a proportion to make it easier to understand.

DataCamp offers courses on market basket analysis that provide information on all six metrics. There are zero editorial competitors for this keyword mentioning conviction, leverage, or Zhang’s metric. In order to create a pipeline for production use, you will need all six.

The Three Types of Market Basket Analysis

Descriptive MBA

Descriptive MBA analyzes the past.

It uses historical transaction data to find the rules in the data. No labels. No training set. No forecasts. You apply the algorithm to past transactions and get the rules out.

This is the most popular type in reality. Descriptive MBA is used to optimize shelf space placement, planograms, and promotion analysis. In truth, this is what people usually refer to when they speak about “market basket analysis.” 

Here is one point that might be useful to remember. Descriptive MBA is vulnerable to seasonality. The rules found by MBA using December data do not coincide with the rules found by MBA using July data. Applying MBA to the whole year will result in some kind of average that does not correspond to either season.

Predictive MBA

Predictive MBA considers the concepts of sequence and timing.

This means that it does not ask what was purchased together but instead seeks to find out what is usually purchased next within distinct transactions occurring on different occasions. Supervised learning algorithms can be used for such an approach, whereby the purchase patterns are analyzed to create rules that cover multiple transactions.

An example of this from above is the rule for purchasing the laptop and the extended warranty. It is not true that all customers who purchase the computer also purchase the warranty within the same basket; sometimes, people purchase the laptop now and the warranty after three days.

Differential MBA

Differential MBA allows you to compare two databases together.

How do baskets from your site differ from those of your competitor? How do baskets of one customer group compare to others? How do rules for this period compare to those from the previous year? It’s all in the comparison.

That’s what to use when you need answers. Say you have a situation when people who purchase brand X shoes on your website also get insoles with a great lift value, but not when they buy those same shoes from your competitor – there’s a difference that can lead to real discoveries. A differential MBA will show you where those differences are without your having to guess how to start comparing things.

The Algorithms Behind Market Basket Analysis

Apriori: The Foundation

The key behind Apriori’s efficiency lies in just one brilliant assumption: If an itemset is frequent, then all its subsets are guaranteed to be frequent too.

This principle is referred to as the Apriori property. This implies that, if {laptop, bag, mouse} satisfies the threshold of minimum support, then all possible two-element and one-element subsets of it will do too. In Apriori, itemsets that have subsets that do not satisfy the condition are pruned right away, thus reducing their number.

Apriori works in two steps: it finds all the itemsets that satisfy the minimum support; it then constructs the association rules from those itemsets using the minimum confidence criterion. Neat and simple.

The problem is candidate generation. On very large datasets – think of supermarkets that sell more than 50,000 products per day – Apriori generates huge amounts of candidates and repeatedly scans the database.

FP-Growth: The Faster Alternative

The reason for developing FP-Growth was precisely to solve the problem with Apriori’s scalability.

Surprisingly, the trick was to avoid generating candidates at all costs. Rather than going over the entire database again and again, FP-Growth converts the entire transaction data set into a compact data structure called the FP-tree (Frequent Pattern Tree), after which the mining process takes place without candidate generation using the divide-and-conquer strategy.

Difference in practical application: While FP-Growth tends to perform much better in terms of processing time (as demonstrated by benchmarks from Han, Pei & Yin’s seminal SIGMOD’00 paper, albeit with the level of performance improvements depending on the dataset density), FP-Growth uses just two scans irrespective of the number of frequent items; however, the downside is increased use of memory space.

In Python, both algorithms are available in the mlxtend library via fpgrowth() and apriori() functions.

Apriori vs. FP-Growth vs. Eclat: Side-by-Side

Eclat (Equivalence Class Clustering and bottom-up Lattice Traversal) is a third algorithm that no current editorial competitor on this keyword mentions.

It uses a vertical data format — instead of listing which items appear in each transaction, it lists which transactions contain each item. This inverted structure makes intersection operations fast and memory-efficient for medium-density datasets.

Algorithm	Speed (Large Data)	Memory Use	DB Passes	Best For
Apriori	Slow	Low	Multiple	Small datasets; learning
FP-Growth	Fast	Medium–High	Two	Large datasets; production
Eclat	Fast	Medium	One	Medium datasets; R workflows

For production work in 2026: use FP-Growth in Python via mlxtend. Use Eclat in R via the arules package. Apriori is fine for learning and for datasets under roughly 50,000 transactions. Above that, the speed difference becomes impossible to ignore.

How to Set Support, Confidence, and Lift Thresholds

But in reality, the element that makes any first MBA assignment difficult is not something anybody ever tells you about: threshold selection.

If you select too low a threshold, your algorithm will spit out 8,000 patterns, and 7,900 of them will be irrelevant. Too high, and you won’t be able to identify the high-revenue-generating relationships.

The trick to setting thresholds is:

Minimum support: If your dataset contains more than 10,000 transactions, begin with 0.01, while if your dataset is less than 1,000 transactions, begin with 0.005. The aim here is to remove all items which occur infrequently enough so that statistical rules cannot be formulated from them.

Minimum confidence: 0.5 (50%) is a reasonable floor for most use cases. Below that, your rule is predicting the wrong outcome more often than not — marginally better than a coin flip at 40% confidence.

Minimum lift: Start from 1.0 to capture everything positive, then analyze the best 20 rules based on lift. A lift value of 1.2 might be statistically significant but practically irrelevant, while a value of 3.0+ is most likely relevant.

The proper process involves iteration. Start with running the algorithm using lax thresholds. Look at the first 20 to 30 rules. Determine whether they surprise you or bore you. Increase the thresholds until the remaining rules become interesting. Repeat.

The determination of thresholds in MBA relies partly on subjective judgment and partly on industry expertise. If any software claims to do the former for you, you can be sure that it is holding back something.

Real-World Applications of Market Basket Analysis

Retail and Grocery

Walmart’s association between beers and diapers took root due to its unpredictability, specificity, and measurability. This was evident once the two items were moved closer together in store layouts, leading to increased profits, measured not just in theory, but by sales performance.

This is the exact strategy Kroger uses through its decades-old collaboration with the data-analytics company Dunnhumby. This involves leveraging information on purchase history from their membership loyalty cards for targeted promotions, a process extensively covered in the retail industry literature, although the precise details regarding the customization of their flyers remains unknown. This methodology also applies in planogramming, or the systematic placement of items on shelves based on their affinity to one another.

E-Commerce and Recommendation Engines

Amazon’s “Customers who bought this also bought” section is the most visible MBA application in the world. But — and this is that kind of the whole point — modern recommendation engines go well beyond pure association rules. Amazon Personalize — Amazon’s cloud ML service for third-party developers — layers collaborative filtering and deep learning models; Amazon’s own internal recommendation engine is a proprietary system whose architecture is not publicly disclosed, though the general principle of combining rule-based interpretability with neural personalization is well-established in the field. 

MBA is still the foundation. It’s just no longer the complete structure.

Healthcare and Drug Co-prescription

Pharmacy chains and hospitals analyze MBA from prescription information to detect harmful drug combinations. If two drug types show up together often enough to demonstrate regular co-prescription, rather than by chance, the red flag will wave for clinical examination. The process is the same in retail, just the consequences are different.

Hospitals have also analyzed MBA of diagnostic test procedures to identify the most common combinations, thereby enabling better design of clinical care pathways and avoiding repetitive testing cycles.

Fraud Detection

Regarding insurance, MBA works by looking for oddball groupings of services that recur across fraudulent claim submissions. Certain groupings of particular services from different providers that always show up together in a particular window of the claims process are something that human reviewers just never see; MBA finds them instantly.

Similarly, credit card fraud detection also relies on MBA to flag transaction baskets that don’t match any of the known archetypes of normal spending behavior. Something like charging a luxury item and also a money order in the same transaction would set off automatic red flags.

Cybersecurity and Clickstream Analysis

The MBA in web server access logs finds sequences of requests that match the signatures of attacks. An “basket” of behaviors – probes to URLs after scanning ports – can recognize the signature of a recon mission before an actual intrusion takes place. The logic behind the association rules remains the same; only the transaction has changed.

Challenges and Limitations of Market Basket Analysis

Most articles treat MBA like a clean, reliable machine. Run the algorithm; get the insights. That framing is wrong.

The Scalability Problem

What becomes painfully obvious is that the Apriori algorithm does not work well at an enterprise level, and even the FP-Growth algorithm encounters memory constraints when the minimum support value is set very low for millions of transactions and thousands of SKUs. At such a large scale, a distributed approach using Apache Spark MLlib becomes necessary.

Spurious Correlations

The rules found using MBA will be statistically sound but have no commercial value. “Customers who purchase whole milk also purchase other dairy products” is an example of such a rule. Such a rule is of no value. Domain expertise is required to identify the truly actionable rules from those that may be statistically significant but obvious; hence, it requires a person who understands the business domain to conduct MBA

The Symmetric Lift Problem

Lift(A→B) = Lift(B→A). Always. The metric cannot tell you which item is driving the other’s purchase. That’s not a corner case — it’s a fundamental limitation of how lift is calculated. Conviction and Zhang’s metric are correct for this, which is why the metrics section earlier in this article covered all six, not just the standard three.

Sparse Data and New Products

The MBA model suffers from being unaware of products in the long tail and anything new. If a product occurs less frequently than your support threshold allows, the algorithm won’t recognize it at all, regardless of whether it’s always associated with the same other product. Product launchings, seasonal products, and specialty products don’t make the cut for the MBA. It’s the same issue faced by collaborative filters, where nothing is recommended until you’ve tried it once already.

How to Interpret and Act on MBA Results

Here’s where the work actually starts.

After your algorithm runs, you’ll have hundreds of rules. The skill is not generating them — that takes seconds. The skill is knowing which ones are worth acting on and which ones to discard.

[VISUAL: 2×2 action matrix — X-axis: Support (Low to High), Y-axis: Lift (Low to High) — four quadrants with specific action labels. Alt text: “market basket analysis results interpretation matrix lift vs support”]

Use this framework:

High Lift + High Support → Act now. The association is both strong and common. Bundle these products, position them together, build homepage cross-sell widgets around them. This is where 80% of your execution budget belongs.

High Lift + Low Support → Test selectively. The relationship is real but rare. Don’t mass-deploy it; run a targeted promotion for the specific segment where the pattern is most concentrated. Track lift in that segment specifically.

Low Lift + High Support → No action needed. Customers are already making this connection without any help from you. The association is happening naturally; your advertising spend is better deployed elsewhere.

Low Lift + Low Support → Ignore. Statistically weak and commercially uncommon. These rules generate noise in your reports.

One more thing, and it’s important: always sanity-check your top rules with someone who knows the product category. An algorithm will cheerfully produce {ketchup → mustard, lift 2.3} and {mustard → ketchup, lift 2.3} as two separate rules. Both are real. Neither requires action — people who want condiments buy both anyway. The rules that matter are the ones that surprise the category expert. That requires a human to identify; no threshold setting will filter for it automatically.

Running Market Basket Analysis: Tools and Getting Started

Python with mlxtend

Python is the standard choice for MBA in 2026. The mlxtend library provides apriori() and fpgrowth() for frequent itemset mining, plus association_rules() to convert itemsets into rules with full metric output including conviction and leverage.

Your transaction data needs to be in a one-hot encoded matrix: rows are transactions, columns are items, values are 0 or 1. The TransactionEncoder class in mlxtend handles that conversion from raw list format.

R with the arules Package

The arules package is the standard for R-based MBA. It supports both Apriori and Eclat, has strong documentation, and pairs with the arulesViz package for rule visualization. DataCamp has a dedicated market basket analysis in R course that covers the full workflow from data prep through rule interpretation.

No-Code Options for Business Users

IBM SPSS Modeler has a built-in MBA node that requires no programming — you drag transaction data in, configure support and confidence thresholds, and review rules in a visual interface. RapidMiner offers similar functionality through a visual pipeline builder.

Both are enterprise tools with significant licensing costs. They’re appropriate for large organizations that need MBA output without a data science team.

Frequently Asked Questions

Closing

From what I’ve seen in articles about Market Basket Analysis, I believe that authors completely fail to grasp what is more important in practice. They dedicate 80% of the article’s word count on explaining the algorithm while leaving only 5% for explaining what needs to be done after it. However, for anyone who is planning to conduct an actual Market Basket Analysis project, the algorithm itself takes only a few seconds to run. The hardest thing is to face the stakeholders armed with 400+ rules sorted by Lift, and tell them what three changes they need to make next quarter.

This decision is not driven by mathematics.

Understanding what exactly each metric measures, what it leaves out, and how to filter out the signal from the noise based on your domain knowledge, which will never be known to the algorithm itself, is critical. Begin with FP-Growth. Apply generous thresholds. Look at the top 20 rules by Lift. Filter out all those which are strong and meaningful enough. Next step: if such a rule is identified, what changes should you apply?

If the answer is “nothing”… the rule has no value whatsoever despite its high lift. Algorithms detect patterns; you determine their value.