Top 5 Data Science Capstone Project Ideas That Will Impress Employers and Sharpen Your Skills

📗 Chapter 2: Market Basket Analysis & Product Recommendation System

Uncover Consumer Patterns and Suggest the Right Products at the Right Time

🧠 Introduction

What if you could predict the next item a customer might buy?

That’s the power of Market Basket Analysis (MBA) and Product Recommendation Systems — foundational pillars of retail analytics and personalization engines used by Amazon, Walmart, and Netflix.

This project helps businesses boost sales by analyzing purchase behavior and delivering personalized recommendations.

In this tutorial, we’ll cover:

• The basics of market basket analysis
• Association rule mining with Apriori
• Building a collaborative filtering-based recommender
• Evaluating and improving recommendations
• Visualizing purchase patterns

Let’s dive into the world of baskets, association rules, and smart recommendations.

📦 Step 1: Define the Project Goals

🔍 Project Objective

• Market Basket Analysis (MBA): Discover product associations using historical transaction data.
• Recommendation System: Suggest products based on user behavior and preferences.

🏢 Business Use Cases

• E-commerce: "Customers who bought X also bought Y"
• Grocery/retail chains: Layout optimization
• Content platforms: Similar movies, songs, etc.

📊 Step 2: Load & Explore the Dataset

We'll use the Instacart Market Basket Dataset or Online Retail Dataset.

python

import pandas as pd

import matplotlib.pyplot as plt

import seaborn as sns

# Example: Online Retail Dataset

df = pd.read_excel("Online Retail.xlsx")

df.head()

🧼 Clean and Filter

python

# Remove cancellations and missing customer IDs

df = df[df['Quantity'] > 0]

df = df[df['CustomerID'].notnull()]

df = df[df['InvoiceNo'].astype(str).str.startswith('5')]

🛒 Step 3: Create Basket Matrix

Convert transactions into a basket format for association rules.

python

basket = (df

          .groupby(['InvoiceNo', 'Description'])['Quantity']

          .sum().unstack().reset_index()

          .fillna(0)

          .set_index('InvoiceNo'))

# Convert quantities to 1/0

basket = basket.applymap(lambda x: 1 if x >= 1 else 0)

basket.head()

📈 Step 4: Market Basket Analysis with Apriori

python

from mlxtend.frequent_patterns import apriori, association_rules

frequent_items = apriori(basket, min_support=0.02, use_colnames=True)

frequent_items.sort_values(by='support', ascending=False).head()

📌 Association Rules

python

rules = association_rules(frequent_items, metric='lift', min_threshold=1)

rules = rules.sort_values(by='confidence', ascending=False)

rules[['antecedents', 'consequents', 'support', 'confidence', 'lift']].head()

📋 Example Output

🎯 Step 5: Build a Product Recommender (Collaborative Filtering)

Now let’s move to personalized product recommendations.

We'll use user-based collaborative filtering with Surprise.

python

from surprise import Dataset, Reader, KNNBasic

from surprise.model_selection import train_test_split

from surprise import accuracy

Prepare Ratings Data

python

df['Rating'] = df['Quantity']  # or create a custom scoring metric

data = df[['CustomerID', 'StockCode', 'Rating']].drop_duplicates()

reader = Reader(rating_scale=(1, 10))

dataset = Dataset.load_from_df(data[['CustomerID', 'StockCode', 'Rating']], reader)

trainset, testset = train_test_split(dataset, test_size=0.2)

Train a KNN Model

python

algo = KNNBasic(sim_options={'user_based': True})

algo.fit(trainset)

predictions = algo.test(testset)

accuracy.rmse(predictions)

💡 Step 6: Recommend Products to a User

python

user_id = str(df['CustomerID'].sample(1).values[0])

stock_codes = df['StockCode'].unique()

recommendations = []

for stock_code in stock_codes:

    pred = algo.predict(user_id, stock_code)

    recommendations.append((stock_code, pred.est))

top_5 = sorted(recommendations, key=lambda x: x[1], reverse=True)[:5]

top_5

📈 Step 7: Visualize Results

Product Frequency Plot

python

top_items = df['Description'].value_counts().head(10)

sns.barplot(x=top_items.values, y=top_items.index)

plt.title("Top Purchased Items")

plt.show()

Association Rule Network

python

import networkx as nx

G = nx.DiGraph()

for _, row in rules.iterrows():

    G.add_edge(list(row['antecedents'])[0], list(row['consequents'])[0], weight=row['lift'])

plt.figure(figsize=(12, 6))

pos = nx.spring_layout(G)

nx.draw(G, pos, with_labels=True, node_color='lightblue', font_size=10, node_size=3000)

🚀 Step 8: Deployment Ideas

• Deploy with Streamlit to create a recommendation dashboard
• Build a REST API to serve recommendations in real-time
• Combine with a churn model to recommend win-back offers

📋 Summary Table