Handling Missing Data Like a Pro: Smart Strategies Every Data Scientist Should Know

📗 Chapter 10: Evaluating Post-Imputation Quality

How to Know If Your Missing Data Strategy Actually Worked

🧠 Introduction

Imputation fills the gaps — but how do you know if you filled them correctly?

Post-imputation evaluation is crucial to ensure you haven’t introduced bias, noise, or distorted patterns.

This chapter teaches you:

• Techniques for validating imputation effectiveness
• Statistical and visual evaluation methods
• Metrics like RMSE, MAE, KL divergence
• Simulating missingness for testing
• Comparing multiple imputation strategies
• Building a feedback loop for model improvement

📊 1. Why Evaluate Imputation?

Imputation should:

• Preserve statistical properties
• Improve (or at least not harm) model performance
• Not introduce fake patterns or bias

If not evaluated:

• You risk degraded accuracy
• Important trends can be flattened or exaggerated
• Your model might overfit to incorrect assumptions

🔄 2. Common Evaluation Approaches

🔬 3. Simulating Missingness for Validation

When true missing values aren't known, simulate them.

Example:

python

import numpy as np

# Make a copy of original

df_eval = df.copy()

# Mask 20% of values from a column

mask = np.random.rand(len(df_eval)) < 0.2

df_eval['Age_masked'] = df_eval['Age']

df_eval.loc[mask, 'Age_masked'] = np.nan

Apply your imputation method, then compare:

python

from sklearn.metrics import mean_squared_error

imputed = imputer.fit_transform(df_eval[['Age_masked']])

rmse = mean_squared_error(df_eval['Age'][mask], imputed[mask], squared=False)

print(f"RMSE: {rmse:.2f}")

📏 4. Useful Metrics for Numerical Imputation

RMSE Code Example:

python

from sklearn.metrics import mean_squared_error

rmse = mean_squared_error(true_values, imputed_values, squared=False)

📏 5. Useful Metrics for Categorical Imputation

Accuracy Example:

python

from sklearn.metrics import accuracy_score

accuracy_score(true_values, imputed_values)

📈 6. Visual Techniques for Evaluation

KDE Plot Comparison

python

import seaborn as sns

sns.kdeplot(df['Income'], label='Original')

sns.kdeplot(df_imputed['Income'], label='Imputed')

• Check shape similarity
• Watch for artificial smoothness or flattening

Boxplots:

python

sns.boxplot(data=[df['Income'].dropna(), df_imputed['Income']], orient='h')

Shows spread, outliers, and symmetry.

🔁 7. Before vs. After Comparison

python

comparison = pd.DataFrame({

    'Before Mean': df['Age'].mean(),

    'After Mean': df_imputed['Age'].mean(),

    'Before Std': df['Age'].std(),

    'After Std': df_imputed['Age'].std()

}, index=['Age'])

🧠 8. Use Modeling to Validate

Build a prediction model on:

1. Raw data
2. Imputed data
3. Imputed + missingness indicator

Compare results.

python

from sklearn.ensemble import RandomForestClassifier

from sklearn.model_selection import cross_val_score

scores = cross_val_score(RandomForestClassifier(), X_imputed, y, cv=5)

print("CV Accuracy:", scores.mean())

A good imputation strategy shouldn’t degrade performance.

🧰 9. Tools & Libraries

📉 10. Common Red Flags in Post-Imputation

✅ 11. Best Practice Tips

📋 Summary Table: Evaluation Strategies