Back to Blog
Machine Learning February 1, 2025 7 min read

Class Imbalance in Production: What Actually Works

After 20+ imbalanced classification projects — fraud, medical, churn — here is what actually moves the needle: SMOTE, class weights, threshold tuning, and cost-sensitive learning.

The Imbalance Spectrum

  • Mild (1:10): Class weights usually sufficient
  • Moderate (1:100): SMOTE + class weights
  • Severe (1:1000+): Anomaly detection framing, cost-sensitive learning

Technique Ranking (from my experience)

  1. Threshold tuning — Always do this. Default 0.5 is almost never optimal.
  2. Class weights — Easy, no data distortion, works 90% of the time.
  3. SMOTE — Helps on small datasets. Can hurt on large ones.
  4. Oversampling minority — Simple, often underrated.
  5. Undersampling majority — Loses information. Use carefully.

The Right Metric

Never use accuracy on imbalanced data. Use:

  • F1 / F2 score
  • PR-AUC (better than ROC-AUC for severe imbalance)
  • Business-relevant cost matrix
Class ImbalanceSMOTEFraud DetectionClassificationMetrics
O

Ossama Elhakki

AI Engineer & ML Systems Builder — Morocco

About me →Contact →