Back to Blog
Machine Learning March 5, 2025 10 min read

Feature Engineering Playbook for Tabular ML Competitions

The 15 feature engineering techniques I use in every Kaggle tabular competition — from target encoding to frequency encoding, lag features, and interaction terms.

The 15 Techniques

1. Target Encoding (with smoothing)

smoothed = (count * category_mean + global_mean * smoothing) / (count + smoothing)

2. Frequency Encoding

Replace categorical value with its frequency in the training set.

3. Lag Features

For time series: lag-1, lag-7, lag-30 values of the target.

4. Rolling Statistics

Mean, std, min, max over rolling windows.

5. Interaction Terms

Multiply or divide two numerical features that have domain meaning.

6. Date Parts

Extract year, month, day, hour, weekday, quarter, is_weekend.

7. Rank Features

Rank within group — useful for normalizing scale.

8. Aggregation Features

Group by entity (user, card, store) and compute statistics.

9-15. More in the full post...

Feature EngineeringTabular DataKaggleTarget EncodingCompetition
O

Ossama Elhakki

AI Engineer & ML Systems Builder — Morocco