Optuna in Production: Smarter Hyperparameter Tuning

Why Optuna Over GridSearch

• Bayesian optimization learns from past trials
• Pruning stops unpromising trials early (saves 60-80% compute)
• Persistent storage resumes studies across sessions

Minimal Example

import optuna

def objective(trial):
    params = {
        'n_estimators': trial.suggest_int('n_estimators', 100, 2000),
        'learning_rate': trial.suggest_float('lr', 0.001, 0.3, log=True),
        'max_depth': trial.suggest_int('max_depth', 3, 10),
    }
    model = LGBMClassifier(**params)
    return cross_val_score(model, X, y, cv=5, scoring='roc_auc').mean()

study = optuna.create_study(direction='maximize')
study.optimize(objective, n_trials=100)

Pruning with Callbacks

from optuna.integration import LightGBMPruningCallback
callback = LightGBMPruningCallback(trial, 'auc')
model.fit(X_train, y_train, callbacks=[callback])