When to Use Genetic Algorithms
- Combinatorial problems (feature selection, scheduling)
- Non-differentiable objective functions
- Multi-modal search spaces
- When Bayesian optimization is too expensive
Feature Selection with GA
from deap import base, creator, tools, algorithms
# Genome: binary vector (1=feature included, 0=excluded)
creator.create("FitnessMax", base.Fitness, weights=(1.0,))
creator.create("Individual", list, fitness=creator.FitnessMax)
def evaluate(individual):
selected_features = [i for i, v in enumerate(individual) if v == 1]
if not selected_features: return (0.0,)
X_sel = X_train[:, selected_features]
score = cross_val_score(model, X_sel, y_train, cv=3).mean()
# Penalize using too many features
penalty = len(selected_features) / len(individual) * 0.05
return (score - penalty,)
Key Parameters
- Population: 50-200 individuals
- Mutation rate: 0.01-0.05 per gene
- Crossover rate: 0.7-0.9
- Generations: 100-500 (early stopping on plateau)