Population Based Training

Maintain population members with model parameters, hyperparameters, an evaluation score and training progress. Each worker trains its parameters with its current hyperparameters and evaluates a chosen performance metric, which need not be differentiable or identical to the local training objective. When a readiness condition holds, an exploitation rule can replace that worker’s parameters and hyperparameters with those of another promising member. An exploration rule then perturbs or resamples the copied hyperparameters, and training continues from the inherited weights. Algorithm 1 calls exploration and re-evaluation only when exploitation changes the weights; its final output is the population member’s parameters with the highest recorded score. Readiness, selection and mutation rules depend on the application. The original procedure is asynchronous: members need access to population scores, parameters and hyperparameters, but no population-wide synchronization barrier is required. Alternating local parameter learning with slower model selection and hyperparameter changes yields schedules along training lineages, rather than restricting each run to one fixed hyperparameter configuration.