FedNTD (Federated Not-True Distillation for Global Knowledge Preservation)

A heterogeneous federated learning algorithm (Lee et al. 2022, Section 3, Equations 10-14, Algorithm 1) that mitigates catastrophic forgetting of out-of-local-distribution classes during client local updates by augmenting the local cross-entropy loss L_CE(q^l, 1_y) with a beta-weighted Not-True Distillation loss L_NTD(q_tilde_tau^l, q_tilde_tau^g) := -sum_{c=1, c != y}^C q_tilde_tau^g(c) log[q_tilde_tau^l(c) / q_tilde_tau^g(c)] (Equations 10-11), where the not-true softmax probabilities q_tilde_tau^l(c) = exp(z_c^l / tau) / sum_{c_tilde != y}^C exp(z_{c_tilde}^l / tau) and q_tilde_tau^g(c) = exp(z_c^g / tau) / sum_{c_tilde != y}^C exp(z_{c_tilde}^g / tau) normalize client and frozen server logits exclusively over the C - 1 non-target classes c != y, yielding identically zero distillation gradient with respect to the ground-truth logit z_y^l while matching the out-local class distribution p_tilde_c = (1 - p_c) / (C - 1) (Definition 1, Propositions 1-2).