FedDecorr (Federated Feature Decorrelation Regularization for Mitigating Dimensional Collapse)

A heterogeneous federated representation learning algorithm (Shi et al. 2023, Sections 3.3-4, Theorem 1, Proposition 1, Equations 6-9, Algorithm 1) that counteracts heterogeneity-induced dimensional collapse of feature representations by adding a local feature-decorrelation regularizer L_FedDecorr(w, X) := (1 / d^2) ||K||_F^2 to the client classification loss min_w ell(w, X, y) + beta * L_FedDecorr(w, X) (Equations 8-9), where K = (1 / N) sum_{i=1}^N z_hat_i z_hat_i^T in R^{d x d} is the empirical sample correlation matrix of z-score standardized d-dimensional penultimate features z_hat_i = (z_i - z_bar) / sqrt(Var(z)), exploiting the algebraic identity sum_{i=1}^d (lambda_i - (1/d) sum_{j=1}^d lambda_j)^2 = ||K||_F^2 - d (Proposition 1, Equation 7) to minimize singular-value variance in O(N d^2) time without computing singular value decompositions.