Zeno++ Robust Fully Asynchronous Stochastic Gradient Descent

A Byzantine-tolerant fully asynchronous stochastic gradient descent algorithm for parameter-server architectures with anonymous workers, unbounded worker delay, and an arbitrary number of Byzantine workers that maintains a lazily updated server validation gradient v = nabla f_s(x_tau) (with server staleness t - tau <= k), normalizes each arriving candidate worker gradient g_tilde to g = c * g_tilde such that ||g||_2^2 = ||v||_2^2, and applies the asynchronous update x_t <- x_{t-1} - gamma * g if and only if the first-order Taylor approximated stochastic descent score satisfies gamma * <v, g> - rho * ||g||_2^2 >= -gamma * epsilon.