ShakeDrop regularization

⚠️ Curated Limitations & Negative Results

• Independent forward and backward multipliers generally do not give the chain-rule derivative of the realized forward computation or an unbiased gradient of its expected loss. The prescribed rule is intentional. The source ratio beta/alpha requires a nonzero denominator and does not by itself imply an unbounded absolute backward multiplier; alpha=0 is an explicit configuration. With negative alpha and positive beta, the sign comparison concerns the residual contribution relative to the derivative of that same perturbed forward pass, not necessarily the gradient of an unperturbed network.
• Averaging local coefficients does not commute with subsequent nonlinearities, losses or training-dependent batch-normalization state. The inference rule is not an exact full-network ensemble average or a calibrated uncertainty estimate. Simplifying its coefficient to p_l when p_l<1 requires E[alpha | b_l=0]=0; this is satisfied, for example, by independent gate and coefficient draws with zero-mean alpha.
• The fail-safe and local-minimum explanations and the conceptual trajectories in Figure 2 are heuristic. They establish no general convergence, nondivergence or superiority guarantee. Tables 2–3 and the later experiments show that coefficient ranges, sharing, depth, architecture and p_L matter. Valid Bernoulli probabilities and compatible residual/skip shapes are prerequisites of the displayed construction.
• Source versions use different defaults. The original preprint selects image-level sharing for memory efficiency and emphasizes batch-normalization-ending branches; the later author copy uses element-level sharing and also studies constant-zero alpha on original architectures. Their Type A and Type B names for separate-versus-summed residual perturbation are reversed: the original preprint describes A before and B after the residual sum, whereas the later Equations (10)–(11) define A after and B before. Those labels and experimental settings are not silently unified.
• The same gate is used across the forward and backward rules. Resampling it between passes would define a different procedure. A zero backward multiplier suppresses the local residual-path loss-gradient contribution, not necessarily every parameter or state change under momentum, weight decay, other paths or running statistics. A forward-zero branch with a nonzero prescribed backward contribution does not justify skipping all residual computation. No implementation-level audit of every sharing or parallelism convention is asserted.
• The principal evidence is the specifically retained 2019 author-hosted accepted-proof-layout copy with local pages 1–11 and placeholder publication dates. Its DOI and bibliographic journal pages identify the article, but this record does not assert byte identity with a final publisher PDF or arXiv v3. The original 2018 v1 preprint is separately cited as version context. File-build timestamps and current acquisition times are not publication dates.
• The evidence concerns specified residual CNNs, datasets and training settings. Architecture changes, controls and training schedules differ across versions. No new numerical benefit, universal stability, causal-ablation result, speedup guarantee or benchmark reproduction is claimed. The source texts and selected formula/table visuals were read, but no author implementation, model training or later library behavior was inspected or executed.