Swapout

⚠️ Curated Limitations & Negative Results

• Compatible tensor shapes, valid Bernoulli probabilities and an acyclic computation are prerequisites. Selector sharing and coupling specify different procedures; the independent two-selector specialization is not silently replaced by complementary SkipForward, whole-branch stochastic depth, recurrent Zoneout, or the separately prescribed backward multipliers of Shake-Shake and ShakeDrop. The displayed Equation 2 outcome set omits the all-zero sum although the selectors permit it; Equation 3 explicitly includes zero. No implementation-level random-number, sharing, device or computation-skipping behavior is certified.
• The generic Equation 2 stability sketch on PDF p. 4 does not justify its pointwise gradient comparison merely by selecting the sampled member with the largest global Lipschitz constant. As a hand example with parameter v, branches F1(v)=v and F2(v)=2 sin(v) and independent half-Bernoulli selectors yield four members with global Lipschitz constants 0,1,2,3. At v=2*pi/3 their derivatives are 0,1,-1,0: the mean gradient magnitude is 1/2, whereas the largest-Lipschitz member has derivative zero there. A common bound of 3 remains valid. This witness concerns the generic source argument, not an asserted instability result for ordinary ReLU residual networks, a full audit of the cited stability theorem, or an author-issued erratum.
• Local expectations do not generally commute with downstream nonlinearities or losses. With fixed contributions X=1 and Y=-1 and independent half-Bernoulli selectors, expected ReLU of their selected sum is 1/4 while ReLU of its mean is zero. Likewise, local dropout/ReLU commutation does not establish exact inference for an entire nonlinear dropout network. Averaging sampled predictions is a finite approximation, not by itself a calibrated uncertainty estimate or a universal sample-count guarantee.
• The source variance comparison requires explicit assumptions. If two Bernoulli(theta) masks are mutually independent and independent of finite-second-moment features X,Y, the variance excess over theta*X+theta*Y is theta*(1-theta)*(E[X squared]+E[Y squared]); X and Y need not be independent. For interior theta it also vanishes when X=Y=0 almost surely. The mean theta*X+theta*Y is conditional on the features; an unconditional mean uses their expectations. Different sharing or feature dependence changes the calculation. This does not prove that batch normalization always fails with deterministic inference or that stochastic inference always cures it.
• Generalization, parameter-tying and co-adaptation explanations and the numerical comparisons are scoped empirical or conjectural observations. In Table 3, PDF p. 7, the first deterministic Swapout error is 8.58 versus ResNet 8.27, while the stochastic column improves every displayed row; nearby prose explicitly says most settings for deterministic inference. The caption does not specify an inference convention, so it is not treated as an unambiguous source contradiction. The preliminary ImageNet setting also performs worse in the source. No universal accuracy, stability, convergence, parameter-efficiency or runtime advantage is asserted.
• Evidence is the exact nine-page official NIPS 2016 proceedings PDF. Its file-build timestamp is not an exact publication date, and separately inspected arXiv submission metadata does not establish byte identity with the preprint. The official supplementary ZIP returned material for Perspective Transformer Nets, a different paper; it is excluded from all Swapout evidence and retained as a separate pending source lead. Historical research read all main-paper text and selected visuals, without inspecting author code, executing training or reproducing benchmarks. Source-access failures and the pre-draft qualification of the Table 3 caption remain separately retained provenance.