True online Sarsa(lambda)

⚠️ Curated Limitations & Negative Results

• This identity is the linear on-policy action-value control algorithm, not fixed-policy state-value prediction, the whole Sarsa or TD family, an isolated Dutch trace, original neural MCQ-L, Peng–Williams two-error control, or a greedy Watkins variant. Shared endpoints and source papers do not establish exact aliases.
• Algorithm 3 uses the constant-alpha unscaled trace convention. A naive varying-alpha substitution is not justified by that pseudocode; a varying-step-size Sarsa extension requires its own matched scaled-trace derivation. Source-display problems documented for the separate Watkins Algorithm 5 are not automatically defects of Sarsa Algorithm 3.
• Finite-time algebraic correspondence on a matched realized sequence does not establish convergence, monotone policy improvement, optimal control, stationary-policy value accuracy or universal exploration. Replaying a different action sequence changes the comparison. Epsilon-greedy is an example of behavior selection, not by itself a guarantee of coverage or a useful learned policy.
• The successor action and its prediction are selected and computed before the current weight update. The carried action may cease to be greedy under the updated weights. At a terminal successor, the zero feature vector removes the bootstrap; the printed pseudocode still selects the successor action before its terminal-feature note, so a dummy terminal action or guard is an implementation convention rather than a required real terminal environment action.
• At lambda=1, interim forward targets can still bootstrap. The terminal full-return forward replay is sequential and does not turn the online procedure into a wait-until-end Monte Carlo algorithm. Continuing undiscounted rewards need appropriate return assumptions. Constant alpha=0 freezes the weights but does not establish a learned solution.
• The exact backward derivation depends on linearity in the learned weights and a fixed feature mapping. Tabular, binary or real features are compatible, but generic nonlinear or parameter-dependent features do not automatically inherit that correspondence. Dutch traces can be negative or exceed one. Dense O(d) update cost does not imply current-feature-sparsity cost, action-count-free cost or equal wall-clock performance.
• The 2014 paper names the control extension and reports a mountain-car comparison; the 2016 article supplies the explicit Sarsa pseudocode and further selected experiments. Their empirical claims do not establish universal superiority. Previously retained proof-display qualifications in the 2014 prediction derivation are separate source context, not a new Sarsa implementation defect or an official erratum. The recorded reading is bounded to primary text and selected visuals; no author software or benchmark reproduction is certified.