True online GTD(lambda)

⚠️ Curated Limitations & Negative Results

• This is the original linear state-value off-policy algorithm, not GTD/GQ in general, an action-value controller, true online TD/Sarsa, WIS variants or Dutch/replacing traces alone. Setting every importance ratio to one does not generally eliminate the auxiliary gradient correction for lambda<1. A zero (1−lambda_(t+1)) factor removes that explicit correction, but does not by itself establish identity with every Monte Carlo algorithm.
• Finite-horizon equality to the specified sampled forward views is an algebraic statement, not a proof that each online step is an unbiased MSPBE gradient or that the coupled iterates converge, are stable, improve monotonically or reach an optimal policy. The expected objective uses a behavior-state distribution and an invertible feature second-moment matrix; existence of relevant returns/expectations, sufficient target-action support under behavior, and other convergence assumptions cannot be inferred merely from the update display.
• The derivation uses linear, fixed features and matched replay sequences. The source initializes e_0=alpha_0 rho_0 phi_0 and e^w_0=beta_0 phi_0; a zero pre-initial accumulating trace gives g_0=rho_0 phi_0. The first theta correction cancels, so no arbitrary theta_(−1) affects that first update. Later theta_(t−1)^T phi_t is a stored preceding-weight prediction, not a recomputation under theta_t. Replacing w_t by w_(t+1), swapping current and previous importance ratios, or simply replacing g with a Dutch trace changes the specified construction. Nonlinear or parameter-dependent features need separate justification.
• Alpha_t and beta_t are inside two different scaled traces; a constant-alpha unscaled TD/Sarsa convention cannot be substituted without a matched derivation. A zero current step size does not in general erase trace contributions from earlier nonzero steps. Importance ratios need not be at most one, and these Dutch or accumulating traces are not generally nonnegative or coordinatewise bounded by one. Linear dense cost excludes feature construction and does not imply cost only in current nonzero features.
• The inspected source retains mixed t/k indexing and the recursive superscript in Equation 6, and a scalar/vector mismatch in Equation 13 and nearby expectation prose. Its first proof restatement on PDF p. 3 also omits x_k despite its presence in the theorem and later algebra. These are bounded source-display observations; the explicit forward and backward updates remain separately identifiable. This record neither certifies nor invalidates the entire proof or author implementation.
• The statement on PDF p. 8 that MSPBE equals MSE at lambda=1 needs qualification. When the full-return operator yields v_pi and Pi is the stated d-orthogonal projection, MSE=J+||Pi*v_pi-v_pi||_d^2. The objectives then have the same minimizers, but their values are equal only when the projection residual vanishes. This direct projection identity does not establish convergence of the online algorithm.
• The evidence source is the exact acquired first-author-hosted version of the 2014 paper, with indexed discount/bootstrap expressions in Equation 16 and the final update. The retained conference, reissue and coauthor copies differ in the discount-product endpoint or some time indices; no verified revision date or comprehensive erratum notice is asserted. Experimental comparisons remain tied to the reported random-walk settings, representations and parameter searches, not universal superiority. No author software or benchmark reproduction is certified.