Recursive prediction-error estimation
Algorithms AIMech:RecursivePredictionErrorEstimation DRAFT
Aliases: Recursive prediction error method
A recursive parameter-estimation method that incrementally adjusts a parameterized predictor using prediction errors and search directions based on its parameter sensitivities.
Identity Notes & Concept Scope
Normalized comparisons covered all 442 pinned AIO IDs, labels and scoped synonyms and 530 native IDs, names and aliases, including expanded prediction-error and pseudo-linear names. No identity match was found. Ignored- and hidden-inclusive full-text searches covered the recovery worktree and readable previous worktree, returning five and zero lines; all five concern the pending source cohort, not existing estimator records. Original Documents untracked files remain inaccessible and are not claimed re-audited. Read all 50 AIO semantic rows and 159 native name/definition neighbors, eight complete annotated AIO CLASS nodes with incident edges, and the complete SGD, L-BFGS and orthogonal least-squares records. The NARX/NARMAX family records were fully read in the preceding round. Generic optimization and regression are related scopes; they do not supply this estimator identity. NARMAX is a model family, forward OLS selects structure, and recursive neural networks denote architectures. Recursion in a name is not equivalence. These methods overlap in special cases without being identical families. LinearRegression does not specify parameter-dependent residual recursion, ARMA is a model family, and TimeSeriesAnalysis/Forecasting and SupervisedLearning are task contexts. None is an exact match. SGD can overlap with a simple search-direction choice, but its generic sampled-gradient definition does not identify this recursive predictor treatment; L-BFGS uses a different curvature-history mechanism.
Claims & Grounded Evidence
6 Claims
Mechanism
#prediction-error-update
For the squared prediction-error criterion, let epsilon_t=y_t-yhat_t and let psi_t denote the recursive predictor-sensitivity approximation. The displayed Gauss–Newton form updates R_t=(1-gamma_t)R_(t-1)+gamma_t*psi_t*psi_t^T and theta_t=theta_(t-1)+gamma_t*R_t^(-1)*psi_t*epsilon_t, with positive-definite curvature handling as needed. The positive update sign corresponds to minus the residual-loss gradient because epsilon=y-yhat. R approximates curvature; it is not the exact Hessian of every nonlinear objective. Other search directions belong to the broader prediction-error method family.
Mechanism
#residual-dependent-sensitivity
For a fixed parameter theta and predictor theta^T*phi_t(theta) built from past residuals, the total sensitivity is psi_t=phi_t-sum_i[theta^T*(partial phi_t/partial epsilon_(t-i))]*psi_(t-i). The online algorithm uses previous parameter estimates and stored residuals/sensitivities. This is a recursive approximation based on the fixed-parameter derivative, not exact differentiation through the entire sequence of changing parameter updates. Treating phi as parameter-independent generally changes the direction.
Mechanism
#linear-noise-filter
With a nonlinear process predictor and a constant-coefficient linear lagged-error polynomial C(q^-1)=1+sum_i c_i*q^-i, the source writes the sensitivity recursion as filtering the current regressor through 1/C. The leading coefficient is 1. Predictor/filter stability concerns residual reconstruction and differs from stability of the physical system dynamics. Merely making the noise terms linear in past errors does not guarantee a stable filter.
Mechanism
#inverse-curvature-form
An algebraically equivalent inverse update uses P_t=[P_(t-1)-P_(t-1)*psi_t*psi_t^T*P_(t-1)/(lambda_t+psi_t^T*P_(t-1)*psi_t)]/lambda_t and theta_t=theta_(t-1)+P_t*psi_t*epsilon_t, where P_t=gamma_t*R_t^(-1) and lambda_t=gamma_(t-1)*(1-gamma_t)/gamma_t. The equivalence requires well-defined inverses and nonzero denominators; the gain and forgetting factors are linked rather than arbitrary independent choices. Exact two-dimensional calculations check this identity for a specified positive-definite common state.
Limitation
#conditional-convergence
The ODE-based convergence discussion assumes a stable predictor domain, admissible parameter restrictions, suitable smoothness and data/moment limits, decreasing gains, and positive-definite curvature. It concerns local behavior under those conditions, not unconditional global minimization. For general nonlinear noise the stable domain can depend on unknown input/output statistics, preventing straightforward enforcement. The source’s recommendation to use simpler noise models is not a universal convergence proof.
Application
#fixed-structure-identification-use
The paper applies recursive prediction-error estimation to a simulated closed-loop polynomial system and a measured pilot liquid-level system. Model structures are determined beforehand. For the liquid-level example, recursive and offline estimators use the same selected NARMAX structure. These examples establish a documented parameter-identification use; they are not held-out accuracy, causal recovery or universal superiority evidence.
⚠️ Curated Limitations & Negative Results
- The inspected mechanism evidence is the complete 1989 polynomial-NARMAX treatment. Earlier recursive-identification sources and broader filtered, vector, nonquadratic, rational and output-affine variants remain follow-up scopes; no priority claim is made.
- Model structure, initialization, gain or forgetting schedule, numerical conditioning and predictor stability require separate choices. A recursive parameter estimator does not select a correct model structure, establish physical causality or guarantee out-of-sample accuracy.
- The source defines m-invertibility by mean-square residual-initialization forgetting. Almost-sure or exponential forgetting is not inferred solely from that definition. Its nonlinear noise examples and convergence extensions require their stated assumptions; no unrestricted theorem is adopted.
- All 27 rendered PDF pages were read, including the experiments and references. Exact examples are curator derivations, not replications of the source experiments. No model training, empirical benchmark or paid service was run. DRAFT status and author self-review do not imply independent scientific approval.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:LeastsquaresAnalysis
Least-squares Analysis
|
related_match | The squared-residual criterion and recursive least-squares algebra directly relate these parameter updates to residual-squared fitting. The pinned target describes an overdetermined fitting analysis, not this recursive estimator; the pseudo-linear approximation also need not minimize the full nonlinear prediction-error objective. A related mapping preserves these differences. |
AIO:RegressionAnalysis
Regression Analysis
|
related_match | The estimator fits a parameterized relationship between observed outputs and predictive lagged information. The pinned target groups statistical fitting processes and does not specify this update or its residual dependence. This is a contextual algorithm/fitting relationship rather than an exact or inferred subclass assertion. |
Evidence Sources & Primary Literature
1 Source| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
chen-billings1989
|
S. Chen, S. A. Billings
|
primary paper
full_text
|
ID: https://doi.org/10.1080/00207178908559655
sha256:7716a5eca47f…
|
Curation Provenance
Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-12T13:23:25Z
Updated: 2026-10-12T13:30:40Z
Source File:
data/curated/RecursivePredictionErrorEstimation.yaml