FastICA
Algorithms AIMech:FastICA DRAFT
Aliases: Fast Independent Component Analysis
A family of independent-component estimation algorithms that repeatedly apply normalized approximate-Newton fixed-point updates to nonlinear projection contrasts, using deflation or symmetric decorrelation to obtain multiple components.
Identity Notes & Concept Scope
Scoped to Hyvarinen-Oja 1997 kurtosis FastICA and Hyvarinen 1999 generalized contrasts. This is an estimation algorithm, not the general ICA model, a trained source separator, ordinary-gradient Infomax ICA, topographic ICA or every fixed-point optimizer. Deflation and symmetric variants share the core update but have distinct ordering and decorrelation behavior; optional stabilization is not a claim that every variant is step-size-free. Three candidate identities were compared against all 442 pinned AIO IDs, labels and scoped synonyms and 454 preceding native IDs, names and aliases, with no normalized match. Searches of both complete checkouts included ignored and hidden files and returned the same 176 matched-line set across 81 paths. All matching excerpts were read and distinguished from existing concept records; historical ledgers were not reread in full solely because they contained a match. All 213 semantic-search rows were read, including 212 unchanged prior rows and the additional KernelLayer row. Twenty-one annotated AIO nodes with outgoing edges were inspected: 19 unchanged prior readings and two new readings. Thirty-one contextual YAMLs, including ten native neighbors, comprise 23 unchanged prior readings and eight complete readings in this round. This identity assessment is bounded to the pinned snapshot and maintained corpus.
Claims & Grounded Evidence
4 Claims
Mechanism
#normalized-fixed-point
For centered whitened observations x with E[xx^T]=I and unit w, choose a differentiable nonlinearity g=Gprime and compute w_plus=E[x*g(w^T x)]-E[gprime(w^T x)]*w, then normalize its nonzero norm. The source derives this from an approximate Newton step for a constrained projection contrast; sample means replace expectations in practice. Cubic g(u)=u^3 gives the original kurtosis step E[x(w^T x)^3]-3w under the whitening and unit-norm conditions. Tanh and Gaussian-weighted-linear nonlinearities are other source choices.
Mechanism
#multiple-components
Deflation subtracts projections on previously estimated orthonormal directions after each update and renormalizes. Symmetric FastICA updates rows together and applies W <- (W W^T)^(-1/2)W for whitened data, requiring full row rank. For nonsphered data, the source uses covariance-metric normalization and decorrelation with nonsingular covariance. Orthogonality prevents exact duplicate directions but does not by itself prove statistical independence. Signs and component ordering are not identified by the ICA objective.
Limitation
#population-convergence-boundary
The generalized proof assumes the linear independent-source model, exact expectations, suitable moments and E[s_i*g(s_i)-gprime(s_i)] nonzero. It establishes local convergence near a source direction, generally quadratic and cubic with the stated symmetry condition. For the original kurtosis rule, source-coordinate updates are proportional to kurt(s_i)*z_i^3. A unique dominant scaled initial coordinate yields the cubic ratio contraction; tied starts can remain mixed up to sign. A source can be non-Gaussian with zero kurtosis, so the kurtosis rule cannot resolve multiple such sources merely from their non-Gaussianity.
Scope
#contrast-and-robustness
The squared contrast [E G(w^T x)-E G(v)]^2, with standard Gaussian v, approximates negentropy rather than giving exact entropy for arbitrary distributions. Contrast choice affects consistency conditions, estimation variance and sensitivity to outliers. The robustness theorem assumes robust whitening and explicitly does not give a globally bounded influence function. The reported outlier experiment estimates covariance from uncontaminated data to emulate robust whitening; it does not certify robustness of an arbitrary complete FastICA pipeline.
⚠️ Curated Limitations & Negative Results
- Standard source identification assumes an appropriate full-rank instantaneous linear mixture, independent sources and at most one Gaussian source; the usable contrasts additionally require their own moments and nondegeneracy. Whitening needs finite covariance and a retained full-rank subspace.
- A zero fixed-point update cannot be normalized. Singular intermediate matrices cannot be repaired to full rank by the displayed symmetric inverse square root or iterative decorrelation formula. Finite-sample errors and model mismatch are outside the exact-expectation convergence proof.
- The original sign-insensitive stopping criterion can declare a mixed tied-start direction unchanged. Finite counterexamples qualify blanket recovery claims; they do not establish typical failure rates for randomized practical implementations.
- Optional stabilized, nonsphered, noisy and overcomplete variants need their stated assumptions. The core algorithm is not universally globally convergent, noise-consistent, step-size-free in all variants, or superior to other estimators.
- Both primary PDFs were read completely, totaling 19 pages. Finite arithmetic verifies zero-kurtosis and tied-start boundaries; no source-separation training, author implementation or historical benchmark was reproduced.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:UnsupervisedLearning
Unsupervised Learning
|
related_match | FastICA estimates source directions from observations without labeled source targets. The task-scoped AIO class is related to this fitting algorithm, without asserting algorithm/task identity or a network architecture. |
Evidence Sources & Primary Literature
2 Sources| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
original-paper
|
Aapo Hyvärinen, Erkki Oja
|
primary paper
full_text
|
ID: https://www.cs.helsinki.fi/u/ahyvarin/papers/NC97.pdf
sha256:7ab4ef35cb53…
|
generalized-method
|
Aapo Hyvärinen
|
primary paper
full_text
|
ID: https://www.cs.helsinki.fi/u/ahyvarin/papers/TNN99_reprint.pdf
sha256:a58d27223e2b…
|
Curation Provenance
Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-12T02:05:59Z
Updated: 2026-10-12T02:05:59Z
Source File:
data/curated/FastICA.yaml