Kernel Principal Component Analysis
Algorithms AIMech:KernelPCA DRAFT
Aliases: Kernel PCA, KPCA
A spectral feature-extraction algorithm that performs principal component analysis on centered implicit feature vectors by eigendecomposing a positive-semidefinite kernel Gram matrix and evaluating new-point coordinates through kernel expansions.
Identity Notes & Concept Scope
The Schölkopf-Smola-Müller kernel eigenvalue construction generalizes ordinary PCA to a kernel-defined feature space. It includes linear PCA as a special case but is not the identity of ordinary input-space PCA, an SVM classifier, a dot-product KernelLayer, Isomap, LLE or Diffusion Maps. A trained embedding, chosen kernel and preimage approximation are not separate aliases for the algorithm. Three candidate identities were compared against all 442 pinned AIO IDs, labels and scoped synonyms and 454 preceding native IDs, names and aliases, with no normalized match. Searches of both complete checkouts included ignored and hidden files and returned the same 176 matched-line set across 81 paths. All matching excerpts were read and distinguished from existing concept records; historical ledgers were not reread in full solely because they contained a match. All 213 semantic-search rows were read, including 212 unchanged prior rows and the additional KernelLayer row. Twenty-one annotated AIO nodes with outgoing edges were inspected: 19 unchanged prior readings and two new readings. Thirty-one contextual YAMLs, including ten native neighbors, comprise 23 unchanged prior readings and eight complete readings in this round. This identity assessment is bounded to the pinned snapshot and maintained corpus.
Claims & Grounded Evidence
4 Claims
Mechanism
#centered-kernel-eigenproblem
For M training points and a positive-semidefinite kernel k, form K_ij=k(x_i,x_j) and Kc=H K H with H=I-11^T/M. If Kc v=kappa v, ||v||=1 and kappa>0, coefficients alpha=v/sqrt(kappa) define a unit feature-space principal axis. The corresponding empirical covariance eigenvalue is kappa/M. Centered rank is at most M-1. Zero Gram eigenvalues do not define additional nonzero covariance directions.
Mechanism
#new-point-projection
For a new x, center its similarity to each training point using the same training feature mean: kc_i(x)=k(x_i,x)-M^(-1)sum_j k(x_j,x)-M^(-1)sum_j K_ij+M^(-2)sum_jl K_jl. Its component coordinate is sum_i alpha_i kc_i(x). Centering input coordinates before a nonlinear kernel does not generally center the feature vectors; centering each test batch independently changes the learned projection.
Scope
#feature-space-optimality
With a fixed valid kernel, leading components maximize retained empirical variance and minimize rank-constrained squared reconstruction error in the centered feature space. Their sample coordinates are uncorrelated, not necessarily statistically independent. They can outnumber input dimensions. A linear kernel recovers ordinary PCA with consistent centering. Truncated feature reconstructions or principal directions need not have an exact input-space preimage, so reconstructing original inputs is a separate problem.
Limitation
#kernel-validity
The feature-space covariance interpretation requires a positive-semidefinite kernel. The paper lists sigmoid similarities, but arbitrary tanh parameters do not meet that requirement: a curator two-point example k(x,y)=tanh(x*y), x in {1,2}, gives a negative Gram determinant and a negative eigenvalue even after centering. Keeping selected positive eigenpairs of an indefinite similarity does not establish that the original similarity is a Hilbert-space inner product.
⚠️ Curated Limitations & Negative Results
- Kernel and hyperparameter selection remain modeling decisions. An eigensolution for a fixed kernel does not solve all kernel-selection or preimage optimization problems.
- The exact Gram construction requires storage quadratic in training sample count; partial eigensolvers, subsets and reduced-set projections are additional approximation choices. Implicit features avoid explicit feature-dimension costs, not all sample-size costs.
- Eigenvector signs and bases within degenerate eigenspaces are nonunique. Variance and reconstruction guarantees concern the chosen feature space, not causal factors, independence, universal input reconstruction or classification accuracy.
- All 21 source pages were inspected. Finite calculations verify centering, eigenvalue scaling, projection and PSD boundaries; the historical USPS classifier experiments and author implementation were not reproduced.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:PrincipalComponentAnalysis
Principal Component Analysis
|
related_match | Kernel PCA uses ordinary PCA in an implicit centered feature space and recovers it for a linear kernel. The pinned task-hierarchy PCA record is related to this particular kernel algorithm; the mapping asserts neither exact identity nor a forced algorithm/task subclass relation. |
Evidence Sources & Primary Literature
1 Source| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
method-source
|
Bernhard Schölkopf, Alexander Smola, Klaus-Robert Müller
|
primary paper
full_text
|
ID: https://alex.smola.org/papers/1998/SchSmoMul98.pdf
sha256:ac778f1c3740…
|
Curation Provenance
Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-12T02:05:59Z
Updated: 2026-10-12T02:05:59Z
Source File:
data/curated/KernelPCA.yaml