Probabilistic Principal Component Analysis
Model Families AIMech:ProbabilisticPCA DRAFT
Aliases: PPCA
A linear Gaussian latent-variable model with a standard-normal latent prior and isotropic observation noise, whose maximum-likelihood loading subspace corresponds to leading principal components under the stated covariance-rank conditions.
Identity Notes & Concept Scope
The single Gaussian observation model is distinct from ordinary PCA as a deterministic projection, diagonal-noise factor analysis, kernel PCA, discrete-latent GTM, generalized component analysis with categorical observations, and supervised PCA. Its eigensolution and EM procedure are fitting alternatives for the same model. Mixtures, missing-data fitting and hierarchical variants require their own scope. Three candidate identities were compared against all 442 pinned AIO IDs, labels and scoped synonyms and 457 preceding native IDs, names and aliases, with no normalized collision. Full searches of both checkouts included ignored and hidden files and returned the same 32 matching excerpts across nine paths. All excerpts were read; frontier leads and neighboring-model comparisons are not duplicate records. Historical ledgers were not reread in full solely because an excerpt matched. All 219 semantic-search rows were read, including six additional adaptive-pooling rows. Nineteen annotated AIO nodes and their outgoing edges were byte-verified against previous complete readings. Twenty-eight contextual YAMLs, including nine native neighbors, comprise 25 unchanged prior readings and three complete current readings. The identity and mapping assessment is bounded to this snapshot and maintained corpus.
Claims & Grounded Evidence
4 Claims
Mechanism
#latent-density
For latent dimension q<d, let x~Normal(0,I_q) and t=Wx+mu+epsilon with independent epsilon~Normal(0,sigma^2 I_d), sigma^2>0. Then t~Normal(mu,C), C=WW^T+sigma^2 I_d. The observed-data model is a normalized full-rank Gaussian; the low-rank loading contribution does not make the positive-noise density singular. Isotropic noise is a stronger restriction than the arbitrary diagonal noise of factor analysis.
Mechanism
#likelihood-eigensolution
For sample covariance S=N^(-1)sum_n(t_n-mu)(t_n-mu)^T at the sample-mean mu, take its q leading eigenpairs U_q,Lambda_q. The likelihood solution has sigma^2 equal to the mean of the d-q discarded eigenvalues and W=U_q(Lambda_q-sigma^2 I)^(1/2)R for any orthogonal R. Rank(S)>q ensures positive residual variance. Tied eigenvalues can make loading columns vanish: for spectrum (2,1,1) and q=2, sigma^2=1 and the effective loading rank is one. Neither the columns nor latent coordinates are uniquely identified.
Mechanism
#posterior-reconstruction
With M=W^T W+sigma^2 I, the latent posterior is Normal(M^(-1)W^T(t-mu),sigma^2 M^(-1)). Mapping its mean back through W generally shrinks the observation within the loading subspace; it is not the ordinary orthogonal PCA reconstruction. When W has full column rank, W(W^T W)^(-1)M times the posterior mean, plus mu, recovers the orthogonal projection. The zero-noise limit loses a nonsingular ambient density for q<d.
Mechanism
#em-fitting
The E-step computes posterior first and second latent moments using the old W and noise variance. The M-step fits W by observation-latent cross-moments times the inverse summed latent second moment, then updates noise from the expected squared residual using the new W and old posterior moments. Exact well-defined EM makes likelihood nondecreasing, but a zero-loading initialization can remain stationary at an inferior isotropic solution. The existence of a principal-eigenspace maximum does not guarantee successful convergence from every initialization.
⚠️ Curated Limitations & Negative Results
- Latent dimension is a model choice. When residual eigenvalues all vanish, the formal noise estimate reaches the zero-variance boundary, outside the nonsingular positive-noise model. Latent rotations and covariance eigenvalue ties prevent unique axes without additional conventions.
- The conditional mean is a probabilistic latent summary, not automatic recovery of physical factors or minimum input-space reconstruction error. Isotropic Gaussian noise can be misspecified.
- Missing-at-random integration and mixtures of PPCA are extensions described in the paper; they are not properties of an arbitrary imputation scheme or aliases for this single Gaussian model. This record does not cover supervised PCA.
- All 12 primary pages were read. Finite arithmetic checks one exact EM step, posterior shrinkage, rotation/rank ambiguity and a stationary initialization. The Tobamovirus illustrations and author software were not reproduced.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:Model
Model
|
broad_match | PPCA is a specific mathematical probability model within the pinned Model class. |
AIO:PrincipalComponentAnalysis
Principal Component Analysis
|
related_match | Its likelihood eigensolution spans a principal subspace, but the normalized latent model, posterior uncertainty and shrinkage differ from the task-scoped PCA record. This relation preserves the model/task distinction. |
Evidence Sources & Primary Literature
1 Source| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
method-source
|
Michael E. Tipping, Christopher M. Bishop
|
primary paper
full_text
|
ID: https://www.cs.columbia.edu/~blei/seminar/2020-representation/readings/TippingBishop1999.pdf
sha256:aad631f02874…
|
Curation Provenance
Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-12T02:24:31Z
Updated: 2026-10-12T02:24:31Z
Source File:
data/curated/ProbabilisticPCA.yaml