Latent Dirichlet allocation
Model Families AIMech:LatentDirichletAllocation DRAFT
A hierarchical model for collections of discrete observations in which each document draws a Dirichlet-distributed mixture over a fixed finite set of topics, and each token draws a topic and then an observation from that topic’s categorical distribution.
Identity Notes & Concept Scope
The defining feature is a shared prior over document-specific topic proportions together with repeated per-token topic selection. This differs from PLSA’s fitted document-index weights and a mixture selecting one topic for a whole document. The model is distinct from its variational EM, sampling or other inference algorithms. The bare LDA alias is omitted because AIMech already uses it for the unrelated Gaussian classifier LinearDiscriminantAnalysis. The base fixed-topic-probability presentation and its Bayesian topic-prior extension are distinguished explicitly. Compared all 442 pinned AIO labels and scoped synonyms and 418 preceding native IDs, names and aliases. The initial normalized LDA hit denotes LinearDiscriminantAnalysis, whose full Gaussian-classifier record was read; the bare LDA alias is omitted from the new topic-model record. No normalized identity match remains for the final candidate names/aliases. Final full-checkout fixed-string searches, including ignored and hidden files in both worktree and main, return two lines each; both unique contents are existing frontier references. An additional broad LDA search returned 289 worktree and 237 main lines before the preceding batch merged; all 84 distinct contents were read, covering the existing classifier, topic-context references, unrelated programming mnemonics and namespace abbreviations. All 188 semantic label/synonym hits were read. Nineteen exact annotated AIO CLASS nodes and outgoing edges were inspected (18 verified unchanged from prior reading, Biclustering read here), along with their generated records and 16 native neighbors. The 35 contextual YAMLs were read or byte-verified after documented full reading. Absence is bounded to this snapshot, maintained corpus and inspected semantic neighborhood.
Claims & Grounded Evidence
1 Claim
Mechanism
#method
For a fixed K, positive alpha_i and normalized topic-word probability rows B_i, draw theta_d~Dirichlet(alpha) independently for each document. Given a document length N_d, draw z_dn~Categorical(theta_d) and w_dn~Categorical(B_zdn) for each token. Conditional on theta_d and B, token probabilities are sum_i theta_di*B_iw and tokens are independent; integrating theta_d gives p(w_d|alpha,B)=integral Dir(theta_d;alpha)*product_n(sum_i theta_di*B_i,w_dn)dtheta_d. This is an exchangeable document distribution, generally with dependent tokens after integration. An unordered count vector additionally includes its multinomial multiplicity. The paper’s Poisson length law is optional to this conditional construction. For fixed B, its mean-field approximation is q(theta,z)=Dir(theta;gamma)*product_n Cat(z_n;phi_n), with phi_ni proportional B_i,w_n*exp[digamma(gamma_i)-digamma(sum_j gamma_j)] and gamma_i=alpha_i+sum_n phi_ni. Variational EM optimizes the evidence lower bound, updating B from expected topic-word counts and alpha numerically. Section 5.4 separately places Dirichlet priors on B; then the assignment update uses expected log topic probabilities, not fixed B values.
⚠️ Curated Limitations & Negative Results
- The number of topics is finite and fixed in this construction. Integrating a continuous Dirichlet mixture does not introduce infinitely many topic classes. Infinite exchangeability supports the representation discussed in Section 3.1; arbitrary finite exchangeability alone is not the asserted theorem.
- Given observed topic assignments, Dirichlet conjugacy is exact. With topics unobserved, the marginal posterior over proportions is generally not a single Dirichlet; gamma=alpha+expected counts is a variational update, not exact marginal conjugacy. Exact inference is generally difficult, and bound optimization need not monotonically improve the true likelihood or attain its global maximum.
- The basic model ignores word order and explicit within-topic dependencies, though integrating document proportions induces dependence between tokens. Zero word probabilities require smoothing or other support handling for novel documents. A prior over document mixtures does not guarantee freedom from overfitting, identifiable topics or meaningful semantic labels.
- The Bayesian topic-prior extension requires expected log topic probabilities in its assignment update. The appendix’s printed alpha Hessian has a sign/factor mismatch with the preceding gradient; the consistent Hessian is M*(trigamma(sum alpha)-delta_ij*trigamma(alpha_i)). This record does not adopt that printed Hessian as verified code. Equation (13) correctly uses the normalized posterior.
- The comparison’s empirical mixture over training-document indices must weight an entire document once; the printed p(d) inside its token product is not adopted. These mathematical qualifications do not establish an error in unseen author software or experiments.
- The source’s document, classification and recommendation experiments have distinct protocols. No exact likelihood replication, generic performance advantage or biological-mechanism conclusion is claimed. All source material concerns the specified 2003 model, not every later topic-model extension.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:Model
Model
|
broad_match | The local concept specifies a mathematical probability model and is narrower than the pinned generic Model class. This is a local scope assessment, not an upstream assertion. |
AIO:ProbabilisticTopicModel
Probabilistic Topic Model
|
related_match | The local model describes latent topic distributions. The inspected AIO target lies in MachineLearningTask and is phrased as analyzing word themes; related match preserves the model-to-task distinction without asserting an is-a relation or temporal dynamics. |
Evidence Sources & Primary Literature
1 Source| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
method-source
|
David M. Blei, Andrew Y. Ng, Michael I. Jordan
|
primary paper
full_text
|
ID: https://www.jmlr.org/papers/v3/blei03a.html
sha256:4667de63545b…
|
Curation Provenance
Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-11T23:19:29Z
Updated: 2026-10-11T23:19:29Z
Source File:
data/curated/LatentDirichletAllocation.yaml