Home / Curated / Model Families / Product of experts

Product of experts

Model Families AIMech:ProductOfExperts DRAFT

Aliases: PoE, Products of experts

A probabilistic model family that combines expert factors over the same observations by multiplying their values and normalizing the product over the observation space.

Identity Notes & Concept Scope This record denotes the general normalized-product construction, including latent-variable experts, rather than an arithmetic mixture, a training algorithm or a language-model-specific factorization. The original cooperative-learning setting does not require experts to be separately trained. AIO FactorizedLLM has Product of experts as a related synonym, but its definition and parent restrict it to large language models; that related synonym does not identify the general family. Logarithmic opinion pooling uses normalized exponent weights and has a separate aggregation scope. All 442 pinned AIO labels/scoped synonyms and 373 preceding native IDs/names/aliases were compared. The sole normalized hit is Product of experts as a related synonym of FactorizedLLM; its language-specific definition and parent distinguish it from the generic candidate. Both full-checkout fixed-string searches included ignored and hidden files and returned 59 lines each. All 29 unique matched contents were read; other hits concern citations, training components, prior boundary discussions or pending leads. All 88 semantic label/synonym hits, 20 exact fully annotated AIO nodes with outgoing edges, their generated records and four complete native neighbors were inspected or verified unchanged after previous full reading. Absence is bounded to this pin and inspected semantic neighborhood.

Claims & Grounded Evidence

1 Claim

Mechanism #method

For observations x, form p(x)=product_m f_m(x;theta_m)/Z(theta), with Z the sum or integral of the product over x and 0<Z<infinity. Expert factors may be unnormalized; multiplying one by a positive x-independent constant cancels in the final normalization. For parameters local to expert m, the exact log-likelihood derivative is its log-factor derivative at the data minus the expectation of that derivative under the full product model. In the latent-expert construction, f_m(x)=sum_hm f_m(x,h_m), and separate expert hidden variables are conditionally independent given a completely observed x, allowing separate inference when each expert is tractable. This does not imply independence of observed coordinates or independence of experts after some observations are marginalized. Restricted Boltzmann machines instantiate the family with one expert per hidden unit, whereas product HMMs use complete HMM sequence distributions as factors.

supports Source: Training Products of Experts by Minimizing Contrastive Divergence · Locator: Section 2, Equations (2.1)–(2.2), printed pp.1772–1774 / PDF pp.2–4; Section 7, pp.1780–1782 / PDF pp.10–12; Sections 12.1 and 12.3 including footnote 9, pp.1794–1795 / PDF pp.24–25
supports Source: Products of Hidden Markov Models: It Takes N>1 to Tango · Locator: Sections 1–2 and Figure 1, PDF pp.1–3; Section 3, Equations (3)–(9) and emission-tempered AIS construction, pp.3–4

⚠️ Curated Limitations & Negative Results

The normalizer must be finite and positive. Continuous expert densities being individually normalized does not alone guarantee that their product is integrable; nonoverlapping supports can also make normalization impossible.
Tractable inference within each expert does not make the product normalizer, model sampling or likelihood learning automatically tractable. Exact likelihood derivatives involve the full product distribution; short-chain CD is a distinct approximate learning procedure.
Experts need not use disjoint input dimensions, be individually well fitted, or produce a sharper or better calibrated distribution in every setting. A product of Gaussian factors remains Gaussian when well defined, so Gaussian factors alone do not represent arbitrary multimodality.
Conditional independence is between separate latent expert states given all modeled observations. Missing observations can couple the experts. The original architecture is not defined by a sparse router or an arithmetic average of expert predictions.
Unit exponent products and geometric opinion averages are not interchangeable: multiplying duplicate factors can sharpen a distribution, while identical distributions pooled with exponent weights summing to one remain unchanged. No empirical advantage or author implementation is certified.

Artificial Intelligence Ontology (AIO) Grounding

Mapped AIO Classes

AIO Target Class	Mapping Relation	Mapping Rationale
`AIO:RestrictedBoltzmannMachine` Restricted Boltzmann Machine	narrow_match	The local product-model family is broader than the AIO RBM target: the primary source explicitly derives RBMs as a product with one expert per hidden unit, while other expert families include HMMs. Narrow match means the target is narrower; this is a local scope assessment, not a new upstream axiom. FactorizedLLM’s related synonym alone does not establish equivalent generic scope.

Evidence Sources & Primary Literature

2 Sources

Source ID	Publication / Document	Type / Access	Reference & Checksum
`method-source`	Training Products of Experts by Minimizing Contrastive Divergence Geoffrey E. Hinton	primary paper full_text	ID: doi:10.1162/089976602760128018 sha256:a1190ee66e69…
`sequence-expert-context`	Products of Hidden Markov Models: It Takes N>1 to Tango Graham W. Taylor, Geoffrey E. Hinton	primary paper full_text	ID: arxiv:1205.2614 sha256:e62e3b6a1876…

Curation Provenance

Curator: Codex

Method: ai_assisted

Model: GPT-6

Created: 2026-10-11T19:31:29Z

Updated: 2026-10-11T19:31:29Z

Source File: data/curated/ProductOfExperts.yaml