Hierarchical harmonium
Model Families AIMech:HierarchicalHarmonium DRAFT
Aliases: HH
A labeled harmonium model family coupling binary words, Gaussian image features and binary category indicators to shared Gaussian latent variables, with a proper joint Gaussian precision and classification through posterior label probabilities.
Identity Notes & Concept Scope
Scoped to Yang and colleagues' shared hierarchical harmonium for semantic video. The label layer couples to the same hidden topics as text and images, unlike FoH's independently parameterized class models. Although its diagram has three tiers, the graph is bipartite between the latent variables and the combined text/image/label layer. The source uses one-hot labeled training shots and a single-label decision rule, while its Bernoulli label conditionals define a joint over the full binary label cube; the conclusion explicitly permits multiple active categories. This is not an ontology class hierarchy or an asserted one-hot generative model. No normalized identity matched four candidates against all 442 pinned AIO IDs, labels and scoped synonyms or 444 preceding native IDs, names and aliases. Full-checkout searches included ignored and hidden files in both checkouts; five matching lines per checkout came from two unique prior discovery-ledger contents, and all matching excerpts were read. All 205 semantic label/synonym hits were byte-verified against prior readings. Twenty-nine annotated AIO nodes and outgoing edges comprise 28 unchanged prior readings and one newly read multimodal-deep-learning node. Thirty-nine contextual YAMLs comprise 34 unchanged prior readings and five fully read current records. Three candidates are curated; the unbounded-count DWH specialization remains deferred because the inspected joint is not normalizable with nonzero count couplings. Absence is bounded to this pinned snapshot and maintained corpus. Unlike the maintained conditional-RBM record, which conditions on an externally supplied input and curates binary hidden units, HH specifies a joint over text, images and labels with Gaussian hidden variables.
Claims & Grounded Evidence
3 Claims
Mechanism
#shared-label-coupling
The model shares word/image parameters across classes and adds label biases tau_t and label-topic couplings V_tk. Given h, words and labels have independent Bernoulli logits alpha+Wh and tau+Vh, while image features have Gaussian mean beta+Uh and identity covariance. Given all x,z,y, hidden variables are independent unit Gaussians with mean W^T x+U^T z+V^T y. As in FoH, the unbounded image/hidden joint requires I-UU^T positive definite; finite binary words and labels do not remove this requirement.
Mechanism
#label-support-and-posterior
For observed x,z, write a=W^T x+U^T z. Integrating hidden variables gives p(y|x,z) proportional to exp(tau dot y+||a+V^T y||^2/2) over all binary label vectors. Labels generally become dependent after hidden marginalization; exact summation over 2^T configurations is possible in principle. Restricting support to exactly one active label would define a different model with categorical, rather than independent Bernoulli, h-conditioned labels. The reported single-label classifier chooses the largest posterior marginal P(Y_t=1|x,z); this decision does not enforce one-hot support in the joint.
Mechanism
#missing-label-inference
When labels are unknown, the simple all-observed hidden Gaussian is replaced by a mixture over label configurations. Substituting the posterior mean hidden vector into a logistic label conditional is not generally the exact label marginal. The source explores fully factorized mean field and finite-chain approximations for prediction and learning, and compares CD with a noisy parameter-gradient procedure. These approximations are distinguished from exact finite label enumeration and from an independently verified posterior sampler over model parameters.
⚠️ Curated Limitations & Negative Results
- One-hot training observations, independent Bernoulli label conditionals and a single-label argmax decision are different statements. The binary-cube law can assign positive probability to zero or multiple labels; posterior marginals need not sum to one. The graph's tiers do not establish hierarchical class subsumption.
- Normalizability requires constraints on Gaussian couplings, which the inspected manuscript does not describe enforcing. The Appendix log-partition display is qualified as for FoH. The source's claimed general convergence of CD to maximum likelihood is not adopted.
- The Gibbs description lists label and hidden updates appropriate to clamped-feature prediction, without fully specifying every free-phase update used during learning. The Langevin display perturbs parameters with likelihood-gradient information and short-chain approximations; its target, prior and discretization controls are not sufficient to certify exact Bayesian parameter inference.
- The selected TRECVID experiments, latent-topic visualizations and class-similarity heatmap do not establish causal relationships, uniquely identified topics or universal classification gains. Finite calculations check binary versus one-hot support and a plug-in inference discrepancy; no author code or benchmark was run.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:Model
Model
|
broad_match | The source-defined probabilistic model family is narrower than the pinned Model class. |
AIO:RestrictedBoltzmannMachine
Restricted Boltzmann Machine
|
related_match | Bipartite conditional-independence structure is shared with harmoniums, but the pinned RBM class does not identify this exact observation, hidden-support and parameter-sharing construction. |
AIO:Classification
Classification
|
related_match | The model supplies classification from labeled examples; the pinned target is a supervised task, not an exact probability-model identity. |
AIO:MultimodalLearning
Multimodal Learning
|
related_match | Text and image features are modeled jointly. The pinned target denotes a task; its deep-learning comment is not used to assert that this classical harmonium is a deep neural network. |
Evidence Sources & Primary Literature
1 Source| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
method-source
|
Jun Yang, Yan Liu, Eric P. Xing, Alexander G. Hauptmann
|
primary paper
full_text
|
sha256:23177fca49a4…
|
Curation Provenance
Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-12T01:13:01Z
Updated: 2026-10-12T01:16:06Z
Source File:
data/curated/HierarchicalHarmonium.yaml