Family-of-harmonium
Model Families AIMech:FamilyOfHarmonium DRAFT
Aliases: FoH
A classification model family comprising separately parameterized harmoniums for each class, with binary text features and Gaussian image features coupled to Gaussian latent variables and compared through normalized class-conditional likelihoods.
Identity Notes & Concept Scope
Scoped to the FoH construction in Yang and colleagues' semantic-video manuscript. Each labeled class owns its parameters and latent coordinates; a categorical class selector has a prior distribution. This differs from the shared label-coupled hierarchical harmonium, an unsupervised single harmonium followed by an SVM, and the earlier unbounded Poisson-text dual-wing specialization. The record covers the proper Gaussian parameter domain; published training or normalizer-estimation safeguards are not inferred from the existence of that domain. No normalized identity matched four candidates against all 442 pinned AIO IDs, labels and scoped synonyms or 444 preceding native IDs, names and aliases. Full-checkout searches included ignored and hidden files in both checkouts; five matching lines per checkout came from two unique prior discovery-ledger contents, and all matching excerpts were read. All 205 semantic label/synonym hits were byte-verified against prior readings. Twenty-nine annotated AIO nodes and outgoing edges comprise 28 unchanged prior readings and one newly read multimodal-deep-learning node. Thirty-nine contextual YAMLs comprise 34 unchanged prior readings and five fully read current records. Three candidates are curated; the unbounded-count DWH specialization remains deferred because the inspected joint is not normalizable with nonzero count couplings. Absence is bounded to this pinned snapshot and maintained corpus.
Claims & Grounded Evidence
3 Claims
Mechanism
#class-specific-modalities
For a selected class y, each word-presence variable is Bernoulli with logit alpha_i^y+sum_k W_ik^y h_k. The demonstrated image-feature conditional is Gaussian with identity covariance and mean beta^y+U^y h. Hidden variables given all text/image features and the class are independent unit-Gaussian variables with mean (W^y)^T x+(U^y)^T z. Separate class parameters produce separate latent coordinates; topic positions across classes are not inherently aligned.
Limitation
#proper-gaussian-domain
For the identity-covariance construction, the Gaussian image/hidden precision must be positive definite; equivalently I-U^y(U^y)^T must be positive definite in every class. Binary word variables add bounded-state linear terms and do not repair an unstable Gaussian coupling. Proper-looking conditional normals alone do not establish a proper joint. The record restricts the probability-model definition to this admissible domain and does not certify that the historical fitting procedure enforced it.
Mechanism
#class-normalized-decision
Integrating Gaussian hidden variables gives class-specific visible log mass F_y(x,z)=alpha^y dot x+beta^y dot z-||z||^2/2+||(W^y)^T x+(U^y)^T z||^2/2. Class posterior scores require pi_y exp(F_y)/Z_y. Assuming uniform class priors removes pi_y but not the separate class normalizers Z_y. In a one-word, one-image-feature example, unnormalized scores favor the second class while normalized densities favor the first. Each class can be fitted from its labeled examples; adding a class does not require sharing its latent coordinates with existing classes.
⚠️ Curated Limitations & Negative Results
- The manuscript acknowledges hidden partition functions but does not give a complete operational recipe for estimating and comparing every class normalizer. Raw energy comparison is not certified as probability comparison; no omission in uninspected author code is alleged.
- The Appendix adds quantities as log-partitions but displays integrals without the required logarithms. The compatible main joint and standard exponential-family algebra determine this record, rather than that inconsistent display. Gaussian color features are a modeling approximation to physical histogram observations.
- Fully factorized mean field and short-chain CD approximate learning statistics. The manuscript's broad CD-to-maximum-likelihood convergence statement is not accepted as a general guarantee. Sampling schedules and stability safeguards remain implementation details not fully specified by the paper.
- Main model comparisons use the five largest TRECVID categories, 1078 shots, an equal training/test split and selected latent sizes. The broader collection has 15 classes and 2468 shots. Selected topic examples and classification curves do not prove universal semantic accuracy, causality or generalization. No historical implementation or benchmark was run.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:Model
Model
|
broad_match | The source-defined probabilistic model family is narrower than the pinned Model class. |
AIO:RestrictedBoltzmannMachine
Restricted Boltzmann Machine
|
related_match | Bipartite conditional-independence structure is shared with harmoniums, but the pinned RBM class does not identify this exact observation, hidden-support and parameter-sharing construction. |
AIO:Classification
Classification
|
related_match | The model supplies classification from labeled examples; the pinned target is a supervised task, not an exact probability-model identity. |
AIO:MultimodalLearning
Multimodal Learning
|
related_match | Text and image features are modeled jointly. The pinned target denotes a task; its deep-learning comment is not used to assert that this classical harmonium is a deep neural network. |
Evidence Sources & Primary Literature
1 Source| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
method-source
|
Jun Yang, Yan Liu, Eric P. Xing, Alexander G. Hauptmann
|
primary paper
full_text
|
sha256:23177fca49a4…
|
Curation Provenance
Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-12T01:13:01Z
Updated: 2026-10-12T01:16:06Z
Source File:
data/curated/FamilyOfHarmonium.yaml