AIMech AI

Hierarchical mixture of experts

Model Families AIMech:HierarchicalMixtureOfExperts DRAFT
Aliases: HME
A conditional mixture-model family that organizes output experts as leaves of a tree and forms each leaf’s mixture weight by multiplying input-dependent probabilistic gates along its root-to-leaf path.
Identity Notes & Concept Scope The inspected Jordan–Jacobs construction uses generalized-linear experts and softmax gates with shared external input, with a finite tree defining nested soft partitions. It is static conditional modeling, distinct from temporal hidden Markov decision trees, hard classification trees, hierarchical clustering and language-specific sparse MoE routing. A mixture density network shares conditional-mixture semantics but need not impose this tree factorization or generalized-linear leaf structure. Flattening a fixed tree gives leaf weights without making every flat gating parameterization the same hierarchical model. EM, IRLS and the online surrogate are fitting procedures, not the model identity. No normalized identity match was found across all 442 pinned AIO labels/scoped synonyms and 382 preceding native IDs/names/aliases for the four researched candidates. Both full-checkout fixed-string searches included ignored and hidden files, with 45 and 45 matching lines. All 31 unique matched contents were read: prior specialized inference records, source/review context and pending leads, not exact records for the three new identities. Expanded titles, source IDs, plural forms and complete/partial-factorizability terminology were included. All 68 semantic label/synonym hits and 23 exact fully annotated AIO CLASS nodes with outgoing edges were read. Their generated YAMLs and seven native neighbors were read or byte-verified unchanged after prior complete reading, totaling 30 contextual records. The broader phrase structured variational inference is searched but not asserted as an unconditional alias of structured mean field. Boltzmann trees remains an unresolved full-text lead. Absence is bounded to the pinned vocabulary and inspected neighborhood.

Claims & Grounded Evidence

1 Claim
Mechanism #method
For input x and leaf l, define w_l(x) as the product of normalized child-gate probabilities along the path to l. Then p(y|x)=sum_l w_l(x) p_l(y|x). In the two-level presentation, p(y|x)=sum_i g_i(x) sum_j g_(j|i)(x) p_ij(y|x), with softmax gates and generalized-linear conditional expert densities. The conditional output mean is the corresponding weighted sum of expert means. Given an observed y, leaf responsibilities are proportional to w_l(x) p_l(y|x); they condition on both x and y, unlike input-only gates. Conditional EM alternates these responsibilities with weighted expert fits and gate fits to soft multinomial targets; a lower gate’s objective is weighted by the posterior probability of reaching its parent. The paper uses IRLS inside those maximum-likelihood subproblems. Its separate Algorithm 2 instead regresses log responsibilities by least squares, and its online variant uses discounted recursive weighted least squares. Those surrogate procedures are approximations to the specified fitting approach.

⚠️ Curated Limitations & Negative Results

Artificial Intelligence Ontology (AIO) Grounding

Mapped AIO Classes
AIO Target Class Mapping Relation Mapping Rationale
AIO:BayesianNetwork
Bayesian Network
broad_match For a fixed input, latent decisions down the finite tree and a leaf-conditioned output define a normalized directed acyclic probabilistic factorization. The pinned Bayesian-network family is broader than this conditional hierarchical mixture.
AIO:DecisionTree
Decision Tree
related_match The model uses a tree of probabilistic decisions, but the pinned target is classification-scoped. The inspected model also performs conditional regression and sums over paths, so this is a structural association rather than an unqualified model-to-classification subclass.

Evidence Sources & Primary Literature

1 Source
Source ID Publication / Document Type / Access Reference & Checksum
method-source
Michael I. Jordan, Robert A. Jacobs
primary paper
full_text
ID: doi:10.1162/neco.1994.6.2.181
sha256:31f0365922df…

Curation Provenance

Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-11T20:15:57Z
Updated: 2026-10-11T20:15:57Z
Source File: data/curated/HierarchicalMixtureOfExperts.yaml