CD-PercLoss
Algorithms AIMech:CDPercLoss DRAFT
A conditional RBM training algorithm that contrasts the observed output free energy with that of a prediction selected by input-initialized stochastic search, treating the selected prediction as fixed during differentiation.
Identity Notes & Concept Scope
This is the CD-PercLoss algorithm defined by Mnih, Larochelle and Hinton as a practical approximation to generalized perceptron-loss training. Unlike standard data-started contrastive divergence, its search begins from the conditioning input and chooses the best free-energy state among its search steps. The training algorithm, the generalized perceptron objective and the conditional RBM architecture are distinct identities. The name does not identify the AIO single-layer linear Perceptron architecture. The paper also allows extension to other models with computable energy or free energy and Gibbs sampling; this record states its conditional-RBM procedure. All 442 pinned AIO labels/scoped synonyms and 367 preceding native IDs/names/aliases were compared, with no normalized identity match. Full-checkout fixed-string searches included ignored and hidden files in both worktree and main, returning 37 lines each. All 27 unique matched contents were read: conditional-RBM source references, contrasting method descriptions, pending leads and saved review context, rather than existing candidate identities. All 45 semantic label/synonym hits and 23 exact annotated AIO nodes were inspected, alongside corresponding generated records and three complete native neighbors. Previously read contextual files were checked for byte identity. Absence is bounded to this snapshot and inspected neighborhood.
Claims & Grounded Evidence
1 Claim
Mechanism
#method
For training pair (u,v), initialize each visible value from the logistic input-to-visible prediction sigmoid(b_v+W_uv^T u) when those connections exist; without them, draw independent Bernoulli(0.5) visible values. Holding u fixed, perform k Gibbs updates and retain the k resulting visible configurations. Choose a configuration v_hat with the lowest conditional free energy among those candidates. Update model parameters using the derivative of F(v,u)-F(v_hat,u) with v_hat held constant, contrasting the observed and searched free-energy derivatives. This replaces the generally intractable global minimum in the generalized perceptron loss by a finite stochastic search. The source uses k mean-field updates instead of sampled Gibbs updates for test-time predictions. No partition function enters the frozen-prediction free-energy-difference gradient.
⚠️ Curated Limitations & Negative Results
- Finite search does not certify the global conditional mode. Its approximate free-energy difference can be negative, and its update can be zero when the searched prediction equals the observed target even though another output has lower free energy.
- Holding the prediction fixed is an explicit surrogate-update convention. It does not differentiate the sampling/selection process or produce the exact conditional log-likelihood gradient, and it does not guarantee calibrated conditional probabilities.
- The selected candidate is the lowest-free-energy state among k generated configurations, not necessarily the last chain state. Search initialization uses the conditioning input rather than the observed target; substituting ordinary data-started CD changes the procedure.
- Mean visible initialization and mean-field test updates are deterministic probability-valued computations, distinct from binary equilibrium samples or exact MAP inference. Tie-breaking and any final hard-decision convention need to be specified by an implementation; this record does not infer undocumented author-code behavior.
- The displayed loss has no explicit margin. The paper proposes margin-based extensions as future work; those are not silently part of CD-PercLoss. Results for the inspected denoising settings do not establish universal suitability, global convergence or equivalence to the linear perceptron.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:RestrictedBoltzmannMachine
Restricted Boltzmann Machine
|
related_match | This is a learning and prediction procedure developed for conditional RBM free energies. The AIO target names the model component/family, not this procedure; the association is related rather than an algorithm-is-model subclass assertion. |
Evidence Sources & Primary Literature
1 Source| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
method-source
|
Volodymyr Mnih, Hugo Larochelle, Geoffrey E. Hinton
|
primary paper
full_text
|
ID: arXiv:1202.3748
sha256:e7811677aa0f…
|
Curation Provenance
Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-11T19:05:47Z
Updated: 2026-10-11T19:05:47Z
Source File:
data/curated/CDPercLoss.yaml