RGDMult Robust Descent Using Smoothed Multiplicative Noise
Algorithms AIMech:RGDMultRobustDescentUsingSmoothedMultiplicativeNoise DRAFT
Aliases: RGDMult, Robust Gradient Descent with Smoothed Multiplicative Noise, Smoothed Multiplicative Noise Robust Descent
A robust first-order stochastic optimization algorithm that minimizes population risk under heavy-tailed data distributions by perturbing model parameters with multiple independent draws of unit-mean multiplicative noise per outer iteration, estimating perturbed gradients on mini-batches via a coordinate-wise Catoni-type M-estimator, and aggregating the inner perturbed gradient estimates with a second Catoni M-estimator to bound excess multiplicative-noise variance by 1/m.
Identity Notes & Concept Scope
No AIO class represents RGDMult or smoothed multiplicative-noise Catoni robust gradient descent; ArtificialNeuralNetwork (AIO:ArtificialNeuralNetwork) is the closest broad match covering neural and statistical learning models trained with multiplicative-noise descent.
Claims & Grounded Evidence
4 Claims
Mechanism
#catoni-coordinatewise-m-estimator-gradient-concentration
RGDMult estimates each coordinate of the gradient by solving a Catoni-type M-estimation root equation with a bounded soft-truncation influence function psi, achieving sub-Gaussian concentration around the true population gradient under only finite variance.
Mechanism
#smoothed-multiplicative-noise-variance-shrinking
Averaging gradient evaluations across m independent draws of unit-mean multiplicative parameter noise per iteration preserves the expected multiplicatively perturbed objective while shrinking the excess multiplicative-noise variance term by a factor of 1/m.
Application
#heavy-tailed-statistical-risk-minimization-application
RGDMult is applied to empirical and population risk minimization under heavy-tailed data and gradient distributions where standard empirical-mean gradient descent and unsmoothed Bernoulli dropout suffer from high variance.
Limitation
#coordinatewise-catoni-dimension-scaling-limitation
Because the Catoni M-estimator is applied coordinate-wise across d parameters and union-bounded over coordinates, the high-probability gradient estimation error bound scales with sqrt(d log(d/delta)/n) and requires solving d scalar root-finding problems per step.
⚠️ Curated Limitations & Negative Results
- Because the Catoni M-estimator is applied coordinate-wise across d parameters and union-bounded over coordinates, the high-probability gradient estimation error bound scales with sqrt(d log(d/delta)/n) and requires solving d scalar root-finding problems per step.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:ArtificialNeuralNetwork
Artificial Neural Network
|
broad_match | RGDMult is a robust first-order optimization algorithm using smoothed multiplicative noise and Catoni M-estimation applicable to training statistical models and artificial neural networks. |
Evidence Sources & Primary Literature
3 Sources| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
r824-primary-pdf
|
Matthew J. Holland
|
primary paper
full_text
|
ID: https://arxiv.org/abs/1810.06207
sha256:668def3e3c18…
|
r824-arxiv-abs
|
Matthew J. Holland
|
primary paper
abstract_only
|
ID: https://arxiv.org/abs/1810.06207
sha256:ce29cd9c5c55…
|
r824-metadata-cr
|
Matthew J. Holland
|
primary paper
abstract_only
|
ID: https://arxiv.org/abs/1810.06207
sha256:7b674594f46a…
|
Ontology & Grounding Topology
Open in interactive network graph →Curation Provenance
Curator: Antigravity
Method: ai_assisted
Model: Gemini
Created: 2026-10-19T15:07:00Z
Updated: 2026-10-19T15:07:00Z
Source File:
data/curated/RGDMultRobustDescentUsingSmoothedMultiplicativeNoise.yaml
· Schema:
curated_concept.yaml
Explore across X-Mech
Related computational & biological Mechs:
Embedding Map
Network Graph
ProteinTraitsMech (ESM-2 pLM)
DUFMech (Domain pLM)
TraitMech
CultureMech
All 14 X-Mech Repositories