Mathematical Methods

aka Diff-Gauss pooling

A parameterized neural pooling unit that averages a group of transformed activations using normalized Gaussian-kernel weights determined by…

1 source

A matrix-valued covariance-update component from the original 2006 active CMA construction that combines retained search covariance and an e…

1 source

aka Barber-Agakov bound, BA bound

A variational lower bound on mutual information formed by adding input entropy to the expected log probability assigned by a normalized cond…

2 sources

An acceptance-probability rule for reversible Monte Carlo proposals that uses R/(1+R), where R is the target-density ratio times the reverse…

1 source

A signal-representation optimization principle that minimizes the L1 norm of real synthesis coefficients subject to exact reconstruction of…

1 source

A beam-dependent sequence rescoring rule that adds per-step rewards derived from the beam’s mean negative log token probability, accumulatin…

1 source

aka Bounded word reward

A sequence scoring rule that adds a fixed nonnegative reward per generated position up to a finite length cap, giving score S(y)+r·min(l,|y|…

2 sources

A sequence rescoring method that adds a fixed word reward only up to an input-specific length predicted from encoder states, replacing a cor…

1 source

aka BP-Norm

A sequence rescoring rule that adds a predicted-length brevity penalty in log space to average token log probability: S(x,y)/|y| + min(1−Lpr…

2 sources

A covariance-valued update component introduced in the 2003 CMA-ES construction that forms an equal-weight second moment of selected mutatio…

1 source

A dispatch-prefix-conditioned mathematical rule that returns, as a function of a contemplated search start time t, the modeled probability o…

1 source

An eligibility-trace update that propagates prior feature credit and adds a current-feature contribution adjusted by the prior trace’s proje…

3 sources

aka Elastic-net regression

Linear regression regularized by a combination of L1 and L2 penalties.

1 source

An acquisition criterion for deterministic scalar minimization that scores a candidate evaluation by the predictive expected positive reduct…

1 source

aka Naive mean-field inference, Fully factorized variational inference

A variational inference method that restricts its approximating distribution to a product of single-variable factors and adjusts those facto…

1 source

aka GAE

An advantage-function estimator for policy-gradient learning formed from an exponentially weighted sum of temporal-difference residuals.

1 source

aka Gumbel-Softmax, Concrete distribution…

A continuous relaxation of a finite categorical random variable obtained by applying softmax to independent Gumbel-perturbed log weights div…

2 sources

aka Gumbel top-k sampling

A categorical sampling construction that adds independent standard Gumbel noise to each category’s log weight and returns the indices of the…

2 sources

aka HiPPO (polynomial projection)

A family of time-dependent operators that represents a signal's past by coefficients of its optimal finite-dimensional polynomial approximat…

1 source

aka InfoNCE loss

A contrastive log-softmax objective that identifies a joint positive among sampled candidates and, under independent marginal-negative sampl…

2 sources

aka IG

A feature-attribution method that integrates a prediction's input gradients along the straight path from a chosen baseline to the input.

1 source

aka I-alpha bound, Nonlinearly interpolated mutual information lower bound

A family of multi-sample mutual-information lower bounds that interpolates the critic’s empirical partition estimate with a positive variati…

3 sources

aka LRP

A prediction-decomposition framework that redistributes relevance backward through a classifier's computational layers toward its input feat…

1 source

aka Lp unit, Learned-norm pooling

A parameterized neural unit that pools a group of learned linear projections by the power mean of their absolute deviations from learned cen…

1 source

aka Leave-one-out variational information upper bound

An expectation-level upper bound on mutual information that compares a known conditional density for each paired sample with a mixture of co…

1 source

aka Lifted structured loss

A metric-learning objective that penalizes a positive pair’s Euclidean distance against negative distances incident to either endpoint, usin…

2 sources

aka Log opinion pool, Logarithmic pooling

A probability-aggregation method that forms a normalized weighted geometric mean of expert distributions over the same outcomes, using nonne…

2 sources

aka Maxout

A parameterized neural unit that outputs the maximum of a finite group of learned affine functions of its input.

2 sources

aka MEX (SimNets)

A scalar aggregation operator that returns log(mean(exp(xi times input)))/xi for a nonzero real parameter xi, with its continuous zero-param…

1 source

aka N-pair-mc loss

A supervised metric-learning objective that averages a joint positive-versus-multiple-negative log-softmax loss across a batch of positive p…

1 source

A metric-dependent differential method that maps the coordinate derivatives of a differentiable objective to a tangent vector by the inverse…

3 sources

aka NWJ bound, MINE-f objective…

A variational lower bound on Kullback–Leibler divergence that subtracts a product-distribution exponential critic moment from a joint-distri…

2 sources

aka Random walk centrality of Noh and Rieger

A node-scoring method for an undirected random walk that divides each node’s stationary visitation probability by its integrated excess retu…

1 source

A stochastic activation that adds zero-mean noise to a preactivation and then applies the positive-part rectifier, producing h=max(0,a+z).

1 source

aka NMF

A matrix-decomposition method that represents or approximates a nonnegative data matrix as the product of two entrywise nonnegative factor m…

1 source

aka NT-Xent, NT-Xent loss

A contrastive cross-entropy objective that uses cosine-normalized, temperature-scaled similarities to identify a paired positive representat…

2 sources

aka N-pair-ovo loss

A supervised metric-learning objective that sums separate positive-versus-negative softplus comparisons for each anchor in a batch of positi…

1 source

aka PLS regression, PLSR

Regression using latent components chosen to capture predictor-response covariance.

1 source

aka Hastie–Stuetzle Principal Curves

A nonlinear statistical summary defined by a smooth curve that is self-consistent for a data distribution: almost every point on the curve e…

1 source

An acquisition criterion for scalar optimization that scores a candidate by the predictive probability of meeting a specified improvement th…

2 sources

aka Random-only abstraction sampling, simpleRand abstraction…

A randomized abstraction rule for graphical-model sampling that uniformly permutes the current search nodes and divides that order into grou…

1 source

An eligibility-trace update that replaces a visited state’s eligibility by one while geometrically attenuating other state traces, preservin…

1 source

aka HiPPO-LegS

The HiPPO projection operator using the uniform measure over the full elapsed interval from zero to the current time, with scaled-Legendre c…

1 source

aka SHAP

An additive feature-attribution framework assigning Shapley values to a prediction through a specified function for evaluating feature subse…

1 source

aka STS unit

A stochastic activation that sets p=sigmoid(a), samples a Bernoulli gate with success probability sqrt(p), and outputs the gate multiplied b…

1 source

aka ST estimator, Identity straight-through estimator

A surrogate-gradient rule that keeps a hard threshold or sampled binary output in the forward computation while propagating its downstream d…

1 source

aka ST Gumbel-Softmax, ST Gumbel estimator

A surrogate-gradient estimator that uses a hard one-hot argmax of a Gumbel-Softmax sample in the forward computation and substitutes the cor…

1 source

aka Structured mean-field approximation

A variational inference method that approximates a target distribution with a tractable family retaining selected dependencies, choosing its…

1 source

aka TUBA, TUBA bound

A tractable unnormalized Barber–Agakov lower-bound parameterization that uses a real critic and a positive output-dependent normalizer to re…

2 sources

aka Value-based abstraction sampling

A family of graphical-model sampling abstractions that assigns a positive scalar value to each current search node and partitions nodes in v…

1 source

aka VRM

A risk-minimization principle that chooses a predictor from a specified function class by minimizing expected loss under an estimated distri…

1 source

aka Zhou neighbor dissimilarity index

A pairwise graph dissimilarity that compares the mean first-passage-time profiles of two adjacent vertices by taking the Euclidean norm of t…

2 sources