Home / Curated / Algorithms / SAWB Statistics-Aware Weight Binning Quantization

SAWB Statistics-Aware Weight Binning Quantization

Algorithms AIMech:SAWBStatisticsAwareWeightBinningQuantization DRAFT

Aliases: SAWB, Statistics-Aware Weight Binning, SAWB Weight Quantizer

A low-bitwidth neural network weight quantization algorithm that determines the symmetric clipping threshold alpha_w* minimizing weight quantization mean-squared error in closed form as a linear combination of the empirical second-moment root sqrt(E[w^2]) and first absolute moment E[|w|] using bitwidth-specific regression coefficients calibrated across canonical weight distributions.

Identity Notes & Concept Scope No AIO class represents Statistics-Aware Weight Binning (SAWB); ArtificialNeuralNetwork (AIO:ArtificialNeuralNetwork) is the closest broad match covering neural network weight quantization.

Claims & Grounded Evidence

4 Claims

Mechanism #two-moment-linear-regression-optimal-clipping-scale

SAWB computes the optimal symmetric weight clipping threshold alpha_w* = c_1 sqrt(E[w^2]) - c_2 E[|w|] using bitwidth-specific constants (c_1, c_2) calibrated across Gaussian, uniform, Laplacian, gamma, Weibull, and logistic distributions, avoiding per-iteration grid search.

supports Source: PACT: Parameterized Clipping Activation for Quantized Neural Networks · Locator: Section 2.2, Equations 4-5, Figure 3, and Table 2, pp. 3-4

Mechanism #outlier-resilient-uniform-weight-binning

By clipping weights outside [-alpha_w*, +alpha_w*] before uniform 2^k-level binning and rescaling to [-alpha_w*, +alpha_w*], SAWB prevents extreme weight outliers from expanding the quantization step size across the bulk of the weight distribution.

supports Source: PACT: Parameterized Clipping Activation for Quantized Neural Networks · Locator: Section 2.2, Equation 4 and Figure 3(a), pp. 3-4

Application #ultra-low-bitwidth-cnn-and-rnn-weight-quantization-application

SAWB is combined with PACT activation clipping to quantize convolutional and recurrent neural network weights down to 2-bit and 4-bit precision with full-precision accuracy on ImageNet and Penn Treebank.

supports Source: PACT: Parameterized Clipping Activation for Quantized Neural Networks · Locator: Section 3, Figures 4-5 and Table 1, pp. 5-8

Limitation #symmetric-zero-mean-distribution-assumption-limitation

Because the SAWB regression coefficients (c_1, c_2) are calibrated on zero-mean symmetric distributions, strongly skewed or multimodal weight distributions can deviate from the predicted MSE-optimal clipping threshold.

supports Source: PACT: Parameterized Clipping Activation for Quantized Neural Networks · Locator: Section 2.2, Figure 3(b) and Table 2, p. 4

⚠️ Curated Limitations & Negative Results

Because the SAWB regression coefficients (c_1, c_2) are calibrated on zero-mean symmetric distributions, strongly skewed or multimodal weight distributions can deviate from the predicted MSE-optimal clipping threshold.

Artificial Intelligence Ontology (AIO) Grounding

Mapped AIO Classes

AIO Target Class	Mapping Relation	Mapping Rationale
`AIO:ArtificialNeuralNetwork` Artificial Neural Network	broad_match	SAWB is a weight quantization algorithm used during low-bitwidth quantization-aware training of deep neural networks.

Evidence Sources & Primary Literature

3 Sources

Source ID	Publication / Document	Type / Access	Reference & Checksum
`r828-primary-pdf`	PACT: Parameterized Clipping Activation for Quantized Neural Networks Jungwook Choi, Zhuo Wang, Swagath Venkataramani, Pierce I-Jen Chuang, Vijayalakshmi Srinivasan, Kailash Gopalakrishnan	primary paper full_text	ID: https://arxiv.org/abs/1805.06085 sha256:bd471647526e…
`r828-arxiv-abs`	PACT: Parameterized Clipping Activation for Quantized Neural Networks (arXiv abstract) Jungwook Choi, Zhuo Wang, Swagath Venkataramani, Pierce I-Jen Chuang, Vijayalakshmi Srinivasan, Kailash Gopalakrishnan	primary paper abstract_only	ID: https://arxiv.org/abs/1805.06085 sha256:c38aa80e1e09…
`r828-metadata-cr`	PACT: Parameterized Clipping Activation for Quantized Neural Networks (Crossref metadata) Jungwook Choi, Zhuo Wang, Swagath Venkataramani, Pierce I-Jen Chuang, Vijayalakshmi Srinivasan, Kailash Gopalakrishnan	primary paper abstract_only	ID: https://arxiv.org/abs/1805.06085 sha256:9638e79904ba…

Ontology & Grounding Topology

Open in interactive network graph →

Curation Provenance

Curator: Antigravity

Method: ai_assisted

Model: Gemini

Created: 2026-10-19T15:11:00Z

Updated: 2026-10-19T15:11:00Z

Source File: data/curated/SAWBStatisticsAwareWeightBinningQuantization.yaml · Schema: curated_concept.yaml

Explore across X-Mech

Related computational & biological Mechs: Embedding Map Network Graph ProteinTraitsMech (ESM-2 pLM) DUFMech (Domain pLM) TraitMech CultureMech All 14 X-Mech Repositories