Home / Curated / Algorithms / TTQ (Trained Ternary Quantization)

TTQ (Trained Ternary Quantization)

Algorithms AIMech:TTQTrainedTernaryQuantization DRAFT

Aliases: TTQ, Trained Ternary Quantization

TTQ (Trained Ternary Quantization) is a neural network weight quantization method that compresses full-precision weights into 2-bit asymmetric ternary values {-W_l^n, 0, +W_l^p} per layer l by normalizing latent full-precision weights w_tilde_l into [-1, 1], thresholding by Delta_l = t * max(|w_tilde_l|) (or a constant sparsity ratio r), and jointly training both the asymmetric positive and negative scaling coefficients (W_l^p, W_l^n) via summed intra-bucket gradients (Eq. 7) and the ternary assignments via asymmetrically scaled latent-weight gradients dL/dw_tilde_l in {W_l^p * dL/dw_l^t, 1 * dL/dw_l^t, W_l^n * dL/dw_l^t} (Eq. 8).

Identity Notes & Concept Scope Distinct from symmetric Ternary Weight Networks (TWN), which tie positive and negative ternary magnitudes to a single analytical scale W_l with heuristic threshold 0.7 * E(|w|), and distinct from uniform k-bit weight quantizers such as DoReFa-Net (AIMech:DoReFaNetTrainingLowBitwidthConvolutionalNeuralNetworksWithLowBitwidthGradients).

Claims & Grounded Evidence

4 Claims

Mechanism #ttq-asymmetric-ternary-quantization-and-threshold-rule

TTQ normalizes latent full-precision weights w_tilde_l to [-1, 1] via maximum-magnitude division (Eq. 5) and assigns ternary weights w_l^t = +W_l^p for w_tilde_l > Delta_l, 0 for |w_tilde_l| <= Delta_l, and -W_l^n for w_tilde_l < -Delta_l (Eq. 6), where the threshold Delta_l = t * max(|w_tilde_l|) (Eq. 9, typically t = 0.05) or is set to maintain a constant layer sparsity ratio r.

supports Source: Trained Ternary Quantization · Locator: Section 3, Equations 5-6 and 9, pp. 3-4
supports Source: Trained Ternary Quantization · Locator: ttq_quantize

Mechanism #ttq-joint-scale-and-latent-assignment-gradient-backpropagation

During backpropagation, TTQ updates the per-layer positive and negative scaling coefficients via dL/dW_l^p = sum_{i in I_l^p} dL/dw_l^t(i) and dL/dW_l^n = sum_{i in I_l^n} dL/dw_l^t(i) (Eq. 7) and updates the latent weights via piecewise scaled gradients dL/dw_tilde_l = W_l^p * dL/dw_l^t for w_tilde_l > Delta_l, 1 * dL/dw_l^t for |w_tilde_l| <= Delta_l, and W_l^n * dL/dw_l^t for w_tilde_l < -Delta_l (Eq. 8).

supports Source: Trained Ternary Quantization · Locator: Section 3, Equations 7-8 and Figure 1, pp. 3-4
supports Source: Trained Ternary Quantization · Locator: ttq_backward and main

Application #ttq-16x-weight-compression-exceeding-or-matching-fp32-on-cifar-and-imagenet

TTQ reduces weight storage from 32 bits to 2 bits (~16x model compression) with 30%-50% zero-weight sparsity while outperforming full-precision baselines on CIFAR-10 (ResNet-20, 32, 44, 56) and AlexNet on ImageNet (57.5% vs. 57.2% top-1 accuracy) and matching ResNet-18 within 3% top-1 accuracy.

supports Source: Trained Ternary Quantization · Locator: Section 4.1-4.2, Tables 1-3 and Figures 2-3, pp. 5-7

Limitation #ttq-full-precision-activations-and-first-last-layer-exclusion

TTQ quantizes only network weights (keeping activations and gradients in 32-bit floating point) and leaves the first convolutional layer and final fully connected layer in full precision across CIFAR-10 and ImageNet experiments.

supports Source: Trained Ternary Quantization · Locator: Section 4.1 (p. 5) and Section 4.2 (AlexNet and ResNet-18, pp. 5-7)

⚠️ Curated Limitations & Negative Results

Quantizes only weights while keeping activations and gradients in full precision.
Leaves the first convolutional layer and the final classification layer in full precision to preserve accuracy.

Artificial Intelligence Ontology (AIO) Grounding

Mapped AIO Classes

AIO Target Class	Mapping Relation	Mapping Rationale
`AIO:ArtificialNeuralNetwork` Artificial Neural Network	broad_match	TTQ is a ternary weight quantization method for compressing deep neural networks to 2-bit weight representations.
`AIO:SparseLearning` Sparse Learning	related_match	TTQ induces controlled zero-weight sparsity via layer threshold Delta_l while learning asymmetric non-zero ternary scales.

Evidence Sources & Primary Literature

3 Sources

Source ID	Publication / Document	Type / Access	Reference & Checksum
`r823-primary-pdf`	Trained Ternary Quantization Chenzhuo Zhu, Song Han, Huizi Mao, William J. Dally	primary paper full_text	ID: https://arxiv.org/bibtex/1612.01064 sha256:05b50804a456…
`r823-arxiv-abs`	Trained Ternary Quantization (arXiv abstract) Chenzhuo Zhu, Song Han, Huizi Mao, William J. Dally	primary paper abstract_only	ID: https://arxiv.org/abs/1612.01064 sha256:f1c7c8c0dde9…
`r823-metadata-cr`	Trained Ternary Quantization (Crossref metadata) Chenzhuo Zhu, Song Han, Huizi Mao, William J. Dally	primary paper abstract_only	ID: https://arxiv.org/bibtex/1612.01064 sha256:6c0f423562fd…

Ontology & Grounding Topology

Open in interactive network graph →

Curation Provenance

Curator: Antigravity

Method: ai_assisted

Model: Gemini

Created: 2026-10-19T15:06:00Z

Updated: 2026-10-19T15:06:00Z

Source File: data/curated/TTQTrainedTernaryQuantization.yaml · Schema: curated_concept.yaml

Explore across X-Mech

Related computational & biological Mechs: Embedding Map Network Graph ProteinTraitsMech (ESM-2 pLM) DUFMech (Domain pLM) TraitMech CultureMech All 14 X-Mech Repositories