Scaling & Shifting Your Features
Algorithms AIMech:SSFScaleAndShiftFeatures DRAFT
Aliases: SSF, SSF-ADA, Scaling and Shifting Your Features, Scale and Shift Features Fine-Tuning
A parameter-efficient fine-tuning algorithm (SSF) for pre-trained deep neural networks (including Vision Transformers, hierarchical Transformers, convolutional networks, and MLPs) that freezes all pre-trained backbone weights and inserts lightweight linear feature modulation modules (SSF-ADA) after backbone operations (such as multi-head self-attention, MLP, convolution, and normalization layers) to alleviate upstream-to-downstream distribution mismatch. Given an intermediate feature tensor x in R^{(N^2+1) x d}, each SSF-ADA module applies an input-independent, channel-wise Hadamard scale vector gamma in R^d (initialized from a normal distribution with mean 1) and additive shift vector beta in R^d (initialized with mean 0) via y = gamma (*) x + beta. Because SSF-ADA is a purely linear affine transformation, at inference time gamma and beta are algebraically absorbed into the frozen preceding linear or convolutional weights w and biases b via structural re-parameterization y = (gamma (*) w) * t + (gamma (*) b + beta), yielding zero extra inference parameters and zero extra inference FLOPs.
Identity Notes & Concept Scope
Distinct from Batch Normalization (AIO:BatchNormalizationLayer), Layer Normalization, FiLM, and AdaIN: SSF-ADA modulates outputs of general backbone operations (MSA, MLP, and normalization layers) rather than only normalization layers, and uses task-level input-independent parameters (gamma, beta) so they can be folded into preceding linear weights at inference time without conditional generator networks or nonlinear activations. Distinct from BitFit (AIMech:BitFit), which only tunes existing bias terms without multiplicative channel scaling or post-operation residual shifts, and from Adapter / VPT methods that retain unmergeable modules or extra tokens at inference time.
Claims & Grounded Evidence
3 Claims
Mechanism
#ssf-reparameterization-zero-inference-overhead
During fine-tuning, SSF inserts SSF-ADA modules after K backbone operations to compute y = gamma (*) x + beta with input-independent scale gamma in R^d and shift beta in R^d (initialized with normal distributions centered at 1 and 0, respectively); during inference, when preceded by a linear or convolutional operation x = w * t + b, SSF re-parameterizes the weights as w' = gamma (*) w and b' = gamma (*) b + beta (Equation 5), incurring m*L*d trainable parameters during training and exactly zero extra parameters and zero extra FLOPs at inference time (Table 2).
Application
#ssf-fgvc-vtab-and-ood-generalization
On ViT-B/16 pre-trained on ImageNet-21K, SSF achieves 90.72% mean Top-1 accuracy across five FGVC datasets (0.39M trainable parameters vs. 85.98M for full fine-tuning at 88.54% and 0.85M for VPT-Deep at 89.11%) and 73.10% mean Top-1 accuracy across the 19-task VTAB-1k benchmark (0.24M trainable parameters vs. 65.57% for full fine-tuning and 69.43% for VPT-Deep), while improving out-of-distribution robustness on ImageNet-A (45.88% vs. 34.49%), ImageNet-R (56.77% vs. 51.29%), and ImageNet-C (41.47 mCE vs. 46.47 mCE).
Limitation
#ssf-dense-prediction-and-independent-task-limitation
On dense prediction tasks such as COCO object detection and instance segmentation with Mask R-CNN (34.9 AP^b / 33.5 AP^m vs. 43.7 AP^b / 39.8 AP^m for full fine-tuning) and ADE20K semantic segmentation with UPerNet (38.9 mIoU vs. 44.5 mIoU for full fine-tuning), modulating only linear scale and shift features leaves a substantial accuracy gap relative to full backbone fine-tuning, and SSF treats each downstream task independently without modeling cross-task correlations.
⚠️ Curated Limitations & Negative Results
- Exhibits a substantial performance gap relative to full fine-tuning on dense prediction benchmarks such as COCO object detection/instance segmentation and ADE20K semantic segmentation (Appendix B, Table 9).
- Zero-mean random initialization of the scale vector gamma disrupts pre-trained feature propagation and can fail to converge, requiring unit-mean normal initialization around gamma = 1 and beta = 0 (Section 4.3, Table 6c).
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:ArtificialNeuralNetwork
Artificial Neural Network
|
related_match | SSF adapts pre-trained artificial neural network backbones via mergeable channel-wise affine feature modulation. |
AIO:BatchNormalizationLayer
BatchNormalization Layer
|
related_match | Like batch normalization folding, SSF uses channel-wise scale and shift parameters that fold into preceding linear or convolutional weights at inference time. |
Evidence Sources & Primary Literature
3 Sources| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
r692-primary-pdf
|
Dongze Lian, Daquan Zhou, Jiashi Feng, Xinchao Wang
|
primary paper
full_text
|
ID: https://dblp.org/search/publ/api?q=Scaling+Shifting+Your+Features+Lian&format=xml
sha256:43899f3e99b5…
|
r692-arxiv-abs
|
Dongze Lian, Daquan Zhou, Jiashi Feng, Xinchao Wang
|
primary paper
abstract_only
|
ID: https://arxiv.org/abs/2210.08823
sha256:1e0a859cdfeb…
|
r692-metadata-cr
|
Dongze Lian, Daquan Zhou, Jiashi Feng, Xinchao Wang
|
primary paper
abstract_only
|
ID: https://dblp.org/search/publ/api?q=Scaling+Shifting+Your+Features+Lian&format=xml
sha256:d2d551f75ad7…
|
Ontology & Grounding Topology
Open in interactive network graph →Curation Provenance
Curator: Antigravity
Method: ai_assisted
Model: Gemini
Created: 2026-10-19T12:55:00Z
Updated: 2026-10-19T12:55:00Z
Source File:
data/curated/SSFScaleAndShiftFeatures.yaml
· Schema:
curated_concept.yaml
Explore across X-Mech
Related computational & biological Mechs:
Embedding Map
Network Graph
ProteinTraitsMech (ESM-2 pLM)
DUFMech (Domain pLM)
TraitMech
CultureMech
All 14 X-Mech Repositories