FourierFT
Algorithms AIMech:FourierFT DRAFT
Aliases: Discrete Fourier Transform Fine-Tuning, Fourier Fine-Tuning, FourierFT Parameter-Efficient Fine-Tuning
A spectral parameter-efficient fine-tuning algorithm that parameterizes the spatial weight update matrix of a frozen pretrained layer as the real part of the two-dimensional inverse discrete Fourier transform of a sparse frequency-domain matrix populated by a small trainable vector of n spectral coefficients.
Identity Notes & Concept Scope
Distinct from AIMech:LowRankAdaptation (LoRA) and AIMech:VeRA, which parameterize weight updates directly in the spatial domain via low-rank matrix products and therefore require at least O(d_1 + d_2) parameters or high rank to represent complex spatial patterns. FourierFT (Ziqi Gao, Qichao Wang, Aochuan Chen, Zijing Liu, Bingzhe Wu, Liang Chen, and Jia Li, ICML 2024, PMLR v235, arXiv:2405.03003) represents the weight update Delta_W in R^{d_1 x d_2} in the orthogonal 2D Fourier basis. A fixed entry index table E in {0, ..., d_1-1} x {0, ..., d_2-1} of length n << d_1 d_2 is sampled once (uniformly or with a central-frequency radial Gaussian bias), a trainable coefficient vector c in R^n is scattered into a sparse spectral matrix S = ToMatrix(c) in R^{d_1 x d_2} at locations E, and the spatial weight update is reconstructed via the 2D inverse discrete Fourier transform Delta_W = alpha * Re(F^{-1}(ToMatrix(c))). Because every spectral coefficient modulates a global 2D complex sinusoid across all d_1 x d_2 spatial entries, FourierFT decouples parameter count n from layer dimensions (d_1, d_2) and can generate full-rank spatial updates when the sampled frequencies span rows and columns.
Claims & Grounded Evidence
4 Claims
Mechanism
#fourierft-sparse-spectral-to-spatial-idft-parameterization
FourierFT defines a layer's weight update as Delta_W = alpha * Re(F^{-1}(ToMatrix(c))), where c in R^n is a trainable spectral vector scattered onto n pre-selected frequency coordinates (u_k, v_k) of a d_1 x d_2 spectral grid and F^{-1} is the 2D inverse discrete Fourier transform, decoupling the trainable parameter count n from the layer dimensions d_1 and d_2.
Mechanism
#fourierft-direct-basis-evaluation-and-frequency-bias-sampling
Rather than executing a full O(d_1 d_2 log(d_1 d_2)) 2D FFT when n is small, FourierFT evaluates the inverse transform directly over the n active frequencies using precomputed complex exponential basis matrices, and optionally samples frequency coordinates with a radial Gaussian bias toward low- or mid-frequency bands.
Application
#fourierft-application-extreme-compression-nlp-and-vision-peft
Across GLUE natural language understanding, instruction tuning on LLaMA-2, and ViT image classification benchmarks, FourierFT matches or exceeds LoRA accuracy while using up to an order of magnitude fewer trainable parameters (for example, n = 1000 spectral coefficients per layer versus rank-8 LoRA on RoBERTa-large).
Limitation
#fourierft-limitation-dense-delta-w-materialization-during-training
Because the inverse discrete Fourier transform produces a dense spatial matrix Delta_W in R^{d_1 x d_2} rather than a low-rank factored intermediate representation, FourierFT must either materialize Delta_W in memory during each forward/backward pass or compute O(n d_1 d_2) basis projections, incurring higher per-layer FLOPs than rank-1 LoRA when n is large.
⚠️ Curated Limitations & Negative Results
- Reconstructing the spatial weight update from n spectral coefficients produces a dense d_1 x d_2 matrix during training rather than a low-rank intermediate vector projection, increasing per-layer compute if n is large.
- Because the n frequency coordinates are sampled randomly before training and held fixed, task-relevant spectral energy that falls outside the sampled frequency support cannot be represented.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:ArtificialNeuralNetwork
Artificial Neural Network
|
related_match | FourierFT adapts linear layers of deep artificial neural networks by learning sparse spectral coefficients in the 2D discrete Fourier domain. |
AIO:SparseLearning
Sparse Learning
|
related_match | FourierFT learns a sparse set of n frequency-domain coefficients whose inverse 2D discrete Fourier transform reconstructs the spatial weight update. |
Evidence Sources & Primary Literature
3 Sources| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
r663-primary-pdf
|
Ziqi Gao, Qichao Wang, Aochuan Chen, Zijing Liu, Bingzhe Wu, Liang Chen, Jia Li
|
primary paper
full_text
|
ID: https://dblp.org/rec/conf/icml/GaoWC0WCL24.xml
sha256:751a4d4b1e34…
|
r663-arxiv-abs
|
Ziqi Gao, Qichao Wang, Aochuan Chen, Zijing Liu, Bingzhe Wu, Liang Chen, Jia Li
|
primary paper
abstract_only
|
ID: https://arxiv.org/abs/2405.03003
sha256:43b22a73641c…
|
r663-metadata-cr
|
Ziqi Gao, Qichao Wang, Aochuan Chen, Zijing Liu, Bingzhe Wu, Liang Chen, Jia Li
|
primary paper
abstract_only
|
ID: https://dblp.org/rec/conf/icml/GaoWC0WCL24.xml
sha256:47dfe2854075…
|
Ontology & Grounding Topology
Open in interactive network graph →Curation Provenance
Curator: Antigravity
Method: ai_assisted
Model: Gemini
Created: 2026-10-19T12:26:00Z
Updated: 2026-10-19T12:26:00Z
Source File:
data/curated/FourierFT.yaml
· Schema:
curated_concept.yaml
Explore across X-Mech
Related computational & biological Mechs:
Embedding Map
Network Graph
ProteinTraitsMech (ESM-2 pLM)
DUFMech (Domain pLM)
TraitMech
CultureMech
All 14 X-Mech Repositories