Convpass
Architectures AIMech:ConvPass DRAFT
Aliases: Convolutional Bypasses, Convpass_attn, Convpass_CLIP, Convolutional Bypass Vision Transformer Adapter
A vision-oriented parameter-efficient transfer learning architecture (Convolutional Bypasses) for Vision Transformers that inserts trainable ResNet-style convolutional bottleneck modules in parallel with the frozen Multi-Head Self-Attention (MHSA) and Multi-Layer Perceptron (MLP) blocks: X' <- X + MHSA/MLP(LN(X)) + s * Convpass(LN(X)). Each Convpass module consists of a 1x1 channel-reducing convolution (d -> h, where h << d, followed by GELU), a 2D spatial reconstruction step that reshapes the flattened 1D patch sequence back to a 2D grid while treating the [cls] token as an individual 1x1 image for a 3x3 spatial convolution (h -> h, followed by GELU), and a 1x1 channel-expanding convolution (h -> d), introducing 2*L*((2*h + 1)*d + 9*h^2 + 2*h) trainable parameters across L layers to inject 2D spatial locality and token-mixing inductive bias.
Identity Notes & Concept Scope
Distinct from language-oriented channel-wise MLP adapters such as Bottleneck Adapters (AIMech:BottleneckAdapter) and AdaptFormer (AIMech:AdaptFormer), whose 1x1 linear projections cannot mix information across spatial neighbors without relying on the frozen MHSA block, and from LoRA (AIMech:LowRankAdaptation) and VPT (AIMech:VisualPromptTuning), which lack 2D neighborhood inductive bias.
Claims & Grounded Evidence
3 Claims
Mechanism
#convpass-2d-spatial-reconstruction-and-parameter-complexity
Each Convpass module restores the 2D spatial grid of image patch tokens and treats the [cls] token as a separate 1x1 image inside a 1x1-down (d -> h) -> GELU -> 3x3-conv (h -> h) -> GELU -> 1x1-up (h -> d) bypass placed parallel to both MHSA and MLP blocks via X' <- X + MHSA/MLP(LN(X)) + s * Convpass(LN(X)), requiring exactly 2*L*((2*h + 1)*d + 9*h^2 + 2*h) trainable parameters (0.33M for ViT-B/16 with L=12, d=768, h=8) and creating trainable unraveled token-mixing paths independent of MHSA.
Application
#convpass-vtab-1k-few-shot-and-3x3-ablation-gains
On the 19-task VTAB-1K benchmark with ViT-B/16, Convpass (0.33M parameters) achieves 76.6% average accuracy (81.7% Natural, 85.3% Specialized, 62.7% Structured), outperforming full fine-tuning (68.9%), VPT (72.0%), Adapter (73.9%), LoRA (74.5%), AdaptFormer (74.7%), NOAH (75.5%), and fully fine-tuned Swin-B (75.0%) and ConvNeXt-B (74.0%); replacing the 3x3 spatial convolutions with 1x1 convolutions drops Structured accuracy from 62.7% to 59.5% (Table 5), and parallel placement outperforms sequential placement (Table 4).
Limitation
#convpass-imagenet-a-resnet-adversarial-bias-limitation
Because Convpass introduces ResNet-style 3x3 convolutional blocks, Convpass_CLIP underperforms CoOp and CoCoOp on the ImageNet-A adversarial out-of-distribution target domain (49.27% vs. 49.71% for CoOp and 50.63% for CoCoOp), which was curated from natural examples misclassified by ResNet architectures, and its relative gain over full fine-tuning diminishes on backbones that already possess built-in spatial inductive bias such as Swin-B (+3.1% vs. +7.7% on ViT-B/16).
⚠️ Curated Limitations & Negative Results
- Inherits ResNet-like convolutional inductive biases that slightly reduce out-of-distribution accuracy on ImageNet-A natural adversarial examples selected against ResNet classifiers (49.27% for Convpass_CLIP vs. 50.63% for CoCoOp, Table 2).
- Requires reshaping the 1D token sequence into a 2D spatial grid before each 3x3 convolution and exhibits smaller relative gains on backbones that already hard-code spatial locality such as Swin Transformer (Section 3.2 & Section 4.4, Table 3).
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:ArtificialNeuralNetwork
Artificial Neural Network
|
related_match | Convpass adapts pre-trained Vision Transformer neural networks using parallel ResNet-style 2D convolutional bottleneck bypasses. |
Evidence Sources & Primary Literature
3 Sources| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
r697-primary-pdf
|
Shibo Jie, Zhi-Hong Deng
|
primary paper
full_text
|
ID: https://dblp.org/search/publ/api?q=Convolutional+Bypasses+Better+Vision+Transformer+Adapters+Jie&format=xml
sha256:6caae432f31c…
|
r697-arxiv-abs
|
Shibo Jie, Zhi-Hong Deng
|
primary paper
abstract_only
|
ID: https://arxiv.org/abs/2207.07039
sha256:dd3a452ea97b…
|
r697-metadata-cr
|
Shibo Jie, Zhi-Hong Deng
|
primary paper
abstract_only
|
ID: https://dblp.org/search/publ/api?q=Convolutional+Bypasses+Better+Vision+Transformer+Adapters+Jie&format=xml
sha256:38f56c13d10f…
|
Ontology & Grounding Topology
Open in interactive network graph →Curation Provenance
Curator: Antigravity
Method: ai_assisted
Model: Gemini
Created: 2026-10-19T13:00:00Z
Updated: 2026-10-19T13:00:00Z
Source File:
data/curated/ConvPass.yaml
· Schema:
curated_concept.yaml
Explore across X-Mech
Related computational & biological Mechs:
Embedding Map
Network Graph
ProteinTraitsMech (ESM-2 pLM)
DUFMech (Domain pLM)
TraitMech
CultureMech
All 14 X-Mech Repositories