Home / Curated / Algorithms / PromptFL: Federated Continuous Prompt Learning for Foundation Models

PromptFL: Federated Continuous Prompt Learning for Foundation Models

Algorithms AIMech:PromptFlFederatedPromptLearningForFoundationModels DRAFT

Aliases: PromptFL, Federated Prompt Training for Foundation Models, CLIP PromptFL

A parameter-efficient federated foundation-model adaptation framework (PromptFL; Guo, Guo, Wang, Tang, and Xu, IEEE Transactions on Mobile Computing 2024) that ships an off-the-shelf pre-trained vision-language foundation model F (such as CLIP with frozen image encoder g(.) and frozen text encoder h(.)) to distributed clients and replaces federated model weight training with collaborative continuous prompt (soft prompt) training. Each client attaches a Prompt Learner consisting of p learnable d-dimensional continuous prompt vectors P in R^{p x d} prepended to each class label embedding K_i (i in {1, ..., k}) to form text features w_i = h(P, K_i) and class probabilities p(y = i | x) = exp(cos[g(x) | h(P, K_i)]) / sum_{j=1}^k exp(cos[g(x) | h(P, K_j)]) (Eq. 1); during local training and server aggregation, gradients are backpropagated and aggregated exclusively with respect to the input-agnostic prompt parameters P (max_{f_prompt} p_{F; f_prompt}(y + v_f | x), Eq. 2), updating only 0.01%--0.1% of the total model parameters.

Identity Notes & Concept Scope Distinct from full-model Federated Averaging (AIMech:FederatedAveraging; McMahan et al., 2017), which updates and communicates 100% of the neural network weights across clients and server. Distinct from centralized vision-language prompt tuning (CoOp; Zhou et al., 2022) and NLP continuous prefix/prompt tuning (Li & Liang, 2021; Lester et al., 2021), which optimize continuous prompts on a single centralized dataset rather than aggregating input-agnostic soft prompt updates across distributed non-IID clients or API-queried frozen backbones. Distinct from federated low-rank weight adapters such as FFA-LoRA (AIMech:FfaLoraFederatedFreezeALowRankAdaptation; Sun et al., ICLR 2024) and FLoRA (AIMech:FloraStackingBasedFederatedHeterogeneousLowRankAdaptation; Wang et al., NeurIPS 2024), which inject low-rank matrix increments into internal linear layers rather than prepending continuous prompt vectors to the text encoder input space.

Claims & Grounded Evidence

4 Claims

Mechanism #promptfl-frozen-clip-continuous-prompt-cosine-alignment-and-aggregation-mechanism

In PromptFL (Figure 1, Eq. 1, p. 3; Eq. 2, p. 6), each client keeps both the CLIP image encoder g(.) and text encoder h(.) frozen and optimizes only p continuous d-dimensional prompt vectors P (d = 512 by default) prepended to class-name word embeddings K_i via the softmax cosine-similarity likelihood p(y = i | x) = exp(cos[g(x) | h(P, K_i)]) / sum_{j=1}^k exp(cos[g(x) | h(P, K_j)]), uploading only the input-agnostic prompt updates to the federated server for global averaging.

supports Source: PromptFL: Let Federated Participants Cooperatively Learn Prompts Instead of Models -- Federated Learning in Age of Foundation Model · Locator: Framework to Learn Prompts in FL & Prompt Learner, Fig. 1, Eq. (1), p. 3; CLIP PromptFL, Eq. (2), pp. 5-6

Scope #promptfl-parameter-and-communication-reduction-under-extreme-non-iid-scope

Across ResNet50 (38.3M parameters) and ViT-B/16 (86.6M parameters) CLIP backbones, PromptFL tunes only 0.1% and 0.01% of parameters respectively (6.3 MB uploaded per round vs 191 MB and 693.3 MB for full fine-tuning FL, a reduction of up to 110x per round and ~3x lower GPU training time per 50 epochs), while maintaining 66.3%--94.12% test accuracy under extreme non-IID (0% class overlap across clients) where full-model Finetuning FL collapses to 22.92%--29.89% (Table 2, p. 5; Figures 2 and 4, pp. 6-7).

supports Source: PromptFL: Let Federated Participants Cooperatively Learn Prompts Instead of Models -- Federated Learning in Age of Foundation Model · Locator: Table 1, p. 4; Table 2, p. 5; Figure 2, p. 6; Figure 4 & Computation and Communication Cost Analysis, p. 7

Limitation #promptfl-backbone-ram-footprint-and-inference-flop-overhead-limitation

Although PromptFL reduces trainable parameters to 0.01%--0.1%, each client must still store and run forward/backward passes through the frozen pre-trained CLIP backbone (requiring ~600 MB disk storage, 60 GFLOPs inference, and 5.95--9.04 GB GPU memory during training unless offloaded to an inference API; Table 1, p. 4; Figure 4, p. 7).

supports Source: PromptFL: Let Federated Participants Cooperatively Learn Prompts Instead of Models -- Federated Learning in Age of Foundation Model · Locator: System Feasibility, Table 1, p. 4; Figure 4 GPU Memory Usage, p. 7

Application #promptfl-few-shot-federated-visual-recognition-application

Applied to few-shot (2, 4, 8, and 16 shots) and non-IID federated visual recognition on Caltech101, Flowers102, OxfordPets, and Food101 using ResNet50 and ViT-B/16 CLIP foundation models.

supports Source: PromptFL: Let Federated Participants Cooperatively Learn Prompts Instead of Models -- Federated Learning in Age of Foundation Model · Locator: Experiments, Table 2, Figures 2-4, pp. 5-7

⚠️ Curated Limitations & Negative Results

Backpropagating gradients from the CLIP cosine similarity loss to the input prompt vectors P still traverses the frozen text encoder Transformer h(.), requiring ~5.95--9.04 GB of device GPU memory (System Feasibility, p. 4; Figure 4, p. 7).
Relies on a pre-trained vision-language foundation model whose pre-training distribution covers the visual concepts of the downstream federated task (Preliminaries & Fine-tuning vs. Prompting, pp. 2, 5).
When using remote Inference APIs instead of on-device CLIP backbones, clients must query the API with their images or intermediate features (Inference APIs, p. 4).

Artificial Intelligence Ontology (AIO) Grounding

Mapped AIO Classes

AIO Target Class	Mapping Relation	Mapping Rationale
`AIO:FederatedLearning` Federated Learning	broad_match	PromptFL is a federated learning framework that collaboratively trains continuous soft prompts on frozen foundation models.

Evidence Sources & Primary Literature

3 Sources

Source ID	Publication / Document	Type / Access	Reference & Checksum
`r509-primary-pdf`	PromptFL: Let Federated Participants Cooperatively Learn Prompts Instead of Models -- Federated Learning in Age of Foundation Model Tao Guo, Song Guo, Junxiao Wang, Xueyang Tang, Wenchao Xu	primary paper full_text	ID: https://doi.org/10.1109/tmc.2023.3302410 sha256:fa87c2b43184…
`r509-arxiv-abs`	PromptFL: Let Federated Participants Cooperatively Learn Prompts Instead of Models -- Federated Learning in Age of Foundation Model (arXiv abstract) Tao Guo, Song Guo, Junxiao Wang, Xueyang Tang, Wenchao Xu	primary paper abstract_only	ID: https://arxiv.org/abs/2208.11625 sha256:466d9d421adc…
`r509-metadata-cr`	PromptFL: Let Federated Participants Cooperatively Learn Prompts Instead of Models -- Federated Learning in Age of Foundation Model (Crossref metadata) Tao Guo, Song Guo, Junxiao Wang, Xueyang Tang, Wenchao Xu	primary paper abstract_only	ID: https://doi.org/10.1109/tmc.2023.3302410 sha256:eaa9848d4a0e…

Ontology & Grounding Topology

Open in interactive network graph →

Curation Provenance

Curator: Antigravity

Method: ai_assisted

Model: Gemini

Created: 2026-10-19T09:52:00Z

Updated: 2026-10-19T09:52:00Z

Source File: data/curated/PromptFlFederatedPromptLearningForFoundationModels.yaml · Schema: curated_concept.yaml

Explore across X-Mech

Related computational & biological Mechs: Embedding Map Network Graph ProteinTraitsMech (ESM-2 pLM) DUFMech (Domain pLM) TraitMech CultureMech All 14 X-Mech Repositories