AutoPrompt Gradient-Guided Discrete Prompt Search

An automated discrete prompt construction and knowledge-probing algorithm for pretrained language models that maps task inputs x_inp into a cloze template containing a prediction token [MASK] and shared discrete trigger tokens x_trig, automatically selects class-associated label token sets V_y via a logistic classifier over [MASK] contextual representations scored against output word embeddings w_out, and iteratively updates each trigger token x_trig^(j) by evaluating the top-k vocabulary candidates V_cand = top-k_{w in V} [w_in^T grad_{x_trig^(j)} log p(y|x_prompt)] under a first-order gradient approximation of the marginalized label log-likelihood p(y|x_prompt) = sum_{w in V_y} p([MASK]=w|x_prompt) while keeping all language model weights frozen.