Set Transformer
Architectures AIMech:SetTransformer DRAFT
A neural encoder-decoder architecture for permutation-invariant set-input prediction that uses attention to encode interactions among elements and learned seed queries to aggregate encoded features.
Identity Notes & Concept Scope
The Set Transformer is a set-input architecture, not the AIO Transformer Language Model class, an individual multihead attention layer, or the later Set Transformer++ encoder revision.
Claims & Grounded Evidence
1 Claim
Mechanism
#architecture-mechanism
Stack self-attention or induced-attention blocks without positional encoding, then pool encoded features by attention from learned seed vectors. The induced variant first attends from learned inducing points to the input and then from the input to the resulting summaries.
⚠️ Curated Limitations & Negative Results
- The encoder is permutation equivariant; the pooled output is invariant to input ordering. Multiple seed outputs are not input-element labels.
- Linear scaling with input-set size concerns a fixed number of inducing points; ordinary self-attention blocks have quadratic interactions. No universal benchmark superiority is asserted.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:ArtificialNeuralNetwork
Artificial Neural Network
|
broad_match | The source specifies a learned neural encoder, attention pooling and output network. ArtificialNeuralNetwork is broader without adding a language-model restriction. |
Evidence Sources & Primary Literature
1 Source| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
original-paper
|
Juho Lee, Yoonho Lee, Jungtaek Kim, Adam R. Kosiorek, Seungjin Choi, Yee Whye Teh
|
primary paper
full_text
|
sha256:965aee34a809…
|
Curation Provenance
Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-11T04:19:31Z
Updated: 2026-10-11T04:19:31Z
Source File:
data/curated/SetTransformer.yaml