PreTTR Precomputing Transformer Term Representations

A decoupled-layer Transformer document re-ranking architecture and token-representation compression method that applies a cross-segment attention mask in the first l layers L_1, ..., L_l of an n-layer Transformer during training—preventing query tokens ([CLS]; q; [SEP]) from attending to document tokens (d; [SEP]) and vice versa—so that layer-l document token representations s_l in R^{m x d} can be precomputed offline at index time, compressed to dimension e < d via a GELU bottleneck encoder-decoder pre-trained to minimize mean squared error on the attention score matrices of the remaining n - l layers, and concatenated at query time with the layer-l query representations to execute only layers L_{l+1}, ..., L_n (computing only the [CLS] token in the final layer L_n).