Illia Polosukhin

3 beliefs on korrents (opens korrents.com)

• Sequence transduction can use a simple architecture based solely on attention, without recurrence or convolutions. (opens korrents.com)

  “We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely.”

  2017-06-12

• A transduction model can rely entirely on self-attention for input and output representations without sequence-aligned RNNs or convolution. (opens korrents.com)

  “To the best of our knowledge, however, the Transformer is the first transduction model relying entirely on self-attention to compute representations of its input and output without using sequence-aligned RNNs or…”

  2017-06-12

• Self-attention can yield more interpretable models. (opens korrents.com)

  “As side benefit, self-attention could yield more interpretable models.”

  2017-06-12