Vaswani et al. · NeurIPS 2017 · arXiv:1706.03762

The Attention Engine

Watch a tiny Transformer translate je suis étudiant into i am a student, one operation at a time. In stages 3 to 9 every number is computed live in your browser with the paper's exact formulas, on a model shrunk to dmodel = 8 so it fits on screen. New to this? Start with the beginner version.

For the researcher