A clean, ground-up implementation of the Transformer architecture in PyTorch, including positional encoding, multi-head attention, encoder-decoder layers, and masking. Great for learning or building upon the core model.
nlp machine-learning deep-learning transformers pytorch sequence-to-sequence attention-mechanism encoder-decoder self-attention positional-encoding transformers-architecture
-
Updated
May 30, 2025 - Jupyter Notebook