Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch
-
Updated
Dec 21, 2024 - Python
Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch
RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.
all kinds of text classification models and more with deep learning
Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch
A concise but complete full-attention transformer with a set of promising experimental features from various papers
A collection of important graph embedding, classification and representation learning papers with implementations.
An ultimately comprehensive paper list of Vision Transformer/Attention, including papers, codes, and related websites
A TensorFlow Implementation of the Transformer: Attention Is All You Need
Graph Attention Networks (https://arxiv.org/abs/1710.10903)
Automatic Speech Recognition (ASR), Speaker Verification, Speech Synthesis, Text-to-Speech (TTS), Language Modelling, Singing Voice Synthesis (SVS), Voice Conversion (VC)
Pytorch implementation of the Graph Attention Network model by Veličković et. al (2017, https://arxiv.org/abs/1710.10903)
Show, Attend, and Tell | a PyTorch Tutorial to Image Captioning
Keras Attention Layer (Luong and Bahdanau scores).
My implementation of the original GAT paper (Veličković et al.). I've additionally included the playground.py file for visualizing the Cora dataset, GAT embeddings, an attention mechanism, and entropy histograms. I've supported both Cora (transductive) and PPI (inductive) examples!
Multilingual Automatic Speech Recognition with word-level timestamps and confidence
Reformer, the efficient Transformer, in Pytorch
The Enterprise-Grade Production-Ready Multi-Agent Orchestration Framework Join our Community: https://discord.com/servers/agora-999382051935506503
To eventually become an unofficial Pytorch implementation / replication of Alphafold2, as details of the architecture get released
Implementation of LambdaNetworks, a new approach to image recognition that reaches SOTA with less compute
Implementation of SoundStorm, Efficient Parallel Audio Generation from Google Deepmind, in Pytorch
Add a description, image, and links to the attention-mechanism topic page so that developers can more easily learn about it.
To associate your repository with the attention-mechanism topic, visit your repo's landing page and select "manage topics."