Kolmogorov-Arnold Transformer

[Adamdad]

KAN was strong but faced scalability issues. We tackled this with 3 simple tricks. By combining KAN with Transformers, we've built a much stronger and more scalable model. 💪

📄 Paper: https://arxiv.org/abs/2409.10594
💻 Code: https://github.com/Adamdad/kat