Spiking Wavelet Transformer (ECCV-2024)

Spiking Wavelet Transformer, ECCV'24: [Paper].

Key contributions

The "Spiking Wavelet Transformer" introduces an innovative approach to enhancing spiking neural networks (SNNs) by integrating wavelet transforms with transformer architectures in an attention-free fashion. This method addresses the challenge of effectively capturing high-frequency patterns crucial for event-driven vision tasks, unlike self-attention, which prioritizes low-frequency elements. Key features include:

• Frequency-Aware Token Mixer (FATM): Learns spatial-frequency features without relying on self-attention.
• Spiking Frequency Representation: Getting robust frequency representation efficiently in a spike-driven, multiplication-free manner.
• Enhanced Performance: Offers improved accuracy and reduced parameter count on datasets like ImageNet.

This approach provides a practical solution for advancing energy-efficient, event-driven computing.

Implementation

Checkpoints for ImageNet

For more details on our training, please check out our paper and supplementary material. (Note: for Imagenet, we used 8×A800 GPU cards for training, total batch size = 512 )

Requirement:

Make sure your PyTorch version is 2.0.0 or higher. For more information, please visit link for details

  pip install timm==0.6.12 spikingjelly==0.0.0.0.12 opencv-python==4.8.1.78 wandb einops PyYAML Pillow six torch

Running the code

Please check the bash file in each folder (cifar10-100, event, imagenet).

Citation

  @inproceedings{fang2025spiking,
    title={Spiking wavelet transformer},
    author={Fang, Yuetong and Wang, Ziqing and Zhang, Lingfeng and Cao, Jiahang and Chen, Honglei and Xu, Renjing},
    booktitle={European Conference on Computer Vision},
    pages={19--37},
    year={2025},
    organization={Springer}
  }