Amphion Alpha Release

[RMSnow]

We release the alpha version of Amphion 🎉. The key features are as follows:

• TTS: Text to Speech
  • Support FastSpeech2 and VITS
• SVC: Singing Voice Conversion
  • Support mutplie content-based features including WeNet, Whisper, and ContentVec
  • Provide the official implementation of the paper "Leveraging Content-based Features from Multiple Acoustic Models for Singing Voice Conversion" (NeurIPS 2023 Workshop on Machine Learning for Audio)
• TTA: Text to Audio
  • Support a latent diffusio model based TTA. It is also the official implementation of the text-to-audio generation part of our NeurIPS 2023 paper "AUDIT: Audio Editing by Following Instructions with Latent Diffusion Models".
• Vocoder
  • Support several GAN-based Vocoders including MelGAN, HiFi-GAN, NSF-HiFiGAN, BigVGAN, and APNet.
  • Provide the official implementation of Multi-Scale Constant-Q Transfrom Discriminator.
  • Release two vocoder checkpoints (see Amphion/pretrained/README.md): Amphion Speech HiFi-GAN and Amphion Singing BigVGAN.
• Evaluation
  • Support 16 objective metrics (see Amphion/egs/metrics/README.md) about F0 Modeling, Energy Modeling, Intelligibility, Spectrogram Distortion, and Speaker Similarity.
• Datasets
  • Support 15 academic datasets (see Amphion/egs/datasets/README.md).

Thanks to all the contributors including:

• Xueyao Zhang* @RMSnow, Liumeng Xue* @lmxue, Yuancheng Wang* @HeCheng0625, and Yicheng Gu* @VocodexElysium (The Chinese University of Hong Kong, Shenzhen)
• Xi Chen @ChenX17 (The Chinese University of Hong Kong, Shenzhen)
• Zihao Fang @Adorable-Qin (The Chinese University of Hong Kong, Shenzhen)
• Haopeng Chen @arsity (The Chinese University of Hong Kong, Shenzhen)
• Lexiao Zou @Lokshaw-Chau (The Chinese University of Hong Kong, Shenzhen & Harbin Institute of Technology, Shenzhen)
• Chaoren Wang @yuantuo666 (The Chinese University of Hong Kong, Shenzhen)
• Jun Han (The Chinese University of Hong Kong, Shenzhen)
• Kai Chen (Shanghai AI Lab & OpenMMLab)
• Haizhou Li (The Chinese University of Hong Kong, Shenzhen)
• Zhizheng Wu @zhizhengwu (The Chinese University of Hong Kong, Shenzhen & Shanghai AI Lab)

*: Equal Contributions.

Also, thanks to Shenzhen Research Institute of Big Data (SRIBD) for partially supporting computing and scholarships to Xueyao, Liumeng, Yuancheng.