Tacotron2 + LPCNET for complete End-to-End TTS System
Thanks to https://github.com/MlWoo/ for most of changes
Checkout the LPCTron , and run tts.sh it converts text_file ( Tacotron2/text.txt ) to test.wav
- Checkout Tacatron2 (https://github.com/alokprasad/LPCTron/blob/master/Tacotron-2/Tacotron2-lpcnet_changes.diff)
python3 preprocess.py --base_dir /media/alok/ws/sandbox/lpc_tacatron2/dataset --dataset LJSpeech-1.1 this will generate /dataset/training_data folder
├── training_data (1)
│ ├── audio (*)
│ ├── linear
│ └── mels
└── train.txt
*all three folder contain npy array, train.txt contains text for each audio.
- this is replaced by script for training to be used by LPCNET
Since Tacatron2 is trained with LPSpeech its good idea to train with Same data. LPCNET uses a single Merged PCM for training. So you need to remove wav headers from each file and merge them for training.
===============================================================================
sudo apt-get install python3-tk
pip3 install lws --user
python -m pip uninstall pip && sudo apt install python3-pip --reinstall
python3 train.py --input_dir ../dataset/training_data --tacotron_input ../dataset/training_data/train.txt --model='Tacotron'
python3 synthesize.py --model='Tacotron' --mode='eval'
sudo apt-get install python3-tk
pip3 install lws --user python -m pip uninstall pip && sudo apt install python3-pip --reinstall
Download https://jmvalin.ca/misc_stuff/lpcnet_models/lpcnet15_384_10_G16_100.h5
apt-get install -y google-perftools LD_PRELOAD = /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4
python3 train.py --input_dir ../dataset/training_data --tacotron_input ../dataset/training_data/train.txt --model='Tacotron'
python3 synthesize.py --model='Tacotron' --mode='eval'
./test_lpcnet f32_for_lpcnet.f32 test.s16
ffmpeg -f s16le -ar 16k -ac 1 -i test.s16 test-out.wav