Inference Run Examples

Tutorial of running the examples

Table of content

Download the example trained models from AWS S3
Download LibriSpeech audio samples from openslr.org
Download and build wav2letter-inference
Simple Streaming Asr Example
Examine the transcription quality
Multithreaded Streaming Asr Example
Simple Streaming Asr Example using Docker

Overview

The supplied examples can be used directly to quickly bootstrap a demo. There are two executables and two libraries. One executable is convenient for transcribing a single audio file or stream. The other is convenient for transcribing a large number of audio files.

simple_streaming_asr_example can by used to quickly transcribe a single audio file.
multithreaded_streaming_asr_example can by used to quickly transcribe many audio files.

The two libraries are useful for bootstrapping your own executables.

AudioToWords has convenient functions to transcribe an audio file or audio stream.
examples/Util.h has utilites to read input from any stream into the inference pipeline. It also has a TimeElapsedReporter a scooped performance measurement utility.

Download the example trained models from AWS S3

~$> mkdir model
~$> cd model
for f in acoustic_model.bin tds_streaming.arch decoder_options.json feature_extractor.bin language_model.bin lexicon.txt tokens.txt ; do wget http://dl.fbaipublicfiles.com/wav2letter/inference/examples/model/${f} ; done

~/model$>ls -sh
total 270M
254M acoustic_model.bin  
1.0K tds_streaming.arch	 
512 decoder_options.json   
512 feature_extractor.bin   
13M language_model.bin	
4.0M lexicon.txt   
82K tokens.txt

Download LibriSpeech audio samples from openslr.org

~$> mkdir audio
~$> cd audio
~/audio$> wget -qO- http://www.openslr.org/resources/12/dev-clean.tar.gz | tar xvz
~/audio$> find LibriSpeech/dev-clean -type f -name "*.flac" -exec sox {} {}".wav"  \;
~/audio$> find "$(pwd)"/LibriSpeech/dev-clean -type f -name "*.wav" > LibriSpeech-dev-clean-wav-all.lst

We should have 2703 audio files.

~/audio$> wc -l LibriSpeech-dev-clean-wav.lst
2703 LibriSpeech-dev-clean-wav.lst

Download and build wav2letter-inference

Download and build kenlm

~/$> git clone https://github.com/kpu/kenlm.git
~/$> cd kenlm
~/kenlm> mkdir build
~/kenlm/build$> cmake .. 
~/kenlm/build$> make -j $(nproc)

Download and build wav2letter-inference

~/$> git clone https://github.com/facebookresearch/wav2letter.git
~/$> cd wav2letter
~/wav2letter$> mkdir build
~/wav2letter/build$> KENLM_ROOT_DIR=~/kenlm/build cmake .. -DW2L_BUILD_LIBRARIES_ONLY=ON -DW2L_BUILD_INFERENCE=ON -DW2L_LIBRARIES_USE_CUDA=OFF

Simple Streaming Asr Example

simple_streaming_asr_example can be used as a unix pipe to dump translation for a wav stream.

~/wav2letter/build$> make simple_streaming_asr_example -j $(nproc)
~/wav2letter/build$ cat ~/audio/LibriSpeech/dev-clean/777/126732/777-126732-0070.flac.wav | inference/inference/examples/simple_streaming_asr_example --input_files_base_path ~/model

Started features model file loading ...
Completed features model file loading elapsed time=46557 microseconds

Started acoustic model file loading ...
Completed acoustic model file loading elapsed time=2058 milliseconds

Started tokens file loading ...
Completed tokens file loading elapsed time=1318 microseconds

Tokens loaded - 9998 tokens
Started decoder options file loading ...
Completed decoder options file loading elapsed time=388 microseconds

Started create decoder ...
[Letters] 9998 tokens loaded.
[Words] 200001 words loaded.
Completed create decoder elapsed time=884 milliseconds

Started converting audio input from stdin to text... ...
Creating LexiconDecoder instance.
#start (msec), end(msec), transcription
0,1000,
1000,2000,he was out of his
2000,3000,mind with something
3000,4000,he overheard about eating
4000,5000,people's flesh
5000,6000,and drinking blood
6000,7000,what's the good of
7000,7315,of talking like that
Completed converting audio input from stdin to text... elapsed time=1302 milliseconds

Examine the transcription quality

We can exmin the transcription quality by inspecting the audio's file transcription. We find the transcription by the file number. For example for the file:~/audio/LibriSpeech/dev-clean/777/126732/777-126732-0070.flac.wav The file number is 0070

~/wav2letter/build$> grep 0070 ~/audio/LibriSpeech/dev-clean/777/126732/777-126732.trans.txt
777-126732-0070 HE WAS OUT OF HIS MIND WITH SOMETHING HE OVERHEARD ABOUT EATING PEOPLE'S FLESH AND DRINKING BLOOD WHAT'S THE GOOD OF TALKING LIKE THAT

It can also transcribe a single file by directly pointing to it:

~/wav2letter/build$> make simple_streaming_asr_example -j $(nproc)
~/wav2letter/build$> inference/inference/examples/simple_streaming_asr_example --input_files_base_path ~/model --input_audio_file ~/audio/LibriSpeech/dev-clean/777/126732/777-126732-0076.flac.wav
...
#start (msec), end(msec), transcription
0,1000,
1000,2000,i wish he
2000,3000,had never been to school
3000,4000,missus
4000,4260,began again brusquely
Completed converting audio input file=/home/avidov/audio/LibriSpeech/dev-clean/777/126732/777-126732-0076.flac.wav to text... elapsed time=914 milliseconds

Multithreaded Streaming ASR Example

multithreaded_streaming_asr_example can convert a large list of audio files using multiple threads.

For running this exeample quickly on a laptop we can start with a smaller number of files. Here we preapre a audio file list with 50 files.

~/wav2letter/build$>head -50 ~/audio/LibriSpeech-dev-clean-wav-all.lst > ~/audio/LibriSpeech-dev-clean-wav-50-files.lst

Now let's run wav2letter on the entire list.

~/wav2letter/build:> mkdir ~/audio/LibriSpeech-dev-clean-transcribed
~/wav2letter/build$> make multithreaded_streaming_asr_example -j $(nproc)

~/wav2letter/build$ inference/inference/examples/multithreaded_streaming_asr_example   --input_audio_file_of_paths ~/audio/LibriSpeech-dev-clean-wav-50-files.lst  --output_files_base_path ~/audio/LibriSpeech-dev-clean-transcribed  --input_files_base_path ~/model --max_num_threads $(nproc)


~/wav2letter/build:> inference/inference/examples/multithreaded_streaming_asr_example   --input_audio_file_of_paths ~/audio/LibriSpeech-dev-clean-wav.lst  --output_files_base_path ~/audio/LibriSpeech-dev-clean-transcribed  --input_files_base_path=$HOME/model --max_num_threads $(nproc)
...
Will process 50 files.
Started features model file loading ...
Completed features model file loading elapsed time=44232 microseconds

Started acoustic model file loading ...
Completed acoustic model file loading elapsed time=2205 milliseconds

Started tokens file loading ...
Completed tokens file loading elapsed time=1521 microseconds

Tokens loaded - 9998 tokens
Started decoder options file loading ...
Completed decoder options file loading elapsed time=342 microseconds

Started create decoder ...
[Letters] 9998 tokens loaded.
[Words] 200001 words loaded.
Completed create decoder elapsed time=804 milliseconds

Started converting audio input files to text ...
Creating thread pool with 80 threads.
audioFileToWordsFile() processing audioFileToWordsFile() processing 1/50 input=2//home/avidov/audio/LibriSpeech/dev-clean/2277/149896/2277-149896-0005.flac.wav50 output=/home/avidov/audio/LibriSpeech-dev-clean-transcribed/2277-149896-0005.flac.wav.txt
audioFileToWordsFile() processing audioFileToWordsFile() processing 4/50 input=/home/avidov/audio/LibriSpeech/dev-clean/2277/149896/2277-149896-0006.flac.wav output=/home/avidov/audio/LibriSpeech-dev-clean-transcribed/2277-149896-0006.flac.wav.txt
audioFileToWordsFile() processing 50/50 input=/home/avidov/audio/LibriSpeech/dev-clean/2277/149897/2277-149897-0004.flac.wav output=/home/avidov/audio/LibriSpeech-dev-clean-transcribed/2277-149897-0004.flac.wav.txt
...
Completed converting audio input files to text elapsed time=18648 milliseconds
Completed create decoder elapsed time=908 milliseconds

Simple Streaming ASR Example using Docker

wav2letter inference Docker allows you to quickly run inference binaries without needing to install dependencies and build wav2letter inference.

sudo docker run --rm -v ~:/root/host/ -it --ipc=host --name w2l -a stdin -a stdout -a stderr wav2letter/wav2letter:inference-latest sh -c "/root/host/audio/LibriSpeech/dev-clean/777/126732/777-126732-0070.flac.wav | /root/wav2letter/build/inference/inference/examples/simple_streaming_asr_example --input_files_base_path /root/host/model"