Script for ASR inference on long files #2373
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Collaborator
jbalam-nv
commented
Jun 18, 2021
jbalam-nv
commented
- speech_to_text_buffered_infer.py supports inference on long audio files by running inference on smaller chunks of audio and them merging the tokens for final transcription.
- This version only supports EncDecCTCModelBPE with AudioMelSpectrogramProcessor as the preprocessor
- Feature normalization is done on each buffer using "per_feature" normalization, future versions will have other methods for experimentation
Signed-off-by: jbalam-nv <4916480+jbalam-nv@users.noreply.github.com>
Signed-off-by: jbalam-nv <4916480+jbalam-nv@users.noreply.github.com>
Signed-off-by: jbalam-nv <4916480+jbalam-nv@users.noreply.github.com>
|
This pull request introduces 2 alerts when merging 4351694 into e070e04 - view on LGTM.com new alerts:
|
|
This pull request introduces 2 alerts when merging e00359a into e5bde15 - view on LGTM.com new alerts:
|
Signed-off-by: jbalam <jbalam@nvidia.com>
titu1994
approved these changes
Jul 13, 2021
| # Create a preprocessor to convert audio samples into raw features, | ||
| # Normalization will be done per buffer in frame_bufferer | ||
| # Do not normalize whatever the model's preprocessor setting is | ||
| preprocessor_cfg.normalize = "None" |
There was a problem hiding this comment.
Hack to get through without any normalization. None throws an error here
, as we are looking for a key in a dict. We should probably clean up this normalize_batch function and add a "no_norm" option.There was a problem hiding this comment.
Ah ok. no_norm sounds like a good option
| frame_overlap: duration of overlaps before and after current frame, seconds | ||
| offset: number of symbols to drop for smooth streaming | ||
| ''' | ||
| self.ZERO_LEVEL_SPEC_DB_VAL = -16.635 |
There was a problem hiding this comment.
Maybe some info can be added on how this was calculated?
Signed-off-by: jbalam-nv <4916480+jbalam-nv@users.noreply.github.com>
fayejf
pushed a commit
that referenced
this pull request
Jul 16, 2021
* First version of script for buffered inference Signed-off-by: jbalam-nv <4916480+jbalam-nv@users.noreply.github.com> * Cleaned up commented code and added comments Signed-off-by: jbalam-nv <4916480+jbalam-nv@users.noreply.github.com> * More clean up and simplified the call to transcribe Signed-off-by: jbalam-nv <4916480+jbalam-nv@users.noreply.github.com> * Removed unused variables Signed-off-by: jbalam <jbalam@nvidia.com> * Style fix Signed-off-by: jbalam <jbalam@nvidia.com> * Added a comment for zero_level_spec_db constant Signed-off-by: jbalam-nv <4916480+jbalam-nv@users.noreply.github.com> * style fix Signed-off-by: jbalam <jbalam@nvidia.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com>
titu1994
added a commit
to titu1994/NeMo
that referenced
this pull request
Jul 20, 2021
* First version of script for buffered inference Signed-off-by: jbalam-nv <4916480+jbalam-nv@users.noreply.github.com> * Cleaned up commented code and added comments Signed-off-by: jbalam-nv <4916480+jbalam-nv@users.noreply.github.com> * More clean up and simplified the call to transcribe Signed-off-by: jbalam-nv <4916480+jbalam-nv@users.noreply.github.com> * Removed unused variables Signed-off-by: jbalam <jbalam@nvidia.com> * Style fix Signed-off-by: jbalam <jbalam@nvidia.com> * Added a comment for zero_level_spec_db constant Signed-off-by: jbalam-nv <4916480+jbalam-nv@users.noreply.github.com> * style fix Signed-off-by: jbalam <jbalam@nvidia.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com>
paarthneekhara
pushed a commit
to paarthneekhara/NeMo
that referenced
this pull request
Sep 17, 2021
* First version of script for buffered inference Signed-off-by: jbalam-nv <4916480+jbalam-nv@users.noreply.github.com> * Cleaned up commented code and added comments Signed-off-by: jbalam-nv <4916480+jbalam-nv@users.noreply.github.com> * More clean up and simplified the call to transcribe Signed-off-by: jbalam-nv <4916480+jbalam-nv@users.noreply.github.com> * Removed unused variables Signed-off-by: jbalam <jbalam@nvidia.com> * Style fix Signed-off-by: jbalam <jbalam@nvidia.com> * Added a comment for zero_level_spec_db constant Signed-off-by: jbalam-nv <4916480+jbalam-nv@users.noreply.github.com> * style fix Signed-off-by: jbalam <jbalam@nvidia.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Paarth Neekhara <paarth.n@gmail.com>
# for free
to join this conversation on GitHub.
Already have an account?
# to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.