Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

Fix reading tokens.txt on Windows #448

Merged
merged 1 commit into from
Nov 25, 2023
Merged

Fix reading tokens.txt on Windows #448

merged 1 commit into from
Nov 25, 2023

Conversation

csukuangfj
Copy link
Collaborator 

fangjuns-MacBook-Pro:sherpa-onnx fangjun$ hexdump -c -x -n 10 ./vits-zh-aishell3/tokens.txt
0000000   s   i   l       0  \n   e   o   s
0000000    6973    206c    0a30    6f65    2073
000000a
fangjuns-MacBook-Pro:sherpa-onnx fangjun$ hexdump -c -x -n 10 ~/Downloads/tokens.txt
0000000   s   i   l       0  \r  \n   e   o   s
0000000    6973    206c    0d30    650a    736f
000000a

~/Downloads/tokens.txt is from a user who downloads it on Windows using git.

This PR handles \r\n correctly in case tokens.txt contains \r\n.

@csukuangfj csukuangfj merged commit 66cad9f into k2-fsa:master Nov 25, 2023
@csukuangfj csukuangfj deleted the fix-reading-tokens branch November 25, 2023 06:22
XiaYucca pushed a commit to XiaYucca/sherpa-onnx that referenced this pull request Jan 9, 2025
@csukuangfj
Fix reading tokens.txt on Windows (k2-fsa#448)
7834ab7
# for free to join this conversation on GitHub. Already have an account? # to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@csukuangfj