Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

skip UTF-8 BOM also #381

Merged
merged 2 commits into from
Jan 28, 2023
Merged

skip UTF-8 BOM also #381

merged 2 commits into from
Jan 28, 2023

Conversation

kmuto
Copy link
Contributor

@kmuto kmuto commented Jan 28, 2023

I noticed that reading a UTF-8 encoded file with a BOM causes an error. Unfortunately, major Windows software adds a BOM to UTF-8 files.
UTF-16 support is fixed in #277 , but in addition, skipping the UTF-8 BOM (ef bb bf, https://en.wikipedia.org/wiki/Byte_order_mark ) also solves the problem, I believe.

A sample is attached.

before:

OK(withoutBOM) あ
ERROR(withBOM) toml: line 1: expected '.' or '=', but got '\ufeff' instead

after:

OK(withoutBOM) あ
OK(withBOM) あ

mini.zip

@arp242 arp242 merged commit 1a6ca6e into BurntSushi:master Jan 28, 2023
@arp242
Copy link
Collaborator

arp242 commented Jan 28, 2023

Thanks!

@kmuto
Copy link
Contributor Author

kmuto commented Jan 30, 2023

@arp242
Thanks for the merge!
Have you decided when you plan to release the next one? If you are keeping the releases at 3 month intervals, I think it's about the right time. 😊

@kmuto kmuto mentioned this pull request May 18, 2023
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants