Speed up \uXXXX parsing and improve WTF-8 handling #1175
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This counterintuitively speeds up War and Peace 275 -> 290 MB/s (+5%) by enabling inlining of encode_utf8 and extend_from_slice.
This speeds up War and Peace 290 MB/s -> 330 MB/s (+15%).
This does not affect performance.
This does not affect performance.
Closes serde-rs#877. This is a good time to make ByteBuf parsing more consistent as I'm rewriting it anyway. This commit integrates the changes from serde-rs#877 and also handles a leading surrogate followed by a surrogate pair correctly. This does not affect performance significantly. Co-authored-by: Luca Casonato <hello@lcas.dev>
purplesyringa
commented
Aug 12, 2024
Comment on lines
+908
to
+909
| // XXX: This is actually a trailing surrogate. | ||
| return error(read, ErrorCode::LoneLeadingSurrogateInHexEscape); |
There was a problem hiding this comment.
Should I do anything about this? This typo was present before the PR.
There was a problem hiding this comment.
I would accept a followup PR to change the ErrorCode enum and fix the error message.
purplesyringa
changed the title
# for free
to join this conversation on GitHub.
Already have an account?
# to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Altogether, this speeds up \u-encoded War and Peace parsing by 20%. Performance on json-benchmark is slightly affected: there are some 5% improvements and a -1% regression, but I'm willing to write that off as noise from an imperfect benchmark setup.
This PR should be more readable per-commit w/o whitespace changes. In addition to the above, it includes a variation on #877, since it's easier to implement with this design.