Optimize surrogate decoding. #894
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Use `char ^ 0xD800 <= 0x3FF` to check if a char code is a lead surrogate. That avoids doing a later `& 0x3FF` to get rid of the top bits. Similar for tail surrogate. This ensures that the `high` function gets values without high bits. Also optimize that function to reduce dependency depth and try to hit `base + (something < small)` expressions that can optimized into a single x64 address computation. Gives a ~7% increase on backwards traversal and 38% increase for forward traversal, based on tool/benchmark.dart compiled with `dart compile exe`.
Package publishing
Documentation at https://github.com/dart-lang/ecosystem/wiki/Publishing-automation. |
PR HealthBreaking changes ✔️
Changelog Entry ✔️
Changes to files need to be accounted for in their respective changelogs.
Coverage
|
| File | Coverage |
|---|---|
| pkgs/characters/lib/src/characters_impl.dart | 💚 90 % ⬆️ 0 % |
| pkgs/characters/lib/src/grapheme_clusters/breaks.dart | 💚 97 % ⬆️ 0 % |
| pkgs/characters/lib/src/grapheme_clusters/table.dart | 💚 100 % |
| pkgs/characters/tool/bin/generate_tables.dart | 💔 Not covered |
| pkgs/characters/tool/src/string_literal_writer.dart | 💔 Not covered |
This check for test coverage is informational (issues shown here will not fail the PR).
This check can be disabled by tagging the PR with skip-coverage-check.
API leaks ✔️
The following packages contain symbols visible in the public API, but not exported by the library. Export these symbols or remove them from your publicly visible API.
| Package | Leaked API symbols |
|---|
License Headers ✔️
// Copyright (c) 2025, the Dart project authors. Please see the AUTHORS file
// for details. All rights reserved. Use of this source code is governed by a
// BSD-style license that can be found in the LICENSE file.
| Files |
|---|
| no missing headers |
All source files should start with a license header.
mosuem
approved these changes
Jun 2, 2025
| var index = chunkStart + (tail & 255); | ||
| return _data.codeUnitAt(index); | ||
| var offset = (tail >> 8) + (lead << 2); | ||
| tail &= 255; |
There was a problem hiding this comment.
Why do the assignment instead of the original chunkStart + (tail & 255)?
| var chunkStart = _start.codeUnitAt(offset >> 8); | ||
| var index = chunkStart + (tail & 255); | ||
| return _data.codeUnitAt(index); | ||
| var offset = (tail >> 8) + (lead << 2); |
There was a problem hiding this comment.
IIUC, this assumes that tail and lead don't need to be masked with 0x3ff. Should this be asserted here?
# for free
to join this conversation on GitHub.
Already have an account?
# to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Use
char ^ 0xD800 <= 0x3FFto check if a char code is a lead surrogate. That avoids doing a later& 0x3FFto get rid of the top bits. Similar for tail surrogate.This ensures that the
highfunction gets values without high bits, which makes it smaller (it tries to get inlined, so a little smaller counts).Also optimize that function to reduce dependency depth and try to hit
base + (something < small)expressions that can optimized into a single x64 address computation.Gives a ~7% increase on backwards traversal and 30% increase for forward traversal, based on tool/benchmark.dart compiled with
dart compile exe.Actually a small decrease in performance on web for forward iteration, and a small increase for backwards iteration, and Wasm follows Web in performance here.
(Also found a bug in the generator, which hasn't worked since it was last committed.)
Interestingly, the change makes little-to-no difference on the
benchmark/benchmark.dartbenchmark.(Maybe even makes it a little slower.)