Streamline `StringReader::bump` #50566

nnethercote · 2018-05-09T10:58:11Z

These patches make bump smaller and nicer. They speed up most runs for coercions and tuple-stress by 1--3%.

rust-highfive · 2018-05-09T10:58:14Z

r? @petrochenkov

(rust_highfive has picked a reviewer for you, use r? to override)

petrochenkov · 2018-05-11T00:10:39Z

Thanks!
@bors r+

I hope we didn't miss any off-by-one errors here.

bors · 2018-05-11T00:10:40Z

📌 Commit f04510d has been approved by petrochenkov

bors · 2018-05-12T13:08:20Z

⌛ Testing commit f04510de8ed288dd1af513b6f2ededa86c83c843 with merge 7f312754402735e825842b33d9222d8eb6abf4f9...

bors · 2018-05-12T13:58:14Z

💔 Test failed - status-travis

rust-highfive · 2018-05-12T13:58:15Z

The job dist-x86_64-freebsd of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

[00:16:01] [RUSTC-TIMING] rustc_plugin test:false 17.541
[00:16:11]    Compiling rustc_save_analysis v0.0.0 (file:///checkout/src/librustc_save_analysis)
[00:16:12]    Compiling rustc_privacy v0.0.0 (file:///checkout/src/librustc_privacy)
[00:16:12] [RUSTC-TIMING] rustc_resolve test:false 63.263
No output has been received in the last 30m0s, this potentially indicates a stalled build or something wrong with the build itself.
Check the details on how to adjust your build configuration on: https://docs.travis-ci.com/user/common-build-problems/#Build-times-out-because-no-output-was-received
The build has been terminated

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

rust-highfive · 2018-05-12T13:58:18Z

The job dist-x86_64-freebsd of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

[00:16:01] [RUSTC-TIMING] rustc_plugin test:false 17.541
[00:16:11]    Compiling rustc_save_analysis v0.0.0 (file:///checkout/src/librustc_save_analysis)
[00:16:12]    Compiling rustc_privacy v0.0.0 (file:///checkout/src/librustc_privacy)
[00:16:12] [RUSTC-TIMING] rustc_resolve test:false 63.263
No output has been received in the last 30m0s, this potentially indicates a stalled build or something wrong with the build itself.
Check the details on how to adjust your build configuration on: https://docs.travis-ci.com/user/common-build-problems/#Build-times-out-because-no-output-was-received
The build has been terminated

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

kennytm · 2018-05-12T14:04:02Z

@bors retry

bors · 2018-05-12T14:40:10Z

⌛ Testing commit f04510de8ed288dd1af513b6f2ededa86c83c843 with merge 63dbff88a50864891f68f27e1d960deedaca87f7...

bors · 2018-05-12T15:29:50Z

💔 Test failed - status-appveyor

kennytm · 2018-05-12T15:56:49Z

src/libsyntax/parse/lexer/comments.rs

-        col -= 1;
+        let len = ch.len_utf8();
+        cursor += len;
+        col_pos -= len;


---- [pretty] pretty\block-comment-wchar.rs stdout ---- error: pretty-printing failed in round 0 revision None status: exit code: 101 command: PATH="C:\projects\rust\build\x86_64-pc-windows-msvc\stage2\bin;C:\projects\rust\build\x86_64-pc-windows-msvc\llvm\build\bin;C:\Program Files (x86)\Windows Kits\10\bin\x64;C:\Program Files (x86)\Windows Kits\10\bin\10.0.14393.0\x64;C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\amd64;C:\projects\rust\build\x86_64-pc-windows-msvc\stage0-tools\x86_64-pc-windows-msvc\release\deps;C:\projects\rust\build\x86_64-pc-windows-msvc\stage0-sysroot\lib\rustlib\x86_64-pc-windows-msvc\lib;C:\Program Files (x86)\Inno Setup 5;C:\Python27;C:\msys64\mingw64\bin;C:\msys64\usr\bin;C:\Perl\site\bin;C:\Perl\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0;C:\Program Files\7-Zip;C:\Program Files\Microsoft\Web Platform Installer;C:\Tools\GitVersion;C:\Tools\PsTools;C:\Program Files\Git LFS;C:\Program Files (x86)\Subversion\bin;C:\Program Files\Microsoft SQL Server\120\Tools\Binn;C:\Program Files\Microsoft SQL Server\Client SDK\ODBC\110\Tools\Binn;C:\Program Files (x86)\Microsoft SQL Server\120\Tools\Binn;C:\Program Files\Microsoft SQL Server\120\DTS\Binn;C:\Program Files (x86)\Microsoft SQL Server\120\Tools\Binn\ManagementStudio;C:\Tools\WebDriver;C:\Program Files (x86)\Microsoft SDKs\TypeScript\1.4;C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\PrivateAssemblies;C:\Program Files (x86)\Microsoft SDKs\Azure\CLI\wbin;C:\Ruby193\bin;C:\Tools\NUnit\bin;C:\Tools\xUnit;C:\Tools\MSpec;C:\Tools\Coverity\bin;C:\Program Files (x86)\CMake\bin;C:\go\bin;C:\Program Files\Java\jdk1.8.0\bin;C:\Python27;C:\Program Files\nodejs;C:\Program Files (x86)\iojs;C:\Program Files\iojs;C:\Users\appveyor\AppData\Roaming\npm;C:\Program Files\Microsoft SQL Server\130\Tools\Binn;C:\Program Files (x86)\MSBuild\14.0\Bin;C:\Tools\NuGet;C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow;C:\Program Files\Microsoft DNX\Dnvm;C:\Program Files\Microsoft SQL Server\Client SDK\ODBC\130\Tools\Binn;C:\Program Files (x86)\Microsoft SQL Server\130\Tools\Binn;C:\Program Files (x86)\Microsoft SQL Server\130\DTS\Binn;C:\Program Files\Microsoft SQL Server\130\DTS\Binn;C:\Program Files (x86)\Microsoft SQL Server\110\DTS\Binn;C:\Program Files (x86)\Microsoft SQL Server\120\DTS\Binn;C:\Program Files (x86)\Apache\Maven\bin;C:\Python27\Scripts;C:\Tools\NUnit3;C:\Program Files\Mercurial;C:\Program Files\LLVM\bin;C:\Program Files\dotnet;C:\Tools\curl\bin;C:\Program Files\Amazon\AWSCLI;C:\Program Files (x86)\Microsoft SQL Server\140\DTS\Binn;C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\Extensions\Microsoft\SQLDB\DAC\140;C:\Tools\vcpkg;C:\Program Files (x86)\Microsoft SQL Server\140\Tools\Binn;C:\Program Files\Microsoft SQL Server\140\Tools\Binn;C:\Program Files\Microsoft SQL Server\140\DTS\Binn;C:\Program Files\PowerShell\6.0.0;C:\Program Files\erl9.2\bin;C:\Program Files (x86)\nodejs;C:\Program Files\Git\cmd;C:\Program Files\Git\usr\bin;C:\Program Files (x86)\Yarn\bin;C:\Program Files (x86)\NSIS;C:\Tools\Octopus;C:\Program Files\Microsoft Service Fabric\bin\Fabric\Fabric.Code;C:\Program Files\Microsoft SDKs\Service Fabric\Tools\ServiceFabricLocalClusterManager;C:\ProgramData\chocolatey\bin;C:\Users\appveyor\AppData\Roaming\npm;C:\Users\appveyor\AppData\Local\Yarn\bin;C:\Program Files\AppVeyor\BuildAgent;C:\projects\rust;C:\projects\rust\handle" "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage2\\bin\\rustc.exe" "-" "-Z" "unpretty=normal" "--target" "x86_64-pc-windows-msvc" "-L" "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\test\\pretty\\block-comment-wchar.stage2-x86_64-pc-windows-msvc.pretty.aux" "-Crpath" "-O" "-Zunstable-options" "-Lnative=C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\native\\rust-test-helpers" stdout: ------------------------------------------ ------------------------------------------ stderr: ------------------------------------------ thread 'main' panicked at 'attempt to subtract with overflow', libsyntax\parse\lexer\comments.rs:215:9 note: Run with `RUST_BACKTRACE=1` for a backtrace. error: internal compiler error: unexpected panic note: the compiler unexpectedly panicked. this is a bug. note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports note: rustc 1.27.0-dev running on x86_64-pc-windows-msvc note: compiler flags: -Z unpretty=normal -Z unstable-options -C rpath ------------------------------------------ thread '[pretty] pretty\block-comment-wchar.rs' panicked at 'explicit panic', tools\compiletest\src\runtest.rs:3033:9 note: Run with `RUST_BACKTRACE=1` for a backtrace. failures: [pretty] pretty\block-comment-wchar.rs test result: FAILED. 46 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out

Because `bump()` is hot.

This patch removes the "old"/"new" names in favour of "foo"/"next_foo", which matches the field names. It also moves the setting of `self.{ch,pos,next_pos}` in the common case to the end, so that the meaning of "foo"/"next_foo" is consistent until the end.

- `source_text` becomes `src`, matching `FileMap::src`. - `byte_offset()` becomes `src_index()`, which makes it clearer that it's an index into `src`. (Likewise for variables containing `byte_offset` in their name.) This function also now returns a `usize` instead of a `BytePos`, because every callsite immediately converted the `BytePos` to a `usize`.

It's silly for a hot function like `bump()` to have such an expensive bounds check. This patch replaces terminator with `end_src_index`. Note that the `self.terminator` check in `is_eof()` wasn't necessary because of the way `StringReader` is initialized.

petrochenkov · 2018-05-14T10:34:10Z

@bors r+

bors · 2018-05-14T10:34:11Z

📌 Commit e913d69 has been approved by petrochenkov

bors · 2018-05-17T10:49:40Z

⌛ Testing commit e913d69 with merge a3412980525c11719c991b27b003e0d2c4a44e0f...

bors · 2018-05-17T11:56:16Z

💔 Test failed - status-travis

rust-highfive · 2018-05-17T11:56:18Z

The job dist-i686-apple of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

[00:03:20]       Memory: 8 GB
[00:03:20]       Boot ROM Version: VMW71.00V.0.B64.1704110547
[00:03:20]       Apple ROM Info: [MS_VM_CERT/SHA1/27d66596a61c48dd3dc7216fd715126e33f59ae7]Welcome to the Virtual Machine
[00:03:20]       SMC Version (system): 2.8f0
[00:03:20]       Serial Number (system): VMh02kWn2/xG
[00:03:20] 
[00:03:20] hw.ncpu: 4
[00:03:20] hw.byteorder: 1234
[00:03:20] hw.memsize: 8589934592
---
  0     0    0     0    0     0      0      0 --:--:--  0:01:15 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:01:16 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:01:17 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:01:18 --:--:--     0curl: (7) Failed to connect to s3-us-west-1.amazonaws.com port 443: Operation timed out
[01:01:04] thread 'main' panicked at 'failed to download openssl source: exit code: 7', bootstrap/native.rs:575:17
[01:01:04] failed to run: /Users/travis/build/rust-lang/rust/build/bootstrap/debug/bootstrap build
[01:01:04] Build completed unsuccessfully in 0:48:12
[01:01:04] Build completed unsuccessfully in 0:48:12
[01:01:04] make: *** [all] Error 1

The command "stamp sh -x -c "$RUN_SCRIPT"" exited with 2.
travis_time:start:0fdde1e8
$ date && (curl -fs --head https://google.com | grep ^Date: | sed 's/Date: //g' || true)
---
travis_fold:start:after_failure.2
travis_time:start:0a4915e8
$ ls -lat $HOME/Library/Logs/DiagnosticReports/
total 0
drwx------+ 15 travis  staff  510 Jan 25 19:20 ..
drwx------   2 travis  staff   68 Dec  6 11:00 .
travis_fold:end:after_failure.2
travis_fold:start:after_failure.3
travis_time:start:0301de80
$ find $HOME/Library/Logs/DiagnosticReports -type f -name '*.crash' -not -name '*.stage2-*.crash' -not -name 'com.apple.CoreSimulator.CoreSimulatorService-*.crash' -exec printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" {} \; -exec head -750 {} \; -exec echo travis_fold":"end:crashlog \; || true
$ find $HOME/Library/Logs/DiagnosticReports -type f -name '*.crash' -not -name '*.stage2-*.crash' -not -name 'com.apple.CoreSimulator.CoreSimulatorService-*.crash' -exec printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" {} \; -exec head -750 {} \; -exec echo travis_fold":"end:crashlog \; || true
travis_time:end:0301de80:start=1526558106946739000,finish=1526558107008585000,duration=61846000
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:360c0f00
$ dmesg | grep -i kill
Unable to obtain kernel buffer: Operation not permitted
usage: sudo dmesg
travis_fold:end:after_failure.4

Done. Your build exited with 1.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

rust-highfive · 2018-05-17T11:56:23Z

The job dist-i686-apple of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

[00:03:20]       Memory: 8 GB
[00:03:20]       Boot ROM Version: VMW71.00V.0.B64.1704110547
[00:03:20]       Apple ROM Info: [MS_VM_CERT/SHA1/27d66596a61c48dd3dc7216fd715126e33f59ae7]Welcome to the Virtual Machine
[00:03:20]       SMC Version (system): 2.8f0
[00:03:20]       Serial Number (system): VMh02kWn2/xG
[00:03:20] 
[00:03:20] hw.ncpu: 4
[00:03:20] hw.byteorder: 1234
[00:03:20] hw.memsize: 8589934592
---
  0     0    0     0    0     0      0      0 --:--:--  0:01:15 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:01:16 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:01:17 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:01:18 --:--:--     0curl: (7) Failed to connect to s3-us-west-1.amazonaws.com port 443: Operation timed out
[01:01:04] thread 'main' panicked at 'failed to download openssl source: exit code: 7', bootstrap/native.rs:575:17
[01:01:04] failed to run: /Users/travis/build/rust-lang/rust/build/bootstrap/debug/bootstrap build
[01:01:04] Build completed unsuccessfully in 0:48:12
[01:01:04] Build completed unsuccessfully in 0:48:12
[01:01:04] make: *** [all] Error 1

The command "stamp sh -x -c "$RUN_SCRIPT"" exited with 2.
travis_time:start:0fdde1e8
$ date && (curl -fs --head https://google.com | grep ^Date: | sed 's/Date: //g' || true)
---
travis_fold:start:after_failure.2
travis_time:start:0a4915e8
$ ls -lat $HOME/Library/Logs/DiagnosticReports/
total 0
drwx------+ 15 travis  staff  510 Jan 25 19:20 ..
drwx------   2 travis  staff   68 Dec  6 11:00 .
travis_fold:end:after_failure.2
travis_fold:start:after_failure.3
travis_time:start:0301de80
$ find $HOME/Library/Logs/DiagnosticReports -type f -name '*.crash' -not -name '*.stage2-*.crash' -not -name 'com.apple.CoreSimulator.CoreSimulatorService-*.crash' -exec printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" {} \; -exec head -750 {} \; -exec echo travis_fold":"end:crashlog \; || true
$ find $HOME/Library/Logs/DiagnosticReports -type f -name '*.crash' -not -name '*.stage2-*.crash' -not -name 'com.apple.CoreSimulator.CoreSimulatorService-*.crash' -exec printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" {} \; -exec head -750 {} \; -exec echo travis_fold":"end:crashlog \; || true
travis_time:end:0301de80:start=1526558106946739000,finish=1526558107008585000,duration=61846000
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:360c0f00
$ dmesg | grep -i kill
Unable to obtain kernel buffer: Operation not permitted
usage: sudo dmesg
travis_fold:end:after_failure.4

Done. Your build exited with 1.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

kennytm · 2018-05-17T12:02:37Z

@bors retry #40474

bors · 2018-05-17T19:32:36Z

⌛ Testing commit e913d69 with merge 9a095e285785af647046c46a5e55edb12ca1bbb0...

Mark-Simulacrum · 2018-05-17T19:51:51Z

@bors p=15

bors · 2018-05-17T21:36:38Z

💔 Test failed - status-appveyor

petrochenkov · 2018-05-17T21:51:19Z

[02:01:54] Uploading artifact "9a095e285785af647046c46a5e55edb12ca1bbb0/rust-docs-nightly-i686-pc-windows-gnu.tar.gz" (0 bytes) to S3 bucket "rust-lang-ci2" as "rustc-builds/9a095e285785af647046c46a5e55edb12ca1bbb0/rust-docs-nightly-i686-pc-windows-gnu.tar.gz"...The difference between the request time and the current time is too large.

@bors retry

bors · 2018-05-18T00:09:43Z

⌛ Testing commit e913d69 with merge bedbf72...

Streamline `StringReader::bump` These patches make `bump` smaller and nicer. They speed up most runs for coercions and tuple-stress by 1--3%.

bors · 2018-05-18T02:39:26Z

☀️ Test successful - status-appveyor, status-travis
Approved by: petrochenkov
Pushing bedbf72 to master...

rust-highfive assigned petrochenkov May 9, 2018

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label May 9, 2018

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 11, 2018

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels May 12, 2018

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 12, 2018

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels May 12, 2018

kennytm added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 12, 2018

kennytm reviewed May 12, 2018

View reviewed changes

nnethercote added 5 commits May 13, 2018 17:16

Inline char_at() and record_width.

4465b2f

Because `bump()` is hot.

Make nextnextch() more closely resemble nextch().

444b770

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 14, 2018

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels May 17, 2018

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 17, 2018

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels May 17, 2018

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 17, 2018

bors added a commit that referenced this pull request May 18, 2018

Auto merge of #50566 - nnethercote:bump, r=petrochenkov

bedbf72

Streamline `StringReader::bump` These patches make `bump` smaller and nicer. They speed up most runs for coercions and tuple-stress by 1--3%.

bors merged commit e913d69 into rust-lang:master May 18, 2018

This was referenced May 18, 2018

add suggestion applicabilities to librustc and libsyntax #50724

Merged

rustc: Fix joint-ness of stringified token-streams #50838

Merged

nnethercote deleted the bump branch May 18, 2018 04:33

Streamline StringReader::bump #50566

Streamline StringReader::bump #50566

Uh oh!

Conversation

nnethercote commented May 9, 2018

Uh oh!

rust-highfive commented May 9, 2018

Uh oh!

petrochenkov commented May 11, 2018

Uh oh!

bors commented May 11, 2018

Uh oh!

bors commented May 12, 2018

Uh oh!

bors commented May 12, 2018

Uh oh!

rust-highfive commented May 12, 2018

Uh oh!

rust-highfive commented May 12, 2018

Uh oh!

kennytm commented May 12, 2018

Uh oh!

bors commented May 12, 2018

Uh oh!

bors commented May 12, 2018

Uh oh!

kennytm May 12, 2018

Choose a reason for hiding this comment

Uh oh!

petrochenkov commented May 14, 2018

Uh oh!

bors commented May 14, 2018

Uh oh!

bors commented May 17, 2018

Uh oh!

bors commented May 17, 2018

Uh oh!

rust-highfive commented May 17, 2018

Uh oh!

rust-highfive commented May 17, 2018

Uh oh!

kennytm commented May 17, 2018

Uh oh!

bors commented May 17, 2018

Uh oh!

Mark-Simulacrum commented May 17, 2018

Uh oh!

bors commented May 17, 2018

Uh oh!

petrochenkov commented May 17, 2018

Uh oh!

bors commented May 18, 2018

Uh oh!

bors commented May 18, 2018

Uh oh!

Uh oh!

Streamline `StringReader::bump` #50566

Streamline `StringReader::bump` #50566