Skip to content

"./x.py test --stage 0 --doc" does not run doc tests #73286

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom
Closed
RalfJung opened this issue Jun 12, 2020 · 9 comments · Fixed by #73411
Closed

"./x.py test --stage 0 --doc" does not run doc tests #73286

RalfJung opened this issue Jun 12, 2020 · 9 comments · Fixed by #73411
Labels
A-doctests Area: Documentation tests, run by rustdoc T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)

Comments

@RalfJung
Copy link
Member

I would expect the following command to run the libcore doc tests:

./x.py test --stage 0 src/libcore --doc

This used to work, but now all it does it build libcore and its test binaries. No tests are actually being run (which I verified by breaking one of them deliberately).

@Mark-Simulacrum any idea what is happening?
@RalfJung RalfJung mentioned this issue Jun 12, 2020
Merged
@jonas-schievink jonas-schievink added A-doctests Area: Documentation tests, run by rustdoc T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Jun 12, 2020
@ehuss
Copy link
Contributor

ehuss commented Jun 12, 2020

I can confirm, doctests are not being run for std. I think a bigger question is, have they ever worked? Cargo will skip doctests whenever --target is used, which AFAIK is how std is always built/tested.

@RalfJung
Copy link
Member Author

I am pretty sure that have used this command before and got test failures when I had screwed up. If you want, I can go back to older versions and verify.

@ehuss
Copy link
Contributor

ehuss commented Jun 12, 2020

I'll try to figure out what's going on.

I tried running with -Zdoctest-xcompile and fortunately nothing looks broken.

It's also very possible I am missing something, but I edited some doctests with errors, and a bare ./x.py test doesn't fail.

@ehuss
Copy link
Contributor

ehuss commented Jun 12, 2020 •
edited
Loading

I see what's going on. It's a consequence of rust-lang/cargo#6892. That means doctests haven't been running since 1.39. This was wrong, see below.

I'll try to put together a fix.

@RalfJung
Copy link
Member Author

That means doctests haven't been running since 1.39.

Ouch. Is there a way we can ensure this does not happen again in the future? Looks like our test suite needs a test suite...

@Mark-Simulacrum
Copy link
Member

It's pretty hard :/

I guess we can add a dummy doc test that writes some "well known" file and we can assert that the file is present -- perhaps do this per-crate.

@ehuss ehuss mentioned this issue Jun 14, 2020
Merged
@ehuss
Copy link
Contributor

ehuss commented Jun 14, 2020

OK, it wasn't as bad as I originally reported. This regressed via rust-lang/cargo#8167 which is only in 1.45 (beta). I have a fix started at rust-lang/cargo#8358. This will require various steps:

  1. Fixing cargo.
  2. Backport to beta.
  3. Update cargo on beta.
  4. Wait for new beta release.
  5. Update bootstrap on nightly.

@RalfJung
Copy link
Member Author

Well, but it does mean that currently on CI we do not run doctests, right? --stage 1 does not change this.

@ehuss
Copy link
Contributor

ehuss commented Jun 14, 2020

Right, I just mean it hasn't been that long (only about a week).

bors added a commit to rust-lang/cargo that referenced this issue Jun 15, 2020
@bors
Auto merge of #8358 - ehuss:fix-target-host-doctest, r=alexcrichton
4e14cc1 
Fix doctests not running with --target=HOST.

There was a regression in #8167 where `cargo test --target=$HOST` stopped running doctests. This caused doctests to silently stop running in rust-lang/rust (rust-lang/rust#73286).  This PR restores the original behavior where `--target=$HOST` behaves as-if it is a normal host test.

There was a discussion about this at #8167 (review), but I think I let it slip through the cracks.
@ehuss ehuss mentioned this issue Jun 16, 2020
Merged
Manishearth added a commit to Manishearth/rust that referenced this issue Jun 20, 2020
@Manishearth
Rollup merge of rust-lang#73411 - ehuss:bump-stage0, r=Mark-Simulacrum
ccef1f5 
Update bootstrap to rustc 1.45.0-beta.2 (1dc0f6d 2020-06-15)

Pulls in changes from rust-lang#73326.

Closes rust-lang#73286
Manishearth added a commit to Manishearth/rust that referenced this issue Jun 20, 2020
@Manishearth
Rollup merge of rust-lang#73411 - ehuss:bump-stage0, r=Mark-Simulacrum
abd4096 
Update bootstrap to rustc 1.45.0-beta.2 (1dc0f6d 2020-06-15)

Pulls in changes from rust-lang#73326.

Closes rust-lang#73286
@bors bors closed this as completed in 9a82736 Jun 21, 2020
# for free to join this conversation on GitHub. Already have an account? # to comment
Assignees
No one assigned
Labels
A-doctests Area: Documentation tests, run by rustdoc T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Update bootstrap to rustc 1.45.0-beta.2 (1dc0f6d8e 2020-06-15) ehuss/rust
4 participants
@ehuss @RalfJung @jonas-schievink @Mark-Simulacrum