Skip to content

On demandify reachability #40873

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Merged
merged 1 commit into from
Apr 7, 2017
Merged

Conversation

cramertj
Copy link
Member

@cramertj cramertj commented Mar 28, 2017

cc #40746

I tried following this guidance from #40746:

The following tasks currently execute before a tcx is built, but they could be easily converted into queries that are requested after tcx is built. The main reason they are the way they are was to avoid a gratuitious refcell (but using the refcell map seems fine)...

but the result of moving region_maps out of TyCtxt and into a query caused a lot of churn, and seems like it could potentially result in a rather large performance hit, since it means a dep-graph lookup on every use of region_maps (rather than just a field access). Possibly TyCtxt could store a RefCell<Option<RegionMap>> internally and use that to prevent repeat lookups, but that feels like it's duplicating the work of the dep-graph. @nikomatsakis What did you have in mind for this?

@rust-highfive
Copy link
Contributor

r? @eddyb

(rust_highfive has picked a reviewer for you, use r? to override)

@cramertj
Copy link
Member Author

r? @nikomatsakis

@rust-highfive rust-highfive assigned nikomatsakis and unassigned eddyb Mar 28, 2017
@nikomatsakis
Copy link
Contributor

Regarding the region-maps:

  • First, we should measure the performance. It may or may not be significant.

  • Second, it seems like it is becoming quite common to have things that are effectively "lazilly computed globals". Right now we are handling those uniformly (with the cache key being a crate number) but maybe it's worth treating them as an (encapsulated) Option field.

  • Third, I expect that some amount of local caching (i.e., if we access region_maps() many times in a loop, we can store it into a local variable) would solve the problem.

Certainly we should solve this problem, because I think this is roughly the pattern we want going forward. But I think it won't be hard to solve.

@eddyb
Copy link
Member

eddyb commented Mar 28, 2017

IMO we would have all the maps per-crate, and HashMap<(), V> should be quite efficient - even if it's not, we could use specialization.

@nikomatsakis
Copy link
Contributor

@eddyb seems reasonable, yes

@cramertj
Copy link
Member Author

cramertj commented Mar 29, 2017

@eddyb Should I take that to mean that I should leave it as-is for now (module fixing the tidy check)? Also, is there interest in having queries include a get_ref function or similar to access the result, rather than having to wrap in an Rc?

@bors
Copy link
Collaborator

bors commented Mar 29, 2017

☔ The latest upstream changes (presumably #40867) made this pull request unmergeable. Please resolve the merge conflicts.

@cramertj cramertj force-pushed the on-demandify-queries branch from 402a57a to 8f4b72e Compare March 29, 2017 05:57
@nikomatsakis
Copy link
Contributor

@cramertj Rc is better, I think, because it avoids refcell issues, and it's simpler to have all of the queries pass ownership.

@bors
Copy link
Collaborator

bors commented Mar 29, 2017

☔ The latest upstream changes (presumably #40540) made this pull request unmergeable. Please resolve the merge conflicts.

@cramertj cramertj force-pushed the on-demandify-queries branch 2 times, most recently from ed2f96b to afebb2b Compare April 4, 2017 14:40
@cramertj cramertj changed the title On demandify reachability and [WIP] region mapping On demandify reachability Apr 4, 2017
@cramertj
Copy link
Member Author

cramertj commented Apr 4, 2017

Removed region-mapping in favor of #41057

@nikomatsakis nikomatsakis reopened this Apr 4, 2017
@cramertj cramertj force-pushed the on-demandify-queries branch from afebb2b to aab2cca Compare April 4, 2017 14:46
@nikomatsakis
Copy link
Contributor

@bors r+

@bors
Copy link
Collaborator

bors commented Apr 6, 2017

📌 Commit aab2cca has been approved by nikomatsakis

frewsxcv added a commit to frewsxcv/rust that referenced this pull request Apr 6, 2017
…komatsakis

On demandify reachability

cc rust-lang#40746

I tried following this guidance from rust-lang#40746:
> The following tasks currently execute before a tcx is built, but they could be easily converted into queries that are requested after tcx is built. The main reason they are the way they are was to avoid a gratuitious refcell (but using the refcell map seems fine)...

but the result of moving `region_maps` out of `TyCtxt` and into a query caused a lot of churn, and seems like it could potentially result in a rather large performance hit, since it means a dep-graph lookup on every use of `region_maps` (rather than just a field access). Possibly `TyCtxt` could store a `RefCell<Option<RegionMap>>` internally and use that to prevent repeat lookups, but that feels like it's duplicating the work of the dep-graph. @nikomatsakis What did you have in mind for this?
@bors
Copy link
Collaborator

bors commented Apr 6, 2017

⌛ Testing commit aab2cca with merge 408c882...

@bors
Copy link
Collaborator

bors commented Apr 6, 2017

💔 Test failed - status-appveyor

@alexcrichton
Copy link
Member

alexcrichton commented Apr 7, 2017 via email

@bors
Copy link
Collaborator

bors commented Apr 7, 2017

⌛ Testing commit aab2cca with merge 4c59c92...

bors added a commit that referenced this pull request Apr 7, 2017
On demandify reachability

cc #40746

I tried following this guidance from #40746:
> The following tasks currently execute before a tcx is built, but they could be easily converted into queries that are requested after tcx is built. The main reason they are the way they are was to avoid a gratuitious refcell (but using the refcell map seems fine)...

but the result of moving `region_maps` out of `TyCtxt` and into a query caused a lot of churn, and seems like it could potentially result in a rather large performance hit, since it means a dep-graph lookup on every use of `region_maps` (rather than just a field access). Possibly `TyCtxt` could store a `RefCell<Option<RegionMap>>` internally and use that to prevent repeat lookups, but that feels like it's duplicating the work of the dep-graph. @nikomatsakis What did you have in mind for this?
@bors
Copy link
Collaborator

bors commented Apr 7, 2017

☀️ Test successful - status-appveyor, status-travis
Approved by: nikomatsakis
Pushing 4c59c92 to master...

@bors bors merged commit aab2cca into rust-lang:master Apr 7, 2017
@aidanhs
Copy link
Member

aidanhs commented Apr 7, 2017

This (spuriously) hung halfway through one of the OSX tests (guess which one), possibly related to #40571, cc @alexcrichton. It then recovered after about 45mins and continued.

@alexcrichton
Copy link
Member

@aidanhs the full logs for that build are super weird. I wonder if there's a few travis bugs in play here. I'm seeing no discontinuities in our logging (which all have timestamps), but I'm seeing stuff like:

[00:56:40]     Finished release [optimized] target(s) in 0.1 secs

[00:32:05]    Compiling rustc_llvm v0.0.0 (file:///Users/travis/build/rust-lang/rust/src/librustc_llvm)

[00:56:40] Copying stage0 std from stage0 (i686-apple-darwin -> i686-apple-darwin / i686-apple-darwin)

[00:32:11]    Compiling syntax_pos v0.0.0 (file:///Users/travis/build/rust-lang/rust/src/libsyntax_pos)

[00:56:41] Building stage0 test artifacts (i686-apple-darwin -> i686-apple-darwin)

[00:56:41]     Finished release [optimized] target(s) in 0.0 secs

[00:32:17]    Compiling rustc_errors v0.0.0 (file:///Users/travis/build/rust-lang/rust/src/librustc_errors)

No idea why it's jumping back and forth...

@cramertj cramertj deleted the on-demandify-queries branch September 21, 2017 16:10
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants