Smaller and more correct generator codegen #69814

jonas-schievink · 2020-03-08T01:07:08Z

This removes unnecessary panicking branches in the resume function when the generator can not return or unwind, respectively.

Closes #66100

It also addresses the correctness concerns wrt poisoning on unwind. These are not currently a soundness issue because any operation inside a generator that could possibly unwind will result in a cleanup path for dropping it, ultimately reaching a Resume terminator, which we already handled correctly. Future MIR optimizations might optimize that out, though.

r? @Zoxc

This didn't cause issues before since generator types were always considered to "need drop", leading to unwind paths (including a `Resume` block) always getting generated.

The indices do not matter here, and this fixes an index out of bounds panic when compiling a generator that can unwind but not return.

src/librustc_mir/transform/generator.rs

Co-Authored-By: bjorn3 <bjorn3@users.noreply.github.com>

Zoxc · 2020-03-08T16:33:45Z

You should be able to write a test that generators can't be resumed once unwound (covering the case fixed here).

jonas-schievink · 2020-03-08T16:58:52Z

Hmm, not sure how I'd do that. Besides a call to box_free I haven't encountered any terminator with unwind = None, and that doesn't unwind (or rather, it's UB if the global allocator unwinds).

src/test/ui/generator/panic-safe.rs tests this case already (but works with the previous code as well, since poisoning on Resume is enough here).

Zoxc · 2020-03-14T14:19:29Z

src/librustc_mir/transform/generator.rs

+            1,
+            (POISONED, insert_panic_block(tcx, body, ResumedAfterPanic(generator_kind))),
+        );
+    }


Maybe swap the order of inserts here so the same ordering is kept.

bjorn3 · 2020-03-14T14:53:56Z

src/librustc_mir/transform/generator.rs

+
+    if can_return {
+        cases.insert(
+            1,


If can_unwind and can_return, this will overwrite the POISONED case.

insert on a Vec can't overwrite.

It should be moved to index 2 though, right?

Ah, missed that reply

I thought cases was an FxHashMap.

Zoxc · 2020-03-14T16:30:22Z

@bors r+

bors · 2020-03-14T16:30:24Z

📌 Commit 38fa378 has been approved by Zoxc

jonas-schievink · 2020-03-14T23:26:49Z

I'm currently looking into making needs_drop smarter when called on a generator, and it seems that that does indeed expose the soundness problem here: ui/async-await/issues/issue-65419/issue-65419-async-fn-resume-after-panic.rs starts failing because the generator isn't poisoned when it panics (since no destructors need to be called, no unwind path is created, so the Resume terminator is missing).

@Zoxc

Smaller and more correct generator codegen This removes unnecessary panicking branches in the resume function when the generator can not return or unwind, respectively. Closes rust-lang#66100 It also addresses the correctness concerns wrt poisoning on unwind. These are not currently a soundness issue because any operation *inside* a generator that could possibly unwind will result in a cleanup path for dropping it, ultimately reaching a `Resume` terminator, which we already handled correctly. Future MIR optimizations might optimize that out, though. r? @Zoxc

@Zoxc

Smaller and more correct generator codegen This removes unnecessary panicking branches in the resume function when the generator can not return or unwind, respectively. Closes rust-lang#66100 It also addresses the correctness concerns wrt poisoning on unwind. These are not currently a soundness issue because any operation *inside* a generator that could possibly unwind will result in a cleanup path for dropping it, ultimately reaching a `Resume` terminator, which we already handled correctly. Future MIR optimizations might optimize that out, though. r? @Zoxc

@ghost

Rollup of 9 pull requests Successful merges: - #69036 (rustc: don't resolve Instances which would produce malformed shims.) - #69443 (tidy: Better license checks.) - #69814 (Smaller and more correct generator codegen) - #69929 (Regenerate tables for Unicode 13.0.0) - #69959 (std: Don't abort process when printing panics in tests) - #69969 (unix: Set a guard page at the end of signal stacks) - #70005 ([rustdoc] Improve visibility for code blocks warnings) - #70088 (Use copy bound in atomic operations to generate simpler MIR) - #70095 (Implement -Zlink-native-libraries) Failed merges: r? @ghost

@ghost

Rollup of 9 pull requests Successful merges: - #68941 (Properly handle Spans that reference imported SourceFiles) - #69036 (rustc: don't resolve Instances which would produce malformed shims.) - #69443 (tidy: Better license checks.) - #69814 (Smaller and more correct generator codegen) - #69929 (Regenerate tables for Unicode 13.0.0) - #69959 (std: Don't abort process when printing panics in tests) - #69969 (unix: Set a guard page at the end of signal stacks) - #70005 ([rustdoc] Improve visibility for code blocks warnings) - #70088 (Use copy bound in atomic operations to generate simpler MIR) Failed merges: r? @ghost

…atthewjasper Make `needs_drop` less pessimistic on generators Generators only have non-trivial drop logic when they may store (in upvars or across yields) a type that does. This prevents generation of some unnecessary MIR in simple generators. There might be some impact on compile times, but this is probably limited in real-world applications. ~~This builds off of rust-lang#69814 since that contains some fixes that are made relevant by *this* PR (see rust-lang#69814 (comment) (this has been merged)

…thewjasper Make `needs_drop` less pessimistic on generators Generators only have non-trivial drop logic when they may store (in upvars or across yields) a type that does. This prevents generation of some unnecessary MIR in simple generators. There might be some impact on compile times, but this is probably limited in real-world applications. ~~This builds off of rust-lang#69814 since that contains some fixes that are made relevant by *this* PR (see rust-lang#69814 (comment) (this has been merged)

jonas-schievink added 5 commits March 7, 2020 21:29

Don't insert panic when generator can not return

425e7e5

Don't insert panic when generator can not unwind

5ac41a1

Poison generators when any terminator unwinds

46aeef6

This didn't cause issues before since generator types were always considered to "need drop", leading to unwind paths (including a `Resume` block) always getting generated.

Change index for SwitchInt case

8d9f633

The indices do not matter here, and this fixes an index out of bounds panic when compiling a generator that can unwind but not return.

Add test for unnecessary panic branches

2cbccce

rust-highfive assigned Zoxc Mar 8, 2020

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Mar 8, 2020

jonas-schievink changed the title ~~Smaller and more correct generator code~~ Smaller and more correct generator codegen Mar 8, 2020

bjorn3 reviewed Mar 8, 2020

View reviewed changes

src/librustc_mir/transform/generator.rs Outdated Show resolved Hide resolved

src/librustc_mir/transform/generator.rs Outdated Show resolved Hide resolved

Apply suggestions from code review

cb58de8

Co-Authored-By: bjorn3 <bjorn3@users.noreply.github.com>

Zoxc reviewed Mar 14, 2020

View reviewed changes

Swap inserts to keep the original ordering

38fa378

bjorn3 reviewed Mar 14, 2020

View reviewed changes

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 14, 2020

jonas-schievink mentioned this pull request Mar 15, 2020

Make needs_drop less pessimistic on generators #70015

Merged

Centril mentioned this pull request Mar 18, 2020

Rollup of 10 pull requests #70110

Closed

Centril mentioned this pull request Mar 19, 2020

Rollup of 9 pull requests #70130

Closed

Centril mentioned this pull request Mar 19, 2020

Rollup of 9 pull requests #70132

Merged

bors merged commit 5570a23 into rust-lang:master Mar 19, 2020

SimonSapin mentioned this pull request Mar 20, 2020

Importing modules from inside include! no longer works #70185

Closed

jonas-schievink deleted the gen-ret-unw branch April 17, 2020 18:36

jonas-schievink mentioned this pull request May 16, 2020

Stack space in async closures is not shared between branches in debug builds #72247

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Smaller and more correct generator codegen #69814

Smaller and more correct generator codegen #69814

Uh oh!

jonas-schievink commented Mar 8, 2020

Uh oh!

Uh oh!

Uh oh!

Zoxc commented Mar 8, 2020

Uh oh!

jonas-schievink commented Mar 8, 2020

Uh oh!

Zoxc Mar 14, 2020

Uh oh!

bjorn3 Mar 14, 2020

Uh oh!

Zoxc Mar 14, 2020

Uh oh!

jonas-schievink Mar 14, 2020

Uh oh!

jonas-schievink Mar 14, 2020

Uh oh!

bjorn3 Mar 14, 2020

Uh oh!

Zoxc commented Mar 14, 2020

Uh oh!

bors commented Mar 14, 2020

Uh oh!

jonas-schievink commented Mar 14, 2020

Uh oh!

Uh oh!

Smaller and more correct generator codegen #69814

Smaller and more correct generator codegen #69814

Uh oh!

Conversation

jonas-schievink commented Mar 8, 2020

Uh oh!

Uh oh!

Uh oh!

Zoxc commented Mar 8, 2020

Uh oh!

jonas-schievink commented Mar 8, 2020

Uh oh!

Zoxc Mar 14, 2020

Choose a reason for hiding this comment

Uh oh!

bjorn3 Mar 14, 2020

Choose a reason for hiding this comment

Uh oh!

Zoxc Mar 14, 2020

Choose a reason for hiding this comment

Uh oh!

jonas-schievink Mar 14, 2020

Choose a reason for hiding this comment

Uh oh!

jonas-schievink Mar 14, 2020

Choose a reason for hiding this comment

Uh oh!

bjorn3 Mar 14, 2020

Choose a reason for hiding this comment

Uh oh!

Zoxc commented Mar 14, 2020

Uh oh!

bors commented Mar 14, 2020

Uh oh!

jonas-schievink commented Mar 14, 2020

Uh oh!

Uh oh!