switch to new LLVM PassManager

[andrewrk]

Upstream, LLVM is transitioning to a new PassManager, and they plan to remove support for the old one. See #8390 for a nice writeup with more details.

So now is the time for us to find problems with it, and get bugs reported upstream before it's too late.

I already tried merging #8390 into the llvm12 branch and found 2 problems:

---

Problem 1. The following test fails:

test.zig

export fn main() i32 {
    @setEvalBranchQuota(1001);
    const y = rec(1001);
    return y - 1;
}

fn rec(n: usize) callconv(.Inline) usize {
    if (n <= 1) return n;
    return rec(n - 1);
}

How to reproduce:

# build self hosted compiler
./zig build
./zig-cache/bin/zig build-exe test.zig

Expected: exit(0)
Actual: Segmentation fault in __zig_probe_stack

---

Problem 2. LLVM is 2x slower for debug builds. Noooooooooooooooo!

---

This issue is to file some friendly bug reports upstream, make sure they get fixed upstream, and then try re-enabling the new pass manager in Zig and see if we run into any more issues.

[mshockwave]

I have some good news for this issue.

TL;DR: I was building the -O0 Pass pipeline in a wrong way

Here is a longer story:
In #8390 , I was measuring the compilation time using our stage2 as the benchmark. More specifically, I was measuring the time (and its timing breakdown) of this command:

/path/to/my/zig build --prefix stage2 -Denable-llvm

But later on I realized that the above command only built stage2 with -O0. And indeed, new PM ran a lot slower in this configuration.
However, if we try to build stage2 with ReleaseFast (which is essentially -O3) via this command:

/path/to/my/zig build --prefix stage2 -Denable-llvm -Drelease=true

New PM actually runs ~20% faster than the legacy PM.

Long story short: With #8390 , New PM only performs poorly in -O0.

It turned out that I was calling the wrong function to build the -O0 Pass pipeline 😛 . More specifically, I should use PassBuilder::buildO0DefaultPipeline rather than PassBuilder::buildPerModuleDefaultPipeline. It is actually forbidden to build -O0 with buildPerModuleDefaultPipeline and LLVM does put an assertion to enforce this rule (but I never turned on assertion during this development so I didn't realize). After changing to buildO0DefaultPipeline, the compilation time of -O0 is the same as legacy PM.

So now we have normal compilation time in debug mode, faster compilation speed in ReleaseFast, and guess what? Problem 1 mentioned above does not crash anymore! Though now it shows the following error message:

test.zig:1:22: error: TODO implement set stack variable with large stack offset

But I also saw the same error message when using legacy PM on llvm12 branch, so I guess this is not caused by new PM.
In short, I think Problem 1 was simply caused by poorly configured -O0 Pass pipeline induced by invalid usage of buildPerModuleDefaultPipeline.

I'm happy to send another PR for this change if everyone here agree.

Also, while I was working on this bug, I added a preliminary support for using LLVM's time tracer in addition to the LLVM timer we are currently using. Time trace can give us a more hierarchical view of the timing breakdown (on the LLVM part). I can send another PR regarding this feature and file an issue to discuss some technical details (e.g. how to enable the time tracing).

[andrewrk]

Thanks for the follow-up.

I'm happy to send another PR for this change if everyone here agree.

Yes please!

Also, while I was working on this bug, I added a preliminary support for using LLVM's time tracer in addition to the LLVM timer we are currently using. Time trace can give us a more hierarchical view of the timing breakdown (on the LLVM part). I can send another PR regarding this feature and file an issue to discuss some technical details (e.g. how to enable the time tracing).

Yes let's look at this in a separate PR please :)

[mshockwave]

Thanks for the follow-up.

I'm happy to send another PR for this change if everyone here agree.

Yes please!

Just sent #8482

Also, while I was working on this bug, I added a preliminary support for using LLVM's time tracer in addition to the LLVM timer we are currently using. Time trace can give us a more hierarchical view of the timing breakdown (on the LLVM part). I can send another PR regarding this feature and file an issue to discuss some technical details (e.g. how to enable the time tracing).

Yes let's look at this in a separate PR please :)

Cool! I'll wait after #8482 is merged since this feature will modify the same function.