Skip to content

High memory usage with cargo build in match statement #131410

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Open
Joshix-1 opened this issue Oct 8, 2024 · 6 comments
Open

High memory usage with cargo build in match statement #131410

Joshix-1 opened this issue Oct 8, 2024 · 6 comments
Labels
A-patterns Relating to patterns and pattern matching C-bug Category: This is a bug. I-compilemem Issue: Problems and improvements with respect to memory usage during compilation. I-crash Issue: The compiler crashes (SIGSEGV, SIGABRT, etc). Use I-ICE instead when the compiler panics. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Comments

@Joshix-1
Copy link

Joshix-1 commented Oct 8, 2024

The memory usage gets high really quickly which causes my pc to crash.

Code

Full git repo: https://github.com/Joshix-1/cargo-memory-leak
https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=03d0fd2c2ceeb588e5297389a01f9fa9

#[derive(Copy, Clone, Debug, Eq, PartialEq)]
#[repr(u8)]
pub enum FieldType {
    Air,
    Wood,
    SandSource,
    BlackHole,
    SandC0, SandC1, SandC2, SandC3, SandC4, SandC5, SandC6, SandC7,
    WaterC0, WaterC1, WaterC2, WaterC3, WaterC4, WaterC5, WaterC6, WaterC7,
}

macro_rules! sand { () => {
    FieldType::SandC0 | FieldType::SandC1 | FieldType::SandC2
    | FieldType::SandC3 | FieldType::SandC4 | FieldType::SandC5
    | FieldType::SandC6 | FieldType::SandC7
}; }

macro_rules! water { () => {
    FieldType::WaterC0 | FieldType::WaterC1 | FieldType::WaterC2
    | FieldType::WaterC3 | FieldType::WaterC4 | FieldType::WaterC5
    | FieldType::WaterC6 | FieldType::WaterC7
}; }

macro_rules! falls {() => { sand!() | water!() };}

macro_rules! not_solid {
    () => { FieldType::Air | water!() };
}

impl FieldType {
    pub const fn is_sand(self) -> bool { matches!(self, sand!()) }

    pub const fn is_water(self) -> bool { matches!(self, water!()) }
}

fn main() {
    let arr = &mut [FieldType::Air; 4];
    let [ref mut a, ref mut b, ref mut c, ref mut d] = arr; 
    let cell: (
        (&mut FieldType, &mut FieldType),
        (&mut FieldType, &mut FieldType),
    ) = ((a, b), (c, d));
    match cell {
        (
            (sand0 @ falls!(), sand1 @ falls!()),
            (unsolid0 @ not_solid!(), unsolid1 @ not_solid!()),
            // compiles without the if in the next line
        ) if unsolid0.is_water() != sand0.is_water() || unsolid1.is_water() != sand1.is_water() => {
            if unsolid0.is_water() != sand0.is_water() {
                (*sand0, *unsolid0) = (*unsolid0, *sand0);
            }
            if unsolid1.is_water() != sand1.is_water() {
                (*sand1, *unsolid1) = (*unsolid1, *sand1);
            }
        }
        _ => {}
    }
}

Meta

rustc --version --verbose:

 rustc --version --verbose
rustc 1.81.0 (eeb90cda1 2024-09-04)
binary: rustc
commit-hash: eeb90cda1969383f56a2637cbd3037bdf598841c
commit-date: 2024-09-04
host: x86_64-unknown-linux-gnu
release: 1.81.0
LLVM version: 18.1.7

also on playground, nightly and stable

Error output

$ env RUST_BACKTRACE=1 cargo build 2> build.err > build.out
# crash of xserver
$ du -bs build.*
49	build.err
0	build.out
$ cat build.err
   Compiling sandrs v0.1.1 (/tmp/tmp.aULMeIa66O)
@Joshix-1 Joshix-1 added C-bug Category: This is a bug. I-ICE Issue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️ T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Oct 8, 2024
@rustbot rustbot added the needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. label Oct 8, 2024
@matthiaskrgr
Copy link
Member

apparently 64 gb are not enough for this to compile 🙃
in perf top I can see some action in _RNvMNtCs8XS4uvkoLYP_11rustc_index8intervalINtB2_11IntervalSetNtNtCsb3wWoxYnthl_18rustc_mir_dat

@matthiaskrgr matthiaskrgr added I-crash Issue: The compiler crashes (SIGSEGV, SIGABRT, etc). Use I-ICE instead when the compiler panics. I-compilemem Issue: Problems and improvements with respect to memory usage during compilation. and removed I-ICE Issue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️ labels Oct 8, 2024
@lqd
Copy link
Member

lqd commented Oct 8, 2024

cc @Nadrieril

@matthiaskrgr
Copy link
Member

digging around a bit with godbolt reveals that this seems to OOM since 1.59, presumably since destructuring_assignment was stabilised #71126

@saethlin saethlin added A-patterns Relating to patterns and pattern matching and removed needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. labels Oct 9, 2024
@Joshix-1
Copy link
Author

Joshix-1 commented Oct 9, 2024

I've reduced the example a bit more. It's now just two match arms

@Nadrieril
Copy link
Member

Nadrieril commented Oct 9, 2024

For this we generate 16*16*9*9 = 21000 branches in MIR, each with their own copy of the if unsolid0.is_water() != sand0.is_water() || ... match guard. I don't know which analysis pass eats all the RAM but I'm also not surprised that this would happen with 21000 branches.

Question is: can we do better? In theory yes, but this is tricky. I think that would require match lowering to be able to tell which patterns are disjoint or something like that.

@fr0xk
Copy link

fr0xk commented Nov 29, 2024

apparently 64 gb are not enough for this to compile 🙃 in perf top I can see some action in _RNvMNtCs8XS4uvkoLYP_11rustc_index8intervalINtB2_11IntervalSetNtNtCsb3wWoxYnthl_18rustc_mir_dat

What? That's ridiculously huge requirement

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
A-patterns Relating to patterns and pattern matching C-bug Category: This is a bug. I-compilemem Issue: Problems and improvements with respect to memory usage during compilation. I-crash Issue: The compiler crashes (SIGSEGV, SIGABRT, etc). Use I-ICE instead when the compiler panics. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

No branches or pull requests

7 participants