OOM with prerelease version

[davidmorgan]

Discovered here

#3834 (reply in thread)

[davidmorgan]

@amrgetment thanks for running with the extra logging.

I was hoping to see some output like "Asset graph size: ", but I don't see that in the output you just posted?

"Failed after 23m" ... 23m is huge, is that normal? i.e. did it take that long before?

A sample repro would be fantastic.

I'll anyway see if I can figure something out, and update here if I do.

Thanks a lot for your help :)

[amrgetment]

23m is too much, it should be like minutes or so
I will prepare a sample repo over the weekend

[davidmorgan]

@amrgetment I tried a bunch of big (1000 file) examples using retrofit_generator, getting generation time up to about 1m20s, but wasn't able to cause anything obviously bad with memory.

Are you using any other generators I should throw into the mix?

Another random question: is your code in one package, or does it split across multiple packages?

I'll keep trying things: I'm close to releasing the new version so any random testing I can do is useful.

Looking forward to that repro :)

[amrgetment]

I use a lot of generators:

  # Generators
  retrofit_generator: ^9.1.9
  build_runner: ^2.4.15
  auto_route_generator: ^10.0.1
  json_serializable: ^6.9.5
  assets_scanner_plus: ^2.0.0
  injectable_generator: ^2.7.0

My code is in a single package

[amrgetment]

My project almost covered all generators 😅

[davidmorgan]

Thanks! I'll take a look at them all. So far I added auto_route_generator, nothing suspicious yet.

I uploaded new version 15e775f which logs which generator is running, that might give us a clue :)

[davidmorgan]

@amrgetment Great news! I have a repro :)

No idea yet what triggered it, but I managed to hit a state where it gets stuck with RAM usage climbing forever until it fails with OOM.

So hopefully, no need for you to spend time on a repro now.

I will update here once I either have a fix, or have failed to find one :)

[amrgetment]

So it is not related to a specific generator, great 👍 , Good Luck

[amrgetment]

I think those logs need to be a part of --verbose for future needs

[davidmorgan]

I believe I have a fix :) ... #4004

Re logging: yes, I definitely plan on improving logging. I have a few ideas, and will try things; filed #4005 for improving logging.

[amrgetment]

I tried with ref: 23e0d98a7d3e46217ea12fff9724e2fab42c8c1f
2gb is Ok for me

[INFO] Generating build script completed, took 571ms
[INFO] Precompiling build script... completed, took 5.3s
[INFO] Building new asset graph completed, took 2.3s
[INFO] Checking for unexpected pre-existing outputs. completed, took 88ms
[INFO] Generating SDK summary completed, took 9.9s
[WARNING] No actions completed for 54.7s, waiting on:
  - freezed on lib/app_router.dart

[INFO] Running build completed, took 4m 10s
[INFO] Caching finalized dependency graph completed, took 1.9s
[INFO] Succeeded after 4m 12s with 723 outputs (15404 actions)

I will keep trying throughout the day, and if I find any issue ,I will report it back to you

[amrgetment]

@davidmorgan It seems to be working fine and no memory issue
I noticed that when I change a field name from foo to boo the time increases from 30 seconds to 90 seconds

╭─projectPath  fix/poo*› 
╰─➤  b
[INFO] Generating build script completed, took 571ms
[INFO] Precompiling build script... completed, took 5.3s
[INFO] Building new asset graph completed, took 2.3s
[INFO] Checking for unexpected pre-existing outputs. completed, took 88ms
[INFO] Generating SDK summary completed, took 9.9s
[WARNING] No actions completed for 54.7s, waiting on:
  - freezed on lib/app_router.dart
[INFO] Running build completed, took 4m 10s
[INFO] Caching finalized dependency graph completed, took 1.9s
[INFO] Succeeded after 4m 12s with 723 outputs (15404 actions)
╭─projectPath  fix/poo*› 
╰─➤  b
[INFO] Generating build script completed, took 617ms
[INFO] Reading cached asset graph completed, took 1.3s
[INFO] Checking for updates since last build completed, took 3.7s
[INFO] Updating asset graph completed, took 25ms
[INFO] Running build completed, took 29.0s
[INFO] Caching finalized dependency graph completed, took 817ms
[INFO] Succeeded after 29.8s with 0 outputs (0 actions)
╭─projectPath  fix/poo*› 
╰─➤  b
[INFO] Generating build script completed, took 687ms
[INFO] Reading cached asset graph completed, took 1.2s
[INFO] Checking for updates since last build completed, took 1.7s
[INFO] Updating asset graph completed, took 27ms
[WARNING] No actions completed for 38.4s, waiting on:
  - freezed on lib/route.dart

[WARNING] No actions completed for 20.5s, waiting on:

[INFO] Running build completed, took 1m 24s
[INFO] Caching finalized dependency graph completed, took 880ms
[INFO] Succeeded after 1m 25s with 20 outputs (270 actions)
╭─projectPath  fix/poo*› 
╰─➤  b
[INFO] Generating build script completed, took 555ms
[INFO] Reading cached asset graph completed, took 1.2s
[INFO] Checking for updates since last build completed, took 3.4s
[INFO] Updating asset graph completed, took 4ms
[INFO] Running build completed, took 26.9s
[INFO] Caching finalized dependency graph completed, took 764ms
[INFO] Succeeded after 27.7s with 0 outputs (0 actions)
╭─projectPath  fix/poo*› 
╰─➤  b
[INFO] Generating build script completed, took 580ms
[INFO] Reading cached asset graph completed, took 1.1s
[INFO] Checking for updates since last build completed, took 1.7s
[INFO] Updating asset graph completed, took 20ms
[INFO] Running build completed, took 26.5s
[INFO] Caching finalized dependency graph completed, took 974ms
[INFO] Succeeded after 27.5s with 0 outputs (0 actions)
╭─projectPath  fix/poo*› 
╰─➤  b
[INFO] Generating build script completed, took 666ms
[INFO] Reading cached asset graph completed, took 1.4s
[INFO] Checking for updates since last build completed, took 1.8s
[INFO] Updating asset graph completed, took 32ms
[WARNING] No actions completed for 48.7s, waiting on:
  - freezed on lib/route.dart

[WARNING] No actions completed for 24.7s, waiting on:

[INFO] Running build completed, took 1m 33s
[INFO] Caching finalized dependency graph completed, took 1.2s
[INFO] Succeeded after 1m 34s with 1 outputs (37 actions)
╭─projectPath  fix/poo*› 
╰─➤  b
[INFO] Generating build script completed, took 722ms
[INFO] Reading cached asset graph completed, took 2.0s
[INFO] Checking for updates since last build completed, took 5.9s
[INFO] Updating asset graph completed, took 56ms
[WARNING] No actions completed for 18.4s, waiting on:
  - freezed on lib/route.dart

[WARNING] No actions completed for 33.1s, waiting on:
  - freezed on lib/route.dart

[WARNING] No actions completed for 33.9s, waiting on:

[INFO] Running build completed, took 1m 47s
[INFO] Caching finalized dependency graph completed, took 1.2s
[INFO] Succeeded after 1m 48s with 1 outputs (42 actions)
╭─projectPath  fix/poo*› 
╰─➤  b
[INFO] Generating build script completed, took 611ms
[INFO] Reading cached asset graph completed, took 1.3s
[INFO] Checking for updates since last build completed, took 4.8s
[INFO] Updating asset graph completed, took 23ms
[WARNING] No actions completed for 15.4s, waiting on:
  - freezed on lib/route.dart

[WARNING] No actions completed for 36.7s, waiting on:
  - freezed on lib/route.dart

[WARNING] No actions completed for 26.2s, waiting on:

[INFO] Running build completed, took 1m 38s
[INFO] Caching finalized dependency graph completed, took 1.4s
[INFO] Succeeded after 1m 40s with 1 outputs (42 actions)
╭─projectPath  fix/poo*› 
╰─➤  b
[INFO] Generating build script completed, took 565ms
[INFO] Reading cached asset graph completed, took 1.2s
[INFO] Checking for updates since last build completed, took 3.6s
[INFO] Updating asset graph completed, took 5ms
[INFO] Running build completed, took 34.4s
[INFO] Caching finalized dependency graph completed, took 1.9s
[INFO] Succeeded after 36.3s with 0 outputs (0 actions)

[davidmorgan]

Re: where the time is going, I plan to add logging that should make that clearer, before releasing :) that way any feedback will be considerably more useful.