CMake: Tweak build order for D executables #3575
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
kinke
force-pushed
the
cmake_parallel
branch
3 times, most recently
from
1ef4502 to
0bdd8de
Compare
|
Nice, that's a good find. |
It's been bothering me for a while that building ldc2.exe makes poor use of parallelization, because the C++ parts are compiled first, and compiling the D frontend to a single object file, the step which takes *way* longer than any C++ file, gets scheduled very late. I've just stumbled upon this in the CMake docs for the custom-command dependencies: > If any dependency is an OUTPUT of another custom command in the same > directory (CMakeLists.txt file), CMake automatically brings the other > custom command into the target in which this command is built. And indeed, using an extra dummy custom-target inbetween makes ninja schedule the D object compilation very early in the process. On my Windows box with `ninja -j4 ldc2`, the wall-time is reduced from ~196 to ~133 secs! I couldn't get this to work with LDC_LINK_MANUALLY=ON, so I made this option now configurable on the CMake cmdline.
Primarily to compile ldc2-unittest with `-g -unittest`, independent from CMAKE_BUILD_TYPE. Also remove unused DDMD_LFLAGS.
As the unittest modules aren't compiled with optimizations anymore, the effect on overall compile time isn't as high anymore. On my box with 4 cores, compiling them all-at-once with a single core takes ~18 secs, whereas compiling them separately with 4 parallel jobs takes ~40 secs. So enforcing separate compilation should only pay off with lots of cores *and* if there aren't other build tasks that can be run in parallel.
Not sure why this hasn't caused problems until now, but using `-conf=ldc2_install.conf.in` to 'compile' the dummy module cannot work before object.d is installed to the referenced import dir. Using `module object;` makes the compiler skip the search for object.d.
kinke
force-pushed
the
cmake_parallel
branch
5 times, most recently
from
0c6f898 to
ae420bf
Compare
With recent LDC and DMD host compilers supporting -Xcc, defaulting to OFF would probably be more appropriate nowadays. For DMD, I don't know how to change the linker driver from the C to the C++ compiler (in order to link the appropriate C+ runtime library). Using ON, i.e., letting CMake link the D executables via the C++ compiler, is still useful for cross-compiling the compiler itself, e.g., for Android, where the NDK toolchain file takes care of setting up cross-linking.
E.g., -Xcc=-Wl,-blub used to expand to `cc -Wl -blub`, which is invalid. With this change, it's now properly forwarded as `cc -Wl,-blub`.
The lit testrunner infers 4 available CPUs, memory seems to be sufficient, so...
kinke
force-pushed
the
cmake_parallel
branch
2 times, most recently
from
0237d35 to
76af5b2
Compare
kinke
added a commit
to kinke/ldc
that referenced
this pull request
Oct 10, 2020
The ldc2 executable wasn't re-linked when only the D object file(s) changed. Fortunately, the custom target still suffices to improve parallelization.
kinke
added a commit
that referenced
this pull request
Oct 10, 2020
kinke
added a commit
to kinke/ldc
that referenced
this pull request
Oct 24, 2020
…mpiler and manual linking Under these circumstances, the `-link-debuglib` had no effect after ldc-developers#3575, so that the D executables were linked against *release* host druntime/ Phobos. Add the flag to DFLAGS_BASE instead, which is used by ExtractDMDSystemLinker.cmake and makes ldc2-unittest use it too again.
kinke
added a commit
that referenced
this pull request
Oct 24, 2020
…mpiler and manual linking (#3597) Under these circumstances, the `-link-debuglib` had no effect after #3575, so that the D executables were linked against *release* host druntime/ Phobos. Add the flag to DFLAGS_BASE instead, which is used by ExtractDMDSystemLinker.cmake and makes ldc2-unittest use it too again.
# for free
to join this conversation on GitHub.
Already have an account?
# to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
It's been bothering me for a while that building
ldc2.exemakes poor use of parallelization, because the C++ parts are compiled first, and compiling the D frontend to a single object file, the step which takes way longer than any C++ file, gets scheduled very late.I've just stumbled upon this in the CMake docs for the custom-command dependencies:
And indeed, using an extra dummy custom-target inbetween makes ninja schedule the D object compilation very early in the process. On my box with
ninja -j4 ldc2, the wall-time is reduced from ~196 to ~133 secs!