Trivial Changes to `spack.yaml` Require Rebuild of Some Packages #12

CodeGat · 2024-09-23T04:52:19Z

Currently, doing trivial modifications to the spack.yaml (seen most prominently in #5) require rebuilding parallelio, esmf and (therefore) access-om3-nuopc. This is a bit of a time sink since esmf can take upwards of 1/2 an hour.

See the following runs, in which nothing was changed relating to the above packages: https://github.com/ACCESS-NRI/ACCESS-OM3/actions/runs/10987685371/job/30502929361?pr=5#step:8:767, https://github.com/ACCESS-NRI/ACCESS-OM3/actions/runs/10985399681/job/30497188155#step:8:773.

Determine if the current spack.packages.*.require statements are not restrictive enough to prevent a rebuild - https://github.com/ACCESS-NRI/ACCESS-OM3/pull/5/files#diff-e8582e74fa156f4e5729a850e52b24f2fde2d815c2c9c360f88c4cf90db851abR10-R20

Or maybe it's another problem?

The text was updated successfully, but these errors were encountered:

harshula · 2024-09-23T04:54:26Z

Double check if the Spack hash is the same or if it has changed.

CodeGat · 2024-09-23T05:00:09Z

parallelio, esmf, access-om3-nuopc do change between runs.

anton-seaice · 2024-09-23T05:04:31Z

Both esmf and access-om3-nuopc have dependencies on parallelio (e.g. https://github.com/spack/spack/blob/c118c7733b9e20f079bb19b84b8ad60cacd2a673/var/spack/repos/builtin/packages/esmf/package.py#L91), so i'd guess its related to parallelio or one of its deps

CodeGat · 2024-10-18T02:32:37Z

Another avenue is the spack gc garbage collection that happens on removal of old Prerelease environments as part of on.pull_request.types.closed, see https://github.com/ACCESS-NRI/ACCESS-OM2/actions/runs/11396341321/job/31710012054#step:6:18

CodeGat · 2025-02-05T00:55:28Z

Due to chats with spack devs, it might be useful to check the difference between the hashes with spack diff /HASH1 /HASH2

anton-seaice · 2025-02-05T05:31:25Z

I don't know what this tells us :-)

spack diff /qcd4giqfkytonx3kmqkjwrfat3kiogai /pj4c37ym76pt3eaqqypzgtxl5rqkve7l
==> Warning: This interface is subject to change.

--- parallelio@2.6.2/qcd4giqfkytonx3kmqkjwrfat3kiogai
+++ parallelio@2.6.2/pj4c37ym76pt3eaqqypzgtxl5rqkve7l
@@ hash @@
-  parallelio qcd4giqfkytonx3kmqkjwrfat3kiogai
+  parallelio pj4c37ym76pt3eaqqypzgtxl5rqkve7l

anton-seaice · 2025-02-05T05:34:36Z

Lots of noise ...

But I have parallelio installed, do a concretise and it decides I need a new one (note - next to parallelio) :

[as2285@gadi-login-03 spack0.22]$ spack find parallelio
==> In environment cice-spack-package
==> 1 root specs
 -  access-om3@git.main=0-git.14

==> Installed packages
-- linux-rocky8-x86_64 / intel@2021.10.0 ------------------------
parallelio@2.6.2
==> 1 installed package
[as2285@gadi-login-03 spack0.22]$ spack concretise -f --fresh
==> Warning: using "cmake@3.24.2" which is a deprecated version
==> Concretized access-om3@git.main=0-git.14
 -   b26hvt5  access-om3@git.main=0-git.14%intel@2021.10.0 build_system=bundle arch=linux-rocky8-x86_64
 -   apv4p3o      ^access3-exe@git.main=0-git.171%intel@2021.10.0~ipo~openmp build_system=cmake build_type=Release configurations=CICE6,MOM6-CICE6 dev_path=/g/data/tm70/as2285/spack0.22/environments/cice-spack-package/access3-share generator=make arch=linux-rocky8-x86_64
 -   gtx27bo          ^access-cice@git.5-port_cmake_build=6.2.0-git.203%intel@2021.10.0+access3+cesmcoupled~ipo build_system=cmake build_type=Release dev_path=/g/data/tm70/as2285/spack0.22/environments/cice-spack-package/access-cice generator=make io_type=PIO arch=linux-rocky8-x86_64
 -   rje7cyo              ^access3-share@git.ca8a1ba8ca2716d2ae57846faf5c5ddc8bc5d35f=0-git.171%intel@2021.10.0~ipo build_system=cmake build_type=Release dev_path=/g/data/tm70/as2285/spack0.22/environments/cice-spack-package/access3-share generator=make arch=linux-rocky8-x86_64
 -   gvv2k6y                  ^esmf@git.v8.7.0=8.7.0%intel@2021.10.0 cflags='-fp-model precise' fflags='-traceback -fp-model precise' ~debug~external-lapack+external-parallelio+mpi+netcdf~pnetcdf+shared~xerces build_system=makefile esmf_comm=auto esmf_os=auto esmf_pio=auto patches=f63d405 snapshot=none arch=linux-rocky8-x86_64
[+]  bcbruwv                  ^fortranxml@4.1.2%intel@2021.10.0 build_system=autotools arch=linux-rocky8-x86_64
 -   pm7t3uw              ^parallelio@2.6.2%intel@2021.10.0 cflags='-std=gnu99 -qno-opt-dynamic-align -fp-model precise' fflags='-convert big_endian -ftz -assume realloc_lhs -assume byterecl -qno-opt-dynamic-align -fp-model source -traceback' +fortran~ipo~logging+mpi~ncint~pnetcdf+shared~timing build_system=cmake build_type=RelWithDebInfo generator=make arch=linux-rocky8-x86_64
[+]  2c4wu5t                  ^netcdf-c@4.9.2%intel@2021.10.0+blosc~byterange~dap~fsync~hdf4~ipo~jna+mpi~nczarr_zip+optimize~parallel-netcdf+pic+shared+szip+zstd build_system=cmake build_type=RelWithDebInfo generator=make patches=0161eb8 arch=linux-rocky8-x86_64
[+]  357vng5                      ^bzip2@1.0.8%intel@2021.10.0~debug~pic+shared build_system=generic arch=linux-rocky8-x86_64
[+]  y7n7vkn                          ^diffutils@3.10%intel@2021.10.0 build_system=autotools arch=linux-rocky8-x86_64
[+]  55ipnye                              ^libiconv@1.17%intel@2021.10.0 build_system=autotools libs=shared,static arch=linux-rocky8-x86_64
[+]  jdlivnc                      ^c-blosc@1.21.5%intel@2021.10.0+avx2~ipo build_system=cmake build_type=RelWithDebInfo generator=make arch=linux-rocky8-x86_64
[+]  7a5olrr                          ^lz4@1.9.4%intel@2021.10.0+pic build_system=makefile libs=shared,static arch=linux-rocky8-x86_64
[+]  ju23cym                          ^snappy@1.1.10%intel@2021.10.0~ipo+pic+shared build_system=cmake build_type=RelWithDebInfo generator=make arch=linux-rocky8-x86_64
[+]  ozfo7fd                      ^hdf5@1.14.3%intel@2021.10.0~cxx~fortran+hl~ipo~java~map+mpi+shared~subfiling~szip~threadsafe+tools api=default build_system=cmake build_type=RelWithDebInfo generator=make patches=82088c8 arch=linux-rocky8-x86_64
[+]  vrupasu                          ^pkgconf@2.2.0%intel@2021.10.0 build_system=autotools arch=linux-rocky8-x86_64
[+]  675weom                      ^libaec@1.0.6%intel@2021.10.0~ipo+shared build_system=cmake build_type=RelWithDebInfo generator=make arch=linux-rocky8-x86_64
[+]  rglvnt5                      ^m4@1.4.19%intel@2021.10.0+sigsegv build_system=autotools patches=9dc5fbd,bfdffa7 arch=linux-rocky8-x86_64
[+]  ou7jkjx                          ^libsigsegv@2.14%intel@2021.10.0 build_system=autotools arch=linux-rocky8-x86_64
[+]  wa2x7rh                      ^zlib-ng@2.1.6%intel@2021.10.0+compat+new_strategies+opt+pic+shared build_system=autotools arch=linux-rocky8-x86_64
[+]  kzc7pcv                      ^zstd@1.5.6%intel@2021.10.0~programs build_system=makefile libs=shared,static arch=linux-rocky8-x86_64
 -   xo3zqcq          ^access-mom6@git.cmake_build=0-git.11338%intel@2021.10.0+access3+cesmcoupled~ipo+mom_symmetric~openmp build_system=cmake build_type=Release dev_path=/g/data/tm70/as2285/spack0.22/environments/cice-spack-package/MOM6 generator=make arch=linux-rocky8-x86_64
[+]  on4aolu              ^fms@git.2024.03=2024.01.02-git.26%intel@2021.10.0~deprecated_io~gfs_phys+internal_file_nml~ipo+large_file~openmp~pic~quad_precision~yaml build_system=cmake build_type=RelWithDebInfo constants=GFDL generator=make precision=32,64 arch=linux-rocky8-x86_64
[+]  bky2d7s              ^netcdf-fortran@4.6.1%intel@2021.10.0~doc+pic+shared build_system=autotools arch=linux-rocky8-x86_64
[e]  tkmozao          ^cmake@3.24.2%intel@2021.10.0~doc+ncurses+ownlibs build_system=generic build_type=RelWithDebInfo arch=linux-rocky8-x86_64
[+]  doeoclg          ^gmake@4.4.1%intel@2021.10.0~guile build_system=generic arch=linux-rocky8-x86_64
[e]  5kn4pgy          ^openmpi@4.1.7%intel@2021.10.0~atomics~cuda~cxx~cxx_exceptions~gpfs~internal-hwloc~internal-libevent~internal-pmix~java~legacylaunchers~lustre~memchecker~openshmem~orterunprefix~romio+rsh~singularity~static+vt+wrapper-rpath build_system=autotools fabrics=none romio-filesystem=none schedulers=none arch=linux-rocky8-x86_64
[e]  5elnsoi      ^glibc@2.28%intel@2021.10.0 build_system=autotools arch=linux-rocky8-x86_64

==> Updating view at /g/data/tm70/as2285/spack0.22/environments/cice-spack-package/.spack-env/view

harshula · 2025-02-05T23:01:37Z

Hi @anton-seaice , Can you please try spack concretise -f --reuse-deps?

harshula · 2025-02-05T23:08:30Z

https://github.com/spack/spack/releases/tag/v0.22.0:

With the install command you now have three options:

    --reuse (default): reuse as many existing installations as possible.
    --reuse-deps / --fresh-roots: upgrade (freshen) roots but reuse dependencies if possible.
    --fresh: install fresh versions of requested packages (roots) and their dependencies.

We've also introduced --fresh-roots as an alias for --reuse-deps to make it more clear
that it may give you fresh versions. More details in #41302 and #43988.

anton-seaice · 2025-02-05T23:21:29Z

Thanks @harshula - that looks better:

[as2285@gadi-cpu-clx-2101 as2285]$ spack concretise -f --reuse-deps
==> Concretized access-om3@git.main=0-git.14
 -   u6sovce  access-om3@git.main=0-git.14%intel@2021.10.0 build_system=bundle arch=linux-rocky8-x86_64
 -   kqd3bkh      ^access3-exe@git.main=0-git.171%intel@2021.10.0~ipo~openmp build_system=cmake build_type=Release configurations=CICE6,MOM6-CICE6 dev_path=/g/data/tm70/as2285/spack0.22/environments/cice-spack-package/access3-share generator=make arch=linux-rocky8-x86_64
[+]  hpiqyr6          ^access-cice@git.5-port_cmake_build=6.2.0-git.203%intel@2021.10.0+access3+cesmcoupled~ipo build_system=cmake build_type=Release dev_path=/g/data/tm70/as2285/spack0.22/environments/cice-spack-package/access-cice generator=make io_type=PIO arch=linux-rocky8-x86_64
[+]  wje7ves              ^access3-share@git.ca8a1ba8ca2716d2ae57846faf5c5ddc8bc5d35f=0-git.171%intel@2021.10.0~ipo build_system=cmake build_type=Release dev_path=/g/data/tm70/as2285/spack0.22/environments/cice-spack-package/access3-share generator=make arch=linux-rocky8-x86_64
[+]  rgw4y5k                  ^esmf@git.v8.7.0=8.7.0%intel@2021.10.0 cflags='-fp-model precise' fflags='-traceback -fp-model precise' ~debug~external-lapack+external-parallelio+mpi+netcdf~pnetcdf+shared~xerces build_system=makefile esmf_comm=auto esmf_os=auto esmf_pio=auto patches=f63d405 snapshot=none arch=linux-rocky8-x86_64
[+]  bcbruwv                  ^fortranxml@4.1.2%intel@2021.10.0 build_system=autotools arch=linux-rocky8-x86_64
[+]  v7eqbrl              ^parallelio@2.6.2%intel@2021.10.0 cflags='-qno-opt-dynamic-align -fp-model precise -std=gnu99' fflags='-assume byterecl -traceback -assume realloc_lhs -convert big_endian -fp-model source -qno-opt-dynamic-align -ftz' +fortran~ipo~logging+mpi~ncint~pnetcdf+shared~timing build_system=cmake build_type=RelWithDebInfo generator=make arch=linux-rocky8-x86_64
[+]  2c4wu5t                  ^netcdf-c@4.9.2%intel@2021.10.0+blosc~byterange~dap~fsync~hdf4~ipo~jna+mpi~nczarr_zip+optimize~parallel-netcdf+pic+shared+szip+zstd build_system=cmake build_type=RelWithDebInfo generator=make patches=0161eb8 arch=linux-rocky8-x86_64
[+]  357vng5                      ^bzip2@1.0.8%intel@2021.10.0~debug~pic+shared build_system=generic arch=linux-rocky8-x86_64
[+]  y7n7vkn                          ^diffutils@3.10%intel@2021.10.0 build_system=autotools arch=linux-rocky8-x86_64
[+]  55ipnye                              ^libiconv@1.17%intel@2021.10.0 build_system=autotools libs=shared,static arch=linux-rocky8-x86_64
[+]  jdlivnc                      ^c-blosc@1.21.5%intel@2021.10.0+avx2~ipo build_system=cmake build_type=RelWithDebInfo generator=make arch=linux-rocky8-x86_64
[+]  7a5olrr                          ^lz4@1.9.4%intel@2021.10.0+pic build_system=makefile libs=shared,static arch=linux-rocky8-x86_64
[+]  ju23cym                          ^snappy@1.1.10%intel@2021.10.0~ipo+pic+shared build_system=cmake build_type=RelWithDebInfo generator=make arch=linux-rocky8-x86_64
[+]  ozfo7fd                      ^hdf5@1.14.3%intel@2021.10.0~cxx~fortran+hl~ipo~java~map+mpi+shared~subfiling~szip~threadsafe+tools api=default build_system=cmake build_type=RelWithDebInfo generator=make patches=82088c8 arch=linux-rocky8-x86_64
[+]  vrupasu                          ^pkgconf@2.2.0%intel@2021.10.0 build_system=autotools arch=linux-rocky8-x86_64
[+]  675weom                      ^libaec@1.0.6%intel@2021.10.0~ipo+shared build_system=cmake build_type=RelWithDebInfo generator=make arch=linux-rocky8-x86_64
[+]  rglvnt5                      ^m4@1.4.19%intel@2021.10.0+sigsegv build_system=autotools patches=9dc5fbd,bfdffa7 arch=linux-rocky8-x86_64
[+]  ou7jkjx                          ^libsigsegv@2.14%intel@2021.10.0 build_system=autotools arch=linux-rocky8-x86_64
[+]  wa2x7rh                      ^zlib-ng@2.1.6%intel@2021.10.0+compat+new_strategies+opt+pic+shared build_system=autotools arch=linux-rocky8-x86_64
[+]  kzc7pcv                      ^zstd@1.5.6%intel@2021.10.0~programs build_system=makefile libs=shared,static arch=linux-rocky8-x86_64
[+]  vygcg6d          ^access-mom6@git.cmake_build=0-git.11338%intel@2021.10.0+access3+cesmcoupled~ipo+mom_symmetric~openmp build_system=cmake build_type=Release dev_path=/g/data/tm70/as2285/spack0.22/environments/cice-spack-package/MOM6 generator=make arch=linux-rocky8-x86_64
[+]  on4aolu              ^fms@git.2024.03=2024.01.02-git.26%intel@2021.10.0~deprecated_io~gfs_phys+internal_file_nml~ipo+large_file~openmp~pic~quad_precision~yaml build_system=cmake build_type=RelWithDebInfo constants=GFDL generator=make precision=32,64 arch=linux-rocky8-x86_64
[+]  bky2d7s              ^netcdf-fortran@4.6.1%intel@2021.10.0~doc+pic+shared build_system=autotools arch=linux-rocky8-x86_64
[e]  tkmozao          ^cmake@3.24.2%intel@2021.10.0~doc+ncurses+ownlibs build_system=generic build_type=RelWithDebInfo arch=linux-rocky8-x86_64
[+]  doeoclg          ^gmake@4.4.1%intel@2021.10.0~guile build_system=generic arch=linux-rocky8-x86_64
[e]  5kn4pgy          ^openmpi@4.1.7%intel@2021.10.0~atomics~cuda~cxx~cxx_exceptions~gpfs~internal-hwloc~internal-libevent~internal-pmix~java~legacylaunchers~lustre~memchecker~openshmem~orterunprefix~romio+rsh~singularity~static+vt+wrapper-rpath build_system=autotools fabrics=none romio-filesystem=none schedulers=none arch=linux-rocky8-x86_64
[e]  5elnsoi      ^glibc@2.28%intel@2021.10.0 build_system=autotools arch=linux-rocky8-x86_64

No promises its robust against compiler changes etc ?

harshula · 2025-02-05T23:34:44Z

Hi @anton-seaice , I think we've been misusing --fresh because it solved a problem that we were experiencing with the default. Most of the time we don't want its functionality. i.e. " --fresh: install fresh versions of requested packages (roots) and their dependencies."

What kind compiler changes are you concerned about? Spack is meant to identify compiler incompatibilities.

anton-seaice · 2025-02-06T00:11:20Z

My memory is that without --fresh spack will try and re-use dependencies from different compiler versions?

e.g. it might try and reuse a netcdf using intel 2021.06.0 even if the top level spec is using intel 2021.10.0

For example, I deactivate and try to install netcdf using intel@2021.06.0 and it uses dependencies from 2021.10.0:

[as2285@gadi-cpu-clx-2101 as2285]$ spack env deactivate
[as2285@gadi-cpu-clx-2101 as2285]$ spack install netcdf-c@4.9.2%intel@2021.06.0
[+] /usr (external glibc-2.28-5elnsoiqgcg5k5zmmwsp33bmnmaa3g5p)
[+] /usr (external glibc-2.28-csotjiltnkhc4qq55pnhueyamwf6uh5q)
==> openmpi@4.1.7 : has external module in ['openmpi/4.1.7']
[+] /apps/openmpi/4.1.7 (external openmpi-4.1.7-5kn4pgyj7aa5e2wcqjgxoybzn5l75i64)
[+] /g/data/tm70/as2285/spack0.22/release/linux-rocky8-x86_64/intel-2021.10.0/libaec-1.0.6-675weomq5zwi6c53fnhntqodgqlm6nqz
[+] /g/data/tm70/as2285/spack0.22/release/linux-rocky8-x86_64/intel-2021.10.0/snappy-1.1.10-ju23cym4rijzildlxsvhql7pv4av7eg5
[+] /g/data/tm70/as2285/spack0.22/release/linux-rocky8-x86_64/intel-2021.10.0/bzip2-1.0.8-357vng5dpd7w7s7lletycxccjbl45ngt
[+] /g/data/tm70/as2285/spack0.22/release/linux-rocky8-x86_64/intel-2021.10.0/gmake-4.4.1-doeoclgsn63cvh7zfx3trjmyk7rlcqn4
[+] /g/data/tm70/as2285/spack0.22/release/linux-rocky8-x86_64/intel-2021.10.0/zstd-1.5.6-kzc7pcve7csxlonb2uaxzgyyuqfx6cz4
[+] /g/data/tm70/as2285/spack0.22/release/linux-rocky8-x86_64/intel-2021.10.0/lz4-1.9.4-7a5olrrnewy7kmlh5x4bstziuheiqkz4
[+] /g/data/tm70/as2285/spack0.22/release/linux-rocky8-x86_64/intel-2021.10.0/pkgconf-2.2.0-vrupasu7smpgcbarzpdwap45fcvjbjoa
[+] /g/data/tm70/as2285/spack0.22/release/linux-rocky8-x86_64/intel-2021.10.0/zlib-ng-2.1.6-wa2x7rho3km6qpiki56dpjlpsce4c5n6
[+] /g/data/tm70/as2285/spack0.22/release/linux-rocky8-x86_64/intel-2021.10.0/c-blosc-1.21.5-jdlivncntjhhznjhl62q7yxdt7mwcaj6
[+] /g/data/tm70/as2285/spack0.22/release/linux-rocky8-x86_64/intel-2021.10.0/hdf5-1.14.3-ozfo7fd2a4ow6ivjte5lfehn5qjhacpp
==> Installing netcdf-c-4.9.2-ulskek34jvensle5yriejgho7vjonctv [14/14]
==> ...

spack install --reuse-deps netcdf-c@4.9.2%intel@2021.06.0 behaves the same

(And for completeness, using different compiler might work in that circumstance), so repeat with fortran code + deps:

e.g. parallelio with intel 2021.06.0 tries to use netcdf-fortran from intel 2021.10.0 :

spack install --reuse-deps parallelio@2.6.2%intel@2021.06.0
[+] /apps/cmake/3.24.2 (external cmake-3.24.2-tkmozaoejrs2hj3hbjh3dt3qdvqy2rya)
[+] /usr (external glibc-2.28-csotjiltnkhc4qq55pnhueyamwf6uh5q)
[+] /usr (external glibc-2.28-5elnsoiqgcg5k5zmmwsp33bmnmaa3g5p)
==> openmpi@4.1.7 : has external module in ['openmpi/4.1.7']
[+] /apps/openmpi/4.1.7 (external openmpi-4.1.7-5kn4pgyj7aa5e2wcqjgxoybzn5l75i64)
[+] /g/data/tm70/as2285/spack0.22/release/linux-rocky8-x86_64/intel-2021.10.0/gmake-4.4.1-doeoclgsn63cvh7zfx3trjmyk7rlcqn4
[+] /g/data/tm70/as2285/spack0.22/release/linux-rocky8-x86_64/intel-2021.10.0/zstd-1.5.6-kzc7pcve7csxlonb2uaxzgyyuqfx6cz4
[+] /g/data/tm70/as2285/spack0.22/release/linux-rocky8-x86_64/intel-2021.10.0/zlib-ng-2.1.6-wa2x7rho3km6qpiki56dpjlpsce4c5n6
[+] /g/data/tm70/as2285/spack0.22/release/linux-rocky8-x86_64/intel-2021.10.0/lz4-1.9.4-7a5olrrnewy7kmlh5x4bstziuheiqkz4
[+] /g/data/tm70/as2285/spack0.22/release/linux-rocky8-x86_64/intel-2021.10.0/bzip2-1.0.8-357vng5dpd7w7s7lletycxccjbl45ngt
[+] /g/data/tm70/as2285/spack0.22/release/linux-rocky8-x86_64/intel-2021.10.0/snappy-1.1.10-ju23cym4rijzildlxsvhql7pv4av7eg5
[+] /g/data/tm70/as2285/spack0.22/release/linux-rocky8-x86_64/intel-2021.10.0/pkgconf-2.2.0-vrupasu7smpgcbarzpdwap45fcvjbjoa
[+] /g/data/tm70/as2285/spack0.22/release/linux-rocky8-x86_64/intel-2021.10.0/libaec-1.0.6-675weomq5zwi6c53fnhntqodgqlm6nqz
[+] /g/data/tm70/as2285/spack0.22/release/linux-rocky8-x86_64/intel-2021.10.0/c-blosc-1.21.5-jdlivncntjhhznjhl62q7yxdt7mwcaj6
[+] /g/data/tm70/as2285/spack0.22/release/linux-rocky8-x86_64/intel-2021.10.0/hdf5-1.14.3-ozfo7fd2a4ow6ivjte5lfehn5qjhacpp
[+] /g/data/tm70/as2285/spack0.22/release/linux-rocky8-x86_64/intel-2021.10.0/netcdf-c-4.9.2-2c4wu5th6gxpm3hsohdqwq43nii5uuuu
[+] /g/data/tm70/as2285/spack0.22/release/linux-rocky8-x86_64/intel-2021.10.0/netcdf-fortran-4.6.1-bky2d7sqvg4kfbiwveqnvr634hj2tdot
==> Installing parallelio-2.6.2-22bnphc3jgvru7f5ttu2kpgwjvctvnk7 [17/17]
==> ...

harshula · 2025-02-06T01:59:31Z

it might try and reuse a netcdf using intel 2021.06.0 even if the top level spec is using intel 2021.10.0

If Spack considers intel 2021.06.0 and intel 2021.10.0 to be compatible, do you think that is not "robust"?

anton-seaice · 2025-02-06T02:03:51Z

Agreed - but I don't think that's the behavior we want - especially for CD ?

anton-seaice · 2025-02-06T02:16:13Z

To convince myself ... I made a new environment where the only difference from the installed environment was to set packages.all.require to an old compiler, and it failed to concretise with --reuse-deps:

[as2285@gadi-login-03 spack0.22]$ spack concretise -f --reuse-deps
==> Error: concretization failed for the following reasons:

   1. cannot satisfy a requirement for package 'fms'.
   2. cannot satisfy a requirement for package 'esmf'.

harshula · 2025-02-06T04:10:32Z

Agreed - but I don't think that's the behavior we want - especially for CD ?

For example, openmpi (external) is a dependency for our models and it isn't compiled with the same compiler version as we use during our CD.

Just to clarify, if your concern about the latency is from the perspective of using spack develop, then please use --reuse-deps first, if it doesn't work, then use --fresh.

In terms of CD, @aidanheerdegen will look at the options and make a decision on what we should use.

anton-seaice · 2025-02-06T04:18:53Z

I would like the cake and to be able to eat it too. (i.e. I want the CI to be fast to encourage folks to build using it so that builds are recorded / shareable / for easier reviews:-)

CodeGat · 2025-02-10T02:56:26Z

Hullo peeps, we will need to keep --fresh in the CD as it resolves a current spack bug. I'll chat to the upstream spack people next meeting about this and update the issue.

CodeGat added priority:medium type:bug labels Sep 23, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Trivial Changes to `spack.yaml` Require Rebuild of Some Packages #12

Trivial Changes to `spack.yaml` Require Rebuild of Some Packages #12

CodeGat commented Sep 23, 2024

harshula commented Sep 23, 2024

CodeGat commented Sep 23, 2024

anton-seaice commented Sep 23, 2024 •

edited

Loading

CodeGat commented Oct 18, 2024

CodeGat commented Feb 5, 2025

anton-seaice commented Feb 5, 2025

anton-seaice commented Feb 5, 2025

harshula commented Feb 5, 2025

harshula commented Feb 5, 2025

anton-seaice commented Feb 5, 2025

harshula commented Feb 5, 2025

anton-seaice commented Feb 6, 2025

harshula commented Feb 6, 2025

anton-seaice commented Feb 6, 2025

anton-seaice commented Feb 6, 2025 •

edited

Loading

harshula commented Feb 6, 2025

anton-seaice commented Feb 6, 2025 •

edited

Loading

CodeGat commented Feb 10, 2025

Trivial Changes to spack.yaml Require Rebuild of Some Packages #12

Trivial Changes to spack.yaml Require Rebuild of Some Packages #12

Comments

CodeGat commented Sep 23, 2024

harshula commented Sep 23, 2024

CodeGat commented Sep 23, 2024

anton-seaice commented Sep 23, 2024 • edited Loading

CodeGat commented Oct 18, 2024

CodeGat commented Feb 5, 2025

anton-seaice commented Feb 5, 2025

anton-seaice commented Feb 5, 2025

harshula commented Feb 5, 2025

harshula commented Feb 5, 2025

anton-seaice commented Feb 5, 2025

harshula commented Feb 5, 2025

anton-seaice commented Feb 6, 2025

harshula commented Feb 6, 2025

anton-seaice commented Feb 6, 2025

anton-seaice commented Feb 6, 2025 • edited Loading

harshula commented Feb 6, 2025

anton-seaice commented Feb 6, 2025 • edited Loading

CodeGat commented Feb 10, 2025

Trivial Changes to `spack.yaml` Require Rebuild of Some Packages #12

Trivial Changes to `spack.yaml` Require Rebuild of Some Packages #12

anton-seaice commented Sep 23, 2024 •

edited

Loading

anton-seaice commented Feb 6, 2025 •

edited

Loading

anton-seaice commented Feb 6, 2025 •

edited

Loading