-
Notifications
You must be signed in to change notification settings - Fork 0
Relocatable Spack packages
On the johnfreeman/issue426_relocatable_installation branch, commit 0a8e92dd19cc46ff9156cc9ad1b51956ccc9a7ca
Four scripts in a scripts/caching/ subdirectory, with fairly self-explanatory names:
-
cache_external_packages.sh: Put the external packages (gcc,boost, etc.) in a Spack buildcache -
cache_dunedaq_packages.sh: Put the DUNE DAQ packages from a given nightly (fddaq,coredaqanddbe) in a Spack buildcache -
install_externals_from_cache.sh:spack installthe external packages from a Spack buildcache -
install_dunedaq_from_cache.sh:spack installthe DUNE DAQ packages from a Spack buildcache
Generally, this would be run by a member of Software Coordination, using a container on daq.fnal.gov as described in the comment at the top of the script. It would be run only occasionally, e.g. in the event that the externals our nightly was based on changed. It takes about 10 minutes to run on daq.fnal.gov and creates a Spack buildcache of about 2 GB. Note that you want to take the externals you put in the buildcache from an installation which includes build-only dependencies; this is because if build-only dependencies aren't put in the buildcache, then when you try to install packages from the buildcache the missing build-only dependencies will be built from source, slowing things down. It's an open question how end users will get their hands on such a buildcache once Software Coordination's created it.
Ultimately this would likely be typically run as part of a nightly workflow. You give it the name of the nightly workflow whose DUNE DAQ packages you want in a cache (e.g., NFD_DEV_250228) and the name of the new buildcache directory where you want to store the DUNE DAQ packages. On daq.fnal.gov it takes about 25 minutes to run, by far the slowest of the four scripts.
One imagines that this script would be run by an end user who wanted a relocatable installation, but probably only once or at most a few times; once you have the externals, you have the externals. Pass it the name of the directory where you want the externals spack installed and the name of the Spack buildcache directory where the externals have been stored. It only takes 3-4 minutes to run on daq.fnal.gov.
This is a script an end user would run whenever they wanted to install DUNE DAQ packages from a particular nightly. It has the most complex signature of the four scripts since it needs to know both about the externals and the nightlies: where the externals are installed, where to install the DUNE DAQ packages, where the DUNE DAQ cache directory is and what the name of the nightly is. This takes about 7-8 minutes to run.
Say that you've run install_dunedaq_from_cache.sh and you passed the script $HOME/NFD_DEV_250228_A9_dunedaq_packages as the directory where you wanted to install the DUNE DAQ packages. To use it, you can do the following:
git clone https://github.com/DUNE-DAQ/daq-buildtools -b johnfreeman/daq-release_issue426_relocatable_installation
source ./daq-buildtools/env.sh
dbt-create -r $HOME/NFD_DEV_250228_A9_dunedaq_packages -n NFD_DEV_250228_A9 <your new workarea>
...and now you can use <your new workarea> as you would any other work area. The difference, transparent to the user, is that the Spack-installed packages are all locally installed from Spack buildcaches, and not on cvmfs.