Deployment 2020-2.0 checklist

[mrakitin]

Previsit

• Make sure that the current latest environments are pushed to all the bl machines
• Skim the IPython startup files in profile_collection
  • Check if it's safe to start bsui remotely (does it touch any hardware?) - use git grep .put or git grep caput
• Check for outstanding PRs and issues on beamline repos
• Check if beamline specific packages pushed to the BL machines
• (Optional) Enable CI

Housekeeping

• Add BL staff to the BL GitHub organization as owners
• Work with BL staff to commit any un-committed changes to their profiles
• Tag the profile as-found as 2019C3.x
• Discuss with BL staff which conda envs they want to keep/delete and perform the cleanup
• Check/update the beamline's inventory with the BL staff (in https://github.com/NSLS-II/playbooks/blob/master/production)

Test deployment

• In ~/.bashrc add the following and remember to source ~/.bashrc afterward.

  export BLUESKY_LOG_FILE=/var/log/bluesky/bluesky.log
  export BLUESKY_IPYTHON_LOG_FILE=/var/log/bluesky/bluesky_ipython.log
  umask 0002

  and (if needed) create those directories, making them world-read/write-able.

  chown xf21id1:xf21id1 /var/log/bluesky/
  chmod g+rws /var/log/bluesky/

• removed proxies and path=/opt/conda... from ~/.bashrc, replaced with /etc/profile.d/conda.sh, /etc/profile.d/proxy.sh
• Append to 00-startup.py https://gist.github.com/jklynch/a4366b8900ec0c03883403455ae711b2
• Install suitcase=0.7.0 into the new env (needed for hdf5 exporting)
  • conda install suitcase=0.7.0 -c nsls-ii-2019C-c defaults-2019C3.0 --override-channels --no-channel-priority --no-deps
• Run profile against new environment
  • BS_ENV=collection-2020-2.0rc7-1 bsui
• Update the profile as needed to run:
  • history.sqlite issue - see https://blueskyproject.io/bluesky/metadata.html#persistence-between-sessions
  • Less verbose ophyd logging:

    # Temporary fix before it's fixed in ophyd
    import logging
    logger = logging.getLogger('ophyd')
    logger.setLevel('WARNING')

    Note: works only when put after nslsii.configure_base(...) in 00-base.py.
  • Replace .value -> .get() if it's in a Device, or with yield from bps.read() if in a plan
• Clean up repo from unused/backup files
• Run acceptance tests (if any) and run a representative set of scans for the beamline
• Tag the updated profile as 2020C2.0
• Update BSUI via ansible to use the new environment (http://nsls-ii.github.io/deployment/upgrading-beamlines.html#change-bsui-s-default-environment)

[cryos]

I see quite a few caput calls:

startup/40-ESM_plans_elio.py:    #caput XF:21IDC-BI{EM:7}EM180:Acquire 0
startup/40-ESM_plans_elio.py:    caput('XF:21IDC-BI{EM:7}EM180:CurrentOffset1', value)
startup/40-ESM_plans_elio.py:    caput('XF:21ID-PPS{Sh:FE}Cmd:Opn-Cmd', 1)
startup/40-ESM_plans_elio.py:    os.system('caput XF:21ID-PPS{Sh:FE}Cmd:Cls-Cmd 1')
startup/40-ESM_plans_elio.py:    os.system('caput XF:21ID-PPS{Sh:FE}Cmd:Opn-Cmd 1')
startup/40-ESM_plans_elio.py:#        os.system('caput XF:21IDB-OP{Mono:1-Ax:8_MP}Mtr.OFF '+ off_M2[GRT_name])    # set offset for M2
startup/40-ESM_plans_elio.py:#        os.system('caput XF:21IDB-OP{Mono:1-Ax:8_GP}Mtr.OFF '+ off_GRT[GRT_name])   # set offset for GRT
startup/40-ESM_plans_elio.py:#        os.system('caput XF:21IDB-OP{Mono:1}:LINES:SET ' + GRT_name)                # tells to the PGM software which grating                                     
startup/esm.py:    caput('XF:21ID-PPS{Sh:FE}Cmd:Cls-Cmd', 1)
startup/esm.py:    #caput XF:21IDC-BI{EM:7}EM180:Acquire 0
startup/esm.py:    caput('XF:21IDC-BI{EM:7}EM180:CurrentOffset1', value)
startup/esm.py:    caput('XF:21ID-PPS{Sh:FE}Cmd:Opn-Cmd', 1)
startup/esm.py:    os.system('caput XF:21ID-PPS{Sh:FE}Cmd:Cls-Cmd 1')
startup/esm.py:    os.system('caput XF:21ID-PPS{Sh:FE}Cmd:Opn-Cmd 1')
startup/esm.py:        os.system('caput XF:21IDB-OP{Mono:1-Ax:8_MP}Mtr.OFF '+ off_M2[GRT_name])    # set offset for M2
startup/esm.py:        os.system('caput XF:21IDB-OP{Mono:1-Ax:8_GP}Mtr.OFF '+ off_GRT[GRT_name])   # set offset for GRT
startup/esm.py:        os.system('caput XF:21IDB-OP{Mono:1}:LINES:SET ' + GRT_name)                # tells to the PGM software which grating

There are 120 .put calls although some are likely duplicates in files that no longer load, cutting it to 88.

[mrakitin]

xref nsls-ii-forge/project-management#19

[mrakitin]

2 more packages were missing, which are used in collection-2019-2.0-esm:

• pyqtgraph (v0.11.0, but had to downgrade to 0.10.0)
• igor v0.3.0

[mrakitin]

Since the current_env_tag is 2019-2.0-esm, the corresponding env did not provide appdirs (2019-3.0-* envs do have it), we had to install it with pip install appdirs, along with a missing msgpack_numpy (installed with pip install msgpack_numpy).

[mrakitin]

The deployment is done.