QA Testing

The following instructions are for Release QA processes only and should not be used on production hardware or in production contexts.

Build Variants/Environments

Three scenarios are supported through the environment key in config.json:

• prod: (yum.securedrop.org, apt.freedom.press)
• staging: (yum-test.securedrop.org, apt-test.freedom.press)
• dev: (local RPM, apt-test.freedom.press nightlies)

One additional scenario is supported manually (instructions follow):

• pre-release QA (production packages on yum-qa.securedrop.org and apt-qa.freedom.press)

QA involves testing dom0 (rpm) packages, VM (deb) packages, or both.

dom0 testing (securedrop-workstation-dom0-config)

Testing RCs

• Staging setup: make staging

Pre-release QA (production packages)

Clean Installation

• Start from clean QubesOS install. Use official install docs at workstation.securedrop.org to download and import the production signing key.
• Follow the official documentation, but replace yum.securedrop.org with yum-qa.securedrop.org when downloading the RPM.
• If you are also testing VM changes (debian packages) at the same time, stop here before continuing and make the needed adjustments to the apt repo (below).
• Proceed with installation.

Upgrade testing

Upgrade testing tests 2 things: that the new RPM is successfully installed on top of the old one, and that the configuration (Salt run) afterwards is successful. While this may change in a current version, at the moment the Salt run (highstate) reinstalls the rpm (overwriting what is on yum-qa with what is on yum prod, effectively a downgrade, and defeating the second part of testing). Therefore, to test qa packages:

• Start from a clean installation of the last production RPM release. Do not switch from an old staging setup to a prod setup; uninstall/unconfigure all VMs, RPC policies, etc (sdw-admin --uninstall) and be sure that your config.json file has the production environment specified if you are reusing this file. Reboot the machine. Ensure basic functionality (VMs boot, basic usage works).
• Edit /etc/yum.repos.d/securedrop-workstation.repo to point to yum-qa instead of yum
• Edit /srv/salt/securedrop_salt/sd-default-config.yml to do the same (in prod config).
• If any Salt runs have occurred since boot time (they should not have if following this test plan), reboot the machine or run sudo qubesctl saltutil.clear_cache && sudo qubesctl saltutil.sync_all
• If you are also testing VM changes (debian packages) at the same time, stop here before continuing and make the needed adjustments to the apt repo (below).
• Run the updater: sdw-updater --skip-delta 0.

VM testing

(Test Debian packages in VMs such as securedrop-client, proxy, log, etc)

Testing RCs

• Staging setup: make staging.

Pre-release QA (prod packages)

Clean install

• Download and install the latest production RPM according to official docs (see above).
• Edit /srv/salt/securedrop_salt/sd-default-config.yml to point to apt-qa.freedom.press.
• in /srv/salt/securedrop_salt/apt_freedom_press.sources.j2, change the apt repo to https://apt-qa.freedom.press
• copy /srv/salt/securedrop_salt/apt_freedom_press.sources.j2 to /srv/salt/securedrop_salt/apt-qa_freedom_press.sources.j2, and edit the path in /srv/salt/securedrop_salt/sd-default.config.yml

Upgrade testing

• Start from a production installation, verifying basic functionality.
• If you are concurrently testing VM changes alongside a dom0 upgrade where a full Salt run (including against templates) will happen (i.e. a template upgrade scenario), perform the VM modifications for the clean install step (see above). If you're not sure, perform the modifications.
• In each of the Debian-based templates (sd-small-$debdistribution-template, sd-large-$debdistribution-template), edit /etc/apt/sources.list.d/securedrop-workstation.list to point to apt-qa.freedom.press, then run sudo apt-get update && sudo apt-get upgrade -y to pull in the latest packages.
• Shut down the templates and any VMs based on them. When you reboot, verify latest packages are present and proceed with QA.

Tip: If you are unsure about the available or installed packages, you can run the command apt-cache policy <package> in a TemplateVM or AppVM, e.g., apt-cache policy securedrop-client. It will tell you the exact version of the package that is available, as well as the package that will be installed next time the VM is updated.