boardwalk (feat): allow execution of playbooks in addition to tasks

[asullivan-blze]

What and why?

This changeset adds a BaseJob class, from which TaskJob and PlaybookJob are derived. As the names suggest, these are Jobs which permit execution of tasks -- as was previously possible when using the Job class -- and full Ansible playbooks. The design features of Boardwalk, however, still exist even when using a PlaybookJob in a Workflow; that is, under the hood, ansible-runner is told to execute the targeted playbook file with a limit to the host which is currently progressing through the workflow.

As such, if the playbook(s) being selected for a given Workflow do not apply to the host(s) going through it, Ansible will skip execution of the playbook, as it would for a standard ansible-playbook invocation.

As a quick overview, the new jobs would look like the following:

class TestJob(TaskJob):
    def tasks(self) -> AnsibleTasksType:
        return [{"ansible.builtin.debug": {"msg": "Hello, Boardwalk!"}}]

class ShouldSucceedPlaybookExecutionTestJob(PlaybookJob):
    def playbooks(self) -> AnsibleTasksType:
        return [
            {"ansible.builtin.import_playbook": path("playbook-job-test-should-succeed.yml")},
            {"ansible.builtin.import_playbook": path("playbook-job-test-should-be-skipped.yml")},
        ]

A Workflow may use TaskJob, PlaybookJob, or a mix of both, as in the following example:

class ShouldSucceedMixedJobTypesWorkflow(Workflow):
    def jobs(self):
        return [
            TestJob(),
            ShouldSucceedPlaybookExecutionTestJob(),
        ]

    def exit_jobs(self):
        return TestJob()

And, of course, the test/server-client/Boardwalkfile.py contains a full example Boardwalkfile.

Other notes

• 🧪 Test enhancements: The majority of test Workspaces in the Boardwalkfile.py have been integrated into the pytest test suite (see: test/integration/test_workspaces.py). No more shall the tedious cycle of running the trinity of commands -- boardwalk workspace use $WORKSPACE, boardwalk init, boardwalk run -- be needed, one command does them all! Unless of course, you want to run them manually; I won't judge.
  • When running pytest, upon reaching these tests, the execution will pause, and prompt for the BECOME password; this is the same sudo password that would be used with boardwalk run; the difference is now we write it -- via tempfile.NamedTemporaryFile -- to a file, and then set the envvar ANSIBLE_BECOME_PASSWORD_FILE to the path returned by the context manager. This results in only needing to specify your password once if you want to run all of the test workspaces.
  • Note, however, having make develop-server running in a different shell beforehand is still needed, as is releasing any caught Workspace(s) that the development server may be aware of. (Having pytest spawn the development server itself is an item to complete for the future.)
  • As there is still the requirement to have the development server up, these tests are marked to be skipped if the CI envvar is found as set (and it is always set in GitHub workflows).
  • 📝 Logging enhancements: If you're like me, you enjoy a good logging output. So loguru has been tapped to transform the prior usages of Python's built-in logger, which looked a little something like this...

    # ... like this ...
    asullivan@MBP-NT9RPG2XV7 tmp % boardwalk run -nsc
    INFO:boardwalk.cli_run:Using workspace: ShouldFailTestWorkspace
    INFO:boardwalk.cli_run:Reading inventory to process any --limit
    INFO:boardwalk.ansible:Processing ansible-inventory
    WARNING:boardwalk.cli_run:localhost: Host started workflow but never completed. Job preconditions are ignored for this host
    [... truncated ...]
    
    # ... and turned it into this! ✨
    (boardwalk-py3.12) asullivan@MBP-NT9RPG2XV7 server-client % boardwalk run -nsc
    2024-10-24 15:27:52.564 | INFO     | boardwalk.cli:cli:77 - Log level is INFO
    2024-10-24 15:27:52.566 | INFO     | boardwalk.cli_run:run:125 - Using workspace: ShouldFailTestWorkspace
    2024-10-24 15:27:52.566 | INFO     | boardwalk.cli_run:filter_hosts_by_limit:375 - Reading inventory to process any --limit
    2024-10-24 15:27:52.566 | INFO     | boardwalk.ansible:ansible_inventory:191 - Processing ansible-inventory
    2024-10-24 15:27:52.933 | WARNING  | boardwalk.cli_run:check_host_preconditions_locally:442 - 127.0.0.1: Host started workflow but never completed. Job preconditions are ignored for this host

    This is admittedly a bit of a minor change, but it should aid with readability.
• 🐛 boardwalk workspace use tab completion - Tab completion via click's
  built-in capabilities was added in
  #81, however only recently
  was it discovered that if no previous workspace was set as active -- that is,
  .boardwalk/workspaces/active_workspaces.txt existed -- a NoActiveWorkspace
  exception would be raised. (Which, is not ideal considering that if you're
  tab-completing, you might not know what workspaces exist to begin with; or,
  more likely, just cannot remember their names.)

Deprecation notes

• Job class - The original Job class is deprecated, and will be removed in a future release; usage of this deprecated class will emit a DeprecationWarning to this effect. To ease transitioning to the new differentiated classes, usage of the Job class will be treated as a TaskJob.

Compatibility notes

Based on running this changeset's pytest session against a test instance of boardwalkd running version 0.8.21, this version should be compatible with prior versions of the server.

Internal ticket reference: SVRENG-608

How was this tested?

Manual invocations of boardwalk workspace use/init/run, up until I shoved it into pytest. Then pytest and make test.

Checklist

• Have you updated the version in the [tool.poetry] section of
  the pyproject.toml file (if applicable)?