[Demo with Helm] Update some bool flags for Flux in Staging

[rytswd]

Change Overview

This updates some values of Flux Helm Chart (which was copied from Flux's official Helm chart), and enable some flags to create more resources.

Challenges

😥 File Diff Does Not Tell What is Changed

You can see how values are being updated, but it is not clear how those changes will affect the actual resources. Are the resources being modified, created, or deleted? Or perhaps those may not take any effect at all? You will need an extra step or two to find out what's actually being done by this change.

😥 Intentional and Unintentional Environment Drift

This particular example chart has 2 Helm Values files - values-staging.yaml and values-prod.yaml ^(*1). It means you can control which flag to enable or disable, and adjust some values based on the environmental requirements.

In this Pull Request, the title specifies that the change is meant to take place only for staging. This is absolutely fine and expected, but it creates the environmental diff. In this simple example, this may seem harmless, but can later cause confusion of "it works in staging, but not in production".

It means that reviewers would need to take extra caution, requiring them to confirm whether new and existing environmental difference is intentional or not.

^{*1: There is another values.yaml, which was simply copied from the official repo. This is not used for any of the demo, but kept for completeness sake.}

Solutions

🎯 Additional Tooling for Better Diff

There are a few very useful tools to help:

• dyff: CLI tool to view the diff between two YAML inputs
• Helm Diff Plugin: Helm plugin to check what changes will take place

This repository has an automation setup to run dyff by comment. Comment with a simple line of /run-dyff (without backticks) can trigger dyff, and the command output will be written as a comment.

NOTE: This Pull Request is affected by the limitation below.

🎯 More Tooling for Environmental Diff

Just like dyff setup with how Kubernetes resources change before and after, you can set up another diff tooling between values-prod.yaml and values-staging.yaml.

Other Considerations and References

🚧 Tooling Limitations

(As of Sep 2021)

Dyff provides a nice diff view of 2 similar files, but it cannot show the diff if the number of documents in YAML does not match in those 2 files. This actually affects this particular change, as enabling some flags in Values often trigger creating or removing resources.

This is specific to Dyff in this case (helm-diff would be able to show the difference), but it is important to note how extra tooling has its own limitation to prevent smooth automation.

⛓ Automation Complexity

Although /run-dyff command is set up to help visualise the actual resource changes, it is not so straightforward to set up. It uses helm template to generate the Kubernetes resource definitions to compare before and after, and then runs dyff with those generated files.

If you want to ensure the deployed chart to be updated without surprises, you'd need more complex setup such as using Helm Diff Plugin with Kubernetes context made available to automation.

In either case, the automation setup tends to get more complicated when you need to deal with more than a single environment (e.g. production, staging, dev, etc.), and often requires custom setup to match your specific business requirements.

[rytswd]

/run-dyff

[github-actions]

Helm Diff

Diff based on the updated file of helm/flux

     _        __  __
 _| |_   _ / _|/ _|  between /tmp/tmp.mvSM5rJuy9/helm_template/prod/helm-flux-before.yaml, nine documents
/ _' | | | | |_| |_       and /tmp/tmp.mvSM5rJuy9/helm_template/prod/helm-flux-after.yaml, nine documents
| (_| | |_| |  _|  _|
\__,_|\__, |_| |_|   returned no differences
      |___/

Diff based on the updated file of helm/flux

[dnson]

/run-dyff