test role via molecule

[ltalirz]

Add continuous integration tests via the molecule package that run on
Github actions.

Test log: https://github.com/ltalirz/ansible-cvmfs-client/runs/4886355529?check_suite_focus=true

[rptaylor]

@ltalirz sorry for the delay, are you still interested in this? I'm afraid I'm not familiar with github actions or molecule but it would be useful to have working.

[ltalirz]

Hey @rptaylor , sure if this is a welcome addition I'll have a go at getting this ready on the weekend

[ltalirz]

When not explicitly providing the CVMFS_SERVER_URL, I now get this:

TASK [cvmfs_contrib.cvmfs_client : CVMFS check setup] **************************
  fatal: [instance]: FAILED! => {"changed": false, "cmd": ["/usr/bin/cvmfs_config", "chksetup"], "delta": "0:00:19.574378", "end": "2022-07-24 20:29:34.598845", "msg": "non-zero return code", "rc": 1, "start": "2022-07-24 20:29:15.024467", "stderr": "", "stderr_lines": [], "stdout": "Warning: cache limit for soft.computecanada.ca is very low (below 1GB)\nWarning: cache limit for cvmfs-config.computecanada.ca is very low (below 1GB)\nError: required parameter CVMFS_SERVER_URL undefined for cvmfs-config.computecanada.ca", "stdout_lines": ["Warning: cache limit for soft.computecanada.ca is very low (below 1GB)", "Warning: cache limit for cvmfs-config.computecanada.ca is very low (below 1GB)", "Error: required parameter CVMFS_SERVER_URL undefined for cvmfs-config.computecanada.ca"]}

Not sure whether this is intentional...

[ltalirz]

After specifying CVMFS_SERVER_URL: http://cvmfs-cache.arbutus.cloud.computecanada.ca:3128 I'm now getting

  TASK [cvmfs_contrib.cvmfs_client : CVMFS probe] ********************************
  fatal: [instance]: FAILED! => {"changed": false, "cmd": ["/usr/bin/cvmfs_config", "probe"], "delta": "0:00:19.474367", "end": "2022-07-24 20:50:43.493059", "msg": "non-zero return code", "rc": 1, "start": "2022-07-24 20:50:24.018692", "stderr": "", "stderr_lines": [], "stdout": "Probing /cvmfs/soft.computecanada.ca... OK\nProbing /cvmfs/cvmfs-config.computecanada.ca... Failed!", "stdout_lines": ["Probing /cvmfs/soft.computecanada.ca... OK", "Probing /cvmfs/cvmfs-config.computecanada.ca... Failed!"]}

Would you happen to know what might be the issue here?

[rptaylor]

It is not normal to set CVMFS_SERVER_URL. That would cause the client to try connect to the same single server for every repository, even ones hosted by different organizations. CVMFS_SERVER_URL has to be removed. Besides, cvmfs-cache.arbutus.cloud.computecanada.ca is a squid proxy, not a stratum server anyway.

You also need to remove all CC config packages. Your setup is trying to open the cvmfs-config.computecanada.ca repo , but that should not be referenced anywhere. You must still have some leftover config after removing cvmfs-config-computecanada 776798f ?

If you grep canada -r /etc/cvmfs there should be nothing.

[ltalirz]

Hi @rptaylor, the purpose of this pull request is to set up continuous integration tests of the ansible role.
For this test it is necessary to pick a reliable server to connect to - otherwise we can't really test whether the client works (the alternative would be to spin up a separate CVMFS server inside the test environment, which would make the setup substantially more complicated).

The choice of the server only affects test runs. When people clone the repository and use the role, they still need to choose the server by themselves, just like before (as you can see, the role itself is left untouched by the pull request).
I personally don't care which server we pick - please let me know your suggestion - but we will need to pick one.

You must still have some leftover config after removing cvmfs-config-computecanada 776798f ?

Every test run starts completely from scratch. There is no interaction between the state of the git repository in different commits.

The outcome of the test is determined by what you see in the "files" tab, and you can see the test runs for each commit on https://github.com/ltalirz/ansible-cvmfs-client/actions
Once the PR is merged, there will be a little icon at each commit that brings you to the corresponding test run, just like here.

[rptaylor]

Thanks for the link to the actions, that helps show what is happening.
I fixed a minor issue #7 that would report a spurious error from the monitoring commands, which should not use cvmfs-config.computecanada.ca anymore (which would be noticed by having automated testing :) )

CVMFS users should never need to pick a reliable stratum server to connect to. If you found that information from any documentation let me know so I can clarify it. CVMFS_SERVER_URL, and other essential information for connecting to repos, is pretty much always distributed from upstream configuration sources (via packages or config repos), regular users should not change it and don't need to worry about it. This way it just works out of the box. (By use of a CDN, users also generally don't need to worry about selecting proxies either.)

I assure you that CVMFS_SERVER_URL: http://cvmfs-cache.arbutus.cloud.computecanada.ca:3128 is incorrect and should be removed. That would set a proxy server to be used as if it were a stratum server, and it would attempt to find all repositories there.
You can e.g. grep CVMFS_SERVER_URL -r /etc/cvmfs/ /cvmfs/cvmfs-config.cern.ch to see what the correct values are, there are dozens of servers in dozens of files.
However what is most mysterious is that this configuration is completely disregarded by the client, as shown by cvmfs_config showconfig | grep SERVER_URL. It must have some built-in failsafe that ignores invalid configuration that would break all repositories, maybe related to the URL not having the right structure (e.g. /cvmfs/@fqrn@)

[ltalirz]

Hi @rptaylor , thanks for the explanation!

I fixed a minor issue #7 that would report a spurious error from the monitoring commands, which should not use cvmfs-config.computecanada.ca anymore (which would be noticed by having automated testing :) )

Ok, I've merged master back into the branch.
The reason I tried adding the CVMFS_SERVER_URL is that, as explained above, I got an error message "Error: required parameter CVMFS_SERVER_URL undefined for cvmfs-config.computecanada.ca" when running the role.

I don't remember this happening before I rebased on your updated master some days ago, so perhaps it was a bug introduced by some intermediate changes and is fixed by #7 . Let's see...

[ltalirz]

Looks like it did :-)
https://github.com/ltalirz/ansible-cvmfs-client/actions

[DrDaveD]

(By use of a CDN, users also generally don't need to worry about selecting proxies either.)

That's true for small numbers of isolated clients, but if there are a lot of clients grouped together at a site, it is still important to have a local site proxy cache.

[rptaylor]

Yes, the original error message was better (clearer what the problem was).

Okay thanks @ltalirz ! I am only familiar with CI on gitlab; it seems more complicated on github but good to have automated testing.