Contents of the system tests archive

[MakisH]

Example of the contents of an archive:

I think we still need some changes:

• reference_results and reference-data seems to be confusing (which is what?). One of them needs to be renamed, maybe to results_to_compare.
• What is the tutorials_ref? It currently contains master, but in that job, I had asked for develop (see Cannot checkout tutorials develop in tests #399).
• The actual git commits of each repository are not reported anywhere. See Report git hash of each repository #401
• Do we need the metadata.yaml, docker-compose.tutorial.yaml, and docker-compose.field_compare.yaml? Are they only useful for debugging the system tests themselves? I would suggest to either delete them, or move them to a subfolder (e.g., system-tests-debug).

cc @valentin-seitz

[valentin-seitz]

* [ ]  Do we need the `metadata.yaml`, `docker-compose.tutorial.yaml`, and `docker-compose.field_compare.yaml`? Are they only useful for debugging the system tests themselves? I would suggest to either delete them, or move them to a subfolder (e.g., `system-tests-debug`).

One of the big benefits of using containers is, that its very easy to "debug" a problem with a precice component, without having to install all of them manually. So having the docker-compose.tutorial.yaml and the fieldcompare.yaml is a crucial step in reproducing the failed pipeline in your local machine. So i would def. keep them around. A related Issue is: #387 where we need to move some absolute paths into relative ones to allow this "debug the pipeline in your local machine" feature.

[MakisH]

Then, let's focus on what is currently not included (or how it is named), and remove stuff later.