You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When running tests either repeatedly or the same tests on different system it is often useful to compare results. Currently, we store the test results in a csv file which is a ok, but a bit tedious to post-process in particular when trying to compare different results.
The proposal is to stash all the tests results as well as the system information and test meta data (such as start/end times etc) in a single json file with a reasonably well defined schema.
Once we have that we should be able to add simple tooling to read/compare multiple json files and generate, for example, a summary csv file from it.
The text was updated successfully, but these errors were encountered:
When running tests either repeatedly or the same tests on different system it is often useful to compare results. Currently, we store the test results in a
csv
file which is a ok, but a bit tedious to post-process in particular when trying to compare different results.The proposal is to stash all the tests results as well as the system information and test meta data (such as start/end times etc) in a single
json
file with a reasonably well defined schema.Once we have that we should be able to add simple tooling to read/compare multiple
json
files and generate, for example, a summarycsv
file from it.The text was updated successfully, but these errors were encountered: