GAC.1: Demonstrate ServiceX -> coffea -> cabinetry -> pyhf pipeline

[alexander-held]

Demonstrate a ServiceX -> coffea -> cabinetry -> pyhf pipeline on coffea-casa. This can be done by extending the ATLAS Open Data H>ZZ* example developed in stormsomething/CoffeaHZZAnalysis.

Target date: Dec 1, 2021

Requires

• Reading histogram input instead of ntuples scikit-hep/cabinetry#219
• notebooks need to run on coffea-casa Add quick fixes for Storm's coffea/servicex notebook #4

[alexander-held]

This pipeline was successfully demonstrated at the AGC tools workshop on Nov 4: https://indico.cern.ch/event/1076231/contributions/4560405/, running on Open Data coffea-casa. As an optional stretch goal, we can add the execution with Dask as well (scikit-hep/coffea#601).

[gordonwatts]

Woooo!

[alexander-held]

This milestone is done as of November 4 2021 when the required pipeline was demonstrated at the AGC workshop. The notebook showcasing this pipeline is provided on GitHub. A recording of the talk is attached to the agenda and available on YouTube.

The notebook runs on an open data instance of coffea-casa, receives columnar data from ServiceX following a func_adl query. The data is processed with coffea, resulting in the creation of histograms. Those histograms are assembled into a statistical model using cabinetry, which calls pyhf for statistical inference. The demonstration includes examples of different types of systematic uncertainties, and also showcases the interoperability of different tools within the ecosystem.