feat: create tables for HEPData

.gitignore

@@ -33,3 +33,30 @@ analyses/cms-open-data-ttbar/metrics
 
 # dask
 dask-worker-space/
+analyses/cms-open-data-ttbar/first_option.py
+analyses/cms-open-data-ttbar/reana_higgs_tautau.yaml
+analyses/cms-open-data-ttbar/reana_samples.yaml
+analyses/cms-open-data-ttbar/Sample_execution.py
+analyses/cms-open-data-ttbar/out.ipynb
+analyses/cms-open-data-ttbar/.snakemake/log/2023-11-13T230445.475964.snakemake.log
+analyses/cms-open-data-ttbar/samples.py
+analyses/cms-open-data-ttbar/uproot.py
+.bash_history
+.cache/matplotlib/fontlist-v330.json
+.ipython/profile_default/startup/README
+analyses/cms-open-data-ttbar/loops.py
+.DS_Store
+analyses/cms-open-data-ttbar/ca-certificates/cerngridca.crt
+analyses/cms-open-data-ttbar/ca-certificates/cernroot.crt
+HEPData_cabinetry/.DS_Store
+analyses/cms-open-data-ttbar/Dockerfile_root_certificates
+.DS_Store
+analyses/cms-open-data-ttbar/Reproducibility_REANA/sample_single_top_s_chan__nominal_paths.txt
+analyses/cms-open-data-ttbar/Reproducibility_REANA/sample_single_top_t_chan__nominal_paths.txt
+analyses/cms-open-data-ttbar/Reproducibility_REANA/sample_single_top_tW__nominal_paths.txt
+analyses/cms-open-data-ttbar/Reproducibility_REANA/sample_ttbar__ME_var_paths.txt
+analyses/cms-open-data-ttbar/Reproducibility_REANA/sample_ttbar__nominal_paths.txt
+analyses/cms-open-data-ttbar/Reproducibility_REANA/sample_ttbar__PS_var_paths.txt
+analyses/cms-open-data-ttbar/Reproducibility_REANA/sample_ttbar__scaledown_paths.txt
+analyses/cms-open-data-ttbar/Reproducibility_REANA/sample_wjets__nominal_paths.txt
+analyses/cms-open-data-ttbar/Reproducibility_REANA/sample_ttbar__scaleup_paths.txt

analyses/cms-open-data-ttbar/README

@@ -0,0 +1,6 @@
+This is the running Analysis Grang Challenge using REANA
+REANA is a reproducible analysis platform for 
+For this you firstly need to  install reana virtual environment
+Further install reana-client 
+you can execute the ttbar analysis.ipynb using this command
+Now you can check your repository.

analyses/cms-open-data-ttbar/README.md

@@ -18,6 +18,7 @@ This directory is focused on running the CMS Open Data $t\bar{t}$ analysis throu
 | models/                       | Contains models used for ML inference task (when `USE_TRITON = False`)                                                                          |
 | utils/                        | Contains code for bookkeeping and cosmetics, as well as some boilerplate. Also contains images used in notebooks.                               |
 | utils/config.py               | This is a general config file to handle different options for running the analysis.                               |
+| utils/hepdata.py              | This is the .py file with function which would create a tables which would be submitted and stored into the [HEP_DATA website](https://www.hepdata.net) (use `HEP_DATA = True`)     |
 
 #### Instructions for paired notebook
 
@@ -51,3 +52,40 @@ argument is the appropriate reference file for the number of files per process a
 For full usage help see the output of `python validate_histograms.py --help`.
 
 `validate_histograms.py` can also be used to create new references by passing the `--dump-json` option.
+
+#### HEP data creation and submision.
+For proper submission, you need to modify the `submission.yaml` with proper explanation of variables and your table.
+To submit the created histograms to HEP data,, you'll need to install the necessary packages and make some modifications to `ttbar_analysis_pipeline.ipynb` notebook.
+``` console
+pip install hepdata_lib hepdata-cli
+```
+and the root (for current version of it is mandatory)
+```console 
+ mamba install -c conda-forge root htcondor=10.8 -y
+```
+Next, modify the notebook to enable the submission in one run. You'll need to create a zip archive of your data for uploading.
+
+```python
+import shutil
+folder_path = "hepdata_model" #name of the folder which was created wiht hepdata syntax
+zip_filename = "hepdata_model.zip"
+temp_folder = "temp_folder"
+# Create a temporary folder without unwanted files
+shutil.copytree(folder_path, temp_folder, ignore=shutil.ignore_patterns('.ipynb_checkpoints'))
+# Create the archive from the temporary folder
+shutil.make_archive(zip_filename, 'zip', temp_folder)
+# Remove the temporary folder
+shutil.rmtree(temp_folder)
+```
+
+```python
+from getpass import getpass
+import os
+# Get the password securely
+password = getpass("Enter your password: ")
+
+command = f"hepdata-cli upload '/home/cms-jovyan/analysis-grand-challenge/analyses/cms-open-data-ttbar/hepdata_model.zip.zip' -e yourname.yoursurname@cern.ch"
+os.system(f'echo {password} | {command}') #insert your passport in the actived window
+```
+If the submission is successful, you'll see your uploaded data in the provided link.
+

analyses/cms-open-data-ttbar/cabinetry_config.yml

@@ -121,4 +121,4 @@ NormFactors:
   - Name: "ttbar_norm"
     Samples: "ttbar"
     Nominal: 1.0
-    Bounds: [0, 10]
+    Bounds: [0, 10]

analyses/cms-open-data-ttbar/cabinetry_config_ml.yml

@@ -214,4 +214,4 @@ NormFactors:
   - Name: "ttbar_norm"
     Samples: "ttbar"
     Nominal: 1.0
-    Bounds: [0, 10]
+    Bounds: [0, 10]

analyses/cms-open-data-ttbar/ttbar_analysis_pipeline.ipynb

@@ -1299,6 +1299,21 @@
     "    utils.plotting.plot_data_mc(model_prediction, model_prediction_postfit, data_ml, config_ml)"
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "54967894",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "if utils.config[\"preservation\"][\"HEP_DATA\"] == True:\n",
+    "    import utils.hepdata\n",
+    "    #Submission of model prediction\n",
+    "    utils.hepdata.submission_hep_data(model, model_prediction, \"hepdata_model\")\n",
+    "    #Submission of model_ml prediction\n",
+    "    utils.hepdata.submission_hep_data(model_ml, model_prediction_ml,\"hepdata_model_ml\")"
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "a2ce2d14",