Documentation for 'The TE correlation coefficient of Planck legacy data' software
This page contains documentation for the codes used for the paper:
The TE correlation coefficient of Planck legacy data.
We explain how to run them at nersc, but this can be generalised to any computer cluster.
The codes can be found at: https://github.com/simonsobs/PSpipe/tree/master/project/correlation_coeff
The final results of the analysis are available at:
https://github.com/simonsobs/PSpipe/tree/master/project/correlation_coeff/results
WARNING: This doc is deprecated, please have a look at https://github.com/simonsobs/PSpipe/tree/master/project/Planck_pspy to reproduce planck spectra
First you need to install PSPy, detailed instruction to the installation at NERSC are provided in: https://github.com/simonsobs/PSpipe
Then you have to download the public 2018 planck data this can be achieved by running:
salloc -N 1 -C haswell -q interactive -t 01:00:00
python get_planck_data.py global.dict
Note that before doing so, you should specify the path to the data directory using the 'data_dir' key in global.dict.
Now we should compute planck power spectra, to do so, we proceed in two different steps:
- First compute the mode coupling matrices
- Compute the alms and the spectra
salloc -N 1 -C haswell -q interactive -t 2:00:00
export OMP_NUM_THREADS=64
python get_planck_mcm_Bbl.py global.dict
python get_planck_spectra.py global.dict
Request a node for 2 hours, define the number of openmp threads to be 64, run the code to compute the mode coupling matrices, run the code to compute the spectra.
To go further we will need to run simulations, we compute the measured noise spectra of Planck in model_planck_noise_spectra.py and we used a theory curve + the best fit Planck foregrounds to construct a signal matrix in get_planck_theory_and_fg.py
salloc -N 1 -C haswell -q interactive -t 01:00:00
python model_planck_spectra.py global.dict
python get_planck_theory_and_fg.py global.dict
In this section, we will describe how to redo Figure 2 of the paper, we will use the code to demonstrate the robustness of the correlation coefficent against instrumental systematics. First let's generate a fake beam and transfer function
salloc -N 1 -C haswell -q interactive -t 01:00:00
python systematic_model.py
Now lets generate 200 simulations with these fake beam and transfer function, and compute their spectra assuming the planck instrument model, on nersc that would be
salloc -N 10 -C haswell -q interactive -t 2:00:00
export OMP_NUM_THREADS=64
srun -n 10 -c 64 --cpu_bind=cores python planck_sim_spectra.py global_syst.dict
global_syst.dict is set up to generate only Planck simulation at 143 GHz, this should be enough for the illustration, it also make sure that we apply the systematic model. This should create a folder: 'sim_spectra_syst' containing the spectra from the simulations.
To analyse the simulations and reproduce the plot of the robustness of the correlation coefficent just run
salloc -N 1 -C haswell -q interactive -t 1:00:00
python planck_sim_analysis.py global_syst.dict
python plot_robustness.py global_syst.dict
This should reproduce a plot similar to: https://github.com/simonsobs/PSpipe/blob/master/project/correlation_coeff/figures/robustness_143x143.pdf
modulo the intrinsic scatter due to the randomness of the simulations.
To get the bias and covariance matrix of the correlation coefficient and compare it to theoretical expectations, you need first to generate many simulations (approx 1500), you can generate them interactively by bunch of 500. Change the iStart, iStop argument in global.dict to choose which bunch of simulation you want to generate (e.g iStart=0,iStop=500 then iStart=500, iStop=1000)
salloc -N 40 -C haswell -q interactive -t 2:00:00
export OMP_NUM_THREADS=64
srun -n 40 -c 64 --cpu_bind=cores python planck_sim_spectra.py global.dict
You can then analyse the simulations and plot the comparison using
salloc -N 1 -C haswell -q interactive -t 1:00:00
python planck_sim_analysis.py global.dict
python plot_bias_covariance.py global_syst.dict
This should reproduce a plot similar to: https://github.com/simonsobs/PSpipe/blob/master/project/correlation_coeff/figures/bias_and_cov.pdf
modulo the intrinsic scatter due to the randomness of the simulations.
For assessing the statistical properties of the Planck data, we used FFP10 planck simulations, planck released 300 sims for each split and frequency, we have downloaded them on nersc (since only 100 are available publicly on nersc) and have run
salloc -N 40 -C haswell -q interactive -t 2:00:00
export OMP_NUM_THREADS=64
srun -n 40 -c 64 --cpu_bind=cores python planck_sim_spectra_ffp10.py global_ffp10.dict
And to analyse and plot them
salloc -N 1 -C haswell -q interactive -t 1:00:00
python planck_sim_analysis.py global_ffp10.dict
python plot_planck_results.py global_ffp10.dict
python plot_planck_results_stacked.py global_ffp10.dict
This should reproduce the plots:
To reproduce the plot showing the dependency of the correlation coefficient to the choice of cosmological parameter, run
python plot_cosmo_variation.py
This should reproduce the plot: https://github.com/simonsobs/PSpipe/blob/master/project/correlation_coeff/figures/cosmo_dependency.pdf
If you want the same plot but now fixing the acoustic scale, change the flag use_H0=True in the code to False.