Improve conservative percentile calculation in live combining single significance fits

[GarethCabournDavies]

We realised that in some cases, the simple percentile of fit coefficient can be less conservative than the maximum likelihood method - particularly for separated fits with a large fit coefficient and a small number of triggers.

This is as the maximum likelihood calculation takes number of triggers into account, but the percentile does not, leading to low-trigger-count fits taking up much more importance than they should.

I have updated the calculation of the conservative fit coefficient to use a calculation more similar to the maximum likelihood method.

The standard calculation is mean(counts) / mean(counts / alpha)

I have updated the 'conservative' fit to use percentile(counts, 100 - p) / percentile(counts / alpha, p)

Using 100 - p for the numerator, but p for the denominator means a smaller fit coefficient, but I'm not sure if this skews the calculation too much.

As a result, I have included an alternative versiion using p for both as "ALT" in the results folder linked below (In fact, I think I prefer this).

Possible alternative methods

• since numpy v2, the percentile function allows for weights. Weighting by number of triggers, we become conservative again and are able to simply use the old calculation. This also allows the weighting to be a command line option if wanted.
• Include the errors from the fit coefficient calculation in the separated fit files, then combine the fits using alpha + sigma_alpha and count + sqrt(count) to give the conservative fit. This would remove the option of conservative percentile in the interface

See plots in https://ldas-jobs.ligo.caltech.edu/~gareth.cabourndavies/pycbclive/various_tests/single_significance_fits_weighted_percentile/

Standard information about the request

This is a new feature
This change affects the live search
This change changes, scientific output
This change follows style guidelines (See e.g. PEP8), has been proposed using the contribution guidelines

• The author of this pull request confirms they will adhere to the code of conduct

[tdent]

Yes, I don't quite see an argument for 1-p in the counts percentile, for several of the cases it seems to be fluctuating too far the other way (ie from being too large due to an outlier the coeff becomes probably too small).

So I vote ALT ... btw, good thing I'm not German given that ALT means 'old' 🇩🇪

[tdent]

Looks fine, see the suggested change