Fixing how stdDev in calibration is calculated

[savuor]

Fixes #19803

This PR should fix the standard deviation calculation for calibration procedure.

TODOs:

• Agree on the root cause of the problem
• (Needs discussion) total should contain 2x the number of object points (one per each coordinate)
• Forbid calibration calculation when a number of variables is less than a total number of residuals
• Forbid stddev calculation when a number of variables is greater than or equal to a total number of residuals
• Fix backport to 4.x
• Make backport for 3.x

EDIT: the previous stddev calculation motivation is considered to be wrong, we need that denominator correction to make our estimates unbiased. An explanation of it can be found in [MVG].
Also, Cramer-Rao bound has nothing to do with it.

Previous incorrect explanation

The stddevs estimation is based on Cramer-Rao bound. While it's not said about the variance of samples, the original code (Camera Calibration Toolbox by Jean-Yves Bouguet) and other available implementations calculate sigma2 based on all object points regardless how many are of them or how many parameters are to optimize.

I invite anyone to discuss that and verify that, especially those who've participated in the original issue discussion:
@Seneral @sanketc001 @lamm45 @alalek

I also invite @vpisarev and @ivashmak because of their experience in calibration.

• MVG: R. Hartley, A. Zisserman, Multiple View Geometry in Computer Vision, 2004, section 5.1.3, page 134

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

• I agree to contribute to the project under Apache 2 License.
• To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
• The PR is proposed to the proper branch
• There is a reference to the original bug report and related work
• There is accuracy test, performance test and test data in opencv_extra repository, if applicable
  Patch to opencv_extra has the same branch name.
• The feature is well documented and sample code can be built with the project CMake

[asmorkalov]

Why not 3.4 or 4.x?

[savuor]

@asmorkalov backport to 4.x: #22996

[lamm45]

Thank you for the invitation! I still think that the existing formula is correct. The proposed formula for sigma2 is only asymptotically correct (that is, when the number of points is huge).

As a "proof", the existing formula seems to be equivalent to the equation (1) of this paper: https://arxiv.org/abs/2107.13484

The issue isn't really specific to camera calibration. Here is a general explanation of why the number of parameters needs to be included in the denominator: https://stats.stackexchange.com/a/64495

I find the Cramer-Rao lower bound a bit confusing and unrelated to this discussion.

[savuor]

@lamm45 Thanks for your contribution!

Not sure if I've really got that explanation on StackExchange (while I've found another reference to it in the Multiple View Geometry in Computer Vision book, see section 5.1.3 at page 134).

Anyway, if we believe in it, then the best solution would be to forbid the calibration when total < nparams_nz since all the calibration makes no sense.

(Maybe this condition is incorrect since each point gives at least 2 equations - one per coordinate - and the prohibition bound should be twice bigger here)

And to leave stddevs empty when total == nparams_nz.

What do you think?

[vpisarev]

my memories about this part of statistics are very fuzzy, but if I recall correctly the formula you propose is sort of biased estimate of the standard deviation. I would keep the original formula and, as you suggest, avoided cases when total is less than nparams_nz.

[lamm45]

(Maybe this condition is incorrect since each point gives at least 2 equations - one per coordinate - and the prohibition bound should be twice bigger here)

I think you are right, the error estimator should use 2*total. And not just for the prohibition bound, but for the actual error computation as well. So basically all estimates have been wrong by up to a factor of two, which is quite significant.

[savuor]

I've finally understood that denominator correction. While it's more related to ordinary least squares estimation variance, I also found a more general explanation of it in MVG book (see the reference in the PR description).
This explanation also contain a note that is strictly related to the calibration problem: when fitting model in the projection of N points, a dimensionality of residuals should be 2*N, not N.
At the same time, I think that error computation (not stddev) is OK since this is how a reprojection error should work: an average across all the points.
So basically the only place where total should be fixed is stddev estimation.

[lamm45]

I should have been more precise, but when I said error estimator or computation, I meant the estimator or computation for the standard error. In this context, standard error is essentially the same as stddev, but I prefer the former.

[savuor]

@asmorkalov Backport to 3.4: #23025

[alalek]

Need align 3.4 / 4.x and this 5.x PRs to minimize conflicts.

[savuor]

@alalek Backport to 4.x is aligned with 5.x code, please check: #22996

[savuor]

Merged through #23189