Extend USAC coverage.

[tfunatomi]

Add estimateSE2(...), estimateSE3(...), estimateSIM2(...), estimateSIM3(...) for estimating an geometric transformation with rotation and translation (with scaling for SIM) using USAC: as alternative for estimateAffinePartial2D and estimateAffine3D.

Problem

Nice feature based on USAC: RANSAC-based universal framework has been introduced for the following functions.

• findHomography
• findFundamentalMat
• findEssentialMat
• solvePnPRansac
• estimateAffine2D

However, the following functions are not supported and only support legacy methods (RANSAC and LMEDS).

• estimateAffinePartial2D
• estimateAffine3D

Meanwhile, these functions provided to estimate a constrained geometric transformations consists of rotation, translation, and scaling with appropriate options.

Proposal

I really needed to estimate for the transformation in the class of Rigid/Similarity transform in 2D and 3D, denoted as $\mathit{SE}(d), \mathit{SIM}(d)$, where $d=2,3$.
In order to clarify the class of the transformation of the output, I propose to add the functions of estimateSE2(...), estimateSE3(...), estimateSIM2(...), estimateSIM3(...) as alternative for estimateAffinePartial2D and estimateAffine3D. These functions takes arguments compatible with the existing functions such as estimateAffine2D.

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

• I agree to contribute to the project under Apache 2 License.
• To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
• The PR is proposed to the proper branch
• There is a reference to the original bug report and related work
• There is accuracy test, performance test and test data in opencv_extra repository, if applicable
  Patch to opencv_extra has the same branch name.
• The feature is well documented and sample code can be built with the project CMake

[tfunatomi]

@alalek Thank you for your reaction.
I did not understand where the test modules are, but my colleague helped to identify it.
I added some modules to test my code and I've gotten the results as follows.

[----------] 1 test from usac_SE2
[ RUN ] usac_SE2.accuracy
[ OK ] usac_SE2.accuracy (579 ms)
[----------] 1 test from usac_SE2 (579 ms total)

[----------] 1 test from usac_SIM2
[ RUN ] usac_SIM2.accuracy
[ OK ] usac_SIM2.accuracy (575 ms)
[----------] 1 test from usac_SIM2 (575 ms total)

[----------] 1 test from usac_SE3
[ RUN ] usac_SE3.accuracy
[ OK ] usac_SE3.accuracy (1362 ms)
[----------] 1 test from usac_SE3 (1362 ms total)

[----------] 1 test from usac_SIM3
[ RUN ] usac_SIM3.accuracy
[ OK ] usac_SIM3.accuracy (1569 ms)
[----------] 1 test from usac_SIM3 (1569 ms total)

[savuor]

It's fully correct, it works and it should be merged. But:

• Why Partial solver is disabled? I've removed it until it's not in use.
• Weights are supported by the algorithm but never actually used. I think we may extend the interface by supporting weights (maybe in next PR?). We can also extend tests by making some ground truth tests.
• I've optimized the function solve_weighted_umeyama a bit: just 1 loop, almost no data copying, plain data types (can be easily vectorized in the future). It's 20% faster on my machine.
• I've pushed my changes into your branch, please check

[tfunatomi]

@savuor
I appreciate your time and efforts for reviewing the request.

• Why Partial solver is disabled? I've removed it until it's not in use.

I tried to cover many functions for SE2/SIM2/SE3/SIM3 by the Partial solver, but I gave up.
I forgot to remove the one. Thank you.

• Weights are supported by the algorithm but never actually used. I think we may extend the interface by supporting weights (maybe in next PR?). We can also extend tests by making some ground truth tests.

I do not understand the design concept of usac module, but I have kept the compatibility with AffineNonMinimalSolverImpl.
Such extension of supporting weights sounds nice :)

• I've optimized the function solve_weighted_umeyama a bit: just 1 loop, almost no data copying, plain data types (can be easily vectorized in the future). It's 20% faster on my machine.

I appreciate your labors for the optimization.

[asmorkalov]

Looks like the function is analog of estimateAffine2D and estimateAffine3D https://docs.opencv.org/4.x/d9/d0c/group__calib3d.html#ga27865b1d26bac9ce91efaee83e94d4dd. They already have overloads with USAC.

[tfunatomi]

@asmorkalov Thank you for your comment.

My estimateSE2() outputs a rigid transform with 3-DOF (rotation+translation) that is different from the output of estimateAffine2D, which has 6-DOF. Such a solution with some constraints is useful for applications like in-plane registration.

estimateSIM2() outputs a similarity transform with 4-DOF (scaling+rotation+translation), which estimateAffinePartial2D estimates, but there is no implementation with USAC as far as I understood.

I think estimateAffine3D does not have an overload with USAC as I stated:

However, the following functions are not supported and only support legacy methods (RANSAC and LMEDS).

• estimateAffinePartial2D
• estimateAffine3D

Meanwhile, these functions provided to estimate a constrained geometric transformations consists of rotation, translation, and scaling with appropriate options.

[asmorkalov]

I propose to extend existing API with USAC then, but not introduce the new one. @vpisarev What is your opinion?

[tfunatomi]

@asmorkalov

I propose to extend existing API with USAC then, but not introduce the new one.

I agree.
I was supposed to do that, but I was concerned about backward compatibility.
estimateAffinePartial2D and estimateAffine3D are similar but different in

• estimateAffinePartial2D only supports similarity transform (SIM2) but estimateAffine3D supports either rigid or similarity with its argument: scale
• estimateAffinePartial2D supports RANSAC but estimateAffine3D does NOT.

The main reason for making the new ones is that the function name sounds inappropriate: the estimates are NOT affine transform.
But, I would welcome your suggestions about the interface, of course.

[tfunatomi]

@asmorkalov
What shall I do for this?
Extending the existing API might lose compatibility, so I think the current form is the second best.

[tfunatomi]

I propose to extend existing API with USAC then, but not introduce the new one. @vpisarev What is your opinion?

@asmorkalov @vpisarev
1.5 years have past already... Can I have your opinion/decision for the next step?

[savuor]

By the way, we have Matx and Vec structs for that

[tfunatomi]

Yes, I prefer to use them, indeed.
But, I just follow the existing 2D version in favor.

class ReprojectionDistanceAffineImpl : public ReprojectionErrorAffine {
private:
const Mat * points_mat;
const float * const points;
float m11, m12, m13, m21, m22, m23;

[tfunatomi]

@asmorkalov Are there any actions I can take to get approved?

[tfunatomi]

Hi @asmorkalov,

It has been 1.5 years since I submitted a pull request, and it seems to have been forgotten.
I still believe that there is no alternative in this project, and this improvement is valuable for the community.

Could you please review the pull request when you have a moment?
If there are any changes needed or concerns, I'd be happy to address them.

Thank you for your time and consideration.

[tfunatomi]

@alalek @asmorkalov Gentle ping – would appreciate your thoughts on whether the current integration is sufficient or if any changes are still needed. Thank you.