Add Loongson Advanced SIMD Extension support: -DCPU_BASELINE=LASX #21833

gititgo · 2022-04-07T11:32:19Z

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

I agree to contribute to the project under Apache 2 License.
To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
The PR is proposed to the proper branch
There is a reference to the original bug report and related work
There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
The feature is well documented and sample code can be built with the project CMake

asmorkalov · 2022-04-08T05:31:48Z

Could you point to cross-complier and may be some notes how to build the code. QEMU or some other emulator will be very useful too. Also I recommend you to take a look on cvRound implementation. It's used everywhere and efficient rounding affects performance a lot:

opencv/modules/core/include/opencv2/core/fast_math.hpp

Line 200 in 9aa6470

cvRound( double value )

gititgo · 2022-04-08T07:20:17Z

Could you point to cross-complier and may be some notes how to build the code. QEMU or some other emulator will be very useful too.

We are preparing such QEMU， but it is not finished yet. We'll provide it as soon as it's available.

TiredNotTear · 2022-04-25T03:09:23Z

modules/core/src/system.cpp

@@ -660,6 +662,10 @@ struct HWFeatures
        have[CV_CPU_RVV] = true;
    #endif

+    #if defined __loongarch_asx
+        have[CV_CPU_LASX] = true;
+    #endif


You are best to enable these SIMD features by reading the CPUID like ARM and x86.

You are best to enable these SIMD features by reading the CPUID like ARM and x86.

Ok，thanks very much.

After I copied the first patch to my local opencv repository (Tag: 4.5.4), the Calib3d_StereoBM.regression test failed.
Note that the cmake option I used on 3A5000 platform is " -DBUILD_PNG=ON -D WITH_OPENCL=OFF -DBUILD_WITH_DEBUG_INFO=ON -DCPU_BASELINE=LASX ".

I‘m so sorry that I didn't get the message or emails. I'm very glad you have a 3a5000 environment as We are considering providing 3a5000 remote environment.

YES, the Calib3d_StereoBM.regression test is currently failed. We suspect that compiler optimization is the reason as DEBUG version is OK. We will follow up and fix the bug.

@gititgo Do you have any news on the Calib3d_StereoBM.regression test failure?

We haven't found the cause of the problem yet. Our colleagues are still working on it.

gititgo · 2022-04-29T07:54:36Z

The cross-complier (build Loongarch on x86) and cmake config file is here:
git clone https://gitee.com/wenux/cross-compiler-la-on-x86.git

How to use it:
(1)tar -xvf toolchain-loongarch64-linux-gnu-cross-830-rc1.0-2022-04-22a.tar.xz
(2)Set "tools" in cmake config file (la64_linux_setup.cmake) to your real path;
(3)cmake with config file:
cmake -DCMAKE_TOOLCHAIN_FILE=path/to/your/la64_linux_setup.cmake -DCPU_BASELINE=LASX -DBUILD_OPENJPEG=ON ../
(4)make

gititgo · 2022-04-30T14:26:37Z

Is QEMU necessary for this PR ?

asmorkalov · 2022-05-04T05:51:11Z

It'll be great to have QEMU to run tests.

gititgo · 2022-05-18T11:19:41Z

It'll be great to have QEMU to run tests.

Is a 3A5000（Loongarch64） environment ok？ Because QEMU may take a long time.

fengyuentau · 2022-05-18T11:27:47Z

How long is it going to take to run all unit tests in QEMU? It is recommanded to have a CI pipeline to test code automatically. Please at least provide something that we can perform tests.

gititgo · 2022-05-19T08:10:41Z

How long is it going to take to run all unit tests in QEMU? It is recommanded to have a CI pipeline to test code automatically. Please at least provide something that we can perform tests.

QEMU is under development, but I haven't got the exact time. We can provide a remote Loongarch environment. Can this be used for automated testing ?

fengyuentau · 2022-05-20T06:58:27Z

Yes, but we are also expecting an environment which we can use for testing if your remote Loongarch environment is expired to us. So it is recommanded to have QEMU to run tests on Loongarch.

asmorkalov · 2022-05-20T07:14:48Z

The provided git link (git clone https://gitee.com/wenux/cross-compiler-la-on-x86.git) is protected by username and password.

gititgo · 2022-05-23T02:28:25Z

The provided git link (git clone https://gitee.com/wenux/cross-compiler-la-on-x86.git) is protected by username and password.

Sorry，it‘s ok now.

asmorkalov · 2022-05-23T05:41:13Z

opencv/modules/core/src/parallel_impl.cpp:63:5: warning: #warning "Can't detect 'pause' (CPU-yield) instruction on the target platform. Specify CV_PAUSE() definition via compiler flags." [-Wcpp]
 #   warning "Can't detect 'pause' (CPU-yield) instruction on the target platform. Specify CV_PAUSE() definition via compiler flags."

asmorkalov · 2022-05-23T05:59:57Z

Also the best way to enable OpenCV cross-compilation for the new architecture is to place toolchain file to opencv/platforms/linux/loongson. I propose to replace cmake local variable tools with CACHE variable with meaningful name to set it from command line. See opencv/platforms/linux/riscv64-clang.toolchain.cmake as example.

asmorkalov · 2022-05-23T06:00:57Z

@gititgo Thanks a lot for the toolchain. I was able to build the source code.

vpisarev · 2022-05-27T09:21:20Z

@gititgo, thank you for the contribution! Please, mark the proper items in the checklist:

  * I agree to contribute to the project under Apache 2 License.
  * To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV

without this confirmation we cannot merge your code into OpenCV

gititgo · 2022-05-27T10:38:04Z

without this confirmation we cannot merge your code into OpenCV

OK，marked

gititgo · 2022-05-30T08:35:24Z

@fengyuentau We prepared a remote LoongArch PC for test. The IP and password have been emailed to you.

fengyuentau · 2022-06-09T05:13:23Z

I run accuracy tests (./bin/opencv_test_*) on your LoongArch PC and only 5 modules (core, flann, highgui, ml and videoio) did not fail at Segmentation fault. The CMake command I used is cmake -B build -D CPU_BASELINE=LASX opencv. Did I miss something?

gititgo · 2022-06-09T07:06:31Z

Please -DBUILD_PNG=ON as there may be a bug in libpng in our system.

fengyuentau · 2022-06-09T12:11:36Z

Rebuilt with CMake option -DBUILD_PNG=ON and no more segmentation faults. But Calib3d_StereoBM.regression failed as discussed above.

gititgo · 2022-06-09T13:08:39Z

Yes，there are two known issues：
1、Calib3d_StereoBM.regression： may be the compiler optimization issue as DEBUG version is OK.
2、videoio/videocapture_acceleration.read（ffmpeg）： ffmpeg has a bug in our system （optimized on loongarch） as it‘s OK when we use Open source ffmpeg.

fengyuentau

Thanks for the contribution! Known issues are related to 3rdparty components on LoongArch PC, so they can be solved outside this pull request. Please take a look at my comments.

cmake/OpenCVCompilerOptimizations.cmake

cmake/checks/cpu_lasx.cpp

fengyuentau · 2022-08-14T06:24:51Z

Yes，there are two known issues： 1、Calib3d_StereoBM.regression： may be the compiler optimization issue as DEBUG version is OK. 2、videoio/videocapture_acceleration.read（ffmpeg）： ffmpeg has a bug in our system （optimized on loongarch） as it‘s OK when we use Open source ffmpeg.

Tested again, those known issues are still there:

# calib
[  FAILED  ] Calib3d_StereoBM.regression
# videoio
[  FAILED  ] videoio_ffmpeg.parallel
[  FAILED  ] videoio/videocapture_acceleration.read/32, where GetParam() = (sample_322x242_15frames.yuv420p.mpeg2video.mp4, FFMPEG, NONE, false)
[  FAILED  ] videoio/videocapture_acceleration.read/33, where GetParam() = (sample_322x242_15frames.yuv420p.mpeg2video.mp4, FFMPEG, NONE, true)
[  FAILED  ] videoio/videocapture_acceleration.read/34, where GetParam() = (sample_322x242_15frames.yuv420p.mpeg2video.mp4, FFMPEG, ANY, false)
[  FAILED  ] videoio/videocapture_acceleration.read/35, where GetParam() = (sample_322x242_15frames.yuv420p.mpeg2video.mp4, FFMPEG, ANY, true)
[  FAILED  ] videoio/videocapture_acceleration.read/64, where GetParam() = (sample_322x242_15frames.yuv420p.libx265.mp4, FFMPEG, NONE, false)
[  FAILED  ] videoio/videocapture_acceleration.read/65, where GetParam() = (sample_322x242_15frames.yuv420p.libx265.mp4, FFMPEG, NONE, true)
[  FAILED  ] videoio/videocapture_acceleration.read/66, where GetParam() = (sample_322x242_15frames.yuv420p.libx265.mp4, FFMPEG, ANY, false)
[  FAILED  ] videoio/videocapture_acceleration.read/67, where GetParam() = (sample_322x242_15frames.yuv420p.libx265.mp4, FFMPEG, ANY, true)
[  FAILED  ] videoio/videocapture_acceleration.read/80, where GetParam() = (sample_322x242_15frames.yuv420p.libvpx-vp9.mp4, FFMPEG, NONE, false)
[  FAILED  ] videoio/videocapture_acceleration.read/81, where GetParam() = (sample_322x242_15frames.yuv420p.libvpx-vp9.mp4, FFMPEG, NONE, true)
[  FAILED  ] videoio/videocapture_acceleration.read/82, where GetParam() = (sample_322x242_15frames.yuv420p.libvpx-vp9.mp4, FFMPEG, ANY, false)
[  FAILED  ] videoio/videocapture_acceleration.read/83, where GetParam() = (sample_322x242_15frames.yuv420p.libvpx-vp9.mp4, FFMPEG, ANY, true)
[  FAILED  ] videoio/videocapture_acceleration.read/96, where GetParam() = (sample_322x242_15frames.yuv420p.libaom-av1.mp4, FFMPEG, NONE, false)
[  FAILED  ] videoio/videocapture_acceleration.read/97, where GetParam() = (sample_322x242_15frames.yuv420p.libaom-av1.mp4, FFMPEG, NONE, true)
[  FAILED  ] videoio/videocapture_acceleration.read/98, where GetParam() = (sample_322x242_15frames.yuv420p.libaom-av1.mp4, FFMPEG, ANY, false)
[  FAILED  ] videoio/videocapture_acceleration.read/99, where GetParam() = (sample_322x242_15frames.yuv420p.libaom-av1.mp4, FFMPEG, ANY, true)

Tests on other modules passed.

gititgo · 2022-08-15T02:45:06Z

gititgo · 2022-08-15T02:49:43Z

Tested again, those known issues are still there

The version of ffmpeg has just been updated on the remote env, videoio module passed now.

asmorkalov · 2022-08-15T09:04:30Z

Thanks a lot for update! Please take a look on "docs" builder. It reports a lot of formatting issues like "modules/core/include/opencv2/core/hal/intrin_lasx.hpp:1865: trailing whitespace."

gititgo · 2022-08-15T11:03:00Z

OK， formatting issues are fixed.

fengyuentau · 2022-08-17T13:11:50Z

issue 1: core

The following test fails from time to time:

# core
[  FAILED  ] Core/HAL.mat_decomp/15, where GetParam() = 15

It fails most likely in the first run of a fresh complilation, and passes from the second run.

issue 2: highgui & gtk

And with the new ffmpeg, issues on videoio module were gone. However, it seems gtk is somehow misconfigured. At first, gtk was missing, but after installing gtk, the issue became:

[----------] 3 tests from Highgui_GUI
[ RUN      ] Highgui_GUI.regression
Exception message: OpenCV(4.6.0-dev) /home/loongson/opencv-workspace/opencv-21833/modules/highgui/src/window_gtk.cpp:635: error: (-2:Unspecified error) Can't initialize GTK backend in function 'cvInitSystem'

/home/loongson/opencv-workspace/opencv-21833/modules/highgui/test/test_gui.cpp:72: Failure
Expected: namedWindow(window_name) doesn't throw an exception.
  Actual: it throws.
[  FAILED  ] Highgui_GUI.regression (2 ms)
[ RUN      ] Highgui_GUI.trackbar_unsafe
Exception message: OpenCV(4.6.0-dev) /home/loongson/opencv-workspace/opencv-21833/modules/highgui/src/window_gtk.cpp:652: error: (-2:Unspecified error) GTK backend is not available in function 'cvInitSystem'

/home/loongson/opencv-workspace/opencv-21833/modules/highgui/test/test_gui.cpp:153: Failure
Expected: namedWindow(window_name) doesn't throw an exception.
  Actual: it throws.
[  FAILED  ] Highgui_GUI.trackbar_unsafe (0 ms)
[ RUN      ] Highgui_GUI.trackbar
Exception message: OpenCV(4.6.0-dev) /home/loongson/opencv-workspace/opencv-21833/modules/highgui/src/window_gtk.cpp:652: error: (-2:Unspecified error) GTK backend is not available in function 'cvInitSystem'

/home/loongson/opencv-workspace/opencv-21833/modules/highgui/test/test_gui.cpp:192: Failure
Expected: namedWindow(window_name) doesn't throw an exception.
  Actual: it throws.
[  FAILED  ] Highgui_GUI.trackbar (0 ms)
[----------] 3 tests from Highgui_GUI (2 ms total)

I guess this is another software compatibility issue.

asmorkalov · 2022-08-17T13:50:33Z

[ FAILED ] Core/HAL.mat_decomp/15, where GetParam() = 15 - it's compute test. Most probably it's a sign of UB somewhere in code or compiler issue.

asmorkalov · 2022-08-17T13:51:39Z

[ RUN ] Highgui_GUI.regression - please check if X session is available and properly configured. Otherwise you need to build OpenCV without UI support.

gititgo · 2022-08-18T01:27:27Z

[----------] 3 tests from Highgui_GUI

By using the following command to test on remote env, you can make the graphics display locally：
$ export DISPLAY=:0.0; ./bin/opencv_test_highgui

fengyuentau · 2022-08-18T02:07:47Z

[ FAILED ] Core/HAL.mat_decomp/15, where GetParam() = 15 - it's compute test. Most probably it's a sign of UB somewhere in code or compiler issue.

@asmorkalov Do you mean undefined behaviour by UB?

[ RUN ] Highgui_GUI.regression - please check if X session is available and properly configured. Otherwise you need to build OpenCV without UI support.

With the env set by export DISPLAY=:0.0, highgui tests are now passed.

By using the following command to test on remote env, you can make the graphics display locally：
$ export DISPLAY=:0.0; ./bin/opencv_test_highgui

Thanks!

asmorkalov · 2022-08-18T09:40:11Z

UB is undefined behavior, bug that triggers randomly depending on not initialized variable or stack content in case of out of bound access.

gititgo · 2022-08-23T03:11:07Z

core

[ FAILED ] Core/HAL.mat_decomp/15, where GetParam() = 15

This is a accuracy issue：
/home/loongson/src/wenxue/opencv/modules/core/test/test_hal_core.cpp:205: Failure Expected: (cvtest::norm(x, x0, NORM_INF | NORM_RELATIVE)) <= (eps), actual: 1.08289e-10 vs 1e-10

The reason is that the compiler uses fmadd class instructions to optimize the "multiply + add" operation, but precision loss may be triggered in parallel. We disable this feature on loongarch paltform by using the compiler option "-ffp-contract = off" and then the test case passes.

Now we add the compiler option "-ffp-contract = off" in opencv/modules/core/CMakeLists.txt. Is there a better place to add this option ?

asmorkalov · 2022-08-23T06:29:29Z

fmadd is important optimization. IMHO we can tune test threshold a bit, but not disable it. @alalek @vpisarev what do you think?

fengyuentau · 2022-09-02T07:59:16Z

@gititgo Any updates regarding the issue on the core module? Can we simply adjust the threshold specifically for LASX?

gititgo · 2022-09-02T08:43:26Z

yes, tunning the test threshold is a good idea. If you have no objection, I will do this specifically for LASX.

vpisarev · 2022-09-02T09:53:46Z

@fengyuentau, @gititgo, I'm fine with increasing tolerance threshold from 1e-10 to 5e-10, for example

fengyuentau

All is working fine except known issues. Thank you👍

asmorkalov

👍

alalek · 2022-09-11T20:51:15Z

This merge has been done without necessary references in commits message on this PR.

Add Loongson Advanced SIMD Extension support: -DCPU_BASELINE=LASX
    
    * Add Loongson Advanced SIMD Extension support: -DCPU_BASELINE=LASX
    * Add resize.lasx.cpp for Loongson SIMD acceleration
    * Add imgwarp.lasx.cpp for Loongson SIMD acceleration
    * Add LASX acceleration support for dnn/conv
    * Add CV_PAUSE(v) for Loongarch
    * Set LASX by default on Loongarch64
    * LoongArch: tune test threshold for Core/HAL.mat_decomp/15
    
    Co-authored-by: shengwenxue <shengwenxue@loongson.cn>

No PR ID at all.

Existed maintenance scripts would not care about this merge.

shengwenxue added 4 commits April 7, 2022 11:23

Add Loongson Advanced SIMD Extension support: -DCPU_BASELINE=LASX

4fafef4

Add resize.lasx.cpp for Loongson SIMD acceleration

652617c

Add imgwarp.lasx.cpp for Loongson SIMD acceleration

703527f

Add LASX acceleration support for dnn/conv

2d955d7

TiredNotTear reviewed Apr 25, 2022

View reviewed changes

Add CV_PAUSE(v) for Loongarch

5bcaf07

asmorkalov mentioned this pull request May 27, 2022

Optimize the cvCeil and cvFloor functions for LoongArch #22040

Merged

6 tasks

fengyuentau reviewed Jun 10, 2022

View reviewed changes

cmake/OpenCVCompilerOptimizations.cmake Outdated Show resolved Hide resolved

cmake/checks/cpu_lasx.cpp Show resolved Hide resolved

Set LASX by default on Loongarch64

ac5d410

Merge branch '4.x' into 4.x

4fb1ce6

gititgo closed this Aug 15, 2022

gititgo reopened this Aug 15, 2022

Fix formatting issues

d977588

Fix formatting issues

2208ace

LoongArch: tune test threshold for Core/HAL.mat_decomp/15

87aadf3

fengyuentau approved these changes Sep 3, 2022

View reviewed changes

asmorkalov approved these changes Sep 10, 2022

View reviewed changes

asmorkalov merged commit 4154bd0 into opencv:4.x Sep 10, 2022

asmorkalov added platform: loongson Loonson CPU architecure and LASX simd and removed platform: other labels Oct 19, 2022

Uh oh!

Add Loongson Advanced SIMD Extension support: -DCPU_BASELINE=LASX #21833

Add Loongson Advanced SIMD Extension support: -DCPU_BASELINE=LASX #21833

Uh oh!

Conversation

gititgo commented Apr 7, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Readiness Checklist

Uh oh!

asmorkalov commented Apr 8, 2022

Uh oh!

gititgo commented Apr 8, 2022

Uh oh!

TiredNotTear Apr 25, 2022

Choose a reason for hiding this comment

Uh oh!

gititgo Apr 30, 2022

Choose a reason for hiding this comment

Uh oh!

TiredNotTear May 9, 2022

Choose a reason for hiding this comment

Uh oh!

gititgo May 17, 2022

Choose a reason for hiding this comment

Uh oh!

asmorkalov Aug 12, 2022

Choose a reason for hiding this comment

Uh oh!

gititgo Aug 12, 2022

Choose a reason for hiding this comment

Uh oh!

gititgo commented Apr 29, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gititgo commented Apr 30, 2022

Uh oh!

asmorkalov commented May 4, 2022

Uh oh!

gititgo commented May 18, 2022

Uh oh!

fengyuentau commented May 18, 2022

Uh oh!

gititgo commented May 19, 2022

Uh oh!

fengyuentau commented May 20, 2022

Uh oh!

asmorkalov commented May 20, 2022

Uh oh!

gititgo commented May 23, 2022

Uh oh!

asmorkalov commented May 23, 2022

Uh oh!

asmorkalov commented May 23, 2022

Uh oh!

asmorkalov commented May 23, 2022

Uh oh!

vpisarev commented May 27, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gititgo commented May 27, 2022

Uh oh!

gititgo commented May 30, 2022

Uh oh!

fengyuentau commented Jun 9, 2022

Uh oh!

gititgo commented Jun 9, 2022

Uh oh!

fengyuentau commented Jun 9, 2022

Uh oh!

gititgo commented Jun 9, 2022

Uh oh!

fengyuentau left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

fengyuentau commented Aug 14, 2022

Uh oh!

gititgo commented Aug 15, 2022

Uh oh!

gititgo commented Apr 7, 2022 •

edited

Loading

gititgo commented Apr 29, 2022 •

edited

Loading

vpisarev commented May 27, 2022 •

edited

Loading