how to improve the speed of network:forward

[perp]

I am using cpu only,and it takes about 4500ms to using the function network:forward to process an image(96x96).
how to use OpenMP or OpenCL to improve the speed?
And it is very strange that I use torch.setnumthreads(1) is much more faster than torch.setnumthreads(4)

[perp]

This must be the reason
torch/image#7

[ExenVitor]

Yes, I can confirm this. When I compile OpenBlas for Android-Torch, it becomes much more faster than before. But it only works on the version of ARM64. Because OpenBlas only supports hard-float for ARMV7.

[antoniosimunovic]

Hi ExenVitor, care to write more detailed instructions on how to compile torch-android with OpenBlas? Thanks!

[antoniosimunovic]

Let me leaborate, I'm building for ARM64 (ARCH=${ARCH:-"v8"}) and linking OpenBLAS library /opt/OpenBLAS/lib/libopenblas.a

I get this error:

/opt/OpenBLAS/lib/libopenblas.a: error adding symbols: File in wrong format
collect2: error: ld returned 1 exit status
make[2]: *** [../install/libs/arm64-v8a/libTH.so] Error 1
make[2]: Leaving directory /home/antonio/torch-android/build' make[1]: *** [distro/pkg/torch/lib/TH/CMakeFiles/TH.dir/all] Error 2 make[1]: Leaving directory/home/antonio/torch-android/build'
make: *** [all] Error 2

OpenBlas compiled using this command:

make TARGET=ARMV8 BINARY=64 HOSTCC=gcc CC=aarch64-linux-android-gcc NOFORTRAN=1 USE_OPENMP=1

[ExenVitor]

@antoniosimunovic Sorry for the late reply. It looks like you are linking System's OpenBLAS library. You need install the libopenblas.a into ${INSTALL_DIR} after compiling. Distro will find it from ${INSTALL_DIR} first.
Just like this:
$MAKE $MAKEARGS TARGET=${OPEN_BLAS_TARGET} BINARY=${OPEN_BLAS_BINARY} HOSTCC="$HOST_CC" CC="${HOST}-gcc" NOFORTRAN=1\ && $MAKE $MAKEARGS TARGET=${OPEN_BLAS_TARGET} BINARY=${OPEN_BLAS_BINARY} HOSTCC="$HOST_CC" CC="${HOST}-gcc" NOFORTRAN=1 PREFIX=${INSTALL_DIR} install
I also create a branch for testing, you can find it here

[antoniosimunovic]

Thanks, that is the solution! I've put the install script before the make command in build.sh.

[LittlePeng]

check notfound

-- Checking for [openblas]
--   Library openblas: /root/torch/torch-android/install/lib/libopenblas.a
-- Checking for [openblas - pthread]
--   Library openblas: /root/torch/torch-android/install/lib/libopenblas.a
--   Library pthread: BLAS_pthread_LIBRARY-NOTFOUND
-- Checking for [goto2 - gfortran]
--   Library goto2: BLAS_goto2_LIBRARY-NOTFOUND
-- Checking for [goto2 - gfortran - pthread]
--   Library goto2: BLAS_goto2_LIBRARY-NOTFOUND
-- Checking for [acml - gfortran]
--   Library acml: BLAS_acml_LIBRARY-NOTFOUND
-- Checking for [Accelerate]
--   Library Accelerate: BLAS_Accelerate_LIBRARY-NOTFOUND
-- Checking for [vecLib]
--   Library vecLib: BLAS_vecLib_LIBRARY-NOTFOUND
-- Checking for [ptf77blas - atlas - gfortran]
--   Library ptf77blas: BLAS_ptf77blas_LIBRARY-NOTFOUND
-- Checking for [blas]
--   Library blas: BLAS_blas_LIBRARY-NOTFOUND
-- Cannot find a library with BLAS API. Not using BLAS.

ll /root/torch/torch-android/install/lib/
drwxr-xr-x 6 root root     4096 Dec 12 20:41 ./
drwxr-xr-x 8 root root     4096 Dec 12 19:38 ../
drwxr-xr-x 3 root root     4096 Dec 12 19:38 cmake/
-rw-r--r-- 1 root root   572836 Dec 12 19:41 libluajit.so
-rw-r--r-- 1 root root    33612 Dec 12 19:46 libluaT.so
lrwxrwxrwx 1 root root       28 Dec 12 20:41 libopenblas.a -> libopenblas_armv7p-r0.2.19.a
-rw-r--r-- 1 root root 20318340 Dec 12 20:41 libopenblas_armv7p-r0.2.19.a
-rwxr-xr-x 1 root root 12963860 Dec 12 20:41 libopenblas_armv7p-r0.2.19.so*
-rwxr-xr-x 1 root root 12963860 Dec 12 20:41 libopenblas.so*
-rwxr-xr-x 1 root root 12963860 Dec 12 20:41 libopenblas.so.0*
drwxr-xr-x 2 root root     4096 Dec 12 20:40 libpng/
-rw-r--r-- 1 root root   537698 Dec 12 19:38 libpng16.a
-rw-r--r-- 1 root root   358080 Dec 12 19:38 libpng16.so
lrwxrwxrwx 1 root root       10 Dec 12 19:38 libpng.a -> libpng16.a
lrwxrwxrwx 1 root root       11 Dec 12 19:38 libpng.so -> libpng16.so
-rw-r--r-- 1 root root  1967524 Dec 12 19:45 libTH.so
drwxr-xr-x 3 root root     4096 Dec 12 19:48 lua/
drwxr-xr-x 2 root root     4096 Dec 12 19:38 pkgconfig/

[huyn]

Thanks @ExenVitor for your excellent contribution