How to activate BLAS? #627
Comments
|
Please review and use the issue template before submitting new issues |
gjmulder
added
the
need more info
|
You'll need to edit the Makefile; look at the LLAMA_OPENBLAS section. Make sure the correct library and include paths are set for the BLAS library you want to use. You'll also need to set LLAMA_OPENBLAS when you build; for example, add LLAMA_OPENBLAS=yes to the command line when you run make. You might not see much improvement; the limit is likely memory bandwidth rather than processing power, and shuffling data between memory and the GPU might slow things down, but it's worth trying. |
OpenBLAS just uses the CPU. I think there are other, separate BLAS implementations that may use GPU but that wouldn't apply in this case. I gave it a try. You'll probably need both openblas and cblas installed. You may also need to add |
|
On Windows I just gave up, wasted 3 hours of my life trying to get that BLASmonster somehow working. The best amount of chaos I had with that: -- Though that still doesn't properly load the DLL |
|
@cmp-nct it's ok I found how to make it work. 1 - Download this https://github.com/xianyi/OpenBLAS/releases/tag/v0.3.22 8 - On your CMakeList.txt, add this line just after "# Build libraries" And there you go... gosh why is it so complicated to install a library seriously 😞 |
|
@BadisG adding the folder to the %PATH% makes cmake find OpenBLAS? It doesn't change anything for me. Switching the LLAMA_OPENBLAS option to ON just enables the broken cmake find_package() check for OpenBLAS, which keeps failing, no matter what I do. That will actually make it build with openblas (it actually depends on libopenblas.dll now). |
In addition to that I had to add the -DBLAS_LIBRARIES= flag into settings.json as configurationArgs entry. Now it indeed is compiled, also detected as BLAS = 1 I guess it might help to use the Intel BLAS lib, I just dislike adding 3.5 GB of libraries for a math multiplication. |
|
@cmp-nct If I had to guess I don't think adding the Intel BLAS lib will change anything, I'm pretty sure it's similar to the github BLAS I found. So same, I didn't see significant increase of performance I also wasted my time 😅 |
Intel has a couple benchmarks up, significant differences to the open blas. The same crap with the makefileon VScode I need to get llama compiled (llama.cpp is written awesomely). |
I was able to compile on Windows adding the following to CMakeLists.txt: Now talking about potential performance gains, it may be my impression, but I notice some gain in the response time of the model in relation to the binary compiled without OpenBLAS, but I will still do more tests. For now, using the llama 7b template using llama.cpp (master) my initial result is: |
|
I was able to turn it on using the following for Windows 11 WSL2 Ubuntu 20.04: Next time you run llama.cpp you'll have BLAS turned on. |
|
It was a little different on Debian 11. There are different versions available: libopenblas[0 | 64[-openmp | -pthread | -serial]]-dev.
Next time you run llama.cpp you'll have BLAS turned on. |
|
Hi, first of all @ggerganov thank you for this awesome project! Problem As a Windows user I also struggled to build llama.cpp with CMake and BLAS acceleration via OpenBLAS. The underlying problem seems to be that I debugged the CMake problem a bit further and described my findings there: Workaround Based on the description of @tomsnunes I have found a reliable way to build llama.cpp with an OpenBLAS library binding on Windows with CMake.
# This is a workaround for a CMake bug on Windows to build llama.cpp
# with OpenBLAS. The find_package(BLAS) call fails to find OpenBLAS,
# so we have to link the 'libopenblas.dll' shared library manually.
#
# @see https://github.com/ggerganov/llama.cpp/issues/627
# @see https://discourse.cmake.org/t/8414
#
if (LLAMA_BLAS AND DEFINED LLAMA_BLAS_VENDOR)
if (${LLAMA_BLAS_VENDOR} MATCHES "OpenBLAS")
set(LLAMA_EXTRA_INCLUDES ${LLAMA_EXTRA_INCLUDES} "C:/absolute/path/to/OpenBLAS/include")
set(LLAMA_EXTRA_LIBS ${LLAMA_EXTRA_LIBS} "C:/absolute/path/to/OpenBLAS/lib/libopenblas.dll.a")
add_compile_definitions(GGML_USE_OPENBLAS)
endif()
endif()You can then build Automation Since This enables you to simply execute the following to keep up to date with the ./rebuild_llama.cpp.ps1 -blasAccelerator "OpenBLAS"Next Steps I think eventually this problem should be fixed in the scope of CMake to not clutter the @ggerganov what do you think? |
I do not see the library files here |
Try installing one of the other versions like |
|
This issue was closed because it has been inactive for 14 days since being marked as stale. |
|
I was compiling whisper.cpp and got the same error. I looked at CMakeLists.txt and reviewed documentation for FindBLAS. Tried supplying -DOpenBLAS to the build line. That didn't work. But grep BLAS revealed more options to try.
|
|
I seem to have found an incantation that works. I hope it helps and I hope it works for everyone. But, YMMV!!! As with the Trolls, and the Whimsy Wasps, sometimes its best to be serious and sometimes its best to be whimsical: |
|
I thought -DBLA_SIZEOF_INTEGER=8 might offer a little boost with quantized models on non-GPU-accl. systems. |
Hello,
I've heard that I could get BLAS activated through my intel i7 10700k by installing this library.
Unfortunatly, nothing happened, after compiling again with Clung I still have no BLAS in llama.cpp
Maybe it's just not possible I don't know, I need someone to tell me the truth 😅
The text was updated successfully, but these errors were encountered: