How to Build

This guide describes the steps to build a Android/Windows release of the QNN backend.

Android

1. Install latest docker engine follow the official steps: Install Docker Engine
2. Clone my llama-cpp-qnn-builder repo (please update to latest main since we're using NDK r23, and there're some optimization flag not correctly applied in Release build, see: https://github.com/android/ndk/issues/1740)
3. In builder folder, run docker\docker_compose_compile_and_share.sh
4. Console output will look like this one, and exectualbe is located at build_qnn_arm64-v8a

Windows

Build with Visual Studio 2022

1. Download Qualcomm AI Engine Direct SDK, from here, and then extract it into a folder

2. Install the latest Visual Studio, make sure clang toolchain and 'cmake' are installed

3. Launch vs2022, tap Continue without code, and then in File menu, select Open -> CMake, in file open dialog, navigate to llama.cpp root directory, select CMakeLists.txt

4. Edit llama.cpp\CMakePresets.json, add following line to config arm64-windows-llvm

   {
       "name": "arm64-windows-llvm", "hidden": true,
       "architecture": { "value": "arm64",    "strategy": "external" },
       "toolset":      { "value": "host=x64", "strategy": "external" },
       "cacheVariables": {
   -        "CMAKE_TOOLCHAIN_FILE": "${sourceDir}/cmake/arm64-windows-llvm.cmake"
   +        "CMAKE_TOOLCHAIN_FILE": "${sourceDir}/cmake/arm64-windows-llvm.cmake",
   +        "GGML_QNN": "ON",
   +        "GGML_QNN_SDK_PATH": "<path to your Qualcomm AI Engine Direct SDK, like x:/ml/qnn_sdk/qairt/2.31.0.250130/>",
   +        "BUILD_SHARED_LIBS": "OFF"
       }
   },

5. Select config arm64-windows-llvm-debug

6. In Build menu, tap Build All, output file are located at build-arm64-windows-llvm-debug\bin\

7. Please copy those files from sdk lib directory (at qairt\2.31.0.250130\lib\aarch64-windows-msvc\) to the output directory before run:

QnnSystem.dll
QnnCpu.dll
QnnGpu.dll
QnnHtp.dll
QnnHtp*.dll