The procedure entry point PrefetchVirtualMemory could not be located in the dynamic link library KERNEL32.dll

[moon91210]

With version master-180b693, and every version after it, including master-2663d2c, when I run the following:

main.exe -m models/ggml-model-q4_1.bin -n 128 --repeat_penalty 1.7 --color -i -r "User: " -f prompts/chat.txt

An error message pops up that reads The procedure entry point PrefetchVirtualMemory could not be located in the dynamic link library KERNEL32.dll.
The last working version for me is: master-f2d1c47

[anzz1]

Implementing this: #890 (comment) will fix it.

[prusnak]

@moon91210 Which version of windows is that?

[moon91210]

@prusnak 7

[prusnak]

@anzz1 Are you sure your change works on Windows 7 and older? The error message reads like there is no function with that name contained in kernel32.lib

[anzz1]

@prusnak
Yes, 100%.

It's dynamically loading the library and thus can solve at runtime whether the function is available or not.

The implementation of loading of shared libraries in the Windows PE environment doesn't make the most sense as even though they are dynamic-link libraries, when importing them normally ( __declspec(dllimport) ) the compilers need to have the function export link definitions (.lib). However after compilation the imported functions are simply stored as C strings (or word ordinals) in the import address table so the part requiring libs is an unnecessary step, idk exactly why the logic was ever set up that way. Anyway, that's what we're stuck with now.

When you run an executable in Windows, before the program code runs the kernel will walk the IAT and load the modules one at a time and resolve the functions by reading the counterpart of IAT, export address table from the module in question and search for the function there. If the procedure aka function isn't found, the program will not start.

The exact same thing will happen however, if you call LoadLibrary/GetModuleHandle and GetProcAddress yourself, the difference being that you can now control the error status of what happens if the function cannot be found and that's what the code in the comment is doing.

By using GetProcAddress, the .lib entry isn't needed at all. The #pragma comment(lib,"kernel32.lib") in the comment is only for GetModuleHandleA and LoadLibraryA, as they are being imported and sure to always exist since Windows NT ever was a thing.

If the compilers were better, dynamically loading like this would be a simple preprocessor attribute on top of a function, but they aren't so that boilerplate abomination in #890 (comment) is required to dynamically load a library. While it's not pretty, it's fast and has no performance penalty though.

In this particular case, as kernel32.dll is already loaded the getmodulehandle/getprocaddress combo doesn't actually load anything, it simply finds the function which is (or isn't) already in memory. It's entirely possible knowing the PE structure to find the pointer to the function in memory manually without using a single function call, but it would make no sense to reinvent the wheel.

TL;DR; The code in the comment will let the program start even when the function doesn't exist and prints a warning instead informing the user of that.

[KrakenTales]

@moon91210 Exactly the same problem. Did you try anzz1' solution or another one?

[anzz1]

The complaint from the compiler is because windows.h already declares those functions, the example posted was a portable one which needn't include it. When the header is included, you'll need to forego the func declarations:

#ifdef _WIN32
#pragma comment(lib,"kernel32.lib")
//typedef struct _WIN32_MEMORY_RANGE_ENTRY {
//  void*  VirtualAddress;
//  size_t NumberOfBytes;
//} WIN32_MEMORY_RANGE_ENTRY, *PWIN32_MEMORY_RANGE_ENTRY;
//extern "C" __declspec(dllimport) void * __stdcall GetProcAddress(void * hModule, const char * lpProcName);
//extern "C" __declspec(dllimport) void * __stdcall GetModuleHandleA(const char * lpModuleName);
typedef int (__stdcall *LPFN_PREFETCHVIRTUALMEMORY)(void * hProcess, size_t NumberOfEntries, PWIN32_MEMORY_RANGE_ENTRY VirtualAddresses, unsigned long Flags);
LPFN_PREFETCHVIRTUALMEMORY fnPrefetchVirtualMemory = 0;
int _fnPrefetchVirtualMemory(void * hProcess, size_t NumberOfEntries, PWIN32_MEMORY_RANGE_ENTRY VirtualAddresses, unsigned long Flags)
{
  if(!fnPrefetchVirtualMemory) {
    fnPrefetchVirtualMemory = (LPFN_PREFETCHVIRTUALMEMORY)GetProcAddress(GetModuleHandleA("kernel32"),"PrefetchVirtualMemory");
    if(!fnPrefetchVirtualMemory) {
      printf("PrefetchVirtualMemory is not supported, use --no-mmap\n");
      return 0;
    }
  }
  return fnPrefetchVirtualMemory(hProcess, NumberOfEntries, VirtualAddresses, Flags);
}
#endif

You also ought to edit the WINNT_WIN8 part out:

        //#if _WIN32_WINNT >= _WIN32_WINNT_WIN8
        // Advise the kernel to preload the mapped memory
        WIN32_MEMORY_RANGE_ENTRY range;
        range.VirtualAddress = addr;
        range.NumberOfBytes = (SIZE_T)size;
        if (!_fnPrefetchVirtualMemory(GetCurrentProcess(), 1, &range, 0)) {
            fprintf(stderr, "warning: PrefetchVirtualMemory failed: %s\n",
                    llama_format_win_err(GetLastError()).c_str());
        }
        //#else
        //#pragma message("warning: You are building for pre-Windows 8; prefetch not supported")
        //#endif // _WIN32_WINNT >= _WIN32_WINNT_WIN8

[KrakenTales]

@anzz1 Wow! Thank you for detailed explanations! Now it works like an incredible charm (*0*)

[MasterDimmy]

please integrate this in code.
main.exe cant launch on windows 7

[feakk]

Following up to this issue with confirmation that @anzz1's patch applied at koboldcpp_win7_test.exe mentioned in #68 and perhaps lost in #890's merge works on Windows 7 for those still lagging OS behind.

[MasterDimmy]

Still waiting for this! We are billions with Win7! Please, implement this fix!

[badrelmers]

i will never leave win7 , please support win7

[sisi399]

Could someone please integrate the patch?

EDIT: Please correct me if I'm wrong, but I believe this issue can be closed since #2930 resolves it.

[EverythingForAI]

The complaint from the compiler is because windows.h already declares those functions, the example posted was a portable one which needn't include it. When the header is included, you'll need to forego the func declarations:

#ifdef _WIN32
#pragma comment(lib,"kernel32.lib")
//typedef struct _WIN32_MEMORY_RANGE_ENTRY {
//  void*  VirtualAddress;
//  size_t NumberOfBytes;
//} WIN32_MEMORY_RANGE_ENTRY, *PWIN32_MEMORY_RANGE_ENTRY;
//extern "C" __declspec(dllimport) void * __stdcall GetProcAddress(void * hModule, const char * lpProcName);
//extern "C" __declspec(dllimport) void * __stdcall GetModuleHandleA(const char * lpModuleName);
typedef int (__stdcall *LPFN_PREFETCHVIRTUALMEMORY)(void * hProcess, size_t NumberOfEntries, PWIN32_MEMORY_RANGE_ENTRY VirtualAddresses, unsigned long Flags);
LPFN_PREFETCHVIRTUALMEMORY fnPrefetchVirtualMemory = 0;
int _fnPrefetchVirtualMemory(void * hProcess, size_t NumberOfEntries, PWIN32_MEMORY_RANGE_ENTRY VirtualAddresses, unsigned long Flags)
{
  if(!fnPrefetchVirtualMemory) {
    fnPrefetchVirtualMemory = (LPFN_PREFETCHVIRTUALMEMORY)GetProcAddress(GetModuleHandleA("kernel32"),"PrefetchVirtualMemory");
    if(!fnPrefetchVirtualMemory) {
      printf("PrefetchVirtualMemory is not supported, use --no-mmap\n");
      return 0;
    }
  }
  return fnPrefetchVirtualMemory(hProcess, NumberOfEntries, VirtualAddresses, Flags);
}
#endif

You also ought to edit the WINNT_WIN8 part out:

        //#if _WIN32_WINNT >= _WIN32_WINNT_WIN8
        // Advise the kernel to preload the mapped memory
        WIN32_MEMORY_RANGE_ENTRY range;
        range.VirtualAddress = addr;
        range.NumberOfBytes = (SIZE_T)size;
        if (!_fnPrefetchVirtualMemory(GetCurrentProcess(), 1, &range, 0)) {
            fprintf(stderr, "warning: PrefetchVirtualMemory failed: %s\n",
                    llama_format_win_err(GetLastError()).c_str());
        }
        //#else
        //#pragma message("warning: You are building for pre-Windows 8; prefetch not supported")
        //#endif // _WIN32_WINNT >= _WIN32_WINNT_WIN8

@anzz1 where should I place/integrate these code? which file should I place/integrate these code? llama.cpp？llama.h? I am confused

[github-actions]

This issue was closed because it has been inactive for 14 days since being marked as stale.