Windows support

[lelayf]

(Py37A) C:\Users\Francois>pip install pykeops
Collecting pykeops
  Downloading https://files.pythonhosted.org/packages/48/72/d1576e0841b1fa6dd65de4ef203362e5eb7748215005ace2975e12ac2679/pykeops-1.3.tar.gz (301kB)
     |████████████████████████████████| 307kB 731kB/s
    ERROR: Command errored out with exit status 1:
     command: 'c:\users\francois\venvs\py37a\scripts\python.exe' -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\Francois\\AppData\\Local\\Temp\\pip-install-hm6v38ko\\pykeops\\setup.py'"'"'; __file__='"'"'C:\\Users\\Francois\\AppData\\Local\\Temp\\pip-install-hm6v38ko\\pykeops\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base 'C:\Users\Francois\AppData\Local\Temp\pip-install-hm6v38ko\pykeops\pip-egg-info'
         cwd: C:\Users\Francois\AppData\Local\Temp\pip-install-hm6v38ko\pykeops\
    Complete output (9 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "C:\Users\Francois\AppData\Local\Temp\pip-install-hm6v38ko\pykeops\setup.py", line 11, in <module>
        from pykeops import __version__ as current_version
      File "C:\Users\Francois\AppData\Local\Temp\pip-install-hm6v38ko\pykeops\pykeops\__init__.py", line 34, in <module>
        from .common.utils import clean_pykeops
      File "C:\Users\Francois\AppData\Local\Temp\pip-install-hm6v38ko\pykeops\pykeops\common\utils.py", line 1, in <module>
        import fcntl
    ModuleNotFoundError: No module named 'fcntl'
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

(Py37A) C:\Users\Francois>pip install fcntl
ERROR: Could not find a version that satisfies the requirement fcntl (from versions: none)
ERROR: No matching distribution found for fcntl

a litlle more search shows that fcntl is only supported on Mac/Linux. It seems it is only used in one place to lock a file, which could be done in a more portable way with portalocker?

[fradav]

Hello, there is a development branch 4f1e6ba which solves the fnctl issue, but this isn't the only one. Pure C++ or numpy part of keops works fine for now.
Pytorch linking is still WIP.
I will update this issue, stay tuned.

Note that ninja, cmake, a working visual studio 2019 and llvm/clang compiler are required.

[fradav]

On hold : due to quarantine, I lost my access to a gpu for windows for testing the port.

[shaheenahmedc]

Hi All,

You may have seen my mention for this issue above.

PyKeOps is a dependancy in a recently released Gaussian Process Regression package, and I was just wondering if installing the above development branch will provide me with a Windows compatible version?

Apologies if you haven't managed to get round to it yet @fradav.

[scottgigante]

Windows support for this package would be much appreciated! Just posting to affirm that there is interest.

[fradav]

Due to various reasons, I'm withdrawing from the project from now on, starting a brand new project with similar goals.

Regards,

[lostmsu]

@fradav is Windows WIP available somewhere?
What is your new project?

[ancestor-mithril]

What is the status of this issue? Is there any plan for adding Windows support for pykeops?

[jeanfeydy]

Hi @ancestor-mithril,

Thanks for your interest in the library!
KeOps works fully on WSL, and @initialneil was kind enough to write a pull request for native support (#281).
I am currently in the process of integrating it - but you may want to try it now, as in #297.
Please feel free to let us know if you face any issue: we don't have much experience with Windows, and welcome any user feedback.

Best regards,
Jean

[An-u-rag]

Hi @jeanfeydy, thanks for continuing to work on Windows Integration.

I have tried the steps mentioned in #281 but I encountered an error. Would appreciate any help regarding this, thanks!

[initialneil]

Hi @An-u-rag, can you provide the cuda, pytorch version that you tested?
If you can use VSCode to debug step-by-step with F11, you can step to the codes around "Cuda libraries were not detected" to see why so.
More information would help to identify what's not working.

[An-u-rag]

Thanks for the reply @initialneil. Pytorch version is 1.13.1 and Cuda version is 11.7.

The test_keopscore.py file tries to import "get_compiler_library" function from the keops package located in miniconda3 site-packages directory. When I navigate to the given location's config.py file and look for it manually, I see that there is no implementation for that function in the config.py file.

However, when I access the local keopscore directory in the cloned keops local repo, I can find a similar config.py file which does seem to have the implementation for the same function.

I am not sure why this is the case. Would it be fine for me to copy this config.py file to the miniconda's site-package version of it or am I doing something wrong?

[initialneil]

@An-u-rag It seems that you did both python setup.py install and python setup.py develop.
Could you try pip uninstall pykeops and keopscore, and follow #281 step 5 to python setup.py develop. There should not be any source code in the site-packages other than a link file.

[An-u-rag]

@initialneil Yep that was the issue. Fixed it with pip uninstall. Thank you for all the help!

I'm not sure if this is a general concern or is it because of your changes to keops for windows but I am using a LazyTensor instantiation with Vi and Vj in my project and I seem to be getting these warnings from keops, is there a way I can suppress them?

[initialneil]

@An-u-rag Right cause this version is not officially tested and merged yet. These are normal logs for verbose purposes.

[An-u-rag]

@initialneil I tried working with it but I keep getting errors. Now, I can't even get the initial import to work due to a circular import error. I am not sure what is happening.

I might just wait for the official release with windows support. Thanks for the help anyway!

[Antoine-insa]

Hi @jeanfeydy,

I get the same error described initially by @lelayf concerning the missing module named fcntl and required for KeOps to work on Windows.
If you have any solution (apart from using Docker) I will be very glad to try it :).

Many thanks for your time in advance !

[jeanfeydy]

Hi @Antoine-insa ,

Thanks for your interest in our library!
The fcntl dependency was kindly removed by @initialneil in the PR #281. I am currently working on testing and integrating his work but this takes some time because:

1. I am 100% new to development on Windows, and didn't even know how to get Python to work...
2. My only Windows computer is a workstation that is located in my office, without remote access. That's not optimal ;-)
3. I would like to add a Windows computer to our CI pipeline before completing the merge.

In any case, you should be able to try the branch by following the instructions here. Please feel free to let us know about your experience with it.
Best regards,
Jean

[Antoine-insa]

Many thanks @jeanfeydy , I will try the instructions you've pinpointed.
Best regards,
Antoine

[abrahamezzeddine]

@jeanfeydy Hello,

do you know if there are any updates to have it work on Windows? Unfortunately, I get the following error

[KeOps] Generating code for formula Sum_Reduction((Var(0,3,0)-Var(1,3,1))|(Var(0,3,0)-Var(1,3,1)),1) ... OK

[KeOps] error: cuMemcpyDtoH(out, (CUdeviceptr) out_d, sizeof(TYPE) * sizeout) failed with error CUDA_ERROR_INVALID_VALUE

I do not know how to solve that because it’s beyond my knowledge of CUDA and C++ programming. I using PyTorch 2.5.1 and CUDA 12.4.

Many thanks. Really looking forward to try this out!