Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Raymarching fails to install, cant find c++17 for PyTorch, SOS help #358

Open
SubhamBurnwal-BiswaGames opened this issue May 4, 2024 · 1 comment
Labels
bug Something isn't working

Comments

@SubhamBurnwal-BiswaGames

Description

Tried setting up this project on local machine but cannot install Raymarching, and other extensions due to various errors. I have solved some step by step but stuck right now.

Steps to Reproduce

I am following the Instant-NGP backbone collab notebook provided in README.md, locally on my machine:
https://colab.research.google.com/drive/1MXT3yfOFvO0ooKEfiUUvTKwUkrrlCHpF?usp=sharing

  1. Created new conda environment starting with python latest.
  2. Loaded the above colab file with this environment
  3. It showed I need to install ipykernel to run the notebook in this environment, so I did:
    ! conda install ipykernel
  4. Installed Pytorch
    ! conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia
  5. Checked installation by importing torch:

image
image

  1. git-cloned the repository as given in the notebook.

  2. once inside the repo, I commented out "torch" since we already installed above
    (this was suggested in another issue solution here)

  3. I install remaining dependencies: ! pip install -r requirements.txt

  4. Everything installed except PyMCubes. The error was it could not find VS C++17 to build.
    It was suggested online that I dont need to build it just install using this, so I did:
    ! pip install PyMCubes --no-cache-dir

  5. Now I reached this portion of building/installing extensions:
    The first extension was raymarching
    ! pip install ./raymarching
    I got this error:
    "Could not locate a supported Microsoft Visual C++ installation"

  6. I tracked this error to the file: stable-dreamfusion/raymarching/setup.py

import os
from setuptools import setup
from torch.utils.cpp_extension import BuildExtension, CUDAExtension

_src_path = os.path.dirname(os.path.abspath(__file__))

nvcc_flags = [
    '-O3', '-std=c++14',
    '-U__CUDA_NO_HALF_OPERATORS__', '-U__CUDA_NO_HALF_CONVERSIONS__', '-U__CUDA_NO_HALF2_OPERATORS__',
]

if os.name == "posix":
    c_flags = ['-O3', '-std=c++14']
elif os.name == "nt":
    c_flags = ['/O2', '/std:c++17']

    # find cl.exe
    def find_cl_path():
        import glob
        for edition in ["Enterprise", "Professional", "BuildTools", "Community"]:
            paths = sorted(glob.glob(r"C:\\Program Files x86\\Microsoft Visual Studio\\*\\%s\\VC\\Tools\\MSVC\\*\\bin\\Hostx64\\x64" % edition), reverse=True)
            if paths:
                return paths[0]

    # If cl.exe is not on path, try to find it.
    if os.system("where cl.exe >nul 2>nul") != 0:
        cl_path = find_cl_path()
        if cl_path is None:
            raise RuntimeError("Could not locate a supported Microsoft Visual C++ installation")
        os.environ["PATH"] += ";" + cl_path

Now my visual studio files were not on the path "C:\Program Files\Microsoft Visual Studio..." instead of "C:\Program Files x86", I might be an idiot here but I removed the x86 from the path this setup script was searching, so that error went away.
But now I get this error log:

Processing d:\python\stable-dreamfusion\raymarching
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Building wheels for collected packages: raymarching
  Building wheel for raymarching (setup.py): started
  Building wheel for raymarching (setup.py): still running...
  Building wheel for raymarching (setup.py): finished with status 'error'
  Running setup.py clean for raymarching
Failed to build raymarching
  error: subprocess-exited-with-error
  
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [100 lines of output]
      running bdist_wheel
      running build
      running build_ext
      building '_raymarching' extension
      creating D:\Python\stable-dreamfusion\raymarching\build
      creating D:\Python\stable-dreamfusion\raymarching\build\temp.win-amd64-cpython-312
      creating D:\Python\stable-dreamfusion\raymarching\build\temp.win-amd64-cpython-312\Release
      creating D:\Python\stable-dreamfusion\raymarching\build\temp.win-amd64-cpython-312\Release\Python
      creating D:\Python\stable-dreamfusion\raymarching\build\temp.win-amd64-cpython-312\Release\Python\stable-dreamfusion
      creating D:\Python\stable-dreamfusion\raymarching\build\temp.win-amd64-cpython-312\Release\Python\stable-dreamfusion\raymarching
      creating D:\Python\stable-dreamfusion\raymarching\build\temp.win-amd64-cpython-312\Release\Python\stable-dreamfusion\raymarching\src
      C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\torch\utils\cpp_extension.py:1967: UserWarning: TORCH_CUDA_ARCH_LIST is not set, all archs for visible cards are included for compilation.
      If this is not desired, please set os.environ['TORCH_CUDA_ARCH_LIST'].
        warnings.warn(
      Emitting ninja build file D:\Python\stable-dreamfusion\raymarching\build\temp.win-amd64-cpython-312\Release\build.ninja...
      Compiling objects...
      Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
      [1/2] C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\bin\nvcc --generate-dependencies-with-compile --dependency-output D:\Python\stable-dreamfusion\raymarching\build\temp.win-amd64-cpython-312\Release\Python\stable-dreamfusion\raymarching\src\raymarching.obj.d -std=c++17 --use-local-env -Xcompiler /MD -Xcompiler /wd4819 -Xcompiler /wd4251 -Xcompiler /wd4244 -Xcompiler /wd4267 -Xcompiler /wd4275 -Xcompiler /wd4018 -Xcompiler /wd4190 -Xcompiler /wd4624 -Xcompiler /wd4067 -Xcompiler /wd4068 -Xcompiler /EHsc -Xcudafe --diag_suppress=base_class_has_different_dll_interface -Xcudafe --diag_suppress=field_without_dll_interface -Xcudafe --diag_suppress=dll_interface_conflict_none_assumed -Xcudafe --diag_suppress=dll_interface_conflict_dllexport_assumed -IC:\Users\BG\.conda\envs\sdf1\Lib\site-packages\torch\include -IC:\Users\BG\.conda\envs\sdf1\Lib\site-packages\torch\include\torch\csrc\api\include -IC:\Users\BG\.conda\envs\sdf1\Lib\site-packages\torch\include\TH -IC:\Users\BG\.conda\envs\sdf1\Lib\site-packages\torch\include\THC "-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\include" -IC:\Users\BG\.conda\envs\sdf1\include -IC:\Users\BG\.conda\envs\sdf1\Include "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.39.33519\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.39.33519\ATLMFC\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\um" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\shared" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\winrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\cppwinrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um" -c D:\Python\stable-dreamfusion\raymarching\src\raymarching.cu -o D:\Python\stable-dreamfusion\raymarching\build\temp.win-amd64-cpython-312\Release\Python\stable-dreamfusion\raymarching\src\raymarching.obj -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -O3 -std=c++14 -U__CUDA_NO_HALF_OPERATORS__ -U__CUDA_NO_HALF_CONVERSIONS__ -U__CUDA_NO_HALF2_OPERATORS__ -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=_raymarching -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_75,code=compute_75 -gencode=arch=compute_75,code=sm_75
      FAILED: D:/Python/stable-dreamfusion/raymarching/build/temp.win-amd64-cpython-312/Release/Python/stable-dreamfusion/raymarching/src/raymarching.obj
      C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\bin\nvcc --generate-dependencies-with-compile --dependency-output D:\Python\stable-dreamfusion\raymarching\build\temp.win-amd64-cpython-312\Release\Python\stable-dreamfusion\raymarching\src\raymarching.obj.d -std=c++17 --use-local-env -Xcompiler /MD -Xcompiler /wd4819 -Xcompiler /wd4251 -Xcompiler /wd4244 -Xcompiler /wd4267 -Xcompiler /wd4275 -Xcompiler /wd4018 -Xcompiler /wd4190 -Xcompiler /wd4624 -Xcompiler /wd4067 -Xcompiler /wd4068 -Xcompiler /EHsc -Xcudafe --diag_suppress=base_class_has_different_dll_interface -Xcudafe --diag_suppress=field_without_dll_interface -Xcudafe --diag_suppress=dll_interface_conflict_none_assumed -Xcudafe --diag_suppress=dll_interface_conflict_dllexport_assumed -IC:\Users\BG\.conda\envs\sdf1\Lib\site-packages\torch\include -IC:\Users\BG\.conda\envs\sdf1\Lib\site-packages\torch\include\torch\csrc\api\include -IC:\Users\BG\.conda\envs\sdf1\Lib\site-packages\torch\include\TH -IC:\Users\BG\.conda\envs\sdf1\Lib\site-packages\torch\include\THC "-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\include" -IC:\Users\BG\.conda\envs\sdf1\include -IC:\Users\BG\.conda\envs\sdf1\Include "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.39.33519\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.39.33519\ATLMFC\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\um" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\shared" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\winrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\cppwinrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um" -c D:\Python\stable-dreamfusion\raymarching\src\raymarching.cu -o D:\Python\stable-dreamfusion\raymarching\build\temp.win-amd64-cpython-312\Release\Python\stable-dreamfusion\raymarching\src\raymarching.obj -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -O3 -std=c++14 -U__CUDA_NO_HALF_OPERATORS__ -U__CUDA_NO_HALF_CONVERSIONS__ -U__CUDA_NO_HALF2_OPERATORS__ -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=_raymarching -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_75,code=compute_75 -gencode=arch=compute_75,code=sm_75
      cl : Command line warning D9025 : overriding '/D__CUDA_NO_HALF_OPERATORS__' with '/U__CUDA_NO_HALF_OPERATORS__'
      cl : Command line warning D9025 : overriding '/D__CUDA_NO_HALF_CONVERSIONS__' with '/U__CUDA_NO_HALF_CONVERSIONS__'
      cl : Command line warning D9025 : overriding '/D__CUDA_NO_HALF2_OPERATORS__' with '/U__CUDA_NO_HALF2_OPERATORS__'
      
      raymarching.cu
      
      C:/Users/BG/.conda/envs/sdf1/Lib/site-packages/torch/include\c10/util/C++17.h(24): fatal error C1189: #error:  You need C++17 to compile PyTorch
      
      nvcc warning : incompatible redefinition for option 'std', the last value of this option was used
      
      raymarching.cu
      
      [2/2] cl /showIncludes /nologo /O2 /W3 /GL /DNDEBUG /MD /MD /wd4819 /wd4251 /wd4244 /wd4267 /wd4275 /wd4018 /wd4190 /wd4624 /wd4067 /wd4068 /EHsc -IC:\Users\BG\.conda\envs\sdf1\Lib\site-packages\torch\include -IC:\Users\BG\.conda\envs\sdf1\Lib\site-packages\torch\include\torch\csrc\api\include -IC:\Users\BG\.conda\envs\sdf1\Lib\site-packages\torch\include\TH -IC:\Users\BG\.conda\envs\sdf1\Lib\site-packages\torch\include\THC "-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\include" -IC:\Users\BG\.conda\envs\sdf1\include -IC:\Users\BG\.conda\envs\sdf1\Include "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.39.33519\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.39.33519\ATLMFC\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\um" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\shared" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\winrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\cppwinrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um" -c D:\Python\stable-dreamfusion\raymarching\src\bindings.cpp /FoD:\Python\stable-dreamfusion\raymarching\build\temp.win-amd64-cpython-312\Release\Python\stable-dreamfusion\raymarching\src\bindings.obj /O2 /std:c++17 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=_raymarching -D_GLIBCXX_USE_CXX11_ABI=0
      ninja: build stopped: subcommand failed.
      Traceback (most recent call last):
        File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\torch\utils\cpp_extension.py", line 2107, in _run_ninja_build
          subprocess.run(
        File "C:\Users\BG\.conda\envs\sdf1\Lib\subprocess.py", line 571, in run
          raise CalledProcessError(retcode, process.args,
      subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.
      
      The above exception was the direct cause of the following exception:
      
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "D:\Python\stable-dreamfusion\raymarching\setup.py", line 44, in <module>
          setup(
        File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\setuptools\__init__.py", line 103, in setup
          return distutils.core.setup(**attrs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\setuptools\_distutils\core.py", line 185, in setup
          return run_commands(dist)
                 ^^^^^^^^^^^^^^^^^^
        File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\setuptools\_distutils\core.py", line 201, in run_commands
          dist.run_commands()
        File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\setuptools\_distutils\dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\setuptools\dist.py", line 989, in run_command
          super().run_command(command)
        File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command
          cmd_obj.run()
        File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\wheel\bdist_wheel.py", line 368, in run
          self.run_command("build")
        File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\setuptools\_distutils\cmd.py", line 318, in run_command
          self.distribution.run_command(command)
        File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\setuptools\dist.py", line 989, in run_command
          super().run_command(command)
        File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command
          cmd_obj.run()
        File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\setuptools\_distutils\command\build.py", line 131, in run
          self.run_command(cmd_name)
        File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\setuptools\_distutils\cmd.py", line 318, in run_command
          self.distribution.run_command(command)
        File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\setuptools\dist.py", line 989, in run_command
          super().run_command(command)
        File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command
          cmd_obj.run()
        File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\setuptools\command\build_ext.py", line 88, in run
          _build_ext.run(self)
        File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\setuptools\_distutils\command\build_ext.py", line 345, in run
          self.build_extensions()
        File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\torch\utils\cpp_extension.py", line 870, in build_extensions
          build_ext.build_extensions(self)
        File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\setuptools\_distutils\command\build_ext.py", line 467, in build_extensions
          self._build_extensions_serial()
        File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\setuptools\_distutils\command\build_ext.py", line 493, in _build_extensions_serial
          self.build_extension(ext)
        File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\setuptools\command\build_ext.py", line 249, in build_extension
          _build_ext.build_extension(self, ext)
        File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\setuptools\_distutils\command\build_ext.py", line 548, in build_extension
          objects = self.compiler.compile(
                    ^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\torch\utils\cpp_extension.py", line 842, in win_wrap_ninja_compile
          _write_ninja_file_and_compile_objects(
        File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\torch\utils\cpp_extension.py", line 1783, in _write_ninja_file_and_compile_objects
          _run_ninja_build(
        File "C:\Users\BG\.conda\envs\sdf1\Lib\site-packages\torch\utils\cpp_extension.py", line 2123, in _run_ninja_build
          raise RuntimeError(message) from e
      RuntimeError: Error compiling objects for extension
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for raymarching
ERROR: Could not build wheels for raymarching, which is required to install pyproject.toml-based projects

I have already installed MSVC++2017 from Visual studio installer. I also have gcc10 in path if the setup could alternatively use that but it still all fails. Please help. My job depends on this!

Expected Behavior

Successfully installed Raymarching

Environment

OS: Windows 11
CUDA Version supported: 12.2

It was suggested in other issues to use 11.8 etc since 12+ versions are not compatibale for this project so I installed that.

CUDA version I installed: cuda_11.8.0_522.06_windows.exe from nvidia site
PyTorch I installed as per instructions for this CUDA version:
! conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia

@SubhamBurnwal-BiswaGames SubhamBurnwal-BiswaGames added the bug Something isn't working label May 4, 2024
@Duodecimus
Copy link

In setup.py there are two places it has c++14. change these to c++17

This lets me compile it with python setup.py build_ext --inplace, but attempting to install the build with pip install . fails during wheel building with ModuleNotFoundError: No module named 'torch'

so your mileage may vary with this advice

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants