CUDA dependencies problem

[FabioLddo]

Hi, i would like to start with some congratulations for the great project.

I am trying to test it out with a docker container, this is the base image that i use:

`

Start from an official CUDA base image

FROM ubuntu:20.04

Set up environment variables for non-interactive apt-get and Conda

ENV DEBIAN_FRONTEND=noninteractive

Update and install dependencies

RUN apt-get update && apt-get install -y
wget
curl
git
build-essential
libgl1-mesa-glx
libglib2.0-0
ninja-build
cmake
python3-dev
&& apt-get clean && rm -rf /var/lib/apt/lists/*

Install Miniconda

ENV CONDA_DIR /opt/conda
RUN wget --quiet https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O /tmp/miniconda.sh &&
bash /tmp/miniconda.sh -b -p $CONDA_DIR &&
rm /tmp/miniconda.sh &&
$CONDA_DIR/bin/conda clean -afy

Add Conda to PATH

ENV PATH=$CONDA_DIR/bin:$PATH

Initialize Conda for bash

RUN echo "source $CONDA_DIR/etc/profile.d/conda.sh" >> ~/.bashrc

Default command

CMD ["/bin/bash"]`

The problem is that when i try to install gorunded sam, i have the following error on this command:

pip install --no-build-isolation -e grounding_dino

Error:

`(G-SAM-2) root@7b07c6566628:/workspaces/FusionSense/Grounded-SAM2-for-masking# pip install --no-build-isolation -e grounding_dino
Obtaining file:///workspaces/FusionSense/Grounded-SAM2-for-masking/grounding_dino
Preparing metadata (setup.py) ... done
Installing collected packages: groundingdino
DEPRECATION: Legacy editable install of groundingdino==0.1.0 from file:///workspaces/FusionSense/Grounded-SAM2-for-masking/grounding_dino (setup.py develop) is deprecated. pip 25.1 will enforce this behaviour change. A possible replacement is to add a pyproject.toml or enable --use-pep517, and use setuptools >= 64. If the resulting installation is not behaving as expected, try using --config-settings editable_mode=compat. Please consult the setuptools documentation for more information. Discussion can be found at pypa/pip#11457
Running setup.py develop for groundingdino
error: subprocess-exited-with-error

× python setup.py develop did not run successfully.
│ exit code: 1
╰─> [48 lines of output]
    Building wheel groundingdino-0.1.0
    Compiling with CUDA
    running develop
    /opt/conda/envs/G-SAM-2/lib/python3.10/site-packages/setuptools/command/develop.py:41: EasyInstallDeprecationWarning: easy_install command is deprecated.
    !!
    
            ********************************************************************************
            Please avoid running ``setup.py`` and ``easy_install``.
            Instead, use pypa/build, pypa/installer or other
            standards-based tools.
    
            See https://github.com/pypa/setuptools/issues/917 for details.
            ********************************************************************************
    
    !!
      easy_install.initialize_options(self)
    /opt/conda/envs/G-SAM-2/lib/python3.10/site-packages/setuptools/_distutils/cmd.py:79: SetuptoolsDeprecationWarning: setup.py install is deprecated.
    !!
    
            ********************************************************************************
            Please avoid running ``setup.py`` directly.
            Instead, use pypa/build, pypa/installer or other
            standards-based tools.
    
            See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
            ********************************************************************************
    
    !!
      self.initialize_options()
    running egg_info
    writing groundingdino.egg-info/PKG-INFO
    writing dependency_links to groundingdino.egg-info/dependency_links.txt
    writing top-level names to groundingdino.egg-info/top_level.txt
    reading manifest file 'groundingdino.egg-info/SOURCES.txt'
    adding license file 'LICENSE'
    writing manifest file 'groundingdino.egg-info/SOURCES.txt'
    running build_ext
    /opt/conda/envs/G-SAM-2/lib/python3.10/site-packages/torch/utils/cpp_extension.py:428: UserWarning: There are no g++ version bounds defined for CUDA version 12.1
      warnings.warn(f'There are no {compiler_name} version bounds defined for CUDA version {cuda_str_version}')
    building 'groundingdino._C' extension
    Emitting ninja build file /workspaces/FusionSense/Grounded-SAM2-for-masking/grounding_dino/build/temp.linux-x86_64-cpython-310/build.ninja...
    Compiling objects...
    Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
    ninja: no work to do.
    g++ -pthread -B /opt/conda/envs/G-SAM-2/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /opt/conda/envs/G-SAM-2/include -fPIC -O2 -isystem /opt/conda/envs/G-SAM-2/include -pthread -B /opt/conda/envs/G-SAM-2/compiler_compat -shared /workspaces/FusionSense/Grounded-SAM2-for-masking/grounding_dino/build/temp.linux-x86_64-cpython-310/workspaces/FusionSense/Grounded-SAM2-for-masking/grounding_dino/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cpu.o /workspaces/FusionSense/Grounded-SAM2-for-masking/grounding_dino/build/temp.linux-x86_64-cpython-310/workspaces/FusionSense/Grounded-SAM2-for-masking/grounding_dino/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cuda.o /workspaces/FusionSense/Grounded-SAM2-for-masking/grounding_dino/build/temp.linux-x86_64-cpython-310/workspaces/FusionSense/Grounded-SAM2-for-masking/grounding_dino/groundingdino/models/GroundingDINO/csrc/cuda_version.o /workspaces/FusionSense/Grounded-SAM2-for-masking/grounding_dino/build/temp.linux-x86_64-cpython-310/workspaces/FusionSense/Grounded-SAM2-for-masking/grounding_dino/groundingdino/models/GroundingDINO/csrc/vision.o -L/opt/conda/envs/G-SAM-2/lib/python3.10/site-packages/torch/lib -L/opt/conda/envs/G-SAM-2/lib -lc10 -ltorch -ltorch_cpu -ltorch_python -lcudart -lc10_cuda -ltorch_cuda -o build/lib.linux-x86_64-cpython-310/groundingdino/_C.cpython-310-x86_64-linux-gnu.so
    /opt/conda/envs/G-SAM-2/compiler_compat/ld: cannot find -lcudart: No such file or directory
    collect2: error: ld returned 1 exit status
    error: command '/usr/bin/g++' failed with exit code 1
    [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.

error: subprocess-exited-with-error

× python setup.py develop did not run successfully.
│ exit code: 1
╰─> [48 lines of output]
Building wheel groundingdino-0.1.0
Compiling with CUDA
running develop
/opt/conda/envs/G-SAM-2/lib/python3.10/site-packages/setuptools/command/develop.py:41: EasyInstallDeprecationWarning: easy_install command is deprecated.
!!

        ********************************************************************************
        Please avoid running ``setup.py`` and ``easy_install``.
        Instead, use pypa/build, pypa/installer or other
        standards-based tools.

        See https://github.com/pypa/setuptools/issues/917 for details.
        ********************************************************************************

!!
  easy_install.initialize_options(self)
/opt/conda/envs/G-SAM-2/lib/python3.10/site-packages/setuptools/_distutils/cmd.py:79: SetuptoolsDeprecationWarning: setup.py install is deprecated.
!!

        ********************************************************************************
        Please avoid running ``setup.py`` directly.
        Instead, use pypa/build, pypa/installer or other
        standards-based tools.

        See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
        ********************************************************************************

!!
  self.initialize_options()
running egg_info
writing groundingdino.egg-info/PKG-INFO
writing dependency_links to groundingdino.egg-info/dependency_links.txt
writing top-level names to groundingdino.egg-info/top_level.txt
reading manifest file 'groundingdino.egg-info/SOURCES.txt'
adding license file 'LICENSE'
writing manifest file 'groundingdino.egg-info/SOURCES.txt'
running build_ext
/opt/conda/envs/G-SAM-2/lib/python3.10/site-packages/torch/utils/cpp_extension.py:428: UserWarning: There are no g++ version bounds defined for CUDA version 12.1
  warnings.warn(f'There are no {compiler_name} version bounds defined for CUDA version {cuda_str_version}')
building 'groundingdino._C' extension
Emitting ninja build file /workspaces/FusionSense/Grounded-SAM2-for-masking/grounding_dino/build/temp.linux-x86_64-cpython-310/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
ninja: no work to do.
g++ -pthread -B /opt/conda/envs/G-SAM-2/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /opt/conda/envs/G-SAM-2/include -fPIC -O2 -isystem /opt/conda/envs/G-SAM-2/include -pthread -B /opt/conda/envs/G-SAM-2/compiler_compat -shared /workspaces/FusionSense/Grounded-SAM2-for-masking/grounding_dino/build/temp.linux-x86_64-cpython-310/workspaces/FusionSense/Grounded-SAM2-for-masking/grounding_dino/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cpu.o /workspaces/FusionSense/Grounded-SAM2-for-masking/grounding_dino/build/temp.linux-x86_64-cpython-310/workspaces/FusionSense/Grounded-SAM2-for-masking/grounding_dino/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cuda.o /workspaces/FusionSense/Grounded-SAM2-for-masking/grounding_dino/build/temp.linux-x86_64-cpython-310/workspaces/FusionSense/Grounded-SAM2-for-masking/grounding_dino/groundingdino/models/GroundingDINO/csrc/cuda_version.o /workspaces/FusionSense/Grounded-SAM2-for-masking/grounding_dino/build/temp.linux-x86_64-cpython-310/workspaces/FusionSense/Grounded-SAM2-for-masking/grounding_dino/groundingdino/models/GroundingDINO/csrc/vision.o -L/opt/conda/envs/G-SAM-2/lib/python3.10/site-packages/torch/lib -L/opt/conda/envs/G-SAM-2/lib -lc10 -ltorch -ltorch_cpu -ltorch_python -lcudart -lc10_cuda -ltorch_cuda -o build/lib.linux-x86_64-cpython-310/groundingdino/_C.cpython-310-x86_64-linux-gnu.so
/opt/conda/envs/G-SAM-2/compiler_compat/ld: cannot find -lcudart: No such file or directory
collect2: error: ld returned 1 exit status
error: command '/usr/bin/g++' failed with exit code 1
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.`

[IrvingF7]

Hi Fabio!

Thanks for your issue report.

I haven't tested this project with Docker before. We will get back to you!

On the first look, it seems like this may be an issue with grounded-sam installation, right? Have you checked their github issue for problems similar to this, or do you think this is caused by some code/install instruction of this project specifically? (Although to be honest I would find it strange since we didn't modify much. Could this be due to some modification in the fork made by @Kairui-SHI ?)

Kim, @KimHe-RGB could you take a look at this? Since you mentioned that you wanted to test the installation independently once.

[FabioLddo]

Thank you so much for the support.
I checked grounded-sam issues but unfortunately there I did not find correlated errors, peraps it is a dependency problem coming from the base image, I will do some more testing.

[IrvingF7]

Hi Fabio!

Sorry for my inadequate knowledge regarding docker. Could you tell me the exact command you used to pull the specific version of cuda image you are using?

Also, now that I get to look at this error message more carefully, it seems to indicate the cuda runtime (lcudart) is missing. In that case, could you run
conda install -c "nvidia/label/cuda-12.1.0" cuda-toolkit in your conda environment? I know the cuda image should've had the runtime in the first place, but maybe for some reasons it's not being recognized by the build system installed in the specific conda environment

[FabioLddo]

Hi,
To quickly test project i am currently using decontainers, so I use simply configure a container that uses my Dockerfile.

I tried again your tutorial at this link: https://github.com/ai4ce/FusionSense/blob/main/instructions/grounded_sam_2.md
and it already includes the conda installation of the toolkit, which is completed without errors.

To answer your first question I was using before a base ubuntu20.04 image to start the process, without cuda or anything, since the runtimes are installed in conda.
Now I tried with this image:
FROM nvcr.io/nvidia/cuda:12.1.0-cudnn8-devel-ubuntu20.04
which should include a complete development kit, but I have a different error, I still fear I am forgetting an ubuntu dependence:

`2 errors detected in the compilation of "/workspaces/FusionSense/Grounded-SAM2-for-masking/grounding_dino/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cuda.cu".
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
File "/opt/conda/lib/python3.12/site-packages/torch/utils/cpp_extension.py", line 2209, in _run_ninja_build
subprocess.run(
File "/opt/conda/lib/python3.12/subprocess.py", line 573, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<string>", line 2, in <module>
  File "<pip-setuptools-caller>", line 34, in <module>
  File "/workspaces/FusionSense/Grounded-SAM2-for-masking/grounding_dino/setup.py", line 208, in <module>
    setup(
  File "/opt/conda/lib/python3.12/site-packages/setuptools/__init__.py", line 117, in setup
    return distutils.core.setup(**attrs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 186, in setup
    return run_commands(dist)
           ^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 202, in run_commands
    dist.run_commands()
  File "/opt/conda/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 983, in run_commands
    self.run_command(cmd)
  File "/opt/conda/lib/python3.12/site-packages/setuptools/dist.py", line 999, in run_command
    super().run_command(command)
  File "/opt/conda/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 1002, in run_command
    cmd_obj.run()
  File "/opt/conda/lib/python3.12/site-packages/setuptools/command/develop.py", line 35, in run
    self.install_for_development()
  File "/opt/conda/lib/python3.12/site-packages/setuptools/command/develop.py", line 112, in install_for_development
    self.run_command('build_ext')
  File "/opt/conda/lib/python3.12/site-packages/setuptools/_distutils/cmd.py", line 339, in run_command
    self.distribution.run_command(command)
  File "/opt/conda/lib/python3.12/site-packages/setuptools/dist.py", line 999, in run_command
    super().run_command(command)
  File "/opt/conda/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 1002, in run_command
    cmd_obj.run()
  File "/opt/conda/lib/python3.12/site-packages/setuptools/command/build_ext.py", line 99, in run
    _build_ext.run(self)
  File "/opt/conda/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 365, in run
    self.build_extensions()
  File "/opt/conda/lib/python3.12/site-packages/torch/utils/cpp_extension.py", line 900, in build_extensions
    build_ext.build_extensions(self)
  File "/opt/conda/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 481, in build_extensions
    self._build_extensions_serial()
  File "/opt/conda/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 507, in _build_extensions_serial
    self.build_extension(ext)
  File "/opt/conda/lib/python3.12/site-packages/setuptools/command/build_ext.py", line 264, in build_extension
    _build_ext.build_extension(self, ext)
  File "/opt/conda/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 562, in build_extension
    objects = self.compiler.compile(
              ^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/torch/utils/cpp_extension.py", line 713, in unix_wrap_ninja_compile
    _write_ninja_file_and_compile_objects(
  File "/opt/conda/lib/python3.12/site-packages/torch/utils/cpp_extension.py", line 1869, in _write_ninja_file_and_compile_objects
    _run_ninja_build(
  File "/opt/conda/lib/python3.12/site-packages/torch/utils/cpp_extension.py", line 2225, in _run_ninja_build
    raise RuntimeError(message) from e
RuntimeError: Error compiling objects for extension
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.`

[IrvingF7]

Hi!

Could you run
python -m torch.utils.collect_env so we can better understand your environment?

Also, not blaming you, but I find this error message really lacking in terms of information, lol. It seems to indicate that the compilation of deformed attention is having troubles, but nothing else.

Grounded-Sam's installation guide does say something about this in this section. Could you try to install Grounded-Sam according to their instructions?

[FabioLddo]

Hi!
Sorry for late response. I actually managed to make the two environments work together, it seems the problem was in the point where i needed to change the cuda path to the cuda toolkit inside the conda environment, for some reason it could not find the runtime. I just used an image with cuda and used that to install everything.
Thank you so much for the availability.

[IrvingF7]

Great! Glad to hear that!