-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? # to your account
[Bug] RuntimeError: /io/build/temp.linux-x86_64-cpython-37/spconv/build/core_cc/src/csrc/sparse/all/SpconvOps/SpconvOps_get_indice_pairs.cc(65) not implemented for CPU ONLY build. #2061
Comments
Maybe you install the CPU-only spconv. Please install the spconv with CUDA. https://github.com/traveller59/spconv |
Ok, thank you, because my cuda version is 10.1.105, the latest spconv2.x supports duda10.2 at least, so I changed the original spconv version to 1.2.1. solved! |
After I replace it with spconv1.2.1, what effect will spconv display as False, and the result of training with pre-training weights is 0!!! 2022-11-28 23:59:39,479 - mmdet - INFO - Set random seed to 0, deterministic: False
|
Hi, we have support spconv1.x in mmcv, so you don't need to install spconv1.x. And can you use default mmcv's spconv to try again? |
And I suggest you check your env by PointPillars model. |
1. May I use when building a virtual environment: conda activate -n mmdtection3d python=3.7 2. When we set up the environment for testing, this error still popped up: **(mmdetection3d) ubuntu@ubuntu-ThinkStation-P920:~/zhangweili/mmdetection3d-master$ python demo/pcd_demo.py demo/data/kitti/kitti_000008.bin configs/second/hv_second_secfpn_6x8_80e_kitti-3d-car.py checkpoints/hv_second_secfpn_6x8_80e_kitti-3d-car_20200620_230238-393f000c.pth --show May I ask how to solve this problem? Is the reason for this problem that all dependencies cannot be installed directly?
Name Version Build Channel_libgcc_mutex 0.1 main |
@VVsssssk @JingweiZhang12 @
TorchVision: 0.11.0+cu111 |
Is it done? I came across this issue recently. |
Prerequisite
Task
I'm using the official example scripts/configs for the officially supported tasks/models/datasets.
Branch
master branch https://github.com/open-mmlab/mmdetection3d
Environment
CUDA_HOME: /usr/local/cuda-10.1/
NVCC: Cuda compilation tools, release 10.1, V10.1.10
GCC: gcc (Ubuntu 5.4.0-6ubuntu1~16.04.12) 5.4.0 20160609
PyTorch: 1.8.0+cu101
PyTorch compiling details: PyTorch built with:
TorchVision: 0.9.0+cu101
OpenCV: 4.6.0
MMCV: 1.6.0
MMCV Compiler: GCC 7.3
MMCV CUDA Compiler: 10.1
MMDetection: 2.25.3
MMSegmentation: 0.29.1
MMDetection3D: 1.0.0rc5+
spconv2.0: True
<<< conda initialize <<<
export CUDA_HOME=/usr/local/cuda-10.1/
export PATH=$PATH:/usr/local/cuda-10.1/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-10.1/lib64
export LIBRARY_PATH=$LIBRARY_PATH:/usr/local/cuda-10.1/lib64
#cuda
export PATH=$PATH:/usr/local/cuda/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/lib64
export LIBRARY_PATH=$LIBRARY_PATH:/usr/local/cuda/lib64
Reproduces the problem - code sample
2022-11-27 23:17:22,114 - mmdet - INFO - workflow: [('train', 1)], max: 40 epochs
2022-11-27 23:17:22,114 - mmdet - INFO - Checkpoints will be saved to /home/dell/zhangweili/mmdetection3d-master/tools/work_dirs/dv_mvx-fpn_second_secfpn_adamw_2x8_80e_kitti-3d-3class by HardDiskBackend.
2022-11-27 23:17:22.798964: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer.so.6'; dlerror: libnvinfer.so.6: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/dell/anaconda3/envs/mmdet3d/lib/python3.7/site-packages/cv2/../../lib64:/usr/local/cuda-9.0/lib64
2022-11-27 23:17:22.799090: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer_plugin.so.6'; dlerror: libnvinfer_plugin.so.6: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/dell/anaconda3/envs/mmdet3d/lib/python3.7/site-packages/cv2/../../lib64:/usr/local/cuda-9.0/lib64
File "/home/dell/zhangweili/mmdetection3d-master/mmdet3d/apis/train.py", line 319, in train_detector
runner.run(data_loaders, cfg.workflow) # 启动
File "/home/dell/anaconda3/envs/mmdet3d/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 136, in run
epoch_runner(data_loaders[i], **kwargs)
File "/home/dell/anaconda3/envs/mmdet3d/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 53, in train
self.run_iter(data_batch, train_mode=True, **kwargs)
File "/home/dell/anaconda3/envs/mmdet3d/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 32, in run_iter
**kwargs)
File "/home/dell/anaconda3/envs/mmdet3d/lib/python3.7/site-packages/mmcv/parallel/data_parallel.py", line 77, in train_step
return self.module.train_step(*inputs[0], **kwargs[0])
File "/home/dell/anaconda3/envs/mmdet3d/lib/python3.7/site-packages/mmdet/models/detectors/base.py", line 248, in train_step
losses = self(**data)
File "/home/dell/anaconda3/envs/mmdet3d/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/dell/anaconda3/envs/mmdet3d/lib/python3.7/site-packages/mmcv/runner/fp16_utils.py", line 116, in new_func
File "/home/dell/anaconda3/envs/mmdet3d/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/dell/anaconda3/envs/mmdet3d/lib/python3.7/site-packages/spconv/pytorch/conv.py", line 384, in forward
raise e
File "/home/dell/anaconda3/envs/mmdet3d/lib/python3.7/site-packages/spconv/pytorch/conv.py", line 375, in forward
self.transposed)
File "/home/dell/anaconda3/envs/mmdet3d/lib/python3.7/site-packages/spconv/pytorch/ops.py", line 162, in get_indice_pairs
stream)
RuntimeError: /io/build/temp.linux-x86_64-cpython-37/spconv/build/core_cc/src/csrc/sparse/all/SpconvOps/SpconvOps_get_indice_pairs.cc(65)
not implemented for CPU ONLY build.
Reproduces the problem - command or script
run directly train.py
configs/mvxnet/dv_mvx-fp_second_secfpn_adamw_2x8_80e_kitti-3d-3class.py
Reproduces the problem - error message
2022-11-27 23:17:22,114 - mmdet - INFO - workflow: [('train', 1)], max: 40 epochs
2022-11-27 23:17:22,114 - mmdet - INFO - Checkpoints will be saved to /home/dell/zhangweili/mmdetection3d-master/tools/work_dirs/dv_mvx-fpn_second_secfpn_adamw_2x8_80e_kitti-3d-3class by HardDiskBackend.
2022-11-27 23:17:22.798964: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer.so.6'; dlerror: libnvinfer.so.6: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/dell/anaconda3/envs/mmdet3d/lib/python3.7/site-packages/cv2/../../lib64:/usr/local/cuda-9.0/lib64
2022-11-27 23:17:22.799090: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer_plugin.so.6'; dlerror: libnvinfer_plugin.so.6: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/dell/anaconda3/envs/mmdet3d/lib/python3.7/site-packages/cv2/../../lib64:/usr/local/cuda-9.0/lib64
2022-11-27 23:17:22.799101: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:30] Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
/home/dell/zhangweili/mmdetection3d-master/mmdet3d/models/fusion_layers/coord_transform.py:35: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
if 'pcd_rotation' in img_meta else torch.eye(
SPCONV_DEBUG_SAVE_PATH not found, you can specify SPCONV_DEBUG_SAVE_PATH as debug data save path to save debug data which can be attached in a issue.
[Exception|native_pair]indices=torch.Size([28776, 4]),bs=2,ss=[41, 1600, 1408],algo=ConvAlgo.Native,ksize=[3, 3, 3],stride=[1, 1, 1],padding=[1, 1, 1],dilation=[1, 1, 1],subm=True,transpose=False
Traceback (most recent call last):
File "/home/dell/zhangweili/mmdetection3d-master/tools/train.py", line 262, in
main()
File "/home/dell/zhangweili/mmdetection3d-master/tools/train.py", line 258, in main
meta=meta)
File "/home/dell/zhangweili/mmdetection3d-master/mmdet3d/apis/train.py", line 351, in train_model
meta=meta)
File "/home/dell/zhangweili/mmdetection3d-master/mmdet3d/apis/train.py", line 319, in train_detector
runner.run(data_loaders, cfg.workflow) # 启动
File "/home/dell/anaconda3/envs/mmdet3d/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 136, in run
epoch_runner(data_loaders[i], **kwargs)
File "/home/dell/anaconda3/envs/mmdet3d/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 53, in train
self.run_iter(data_batch, train_mode=True, **kwargs)
File "/home/dell/anaconda3/envs/mmdet3d/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 32, in run_iter
**kwargs)
File "/home/dell/anaconda3/envs/mmdet3d/lib/python3.7/site-packages/mmcv/parallel/data_parallel.py", line 77, in train_step
return self.module.train_step(*inputs[0], **kwargs[0])
File "/home/dell/anaconda3/envs/mmdet3d/lib/python3.7/site-packages/mmdet/models/detectors/base.py", line 248, in train_step
losses = self(**data)
File "/home/dell/anaconda3/envs/mmdet3d/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/dell/anaconda3/envs/mmdet3d/lib/python3.7/site-packages/mmcv/runner/fp16_utils.py", line 116, in new_func
return old_func(*args, **kwargs)
File "/home/dell/zhangweili/mmdetection3d-master/mmdet3d/models/detectors/base.py", line 60, in forward
return self.forward_train(**kwargs)
File "/home/dell/zhangweili/mmdetection3d-master/mmdet3d/models/detectors/mvx_two_stage.py", line 274, in forward_train
points, img=img, img_metas=img_metas)
File "/home/dell/zhangweili/mmdetection3d-master/mmdet3d/models/detectors/mvx_two_stage.py", line 208, in extract_feat
pts_feats = self.extract_pts_feat(points, img_feats, img_metas)
File "/home/dell/zhangweili/mmdetection3d-master/mmdet3d/models/detectors/mvx_faster_rcnn.py", line 57, in extract_pts_feat
x = self.pts_middle_encoder(voxel_features, feature_coors, batch_size)
File "/home/dell/anaconda3/envs/mmdet3d/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, inFile "/home/dell/anaconda3/envs/mmdet3d/lib/python3.7/site-packages/mmcv/runner/fp16_utils.py", line 116, in new_func
return old_func(*args, **kwargs)
File "/home/dell/zhangweili/mmdetection3d-master/mmdet3d/models/middle_encoders/sparse_encoder.py", line 123, in forward
x = self.conv_input(input_sp_tensor)
File "/home/dell/anaconda3/envs/mmdet3d/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/dell/anaconda3/envs/mmdet3d/lib/python3.7/site-packages/spconv/pytorch/modules.py", line 137, in forward
input = module(input)
File "/home/dell/anaconda3/envs/mmdet3d/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/dell/anaconda3/envs/mmdet3d/lib/python3.7/site-packages/spconv/pytorch/conv.py", line 384, in forward
raise e
File "/home/dell/anaconda3/envs/mmdet3d/lib/python3.7/site-packages/spconv/pytorch/conv.py", line 375, in forward
self.transposed)
File "/home/dell/anaconda3/envs/mmdet3d/lib/python3.7/site-packages/spconv/pytorch/ops.py", line 162, in get_indice_pairs
stream)
RuntimeError: /io/build/temp.linux-x86_64-cpython-37/spconv/build/core_cc/src/csrc/sparse/all/SpconvOps/SpconvOps_get_indice_pairs.cc(65)
not implemented for CPU ONLY build.
Additional information
Hope it can help us solve or suggest, thank you!
The text was updated successfully, but these errors were encountered: