Skip to content

Latest commit

 

History

History
106 lines (87 loc) · 6.43 KB

README.md

File metadata and controls

106 lines (87 loc) · 6.43 KB

About

This package allows you to run object detection models on the machine running ROS. This is intended as an alternative to the OAK cameras while doing simulations, but can also be potentially used on the Jetson itself.

Usage

The package supports TensorFlow 1 GraphDef models such as MobileNetV2 and PyTorch Hub models such as YOLOv5. Each approach has a more detailed guide below (TF1 GraphDef, PyTorch Hub).

The faux_detector.launch file is the main launch file of the package. These are parameters common to both approaches:

  • image_topic: topic of the image topic that the dectector takes as input.
  • detection_topic: topic of the detection result, has type vision_msgs/Detection2DArray.
  • min_score_thresh: minimum confidence to accept detection, in range [0, 1].
  • label_file: label file to match class number and class name. This is, at the moment, in the style of the .pbtxt files generated by TF1 GraphDef models.
  • model_type: can either be "graphdef" for TF1 GraphDef or "torchhub" for PyTorch Hub.

For TF1 GraphDef models:

  • model_file: path to the .pb file of the model.

For PyTorch Hub models:

  • model_path: path to the model folder, such as the YOLOv5 repository ultralytics/yolov5. Does not need to be changed if you're using YOLOv5, since it is included as a submodule.
  • weights_file: path to the .pt file of the trained model that contains the weights.
  • subprocess_file: path to the subprocess file, and does not need to be changed by default (more details in the PyTorch Hub section.)
  • subprocess_python_file: path to the Python 3 executable (more details in the PyTorch Hub section.)

Tensorflow 1 (MobileNetV2)

This approach keeps a Tensorflow 1 session open while running inference on each new image, which achieves real-time speeds like a deployed model. You can read graphdef.py to learn more.

Install object_detection

The package uses the object_detection package provided in TensorFlow's repository, which is included in nr22-software/include. To install, do the following:

  • cd into nr22-software/include/models/research
  • Install using the following (courtesy of rakidedigama):
python setup.py build
python setup.py install

If the above gives you the [Errno 13] Permission denied error, it means you're using the system python, and it is not recommended to use setup.py to install for all users. Try this instead to install for local user.

python setup.py install --user
  • Still in models/research, run the following to add the library to PYTHON_PATH:
echo "export PYTHONPATH=\$PYTHONPATH:$PWD:$PWD/slim" >> ~/.profile
export PYTHONPATH=$PYTHONPATH:$PWD:$PWD/slim
  • Still in models/research, run the following (if you don't have protoc, install with sudo apt-get install protobuf-compiler.)
protoc object_detection/protos/*.proto --python_out=.
  • If you cannot install protoc with apt as above, you can install it manually. Download and install the 3.0 release of protoc, then unzip the file.
# From tensorflow/models/research (be in this directory)
wget -O protobuf.zip https://github.com/google/protobuf/releases/download/v3.0.0/protoc-3.0.0-linux-x86_64.zip
unzip protobuf.zip
rm -rf protobuf.zip
./bin/protoc object_detection/protos/*.proto --python_out=.

Install other dependencies

The following installs the rest of the dependencies. Note that this installs a non-GPU version of tensorflow. If you want to use GPU, install tensorflow-gpu instead.

pip2 install --user tensorflow==1.15.0 Cython contextlib2 pillow lxml

PyTorch Hub (Yolov5)

This approach uses the PyTorch Hub to open a model and run inferences, but there's a tricky issue that PyTorch no longer supports Python 2.7. Instead, this uses the subprocess package to run a Python 3 subprocess with PyTorch installed. Back-and-forth communication is facilitated with subprocess.PIPEs. You can read torchhub.py to learn more.

At the moment, it also relies on the object_detection package described in the above section (Install object_detection). Please go there and install accordingly.

Launch parameters

The subprocess_file parameter in the launch file does not need to be changed, as the default path points to the correctly-working faux_subprocess.py.

The subprocess_python_file parameter, however, needs to be modified to point to a Python 3 executable that has PyTorch installed. The next section discusses how you can do this.

Install PyTorch

If you're in a Python 2 environment, it is best to use virtual environment managers such as Miniconda to create a new environment with Python 3 and proper packages.

In your Python 3 environment, refer to PyTorch's Start Locally guide to find out what to install for your machine. If you have an unlisted version of CUDA, the following might help (using pip):

pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu<CUDA VERSION NUMBER>

where <CUDA VERSION NUMBER> is your CUDA version without a dot, e.g. 116, 117 or 118. If this does not work, you can try PyTorch's Previous Versions.

Install other dependencies

The following installs the rest of the dependencies.

pip install --upgrade opencv-python pandas psutil pyyaml tqdm ultralytics gitpython>=3.1.30

Get Python executable path

Using your new Python 3, run the following:

import sys
print(sys.executable)

Or, from the terminal:

python -c "import sys; print(sys.executable)"

This will give you the full path to the Python executable that you can put in the subprocess_python_file parameter.

Troubleshooting

If there is an issue with CUDA, possibly your graphics card is being used for display and cannot run TensorFlow (see this and this). Remove availability of CUDA devices could help (only show available devices not used for display):

export CUDA_VISIBLE_DEVICES=1