diff --git a/README.md b/README.md index 74fefa5..a5af2be 100644 --- a/README.md +++ b/README.md @@ -1,23 +1,65 @@ + # articulated_objects_scene_builder -## About -ROS node that builds point cloud-based model of the scene enhanced with information about articulated objects. It subscribes to following topics: -- topic with point cloud to process that can be set using `rosparam set rosparam set input_point_cloud_topic "input_point_cloud_topic"` -- `front_prediction` which contains information about detected fronts of articulated objects. [See node](https://github.com/arekmula/ros_front_detection_segmentation) -- `handler_prediction_topic` which contains information about detected handlers of articulated objects. [See node](https://github.com/arekmula/ros_handler_detector) -- `joint_prediction_topic` which contains information about detected joints of articulated objects that are rotational. [See node](https://github.com/arekmula/ros_joint_segmentation) +
+ + + + + +
+ +The goal of the project is to build a ROS system that would be able to build a point-cloud-based model of the scene enhanced with information about articulated objects based on a **single** RGB-D image. The articulated objects defined by this work are drawers and cabinets. + +This module is part of my master thesis "Point cloud-based model of the scene enhanced with information about articulated objects" + +This node concatenates results and runs on top of the previous three nodes that are available here: + +- [Handler detector](https://github.com/arekmula/ros_handler_detector) +- [Front detector](https://github.com/arekmula/ros_front_detection_segmentation) +- [Rotational joint detector](https://github.com/arekmula/ros_joint_detection) + +to build a final model of the scene in a 3D environment. + +The node subsribes to the following topics: +- topic with Microsoft Kinect point cloud, that can be set using `rosparam set input_point_cloud_topic "input_point_cloud_topic"` +- `front_prediction` which contains information about detected fronts of articulated objects. +- `handler_prediction_topic` which contains information about detected handlers of articulated objects. +- `joint_prediction_topic` which contains information about detected joints of articulated objects that are rotational. The node publish following topics: -- `image_to_process` - RGB image obtained from input point cloud, which can be processed by external nodes -- `currently_processed_point_cloud` - currently processed point cloud -- `processed_point_cloud` - processed point cloud with marked data +- `image_to_process` - RGB image obtained from input point cloud, which has to be processed by the rest of the nodes +- `cloud_to_process` - currently processed point cloud +- `processed_fronts_point_cloud` - Processed point cloud with fronts data +- `processed_handlers_point_cloud` - Processed point cloud with handlers data +- `trans_fronts_normals` - Normals of transitional objects +- `rot_front_joints` - Rotational joints +- `last_processed_point_cloud` - Point cloud that was processed a moment ago + +## Dataset +The dataset that was used using developing the project is available here: +- [part1](https://drive.google.com/file/d/1fhE5tN_5AM1CKty76QT63WNsISecXsl7/view?usp=sharing) +- [part2](https://drive.google.com/file/d/1k008_vaWegVhvY-ULVqCNqPQ12pFojvO/view?usp=sharing) +The dataset consists of 38 rosbag sequences, which contain 32 unique transitional objects and 53 unique rotational objects. +## Results + +The results gathered on the dataset above looks as follow: +- Mean time of scene building: 1.9393s +```markdown +| | Transitional objects | Rotational Objects | +|:---------------------------------------:|:--------------------:|--------------------| +| Found fronts/Total number of fronts | 31/32 | 50/53 | +| Found joints/Total number of joints | 31/32 | 46/53 | +| Found handlers/Total number of handlers | 39/40 | 50/53 | +``` ## Dependencies - ROS Noetic - PCL library `sudo apt install libpcl-dev` ## Run with ``` -rosparam set rosparam set input_point_cloud_topic "input_point_cloud_topic" +rosparam set input_point_cloud_topic "input_point_cloud_topic" roslaunch model_builder model_builder.launch ``` +Run nodes responsible for handler, front and rotational joint detection. diff --git a/imgs/object1.gif b/imgs/object1.gif new file mode 100644 index 0000000..27f0f66 Binary files /dev/null and b/imgs/object1.gif differ diff --git a/imgs/object2.gif b/imgs/object2.gif new file mode 100644 index 0000000..2a0ab89 Binary files /dev/null and b/imgs/object2.gif differ diff --git a/imgs/object3.gif b/imgs/object3.gif new file mode 100644 index 0000000..d2eaee9 Binary files /dev/null and b/imgs/object3.gif differ