The visual-tracking-control project is a suite of cross-platform applications for visual tracking and visual servoing for the humanoid robot platform iCub.
The suite includes:
- hand-tracking: a visual end-effector tracker using a 3D model-aided particle filter [1];
- visual-servoing: a visual servoing
YARP
plugin to control the pose (position and orientation) of the end-effector using image feedback [2].- The
visual-servoing
consist of two plugins, a client and server version. On one hand, the server must be always running viayarpdev
and is responsible to command the robot. On the other hand, the client is dynamically loaded by any application using visual servoing interfaces and has the sole purpose of communicating commands to the server. This architecture enable distributed computation over different machine and decouples client and server implementations.
- The
The output of the hand-tracking application can be visualized by means of the iCubProprioception module. iCubProprioception provides an augmented-reality application which superimposes the 3D model of the end-effector onto the camera images, using the estimated pose provided by hand-tracking.
visual-tracking-control suite depends on
- BayesFilters -
version >= 0.6
- iCub
- iCubContrib
- OpenCV -
version >= 3.3
, built withCUDA >= 8.0
- SuperimposeMesh -
version >= 0.9
- YARP
Use the following commands to build, install and link the library.
With make
facilities:
$ git clone https://github.com/robotology/visual-tracking-control
$ cd visual-tracking-control
$ mkdir build && cd build
$ cmake -DBUILD_HAND_TRACKING=ON -DBUILD_VISUAL_SERVOING_CLIENT=ON -DBUILD_VISUAL_SERVOING_SERVER=ON ..
$ make
$ [sudo] make install
With IDE build tool facilities:
$ git clone https://github.com/robotology/visual-tracking-control
$ cd visual-tracking-control
$ mkdir build && cd build
$ cmake -DBUILD_HAND_TRACKING=ON -DBUILD_VISUAL_SERVOING_CLIENT=ON -DBUILD_VISUAL_SERVOING_SERVER=ON ..
$ cmake --build . --target ALL_BUILD --config Release
$ cmake --build . --target INSTALL --config Release
Doxygen-generated documentation is available here.
[1] C. Fantacci, U. Pattacini, V. Tikhanoff and L. Natale, "Visual end-effector tracking using a 3D model-aided particle filter for humanoid robot platforms", IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada, 2017. arXiv preprint arXiv:1703.04771.
[2] C. Fantacci, G. Vezzani, U. Pattacini, V. Tikhanoff and L. Natale, "Markerless visual servoing on unknown objects for humanoid robot platforms", IEEE International Conference on Robotics and Automation (ICRA), Brisbane, AU, 2018. arXiv preprint arXiv:1710.04465.