Fer va estimation

[negarhdr]

This PR adds image-based facial expression recognition and valence-arousal estimation.
It includes learner, unit-tests, demo, document, and ROS node.
This is a replacement for the previous PR which had conflicts with other tools.

[ad-daniel]

Hi, maybe there was a misunderstanding. It was correct for you to create your branch from the develop (since new additions like this should target develop, not master). The issue with your previous attempt likely was that when you created the branch, develop was not up to date or you committed unrelated stuff. In short you should:

git checkout develop
git pull origin develop
git submodule update

do a git status to ensure everything looks clean, there should not be any change showing up

git checkout -b my-fancy-new-branch
do the changes and push

[negarhdr]

Thank you for the comment.
I found some other issues in the previous branch so I deleted it and made this new branch and the new PR.
My branch is based on "develop" branch and now its merge target is also the "develop" branch.

[passalis]

Thank you! I have added a few comments directly on files.

Furthermore, there are few additional points:

• Please make sure that you use the conventions for ROS argument passing (check also the names of the topics and arguments in ROS1 updates and fixes for several nodes #281)
• Should this file be empty src/opendr/perception/facial_expression_recognition/image_based_facial_emotion_estimation/README.md ?
• The demo is nice. However, for most other tools for inference demo, we provide small examples of how each tool can be used. This demo incorporates too much functionality (which is good, but it can be also hard for new users to follow). So I would propose to also include a small inference demo that runs on a single image and clearly imports and uses the learner (similar to other tools, e.g., https://github.com/opendr-eu/opendr/blob/master/projects/perception/lightweight_open_pose/demos/inference_demo.py). In this way it would be clear for new users how they should use the tool.

[negarhdr]

Thank you! I have added a few comments directly on files.

Furthermore, there are few additional points:

• Please make sure that you use the conventions for ROS argument passing (check also the names of the topics and arguments in ROS1 updates and fixes for several nodes #281)
• Should this file be empty src/opendr/perception/facial_expression_recognition/image_based_facial_emotion_estimation/README.md ?
• The demo is nice. However, for most other tools for inference demo, we provide small examples of how each tool can be used. This demo incorporates too much functionality (which is good, but it can be also hard for new users to follow). So I would propose to also include a small inference demo that runs on a single image and clearly imports and uses the learner (similar to other tools, e.g., https://github.com/opendr-eu/opendr/blob/master/projects/perception/lightweight_open_pose/demos/inference_demo.py). In this way it would be clear for new users how they should use the tool.

Thanks for the comments.
The ros1_node is updated and the demo is simplified as much as possible.

[tsampazk]

@ad-daniel @negarhdr The ROS1 node now uses retina face, as the cv face detector proved problematic with the conversions required for ROS. I also included the publishing of an annotated image which draws bounding boxes with all facial emotion info printed on the corner of the box. I tested it with multiple faces and it works well. It is similar to the face recognition pipeline in the corresponding node. Note that it is quite slow, but the detector is far superior in the performed qualitative testing.

[ad-daniel]

When running the ROS node I'm getting this:

rosrun opendr_perception facial_emotion_estimation_node.py
[rospack] Error: package 'opendr_perception' not found
find: ‘’: No such file or directory
Tokenization took: 0.01 ms

however it doesn't prevent it from working, so not a big deal.

As for the demo, I've tested the webcam and it works nicely. Image and video are failing, from the error it seems to be related to the fact it cannot find the resources in the FTP server. @Pavlos-Tosidis has it been moved from owncloud? I think we forgot to ping him

Other than that, it looks good to me, thank you both!

[tsampazk]

When running the ROS node I'm getting this:

rosrun opendr_perception facial_emotion_estimation_node.py
[rospack] Error: package 'opendr_perception' not found
find: ‘’: No such file or directory
Tokenization took: 0.01 ms

Hmm that's weird, never gotten this error before. Maybe this helps?

Thanks!

[ad-daniel]

Nevermind my bad, I did a source devel/local_setup.bash rather than source devel/setup.bash.

[Pavlos-Tosidis]

@ad-daniel
Truth is I never saw this comment for moving the demo on the FTP. Files are up now!

[tsampazk]

Went ahead and resolved the issue with the path in download. Also fixed the demo for image and video modes, as the cv2 operations to show the images were used incorrectly. Should be good to go now!

[ad-daniel]

Working for me as well, thank you!

[tsampazk]

Sorry Daniel just merged hr pose estimation and there was a conflict.

done