Convert smplx_params from PyMAF-X to X-avatar

[Darius-H]

I'm trying to run data_process/preprocess_PymafX_pose.py to convert the smplx_output from PyMAF-X to the format that X-avatar demands. I meet a problem, as the PymafX's expression dim is 50, and X-avatar needs expression dim is 10.
The config file of PymafX:

The output params' shape of Pymafx:

Can you tell me how X-avatar do to convert it to 10? Thank you very much.

[MoyGcc]

Is it because of the SMPL-X version mismatch? like #11

[Darius-H]

Is it because of the SMPL-X version mismatch? like #11

I think it's not the SMPL-X version mismatch. I check the smplx expression params from X-Humans dataset and they are 10. But the output of official Pymafx is 50. If there is a version mismatch, it should be the Pymafx version mismatch. Do you get 10 smplx expression params when using Pymaf-x ? And I found #9 has the same problem about expression dim.

[MoyGcc]

Yes, I suspect that you used another version of SMPL-X for PyMAF-X. We directly got 10 dim expression of SMPL-X from PyMAF-X and process it in the way here. We didn't do any specific operation like only keeping the top 10 dimensions for expression space. I attach one example of PyMAF-X output file from our side here so you can have a comparison: https://drive.google.com/drive/folders/1RQSQ3xjAYTq5lGTYF0V7lcahdhCspgib?usp=sharing.

[Darius-H]

I have figured out where the problem is. Since this time: HongwenZhang/PyMAF-X@d5fe7a4, Pymafx has changed their smplx model which is different from SMPLXv1.1, and outputs a 50 expression dim. Current smplx can not load this expression param. Thanks very much for your patient instruction.

For people who want Pymaf-x �with expression params fitting with SMPLXv1.1, i recommend https://github.com/open-mmlab/mmhuman3d/tree/main/configs/pymafx.

[felixshing]

I have figured out where the problem is. Since this time: HongwenZhang/PyMAF-X@d5fe7a4, Pymafx has changed their smplx model which is different from SMPLXv1.1, and outputs a 50 expression dim. Current smplx can not load this expression param. Thanks very much for your patient instruction.

For people who want Pymaf-x �with expression params fitting with SMPLXv1.1, i recommend https://github.com/open-mmlab/mmhuman3d/tree/main/configs/pymafx.

Hi, thank you for recommending this version of pymaf-x. I have successfully ran it but I found out somehow its output is weird. The shape of fullpose is only (55,3). Did you also meet this problem and how to convert it to the shape we want?

[Darius-H]

I have figured out where the problem is. Since this time: HongwenZhang/PyMAF-X@d5fe7a4, Pymafx has changed their smplx model which is different from SMPLXv1.1, and outputs a 50 expression dim. Current smplx can not load this expression param. Thanks very much for your patient instruction.
For people who want Pymaf-x �with expression params fitting with SMPLXv1.1, i recommend https://github.com/open-mmlab/mmhuman3d/tree/main/configs/pymafx.

Hi, thank you for recommending this version of pymaf-x. I have successfully ran it but I found out somehow its output is weird. The shape of fullpose is only (55,3). Did you also meet this problem and how to convert it to the shape we want?

Use the result here, before they concat everything into full pose.
https://github.com/open-mmlab/mmhuman3d/blob/9431addec32f7fbeffa1786927a854c0ab79d9ea/demo/pymafx_estimate_smplx.py#L315

[felixshing]

I have figured out where the problem is. Since this time: HongwenZhang/PyMAF-X@d5fe7a4, Pymafx has changed their smplx model which is different from SMPLXv1.1, and outputs a 50 expression dim. Current smplx can not load this expression param. Thanks very much for your patient instruction.
For people who want Pymaf-x �with expression params fitting with SMPLXv1.1, i recommend https://github.com/open-mmlab/mmhuman3d/tree/main/configs/pymafx.

Hi, thank you for recommending this version of pymaf-x. I have successfully ran it but I found out somehow its output is weird. The shape of fullpose is only (55,3). Did you also meet this problem and how to convert it to the shape we want?

Use the result here, before they concat everything into full pose. https://github.com/open-mmlab/mmhuman3d/blob/9431addec32f7fbeffa1786927a854c0ab79d9ea/demo/pymafx_estimate_smplx.py#L315

Hello, sorry to bother you again. I would like to ask did you meet the problem that the number of output frames of pymafx is different from that of the video? I also post this issue here: open-mmlab/mmhuman3d#380

[Darius-H]

I have figured out where the problem is. Since this time: HongwenZhang/PyMAF-X@d5fe7a4, Pymafx has changed their smplx model which is different from SMPLXv1.1, and outputs a 50 expression dim. Current smplx can not load this expression param. Thanks very much for your patient instruction.
For people who want Pymaf-x �with expression params fitting with SMPLXv1.1, i recommend https://github.com/open-mmlab/mmhuman3d/tree/main/configs/pymafx.

Hi, thank you for recommending this version of pymaf-x. I have successfully ran it but I found out somehow its output is weird. The shape of fullpose is only (55,3). Did you also meet this problem and how to convert it to the shape we want?

Use the result here, before they concat everything into full pose. https://github.com/open-mmlab/mmhuman3d/blob/9431addec32f7fbeffa1786927a854c0ab79d9ea/demo/pymafx_estimate_smplx.py#L315

Hello, sorry to bother you again. I would like to ask did you meet the problem that the number of output frames of pymafx is different from that of the video? I also post this issue here: open-mmlab/mmhuman3d#380

Maybe there is more than one person detected in your video. One way to get the main person is that make renderer render the smplx person to img, and turn it to mask to figure out which person occupies the most pixels.

[felixshing]

I have figured out where the problem is. Since this time: HongwenZhang/PyMAF-X@d5fe7a4, Pymafx has changed their smplx model which is different from SMPLXv1.1, and outputs a 50 expression dim. Current smplx can not load this expression param. Thanks very much for your patient instruction.
For people who want Pymaf-x �with expression params fitting with SMPLXv1.1, i recommend https://github.com/open-mmlab/mmhuman3d/tree/main/configs/pymafx.

Hi, thank you for recommending this version of pymaf-x. I have successfully ran it but I found out somehow its output is weird. The shape of fullpose is only (55,3). Did you also meet this problem and how to convert it to the shape we want?

Use the result here, before they concat everything into full pose. https://github.com/open-mmlab/mmhuman3d/blob/9431addec32f7fbeffa1786927a854c0ab79d9ea/demo/pymafx_estimate_smplx.py#L315

Hello, sorry to bother you again. I would like to ask did you meet the problem that the number of output frames of pymafx is different from that of the video? I also post this issue here: open-mmlab/mmhuman3d#380

Maybe there is more than one person detected in your video. One way to get the main person is that make renderer render the smplx person to img, and turn it to mask to figure out which person occupies the most pixels.

Oh okay, but actually my video indeed has one person lol. I will try to figure out the reason