Critic Network Error and Poor Rewards in mujoco_sac.py

[payamparvizi]

Hello,

I am trying to run the code in tianshou/examples/mujoco/mujoco_sac.py. However, I got an error in the critic networks (net_c1 and net_c2). The error message indicates that setting concat=True causes a size mismatch.

I commented that part and ran the code, but I did not get good rewards from the Ant-v4 environment as expected. To enable concat=True, I customized the network, but the results were still not sufficient. The rewards I get are always below 0, no matter how long I run it.

Is it possible that the hyperparameters are incorrect?
Please let me know how I can solve this problem.

Regards

[MischaPanch]

Hi, sorry for the late answer. It's better to open an issue instead of a discussion, I rarely look at those ;)

I'm going to look into it