Fixed Ant never sending the done signal

[sash-a]

Noticed this when I was trying to find a fix for this issue. The bug seems to be with these two lines:

pybullet-gym/pybulletgym/envs/mujoco/robots/locomotors/ant.py

Line 29 in bc68201

return +1 if z > 0.26 else -1 # 0.25 is central sphere rad, die if it scrapes the ground

and

pybullet-gym/pybulletgym/envs/mujoco/envs/locomotion/walker_base_env.py

Lines 59 to 60 in bc68201

	alive = float(self.robot.alive_bonus(state[0]+self.robot.initial_z, self.robot.body_rpy[1])) # state[0] is body height above ground, body_rpy[1] is pitch
	done = alive < 0

I'm not sure if it should be changed in Ant or in WalkerBaseMuJoCoEnv not sure if other envs could benefit from this too? I made the change in Ant so as not to affect other envs.

Steps to reproduce the bug, simply run:

env = gym.make('AntMuJoCoEnv-v0')
env.render()
env.reset()
done = False
step = 0
while not done:
    _, _, done, _ = env.step([0] * 8)
    step += 1

print(step)

It should print 1000 indicating that the Ant run for the max number of steps, but its torso should have been touching the ground the whole time, which is undesired behavior.

[benelot]

Sounds like a reasonable fix for now. I think I will have to go through the code again at some point. I did not really find time and motivation as I was stuck with the missing observations anyway, making the project hard to use compared to mujoco. Maybe there will be a resolution to all this soon.