[Proposal] Explicitly requires to allow closing a closed environment

[qgallouedec]

Proposal

• Add a warning in the env checker when the custom environment that doesn't allow to be closed multiple times.
• Mention that it is recommended to be able to call the close function on an already closed environment.

Motivation

It's usually possible to reset an environment several times.

env = gym.make("CartPole-v1")
env.close()
env.close()

Is it good practice to do so? Since there's no simple way of checking whether an environment is closed, I propose to consider that calling the close method on a closed environment should not raise an error.

Pitch

Give your opinion, I'll handle the PR if you're ok.

Alternatives

Add an is_closed method, and always raise an error when the user tries to close a closed environment.

(But it would break so many things...)

Additional context

See DLR-RM/stable-baselines3#1525, and especially DLR-RM/stable-baselines3#1525 (comment)

Checklist

• I have checked that there is no similar issue in the repo

[pseudo-rnd-thoughts]

Yes, I think this is a good idea and yes on the PR, thanks for offering
Would we prevent the second close in the PassiveEnvCheckerWrapper from being forwarded to the base environment?

[qgallouedec]

Would we prevent the second close in the PassiveEnvCheckerWrapper` from being forwarded to the base environment?

I would say no. From a user's perspective, I expect the PassiveEnvChecker to check the environment passively. Preventing a second close can't be considered "passive" anymore.