Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

Add a tutorial for vectorized envs using A2C. #234

Merged
merged 11 commits into from
Jan 4, 2023
Merged

Add a tutorial for vectorized envs using A2C. #234

merged 11 commits into from
Jan 4, 2023

Conversation

till2
Copy link
Contributor

@till2 till2 commented Jan 2, 2023

Content:

  • how to create vectorized envs?
  • using vectorized envs for domain randomization to train more robust agents.
  • building an A2C agent and training it on LunarLander-v2 using vector envs

till2 added 4 commits January 2, 2023 18:32
@till2
Add the training plots img for A2C on vector envs
d93b5f8
@till2
Add a tutorial for vectorized envs using A2C.
376e3c8 
Content:
- how to create vectorized envs? 
- using vectorized envs for domain randomization to train more robust agents.
- building an A2C agent and training it on LunarLander-v2 using vector envs
@till2
Fix a typo.
acfcba3
@till2
run pre-commit
da35417
pseudo-rnd-thoughts
pseudo-rnd-thoughts requested changes Jan 2, 2023
View reviewed changes
Copy link
Member

@pseudo-rnd-thoughts pseudo-rnd-thoughts left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tutorial looks good.
You note that the purpose of the tutorial is on speeding up training.

Not sure how painful this would be but could you

  1. Create a new graph with the time (I believe that RecordEpisodes wrapper includes this info) to replace for X time steps using different numbers of environments, 1, 3, 10, 32. Log the performance as well.
  2. Compare the performance for different vector implementations, AsyncVectorEnv, SyncVectorEnv, + EnvPool or SampleFactory if you have time.

docs/tutorials/vector_envs_tutorial.py Outdated Show resolved Hide resolved
till2
till2 commented Jan 4, 2023
View reviewed changes
Copy link
Contributor Author

@till2 till2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything incorporated now.

  • fix type hints
  • add performance analysis section

till2
till2 commented Jan 4, 2023
View reviewed changes
Copy link
Contributor Author

@till2 till2 left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure why isort is failing here, locally it's passing.

pseudo-rnd-thoughts
pseudo-rnd-thoughts approved these changes Jan 4, 2023
View reviewed changes
Copy link
Member

@pseudo-rnd-thoughts pseudo-rnd-thoughts left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New performance section looks good and interesting.
For pre-commit, if you run it twice in a row does it fix the issue?

@pseudo-rnd-thoughts pseudo-rnd-thoughts merged commit 300b383 into Farama-Foundation:main Jan 4, 2023
# for free to join this conversation on GitHub. Already have an account? # to comment
Reviewers

@pseudo-rnd-thoughts pseudo-rnd-thoughts pseudo-rnd-thoughts approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@till2 @pseudo-rnd-thoughts