-
Notifications
You must be signed in to change notification settings - Fork 8.6k
Leaderboard
- This page tracks the performance of user algorithms for various tasks in gym. Previously, users could submit their scores directly to [gym.openai.com/envs](https://gym.openai.com/envs/), but it has been decided that a simpler wiki might do this task more efficiently.
This wiki page is a community driven page. Anyone can edit this page and add to it. We encourage you to contribute and modify this page and add your scores and links to your write-ups and code to reproduce your results. We also encourage you to add new tasks with the gym interface, but not in the core gym library (such as roboschool) to this page as well.
Links to videos are optional, but encouraged. Videos can be youtube, instagram, a tweet, or other public links. Write-ups should explain how to reproduce the result, and can be in the form of a simple gist link, blog post, or github repo.
We have begun to copy over the previous performance scores and write-up links over from the [previous page](https://gym.openai.com/envs/). This is an ongoing effort, and we can use some help.
- Environments
- Classic control
- CartPole-v0
<img align="right" width="200" src="https://user-images.githubusercontent.com/8510097/31701291-3b9f3d94-b384-11e7-8ee1-70fb1e7deb63.PNG"> A pole is attached by an un-actuated joint to a cart, which moves along a frictionless track. The system is controlled by applying a force of +1 or -1 to the cart. The pendulum starts upright, and the goal is to prevent it from falling over. A reward of +1 is provided for every timestep that the pole remains upright. The episode ends when the pole is more than 15 degrees from vertical, or the cart moves more than 2.4 units from the center.
![https://user-images.githubusercontent.com/8510097/31701291-3b9f3d94-b384-11e7-8ee1-70fb1e7deb63.PNG">](https://user-images.githubusercontent.com/8510097/31701291-3b9f3d94-b384-11e7-8ee1-70fb1e7deb63.PNG">){kind=link}
- [Environment Details](https://github.com/openai/gym/wiki/CartPole-v0)
- *CartPole-v0 defines "solving" as getting average reward of 195.0 over 100 consecutive trials.*
- *This environment corresponds to the version of the cart-pole problem described by Barto, Sutton, and Anderson [Barto83].*
- MountainCar-v0
<img align="right" width="200" src="https://user-images.githubusercontent.com/8510097/31701297-3ebf291c-b384-11e7-8289-24f1d392fb48.PNG"> A car is on a one-dimensional track, positioned between two "mountains". The goal is to drive up the mountain on the right; however, the car's engine is not strong enough to scale the mountain in a single pass. Therefore, the only way to succeed is to drive back and forth to build up momentum.
![https://user-images.githubusercontent.com/8510097/31701297-3ebf291c-b384-11e7-8289-24f1d392fb48.PNG">](https://user-images.githubusercontent.com/8510097/31701297-3ebf291c-b384-11e7-8289-24f1d392fb48.PNG">){kind=link}
- [Environment details](https://github.com/openai/gym/wiki/MountainCar-v0)
- *MountainCar-v0 defines "solving" as getting average reward of -110.0 over 100 consecutive trials.*
- *This problem was first described by Andrew Moore in his PhD thesis [Moore90].*
- MountainCarContinuous-v0
<img align="right" width="200" src="https://user-images.githubusercontent.com/8510097/31701297-3ebf291c-b384-11e7-8289-24f1d392fb48.PNG"> A car is on a one-dimensional track, positioned between two "mountains". The goal is to drive up the mountain on the right; however, the car's engine is not strong enough to scale the mountain in a single pass. Therefore, the only way to succeed is to drive back and forth to build up momentum. Here, the reward is greater if you spend less energy to reach the goal
Here, this is the continuous version.
- [Environment details](https://github.com/openai/gym/wiki/MountainCarContinuous-v0)
- *MountainCarContinuous-v0 defines "solving" as getting average reward of 90.0 over 100 consecutive trials.*
- *This problem was first described by Andrew Moore in his PhD thesis [Moore90].*
- Pendulum-v0 <img align="right" width="200" src="https://user-images.githubusercontent.com/8510097/31701471-726f54c0-b385-11e7-9f05-5c50f2affbb4.PNG"> The inverted pendulum swingup problem is a classic problem in the control literature. In this version of the problem, the pendulum starts in a random position, and the goal is to swing it up so it stays upright.
![https://user-images.githubusercontent.com/8510097/31701471-726f54c0-b385-11e7-9f05-5c50f2affbb4.PNG">](https://user-images.githubusercontent.com/8510097/31701471-726f54c0-b385-11e7-9f05-5c50f2affbb4.PNG">){kind=link}
- [Environment details](https://github.com/openai/gym/wiki/Pendulum-v0)
- *Pendulum-v0 is an unsolved environment, which means it does not have a specified reward threshold at which it's considered solved.*
- Acrobot-v1 <img align="right" width="200" src="https://user-images.githubusercontent.com/8510097/32645660-4117b140-c59d-11e7-8290-43ed1b384dac.PNG">
![https://user-images.githubusercontent.com/8510097/32645660-4117b140-c59d-11e7-8290-43ed1b384dac.PNG">](https://user-images.githubusercontent.com/8510097/32645660-4117b140-c59d-11e7-8290-43ed1b384dac.PNG">){kind=link}
The acrobot system includes two joints and two links, where the joint between the two links is actuated. Initially, the links are hanging downwards, and the goal is to swing the end of the lower link up to a given height.
- *Acrobot-v1* is an unsolved environment, which means it does not have a specified reward threshold at which it's considered solved.*
- *Control of Acrobot around equilibrium was described by J. Hauser and R. Murray in ACC 1990. Swing-up control of Acrobot is in M. W. Spong, IEEE Control Systems Magazine, 1995.*
- *Learning control on Acrobot was first described by Sutton [Sutton96]. We are using the version from RLPy [Geramiford15], which uses Runge-Kutta integration for better accuracy.*
| User | Best 100-episode performance | Write-up | Video |
| -------------------- | ------------------------------ | ---------- | ------- |
| mallochio | -42.37 ± 4.83 | [taken down](https://gym.openai.com/evaluations/eval_AXtZeKiTzilcUHfpwjrwA/#reproducibility) | |
| marunowskia | -59.31 ± 1.23 | ||
| MontrealAI | -60.82 ± 0.06 | ||
| [BS Haney](https://github.com/Bhaney44) | -61.8 | [Write-up](https://github.com/Bhaney44/Acrobot/tree/master) | [YouTube](https://www.youtube.com/watch?v=7x5KrtiOaCM&feature=youtu.be) |
| [Felix Nica](https://github.com/FelixNica) | -63.13 ± 2.65 | [Write-up](https://github.com/FelixNica/OpenAI_Acrobot_D3QN) | [YouTube](https://www.youtube.com/watch?v=JQP8gL-WAwE) |
[Nick Kaparinos](https://github.com/NickKaparinos) |-64.30 ± 4.10|[Write-up](https://github.com/NickKaparinos/OpenAI-Gym-Projects) |[gif](https://github.com/NickKaparinos/OpenAI-Gym-Projects/blob/master/Classic%20Control/Acrobot/results/openaigym.video.38.37714.video000000.gif) |
- Box2D
- LunarLander-v2
<img align="right" width="200" src="https://user-images.githubusercontent.com/8510097/31702728-c2b8db88-b38d-11e7-8d1e-d15450303bdd.PNG"> Landing pad is always at coordinates (0,0). Coordinates are the first two numbers in state vector. Reward for moving from the top of the screen to landing pad and zero speed is about 100..140 points. If lander moves away from landing pad it loses reward back. Episode finishes if the lander crashes or comes to rest, receiving additional -100 or +100 points. Each leg ground contact is +10. Firing main engine is -0.3 points each frame. Solved is 200 points. Landing outside landing pad is possible. Fuel is infinite, so an agent can learn to fly and then land on its first attempt. Four discrete actions available: do nothing, fire left orientation engine, fire main engine, fire right orientation engine.
![https://user-images.githubusercontent.com/8510097/31702728-c2b8db88-b38d-11e7-8d1e-d15450303bdd.PNG">](https://user-images.githubusercontent.com/8510097/31702728-c2b8db88-b38d-11e7-8d1e-d15450303bdd.PNG">){kind=link}
- *LunarLander-v2 defines "solving" as getting average reward of 200 over 100 consecutive trials.*=
- *by @olegklimov*
- LunarLanderContinuous-v2
<img align="right" width="200" src="https://user-images.githubusercontent.com/8510097/31702728-c2b8db88-b38d-11e7-8d1e-d15450303bdd.PNG"> Landing pad is always at coordinates (0,0). Coordinates are the first two numbers in state vector. Reward for moving from the top of the screen to landing pad and zero speed is about 100..140 points. If lander moves away from landing pad it loses reward back. Episode finishes if the lander crashes or comes to rest, receiving additional -100 or +100 points. Each leg ground contact is +10. Firing main engine is -0.3 points each frame. Solved is 200 points. Landing outside landing pad is possible. Fuel is infinite, so an agent can learn to fly and then land on its first attempt. Action is two real values vector from -1 to +1. First controls main engine, -1..0 off, 0..+1 throttle from 50% to 100% power. Engine can't work with less than 50% power. Second value -1.0..-0.5 fire left engine, +0.5..+1.0 fire right engine, -0.5..0.5 off.
- *LunarLanderContinuous-v2 defines "solving" as getting average reward of 200 over 100 consecutive trials.*
- BipedalWalker-v2 and BipedalWalker-v3
<img align="right" width="200" src="https://user-images.githubusercontent.com/8510097/31702845-786ae5a2-b38e-11e7-8a8c-0952bf6490c0.PNG"> Reward is given for moving forward, total 300+ points up to the far end. If the robot falls, it gets -100. Applying motor torque costs a small amount of points, more optimal agent will get better score. State consists of hull angle speed, angular velocity, horizontal speed, vertical speed, position of joints and joints angular speed, legs contact with ground, and 10 lidar rangefinder measurements. There's no coordinates in the state vector.
![https://user-images.githubusercontent.com/8510097/31702845-786ae5a2-b38e-11e7-8a8c-0952bf6490c0.PNG">](https://user-images.githubusercontent.com/8510097/31702845-786ae5a2-b38e-11e7-8a8c-0952bf6490c0.PNG">){kind=link}
- [Environment Details](https://github.com/openai/gym/wiki/BipedalWalker-v2)
- *BipedalWalker-v2 defines "solving" as getting average reward of 300 over 100 consecutive trials.*
- *by @olegklimov*
- BipedalWalkerHardcore-v2 and BipedalWalkerHardcore-v3
<img align="right" width="200" src="https://user-images.githubusercontent.com/8510097/31704513-996d6a56-b396-11e7-8a05-5bd45b35d352.PNG"> Hardcore version with ladders, stumps, pitfalls. Time limit is increased due to obstacles. Reward is given for moving forward, total 300+ points up to the far end. If the robot falls, it gets -100. Applying motor torque costs a small amount of points, more optimal agent will get better score. State consists of hull angle speed, angular velocity, horizontal speed, vertical speed, position of joints and joints angular speed, legs contact with ground, and 10 lidar rangefinder measurements. There's no coordinates in the state vector.
![https://user-images.githubusercontent.com/8510097/31704513-996d6a56-b396-11e7-8a05-5bd45b35d352.PNG">](https://user-images.githubusercontent.com/8510097/31704513-996d6a56-b396-11e7-8a05-5bd45b35d352.PNG">){kind=link}
- *BipedalWalkerHardcore-v2 defines "solving" as getting average reward of 300 over 100 consecutive trials.*
- CarRacing-v0
<img align="right" width="200" src="https://user-images.githubusercontent.com/8510097/31704419-298c707e-b396-11e7-8858-b9041db8198e.PNG"> Easiest continuous control task to learn from pixels, a top-down racing environment. Discreet control is reasonable in this environment as well, on/off discretisation is fine. State consists of 96x96 pixels. Reward is -0.1 every frame and +1000/N for every track tile visited, where N is the total number of tiles in track. For example, if you have finished in 732 frames, your reward is 1000 - 0.1*732 = 926.8 points. Episode finishes when all tiles are visited. Some indicators shown at the bottom of the window and the state RGB buffer. From left to right: true speed, four ABS sensors, steering wheel position, gyroscope.
![https://user-images.githubusercontent.com/8510097/31704419-298c707e-b396-11e7-8858-b9041db8198e.PNG">](https://user-images.githubusercontent.com/8510097/31704419-298c707e-b396-11e7-8858-b9041db8198e.PNG">){kind=link}
- by @olegklimov
- *CarRacing-v0 defines "solving" as getting average reward of 900 over 100 consecutive trials.*
(*) They used reward shaping (added some score back when the agent dies) during training to make training work better, but unfortunately kept the artificial shaped score for evaluation. When testing their agent using their model (and also trying to train it from scratch, which performed worse), we got a score of 820. We have filed an [issue](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Udacity/issues/3). We found a similar problem with another PPO repo [here](https://github.com/xtma/pytorch_car_caring/issues/2).
- CarRacing-v1
v1: Changed track completion logic and added domain randomization (0.24.0)
| User | Episodes before solve | 100-Episode Average Score | Write-up | Video |
| --------------------------------------- | ----------------------- | --------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- | ------- |
| [Ray Coden Mercurius](https://github.com/Ceudan) | 925 | 917 | [writeup](https://github.com/Ceudan/Car-Racing) | [video](https://user-images.githubusercontent.com/78922263/177021714-fc82a6ff-e44c-4936-bf50-61a8f3a372f1.mp4) |
- MuJoCo
- Inverted Pendulum
This environment involves a cart that can be moved linearly, with a pole fixed on it at one end and having another end free. The cart can be pushed left or right, and the goal is to balance the pole on the top of the cart by applying forces on the cart.
| User | Episode | 100-Episode Average Score | Write-up | Video |
| --- | --- | --- | --- | --- |
| [timurgepard](https://github.com/timurgepard/Simphony/tree/main) | 56 | 1000.0 (Symphony🎹 ver 2.0) | [writeup](https://github.com/timurgepard/Simphony/tree/main) |
- Walker2d-v1 and Walker 2d-v2
<img align="right" width="200" src="https://user-images.githubusercontent.com/8510097/31702336-36cd4606-b38b-11e7-9b59-4d5018de8572.PNG"> Make a two-dimensional bipedal robot walk forward as fast as possible.
![https://user-images.githubusercontent.com/8510097/31702336-36cd4606-b38b-11e7-9b59-4d5018de8572.PNG">](https://user-images.githubusercontent.com/8510097/31702336-36cd4606-b38b-11e7-9b59-4d5018de8572.PNG">){kind=link}
- *Walker2d-v1 is an unsolved environment, which means it does not have a specified reward threshold at which it's considered solved.*
- *The robot model is based on work by Erez, Tassa, and Todorov [Erez11].*
- Ant-v1
<img align="right" width="200" src="https://user-images.githubusercontent.com/8510097/31752126-30384f76-b43e-11e7-94b9-2b32b52abe85.PNG"> Make a four-legged creature walk forward as fast as possible.
![https://user-images.githubusercontent.com/8510097/31752126-30384f76-b43e-11e7-94b9-2b32b52abe85.PNG">](https://user-images.githubusercontent.com/8510097/31752126-30384f76-b43e-11e7-94b9-2b32b52abe85.PNG">){kind=link}
- *Ant-v1 defines "solving" as getting average reward of 6000.0 over 100 consecutive trials.*
- *This task originally appeared in [Schulman15].*
| User | Episode | 100-Episode Average Score | Write-up | Video |
| --- | --- | --- | --- | --- |
| [timurgepard](https://github.com/timurgepard/Simphony/tree/main) | 700 | 9320.0 (Symphony🎹 ver 2.0) | [writeup](https://github.com/timurgepard/Simphony/tree/main) | [video](https://youtu.be/tA7D6aakUA4) |
| [zlw21gxy](https://github.com/zlw21gxy) | 1000 | N/A | [writeup](https://arxiv.org/pdf/1908.11494.pdf) | |
| [pat-coady](https://github.com/pat-coady) | 69154 | N/A | [writeup](https://gist.github.com/pat-coady/bac60888f011199aad72d2f1e6f5a4fa#file-ant-ipynb) | |
| [joschu](https://github.com/joschu) | N/A | N/A | [writeup](https://gist.github.com/joschu/f8c5e6ba94d8bbd8c13ee2fbd6c4a604#file-1-cem-v1-writeup-md) |
- HalfCheetah-v4 <img align="right" width="200" alt="half_cheetah" src="https://github.com/openai/gym/assets/60650661/461b6400-9851-4f51-8d5d-70042772f0f3"> Make a 2-dimensional robot walk forward as fast as possible.
- The HalfCheetah is a 2-dimensional robot consisting of 9 body parts and 8 joints connecting them (including two paws).
- The goal is to apply a torque on the joints to make the cheetah run forward (right) as fast as possible.
- *This environment is based on the work by P. Wawrzyński*
| User | Episodes before solve | Write-up | Video |
| ------------------------------------------- | ----------------------- | ----------------------------------------------------------------------------------------------------- | ------- |
| [timurgepard](https://github.com/timurgepard) | 25 (Symphony🎹 ver 2.0) | [writeup](https://github.com/timurgepard/Simphony/tree/main) | [video](https://youtu.be/aAuvCO_LDvA) |
| [tareknaser](https://github.com/tareknaser) | N/A | [writeup](https://github.com/tareknaser/GSoC23-mlpack-RL-Report/blob/main/src/half_cheetah/HalfCheetah_Writeup.md) | [video](https://github.com/tareknaser/GSoC23-mlpack-RL-Report/blob/main/src/half_cheetah/half_cheetah.gif) |
- Humanoid-v4
<img align="right" width="200" src="https://user-images.githubusercontent.com/13238473/274489739-bc4c08e4-f5ff-4cb1-83c8-9bc531eab61f.png"> Make 3D humanoid robot walk forward as fast as possible.
![https://user-images.githubusercontent.com/13238473/274489739-bc4c08e4-f5ff-4cb1-83c8-9bc531eab61f.png">](https://user-images.githubusercontent.com/13238473/274489739-bc4c08e4-f5ff-4cb1-83c8-9bc531eab61f.png">){kind=link}
- Humanoid-v4 is an unsolved environment, which means it does not have a specified reward threshold at which it's considered solved.
- The 3D bipedal robot is designed to simulate a human. Humanoid-v4 defines "solving" as acquiring human like motions.
- The robot model is based on work by Tassa, Erez, and Todorov [Tassa12].*
| User | Episodes before solve | 100-Episode Average Score | Write-up | Video |
| --- | --- | --- | --- | --- |
| [timurgepard](https://github.com/timurgepard) | 1500 | 12,600.0 (Symphony🎹 ver 2.0) | [writeup](https://github.com/timurgepard/Simphony/tree/main) | [video](https://youtu.be/20yuPY0lA-M) |
- HumanoidStandup-v4
<img align="right" width="200" src="https://user-images.githubusercontent.com/13238473/283983923-3c467b84-0034-4dd2-8878-11704f734635.png">
![https://user-images.githubusercontent.com/13238473/283983923-3c467b84-0034-4dd2-8878-11704f734635.png">](https://user-images.githubusercontent.com/13238473/283983923-3c467b84-0034-4dd2-8878-11704f734635.png">){kind=link}
Make the humanoid standup and then keep it standing by applying torques on the various hinges.
- The environment starts with the humanoid laying on the ground, and then the goal of the environment is to make the humanoid standup and then keep it standing by applying torques on the various hinges.
- The 3D bipedal robot is designed to simulate a human. It has a torso (abdomen) with a pair of legs and arms. The legs each consist of two links, and so the arms (representing the knees and elbows respectively).
- This environment is based on the environment introduced by Tassa, Erez and Todorov in [“Synthesis and stabilization of complex behaviors through online trajectory optimization”](https://ieeexplore.ieee.org/document/6386025).
| User | Episodes before solve | 100-Episode Average Score | Write-up | Video |
| --- | --- | --- | --- | --- |
| [timurgepard](https://github.com/timurgepard) | 3200 (step 960k, ep steps 300) | 320000.0 (Symphony🎹 ver 2.0) | [writeup](https://github.com/timurgepard/Simphony/tree/main) | [video](https://youtu.be/GrOEHvgwMAs) |
- Pusher-v4
![https://private-user-images.githubusercontent.com/13238473/300699741-9ec11eab-d119-46dd-a491-44db9d7552f3.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MDY1OTU5MzMsIm5iZiI6MTcwNjU5NTYzMywicGF0aCI6Ii8xMzIzODQ3My8zMDA2OTk3NDEtOWVjMTFlYWItZDExOS00NmRkLWE0OTEtNDRkYjlkNzU1MmYzLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDAxMzAlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwMTMwVDA2MjAzM1omWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWNhYmRkNTYyYmYyZjQ5M2E5MzdmNzA3MjI2NjUyMDJkM2E1YTBlZWU3OWRhMzY5MzE3NjJlMDI1YTlhNGQ1YmYmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.QxRb4Tv4rVQ2Li9S0S2QTGtbwmik8iXaZsXzMKz8kqE">](https://private-user-images.githubusercontent.com/13238473/300699741-9ec11eab-d119-46dd-a491-44db9d7552f3.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MDY1OTU5MzMsIm5iZiI6MTcwNjU5NTYzMywicGF0aCI6Ii8xMzIzODQ3My8zMDA2OTk3NDEtOWVjMTFlYWItZDExOS00NmRkLWE0OTEtNDRkYjlkNzU1MmYzLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDAxMzAlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwMTMwVDA2MjAzM1omWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWNhYmRkNTYyYmYyZjQ5M2E5MzdmNzA3MjI2NjUyMDJkM2E1YTBlZWU3OWRhMzY5MzE3NjJlMDI1YTlhNGQ1YmYmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.QxRb4Tv4rVQ2Li9S0S2QTGtbwmik8iXaZsXzMKz8kqE">){kind=link}
“Pusher” is a multi-jointed robot arm which is very similar to that of a human. The goal is to move a target cylinder (called object) to a goal position using the robot’s end effector (called fingertip). The robot consists of shoulder, elbow, forearm, and wrist joints.
| User | Episodes before solve | 100-Episode Average Score | Write-up | Video |
| --- | --- | --- | --- | --- |
| [timurgepard](https://github.com/timurgepard) | 350 | -45.0 (Symphony🎹 ver 2.0) | [writeup](https://github.com/timurgepard/Simphony/tree/main) | [video](https://youtu.be/QTG2-n-pWEI) |
- Swimmer-v4
![https://private-user-images.githubusercontent.com/13238473/300699425-a267238f-c996-43c1-9739-91dcbeadf3da.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MDY1OTU4MzksIm5iZiI6MTcwNjU5NTUzOSwicGF0aCI6Ii8xMzIzODQ3My8zMDA2OTk0MjUtYTI2NzIzOGYtYzk5Ni00M2MxLTk3MzktOTFkY2JlYWRmM2RhLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDAxMzAlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwMTMwVDA2MTg1OVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTM2OWZiYjUxYTJkMzZjMmU2ZDlkZDUxMzE1NGExNzE2NjNkNDZlMGJhYmNjODE4NzIwN2VhMWI3ZDgzNTc5NDYmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.h7cSO0EuVQeCa2bjR1fQ3PudZOQ9T5zrX2IcPtEQc9E">](https://private-user-images.githubusercontent.com/13238473/300699425-a267238f-c996-43c1-9739-91dcbeadf3da.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MDY1OTU4MzksIm5iZiI6MTcwNjU5NTUzOSwicGF0aCI6Ii8xMzIzODQ3My8zMDA2OTk0MjUtYTI2NzIzOGYtYzk5Ni00M2MxLTk3MzktOTFkY2JlYWRmM2RhLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDAxMzAlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwMTMwVDA2MTg1OVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTM2OWZiYjUxYTJkMzZjMmU2ZDlkZDUxMzE1NGExNzE2NjNkNDZlMGJhYmNjODE4NzIwN2VhMWI3ZDgzNTc5NDYmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.h7cSO0EuVQeCa2bjR1fQ3PudZOQ9T5zrX2IcPtEQc9E">){kind=link}
The swimmer consist of three segments ('links') and two articulation joints (’rotors’) - one rotor joint connecting exactly two links to form a linear chain. The swimmer is suspended in a two dimensional pool, and the goal is to move as fast as possible towards the right by applying torque on the rotors and using the fluids friction.
| User | Episodes before solve | 100-Episode Average Score | Write-up | Video |
| --- | --- | --- | --- | --- |
| [timurgepard](https://github.com/timurgepard) | 55 | 205.0 (Symphony🎹 ver 2.0) | [writeup](https://github.com/timurgepard/Simphony/tree/main) |
- PyGame Learning Environment
- FlappyBird-v0
<img align="right" height="200" src="https://user-images.githubusercontent.com/8510097/31752540-4e2cf6f6-b440-11e7-9186-852fa5b93ccf.PNG">
![https://user-images.githubusercontent.com/8510097/31752540-4e2cf6f6-b440-11e7-9186-852fa5b93ccf.PNG">](https://user-images.githubusercontent.com/8510097/31752540-4e2cf6f6-b440-11e7-9186-852fa5b93ccf.PNG">){kind=link}
This environment adapts a game from the [PyGame Learning Environment](https://pygame-learning-environment.readthedocs.io/en/latest/) (PLE). To run it, you will need to install gym-ple from https://github.com/lusob/gym-ple.
Flappybird is a side-scrolling game where the agent must successfully navigate through gaps between pipes. The up arrow causes the bird to accelerate upwards. If the bird makes contact with the ground or pipes, or goes above the top of the screen, the game is over. For each pipe it passes through it gains a positive reward of +1. Each time a terminal state is reached it receives a negative reward of -1.
- *FlappyBird-v0 is an unsolved environment, which means it does not have a specified reward threshold at which it's considered solved.*
- *by @lusob*
- Snake-v0
Snake is a game where the agent must maneuver a line which grows in length each time food is touched by the head of the segment. The line follows the previous paths taken which eventually become obstacles for the agent to avoid.
The food is randomly spawned inside of the valid window while checking it does not make contact with the snake body.
| User | Best 100-episode performance | Write-up | Video |
| --------------------------------------------------- | ------------------------------ | -------------------------------------------------------------------------------------------------- | ------- |
| [carsonprindle](https://github.com/carsonprindle) | .44 ± .04 | [writeup](https://github.com/carsonprindle/OpenAIExam2018) |
- Atari Games
- Atlantis-v0
- Atari Games
| User | Best 100-episode performance | Write-up |
| --------------------------------------------------- | ------------------------------ | -------------------------------------------------------------------------------------------------- |
| msemple1111 | 62,500 ± 0 | [writeup](https://github.com/MElena14/COMP341/blob/main/assignment2.ipynb) |
- Breakout-v0
| User | Best 100-episode performance | Write-up |
| --------------------------------------------------- | ------------------------------ | -------------------------------------------------------------------------------------------------- |
| ppwwyyxx | 760.07 ± 18.37 | [writeup](https://github.com/tensorpack/tensorpack/tree/master/examples/A3C-Gym) |
- Pong-v5
| User | Best 100-episode performance | Write-up |
| --------------------------------------------------- | ------------------------------ | -------------------------------------------------------------------------------------------------- |
[Nick Kaparinos](https://github.com/NickKaparinos) |21.00 ± 0.00|[Write-up](https://github.com/NickKaparinos/OpenAI-Gym-Projects) |[gif](https://github.com/NickKaparinos/OpenAI-Gym-Projects/blob/master/Atari/Pong/Results/pong.gif) |
| ppwwyyxx | 20.81 ± 0.04 | [writeup](https://github.com/tensorpack/tensorpack/tree/master/examples/A3C-Gym) |
- MsPacman-v0
| User | Best 100-episode performance | Write-up |
| --------------------------------------------------- | ------------------------------ | -------------------------------------------------------------------------------------------------- |
| ppwwyyxx | 5738.30 ± 171.99 | [writeup](https://github.com/tensorpack/tensorpack/tree/master/examples/A3C-Gym) |
- SpaceInvaders-v0
| User | Best 100-episode performance | Write-up |
| --------------------------------------------------- | ------------------------------ | -------------------------------------------------------------------------------------------------- |
| ppwwyyxx | 3454.00 ± 0 | [writeup](https://github.com/tensorpack/tensorpack/tree/master/examples/A3C-Gym) |
- Seaquest-v0
| User | Best 100-episode performance | Write-up |
| --------------------------------------------------- | ------------------------------ | -------------------------------------------------------------------------------------------------- |
| ppwwyyxx | 50209 ± 2440.07 | [writeup](https://github.com/tensorpack/tensorpack/tree/master/examples/A3C-Gym) |
- Toy text Simple text environments to get you started.
<a name="TaxiV2"></a>
- Taxi-v2 This task was introduced in [Dietterich2000] to illustrate some issues in hierarchical reinforcement learning. There are 4 locations (labeled by different letters) and your job is to pick up the passenger at one location and drop him off in another. You receive +20 points for a successful dropoff, and lose 1 point for every timestep it takes. There is also a 10 point penalty for illegal pick-up and drop-off actions.
[Dietterich2000] T Erez, Y Tassa, E Todorov, "Hierarchical Reinforcement Learning with the MAXQ Value Function Decomposition", 2011.
<a name="TaxiV3"></a>
- Taxi-v3 This task was introduced in [Dietterich2000] to illustrate some issues in hierarchical reinforcement learning. There are 4 locations (labeled by different letters) and your job is to pick up the passenger at one location and drop him off in another. You receive +20 points for a successful dropoff, and lose 1 point for every timestep it takes. There is also a 10 point penalty for illegal pick-up and drop-off actions.
[Dietterich2000] T Erez, Y Tassa, E Todorov, "Hierarchical Reinforcement Learning with the MAXQ Value Function Decomposition", 2011.
- GuessingGame-V0 The goal of the game is to guess within 1% of the randomly chosen number within 200 time steps
After each step the agent is provided with one of four possible observations which indicate where the guess is in relation to the randomly chosen number
| User | Average Episode Steps | Write-up | Video | Solved In Episode |
| -------------- | ---------------------------------- | ---------- | ------- | ------------------- |
| [Anandha Krishnan H](https://github.com/AKing1998) | 51 (use close-form preset policy) | [writeup](https://github.com/AKing1998/Gym-Toy-Text-Guessing-Game-V0) | ||
| [Britto Sabu](https://github.com/brittosabu) | 53 (use close-form preset policy) | [writeup](https://github.com/brittosabu/Guessing-game) |
- FrozenLake-v0 The agent controls the movement of a character in a grid world. Some tiles of the grid are walkable, and others lead to the agent falling into the water. Additionally, the movement direction of the agent is uncertain and only partially depends on the chosen direction. The agent is rewarded for finding a walkable path to a goal tile.
| User | Episodes Before Solve | Write-up | Video | Solved In Episode |
| -------------- | ---------------------------------- | ---------- | ------- | ------------------- |
| [Nitish tom michael](https://github.com/knitemblazor) | 100 | [writeup](https://github.com/knitemblazor/frozen_lake_openaigym) |
- FrozenLake8x8-v0 The agent controls the movement of a character in a grid world. Some tiles of the grid are walkable, and others lead to the agent falling into the water. Additionally, the movement direction of the agent is uncertain and only partially depends on the chosen direction. The agent is rewarded for finding a walkable path to a goal tile.
| User | 100 Episodes Best Average Reward | Write-up | Video | Solved In Episode |
| -------------- | ---------------------------------- | ---------- | ------- | ------------------- |
| [Sukesh Shenoy](https://github.com/sukesh167) | 85 | [writeup](https://github.com/sukesh167/OpenAI_GYM_TOY_TEXT_FrozenLake8x8-v0/tree/master) |
- Gym Repository
- Wiki Home
- Leaderboard
- Environments
- FAQ
- Resources
- Feature Requests