Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Implement video capturing system #29

Draft
wants to merge 23 commits into
base: main
Choose a base branch
from

Conversation

lorycontixd
Copy link

No description provided.

lorycontixd and others added 21 commits October 9, 2024 11:22
- Generalised urdf_path input for robotModel class to pass either string or a pathlib object
- [To review] Added save_xml flag on robotModel.get_mujoco_model to save the modified xml file
… with associated default values taken from mujoco
Still work in progress, implemented basic structure for PR.

- ScoreCallback associates a simulation to a score which can be customly defined
- TrackerCallback helps to store variable values for either printing or eventual plotting
- Dumping urdf string to temp file if mujoco fails to load
- Added should_stop flag to mujocoSimulator class
@lorycontixd lorycontixd marked this pull request as draft December 13, 2024 14:55
@lorycontixd lorycontixd self-assigned this Dec 13, 2024
@lorycontixd
Copy link
Author

First draft of a video recording system for mujoco.

The idea is to read an image at each frame and save it in a temporary folder. At the end of the simulation, save the video using opencv.

Unfortunately for now it is quite slow.

@lorycontixd
Copy link
Author

@flferretti Do you have any suggestions on the implementation by any chance? (The changes are in the last commit only)

@flferretti
Copy link
Contributor

@flferretti Do you have any suggestions on the implementation by any chance? (The changes are in the last commit only)

I'd avoid writing each frame to a tempfile as it is extremely slow. You can just save it as a numpy array (the default output) in a class attribute. Moreover, try to use the mujoco passive viewer, so the process of headless rendering does not overlap with the interactive viewer used in MujocoSimulator:

self.viewer = mujoco_viewer.MujocoViewer(self.model, self.data)

You might also take inspiration from how we use the mujoco passive viewer in JaxSim https://github.com/ami-iit/jaxsim/blob/main/src/jaxsim/mujoco/visualizer.py

@lorycontixd
Copy link
Author

As of the last commit, the recorder kills the python process because it runs out of memory. Investigating...

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants