Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Enhanced monitor with system metrics logging and wandb support #90

Merged
merged 14 commits into from
May 24, 2022

Conversation

yxdyc
Copy link
Collaborator

@yxdyc yxdyc commented May 16, 2022

Enhanced monitor with system metrics logging and wandb support:

@yxdyc yxdyc added the enhancement New feature or request label May 16, 2022
@yxdyc yxdyc added the Feature New feature label May 16, 2022
Copy link
Collaborator

@rayrayraykk rayrayraykk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good job. And please see the inline comments.

):
log_res_best[
f"{final_type}/{inner_key}"] = inner_val
# log_res_best = {}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are the commented codes for?

except ImportError:
logger.error(
"cfg.wandb.use=True but not install the wandb package")
exit()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about raising ValueError?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If import wandb failed, the system will trigger ImportError

Copy link
Collaborator

@joneswong joneswong left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved! BTW, I noticed that several customized trainers have to explicitly register the newly added hook (seems to estimate flops). I am curious about when we should do that for a new trainer (e.g., the newly added FLIT by Haohui).

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
enhancement New feature or request Feature New feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants