New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

task-based LR scheduling, finer-grained val reporting #33

Open

2 tasks

TimotheeMickus opened this issue Oct 16, 2023 · 0 comments

Open

2 tasks

task-based LR scheduling, finer-grained val reporting #33

TimotheeMickus opened this issue Oct 16, 2023 · 0 comments

Assignees

Labels

Collaborator

TimotheeMickus commented Oct 16, 2023 •

edited

Loading

break down accuracy and validation loss per task when reporting stats
introduce validation-based LR scheduling (reduce LR on plateau) that's properly adjusted depending on the performances of individual modules:
- for each module, initialize module-specific LRs to the default value
- at validation:
  - compute perf per task
  - aggregate (mean or task-weighted mean) perf per module across task
  - reduce module-specific LR based on the aggregate perf

TimotheeMickus self-assigned this

TimotheeMickus added the enhancement label

# for free to join this conversation on GitHub. Already have an account? # to comment