Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Support custom SweepContainer image #3

Open
3 tasks
davidagold opened this issue Jan 15, 2022 · 1 comment
Open
3 tasks

Support custom SweepContainer image #3

davidagold opened this issue Jan 15, 2022 · 1 comment

Comments

@davidagold
Copy link
Owner

davidagold commented Jan 15, 2022

What

Currently, sweep tasks use a single image defined in the sweep experiment with the model definition and training script backed in. To scale to arbitrary model/training regimes, we need to allow users to run sweep tasks with their own image. The workflow looks like this:

Desired workflow:

  1. Install bacon CLI and deploy a Bacon stack (this step only need to be done once)
  2. In an experiment repository, define your architecture and training script. Build and push (to ECR)
  3. Deploy a capacity provider
  4. Use the CLI to trigger a sweep_dag run, where the dag_config contains:

Acceptance Criteria

  • Export a shortcut to deploy sweep task definition
  • sweep_task is parametrized
  • Sweep experiment config accepts task definition as argument
@davidagold
Copy link
Owner Author

Note that we'll need to modify the init_sweep operator to pass a parametrized project name.

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant