Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

[GCP DWS] Fix None issue when no provision timeout is provided #3835

Merged
merged 3 commits into from
Aug 16, 2024

Conversation

Michaelvll
Copy link
Collaborator

@Michaelvll Michaelvll commented Aug 16, 2024

It is possible that the provision_timeout is set to None in our cluster yaml, which will cause a wrong CLI to wait for MIG.

To reproduce:

experimental:
  config_overrides:
    gcp:
      managed_instance_group:
        # Setup the DWS config
        run_duration: 36000
        # Failover to other regions after 30 minutes.
        provision_timeout: 1800
resources:
  accelerators: H100:8
  cpus: 32+

Tested (run the relevant ones):

  • Code formatting: bash format.sh
  • Any manual or new tests for this PR (please specify below)
  • All smoke tests: pytest tests/test_smoke.py
  • Relevant individual smoke tests: pytest tests/test_smoke.py::test_fill_in_the_name
  • Backward compatibility tests: conda deactivate; bash -i tests/backward_compatibility_tests.sh

@Michaelvll Michaelvll added this pull request to the merge queue Aug 16, 2024
Merged via the queue into master with commit 69838f5 Aug 16, 2024
20 checks passed
@Michaelvll Michaelvll deleted the mig-provision-timeout branch August 16, 2024 07:33
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants