-
Notifications
You must be signed in to change notification settings - Fork 284
Fixed Hyper-V VM remaining in invalid state #4012
Conversation
If creating the machine failed during virtual disk creation then the VM would remain in a an invalid state (actually not the VM but some config files used by docker-machine). This in turn would make it impossible to re-create the VM. The error was unsuccessfully mitigated against by adding a remove command in HyperVHypervisor._failed_to_create() method. This was not effective because removing the VM sometimes fails if it done shortly after the unsuccessful creation. To make the cleanup more robust a retry loop was added. It required changing the logic of DockerMachineHypervisor.vms() because `docker- machine remove -q` doesn't print out VMs in invalid states.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice find on the -q
!
Sad the retry is needed but good its more robust now
Codecov Report
@@ Coverage Diff @@
## b0.19 #4012 +/- ##
==========================================
+ Coverage 89.18% 89.25% +0.06%
==========================================
Files 205 205
Lines 18414 18422 +8
==========================================
+ Hits 16423 16443 +20
+ Misses 1991 1979 -12 |
f'machine "{name}" failed') | ||
time.sleep(0.1) | ||
|
||
else: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Doesn't this always trigger on the last iteration?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That was the idea. If all the iterations passed then we assume that removing the VM failed. If it succeeded then break
would be triggered and else
clause wouldn't be executed.
If creating the machine failed during virtual disk creation then the VM would remain in a an invalid state (actually not the VM but some config files used by docker-machine). This in turn would make it impossible to re-create the VM.
The error had been unsuccessfully mitigated against by adding a remove command in
HyperVHypervisor._failed_to_create()
method. This was not effective because removing the VM sometimes fails if it done shortly after the unsuccessful creation.To make the cleanup more robust a retry loop was added. It required changing the logic of
DockerMachineHypervisor.vms()
becausedocker-machine remove -q
doesn't print out VMs in invalid states.