[Cloud] sh: 1: Bad substitution error when using docker as runtime environment

[novahow]

when I use my docker image as runtime, the setup fails with

I 05-15 21:00:14 provisioner.py:76] Launching on GCP us-central1 (us-central1-a)
I 05-15 21:00:30 provisioner.py:458] Successfully provisioned or found existing instance.
I 05-15 21:01:45 provisioner.py:560] Successfully provisioned cluster: gcp0
I 05-15 21:01:45 cloud_vm_ray_backend.py:3213] Running setup on 1 node.
> /mnt/d/ebook/FlyteDev/skyTest/skypilot/sky/backends/cloud_vm_ray_backend.py(3176)_setup_node()
-> returncode = runner.run(
(Pdb) c
Warning: Permanently added '34.70.197.193' (ED25519) to the list of known hosts.
Warning: Permanently added '[localhost]:10022' (ED25519) to the list of known hosts.
bash: no job control in this shell
bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
total 12
drwxr-xr-x 2 flytekit flytekit 4096 May 15 11:30 .
drwxr-xr-x 1 flytekit flytekit 4096 May 15 13:01 ..
sh: 1: Bad substitution
setuped
Clusters
NAME  LAUNCHED    RESOURCES                                                                  STATUS  AUTOSTOP  COMMAND                       
gcp0  3 mins ago  1x GCP(e2-small, image_id={'us-central1': 'docker:flytesky/plugins:sky...  UP      -         sky launch -c gcp0 hello_...  

sky.exceptions.CommandError: Command /bin/bash -i /tmp/sky_setup_sky-2024-05-15-21-00-04-823032 2>&1 failed with return code 2.
Failed to setup with return code 2. Check the details in log: ~/sky_logs/sky-2024-05-15-21-00-04-823032/setup-34.70.197.193-22.log

****** START Last lines of setup output ******
bash: no job control in this shell
bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
total 12
drwxr-xr-x 2 flytekit flytekit 4096 May 15 11:30 .
drwxr-xr-x 1 flytekit flytekit 4096 May 15 13:01 ..
setuped
******* END Last lines of setup output *******

I believe the cause was

skypilot/sky/backends/cloud_vm_ray_backend.py

Line 3178 in 5a2f1b8

source_bashrc=True,

, since when I deleted the source_bashrc=True statement, my setup worked fine. the task fails especially when my docker image uses /bin/sh by default, which can't get ${PIPESTATUS[0]}

Version & Commit info:

• sky -v: PLEASE_FILL_IN
• sky -c: PLEASE_FILL_IN

[Michaelvll]

Can’t believe we missed this issue. Related to #3934. We should take a look at this.

[cblmemo]

Hi @novahow , could you share your skypilot version (output of sky -c and sky -v) and the docker image you are using?

Edited: I think this is already fixed by #3867. I tested the following command on latest master and it works well. Do you want to try the nightly build of SkyPilot (pip install "skypilot-nightly[all]") and see if this problem persists?

• sky launch --cloud gcp --image-id docker:flytesky/plugins:skypilot
• sky launch --cloud gcp --image-id docker:flytesky/plugins:skypilot_bash_entry

[Michaelvll]

Closing this issue now due to the fix in #3867. Please feel free to re-open it if the problem still exists.