Python debugging fails to unpack env variables in launch args

[DonJayamanne]

@AjawadMahmoud commented on Wed Jul 10 2019

Issue Type: Bug

Using the following launch.js config:

		{
			"name": "Debug with env variables",
			"type": "python",
			"request": "launch",
			"env":{
				"FW_PACKAGE":"my_package",
				"FW_TEST":"test_name"
			},
			"program": "${workspaceFolder}/appd.py",
			"args": ["--env", "dev", "--packages", "${env:LIMP_PACKAGE}", "--test-flush", "--test", "${env:LIMP_TEST}"],
			"console": "integratedTerminal"
		}

I get in terminal:

cd 'd:\path\to\project'; ${env:FW_PACKAGE}='my_package'; ${env:FW_TEST}='test_name'; ${env:PYTHONIOENCODING}='UTF-8'; ${env:PYTHONUNBUFFERED}='1'; & 'C:\Python37\python.exe' 'c:\Users\USER\.vscode\extensions\ms-python.python-2019.6.24221\pythonFiles\ptvsd_launcher.py' '--default' '--client' '--host' 'localhost' '--port' '58452' 'D:\path\to\project/appd.py' '--env' 'dev' '--packages' '' '--test-flush' '--test' '' 

As it's seen in the terminal debug command, it's failing to unpack both env variables although the command was prefixed with env variables setter. And to confirm the env variables are assigned I was able to print them in the terminal after the debug command failed, and I got the env variables with the respective values correctly.

VS Code version: Code 1.36.1 (2213894, 2019-07-08T22:59:35.033Z)
OS version: Windows_NT x64 10.0.18932

System Info

Item	Value
CPUs	Intel(R) Core(TM) i7-6650U CPU @ 2.20GHz (4 x 2208)
GPU Status	2d_canvas: enabled flash_3d: enabled flash_stage3d: enabled flash_stage3d_baseline: enabled gpu_compositing: enabled multiple_raster_threads: enabled_on native_gpu_memory_buffers: disabled_software oop_rasterization: disabled_off protected_video_decode: unavailable_off rasterization: enabled skia_deferred_display_list: disabled_off skia_renderer: disabled_off surface_synchronization: enabled_on video_decode: enabled viz_display_compositor: disabled_off webgl: enabled webgl2: enabled
Load (avg)	undefined
Memory (System)	15.93GB (6.88GB free)
Process Argv
Screen Reader	no
VM	0%

Extensions (14)

Extension	Author (truncated)	Version
vscode-angular2-files	ale	1.6.2
githistory	don	0.4.6
jbockle-format-files	jbo	3.0.0
chat	kar	0.15.5
code-beautifier	mic	2.2.1
azure-pipelines	ms-	1.152.0
vscode-azureappservice	ms-	0.15.0
vscode-cosmosdb	ms-	0.10.2
python	ms-	2019.6.24221
azure-account	ms-	0.8.4
vsliveshare	ms-	1.0.488
vsliveshare-audio	ms-	0.1.59
vsliveshare-pack	ms-	0.3.2
team	ms-	1.149.2

(2 theme extensions excluded)

[DonJayamanne]

Note: This is not an extension issue, we're sending the right information to the Debug Adapter.

[mahmoudajawad]

For reference, I created #77050 but was refused.

[mahmoudajawad]

Any luck on this? @DonJayamanne, since you confirmed this is not python extension issue do you have any workaround?

[weinand]

Sorry, but I don't understand the intended behavior of the launch config from above.

In the launch config from above the "env" section defines "FW_PACKAGE" and "FW_TEST" but then later "LIMP_PACKAGE" and "LIMP_TEST" is used. How are they related?

In addition please note: the "env" section is passed to the debug extension which sets the environment for the target program. The "env" section does not affect the VS Code environment variables and the variables defined in the "env" section are not available for "${env:ENV_NAME}" variable substitution in the launch config.

[mahmoudajawad]

@weinand, you are right I wrongly copied two different versions of the launch config when I was reporting this bug, but this mistake doesn't invalidate the report.

Also there seems a confusion here. Let me try to clear it with few points:

1. env attribute of Python extension launch config has the following description:

Environment variables defined as a key value pair. Property ends up being the Environment Variable and the value of the property ends up being the value of the Env Variable.

2. Passing any pair of key and value to env is correctly being captured by the process by prefixing the debug command with the setters of these pairs. The issue has sample debug command showing the correct variables being set BEFORE running the debugger.
3. Before launching the project with debugger, the env variables value is null, however after running it (at least once) I can print the value of the environment variable from [power]shell and get the coorect value, which means the prefixed setters to the debug command are doing correct job.
4. Even if I manually set the environment variable without using env attribute, I'm still getting empty values in launch args where the environment variables like {env:ANYTHING_WOULD_NOT_WORK} would always be translated into empty string.

Point 4 is what made me sure this was a bug when I first report it, because whether I'm using env or manually setting the environment variable before launching the debugger I'm still getting no hope.

[weinand]

The "env" attribute is implemented by the Python extension. Since VS Code does know what "env" is, it does not interpret "env" in any way (and it does not change any environment variables because of it).

The Python extension sees the "env" attribute, knows its semantic and can do something with it. But whatever the Python extension does with "env" it can not influence VS Code directly.

And since VS Code performs the variable substitution before the Python extension sees "env", the variable substitution can not be affected by "env" directly.

Another confusing thing from the initial bug report is the statement:

"command was prefixed with env variables setter"

I do not really understand what this means, but please note that VS Code's variable substitution takes place before anything is passed to the terminal for execution. So when setting an env var in the terminal the "${env:LIMP_PACKAGE}" has been already processed by VS Code (and it will not be processed again).

[weinand]

Closing since this works as designed:

VS Code's variable substitution is done before the command line is executed in a command prompt/terminal. As a consequence variable substitution cannot take into account anything that is the result of running a command in the command prompt/terminal.

[mahmoudajawad]

@weinand, this makes sense now. Is this something Python extension team can manipulate or that this sequence can never be updated by an extension? IOW is this an extension bug that they are not timing env passing correctly or that vscode doesn't allow this to happen in the first place, therefore it should become a feature request?

[weinand]

@AjawadMahmoud this is neither a bug of an extension nor of VS Code. It is just a consequence of a design decision that we made for variable substitution in launch configs.