show python requirements not found message and option to install #712

mbektas · 2023-10-02T14:40:23Z

Shows relevant error message when Python dependency packages such as jupyterlab are not found.
Provides option to copy the command to clipboard and installing the missing packages for user
Adds ability to open app logs from Settings dialog and progress views
copyable-span web component to support copying to clipboard from web views
Additional logging for shell executions and environment errors

krassowski · 2023-10-05T22:33:37Z

I started testing it, but did not get to it - I will report back tomorrow.

krassowski · 2023-10-06T08:36:14Z

So when I tried to open an environment with an older version of JupyterLab:

jupyterlab             3.4.4
jupyterlab-server      2.15.0
jupyter-server         1.18.1

I got this distressing screen:

which has no environment picker so I cannot change the environment (without restarting app). I was able to use the hamburger menu to go to settings and grab the logs using the new "Show logs" button.

source "/home/krassowski/.pyenv/versions/<sanitized-out>/bin/activate"
        python -m jupyterlab --no-browser --expose-app-in-browser --ServerApp.port=43547 --ServerApp.password="" --ServerApp.token="jlab:srvr:<sanitized out>" --LabApp.quit_button=False --JupyterApp.config_file_name="" --ContentsManager.allow_hidden=True
[2023-10-06 09:31:09.239] [debug] Server launch parameters:
  [script]: /tmp/jlab_desktopIaMVVT/launch.sh
  [options]: {<sanitized out>}
[2023-10-06 09:31:09.589] [error] Error: Reader received error. Reason: Error during socket reconnect: code = 1006, reason = ,,
[2023-10-06 09:31:09.589] [error] Connection closed by the server
[2023-10-06 09:31:09.589] [error] Connection unexpectedly disconnected
[2023-10-06 09:31:10.074] [error] Failed to start server Error: Jupyter Server process terminated before the initialization completed
    at ChildProcess.<anonymous> (/opt/JupyterLab/resources/app.asar/build/out/main/server.js:306:28)
    at ChildProcess.emit (node:events:513:28)
    at Process.onexit (node:internal/child_process:291:12)
[2023-10-06 09:31:10.359] [warn]  Showing error: [object Object]
[2023-10-06 09:31:16.357] [warn]  Showing error: [object Object]

Note: running the command manually in the environment works fine, so I am not sure what is the error. In any way, the point is that there might be something wrong with version checks for Jupyter Server if it does not work?

mbektas · 2023-10-06T14:32:34Z

@krassowski was this environment shown in your environment selection list on top right or did you browse the python path to select it? This failure is not due to version incompatibility, running the python -m jupyterlab somehow failed.

krassowski · 2023-10-06T14:37:36Z

It's both. I do not have any suggestions in the environment list auto-populated because I use pyenv so I always have to browser for path to select it (hence I wish there was auto-detection for non-conda environments). But once I select it it stays in the environment selection list forever. It used to work previously, so I guess there may be something broken.
Re-iterating, running python -m jupyterlab with exact line as above from command line works fine.

krassowski · 2023-10-06T14:45:06Z

I tried another environment which starts up fine with jupyterlab 3.6.3. My hypothesis was that it was the older JupyterLab version, so I downgraded. After downgrading the jupyterlab in the environment the picker tooltip still says that it sees 3.6.3. This is odd. How does it get jupyterlab version?

mbektas · 2023-10-06T14:48:41Z

I tried another environment which starts up fine with jupyterlab 3.6.3. My hypothesis was that it was the older JupyterLab version, so I downgraded. After downgrading the jupyterlab in the environment the picker tooltip still says that it sees 3.6.3. This is odd. How does it get jupyterlab version?

Environment metadata is updated after a rediscovery, it should update the version shown in the list after few seconds from launch.

mbektas · 2023-10-06T14:49:38Z

I tried in a conda environment with the same versions you posted above, and it worked just fine. jupyterlab >= 3.0.0 is supported

krassowski · 2023-10-06T14:50:43Z

Ok, I know how to repro it reliably. It is a conflict of versions problem - it was happening when I stated jlab from my jupyterlab dev. Because of path overshadowing and not using path isolation it was pulling JupyterLab 4.0 Python executables, but then I selected an old environment to test it and it failed with the screen above.

So this is not a typical failure scenario, but it does show that this error screen needs work (in particular it would be lovely if it had the environment selector).

krassowski · 2023-10-06T14:54:27Z

src/main/sessionwindow/sessionwindow.ts

+            const requirementInstallCmd = encodeURIComponent(
+              this._registry.getRequirementsPipInstallCommand()
+            );
+            message = `Error! Required Python packages not found in the selected environment. You can install using <copyable-span label="${requirementInstallCmd}" copied="${requirementInstallCmd}"></copyable-span> command in the selected environment.`;


Is it possible to show the detected version here>

it is possible. it is easier to just log them though.

krassowski · 2023-10-06T14:55:17Z

src/main/registry.ts

    this._requirements = [
      {
        name: 'jupyterlab',
        moduleName: 'jupyterlab',
        commands: ['--version'],
-        versionRange: new semver.Range(`>=${minJLabVersionRequired}`)
+        versionRange: new semver.Range(`>=${MIN_JLAB_VERSION_REQUIRED}`),
+        pipCommand: 'jupyterlab'


should it include >=${MIN_JLAB_VERSION_REQUIRED} as well?

it was causing installation to fail on Windows. I will take another look.

krassowski · 2023-10-06T14:56:57Z

src/main/registry.ts

@@ -446,6 +468,16 @@ export class Registry implements IRegistry, IDisposable {
    return this._requirements;
  }

+  getRequirementsPipInstallCommand(): string {


I am afraid that suggesting pip install will lead to many problems if novice conda users end up executing this in their env. Worst case scenario is having two conflicting versions with assets from one and scripts form another which is quite common problem but hard to debug.

is this an issue when an old jupyterlab is installed with conda install, then updated with pip install. I thought pip install was fine in a conda environment.

It will only work when NOT in the base environment (ideally in isolated environment) and if pip was explicitly installed. Many users fall into this trap.

Reference: https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#using-pip-in-an-environment

Of course if you install the same version in base and leaf env, one using pip and the other using conda, it does not matter. Up until the very moment one gets updated and then it breaks.

mbektas · 2023-10-15T03:27:23Z

@krassowski could you take another look as I pushed some updates addressing your comments.

krassowski · 2023-10-18T15:30:39Z

I think I am seeing an issue with PYTHONPATH. Previously I used to set it like this (simplifying) :

PYTHONPATH=/my/project/root/dir:$PYTHONPATH jlab

but this does not seem to work now? I am not 100% positive if this is due to this PR or because some setting got lost during uninstall/install sequence, but given that this PR touches python path logic it could be related.

This is kind of a deal breaker for me, but possibly not a common usage pattern. Note that configuring this in settings does not seem to work either. Is Python path overwritten at some point, or am I fooling myself?

I am going to try an older version now.

mbektas · 2023-10-19T15:40:35Z

I think I am seeing an issue with PYTHONPATH

I tested both from CLI like your example and also from the settings dialog and both worked fine. This PR doesn't make any changes to environment variable setting.

krassowski · 2023-10-19T15:41:47Z

Thank you for looking into this! Apologies for the delay with the actual review, will take another look soon.

krassowski

Works great, thank you!

A spinner, or even better logs during live installation would be a great future enhancement.

mbektas · 2023-10-25T14:39:50Z

A spinner, or even better logs during live installation would be a great future enhancement.

there is actually a subtle animation of the orange part of the logo, like heart beat. but probably too subtle to notice.

mbektas added 3 commits October 2, 2023 07:39

show python reqs not found messages and option to install

370a7ef

fix log path on Windows, log command executions

532213b

copy to clipboard, fix windows install issue

135bc0a

mbektas mentioned this pull request Oct 4, 2023

Environment selection dialog improvements #479

Open

mbektas added 2 commits October 5, 2023 09:00

handle opening projects with incompatible Python env

efccb88

update copyable-span styles

e0f592a

mbektas marked this pull request as ready for review October 5, 2023 16:25

mbektas requested a review from krassowski October 5, 2023 16:25

krassowski reviewed Oct 6, 2023

View reviewed changes

mbektas added 4 commits October 9, 2023 19:20

fix pip install on windows with version pin

f3cccb4

support quotes in windows install script

20249e4

use conda install for installing missing requirements

d61b732

log versions of mismatched packages in requirements

5cad694

check if conda env is a base env

30aff0c

krassowski approved these changes Oct 25, 2023

View reviewed changes

mbektas merged commit bfd38e2 into master Oct 25, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

show python requirements not found message and option to install #712

show python requirements not found message and option to install #712

mbektas commented Oct 2, 2023 •

edited

Loading

krassowski commented Oct 5, 2023

krassowski commented Oct 6, 2023 •

edited

Loading

mbektas commented Oct 6, 2023

krassowski commented Oct 6, 2023

krassowski commented Oct 6, 2023

mbektas commented Oct 6, 2023

mbektas commented Oct 6, 2023 •

edited

Loading

krassowski commented Oct 6, 2023

krassowski Oct 6, 2023

mbektas Oct 8, 2023

krassowski Oct 6, 2023

mbektas Oct 8, 2023

krassowski Oct 6, 2023

mbektas Oct 8, 2023

krassowski Oct 8, 2023

krassowski Oct 8, 2023

mbektas commented Oct 15, 2023

krassowski commented Oct 18, 2023

mbektas commented Oct 19, 2023 •

edited

Loading

krassowski commented Oct 19, 2023

krassowski left a comment

mbektas commented Oct 25, 2023

show python requirements not found message and option to install #712

show python requirements not found message and option to install #712

Conversation

mbektas commented Oct 2, 2023 • edited Loading

krassowski commented Oct 5, 2023

krassowski commented Oct 6, 2023 • edited Loading

mbektas commented Oct 6, 2023

krassowski commented Oct 6, 2023

krassowski commented Oct 6, 2023

mbektas commented Oct 6, 2023

mbektas commented Oct 6, 2023 • edited Loading

krassowski commented Oct 6, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mbektas commented Oct 15, 2023

krassowski commented Oct 18, 2023

mbektas commented Oct 19, 2023 • edited Loading

krassowski commented Oct 19, 2023

krassowski left a comment

Choose a reason for hiding this comment

mbektas commented Oct 25, 2023

mbektas commented Oct 2, 2023 •

edited

Loading

krassowski commented Oct 6, 2023 •

edited

Loading

mbektas commented Oct 6, 2023 •

edited

Loading

mbektas commented Oct 19, 2023 •

edited

Loading