Python virtual environment detection?

[ianyepan]

I'm on macOS Catalina with Emacs 27.0.50, using pyls as my lsp-server.
After I activate the venv I created in the terminal, when I open a python file in emacs, it doesn't automatically detect the virtual environment I'm supposed to be in. I noticed this simply by testing the django-related completions, which is only installed in the venv and not globally.
A workaround would be that I restart emacs using the same terminal instance that I activated the venv with, e.g.

$ source venv/bin/activate 
$ emacs

Is there a way to do this automatically without having to specifically start emacs from that specific CLI?

[yyoncho]

Not a python expert, but isn't this what you are looking for http://www.kotaweaver.com/blog/emacs-python-lsp/ ? If not, please provide steps to run + sample project + expected behaviour so that they are understandable by someone with no python experience.

[ianyepan]

@yyoncho I just read through the blog, and it appears that the thing I'm trying to achieve is much simpler than what the author was doing. All I really wanted was emacs python lsp-mode to be able to automatically choose the virtual environment python interpreter for the current project. If not automatically, a manual python interpreter selection would be fine too. Much like the "Python: Select Interpreter" option in visual studio code.

To reproduce a sample project for context purposes:

$ cd My_Cool_App

$ which python
/usr/bin/python

$ virtualenv cool_venv

$ source cool_venv/bin/activate

(cool_venv) $ which python
/Users/xyz/abc/My_Cool_App/cool_venv/bin/python

(cool_venv) $ pip install django
...Successfully installed

Then back in emacs, I open a python file in that folder. The django module, which I installed locally in this activated virtual environment (cool_venv) seems to be invisible to the lsp-server. And no django-related completions were provided.

However, the workaround I mentioned to force lsp-server to select the virtual env python interpreter is to launch emacs "right here in this CLI instance":

(cool_venv) $ emacs

To restate my original question: does python lsp-mode have the ability to either automatically detect, or at least manually select, a python interpreter for a python file/project without us having to restart emacs in the virtual-env-activated-CLI?

[yyoncho]

ATM AFAIK the users were installing pyls in the venv and then they were pointing lsp-mode to it via dir-local lsp-pyls-server-command

[ianyepan]

I uninstalled the global installation of python-language-server and installed the server only in my venv.

When I revisit the file, emacs cannot find the server: "No LSP server found for python-mode"

You mentioned "dir-local lsp-pyls-server-command", can you please elaborate on what those are? Maybe that'll solve my problem

[yyoncho]

You could use (setq lsp-pyls-server-command "path to pyls in your env"). And if you want to have it per project refer to https://www.gnu.org/software/emacs/manual/html_node/emacs/Directory-Variables.html and lsp-mode faq on how to make it work.

[ianyepan]

@yyoncho Thanks for the reply.
I've actually got another workaround that's good enough from another answer on Reddit.
For others who are wondering, I ended up using the pyvenv package, and use M-x pyvenv-activate to interactively select a virtual environment folder, then I navigate to a relevant python file and lsp-mode will pick up the venv automatically. If the python buffer was already open before I do M-x pyvenv-activate, simply restart the lsp workspace. The whole emacs instance is now put into the virtual env "state", until you M-x pyvenv-deactivate.
Since we're targeting the entire venv folder, the python interpreter is also automatically targeted and selected, as well as the other locally pip installed packages.

Here's an example config:

(use-package pyvenv
  :diminish
  :config
  (setq pyvenv-mode-line-indicator
        '(pyvenv-virtual-env-name ("[venv:" pyvenv-virtual-env-name "] ")))
  (pyvenv-mode +1))

[jakob1379]

Using a .python-version created with pyenv local this pyvenv setup auto detects the correct virtualenv, enables it before calling lsp which will then use the correct virutalenv.

;; Use pyvenv package for managing Python virtual environments
(use-package pyvenv
  :ensure t
  :init
  (setenv "WORKON_HOME" "~/.pyenv/versions")
  :hook (python-mode . my-python-mode-hook)
  :config

  ;; Custom Python mode hook
  (defun my-python-mode-hook ()
    ;; Locating the virtual environment file (.python-version) in the directory tree
    ;; and extracting its contents to use for setting up the virtual environment
    (let* ((root (locate-dominating-file
                  (or (buffer-file-name) default-directory)
                  ".python-version"))
           (venv (and root
                      (with-temp-buffer
                        (insert-file-contents
                         (expand-file-name ".python-version" root))
                        (string-trim (buffer-string))))))
      ;; If a virtual environment is found, enable pyvenv mode,
      ;; activate the virtual environment and restart lsp workspace
      (when venv
        (pyvenv-mode 1)
        (pyvenv-workon venv)
        (lsp-restart-workspace)))))