Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

utils: Ignore path errors when importing plugins for i18n #11205

Merged
merged 1 commit into from
Sep 13, 2023

Conversation

pwithnall
Copy link
Contributor

Fixes:

$ kolibri plugin list
INFO     2023-08-22 13:19:35,023 Option DEBUG in section [Server] being overridden by environment variable KOLIBRI_DEBUG
INFO     2023-08-22 13:19:35,023 Option DEBUG_LOG_DATABASE in section [Server] being overridden by environment variable KOLIBRI_DEBUG_LOG_DATABASE
kolibri_explore_plugin
<module 'kolibri_explore_plugin' (<_frozen_importlib_external.NamespaceLoader object at 0x7fb4ccdac1d0>)>
None
Traceback (most recent call last):
  File "/home/philip/.local/bin/pyenv.git/versions/kolibri-py3.9/bin/kolibri", line 33, in <module>
    sys.exit(load_entry_point('kolibri', 'console_scripts', 'kolibri')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/philip/.local/bin/pyenv.git/versions/kolibri-py3.9/lib/python3.11/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/philip/.local/bin/pyenv.git/versions/kolibri-py3.9/lib/python3.11/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/home/philip/.local/bin/pyenv.git/versions/kolibri-py3.9/lib/python3.11/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/philip/Projects/kolibri/kolibri/utils/cli.py", line 172, in invoke
    return super(KolibriGroupCommand, self).invoke(ctx)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/philip/.local/bin/pyenv.git/versions/kolibri-py3.9/lib/python3.11/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/philip/.local/bin/pyenv.git/versions/kolibri-py3.9/lib/python3.11/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/philip/.local/bin/pyenv.git/versions/kolibri-py3.9/lib/python3.11/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/philip/Projects/kolibri/kolibri/utils/cli.py", line 453, in list
    max_name_len = max((len(plugin.name(lang)) for plugin in plugins))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/philip/Projects/kolibri/kolibri/utils/cli.py", line 453, in <genexpr>
    max_name_len = max((len(plugin.name(lang)) for plugin in plugins))
                            ^^^^^^^^^^^^^^^^^
  File "/home/philip/Projects/kolibri/kolibri/plugins/coach/kolibri_plugin.py", line 28, in name
    with translation.override(lang):
  File "/home/philip/Projects/kolibri/kolibri/utils/translation.py", line 116, in __enter__
    self.old_language = get_language()
                        ^^^^^^^^^^^^^^
  File "/home/philip/Projects/kolibri/kolibri/utils/translation.py", line 58, in wrapped
    return getattr(django_translation_module, fn.__name__)(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/philip/.local/bin/pyenv.git/versions/kolibri-py3.9/lib/python3.11/site-packages/django/utils/translation/__init__.py", line 195, in get_language
    return _trans.get_language()
           ^^^^^^^^^^^^^^^^^^^
  File "/home/philip/.local/bin/pyenv.git/versions/kolibri-py3.9/lib/python3.11/site-packages/django/utils/translation/__init__.py", line 59, in __getattr__
    if settings.USE_I18N:
       ^^^^^^^^^^^^^^^^^
  File "/home/philip/.local/bin/pyenv.git/versions/kolibri-py3.9/lib/python3.11/site-packages/django/conf/__init__.py", line 56, in __getattr__
    self._setup(name)
  File "/home/philip/.local/bin/pyenv.git/versions/kolibri-py3.9/lib/python3.11/site-packages/django/conf/__init__.py", line 41, in _setup
    self._wrapped = Settings(settings_module)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/philip/.local/bin/pyenv.git/versions/kolibri-py3.9/lib/python3.11/site-packages/django/conf/__init__.py", line 110, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/philip/Projects/kolibri/kolibri/deployment/default/settings/base.py", line 422, in <module>
    apply_settings(sys.modules[__name__])
  File "/home/philip/Projects/kolibri/kolibri/plugins/utils/settings.py", line 133, in apply_settings
    _apply_base_settings(plugin_instance, settings_module)
  File "/home/philip/Projects/kolibri/kolibri/plugins/utils/settings.py", line 106, in _apply_base_settings
    ) and i18n.get_installed_app_locale_path(plugin_instance.module_path):
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/philip/Projects/kolibri/kolibri/utils/i18n.py", line 24, in get_installed_app_locale_path
    module_path = os.path.dirname(m.__file__)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen posixpath>", line 152, in dirname
TypeError: expected str, bytes or os.PathLike object, not NoneType

Summary

References

Reviewer guidance


Testing checklist

  • Contributor has fully tested the PR manually
  • If there are any front-end changes, before/after screenshots are included
  • Critical user journeys are covered by Gherkin stories
  • Critical and brittle code paths are covered by unit tests

PR process

  • PR has the correct target branch and milestone
  • PR has 'needs review' or 'work-in-progress' label
  • If PR is ready for review, a reviewer has been added. (Don't use 'Assignees')
  • If this is an important user-facing change, PR or related issue has a 'changelog' label
  • If this includes an internal dependency change, a link to the diff is provided

Reviewer checklist

  • Automated test coverage is satisfactory
  • PR is fully functional
  • PR has been tested for accessibility regressions
  • External dependency files were updated if necessary (yarn and pip)
  • Documentation is updated
  • Contributor is in AUTHORS.md

@github-actions github-actions bot added DEV: backend Python, databases, networking, filesystem... SIZE: very small labels Sep 7, 2023
@rtibbles
Copy link
Member

rtibbles commented Sep 7, 2023

Just out of interest, what environment is this happening in?

@rtibbles
Copy link
Member

rtibbles commented Sep 7, 2023

Also - is this something that would need to be fixed in the 0.16 release? If so, it's targeted at the wrong branch (and would also need to be rebased).

@dbnicholson
Copy link
Contributor

I think this should be targeted at release-v0.16.x. Certainly that's where we plan to use it. The context is that @pwithnall has been trying to wire up the translation infrastructure in kolibri-explore-plugin. The environment is Linux (Endless OS) where the error is happening above.

@rtibbles
Copy link
Member

OK - the thing I was finding confusing was that m.__file__ is evaluating as None and it seems to suggest it's a frozen path, which is usually only when in a PyInstaller type environment, so I was curious if there was anything unusual happening here.

@pwithnall
Copy link
Contributor Author

I can rebase it on release-v0.16.x if people want? I don’t know what the process is for backporting — the process I’m used to is that something lands on the main development branch and then is cherry-picked back to older branches (like 0.16) if needed.

@dbnicholson
Copy link
Contributor

I can rebase it on release-v0.16.x if people want? I don’t know what the process is for backporting — the process I’m used to is that something lands on the main development branch and then is cherry-picked back to older branches (like 0.16) if needed.

Yes, please do. The process is more "forward porting". The stable branches are periodically merged into develop, so you want to target the oldest branch you want it supported on.

@pwithnall
Copy link
Contributor Author

pwithnall commented Sep 12, 2023

Done! The PR labels will need updating and I don’t think I have permissions to do that.

@github-actions github-actions bot added DEV: dev-ops Continuous integration & deployment APP: Device Re: Device App (content import/export, facility-syncing, user permissions, etc.) APP: Facility Re: Facility App (user/class management, facility settings, csv import/export, etc.) APP: Learn Re: Learn App (content, quizzes, lessons, etc.) APP: Coach Re: Coach App (lessons, quizzes, groups, reports, etc.) APP: User Re: User app (sign-in, sign-up, user profile, etc.) APP: Setup Wizard Re: Setup Wizard (facility import, superuser creation, settings, etc.) SIZE: medium labels Sep 12, 2023
@pwithnall pwithnall changed the base branch from develop to release-v0.16.x September 12, 2023 17:07
@rtibbles
Copy link
Member

Any more info on the environment that this was triggered in btw?

@pwithnall
Copy link
Contributor Author

pwithnall commented Sep 12, 2023

Any more info on the environment that this was triggered in btw?

No, I’m not sure what about my environment triggered it, and I actually haven’t seen the issue for a little while. My guess would be a pip install of kolibri_explore_plugin in the same pyenv as I build/run Kolibri. As opposed to running kolibri_explore_plugin using yarn dev.

Sorry that’s probably not very helpful.

Fixes:
```
$ kolibri plugin list
INFO     2023-08-22 13:19:35,023 Option DEBUG in section [Server] being overridden by environment variable KOLIBRI_DEBUG
INFO     2023-08-22 13:19:35,023 Option DEBUG_LOG_DATABASE in section [Server] being overridden by environment variable KOLIBRI_DEBUG_LOG_DATABASE
kolibri_explore_plugin
<module 'kolibri_explore_plugin' (<_frozen_importlib_external.NamespaceLoader object at 0x7fb4ccdac1d0>)>
None
Traceback (most recent call last):
  File "/home/philip/.local/bin/pyenv.git/versions/kolibri-py3.9/bin/kolibri", line 33, in <module>
    sys.exit(load_entry_point('kolibri', 'console_scripts', 'kolibri')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/philip/.local/bin/pyenv.git/versions/kolibri-py3.9/lib/python3.11/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/philip/.local/bin/pyenv.git/versions/kolibri-py3.9/lib/python3.11/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/home/philip/.local/bin/pyenv.git/versions/kolibri-py3.9/lib/python3.11/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/philip/Projects/kolibri/kolibri/utils/cli.py", line 172, in invoke
    return super(KolibriGroupCommand, self).invoke(ctx)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/philip/.local/bin/pyenv.git/versions/kolibri-py3.9/lib/python3.11/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/philip/.local/bin/pyenv.git/versions/kolibri-py3.9/lib/python3.11/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/philip/.local/bin/pyenv.git/versions/kolibri-py3.9/lib/python3.11/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/philip/Projects/kolibri/kolibri/utils/cli.py", line 453, in list
    max_name_len = max((len(plugin.name(lang)) for plugin in plugins))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/philip/Projects/kolibri/kolibri/utils/cli.py", line 453, in <genexpr>
    max_name_len = max((len(plugin.name(lang)) for plugin in plugins))
                            ^^^^^^^^^^^^^^^^^
  File "/home/philip/Projects/kolibri/kolibri/plugins/coach/kolibri_plugin.py", line 28, in name
    with translation.override(lang):
  File "/home/philip/Projects/kolibri/kolibri/utils/translation.py", line 116, in __enter__
    self.old_language = get_language()
                        ^^^^^^^^^^^^^^
  File "/home/philip/Projects/kolibri/kolibri/utils/translation.py", line 58, in wrapped
    return getattr(django_translation_module, fn.__name__)(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/philip/.local/bin/pyenv.git/versions/kolibri-py3.9/lib/python3.11/site-packages/django/utils/translation/__init__.py", line 195, in get_language
    return _trans.get_language()
           ^^^^^^^^^^^^^^^^^^^
  File "/home/philip/.local/bin/pyenv.git/versions/kolibri-py3.9/lib/python3.11/site-packages/django/utils/translation/__init__.py", line 59, in __getattr__
    if settings.USE_I18N:
       ^^^^^^^^^^^^^^^^^
  File "/home/philip/.local/bin/pyenv.git/versions/kolibri-py3.9/lib/python3.11/site-packages/django/conf/__init__.py", line 56, in __getattr__
    self._setup(name)
  File "/home/philip/.local/bin/pyenv.git/versions/kolibri-py3.9/lib/python3.11/site-packages/django/conf/__init__.py", line 41, in _setup
    self._wrapped = Settings(settings_module)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/philip/.local/bin/pyenv.git/versions/kolibri-py3.9/lib/python3.11/site-packages/django/conf/__init__.py", line 110, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/philip/Projects/kolibri/kolibri/deployment/default/settings/base.py", line 422, in <module>
    apply_settings(sys.modules[__name__])
  File "/home/philip/Projects/kolibri/kolibri/plugins/utils/settings.py", line 133, in apply_settings
    _apply_base_settings(plugin_instance, settings_module)
  File "/home/philip/Projects/kolibri/kolibri/plugins/utils/settings.py", line 106, in _apply_base_settings
    ) and i18n.get_installed_app_locale_path(plugin_instance.module_path):
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/philip/Projects/kolibri/kolibri/utils/i18n.py", line 24, in get_installed_app_locale_path
    module_path = os.path.dirname(m.__file__)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen posixpath>", line 152, in dirname
TypeError: expected str, bytes or os.PathLike object, not NoneType
```

Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
@github-actions github-actions bot removed DEV: dev-ops Continuous integration & deployment APP: Device Re: Device App (content import/export, facility-syncing, user permissions, etc.) APP: Facility Re: Facility App (user/class management, facility settings, csv import/export, etc.) APP: Learn Re: Learn App (content, quizzes, lessons, etc.) APP: Coach Re: Coach App (lessons, quizzes, groups, reports, etc.) APP: User Re: User app (sign-in, sign-up, user profile, etc.) labels Sep 12, 2023
@github-actions github-actions bot removed the APP: Setup Wizard Re: Setup Wizard (facility import, superuser creation, settings, etc.) label Sep 12, 2023
@pwithnall
Copy link
Contributor Author

(I just pushed a small update to the commit message to re-trigger the github actions)

@rtibbles rtibbles merged commit 93c9786 into learningequality:release-v0.16.x Sep 13, 2023
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
DEV: backend Python, databases, networking, filesystem... SIZE: medium SIZE: very small
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants