ImportWarning re-triggered with unittest.mock.patch().__enter__

[robsdedude]

Bug report

This only happens with Python 3.11

The minimal reproducer I could come up with:

.
├── foo
│   └── __init__.py
└── tests
    └── __init__.py

./foo/__init__.py

class Bar:
    def baz(self):
        ...

./tests/__init__.py

import unittest
import unittest.mock
import warnings

import mock  # version 4.0.3


with warnings.catch_warnings():
    warnings.simplefilter("ignore", ImportWarning)

    # this import causes multiple import warnings
    # warnings.simplefilter("once")

    # <frozen importlib._bootstrap>:1049: ImportWarning: _SixMetaPathImporter.find_spec() not found; falling back to find_module()
    # <frozen importlib._bootstrap>:673: ImportWarning: _SixMetaPathImporter.exec_module() not found; falling back to load_module()

    # urllib3 vesrion 1.24.3
    from urllib3.packages.six.moves.http_client import IncompleteRead


class TestFoo(unittest.TestCase):
    def test_example(self):
        # make import warnings raise to get a stack trace
        warnings.simplefilter("error", ImportWarning)

        with mock.patch("foo.Bar.baz"):  # this is fine
            ...

        # but this causes another import warning but only if IncompleteRead is
        # imported above
        with unittest.mock.patch("foo.Bar.baz"):
            ...

Running this with python -m unittest tests, I get the following error

$ python -m unittest tests
E
======================================================================
ERROR: test_example (tests.TestFoo.test_example)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 1074, in _find_spec
AttributeError: '_SixMetaPathImporter' object has no attribute 'find_spec'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File ".../example/tests/__init__.py", line 29, in test_example
    with unittest.mock.patch("foo.Bar.baz"):
  File ".../.pyenv/versions/3.11.0/lib/python3.11/unittest/mock.py", line 1411, in __enter__
    self.target = self.getter()
                  ^^^^^^^^^^^^^
  File ".../.pyenv/versions/3.11.0/lib/python3.11/pkgutil.py", line 705, in resolve_name
    mod = importlib.import_module(s)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.pyenv/versions/3.11.0/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1206, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1140, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1076, in _find_spec
  File "<frozen importlib._bootstrap>", line 1049, in _find_spec_legacy
ImportWarning: _SixMetaPathImporter.find_spec() not found; falling back to find_module()

----------------------------------------------------------------------
Ran 1 test in 0.002s

FAILED (errors=1)

A clear and concise description of what the bug is.
Include a minimal, reproducible example (https://stackoverflow.com/help/minimal-reproducible-example), if possible.

Your environment

• CPython versions tested on:

$ python -VV
Python 3.11.0 (main, Nov  2 2022, 13:45:57) [GCC 11.3.0]

• Operating system and architecture:

Operating System: Linux Mint 21                   
          Kernel: Linux 5.15.0-53-generic
    Architecture: x86-64

In case you were wondering why I would import from urllib3.packages.six.moves.http_client import IncompleteRead: that's an indirect (over multiple hops) dependency of my actual test code. I found this line in urllib3 and figured I can reduce the reproducer for this weird issue with directly using that import.

Originally, if that matters, it's

from .packages.six.moves.http_client import (
    IncompleteRead as httplib_IncompleteRead
)

in urllib3/exceptions.py.

I'm aware that there is an external library involved, but seeing that the stack trace goes through some C extensions in importlib, I suspect this, while being triggered with the external lib, has it's root-cause somewhere else.

[tirkarthi]

Does updating urllib3 fix this warning? The fix seems to be in urllib3 1.26.x version.

See : urllib3/urllib3#2238

[robsdedude]

Updating urllib3 to 1.24.3 mitigates the issue indeed. But I'm still confused how user code (urllib3 together with mock in this case) is even able to trigger the warning that should be ignored.

Alright, I now got what's going on under the hood. The old urllib3 injects a custom meta path finder into sys.meta_path. And that meta path finder is triggering an ImportWarning (in it's code, not mine) when importlib.import_module("foo") is called by mock (or when called manually in tests/__init__.py). Interestingly enough there is no warning when adding import foo to tests/__init__.py.

Sorry for the noise. The source of the warning was just very hard to track down because of the meta path magic.