[release 0.780] Source file found twice under different module names error.

[ayushr2]

Maintainer edit from 2023

Most likely, you want to run mypy via mypy -p package instead of mypy package/.

To investigate your situation further, see https://mypy.readthedocs.io/en/stable/running_mypy.html#mapping-file-paths-to-modules

I don't recommend reading this thread, there's a lot of out of date information in here.

---

After yesterday's release 0.780, our CI starting failing with the message Source file found twice under different module names: 'grader.testlib.framework' and 'testlib.framework' but we only have one file named framework.py in grader/testlib/framework.py. This error does not show up in version 0.770.

[emmatyping]

How are you invoking mypy, and what does your directory structure look like? Also, please share your config file/command line flags if possible.

[ayushr2]

Directory Structure:

/grader
              /testlib
                         /components
                         framework.py
              /testers
              .mypy.ini
              ...

Invocation:
mypy grader/ --config-file grader/.mypy.ini

Config .mypy.ini

# Global options:

[mypy]
python_version = 3.6
mypy_path = grader

[mypy-pytest]
ignore_missing_imports = True

[roitk]

We're running into the same problem. I thought it might have been caused by mypy-protobuf, but it doesn't look like the OP was using that. I posted some details in the above issue, but here are some more:

Error message:

buildgrid/_protos/build/bazel/semver/semver_pb2.pyi: error: Source file found twice under different module names: 'buildgrid._protos.build.bazel.semver.semver_pb2' and 'build.bazel.semver.semver_pb2'
Found 1 error in 1 file (checked 144 source files)

Directory structure (all files outside of buildgrid/_protos omitted)

.
├── buildgrid
│   ├── _app
│   │   ├── bots
│   │   ├── commands
│   │   └── settings
│   ├── _protos
│   │   ├── __init__.py
│   │   ├── build
│   │   │   ├── __init__.py
│   │   │   └── bazel
│   │   │       ├── __init__.py
│   │   │       ├── remote
│   │   │       │   ├── __init__.py
│   │   │       │   └── execution
│   │   │       │       ├── __init__.py
│   │   │       │       └── v2
│   │   │       │           ├── __init__.py
│   │   │       │           ├── remote_execution.proto
│   │   │       │           ├── remote_execution_pb2.py
│   │   │       │           ├── remote_execution_pb2.pyi
│   │   │       │           └── remote_execution_pb2_grpc.py
│   │   │       └── semver
│   │   │           ├── __init__.py
│   │   │           ├── semver.proto
│   │   │           ├── semver_pb2.py
│   │   │           ├── semver_pb2.pyi
│   │   │           └── semver_pb2_grpc.py
│   │   ├── buildgrid
│   │   │   ├── __init__.py
│   │   │   └── v2
│   │   │       ├── __init__.py
│   │   │       ├── monitoring.proto
│   │   │       ├── monitoring_pb2.py
│   │   │       ├── monitoring_pb2.pyi
│   │   │       └── monitoring_pb2_grpc.py
│   │   ├── buildstream
│   │   │   ├── __init__.py
│   │   │   └── v2
│   │   │       ├── __init__.py
│   │   │       ├── buildstream.proto
│   │   │       ├── buildstream_pb2.py
│   │   │       ├── buildstream_pb2.pyi
│   │   │       └── buildstream_pb2_grpc.py
│   │   └── google
│   │       ├── __init__.py
│   │       ├── api
│   │       │   ├── __init__.py
│   │       │   ├── annotations.proto
│   │       │   ├── annotations_pb2.py
│   │       │   ├── annotations_pb2.pyi
│   │       │   ├── annotations_pb2_grpc.py
│   │       │   ├── http.proto
│   │       │   ├── http_pb2.py
│   │       │   ├── http_pb2.pyi
│   │       │   └── http_pb2_grpc.py
│   │       ├── bytestream
│   │       │   ├── __init__.py
│   │       │   ├── bytestream.proto
│   │       │   ├── bytestream_pb2.py
│   │       │   ├── bytestream_pb2.pyi
│   │       │   └── bytestream_pb2_grpc.py
│   │       ├── devtools
│   │       │   ├── __init__.py
│   │       │   └── remoteworkers
│   │       │       ├── __init__.py
│   │       │       └── v1test2
│   │       │           ├── __init__.py
│   │       │           ├── artman_remoteworkers.yaml
│   │       │           ├── bots.proto
│   │       │           ├── bots_pb2.py
│   │       │           ├── bots_pb2.pyi
│   │       │           ├── bots_pb2_grpc.py
│   │       │           ├── command.proto
│   │       │           ├── command_pb2.py
│   │       │           ├── command_pb2.pyi
│   │       │           ├── command_pb2_grpc.py
│   │       │           ├── remote_workers.yaml
│   │       │           ├── remoteworkers_gapic.yaml
│   │       │           ├── tasks.proto
│   │       │           ├── tasks_pb2.py
│   │       │           ├── tasks_pb2.pyi
│   │       │           ├── tasks_pb2_grpc.py
│   │       │           ├── worker.proto
│   │       │           ├── worker_pb2.py
│   │       │           ├── worker_pb2.pyi
│   │       │           └── worker_pb2_grpc.py
│   │       ├── longrunning
│   │       │   ├── __init__.py
│   │       │   ├── operations.proto
│   │       │   ├── operations_pb2.py
│   │       │   ├── operations_pb2.pyi
│   │       │   └── operations_pb2_grpc.py
│   │       └── rpc
│   │           ├── __init__.py
│   │           ├── code.proto
│   │           ├── code_pb2.py
│   │           ├── code_pb2.pyi
│   │           ├── code_pb2_grpc.py
│   │           ├── status.proto
│   │           ├── status_pb2.py
│   │           ├── status_pb2.pyi
│   │           └── status_pb2_grpc.py
│   ├── bot
│   │   └── hardware
│   ├── cleanup
│   ├── client
│   └── server
│       ├── actioncache
│       ├── bots
│       ├── capabilities
│       ├── cas
│       │   └── storage
│       │       └── index
│       │           └── sql_dialect_delegates
│       ├── execution
│       ├── operations
│       ├── persistence
│       │   ├── mem
│       │   └── sql
│       │       └── alembic
│       │           └── versions
│       └── referencestorage
├── data
│   ├── bin
│   └── config
│       └── grafana
│           ├── dashboards
│           │   └── buildgrid
│           └── datasources
├── dockerfile-scripts
├── docs
│   └── source
│       ├── data
│       ├── developer
│       └── user
└── tests
    ├── auth
    │   └── data
    ├── cas
    │   ├── data
    │   │   └── hello
    │   │       ├── docs
    │   │       │   └── reference
    │   │       └── utils
    │   └── index
    ├── integration
    ├── persistence
    ├── stress-testing-dockerfiles
    │   └── mnt
    │       ├── configs
    │       └── scripts
    └── utils

mypy.ini

[mypy]
plugins = sqlmypy
mypy_path = buildgrid/_protos

[mypy-botocore.*]
ignore_missing_imports = True

[mypy-alembic.*]
ignore_missing_imports = True

[mypy-grpc_reflection.*]
ignore_missing_imports = True

[mypy-janus.*]
ignore_missing_imports = True

mypy invocation (run from the root of the repository):

mypy buildgrid/

[emmatyping]

@ayushr2 @roitk it seems that mypy is now flagging those files as duplicates because they are both searched by being in the MYPYPATH and the sources list. This seems correct to me. This may have to do with a change in how mypy searches namespace packages but I am not sure.

[JelleZijlstra]

I ran into this too, even though I'm not setting $MYPYPATH. It's dependent on exactly how I invoke mypy:

jelle@devshared28:~/ans/yo$ /home/jelle/ans/yo/.tox/py36/bin/python3.6 -m mypy --config-file /home/jelle/ans/yo/src/mypy.ini /home/jelle/ans/yo/src
src/__init__.py: error: Source file found twice under different module names: 'yo.src' and 'src'
Found 1 error in 1 file (checked 37 source files)
jelle@devshared28:~/ans/yo$ /home/jelle/ans/yo/.tox/py36/bin/python3.6 -m mypy --config-file /home/jelle/ans/yo/src/mypy.ini src
Success: no issues found in 37 source files

The config file doesn't have options that should be relevant, and the same error appears if I don't pass any config file.

[emmatyping]

Hm, then perhaps this is a regression after all? I think this definitely merits further investigation.

[JukkaL]

MYPYPATH (or mypy_path) should not point to a directory with an __init__.py file, as this can result in a package with an ambiguous name. Previously mypy sometimes accepted this. This could cause crashes so mypy now complains about it.

@ayushr2 If /grader/__init__.py exists, you've hit the above issue. You should perhaps point mypy_path to the parent directory of grader.

@roitk You have mypy_path that points to a directory with a __init__.py file, so mypy gets confused.

@JelleZijlstra Your issue might be different. Can you give a full example, including all relevant source files?

[hauntsaninja]

I suspect Jelle's issue is also a spurious __init__.py and that it manifests differently with relative paths because of how we crawl up to find the base directory:

mypy/mypy/find_sources.py

Line 101 in 3adb2e9

def crawl_up(self, arg: str) -> Tuple[str, str]:

[JelleZijlstra]

@hauntsaninja is correct. The directory structure in my example is as follows:

├── __init__.py
├── src
    ├── __init__.py
    ├── mypy.ini
    └── yo_bot.py

(plus numerous other files not relevant here). If I remove the outer __init__.py, the error goes away.

[wadetregaskis-linkedin]

I'm hitting this. It happens because I have to specify all the subfolders explicitly in order to get mypy to process them, since it doesn't process folders recursively (contrary to what its documentation says). e.g.

$ echo $MYPYPATH

$ mypy src test test/jobs
test/jobs/job_utils.py: error: Source file found twice under different module names: 'job_utils' and 'jobs.job_utils'  [misc]

If I turn off namespace_packages it seems to work, but it's unclear to me what the ramifications of that will be (the documentation is unclear to its practical purpose, but seems to imply this will cause some modules I import to not be recognised, and thus not have their use by my code typechecked?).

I've noticed for some time that it was double-reporting everything, implying it's been processing things twice, but it didn't originally do that - I'm not certain, but I think this was a mypy change, not something I did. Alas I don't know which version of mypy introduced that duplicate processing.

[hauntsaninja]

mypy should recursively process your directories, and in fact it definitely is doing that to some extent, since it's clearly finding job_utils.py twice. Could you confirm whether mypy src test is checking job_utils.py (perhaps by introducing a type error)? Is there a tests/__init__.py?

[httpstergeek]

I found I had this problem when upgrading from 0.770 to 0.780 in multiple projects. When I removed MYPY_PATH from the environment variable and defining mypy_path in mypy.ini as a gobal fixed the issue.

[JukkaL]

@wadetregaskis-linkedin You might be hitting #5759. Namespace package support is currently incomplete but a major update to namespace packages is being planned (#8584).

@httpstergeek It would be helpful if you can provide a simplified but complete example which reproduces your issue. It seems that things are working unexpectedly in some specific scenarios that we don't yet fully understand.

[wadetregaskis-linkedin]

@hauntsaninja, no, it definitely doesn't process directories recursively unless they're modules (i.e. it gets 'src' and all its subfolders, because 'src' has a 'foo' subfolder with an __init__.py in it, but the associated 'test' folder contains no __init__.pys, at any level). At one point I believe it did - I used to just provide 'src' and 'test' arguments - but that was a long time ago. I don't recall all the details now, but there's been quite a struggle over various different versions of mypy to get it to just process these two folders recursively.

The duplicates only arise when I specify all the subfolders explicitly, and only for the subfolders I specify (since otherwise mypy ignores them).

IIRC it also reports things twice from not just the 'test' folder but also 'src', implying that somehow merely mentioning both, and having the files in 'test' import those from 'src', somehow causes it to do a redundant pass.

[jvm3487]

@JukkaL I also hit this issue after upgrade. I think it is the same issue reported here. Here is a minimal recreate (using CentOS 7):

mkdir -p lib/dir1/dir2/dir3/dir4
touch lib/dir1/dir2/dir3/file1.py
echo "import dir1.dir2.dir3.file1 as file1" > lib/dir1/dir2/dir3/dir4/file2.py

With mypy 0.781:
MYPYPATH=lib mypy --namespace-packages --strict-optional --strict lib/dir1/dir2/dir3/*
lib/dir1/dir2/dir3/file1.py: error: Source file found twice under different module names: 'file1' and 'dir1.dir2.dir3.file1'
Found 1 error in 1 file (checked 2 source files)

With mypy 0.770:
MYPYPATH=lib mypy --namespace-packages --strict-optional --strict lib/dir1/dir2/dir3/*
Success: no issues found in 2 source files

[richtong]

I have the same issue, but just a single init.py in our main directory. If I remove the init.py it works fine.

[sobolevn]

The same happens for https://github.com/typeddjango/djangorestframework-stubs

Output:

» mypy -p rest_framework-stubs
rest_framework-stubs/request.pyi: error: Source file found twice under different module names: 'rest_framework-stubs.request' and 'rest_framework.request'  [misc]
Found 1 error in 1 file (checked 48 source files)

When I delete __init__.pyi - it works.
It also work when I downgrade to pip install 'mypy==0.770' (my current solution)
Related typeddjango/djangorestframework-stubs#75

[killthekitten]

I experienced the opposite – adding an __init__.py solved the issue. With every folder being a package, this worked fine:

mypy app/mailers app/models app/services utils

We are gradually adding typechecks to the project, and the next step was to add the app/rest folder. It didn't contain an __init__.py, which caused this:

app/rest/errors.py: error: Source file found twice under different module names: 'errors' and 'app.rest.errors'

At first I tried moving errors.py away, but then mypy complained about other files under app/rest. So, adding app/rest/__init__.py helped.

Edit: this must be #5759, as we use mypy with namespace_packages = True.

[jamesbraza]

I am hitting this too today (mypy==0.782) after converting a library to use PEP 420 native namespacing. Here's the error message I'm seeing:

namespace/package/foo/bar.py: error: Source file found twice under different module names: 'package.foo.bar' and 'namespace.package.foo.bar'  [misc]

As per @richtong 's suggestion above, removing the __init__.py in our subpackage_a directory solved the problem.

Unfortunately, that __init__.py contains important information, so deleting it isn't a viable solution in my use case.

Aside: what's interesting is this error is only triggered for one source file, even though this error could exist in many places within the library.

I am coming here for one question and one suggestion:

• Question: As a temporary workaround, how can one disable this warning on a per-module basis?
  • Placing # type: ignore or # type: ignore[misc] at the top of the namespace/package/foo/bar.py module is not desirable for me and doesn't actually suppress this message
  • I am unsure of how to add ignores for this to the config file, the below isn't working for me with setup.cfg:

[mypy-namespace.package.foo.bar]
ignore_errors = True

• Suggestion: Provide a more verbose message for this error.
  • Ex: over two more lines, enumerate where exactly the duplicate source files were found, with line numbers

[killthekitten]

@jamesbraza could you also share the command you use to run mypy?

[jamesbraza]

@killthekitten yeah I use the files section of a config file, and just invoke mypy via entering solely mypy. Currently, I have three args to files:

• .: to type check setup.py in my repo's root
• namespace: type check the actual package
• tests: type check unit tests

I noticed that if I get rid of the ., that this error actually goes away. So a new workaround for me is to no longer use files, and in my CI config just invoke mypy two times:

• mypy namespace tests
• mypy setup.py

In my opinion, the more desirable workaround would be to make this error message suppressible.

---

Update: I am no longer hitting this error, as it turns out (found out through #9341) I was improperly invoking mypy. I needed to use -p and -m as opposed to files in the config file.

Perhaps that might be the root of your problem @ayushr2 ?

[gvanrossum]

ALL: The solution is to use mypy -p <folder> instead of mypy <folder>. This is because PEP 420 support only works on import, not on files/directories passed on the command line. The -p takes a package name which is treated as an import.

[arielf]

Problem is back in 0.80.
I started getting this error after unintentionally upgrading mypy.

If I pin pypy to 0.782 the error is gone.

Unfortunately, the code in question has a complex __init__.py which looks for the library in multiple places to support various legacy envs and situations so I can't remove it. The two "different name" locations "lib/XXX.py" and "XXX.py" are actually one and the same.

Is there a simple way to suppress this particular error? Thanks!
(I searched the docs, and couldn't find anything I could simply do to suppress it).

[jakob-keller]

Problem is back in 0.80.
I started getting this error after unintentionally upgrading mypy.

If I pin pypy to 0.782 the error is gone.

Unfortunately, the code in question has a complex __init__.py which looks for the library in multiple places to support various legacy envs and situations so I can't remove it. The two "different name" locations "lib/XXX.py" and "XXX.py" are actually one and the same.

Is there a simple way to suppress this particular error? Thanks!
(I searched the docs, and couldn't find anything I could simply do to suppress it).

I experienced the same issue in a mono-repo with namespace packages, but passing --explicit-package-bases as a command line argument fixed it for me. Also check the release blog and documentation.

[hackermd]

Specifying the name of the package (full namespace) instead of the path to the directory, resolved the issue for me with mypy v0.800.

mypy --namespace-packages -p "${NAMESPACE}.${PACKAGE}"

Not ideal, but a temporary workaround.

[avatar-lavventura]

@JukkaL Than should we create __init__.py file in all the folders?

[hardikkat24]

@JukkaL I am facing the same error. I have mypy.ini in the root directory but it must have __init__.py and this cannot be removed.
What should I do?

mypy = 0.812

[chaoflow]

After seeing the same or similar issue with namespace packages with mypy==0.910, things are working with:

All good setup

layout:

src/namespace/mod.py
tests/test_mod.py

setup.cfg:

[mypy]
explicit_package_bases = True
files = src,tests
mypy_path = $MYPY_CONFIG_FILE_DIR/src
namespace_packages = True

in project root:

mypy
mypy src tests

Error cases

No namespace_packages = True or no mypy_path:

tepsts/test_mod.py:5: error: Skipping analyzing "namespace.mod": found module but no type hints or library stubs

No explicit_package_bases = True:

tests/test_mod.py: error: Source file found twice under different module names: "mod" and "namespace.mod"

As @jakob-keller already pointed out, the explicit_package_bases is essential.

[mariusnita]

I'm seeing this error because I have a module foo/bar/select.py and mypy raises this error when run in the bar directory:

select.py: error: Source file found twice under different module names: "foo.bar.select" and "select"

(Note that mypy was straight up crashing before this, because internally it does import select and that would attempt to import the local select.py. But I think this is an issue with Python, not mypy. I worked around that issue by wrapping mypy with some code that excludes the current directory from sys.path.)

This error occurs when you have any module in the current directory that is named the same as a Python builtin module, such as csv.py.

I tried the previous commenter's suggestion to use

explicit_package_bases = True
namespace_packages = True

but that spits out a slew of errors error: Relative import climbs too many namespaces, for basically every single file.

[hauntsaninja]

These docs explain the tools you have to solve your problem: https://mypy.readthedocs.io/en/stable/running_mypy.html#mapping-file-paths-to-modules

[atos-ghassen-bchir]

ALL: The solution is to use mypy -p <folder> instead of mypy <folder>. This is because PEP 420 support only works on import, not on files/directories passed on the command line. The -p takes a package name which is treated as an import.

I am facing the same issue (v0.812). However I don't see how the "-p" option can be provided if I am using a mypy.ini file. The "files=" option provides the directories to inspect, but it does not allow for a packages-related configuration item.

In my case, removing a (useful) init.py file did the trick. However, that file being necessary (as it used in a typing_stubs folder, over the structure of which I have no control), I have to put it back and figure out another solution...

[atos-ghassen-bchir]

Answering my own previous comment: there is no such equivalent.
As per #10728, it is a requested feature, but not implemented yet. To be continued!
I had to downgrade mypy on my side

[Hi-Angel]

ALL: The solution is to use mypy -p <folder> instead of mypy <folder>. This is because PEP 420 support only works on import, not on files/directories passed on the command line. The -p takes a package name which is treated as an import.

Idk if I'm missing something obvious, but mypy refuses to even try to type-check that way:

$ mypy -p .
Can't find package '.'
$ mypy -p ./
Package name './' cannot have a slash in it.

This answer suggests one have to create a __init__.py file, but that changed nothing.

[hauntsaninja]

-p takes a package name and . is not a valid package name, you want something more like mypy -p numpy. Anyway, I'm closing and locking this issue since there's a lot of out of date stuff here causing confusion.

For most people encountering this issue, your problems are likely solvable by reading https://mypy.readthedocs.io/en/stable/running_mypy.html#mapping-file-paths-to-modules or https://mypy.readthedocs.io/en/stable/command_line.html#cmdoption-mypy-exclude

https://gitter.im/python/typing is a good place to get help and you can always open a new issue.