Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

stubtest_stdlib: get rid of --ignore-missing-stub #6491

Merged
merged 4 commits into from
Dec 5, 2021

Conversation

Akuli
Copy link
Collaborator

@Akuli Akuli commented Dec 4, 2021

Fixes #6401.

Akuli added 3 commits December 4, 2021 22:33
import re

platforms = ["linux", "win32", "darwin"]
versions = ["py36", "py37", "py38", "py39", "py310"]

entries_by_pv = {}
for p in platforms:
    for v in versions:
        p_name = {"linux": "ubuntu", "darwin": "macos", "win32": "windows"}[p]
        v_name = "3." + v.replace("py3", "")
        if v_name == "3.9":
            v_name = "3.9.7"

        entries = set()
        with open(f"la/Check stdlib with stubtest ({p_name}-latest, {v_name})/6_Run stubtest.txt") as file:
            for line in file:
                m = re.search(r"error: (.*) is not present in stub$", line.strip())
                if m:
                    entries.add(m.group(1))
        entries_by_pv[p, v] = entries

def remove_intersection(sets):
    sets = list(sets)
    result = set(sets[0])
    for s in sets[1:]:
        result &= s
    for s in sets:
        for r in result:
            s.remove(r)
    return result

common_to_all = remove_intersection(entries_by_pv.values())

common_to_version = {}
for v in versions:
    common_to_version[v] = remove_intersection([
        entries
        for (p, v2), entries in entries_by_pv.items()
        if v == v2
    ])

common_to_platform = {}
for p in platforms:
    common_to_platform[p] = remove_intersection([
        entries
        for (p2, v), entries in entries_by_pv.items()
        if p == p2
    ])

def write(fname, entries):
    with open(f"tests/stubtest_allowlists/{fname}.txt", "a") as file:
        file.write("\n# Exists at runtime, but missing from stubs\n")
        for i in sorted(entries):
            file.write(i + "\n")

write("py3_common", common_to_all)
for v, entries in common_to_version.items():
    write(v, entries)
for p, entries in common_to_platform.items():
    write(p, entries)
for (p, v), entries in entries_by_pv.items():
    write(p + "-" + v, entries)
@Akuli Akuli marked this pull request as ready for review December 4, 2021 21:03
@Akuli
Copy link
Collaborator Author

Akuli commented Dec 4, 2021

This is easiest to review commit by commit.

There are a lot of new entries, but I don't think combining them with wildcards, e.g. distutils..+, is a good idea, because that would hide all errors from distutils, not just the errors where something is missing from stubs.

Copy link
Member

@AlexWaygood AlexWaygood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this looks great! A few thoughts:

rlcompleter.Completer.global_matches
sunau.Au_read.initfp
sunau.Au_write.initfp
threading.Event.isSet
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a deprecated alias to another method (https://docs.python.org/3/library/threading.html#threading.Event.is_set), so shouldn't ever be added to the stub in my opinion -- I'd put it in a "Won't fix" section

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

threading has other aliases too. We can remove them, or add isSet, but either way it's beyond the scope of this PR.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair!

@github-actions
Copy link
Contributor

github-actions bot commented Dec 4, 2021

According to mypy_primer, this change has no effect on the checked open source code. 🤖🎉

@JelleZijlstra JelleZijlstra merged commit 24afb53 into python:master Dec 5, 2021
@Akuli Akuli deleted the stdlib-missing-stub branch December 5, 2021 09:48
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

stubtest: disable --ignore-missing-stubs for stdlib?
3 participants