Skip to content

Replace validate_stubs scripts with stubtest #371

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Open
wants to merge 12 commits into
base: main
Choose a base branch
from

Conversation

Avasam
Copy link
Contributor

@Avasam Avasam commented May 23, 2025

validate_stubs.py doesn't really work in its current state. I'm sure there's plenty more issues, but mainly it tries to import stub files and stubs-only symbols.

Stubtest is as actively maintained as mypy and is a core tool to typeshed's own testing to validate that stubs match the content of implementation.
https://mypy.readthedocs.io/en/stable/stubtest.html

  • Updated .github/workflows/test.yml to test both oldest and newest python version. This will help uncover issues like numpy.distutils not being available on Python 3.12+
  • In testing dependency group, explicitly install the libraries we're stubbing. This is helpful to quickly see the implementation using Pylance. And required for stubtest.
  • All found discrepancy between current stubs and runtime have been added to an allowlist
  • Issues that couldn't be ignored have been resolved
  • Removed utils/validate_stubs.py in favor of tests/run_stubtest.py

Whilst all other tools help you ensure your stubs are valid, sane, or simply follow modern standards. This is actually the best metric you'll have to see how close the stubs are to actual implementation (or rather, how much they've diverged over time)

@Avasam Avasam force-pushed the Replace-validate_stubs-scripts-with-stubtest branch from c700e37 to dbf819b Compare May 23, 2025 04:58
@Avasam
Copy link
Contributor Author

Avasam commented May 23, 2025

@debonte If you prefer, I can split off the changes to test across different Python versions

@debonte
Copy link
Contributor

debonte commented May 23, 2025

@debonte If you prefer, I can split off the changes to test across different Python versions

@Avasam, I don't have a preference. If it makes things easier for you feel free, but I'm ok with the PR as is.

@Avasam
Copy link
Contributor Author

Avasam commented May 23, 2025

I think I'm through all the issues anyway ^^
And I had to fix some of these locally either way since I'm running on a Python 3.13 .venv

@Avasam
Copy link
Contributor Author

Avasam commented May 24, 2025

As this PR grows, I ended up splitting it off after all: #375

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants