[pydocstyle] Improve heuristics for detecting Google-style docstrings
#13142
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
AlexWaygood
added
bug
|
| code | total | + violation | - violation | + fix | - fix |
|---|---|---|---|---|---|
| D407 | 983 | 0 | 983 | 0 | 0 |
| D406 | 164 | 0 | 164 | 0 | 0 |
Linter (preview)
ℹ️ ecosystem check detected linter changes. (+0 -1147 violations, +0 -0 fixes in 4 projects; 50 projects unchanged)
apache/airflow (+0 -34 violations, +0 -0 fixes)
ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview --select ALL
- airflow/decorators/base.py:137:5: D407 [*] Missing dashed underline after section ("Example") - airflow/hooks/filesystem.py:32:5: D407 [*] Missing dashed underline after section ("example") - airflow/models/dag.py:3909:5: D407 [*] Missing dashed underline after section ("Args") - airflow/providers/amazon/aws/hooks/s3.py:1401:9: D407 [*] Missing dashed underline after section ("Note") - airflow/providers/amazon/aws/operators/base_aws.py:40:5: D406 [*] Section name should end with a newline ("Examples") - airflow/providers/amazon/aws/operators/base_aws.py:40:5: D407 [*] Missing dashed underline after section ("Examples") - airflow/providers/amazon/aws/sensors/base_aws.py:40:5: D406 [*] Section name should end with a newline ("Examples") - airflow/providers/amazon/aws/sensors/base_aws.py:40:5: D407 [*] Missing dashed underline after section ("Examples") ... 15 additional changes omitted for rule D407 - airflow/providers/amazon/aws/utils/mixins.py:69:9: D406 [*] Section name should end with a newline ("Examples") - airflow/providers/apache/hive/hooks/hive.py:840:5: D406 [*] Section name should end with a newline ("Notes") ... 24 additional changes omitted for project
apache/superset (+0 -15 violations, +0 -0 fixes)
ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview --select ALL
- scripts/cancel_github_workflows.py:21:1: D407 [*] Missing dashed underline after section ("Example") - scripts/cancel_github_workflows.py:68:5: D406 [*] Section name should end with a newline ("Returns") - scripts/cancel_github_workflows.py:68:5: D407 [*] Missing dashed underline after section ("Returns") - superset/cli/test_db.py:154:5: D407 [*] Missing dashed underline after section ("TODO") - superset/commands/chart/importers/v1/utils.py:35:5: D407 [*] Missing dashed underline after section ("TODO") - superset/commands/sql_lab/query.py:39:5: D406 [*] Section name should end with a newline ("Attributes") - superset/commands/sql_lab/query.py:39:5: D407 [*] Missing dashed underline after section ("Attributes") - superset/daos/tag.py:312:9: D407 [*] Missing dashed underline after section ("Args") ... 7 additional changes omitted for rule D407 - superset/db_engine_specs/base.py:194:5: D406 [*] Section name should end with a newline ("Attributes") ... 6 additional changes omitted for project
bokeh/bokeh (+0 -1081 violations, +0 -0 fixes)
ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview --select ALL
- src/bokeh/__init__.py:65:5: D406 [*] Section name should end with a newline ("Returns") - src/bokeh/__init__.py:65:5: D407 [*] Missing dashed underline after section ("Returns") - src/bokeh/application/application.py:157:9: D407 [*] Missing dashed underline after section ("Args") - src/bokeh/application/application.py:253:9: D407 [*] Missing dashed underline after section ("Args") - src/bokeh/application/application.py:256:9: D407 [*] Missing dashed underline after section ("Returns") - src/bokeh/application/application.py:340:9: D407 [*] Missing dashed underline after section ("Args") - src/bokeh/application/application.py:344:9: D407 [*] Missing dashed underline after section ("Returns") - src/bokeh/application/application.py:91:9: D407 [*] Missing dashed underline after section ("Args") - src/bokeh/application/application.py:95:9: D407 [*] Missing dashed underline after section ("Keyword Args") - src/bokeh/application/handlers/code.py:139:9: D407 [*] Missing dashed underline after section ("Args") - src/bokeh/application/handlers/code.py:85:9: D407 [*] Missing dashed underline after section ("Args") - src/bokeh/application/handlers/code_runner.py:171:9: D406 [*] Section name should end with a newline ("Returns") - src/bokeh/application/handlers/code_runner.py:171:9: D407 [*] Missing dashed underline after section ("Returns") - src/bokeh/application/handlers/code_runner.py:207:9: D407 [*] Missing dashed underline after section ("Args") - src/bokeh/application/handlers/code_runner.py:75:9: D407 [*] Missing dashed underline after section ("Args") - src/bokeh/application/handlers/code_runner.py:89:9: D407 [*] Missing dashed underline after section ("Raises") - src/bokeh/application/handlers/directory.py:247:9: D407 [*] Missing dashed underline after section ("Args") - src/bokeh/application/handlers/directory.py:261:9: D407 [*] Missing dashed underline after section ("Args") - src/bokeh/application/handlers/directory.py:276:9: D407 [*] Missing dashed underline after section ("Args") - src/bokeh/application/handlers/directory.py:286:9: D407 [*] Missing dashed underline after section ("Args") - src/bokeh/application/handlers/directory.py:296:9: D407 [*] Missing dashed underline after section ("Args") - src/bokeh/application/handlers/directory.py:299:9: D407 [*] Missing dashed underline after section ("Returns") - src/bokeh/application/handlers/function.py:94:9: D407 [*] Missing dashed underline after section ("Args") - src/bokeh/application/handlers/handler.py:134:9: D407 [*] Missing dashed underline after section ("Args") - src/bokeh/application/handlers/handler.py:137:9: D407 [*] Missing dashed underline after section ("Returns") - src/bokeh/application/handlers/handler.py:150:9: D407 [*] Missing dashed underline after section ("Args") ... 918 additional changes omitted for rule D407 - src/bokeh/client/connection.py:174:9: D406 [*] Section name should end with a newline ("Returns") - src/bokeh/client/connection.py:242:9: D406 [*] Section name should end with a newline ("Returns") - src/bokeh/client/session.py:395:9: D406 [*] Section name should end with a newline ("Returns") - src/bokeh/client/session.py:407:9: D406 [*] Section name should end with a newline ("Returns") - src/bokeh/client/session.py:473:9: D406 [*] Section name should end with a newline ("Returns") - src/bokeh/colors/color.py:175:9: D406 [*] Section name should end with a newline ("Returns") - src/bokeh/colors/color.py:187:9: D406 [*] Section name should end with a newline ("Returns") - src/bokeh/colors/color.py:199:9: D406 [*] Section name should end with a newline ("Returns") - src/bokeh/colors/color.py:238:9: D406 [*] Section name should end with a newline ("Returns") - src/bokeh/colors/color.py:317:9: D406 [*] Section name should end with a newline ("Returns") - src/bokeh/colors/color.py:332:9: D406 [*] Section name should end with a newline ("Returns") - src/bokeh/colors/color.py:345:9: D406 [*] Section name should end with a newline ("Returns") - src/bokeh/colors/color.py:355:9: D406 [*] Section name should end with a newline ("Returns") - src/bokeh/colors/color.py:406:9: D406 [*] Section name should end with a newline ("Returns") - src/bokeh/colors/color.py:456:9: D406 [*] Section name should end with a newline ("Returns") - src/bokeh/colors/color.py:468:9: D406 [*] Section name should end with a newline ("Returns") - src/bokeh/colors/color.py:478:9: D406 [*] Section name should end with a newline ("Returns") - src/bokeh/command/subcommands/file_output.py:222:9: D406 [*] Section name should end with a newline ("Raises") - src/bokeh/core/has_props.py:459:9: D406 [*] Section name should end with a newline ("Returns") - src/bokeh/core/has_props.py:462:9: D406 [*] Section name should end with a newline ("Examples") - src/bokeh/core/has_props.py:529:9: D406 [*] Section name should end with a newline ("Returns") ... 1034 additional changes omitted for project
zulip/zulip (+0 -17 violations, +0 -0 fixes)
ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview --select ALL
- zerver/data_import/slack.py:158:5: D406 [*] Section name should end with a newline ("Returns") - zerver/data_import/slack.py:158:5: D407 [*] Missing dashed underline after section ("Returns") - zerver/data_import/slack.py:253:5: D406 [*] Section name should end with a newline ("Returns") - zerver/data_import/slack.py:253:5: D407 [*] Missing dashed underline after section ("Returns") - zerver/data_import/slack.py:510:5: D406 [*] Section name should end with a newline ("Returns") - zerver/data_import/slack.py:510:5: D407 [*] Missing dashed underline after section ("Returns") - zerver/data_import/slack.py:733:5: D406 [*] Section name should end with a newline ("Returns") - zerver/data_import/slack.py:733:5: D407 [*] Missing dashed underline after section ("Returns") - zerver/data_import/slack.py:876:5: D406 [*] Section name should end with a newline ("Returns") - zerver/data_import/slack.py:876:5: D407 [*] Missing dashed underline after section ("Returns") ... 7 additional changes omitted for project
Changes by rule (2 rules affected)
| code | total | + violation | - violation | + fix | - fix |
|---|---|---|---|---|---|
| D407 | 983 | 0 | 983 | 0 | 0 |
| D406 | 164 | 0 | 164 | 0 | 0 |
|
The ecosystem hits look good to me, but it's a big change... |
|
Can we quickly check which of those projects actually have |
|
charliermarsh
approved these changes
Aug 29, 2024
# for free
to join this conversation on GitHub.
Already have an account?
# to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR improves our heuristics for detecting which docstring convention users have in their docstrings. Specifically, we used to infer the following docstring as being a numpy-style docstring, when it's clearly a Google-style docstring (because the section title ends with a colon, and does not have an underline on the following line):
While fixing the bug, I noticed that we were also checking D407 on docstrings we infer as being Google-style, even though our docs say we don't. I fixed that as well in this PR, since otherwise the fixture I added would have led to an incorrect snapshot being added (D407 would have been emitted on the snippet, when it shouldn't have been). This D407 change resulted in a several changes to the existing snapshots, but I think they're all for the better. It looks like we no longer emit D407 on several docstrings in our fixtures that seem pretty clearly to be Google-style docstrings to me, and this rule definitely shouldn't be enforced for Google-style docstrings in my opinion.
Fixes #13139
Test Plan
cargo test -p ruff_linter --lib