Deduce model field names from custom prefetches

[diesieben07]

Description

Currently the QuerySet optimizer is oblivious to custom prefetch_related like the following:

@strawberry_django.type(models.Project)
class ProjectType(models.Project):
      @strawberry_django.field(
        prefetch_related=lambda _: Prefetch(
            "milestones",
            to_attr="next_milestones_pf",
            queryset=Milestone.objects.filter(due_date__isnull=False).order_by("due_date")
        )
    )
    def next_milestones(self) -> "list[MilestoneType]":
        return self.next_milestones_pf

The current optimizer cannot understand how to optimize "through" this prefetch, because it cannot find the model field (milestones in Project) that the field uses.
This results in the optimization essentially stopping at the next_milestones field and any further nested relations will cause N+1 issues.

This pull request attempts to address this shortcoming.

Types of Changes

Now the optimizer first inspects the field configuration to find any custom Prefetch. A custom prefetch is one which has a custom to_attr set. The optimizer then uses these custom Prefetches to perform further optimization instead of stopping here.

• Core
• Bugfix
• New feature
• Enhancement/optimization
• Documentation

Issues Fixed or Closed by This PR

• Optimiser not working with custom prefetch query #389

Checklist

• My code follows the code style of this project.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I have read the CONTRIBUTING document.
• I have added tests to cover my changes.
• I have tested the changes and verified that they work and don't break anything (as well as I can manage).

[sourcery-ai]

PR Type: Enhancement

PR Summary: The pull request introduces an enhancement to the QuerySet optimizer by enabling it to understand and optimize queries involving custom prefetch_related configurations. This is achieved by deducing model field names from custom prefetches, allowing the optimizer to continue its optimization process beyond fields with custom prefetches, thereby addressing potential N+1 issues in nested relations.

Decision: Comment

📝 Type: 'Enhancement' - not supported yet.

• Sourcery currently only approves 'Typo fix' PRs.

✅ Issue addressed: this change correctly addresses the issue or implements the desired feature.

No details provided.

✅ Small diff: the diff is small enough to approve with confidence.

No details provided.

General suggestions:

• Consider adding more inline documentation or comments explaining the rationale behind key changes, especially for complex logic around handling custom prefetches. This will aid future maintainability.
• Review the use of suppression comments like # noqa: PLW2901 to ensure their necessity is clear, possibly by adding a brief explanation next to each suppression.
• Ensure thorough testing of the new logic, particularly in scenarios with multiple custom prefetches that might lead to ambiguous model field deductions.

Thanks for using Sourcery. We offer it for free for open source projects and would be very grateful if you could help us grow. If you like it, would you consider sharing Sourcery on your favourite social media? ✨

Share Sourcery

• X
• Mastodon
• LinkedIn
• Facebook

---

Help me be more useful! Please click 👍 or 👎 on each comment to tell me if it was helpful.

[sourcery-ai]

suggestion (llm): This block introduces a significant change in how prefetches are handled based on the presence of custom_prefetches. It's a complex addition that could benefit from a bit more inline documentation to explain the rationale behind this approach, especially for future maintainers.

[diesieben07]

Some guidance on the test failures would be great. I am not 100% sure what the correct approach would be for the changed GraphQL schema.

[bellini666]

Wow, sorry for taking to long to review this. For some reason I started reviewing and stopped, had a some comments drafted...

Some guidance on the test failures would be great. I am not 100% sure what the correct approach would be for the changed GraphQL schema.

Most of the issues I'm seeing here are related to typing. Try to run pyright locally and it will help you fix those. If you need specific guidance feel free to reach me on discord :)

There are some related to features not supported on older python versions. e.g. X | Y cannot be used yet because we are targeting python 3.8+, Annotated needs to be imported from typing_extensions (it was introduced on 3.9), etc

About the schema, just run pytest with --snapshot-update and it should update the schema file :)

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 83.01887% with 9 lines in your changes are missing coverage. Please review.

Project coverage is 88.21%. Comparing base (5c999e9) to head (746ec95).
Report is 20 commits behind head on main.

Files	Patch %	Lines
strawberry_django/optimizer.py	83.01%	9 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #473      +/-   ##
==========================================
+ Coverage   87.90%   88.21%   +0.30%     
==========================================
  Files          37       37              
  Lines        3166     3215      +49     
==========================================
+ Hits         2783     2836      +53     
+ Misses        383      379       -4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[diesieben07]

@bellini666 I think all the linting and type errors should be resolved now and the snapshot test fixed.

[bellini666]

The solution don't seem to be working for django 3.2 and 4.0 (see the tests that broke)

[bellini666]

Is this a left over from a test attempt? =P

This is not being used, so it should be removed