gh-129569: The function unicodedata.normalize() always returns built-in str #129570
Conversation
Hizuru3
commented
Feb 2, 2025
•
Hizuru3
commented
- Issue: The function unicodedata.normalize() should always return an instance of the built-in str type. #129569
|
All commit authors signed the Contributor License Agreement. |
|
Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool. If this change has little impact on Python users, wait for a maintainer to apply the |
corona10
added
needs backport to 3.12
|
It will change the behavior, I am not sure it should be fixed. But from a consistent view, I think that the suggestion looks good to me. And AKAIK, no documentation is written about this behavior: https://docs.python.org/3/library/unicodedata.html#unicodedata.normalize |
corona10
removed
needs backport to 3.12
There was a problem hiding this comment.
Please add a blurb entry, a What's New entry, and update the docs to indicate that a fresh copy is returned.
Optional: I would also expect that we have a fast path for exact strings where we return a reference on the object rather than calling FromObject. Otherwise we'll create new strings if the string is already normalized (unless they are interned but that's not always the case).
|
A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated. Once you have made the requested changes, please leave a comment on this pull request containing the phrase |
corona10
added
needs backport to 3.12
|
I have made the requested changes; please review again. I added a news entry via blurb_it. Note that as mentioned in the discussion forum, PyUnicode_FromObject() returns a new reference without cloning the whole string for an instance of the exact str type. It performs copying only for instances of str subclasses. So the impact of this adjustment will not be that big. If a visible performance regression is observed, I'll consider inlining PyUnicode_FromObject() by hand. As the previous behavior is considered to be a bug, does the documentation still need an update? If so, I think it would be good to open a new issue that requests to adjust the wording in the documentation. |
|
Thanks for making the requested changes! @picnixz: please review the changes made to this pull request. |
|
I have made the requested changes; please review again; I have synced the fork. |
|
Thanks for making the requested changes! @picnixz: please review the changes made to this pull request. |
|
I think I've already approved this PR. Unless conflicts are to be solved or a CI job is to be merged/updated/created, there's no need to sync the fork. |
…built-in str (pythonGH-129570) (cherry picked from commit c359fcd) Co-authored-by: Hizuru <106918920+Hizuru3@users.noreply.github.com> Co-authored-by: Victor Stinner <vstinner@python.org>
|
GH-130403 is a backport of this pull request to the 3.13 branch. |
…built-in str (pythonGH-129570) (cherry picked from commit c359fcd) Co-authored-by: Hizuru <106918920+Hizuru3@users.noreply.github.com> Co-authored-by: Victor Stinner <vstinner@python.org>
|
GH-130404 is a backport of this pull request to the 3.12 branch. |
|
Merged. Thanks for your contribution @Hizuru3. |
I disagree. I don't think that this change should not be backported to stable branches. It would be surprising that the function returns a str subclass in Python 3.13.x but return str in Python 3.13.(x+1). It can break applications relying on the current behavior. This change should only land in Python 3.14 branch (main branch). |
|
Yeah, it's up to consider it as a bug or behavior change. |
|
@serhiy-storchaka @corona10: Do you consider that this change must be backported? IMO it's a corner case and it's safer to not backport it. It can wait for Python 3.14. |
