Upgrade vendored requests and urllib3 #12026
Conversation
urllib3 is upgraded to 2.0.2 for thread safety. requests is upgraded to 2.30.0 for urllib3 2.x support. chardet is replaced by charset-normalizer. Brotli patch to urllib3 is removed temporarily sicne it does not apply cleanly. It will be re-applied in subsequent commits.
This removes brotli (as previously) and zstandard (added in urllib3 2.0 in a similar fashion).
uranusjr
force-pushed
the
requests-2.30-urllib3-2.0
branch
from
b7c0af6 to
6fa8356
Compare
|
Urgh, psf/cachecontrol#292 The linked issue is a 🗑️🔥, tldr: cachecontrol uses psf/cachecontrol#292 (comment)
So I think I’ll just patch it out for now. |
uranusjr
force-pushed
the
requests-2.30-urllib3-2.0
branch
from
a6f9194 to
8f99623
Compare
We can't vendor those with pip, unfortunately.
uranusjr
force-pushed
the
requests-2.30-urllib3-2.0
branch
from
22ada68 to
1f92c11
Compare
|
Apparently the network stack is completely broken now. Persumably from some thing I patched? Not sure exactly what. |
|
As @nateprewitt said here, upgrading to v2.0 of urllib3 has some restrictions that pip may not want to take on yet (specifically we only support OpenSSL 1.1.1+, not LibreSSL or other flavors). I believe the fix that spurred this change has been backported to 1.26.x so should be available to pip even if you don't upgrade to v2.0. Let me know if you have questions! :) |
This comment was marked as outdated.
This comment was marked as outdated.
|
@uranusjr given the limitations re supported ssl libs do you still want to target updating urllib3 to 2.x now? |
|
No, I just haven’t had the time to get back to this. I still want to move to chardet-normalizer at some point though. Also, for reference in the future, what do y’all think would be a good time to upgrade to urllib3? |
Personally, I would say whenever one of the maintainers has the time to do the upgrade (or review a submitted PR) and keep an eye on things post-merge in case of issues. But I’d say it should be done in the first half of a release cycle, so it’s settled in by the time of the release. |
|
Not sure if you were asking in terms of when in the release cycle or when it makes sense to jump to the 2.x branch? On the 2.x branch, 2.x requires openssl 1.1.1+ , pep-644 is python 3.10+. So my possibly overly conservative leaning would be pip should stay on urllib3<2 until support for python 3.9 is dropped? |
Since it’s not practical to vendor two versions of urllib3, I guess that’s the most reasonable timeline. See ya’ll in 2025! |
urllib3 is upgraded to 2.0.2 for thread safety.
requests is upgraded to 2.30.0 for urllib3 2.x support.
chardet is replaced by charset-normalizer.
Brotli patch to urllib3 has been re-generated since the old one does not apply cleanly anymore. urllib3 2.0 also added detection for zstandard, which I also removed since it essentially has the same issue as brotli.
Since chardet is replaced, all its usages are now replaced by charset-normalizer. This is done for pygments, which is trivial (just one line), and requests, which is much more involved. I split the previous requests patch into two to better keep track of this.
This should unblock #10480.