Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

gh-104773: PEP 594: Remove cgi and cgitb modules #104775

Merged
merged 2 commits into from
May 24, 2023
Merged

gh-104773: PEP 594: Remove cgi and cgitb modules #104775

merged 2 commits into from
May 24, 2023

Conversation

vstinner
Copy link
Member

@vstinner vstinner commented May 22, 2023 •
edited
Loading

@vstinner vstinner requested a review from ethanfurman as a code owner May 22, 2023 23:50
@bedevere-bot bedevere-bot mentioned this pull request May 22, 2023
Closed
19 tasks
@vstinner
Copy link
Member Author

vstinner commented May 23, 2023 •
edited
Loading

PR updated to fix the doc: Warning: (...)/Doc/library/security_warnings.rst:12: WARNING: undefined label: cgi-security.

@vstinner vstinner changed the title gh-104773: PEP 594: Remove cgi and cgitb modules gh-104773: PEP 594: Remove cgi and cgitb modules May 23, 2023
@vstinner vstinner force-pushed the remove_cgi branch 2 times, most recently from 7a4a015 to 7cbc500 Compare May 23, 2023 00:36
@vstinner
Copy link
Member Author

Update: fix test_pyclbr.

@vstinner
Copy link
Member Author

It seems like cgi.escape() was used in Python 2. There is still compatibility code in the wild using it. Example with distlib/compat.py:

try:
    from html import escape
except ImportError:
    from cgi import escape

src: https://github.com/pypa/distlib/blob/05375908c1b2d6b0e74bdeb574569d3609db9f56/distlib/compat.py#L470-L474

@vstinner
Copy link
Member Author

Using import cgi|from cgi regex, I found a bunch of PyPI top 5,000 projects (PYPI-2023-04-13) using cgi and cgitb modules.

Affected projects (91):

  • Cheetah3-3.2.6.post1
  • CherryPy (18.8.0)
  • Cython (0.29.34)
  • FormEncode (2.0.1)
  • Mako (1.2.4)
  • PTable (0.9.2)
  • Paste (3.5.2)
  • PasteDeploy (3.0.1)
  • PasteScript (3.3.0)
  • PyVISA (1.13.0)
  • Pyro4 (4.82)
  • ShopifyAPI (12.3.0)
  • Tempita (0.5.2)
  • Twisted (22.10.0)
  • WMI (1.5.1)
  • WebHelpers (1.3)
  • WebOb (1.8.7)
  • WebTest (3.0.0)
  • Whoosh (2.7.4)
  • alexapy (1.26.5)
  • aliyun-python-sdk-core (2.13.36)
  • ansible (7.4.0)
  • asana (3.2.0)
  • astropy (5.2.2)
  • asyncio (3.4.3)
  • atlassian-jwt-auth (17.0.0)
  • boltons (23.0.0)
  • boto (2.49.0)
  • braintree (4.19.0)
  • chispa (0.9.2)
  • cinemagoer (2022.12.27)
  • conda (4.3.16)
  • coreapi (2.3.3)
  • cssutils (2.6.0)
  • devpi-client (6.0.4)
  • distlib (0.3.6)
  • docxtpl (0.16.6)
  • eli5 (0.13.0)
  • emails (0.6)
  • eventlet (0.33.3)
  • falcon (3.1.1)
  • feedparser (6.0.10)
  • fissix (21.11.13)
  • gevent (22.10.2)
  • google-apitools (0.5.32)
  • gsutil (5.23)
  • gviz_api (1.10.0)
  • htmlmin (0.1.12)
  • ibm-cos-sdk-core (2.13.0)
  • isort (5.12.0)
  • jenkinsapi (0.3.13)
  • json-ref-dict (0.7.1)
  • json2html (1.3.0)
  • jusText (3.0.0)
  • lxml (4.9.2)
  • mechanize (0.4.8)
  • mercurial (6.4.1)
  • mozlog (7.1.1)
  • nibabel (5.1.0)
  • nltk (3.8.1)
  • numpy (1.24.2)
  • oauth (1.0.1)
  • oauth2client (4.1.3)
  • odfpy (1.4.1)
  • oletools (0.60.1)
  • os_sys (2.1.4)
  • pdfminer (20191125)
  • peppercorn (0.6)
  • pex (2.1.131)
  • prov (2.0.0)
  • pyLDAvis (3.4.0)
  • pyrocko (2023.3.27)
  • pysaml2 (7.4.1)
  • pystache (0.6.0)
  • python-mimeparse (1.6.0)
  • python-openid (2.2.5)
  • python3-openid (3.2.0)
  • pyvmomi (8.0.0.1.2)
  • scapy (2.5.0)
  • sentry-sdk (1.19.1)
  • setuptools (67.6.1)
  • simplekml (1.3.6)
  • speedtest-cli (2.1.3)
  • supervisor (4.2.5)
  • tableauserverclient (0.25)
  • tqdm (4.65.0)
  • twine (4.0.2)
  • uri (2.0.1)
  • web.py (0.62)
  • webapp2 (2.5.2)
  • zeep (4.2.1)
Code: 
Cheetah3-3.2.6.post1/Cheetah/Filters.py: # These substitutions are copied from cgi.escape().
Cheetah3-3.2.6.post1/Cheetah/Template.py: import cgi  # Used by .webInput() if the template is a CGI script.
Cheetah3-3.2.6.post1/Cheetah/Tests/Regressions.py: from cgi import escape as html_escape
CherryPy-18.8.0/cherrypy/lib/covercp.py: import cgi
CherryPy-18.8.0/cherrypy/lib/httputil.py: from cgi import parse_header
Cython-0.29.34/Cython/Tempita/_tempita.py: import cgi
FormEncode-2.0.1/formencode/fieldstorage.py: import cgi
FormEncode-2.0.1/formencode/htmlgen.py: from cgi import escape
FormEncode-2.0.1/formencode/rewritingparser.py: from cgi import escape
FormEncode-2.0.1/formencode/validators.py: import cgi
Mako-1.2.4/examples/wsgi/run_wsgi.py: import cgi
PTable-0.9.2/prettytable/_compact.py: from cgi import escape
Paste-3.5.2/paste/auth/open_id.py: import cgi
Paste-3.5.2/paste/cgitb_catcher.py: import cgitb
Paste-3.5.2/paste/debug/doctest_webapp.py: import cgi
Paste-3.5.2/paste/debug/prints.py: import cgi
Paste-3.5.2/paste/debug/profile.py: import cgi
Paste-3.5.2/paste/debug/watchthreads.py: import cgi
Paste-3.5.2/paste/debug/wdg_validate.py: import cgi
Paste-3.5.2/paste/exceptions/errormiddleware.py: import cgi
Paste-3.5.2/paste/request.py: import cgi
Paste-3.5.2/paste/util/multidict.py: import cgi
Paste-3.5.2/tests/cgiapp_data/form.cgi: import cgi
Paste-3.5.2/tests/test_fixture.py: import cgi
Paste-3.5.2/tests/test_multidict.py: import cgi
Paste-3.5.2/tests/test_wsgiwrappers.py: import cgi
PasteDeploy-3.0.1/src/paste/deploy/paster_templates/paste_deploy/+package+/sampleapp.py_tmpl: import cgi
PasteScript-3.3.0/paste/script/copydir.py: import cgi as html
PasteScript-3.3.0/paste/script/exe.py: # Maybe import cgitb or something?
PasteScript-3.3.0/paste/script/testapp.py: import cgi as html
PyVISA-1.13.0/pyvisa/thirdparty/prettytable.py: from cgi import escape
Pyro4-4.82/src/Pyro4/utils/httpgateway.py: import cgi
ShopifyAPI-12.3.0/shopify/collection.py: import cgi
Tempita-0.5.2/tempita/__init__.py: import cgi
Twisted-22.10.0/docs/core/howto/tutorial/listings/finger/finger15.tac: import cgi
Twisted-22.10.0/docs/core/howto/tutorial/listings/finger/finger16.tac: import cgi
Twisted-22.10.0/docs/core/howto/tutorial/listings/finger/finger17.tac: import cgi
Twisted-22.10.0/docs/core/howto/tutorial/listings/finger/finger18.tac: import cgi
Twisted-22.10.0/docs/core/howto/tutorial/listings/finger/finger19.tac: import cgi
Twisted-22.10.0/docs/core/howto/tutorial/listings/finger/finger19a.tac: import cgi
Twisted-22.10.0/docs/core/howto/tutorial/listings/finger/finger19b.tac: import cgi
Twisted-22.10.0/docs/core/howto/tutorial/listings/finger/finger19c.tac: import cgi
Twisted-22.10.0/docs/core/howto/tutorial/listings/finger/finger20.tac: import cgi
Twisted-22.10.0/docs/core/howto/tutorial/listings/finger/finger21.tac: import cgi
Twisted-22.10.0/docs/core/howto/tutorial/listings/finger/finger22.py: import cgi
Twisted-22.10.0/src/twisted/web/http.py: import cgi
Twisted-22.10.0/src/twisted/web/test/test_cgi.py: import cgi
WMI-1.5.1/wmiweb.py: from cgi import escape
WebHelpers-1.3/webhelpers/util.py: from cgi import parse_qs
WebHelpers-1.3/webhelpers/util.py: import cgi
WebOb-1.8.7/src/webob/compat.py: from cgi import FieldStorage as _cgi_FieldStorage
WebOb-1.8.7/src/webob/compat.py: from cgi import FieldStorage as cgi_FieldStorage
WebOb-1.8.7/src/webob/compat.py: from cgi import escape
WebOb-1.8.7/src/webob/compat.py: from cgi import parse_header
WebOb-1.8.7/src/webob/compat.py: import cgi
WebOb-1.8.7/tests/test_compat.py: from webob.compat import cgi_FieldStorage
WebOb-1.8.7/tests/test_compat.py: from webob.compat import cgi_FieldStorage
WebOb-1.8.7/tests/test_compat.py: from webob.compat import cgi_FieldStorage
WebOb-1.8.7/tests/test_compat.py: from webob.compat import cgi_FieldStorage
WebOb-1.8.7/tests/test_in_wsgiref.py: import cgi
WebOb-1.8.7/tests/test_multidict.py: from cgi import FieldStorage
WebOb-1.8.7/tests/test_multidict.py: from cgi import FieldStorage
WebOb-1.8.7/tests/test_multidict.py: from cgi import FieldStorage
WebOb-1.8.7/tests/test_request.py: from cgi import FieldStorage
WebOb-1.8.7/tests/test_request.py: import cgi
WebOb-1.8.7/tests/test_request.py: import cgi
WebOb-1.8.7/tests/test_request.py: import cgi
WebTest-3.0.0/tests/test_forms.py: import cgi
Whoosh-2.7.4/src/whoosh/compat.py: import cgi
alexapy-1.26.5/alexapy/aiohttp/helpers.py: import cgi
aliyun-python-sdk-core-2.13.36/aliyunsdkcore/vendored/requests/utils.py: import cgi
ansible-7.4.0/ansible_collections/community/general/plugins/modules/campfire.py: import cgi
asana-3.2.0/tests/test_client_attachments.py: import cgi
astropy-5.2.2/cextern/cfitsio/docs/changes.txt: that even fits files returned from cgi queries which have the wrong
asyncio-3.4.3/examples/crawl.py: import cgi
atlassian-jwt-auth-17.0.0/atlassian_jwt_auth/key.py: import cgi
boltons-23.0.0/boltons/tableutils.py: from cgi import escape as html_escape
boltons-23.0.0/boltons/tableutils.py: from cgi import escape as html_escape
boto-2.49.0/boto/ecs/item.py: import cgi
braintree-4.19.0/braintree/apple_pay_gateway.py: from cgi import escape
chispa-0.9.2/chispa/prettytable.py: from cgi import escape
cinemagoer-2022.12.27/imdb/helpers.py: from cgi import escape
cinemagoer-2022.12.27/imdb/parser/http/piculet.py: from cgi import escape as html_escape
conda-4.3.16/conda/gateways/adapters/ftp.py: import cgi
coreapi-2.3.3/coreapi/codecs/download.py: import cgi
cssutils-2.6.0/cssutils/_fetchgae.py: import cgi
cssutils-2.6.0/cssutils/tests/test_util.py: import cgi
cssutils-2.6.0/encutils/__init__.py: import cgi
cssutils-2.6.0/tools/try.py: import cgi
devpi-client-6.0.4/testing/simpypi.py: from cgi import escape  # type: ignore
distlib-0.3.6/tests/test_database.py: import cgi
docxtpl-0.16.6/docxtpl/listing.py: from cgi import escape
docxtpl-0.16.6/docxtpl/richtext.py: from cgi import escape
docxtpl-0.16.6/docxtpl/template.py: from cgi import escape  # noqa: F401
eli5-0.13.0/eli5/formatters/html.py: from cgi import escape  # type: ignore
emails-0.6/emails/loader/helpers.py: import cgi
eventlet-0.33.3/tests/wsgi_test.py: import cgi
falcon-3.1.1/falcon/asgi/multipart.py: import cgi
falcon-3.1.1/falcon/media/multipart.py: import cgi
falcon-3.1.1/falcon/testing/helpers.py: import cgi
falcon-3.1.1/falcon/vendor/mimeparse/mimeparse.py: import cgi
feedparser-6.0.10/feedparser/encodings.py: import cgi
fissix-21.11.13/fissix/tests/test_util.py: self.assertEqual(str(node), "import cgi\nbar()\n\n")
gevent-22.10.2/examples/webproxy.py: from cgi import escape
gevent-22.10.2/src/greentest/2.7/test_httpservers.py: import cgi
gevent-22.10.2/src/greentest/2.7pypy/test_httpservers.py: import cgi
gevent-22.10.2/src/greentest/3.7pypy/test_httpservers.py: import cgi
google-apitools-0.5.32/apitools/base/protorpclite/test_util.py: import cgi
gsutil-5.23/gslib/vendored/boto/boto/ecs/item.py: import cgi
gsutil-5.23/gslib/vendored/oauth2client/oauth2client/contrib/appengine.py: import cgi
gviz_api-1.10.0/gviz_api.py: import cgi as html  # Only used for .escape()
htmlmin-0.1.12/htmlmin/escape.py: from cgi import escape
htmlmin-0.1.12/htmlmin/main.py: import cgi
ibm-cos-sdk-core-2.13.0/ibm_botocore/utils.py: import cgi
isort-5.12.0/tests/unit/profiles/test_django.py: """import cgi
jenkinsapi-0.3.13/examples/low_level/post_watcher.py: import cgi
jenkinsapi-0.3.13/jenkinsapi/utils/simple_post_logger.py: import cgi
json-ref-dict-0.7.1/json_ref_dict/loader.py: import cgi
json2html-1.3.0/json2html/jsonconv.py: from cgi import escape as html_escape
jusText-3.0.0/justext/_compat.py: from cgi import escape
lxml-4.9.2/doc/s5/ep2008/atom.py: import cgi
lxml-4.9.2/src/lxml/doctestcompare.py: from cgi import escape as html_escape
lxml-4.9.2/src/lxml/html/diff.py: from cgi import escape as html_escape
mechanize-0.4.8/examples/forms/echo.cgi: import cgi
mercurial-6.4.1/contrib/hgweb.fcgi: # import cgitb; cgitb.enable()
mercurial-6.4.1/contrib/hgweb.wsgi: #import cgitb; cgitb.enable()
mercurial-6.4.1/hgweb.cgi: # import cgitb; cgitb.enable()
mercurial-6.4.1/tests/test-clone-cgi.t: > import cgitb
mercurial-6.4.1/tests/test-mq.t: > import cgitb
mercurial-6.4.1/tests/test-newcgi.t: > import cgitb
mercurial-6.4.1/tests/test-newcgi.t: > import cgitb
mercurial-6.4.1/tests/test-newercgi.t: > import cgitb
mercurial-6.4.1/tests/test-newercgi.t: > import cgitb
mercurial-6.4.1/tests/test-oldcgi.t: > import cgitb, os, sys
mercurial-6.4.1/tests/test-oldcgi.t: > import cgitb, sys
mercurial-6.4.1/tests/test-push-cgi.t: > import cgitb
mozlog-7.1.1/mozlog/formatters/html/html.py: from cgi import escape
nibabel-5.1.0/tools/dicomfs.wsgi: import cgi
nltk-3.8.1/nltk/tree/prettyprinter.py: from cgi import escape
numpy-1.24.2/tools/npy_tempita/__init__.py: from cgi import escape as html_escape
oauth-1.0.1/oauth/oauth.py: import cgi
oauth2client-4.1.3/oauth2client/contrib/appengine.py: import cgi
odfpy-1.4.1/contrib/html2odt/html2odt.py: from cgi import escape,parse_header
odfpy-1.4.1/contrib/html2odt/shtml2odt.py: from cgi import escape,parse_header
oletools-0.60.1/oletools/thirdparty/prettytable/prettytable.py: from cgi import escape
os_sys-2.1.4/bars/bars/notebook.py: from cgi import escape
os_sys-2.1.4/bars/notebook.py: from cgi import escape
os_sys-2.1.4/os_sys/download.py: import cgi
os_sys-2.1.4/server/core/handlers/wsgi.py: import cgi
os_sys-2.1.4/server/core/management/templates.py: import cgi
os_sys-2.1.4/server/http/multipartparser.py: import cgi
pdfminer-20191125/tools/pdf2html.cgi: #import cgitb; cgitb.enable()
pdfminer-20191125/tools/pdf2html.cgi: import cgi, logging, traceback, random
peppercorn-0.6/peppercorn/tests.py: from cgi import FieldStorage
pex-2.1.131/pex/vendor/_vendored/pip/pip/_internal/index/collector.py: import cgi
pex-2.1.131/pex/vendor/_vendored/pip/pip/_internal/network/download.py: import cgi
prov-2.0.0/src/prov/dot.py: from cgi import escape
pyLDAvis-3.4.0/pyLDAvis/lib/python3.11/site-packages/setuptools/_distutils/config.py: import cgi
pyrocko-2023.3.27/src/gf/server.py: from cgi import escape
pyrocko-2023.3.27/src/gf/server.py: import cgi
pysaml2-7.4.1/src/saml2/httputil.py: import cgi
pysaml2-7.4.1/src/saml2/pack.py: import cgi as html  # type: ignore[no-redef]
pystache-0.6.0/pystache/defaults.py: from cgi import escape
python-mimeparse-1.6.0/mimeparse.py: import cgi
python-openid-2.2.5/contrib/openid-parse: import cgi, re, subprocess, sys
python-openid-2.2.5/examples/consumer.py: import cgi
python-openid-2.2.5/examples/consumer.py: import cgitb
python-openid-2.2.5/examples/djopenid/server/views.py: import cgi
python-openid-2.2.5/examples/server.py: import cgi
python-openid-2.2.5/examples/server.py: import cgitb
python-openid-2.2.5/openid/consumer/consumer.py: import cgi
python-openid-2.2.5/openid/test/test_auth_request.py: import cgi
python-openid-2.2.5/openid/test/test_consumer.py: import cgi
python-openid-2.2.5/openid/test/test_message.py: import cgi
python-openid-2.2.5/openid/test/test_server.py: import cgi
python3-openid-3.2.0/contrib/openid-parse: import cgi, re, subprocess, sys
python3-openid-3.2.0/examples/consumer.py: import cgitb
python3-openid-3.2.0/examples/djopenid/server/views.py: import cgi
python3-openid-3.2.0/examples/server.py: import cgitb
pyvmomi-8.0.0.1.2/pyVim/sso.py: from cgi import escape
scapy-2.5.0/scapy/compat.py: import cgi
sentry-sdk-1.19.1/sentry_sdk/integrations/pyramid.py: from webob.compat import cgi_FieldStorage  # type: ignore
sentry-sdk-1.19.1/sentry_sdk/utils.py: from cgi import parse_qs  # type: ignore
setuptools-67.6.1/setuptools/_distutils/config.py: import cgi
simplekml-1.3.6/simplekml/base.py: import cgi
speedtest-cli-2.1.3/speedtest.py: from cgi import parse_qs
supervisor-4.2.5/supervisor/compat.py: from cgi import escape
tableauserverclient-0.25/tableauserverclient/server/endpoint/datasources_endpoint.py: import cgi
tableauserverclient-0.25/tableauserverclient/server/endpoint/flows_endpoint.py: import cgi
tableauserverclient-0.25/tableauserverclient/server/endpoint/workbooks_endpoint.py: import cgi
tqdm-4.65.0/tqdm/notebook.py: from cgi import escape
twine-4.0.2/twine/commands/check.py: import cgi
uri-2.0.1/uri/compat.py: from cgi import escape
web.py-0.62/web/webapi.py: import cgi
webapp2-2.5.2/tests/request_test.py: import cgi
webapp2-2.5.2/tests/request_test.py: import cgi
webapp2-2.5.2/webapp2.py: import cgi
zeep-4.2.1/src/zeep/utils.py: import cgi

@hugovk hugovk self-requested a review May 23, 2023 09:44
hugovk
hugovk approved these changes May 23, 2023
View reviewed changes
Copy link
Member

@hugovk hugovk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹

@vstinner vstinner force-pushed the remove_cgi branch 2 times, most recently from c0001e7 to 734ae51 Compare May 24, 2023 07:38
@vstinner
Copy link
Member Author

test_importlib modified the environment on the Ubuntu job. I wrote a fix for that: #104840

vstinner added 2 commits May 24, 2023 10:30
@vstinner
pythongh-104773: PEP 594: Remove cgi and cgitb modules
10b3f34 
* Replace "cgi" with "!cgi" in the Sphinx documentation to avoid
  warnings on broken references.
* test_pyclbr no longer tests the cgi module.
@vstinner
Explain how to port existing code
5ad9cf6
@vstinner
Copy link
Member Author

I re-ran the Ubuntu job and test_importlib failed again.

test_importlib modified the environment on the Ubuntu job. I wrote a fix for that: #104840

I merged the test_importlib fix. I rebased my PR on top of it.

@vstinner vstinner enabled auto-merge (squash) May 24, 2023 08:57
@vstinner vstinner merged commit 08d5923 into python:main May 24, 2023
@vstinner vstinner deleted the remove_cgi branch May 24, 2023 09:05
@vstinner
Copy link
Member Author

Bye bye cgi! I added instructions on how to port existing code.

@JelleZijlstra JelleZijlstra mentioned this pull request May 25, 2023
Closed
@vstinner
Copy link
Member Author

@JulienPalard: Oh, I'm surprising that sphinx-lint missing this typo:

* :pep:`594`: Remove the :mod:`!cgi`` and :mod:`!cgitb` modules,
  deprecated in Python 3.11.

@vstinner vstinner mentioned this pull request May 31, 2023
Open
@Uzume Uzume mentioned this pull request May 18, 2024
Open
# for free to join this conversation on GitHub. Already have an account? # to comment
Reviewers

@gpshead gpshead gpshead approved these changes

@hugovk hugovk hugovk approved these changes

@ethanfurman ethanfurman Awaiting requested review from ethanfurman ethanfurman is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@vstinner @gpshead @hugovk @bedevere-bot