Test failures with Python 3.14.0b3

[mgorny]

After upgrading Python 3.14.0b2 to Python 3.14.0b3, I'm seeing the following test failures:

$ tox -e py314
py314: recreate env because python changed version_info=[3, 14, 0, 'beta', 2]->[3, 14, 0, 'beta', 3]
py314: remove tox env folder /tmp/markdown/.tox/py314
py314: install_deps> python -I -m pip install pytidylib
.pkg: recreate env because python changed version_info=[3, 14, 0, 'beta', 2]->[3, 14, 0, 'beta', 3]
.pkg: remove tox env folder /tmp/markdown/.tox/.pkg
.pkg: install_requires> python -I -m pip install 'setuptools>=77.0'
.pkg: _optional_hooks> python /usr/lib/python3.14/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg: get_requires_for_build_sdist> python /usr/lib/python3.14/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg: build_sdist> python /usr/lib/python3.14/site-packages/pyproject_api/_backend.py True setuptools.build_meta
py314: install_package_deps> python -I -m pip install coverage 'importlib-metadata>=4.4; python_version < "3.10"' pyyaml
py314: install_package> python -I -m pip install --force-reinstall --no-deps /tmp/markdown/.tox/.tmp/package/6/markdown-3.8.1.tar.gz
py314: commands[0]> coverage run --source=markdown -m unittest discover /tmp/markdown/tests
......................../tmp/markdown/markdown/__main__.py:103: DeprecationWarning: codecs.open() is deprecated. Use open() instead.
  with codecs.open(
................../tmp/markdown/markdown/core.py:420: DeprecationWarning: codecs.open() is deprecated. Use open() instead.
  input_file = codecs.open(input, mode="r", encoding=encoding)
/tmp/markdown/markdown/core.py:436: DeprecationWarning: codecs.open() is deprecated. Use open() instead.
  output_file = codecs.open(output, "w",
......................................................................................s...................................................................................sssssssssssssssssssssssssssssssssssssssssssssss...........................................................................ss...........................................................................................F.................................F.......................................F.....................ss...........................................................................................................................................................................................F.................................F.......................................F.............................................................................................................................................................................................................................................................................
======================================================================
FAIL: test_not_actually_cdata (test_syntax.blocks.test_html_blocks.TestHTMLBlocks.test_not_actually_cdata)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/markdown/tests/test_syntax/blocks/test_html_blocks.py", line 1280, in test_not_actually_cdata
    self.assertMarkdownRenders(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~^
        '<![',
        ^^^^^^
        '<p>&lt;![</p>'
        ^^^^^^^^^^^^^^^
    )
    ^
  File "/tmp/markdown/markdown/test_tools.py", line 77, in assertMarkdownRenders
    self.assertMultiLineEqual(output, expected)
    ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
AssertionError: '<!--[\n\n-->' != '<p>&lt;![</p>'
+ <p>&lt;![</p>
- <!--[
- 
- -->


======================================================================
FAIL: test_raw_missing_close_bracket (test_syntax.blocks.test_html_blocks.TestHTMLBlocks.test_raw_missing_close_bracket)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/markdown/tests/test_syntax/blocks/test_html_blocks.py", line 662, in test_raw_missing_close_bracket
    self.assertMarkdownRenders(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~^
        '<foo',
        ^^^^^^^
        '<p>&lt;foo</p>'
        ^^^^^^^^^^^^^^^^
    )
    ^
  File "/tmp/markdown/markdown/test_tools.py", line 77, in assertMarkdownRenders
    self.assertMultiLineEqual(output, expected)
    ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
AssertionError: '' != '<p>&lt;foo</p>'
+ <p>&lt;foo</p>


======================================================================
FAIL: test_unclosed_comment_ (test_syntax.blocks.test_html_blocks.TestHTMLBlocks.test_unclosed_comment_)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/markdown/tests/test_syntax/blocks/test_html_blocks.py", line 1022, in test_unclosed_comment_
    self.assertMarkdownRenders(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self.dedent(
        ^^^^^^^^^^^^
    ...<11 lines>...
        )
        ^
    )
    ^
  File "/tmp/markdown/markdown/test_tools.py", line 77, in assertMarkdownRenders
    self.assertMultiLineEqual(output, expected)
    ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
AssertionError: '<!-- unclosed comment\n\n*not* a comment\n\n-->' != '<p>&lt;!-- unclosed comment</p>\n<p><em>not</em> a comment</p>'
- <!-- unclosed comment
+ <p>&lt;!-- unclosed comment</p>
?  ++++++                    ++++
+ <p><em>not</em> a comment</p>
- 
- *not* a comment
- 
- -->


======================================================================
FAIL: test_not_actually_cdata (test_syntax.extensions.test_md_in_html.TestDefaultwMdInHTML.test_not_actually_cdata)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/markdown/tests/test_syntax/blocks/test_html_blocks.py", line 1280, in test_not_actually_cdata
    self.assertMarkdownRenders(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~^
        '<![',
        ^^^^^^
        '<p>&lt;![</p>'
        ^^^^^^^^^^^^^^^
    )
    ^
  File "/tmp/markdown/markdown/test_tools.py", line 77, in assertMarkdownRenders
    self.assertMultiLineEqual(output, expected)
    ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
AssertionError: '<!--[\n\n-->' != '<p>&lt;![</p>'
+ <p>&lt;![</p>
- <!--[
- 
- -->


======================================================================
FAIL: test_raw_missing_close_bracket (test_syntax.extensions.test_md_in_html.TestDefaultwMdInHTML.test_raw_missing_close_bracket)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/markdown/tests/test_syntax/blocks/test_html_blocks.py", line 662, in test_raw_missing_close_bracket
    self.assertMarkdownRenders(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~^
        '<foo',
        ^^^^^^^
        '<p>&lt;foo</p>'
        ^^^^^^^^^^^^^^^^
    )
    ^
  File "/tmp/markdown/markdown/test_tools.py", line 77, in assertMarkdownRenders
    self.assertMultiLineEqual(output, expected)
    ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
AssertionError: '' != '<p>&lt;foo</p>'
+ <p>&lt;foo</p>


======================================================================
FAIL: test_unclosed_comment_ (test_syntax.extensions.test_md_in_html.TestDefaultwMdInHTML.test_unclosed_comment_)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/markdown/tests/test_syntax/blocks/test_html_blocks.py", line 1022, in test_unclosed_comment_
    self.assertMarkdownRenders(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self.dedent(
        ^^^^^^^^^^^^
    ...<11 lines>...
        )
        ^
    )
    ^
  File "/tmp/markdown/markdown/test_tools.py", line 77, in assertMarkdownRenders
    self.assertMultiLineEqual(output, expected)
    ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
AssertionError: '<!-- unclosed comment\n\n*not* a comment\n\n-->' != '<p>&lt;!-- unclosed comment</p>\n<p><em>not</em> a comment</p>'
- <!-- unclosed comment
+ <p>&lt;!-- unclosed comment</p>
?  ++++++                    ++++
+ <p><em>not</em> a comment</p>
- 
- *not* a comment
- 
- -->


----------------------------------------------------------------------
Ran 1056 tests in 1.743s

FAILED (failures=6, skipped=52)
py314: exit 1 (2.02 seconds) /tmp/markdown> coverage run --source=markdown -m unittest discover /tmp/markdown/tests pid=3540134
  py314: FAIL code 1 (8.97=setup[6.96]+cmd[2.02] seconds)
  evaluation failed :( (9.06 seconds)

[facelessuser]

This may be a duplicate of: #1534

[facelessuser]

Oh, you seem to be using the latest release (that was about 3 hours ago: 3.8.1).

[mgorny]

Yes, sorry for not mentioning it. I'm so used to always retesting latest git that I forget to mention it.

[mgorny]

Also it's definitely related to 3.14.0b3. The 3.8 release also fails like this, and both pass with 3.14.0b2. I just hope this isn't another regression in CPython.

[facelessuser]

These differences bother me ☹. They will definitely require further investigation.

[facelessuser]

I have confirmed these failures locally.

[facelessuser]

It seems the parser is now auto closing unclosed comments and then dropping content like <foo if not completed properly. I'm not sure how to fix it yet, but that seems to be what is happening at least.

[facelessuser]

The <foo case seems easy to handle in a backwards compatible way. I'm not sure about the autocompletion of comments yet.

[mgorny]

I'm guessing this is python/cpython#135464. Given it's a security fix, it will probably be backported to all older Python versions too.

[mgorny]

And confirmed. Tests pass prior to that change, fail with it.

[facelessuser]

So far, I can fix all cases except <!-- unclosed comment. It seems that right now, if the comment is unclosed, it consumes everything until the end.

[waylan]

Looks like this is getting backported to Python 3.9, 3.10, 3.11, 3.12, 3.13, and 3.14 so we need to get this fix out soon.

Apparently, Python has actually been applying fixes to the HTML parser recently, This is the third I'm aware of in the last few months. It took them 5 years to finally merge my pull request (see gh-86155. Maybe a new dev is actually interested in working on the lib!?

[waylan]

So far, I can fix all cases except <!-- unclosed comment. It seems that right now, if the comment is unclosed, it consumes everything until the end.

I was going to say that that behavior actually makes more sense as it matches how browsers parse HTML. Then I checked the tests and see this comment (at test_html_blocks.py#L1018-L1020):

    # Note: This is a change in behavior. Previously, Python-Markdown interpreted this in the same manner
    # as browsers and all text after the opening comment tag was considered to be in a comment. However,
    # that did not match the reference implementation. The new behavior does.

🤷‍♂