`end_offset` is ignored in subclasses of SyntaxError (3.12-only) #111609

stbech · 2023-11-01T16:05:04Z

Bug report

Bug description:

When I tried to implement a subclass of SyntaxError, the exception details ignored the end_offset-variable and just showed one caret at the position of offset.

raise SyntaxError("Error message", (None, 1, 5, 'a = sin(3)', 1, 9))

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<string>", line 1
    a = sin(3)
        ^^^^
SyntaxError: Error message

class CustomSyntaxError(SyntaxError):
    pass
raise CustomSyntaxError("Error message", (None, 1, 5, 'a = sin(3)', 1, 9))

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<string>", line 1
    a = sin(3)
        ^
CustomSyntaxError: Error message

CPython versions tested on:

3.11, 3.12

Operating systems tested on:

Windows

Linked PRs

The text was updated successfully, but these errors were encountered:

sobolevn · 2023-11-01T16:08:32Z

It works correctly for me on 3.13a1:

>>> class CustomSyntaxError(SyntaxError):
...     pass
... 
>>> raise CustomSyntaxError("Error message", (None, 1, 5, 'a = sin(3)', 1, 9))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
    raise CustomSyntaxError("Error message", (None, 1, 5, 'a = sin(3)', 1, 9))
  File "<string>", line 1
    a = sin(3)
        ^^^^
CustomSyntaxError: Error message

harshitrathor3 · 2023-11-01T17:29:57Z

Also worked correctly for me on 3.10.10

raise SyntaxError("Error message", (None, 1, 5, 'a = sin(3)', 1, 9))
Traceback (most recent call last):
  File "<pyshell#0>", line 1, in <module>
    raise SyntaxError("Error message", (None, 1, 5, 'a = sin(3)', 1, 9))
  File "<string>", line 1
    a = sin(3)
        ^^^^
SyntaxError: Error message


class CustomSyntaxError(SyntaxError):
    pass
raise CustomSyntaxError("Error message", (None, 1, 5, 'a = sin(3)', 1, 9))
Traceback (most recent call last):
  File "<pyshell#4>", line 1, in <module>
    raise CustomSyntaxError("Error message", (None, 1, 5, 'a = sin(3)', 1, 9))
  File "<string>", line 1
    a = sin(3)
        ^^^^
CustomSyntaxError: Error message

stbech · 2023-11-02T20:58:48Z

For me (Python 3.11.6 on Windows 10) the console does not print the carets in the original error message, only when using the traceback package:

Python 3.11.6 (tags/v3.11.6:8b6ee5b, Oct  2 2023, 14:57:12) [MSC v.1935 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import traceback
>>> class CustomSyntaxError(SyntaxError):
...     pass
...
>>> try:
...     raise CustomSyntaxError("Error message", ('<py>', 1, 5, 'a = sin(3)', 1, 9))
... except Exception as e:
...     traceback.print_exc()
...     print('-'*20)
...     raise e
...
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "<py>", line 1
    a = sin(3)
        ^^^^
CustomSyntaxError: Error message
--------------------
Traceback (most recent call last):
  File "<stdin>", line 6, in <module>
  File "<stdin>", line 2, in <module>
  File "<py>", line 1
    a = sin(3)
        ^
CustomSyntaxError: Error message
>>> raise SyntaxError("Error message", ('<py>', 1, 5, 'a = sin(3)', 1, 9))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<py>", line 1
    a = sin(3)
        ^^^^
SyntaxError: Error message

See also this gist for the behavior in a Ipython console or Jupyter Notebook. There neither SyntaxError nor its subclass showed the desired result.
Maybe it has something to do with the error handling in the console, e.g. the console in PyCharm Community Edition works fine for me.

zedzhen · 2024-04-08T14:46:33Z

Python 3.11.1, 3.12.2 (Windows 10)
IDLE

import sys
import traceback
class CustomSyntaxError(SyntaxError):
    pass

try:
    raise CustomSyntaxError("Error message", ('<py>', 1, 5, 'a = sin(3)', 1, 9))
except Exception as e:
    traceback.print_exc()
    print('-' * 20, file=sys.stderr)
    raise e from None

Traceback (most recent call last):
  File "<pyshell#4>", line 2, in <module>
  File "<py>", line 1
    a = sin(3)
        ^^^^
CustomSyntaxError: Error message
--------------------
Traceback (most recent call last):
  File "<pyshell#4>", line 6, in <module>
    raise e from None
  File "<pyshell#4>", line 2, in <module>
    raise CustomSyntaxError("Error message", ('<py>', 1, 5, 'a = sin(3)', 1, 9))
  File "<py>", line 1
    a = sin(3)
        ^^^^
CustomSyntaxError: Error message

========= RESTART: D:\path\test.py =========
Traceback (most recent call last):
  File "D:\path\test.py", line 10, in <module>
    raise CustomSyntaxError("Error message", ('<py>', 1, 5, 'a = sin(3)', 1, 9))
  File "<py>", line 1
    a = sin(3)
        ^^^^
CustomSyntaxError: Error message
--------------------
Traceback (most recent call last):
  File "D:\path\test.py", line 14, in <module>
    raise e from None
  File "D:\path\test.py", line 10, in <module>
    raise CustomSyntaxError("Error message", ('<py>', 1, 5, 'a = sin(3)', 1, 9))
  File "<py>", line 1
    a = sin(3)
        ^^^^
CustomSyntaxError: Error message

cmd

C:\path\Python312>python
Python 3.12.2 (tags/v3.12.2:6abddd9, Feb  6 2024, 21:26:36) [MSC v.1937 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
random, math imported
>>> import sys
>>> import traceback
>>>
>>>
>>> class CustomSyntaxError(SyntaxError):
...     pass
...
>>>
>>> try:
...     raise CustomSyntaxError("Error message", ('<py>', 1, 5, 'a = sin(3)', 1, 9))
... except Exception as e:
...     traceback.print_exc()
...     print('-' * 20, file=sys.stderr)
...     raise e from None
...
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "<py>", line 1
    a = sin(3)
        ^^^^
CustomSyntaxError: Error message
--------------------
Traceback (most recent call last):
  File "<stdin>", line 6, in <module>
  File "<stdin>", line 2, in <module>
  File "<py>", line 1
    a = sin(3)
        ^
CustomSyntaxError: Error message
>>> ^Z


C:\path\Python312>python D:\path\test.py
Traceback (most recent call last):
  File "D:\path\test.py", line 10, in <module>
    raise CustomSyntaxError("Error message", ('<py>', 1, 5, 'a = sin(3)', 1, 9))
  File "<py>", line 1
    a = sin(3)
        ^^^^
CustomSyntaxError: Error message
--------------------
Traceback (most recent call last):
  File "D:\path\test.py", line 14, in <module>
    raise e from None
  File "D:\path\test.py", line 10, in <module>
    raise CustomSyntaxError("Error message", ('<py>', 1, 5, 'a = sin(3)', 1, 9))
  File "<py>", line 1
    a = sin(3)
        ^
CustomSyntaxError: Error message

PyCharm 2023.3.5 (Professional Edition)

Traceback (most recent call last):
  File "D:\path\test.py", line 10, in <module>
    raise CustomSyntaxError("Error message", ('<py>', 1, 5, 'a = sin(3)', 1, 9))
  File "<py>", line 1
    a = sin(3)
        ^^^^
CustomSyntaxError: Error message
--------------------
Traceback (most recent call last):
  File "D:\path\test.py", line 14, in <module>
    raise e from None
  File "D:\path\test.py", line 10, in <module>
    raise CustomSyntaxError("Error message", ('<py>', 1, 5, 'a = sin(3)', 1, 9))
  File "<py>", line 1
    a = sin(3)
        ^
CustomSyntaxError: Error message

test.py

import sys
import traceback


class CustomSyntaxError(SyntaxError):
    pass


try:
    raise CustomSyntaxError("Error message", ('<py>', 1, 5, 'a = sin(3)', 1, 9))
except Exception as e:
    traceback.print_exc()
    print('-' * 20, file=sys.stderr)
    raise e from None

zedzhen · 2024-04-08T14:54:27Z

Python 3.13.0a5 (windows 10) correct print

C:\path\Python313>python
Python 3.13.0a5 (tags/v3.13.0a5:076d169, Mar 12 2024, 21:29:03) [MSC v.1938 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
random, math imported
>>> import sys
>>> import traceback
>>>
>>>
>>> class CustomSyntaxError(SyntaxError):
...     pass
...
>>>
>>> try:
...     raise CustomSyntaxError("Error message", ('<py>', 1, 5, 'a = sin(3)', 1, 9))
... except Exception as e:
...     traceback.print_exc()
...     print('-' * 20, file=sys.stderr)
...     raise e from None
...
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
    raise CustomSyntaxError("Error message", ('<py>', 1, 5, 'a = sin(3)', 1, 9))
  File "<py>", line 1
    a = sin(3)
        ^^^^
CustomSyntaxError: Error message
--------------------
Traceback (most recent call last):
  File "<stdin>", line 6, in <module>
    raise e from None
  File "<stdin>", line 2, in <module>
    raise CustomSyntaxError("Error message", ('<py>', 1, 5, 'a = sin(3)', 1, 9))
  File "<py>", line 1
    a = sin(3)
        ^^^^
CustomSyntaxError: Error message
>>> ^Z


C:\path\Python313>python D:\path\test.py
Traceback (most recent call last):
  File "D:\path\test.py", line 10, in <module>
    raise CustomSyntaxError("Error message", ('<py>', 1, 5, 'a = sin(3)', 1, 9))
  File "<py>", line 1
    a = sin(3)
        ^^^^
CustomSyntaxError: Error message
--------------------
Traceback (most recent call last):
  File "D:\path\test.py", line 14, in <module>
    raise e from None
  File "D:\path\test.py", line 10, in <module>
    raise CustomSyntaxError("Error message", ('<py>', 1, 5, 'a = sin(3)', 1, 9))
  File "<py>", line 1
    a = sin(3)
        ^^^^
CustomSyntaxError: Error message

nineteendo · 2024-07-19T18:41:34Z

Can confirm on 3.12.4:

Python 3.12.4 (main, Jun  6 2024, 18:26:44) [Clang 15.0.0 (clang-1500.1.0.2.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from jsonyx import loads
>>> loads("NaN")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/wannes/Library/CloudStorage/OneDrive-Personal/Personal/GitHub/pyvz2/alpha/src/jsonyx/__init__.py", line 248, in loads
    return JSONDecoder(allow=allow).loads(s, filename=filename)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/wannes/Library/CloudStorage/OneDrive-Personal/Personal/GitHub/pyvz2/alpha/src/jsonyx/__init__.py", line 123, in loads
    return self._scanner(filename, s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 1
    NaN
    ^
jsonyx._decoder.JSONSyntaxError: NaN is not allowed

It works correctly on 3.13.0b4:

Python 3.13.0b4 (v3.13.0b4:567c38b4eb, Jul 18 2024, 07:28:29) [Clang 15.0.0 (clang-1500.3.9.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from jsonyx import loads
>>> loads("NaN")
Traceback (most recent call last):
  File "<python-input-1>", line 1, in <module>
    loads("NaN")
    ~~~~~^^^^^^^
  File "/Users/wannes/Library/CloudStorage/OneDrive-Personal/Personal/GitHub/pyvz2/alpha/src/jsonyx/__init__.py", line 248, in loads
    return JSONDecoder(allow=allow).loads(s, filename=filename)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/wannes/Library/CloudStorage/OneDrive-Personal/Personal/GitHub/pyvz2/alpha/src/jsonyx/__init__.py", line 123, in loads
    return self._scanner(filename, s)
           ~~~~~~~~~~~~~^^^^^^^^^^^^^
  File "/Users/wannes/Library/CloudStorage/OneDrive-Personal/Personal/GitHub/pyvz2/alpha/src/jsonyx/_decoder.py", line 398, in scanner
    raise exc.with_traceback(None) from None
  File "<string>", line 1
    NaN
    ^^^
jsonyx._decoder.JSONSyntaxError: NaN is not allowed

picnixz · 2024-12-02T15:46:07Z

The bug is affecting 3.12.6 (haven't tested on 3.12.7) so I'll mark it as 3.12-only.

nineteendo · 2024-12-02T16:05:08Z

Could you do a bisect?

picnixz · 2024-12-02T16:06:16Z

I don't really have time for now (I'm triaging the pending issues), so I'll leave it to anyone willing to do it.

nineteendo · 2024-12-03T11:01:46Z

This also wasn't working on 3.10 and 3.11. It was fixed in 3.13 by #110702.
~~Should we back port it to fix this?~~ We should fix it differently.

I don't want to choose between end_offset and storing the column number.

nineteendo · 2024-12-03T11:33:56Z

This fixes the problem:

diff --git a/Python/pythonrun.c b/Python/pythonrun.c
index 5f3d249df4..cf84573a8e 100644
--- a/Python/pythonrun.c
+++ b/Python/pythonrun.c
@@ -538,7 +538,6 @@ parse_syntax_error(PyObject *err, PyObject **message, PyObject **filename,
         *offset = hold;
     }
 
-    if (Py_TYPE(err) == (PyTypeObject*)PyExc_SyntaxError) {
     v = PyObject_GetAttr(err, &_Py_ID(end_lineno));
     if (!v) {
         PyErr_Clear();
@@ -570,11 +569,6 @@ parse_syntax_error(PyObject *err, PyObject **message, PyObject **filename,
             goto finally;
         *end_offset = hold;
     }
-    } else {
-        // SyntaxError subclasses
-        *end_lineno = *lineno;
-        *end_offset = -1;
-    }
 
     v = PyObject_GetAttr(err, &_Py_ID(text));
     if (!v)

nineteendo · 2024-12-03T11:42:13Z

The weird thing is that this check isn't even necessary:

>>> class CustomSyntaxError(SyntaxError):
...     def __init__(self, msg, filename, lineno, offset, text, end_lineno, end_offset):
...         self.msg = msg
...         self.filename = filename
...         self.lineno = lineno
...         self.offset = offset
...         self.text = text
...         # self.end_lineno = end_lineno
...         # self.end_offset = end_offset
... 
>>> raise CustomSyntaxError("Error message", None, 1, 5, 'a = sin(3)', 1, 9)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<string>", line 1
    a = sin(3)
        ^
CustomSyntaxError: Error message

#127554) * `end_offset` is ignored in subclasses of SyntaxError * 📜🤖 Added by blurb_it. * Add test --------- Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>

Test `end_offset` in SyntaxError subclass

…127830) Test `end_offset` in SyntaxError subclass (cherry picked from commit e8f4e27) Co-authored-by: Nice Zombies <nineteendo19d0@gmail.com>

…7830) Test `end_offset` in SyntaxError subclass

…127830) Test `end_offset` in SyntaxError subclass (cherry picked from commit e8f4e27) Co-authored-by: Nice Zombies <nineteendo19d0@gmail.com>

… (#130407) Co-authored-by: Nice Zombies <nineteendo19d0@gmail.com>

stbech added the type-bug An unexpected behavior, bug, or error label Nov 1, 2023

iritkatriel added the pending The issue will be closed if no feedback is provided label Nov 2, 2023

picnixz added the 3.12 bugs and security fixes label Dec 2, 2024

picnixz changed the title ~~end_offset is ignored in subclasses of SyntaxError~~ end_offset is ignored in subclasses of SyntaxError (3.12-only) Dec 2, 2024

picnixz added stdlib Python modules in the Lib dir topic-repl Related to the interactive shell and removed pending The issue will be closed if no feedback is provided labels Dec 2, 2024

bedevere-app bot mentioned this issue Dec 3, 2024

[3.12] gh-111609: end_offset is ignored in subclasses of SyntaxError #127554

Merged

bedevere-app bot mentioned this issue Dec 11, 2024

gh-111609: Test end_offset in SyntaxError subclass #127830

Merged

pablogsal pushed a commit that referenced this issue Dec 11, 2024

gh-111609: Test end_offset in SyntaxError subclass (#127830)

e8f4e27

Test `end_offset` in SyntaxError subclass

srinivasreddy pushed a commit to srinivasreddy/cpython that referenced this issue Jan 8, 2025

pythongh-111609: Test end_offset in SyntaxError subclass (python#12…

561297c

…7830) Test `end_offset` in SyntaxError subclass

bedevere-app bot mentioned this issue Feb 21, 2025

[3.13] gh-111609: Test end_offset in SyntaxError subclass (GH-127830) #130407

Merged

hugovk pushed a commit that referenced this issue Feb 21, 2025

[3.13] gh-111609: Test end_offset in SyntaxError subclass (GH-127830)…

cb8d89d

… (#130407) Co-authored-by: Nice Zombies <nineteendo19d0@gmail.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`end_offset` is ignored in subclasses of SyntaxError (3.12-only) #111609

`end_offset` is ignored in subclasses of SyntaxError (3.12-only) #111609

stbech commented Nov 1, 2023 •

edited by bedevere-app bot

Loading

sobolevn commented Nov 1, 2023

harshitrathor3 commented Nov 1, 2023 •

edited by hugovk

Loading

stbech commented Nov 2, 2023 •

edited by hugovk

Loading

zedzhen commented Apr 8, 2024 •

edited by hugovk

Loading

zedzhen commented Apr 8, 2024 •

edited by hugovk

Loading

nineteendo commented Jul 19, 2024

picnixz commented Dec 2, 2024

nineteendo commented Dec 2, 2024

picnixz commented Dec 2, 2024 •

edited

Loading

nineteendo commented Dec 3, 2024 •

edited

Loading

nineteendo commented Dec 3, 2024

nineteendo commented Dec 3, 2024

end_offset is ignored in subclasses of SyntaxError (3.12-only) #111609

end_offset is ignored in subclasses of SyntaxError (3.12-only) #111609

Comments

stbech commented Nov 1, 2023 • edited by bedevere-app bot Loading

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

sobolevn commented Nov 1, 2023

harshitrathor3 commented Nov 1, 2023 • edited by hugovk Loading

stbech commented Nov 2, 2023 • edited by hugovk Loading

zedzhen commented Apr 8, 2024 • edited by hugovk Loading

zedzhen commented Apr 8, 2024 • edited by hugovk Loading

nineteendo commented Jul 19, 2024

picnixz commented Dec 2, 2024

nineteendo commented Dec 2, 2024

picnixz commented Dec 2, 2024 • edited Loading

nineteendo commented Dec 3, 2024 • edited Loading

nineteendo commented Dec 3, 2024

nineteendo commented Dec 3, 2024

`end_offset` is ignored in subclasses of SyntaxError (3.12-only) #111609

`end_offset` is ignored in subclasses of SyntaxError (3.12-only) #111609

stbech commented Nov 1, 2023 •

edited by bedevere-app bot

Loading

harshitrathor3 commented Nov 1, 2023 •

edited by hugovk

Loading

stbech commented Nov 2, 2023 •

edited by hugovk

Loading

zedzhen commented Apr 8, 2024 •

edited by hugovk

Loading

zedzhen commented Apr 8, 2024 •

edited by hugovk

Loading

picnixz commented Dec 2, 2024 •

edited

Loading

nineteendo commented Dec 3, 2024 •

edited

Loading