fails to build with Python 3.13 #1181

Cikmo · 2024-09-15T09:26:09Z

asyncpg version: 0.29.0
Python version: 3.13.0rc2
Platform: Linux
Did you install asyncpg with pip?: Yes
If you built asyncpg locally, which version of Cython did you use?: n/a
Can the issue be reproduced under both asyncio and
uvloop?: yes

Fails to build on Python 3.13. There's been various API changes.

      asyncpg/pgproto/pgproto.c:23886:18: warning: ‘__pyx_f_7asyncpg_7pgproto_7pgproto_json_encode’ defined but not used [-Wunused-function]
      23886 | static PyObject *__pyx_f_7asyncpg_7pgproto_7pgproto_json_encode(struct __pyx_obj_7asyncpg_7pgproto_7pgproto_CodecContext *__pyx_v_settings, struct __pyx_obj_7asyncpg_7pgproto_7pgproto_WriteBuffer *__pyx_v_buf, PyObject *__pyx_v_obj) {
            |                  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

      /home/username/.pyenv/versions/3.13.0rc2/include/python3.13/cpython/longobject.h:111:17: note: declared here
        111 | PyAPI_FUNC(int) _PyLong_AsByteArray(PyLongObject* v,
            |                 ^~~~~~~~~~~~~~~~~~~

Read more: MagicStack/uvloop#603

The text was updated successfully, but these errors were encountered:

musicinmybrain · 2024-09-28T13:15:56Z

If it helps, we’re successfully building and testing asyncpg with Python 3.13 in Fedora using the following adaptations:

A downstream-only patch to remove the upper bound on the uvloop test dependency – although writing this prompts me to open Allow testing with uvloop on Python 3.12 #1182 to at least allow testing with uvloop on Python 3.12.
A patch for Allow Cython 3 (fix #1083) #1101, which has been merged but not released.
A patch for Replace obsolete, unsafe Py_TRASHCAN_SAFE_BEGIN/END #1150, which has been merged but not released.
Skip tests tests/test_connect.py::TestClientSSLConnection::test_ssl_connection_client_auth_custom_context and tests/test_connect.py::TestClientSSLConnection::test_ssl_connection_client_auth_fails_with_wrong_setup, as discussed in Replace obsolete, unsafe Py_TRASHCAN_SAFE_BEGIN/END #1150 (comment). Ideally, the underlying certificate issue could be fixed.

And, of course, we remove any pre-generated Cythonized C sources and re-generate them with a current Cython (3.0.11):

# Remove pre-generated C sources from Cython to ensure they are re-generated
# and not used in the build. Note that recordobj.c is not a generated source,
# and must not be removed!
find asyncpg -type f -name '*.c' ! -name 'recordobj.c' -print -delete

(We also export the environment variable ASYNCPG_BUILD_CYTHON_ALWAYS=1, which might not be necessary once we’ve removed any pre-generated Cythonized C sources.)

kamikaze · 2024-10-10T16:52:32Z

same issue here

sylvainmouquet · 2024-10-11T08:42:48Z

same error but when i set the github repository, it works with python versions 3.9, 3.10, 3.11, 3.12 and 3.13:

[tool.uv.sources]
asyncpg = { git = "https://github.com/MagicStack/asyncpg"}

When the next release is planned ?

juergenzimmermann · 2024-10-12T09:27:10Z

pip install ... also fails on a Windows box. The output is attached.
windows-python-3.13..txt

Kwieeciol · 2024-10-13T11:06:32Z

same issue on Mac

ERROR: Failed building wheel for asyncpg
Failed to build asyncpg
ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (asyncpg)

EwertonDCSilv · 2024-10-17T02:05:25Z

@musicinmybrain Existing any alternative for python-3.13.0 ? I am using docker image python-3.13.0-slim

EwertonDCSilv · 2024-10-17T02:06:30Z

@musicinmybrain Existing any alternative for python-3.13.0 ? I am using docker image python-3.13.0-slim

21.42       gcc -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall -fPIC -I/usr/local/include/python3.13 -c asyncpg/pgproto/pgproto.c -o build/temp.linux-x86_64-cpython-313/asyncpg/pgproto/pgproto.o -O2 -fsigned-char -Wall -Wsign-compare -Wconversion
21.42       asyncpg/pgproto/pgproto.c:864:1: warning: ‘Py_UNICODE’ is deprecated [-Wdeprecated-declarations]
21.42         864 | static CYTHON_INLINE size_t __Pyx_Py_UNICODE_strlen(const Py_UNICODE *u) {
21.42             | ^~~~~~
21.42       asyncpg/pgproto/pgproto.c: In function ‘__Pyx_Py_UNICODE_strlen’:
21.42       asyncpg/pgproto/pgproto.c:865:5: warning: ‘Py_UNICODE’ is deprecated [-Wdeprecated-declarations]
21.42         865 |     const Py_UNICODE *u_end = u;
21.42             |     ^~~~~
21.42       asyncpg/pgproto/pgproto.c: In function ‘__Pyx_init_assertions_enabled’:
21.42       asyncpg/pgproto/pgproto.c:1667:39: warning: implicit declaration of function ‘_PyInterpreterState_GetConfig’; did you mean ‘PyInterpreterState_GetID’? [-Wimplicit-function-declaration]
21.42        1667 |     __pyx_assertions_enabled_flag = ! _PyInterpreterState_GetConfig(__Pyx_PyThreadState_Current->interp)->optimization_level;
21.42             |                                       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
21.42             |                                       PyInterpreterState_GetID
21.42       asyncpg/pgproto/pgproto.c:1667:105: error: invalid type argument of ‘->’ (have ‘int’)
21.42        1667 |     __pyx_assertions_enabled_flag = ! _PyInterpreterState_GetConfig(__Pyx_PyThreadState_Current->interp)->optimization_level;
21.42             |                                                                                                         ^~
21.42       asyncpg/pgproto/pgproto.c: In function ‘__Pyx_PyUnicode_Join’:
21.42       asyncpg/pgproto/pgproto.c:38932:13: warning: implicit declaration of function ‘_PyUnicode_FastCopyCharacters’; did you mean ‘PyUnicode_CopyCharacters’? [-Wimplicit-function-declaration]
21.42       38932 |             _PyUnicode_FastCopyCharacters(result_uval, char_pos, uval, 0, ulength);
21.42             |             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
21.42             |             PyUnicode_CopyCharacters
21.42       In file included from /usr/local/include/python3.13/internal/pycore_code.h:12,
21.42                        from /usr/local/include/python3.13/internal/pycore_frame.h:13,
21.42                        from asyncpg/pgproto/pgproto.c:41520:
21.42       /usr/local/include/python3.13/internal/pycore_backoff.h: In function ‘make_backoff_counter’:
21.42       /usr/local/include/python3.13/internal/pycore_backoff.h:60:20: warning: conversion from ‘uint16_t’ {aka ‘short unsigned int’} to ‘short unsigned int:12’ may change value [-Wconversion]
21.42          60 |     result.value = value;
21.42             |                    ^~~~~
21.42       /usr/local/include/python3.13/internal/pycore_backoff.h:61:22: warning: conversion from ‘uint16_t’ {aka ‘short unsigned int’} to ‘unsigned char:4’ may change value [-Wconversion]
21.42          61 |     result.backoff = backoff;
21.42             |                      ^~~~~~~
21.42       /usr/local/include/python3.13/internal/pycore_backoff.h: In function ‘restart_backoff_counter’:
21.42       /usr/local/include/python3.13/internal/pycore_backoff.h:78:66: warning: conversion from ‘int’ to ‘uint16_t’ {aka ‘short unsigned int’} may change value [-Wconversion]
21.42          78 |         return make_backoff_counter((1 << (counter.backoff + 1)) - 1, counter.backoff + 1);
21.42             |                                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~
21.42       asyncpg/pgproto/pgproto.c: In function ‘__Pyx_PyInt_As_int’:
21.42       asyncpg/pgproto/pgproto.c:41809:27: error: too few arguments to function ‘_PyLong_AsByteArray’
21.42       41809 |                 int ret = _PyLong_AsByteArray((PyLongObject *)v,
21.42             |                           ^~~~~~~~~~~~~~~~~~~
21.42       In file included from /usr/local/include/python3.13/longobject.h:107,
21.42                        from /usr/local/include/python3.13/Python.h:81,
21.42                        from asyncpg/pgproto/pgproto.c:33:
21.42       /usr/local/include/python3.13/cpython/longobject.h:111:17: note: declared here
21.42         111 | PyAPI_FUNC(int) _PyLong_AsByteArray(PyLongObject* v,
21.42             |                 ^~~~~~~~~~~~~~~~~~~
21.42       asyncpg/pgproto/pgproto.c: In function ‘__Pyx_PyInt_As_int64_t’:
21.42       asyncpg/pgproto/pgproto.c:42043:27: error: too few arguments to function ‘_PyLong_AsByteArray’
21.42       42043 |                 int ret = _PyLong_AsByteArray((PyLongObject *)v,
21.42             |                           ^~~~~~~~~~~~~~~~~~~
21.42       /usr/local/include/python3.13/cpython/longobject.h:111:17: note: declared here
21.42         111 | PyAPI_FUNC(int) _PyLong_AsByteArray(PyLongObject* v,
21.42             |                 ^~~~~~~~~~~~~~~~~~~
21.42       asyncpg/pgproto/pgproto.c: In function ‘__Pyx_PyInt_As_int32_t’:
21.42       asyncpg/pgproto/pgproto.c:42239:27: error: too few arguments to function ‘_PyLong_AsByteArray’
21.42       42239 |                 int ret = _PyLong_AsByteArray((PyLongObject *)v,
21.42             |                           ^~~~~~~~~~~~~~~~~~~
21.42       /usr/local/include/python3.13/cpython/longobject.h:111:17: note: declared here
21.42         111 | PyAPI_FUNC(int) _PyLong_AsByteArray(PyLongObject* v,
21.42             |                 ^~~~~~~~~~~~~~~~~~~
21.42       asyncpg/pgproto/pgproto.c: In function ‘__Pyx_PyInt_As_long’:
21.42       asyncpg/pgproto/pgproto.c:42435:27: error: too few arguments to function ‘_PyLong_AsByteArray’
21.42       42435 |                 int ret = _PyLong_AsByteArray((PyLongObject *)v,
21.42             |                           ^~~~~~~~~~~~~~~~~~~
21.42       /usr/local/include/python3.13/cpython/longobject.h:111:17: note: declared here
21.42         111 | PyAPI_FUNC(int) _PyLong_AsByteArray(PyLongObject* v,
21.42             |                 ^~~~~~~~~~~~~~~~~~~
21.42       asyncpg/pgproto/pgproto.c: In function ‘__Pyx_PyInt_As_unsigned_char’:
21.42       asyncpg/pgproto/pgproto.c:42745:27: error: too few arguments to function ‘_PyLong_AsByteArray’
21.42       42745 |                 int ret = _PyLong_AsByteArray((PyLongObject *)v,
21.42             |                           ^~~~~~~~~~~~~~~~~~~
21.42       /usr/local/include/python3.13/cpython/longobject.h:111:17: note: declared here
21.42         111 | PyAPI_FUNC(int) _PyLong_AsByteArray(PyLongObject* v,
21.42             |                 ^~~~~~~~~~~~~~~~~~~
21.42       asyncpg/pgproto/pgproto.c: In function ‘__Pyx_PyInt_As_int16_t’:
21.42       asyncpg/pgproto/pgproto.c:43055:27: error: too few arguments to function ‘_PyLong_AsByteArray’
21.42       43055 |                 int ret = _PyLong_AsByteArray((PyLongObject *)v,
21.42             |                           ^~~~~~~~~~~~~~~~~~~
21.42       /usr/local/include/python3.13/cpython/longobject.h:111:17: note: declared here
21.42         111 | PyAPI_FUNC(int) _PyLong_AsByteArray(PyLongObject* v,
21.42             |                 ^~~~~~~~~~~~~~~~~~~
21.42       asyncpg/pgproto/pgproto.c: In function ‘__Pyx_PyInt_As_char’:
21.42       asyncpg/pgproto/pgproto.c:43289:27: error: too few arguments to function ‘_PyLong_AsByteArray’
21.42       43289 |                 int ret = _PyLong_AsByteArray((PyLongObject *)v,
21.42             |                           ^~~~~~~~~~~~~~~~~~~
21.42       /usr/local/include/python3.13/cpython/longobject.h:111:17: note: declared here
21.42         111 | PyAPI_FUNC(int) _PyLong_AsByteArray(PyLongObject* v,
21.42             |                 ^~~~~~~~~~~~~~~~~~~
21.42       asyncpg/pgproto/pgproto.c: In function ‘__Pyx_PyInt_As_int8_t’:
21.42       asyncpg/pgproto/pgproto.c:43485:27: error: too few arguments to function ‘_PyLong_AsByteArray’
21.42       43485 |                 int ret = _PyLong_AsByteArray((PyLongObject *)v,
21.42             |                           ^~~~~~~~~~~~~~~~~~~
21.42       /usr/local/include/python3.13/cpython/longobject.h:111:17: note: declared here
21.42         111 | PyAPI_FUNC(int) _PyLong_AsByteArray(PyLongObject* v,
21.42             |                 ^~~~~~~~~~~~~~~~~~~
21.42       asyncpg/pgproto/pgproto.c: In function ‘__Pyx_PyInt_As_uint32_t’:
21.42       asyncpg/pgproto/pgproto.c:43719:27: error: too few arguments to function ‘_PyLong_AsByteArray’
21.42       43719 |                 int ret = _PyLong_AsByteArray((PyLongObject *)v,
21.42             |                           ^~~~~~~~~~~~~~~~~~~
21.42       /usr/local/include/python3.13/cpython/longobject.h:111:17: note: declared here
21.42         111 | PyAPI_FUNC(int) _PyLong_AsByteArray(PyLongObject* v,
21.42             |                 ^~~~~~~~~~~~~~~~~~~
21.42       asyncpg/pgproto/pgproto.c: In function ‘__Pyx_PyInt_As_uint64_t’:
21.42       asyncpg/pgproto/pgproto.c:43915:27: error: too few arguments to function ‘_PyLong_AsByteArray’
21.42       43915 |                 int ret = _PyLong_AsByteArray((PyLongObject *)v,
21.42             |                           ^~~~~~~~~~~~~~~~~~~
21.42       /usr/local/include/python3.13/cpython/longobject.h:111:17: note: declared here
21.42         111 | PyAPI_FUNC(int) _PyLong_AsByteArray(PyLongObject* v,
21.42             |                 ^~~~~~~~~~~~~~~~~~~
21.42       asyncpg/pgproto/pgproto.c: At top level:
21.42       asyncpg/pgproto/pgproto.c:23994:18: warning: ‘__pyx_f_7asyncpg_7pgproto_7pgproto_json_decode’ defined but not used [-Wunused-function]
21.42       23994 | static PyObject *__pyx_f_7asyncpg_7pgproto_7pgproto_json_decode(struct __pyx_obj_7asyncpg_7pgproto_7pgproto_CodecContext *__pyx_v_settings, struct __pyx_t_7asyncpg_7pgproto_7pgproto_FRBuffer *__pyx_v_buf) {
21.42             |                  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
21.42       asyncpg/pgproto/pgproto.c:23886:18: warning: ‘__pyx_f_7asyncpg_7pgproto_7pgproto_json_encode’ defined but not used [-Wunused-function]
21.42       23886 | static PyObject *__pyx_f_7asyncpg_7pgproto_7pgproto_json_encode(struct __pyx_obj_7asyncpg_7pgproto_7pgproto_CodecContext *__pyx_v_settings, struct __pyx_obj_7asyncpg_7pgproto_7pgproto_WriteBuffer *__pyx_v_buf, PyObject *__pyx_v_obj) {
21.42             |                  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
21.42       error: command '/usr/bin/gcc' failed with exit code 1
21.42       [end of output]

eirnym · 2024-10-17T07:00:21Z

There's many functions in cPython become suddenly internal therefore breaking to much binary packages. Probably there's an alternative for long to bytes function we can use.

python/cpython#106320

musicinmybrain · 2024-10-17T19:00:13Z

@musicinmybrain Existing any alternative for python-3.13.0 ? I am using docker image python-3.13.0-slim

I do not maintain asyncpg, and I can’t do anything to make it install properly from PyPI.

I maintain the python-asyncpg package in Fedora. In #1181 (comment), I described what was necessary for it to build and pass its test suite there, so that dnf install python3-asyncpg works as expected in the soon-to-be-released Fedora 41, where the system Python is 3.13. Sharing that experience and all of the necessary patches is really all I have to offer here.

elprans · 2024-10-17T19:04:06Z

I'll look into releasing 3.13 wheels soon.

musicinmybrain · 2024-10-17T19:04:39Z

There's many functions in cPython become suddenly internal therefore breaking to much binary packages. Probably there's an alternative for long to bytes function we can use.

python/cpython#106320

The logs in #1181 (comment) just indicate that the Cythonized C sources were generated by a too-old version of Cython. Try export ASYNCPG_BUILD_CYTHON_ALWAYS=1 before installing so that the pre-generated C sources included in the wheel are discarded, and new ones are generated with a current Cython at build time. (That’s assuming you are building from a source that has all other necessary patches.)

elprans · 2024-10-20T00:35:59Z

0.30.0 with support for Python 3.13 has been released.

EwertonDCSilv · 2024-10-24T04:29:21Z

0.30.0 com suporte para Python 3.13 foi lançado.

He was ! But it is likely that you will need to install greenlet. I believe that to solve the problems mentioned by @musicinmybrain...

Há muitas funções no cPython que se tornam internas de repente, quebrando muitos pacotes binários. Provavelmente há uma alternativa para a função long to bytes que podemos usar.
python/cpython#106320

Os logs em #1181 (comentário) apenas indicam que as fontes C Cythonizadas foram geradas por uma versão muito antiga do Cython. Tente export ASYNCPG_BUILD_CYTHON_ALWAYS=1antes de instalar para que as fontes C pré-geradas incluídas no wheel sejam descartadas, e novas sejam geradas com um Cython atual no momento da compilação. (Isso pressupõe que você esteja compilando a partir de uma fonte que tenha todos os outros patches necessários.)

After install asyncpg==0.30.0 from Python docker image python-3.13.0-slim, I received this error:

Traceback (most recent call last):
  File "/usr/local/lib/python3.13/site-packages/coverage/collector.py", line 145, in __init__
    import greenlet
ModuleNotFoundError: No module named 'greenlet'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/bin/pytest", line 8, in <module>
    sys.exit(console_main())
             ~~~~~~~~~~~~^^
  File "/usr/local/lib/python3.13/site-packages/_pytest/config/__init__.py", line 201, in console_main
    code = main()
  File "/usr/local/lib/python3.13/site-packages/_pytest/config/__init__.py", line 156, in main
    config = _prepareconfig(args, plugins)
  File "/usr/local/lib/python3.13/site-packages/_pytest/config/__init__.py", line 341, in _prepareconfig
    config = pluginmanager.hook.pytest_cmdline_parse(
        pluginmanager=pluginmanager, args=args
    )
  File "/usr/local/lib/python3.13/site-packages/pluggy/_hooks.py", line 513, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/pluggy/_callers.py", line 139, in _multicall
    raise exception.with_traceback(exception.__traceback__)
  File "/usr/local/lib/python3.13/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/_pytest/helpconfig.py", line 105, in pytest_cmdline_parse
    config = yield
             ^^^^^
  File "/usr/local/lib/python3.13/site-packages/pluggy/_callers.py", line 103, in _multicall
    res = hook_impl.function(*args)
  File "/usr/local/lib/python3.13/site-packages/_pytest/config/__init__.py", line 1140, in pytest_cmdline_parse
    self.parse(args)
    ~~~~~~~~~~^^^^^^
  File "/usr/local/lib/python3.13/site-packages/_pytest/config/__init__.py", line 1494, in parse
    self._preparse(args, addopts=addopts)
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/_pytest/config/__init__.py", line 1398, in _preparse
    self.hook.pytest_load_initial_conftests(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        early_config=self, args=args, parser=self._parser
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

laggron42 mentioned this issue Oct 8, 2024

Update to Python 3.13 Ballsdex-Team/BallsDex-DiscordBot#434

Closed

jvstme mentioned this issue Oct 15, 2024

Support Python 3.13, deprecate Python 3.8 dstackai/dstack#1833

Closed

7 tasks

jackwotherspoon mentioned this issue Oct 17, 2024

feat: add support for Python 3.13 GoogleCloudPlatform/cloud-sql-python-connector#1177

Merged

dantownsend mentioned this issue Oct 18, 2024

1094 Add Python 3.13 support piccolo-orm/piccolo#1104

Merged

elprans closed this as completed Oct 20, 2024

chenrui333 mentioned this issue Oct 24, 2024

dstack: migrate to python@3.13 Homebrew/homebrew-core#195491

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fails to build with Python 3.13 #1181

fails to build with Python 3.13 #1181

Cikmo commented Sep 15, 2024

musicinmybrain commented Sep 28, 2024 •

edited

Loading

kamikaze commented Oct 10, 2024

sylvainmouquet commented Oct 11, 2024 •

edited

Loading

juergenzimmermann commented Oct 12, 2024

Kwieeciol commented Oct 13, 2024

EwertonDCSilv commented Oct 17, 2024

EwertonDCSilv commented Oct 17, 2024

eirnym commented Oct 17, 2024

musicinmybrain commented Oct 17, 2024

elprans commented Oct 17, 2024

musicinmybrain commented Oct 17, 2024

elprans commented Oct 20, 2024

EwertonDCSilv commented Oct 24, 2024

fails to build with Python 3.13 #1181

fails to build with Python 3.13 #1181

Comments

Cikmo commented Sep 15, 2024

musicinmybrain commented Sep 28, 2024 • edited Loading

kamikaze commented Oct 10, 2024

sylvainmouquet commented Oct 11, 2024 • edited Loading

juergenzimmermann commented Oct 12, 2024

Kwieeciol commented Oct 13, 2024

EwertonDCSilv commented Oct 17, 2024

EwertonDCSilv commented Oct 17, 2024

eirnym commented Oct 17, 2024

musicinmybrain commented Oct 17, 2024

elprans commented Oct 17, 2024

musicinmybrain commented Oct 17, 2024

elprans commented Oct 20, 2024

EwertonDCSilv commented Oct 24, 2024

musicinmybrain commented Sep 28, 2024 •

edited

Loading

sylvainmouquet commented Oct 11, 2024 •

edited

Loading