When having on --with-system-expat, system still builds using bundled expat for tests #92875

mcepl · 2022-05-17T10:22:55Z

Bug report

When building 3.11.0b1 for openSUSE (openSUSE/Tumbleweed as of 2022-05-17) we have --with-system-expat on (of course) and just to be sure we also rm -rf Modules/expat (to be sure, we really don't use bundled expat). Unfortunately, in such situation test_freeze_simple_script in test.test_tools.test_freeze.TestFreeze fails:

[ 1020s] test_freeze_simple_script (test.test_tools.test_freeze.TestFreeze.test_freeze_simple_script) ... creating the script to be frozen at /tmp/tmpl92b_12m/app.py
[ 1020s] copying the source tree into /tmp/tmpl92b_12m/cpython...
[ 1020s] configuring python in /tmp/tmpl92b_12m/python-build...
[ 1020s] building python in /tmp/tmpl92b_12m/python-build...
[ 1020s] CalledProcessError: Command '['/usr/bin/make', '-C', '/tmp/tmpl92b_12m/python-build', '-j8']' returned non-zero exit status 2.

[Plenty of lines of gcc spew skipped]

[ 1139s] make[1]: Entering directory '/tmp/tmpraer2tjx/python-build'
[ 1139s] gcc -c -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall    -std=c11 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden  -I/tmp/tmpraer2tjx/cpython/Include/internal -IObjects -IInclude -IPython -I. -I/tmp/tmpraer2tjx/cpython/Include    -fPIC -I/tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec -DCONFIG_64=1 -DANSI=1 -DHAVE_UINT128_T=1 -o Modules/_decimal/libmpdec/mpdecimal.o /tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec/mpdecimal.c
[ 1139s] make[1]: Leaving directory '/tmp/tmpraer2tjx/python-build'
[ 1139s] 
[ 1139s] --- STDERR ---
[ 1139s] /tmp/tmpraer2tjx/cpython/Python/sysmodule.c: In function ‘PySys_SetArgv’:
[ 1139s] /tmp/tmpraer2tjx/cpython/Python/sysmodule.c:3305:5: warning: ‘PySys_SetArgvEx’ is deprecated [-Wdeprecated-declarations]
[ 1139s]  3305 |     PySys_SetArgvEx(argc, argv, Py_IsolatedFlag == 0);
[ 1139s]       |     ^~~~~~~~~~~~~~~
[ 1139s] /tmp/tmpraer2tjx/cpython/Python/sysmodule.c:3259:1: note: declared here
[ 1139s]  3259 | PySys_SetArgvEx(int argc, wchar_t **argv, int updatepath)
[ 1139s]       | ^~~~~~~~~~~~~~~
[ 1139s] make[1]: *** No rule to make target '/tmp/tmpraer2tjx/cpython/Modules/expat/xmlparse.c', needed by 'Modules/expat/xmlparse.o'.  Stop.
[ 1139s] make[1]: *** Waiting for unfinished jobs....
[ 1139s] In function ‘word_to_string’,
[ 1139s]     inlined from ‘coeff_to_string’ at /tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec/io.c:410:13,
[ 1139s]     inlined from ‘_mpd_to_string’ at /tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec/io.c:611:18:
[ 1139s] /tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec/io.c:348:40: warning: array subscript 0 is outside array bounds of ‘char[0]’ [-Warray-bounds]
[ 1139s]   348 |         if (s == dot) *s++ = '.'; *s++ = '0' + (char)(x / d); x %= d
[ 1139s]       |                                   ~~~~~^~~~~~~~~~~~~~~~~~~~~
[ 1139s] /tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec/io.c:355:14: note: in expansion of macro ‘EXTRACT_DIGIT’
[ 1139s]   355 |     case 19: EXTRACT_DIGIT(s, x, 1000000000000000000ULL, dot);
[ 1139s]       |              ^~~~~~~~~~~~~
[ 1139s] In function ‘word_to_string’,
[ 1139s]     inlined from ‘coeff_to_string’ at /tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec/io.c:410:13,
[ 1139s]     inlined from ‘_mpd_to_string’ at /tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec/io.c:607:18:
[ 1139s] /tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec/io.c:348:40: warning: array subscript 0 is outside array bounds of ‘char[0]’ [-Warray-bounds]
[ 1139s]   348 |         if (s == dot) *s++ = '.'; *s++ = '0' + (char)(x / d); x %= d
[ 1139s]       |                                   ~~~~~^~~~~~~~~~~~~~~~~~~~~
[ 1139s] /tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec/io.c:355:14: note: in expansion of macro ‘EXTRACT_DIGIT’
[ 1139s]   355 |     case 19: EXTRACT_DIGIT(s, x, 1000000000000000000ULL, dot);
[ 1139s]       |              ^~~~~~~~~~~~~
[ 1139s] In function ‘word_to_string’,
[ 1139s]     inlined from ‘coeff_to_string’ at /tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec/io.c:410:13,
[ 1139s]     inlined from ‘_mpd_to_string’ at /tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec/io.c:501:22:
[ 1139s] /tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec/io.c:348:40: warning: array subscript 0 is outside array bounds of ‘char[0]’ [-Warray-bounds]
[ 1139s]   348 |         if (s == dot) *s++ = '.'; *s++ = '0' + (char)(x / d); x %= d
[ 1139s]       |                                   ~~~~~^~~~~~~~~~~~~~~~~~~~~
[ 1139s] /tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec/io.c:355:14: note: in expansion of macro ‘EXTRACT_DIGIT’
[ 1139s]   355 |     case 19: EXTRACT_DIGIT(s, x, 1000000000000000000ULL, dot);
[ 1139s]       |              ^~~~~~~~~~~~~
[ 1139s] /tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec/io.c:348:40: warning: array subscript 0 is outside array bounds of ‘char[0]’ [-Warray-bounds]
[ 1139s]   348 |         if (s == dot) *s++ = '.'; *s++ = '0' + (char)(x / d); x %= d
[ 1139s]       |                                   ~~~~~^~~~~~~~~~~~~~~~~~~~~
[ 1139s] /tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec/io.c:356:14: note: in expansion of macro ‘EXTRACT_DIGIT’
[ 1139s]   356 |     case 18: EXTRACT_DIGIT(s, x, 100000000000000000ULL, dot);
[ 1139s]       |              ^~~~~~~~~~~~~
[ 1139s] 
[ 1139s] ---- END ----
[ 1139s] ERROR
[ 1139s] 
[ 1139s] ======================================================================
[ 1139s] ERROR: test_freeze_simple_script (test.test_tools.test_freeze.TestFreeze.test_freeze_simple_script)
[ 1139s] ----------------------------------------------------------------------
[ 1139s] Traceback (most recent call last):
[ 1139s]   File "/home/abuild/rpmbuild/BUILD/Python-3.11.0b1/Lib/test/test_tools/test_freeze.py", line 27, in test_freeze_simple_script
[ 1139s]     outdir, scriptfile, python = helper.prepare(script, outdir)
[ 1139s]                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ 1139s]   File "/home/abuild/rpmbuild/BUILD/Python-3.11.0b1/Tools/freeze/test/freeze.py", line 159, in prepare
[ 1139s]     _run_quiet([MAKE, '-C', builddir, '-j8'])
[ 1139s]     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ 1139s]   File "/home/abuild/rpmbuild/BUILD/Python-3.11.0b1/Tools/freeze/test/freeze.py", line 25, in _run_quiet
[ 1139s]     return subprocess.run(
[ 1139s]            ^^^^^^^^^^^^^^^
[ 1139s]   File "/home/abuild/rpmbuild/BUILD/Python-3.11.0b1/Lib/subprocess.py", line 558, in run
[ 1139s]     raise CalledProcessError(retcode, process.args,
[ 1139s]     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ 1139s] subprocess.CalledProcessError: Command '['/usr/bin/make', '-C', '/tmp/tmpraer2tjx/python-build', '-j8']' returned non-zero exit status 2.
[ 1139s]

Complete build log with all details of building the package.

The text was updated successfully, but these errors were encountered:

mcepl · 2022-05-17T17:44:57Z

OK, so it is enough just to set CONFIG_ARGS variable to something like "--enable-ipv6 --enable-shared --with-ensurepip=no --with-system-ffi --with-system-expat --with-lto --enable-optimizations --with-system-libmpdec --enable-loadable-sqlite-extensions".

mcepl · 2022-05-17T18:42:34Z

Nope, it doesn't help. When I remove the bundled expat it fails just the same even with CONFIG_ARGS variable set.

sharewax · 2022-08-12T21:30:19Z

One more report for it.
Trying to build SCL for Centos 7, issue is the same:

make: Leaving directory `/tmp/tmpzdbz0bvg/python-build'
--- STDERR ---
/tmp/tmpzdbz0bvg/cpython/Objects/obmalloc.c:1449:1: warning: always_inline function might not be inlinable [-Wattributes]
 1449 | arena_map_get(block *p, int create)
      | ^~~~~~~~~~~~~
make: *** No rule to make target `/tmp/tmpzdbz0bvg/cpython/Modules/expat/xmlparse.c', needed by `Modules/expat/xmlparse.o'.  Stop.
make: *** Waiting for unfinished jobs....
---- END ----
ERROR
======================================================================
ERROR: test_freeze_simple_script (test.test_tools.test_freeze.TestFreeze.test_freeze_simple_script)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/Python-3.11.0rc1/Lib/test/test_tools/test_freeze.py", line 27, in test_freeze_simple_script
    outdir, scriptfile, python = helper.prepare(script, outdir)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/builddir/build/BUILD/Python-3.11.0rc1/Tools/freeze/test/freeze.py", line 159, in prepare
    _run_quiet([MAKE, '-C', builddir, '-j8'])
  File "/builddir/build/BUILD/Python-3.11.0rc1/Tools/freeze/test/freeze.py", line 25, in _run_quiet
    return subprocess.run(
           ^^^^^^^^^^^^^^^
  File "/builddir/build/BUILD/Python-3.11.0rc1/Lib/subprocess.py", line 569, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/usr/bin/make', '-C', '/tmp/tmpzdbz0bvg/python-build', '-j8']' returned non-zero exit status 2.
----------------------------------------------------------------------
Ran 1 test in 78.506s
FAILED (errors=1)
test test_tools failed
1 test failed again:
    test_tools
== Tests result: FAILURE then FAILURE ==
409 tests OK.
10 slowest tests:
- test_concurrent_futures: 2 min 36 sec
- test_multiprocessing_spawn: 2 min 28 sec
- test_subprocess: 2 min 27 sec
- test_asyncio: 1 min 36 sec
- test_tools: 1 min 36 sec
- test_multiprocessing_forkserver: 1 min 28 sec
- test_multiprocessing_fork: 1 min 16 sec
- test_signal: 49.0 sec
- test_io: 35.8 sec
- test_compileall: 33.4 sec
1 test failed:
    test_tools
24 tests skipped:
    test_curses test_devpoll test_idle test_ioctl test_kqueue
    test_launcher test_msilib test_ossaudiodev test_smtpnet
    test_socketserver test_startfile test_tcl test_tix test_tk
    test_ttk_guionly test_ttk_textonly test_turtle test_urllib2net
    test_urllibnet test_winconsoleio test_winreg test_winsound
    test_xmlrpc_net test_zipfile64
1 re-run test:
    test_tools
Total duration: 4 min 27 sec
Tests result: FAILURE then FAILURE

I removed the internal Modules/expat, but have no idea then it takes make target for xmlparse in this case.

# grep expat build.log
+ rm -r Modules/expat
checking for --with-system-expat... yes
checking for stdlib extension module pyexpat... yes
building 'pyexpat' extension
...
0:00:33 load avg: 7.96 [264/434] test_pyexpat passed -- running: test_asyncio (32.8 sec)
make: *** No rule to make target `/tmp/tmp02i4l09d/cpython/Modules/expat/xmlparse.c', needed by `Modules/expat/xmlparse.o'.  Stop.
make: *** No rule to make target `/tmp/tmpzdbz0bvg/cpython/Modules/expat/xmlparse.c', needed by `Modules/expat/xmlparse.o'.  Stop.

Looks like #91752

Removed rm -r Modules/expat from spec file - all compiled ok. Centos7.

mcepl added the type-bug An unexpected behavior, bug, or error label May 17, 2022

AlexWaygood added the tests Tests in the Lib/test dir label May 17, 2022

mcepl closed this as completed May 17, 2022

mcepl reopened this May 17, 2022

erlend-aasland added the build The build process and cross-build label May 23, 2022

mcepl mentioned this issue Jan 27, 2025

Move vendored modules into a vendor directory #129222

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

When having on --with-system-expat, system still builds using bundled expat for tests #92875

When having on --with-system-expat, system still builds using bundled expat for tests #92875

mcepl commented May 17, 2022 •

edited

Loading

mcepl commented May 17, 2022

mcepl commented May 17, 2022

sharewax commented Aug 12, 2022 •

edited

Loading

When having on --with-system-expat, system still builds using bundled expat for tests #92875

When having on --with-system-expat, system still builds using bundled expat for tests #92875

Comments

mcepl commented May 17, 2022 • edited Loading

mcepl commented May 17, 2022

mcepl commented May 17, 2022

sharewax commented Aug 12, 2022 • edited Loading

mcepl commented May 17, 2022 •

edited

Loading

sharewax commented Aug 12, 2022 •

edited

Loading