Skip to content

gh-101282: move BOLT config after PGO #104493

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Merged
merged 1 commit into from
May 15, 2023

Conversation

indygreg
Copy link
Contributor

@indygreg indygreg commented May 15, 2023

A future commit will make bolt's configure logic depend on PGO state. I'm moving the block in a standalone commit to make the diff easier to read.

@indygreg
Copy link
Contributor Author

indygreg commented May 15, 2023

FWIW this change builds up to a build system refactor that fixes some bugs/quirks with BOLT and enables the use of BOLT on libpython, which results in significant pyperformance impact. The remaining rough set of commits are available in #101093.

@corona10 corona10 changed the title gh-104492: move BOLT config after PGO gh-101282: move BOLT config after PGO May 15, 2023
Copy link
Member

@corona10 corona10 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with nit comment
But for future work, I would like to say.
For the BOLT, it could also be able to be enabled without enabling PGO. (Might be a weird configuration, but someone may need it)

A future commit will make bolt's configure logic depend on PGO
state. I'm moving the block in a standalone commit to make the
diff easier to read.

skip news
@indygreg indygreg force-pushed the build-bolt-after-pgo branch from 077503c to f9fdde5 Compare May 15, 2023 04:00
@corona10 corona10 enabled auto-merge (squash) May 15, 2023 04:00
@indygreg
Copy link
Contributor Author

LGTM with nit comment But for future work, I would like to say. For the BOLT, it could also be able to be enabled without enabling PGO. (Might be a weird configuration, but someone may need it)

Agree that we should support {none, either, both} configs. But it may require a bit of work to get there. I'll keep this in mind and try to implement it. I think the refactoring I'm doing to the PGO/BOLT make rules should make this vastly easier to achieve.

@corona10 corona10 merged commit 27d8ecd into python:main May 15, 2023
@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot wasm32-emscripten node (dynamic linking) 3.x has failed when building commit 27d8ecd.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/1056/builds/2009) and take a look at the build logs.
  4. Check if the failure is related to this commit (27d8ecd) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/1056/builds/2009

Failed tests:

  • test_bytes

Summary of the results of the build (if available):

== Tests result: FAILURE ==

335 tests OK.

10 slowest tests:

  • test_hashlib: 2 min 2 sec
  • test_math: 1 min 48 sec
  • test_tokenize: 1 min 13 sec
  • test_unparse: 47.8 sec
  • test_lib2to3: 45.5 sec
  • test_capi: 32.6 sec
  • test_tarfile: 25.0 sec
  • test_fstring: 23.6 sec
  • test_unicodedata: 21.4 sec
  • test_pickle: 17.0 sec

1 test failed:
test_bytes

127 tests skipped:
test.test_asyncio.test_base_events
test.test_asyncio.test_buffered_proto
test.test_asyncio.test_context
test.test_asyncio.test_eager_task_factory
test.test_asyncio.test_events test.test_asyncio.test_futures
test.test_asyncio.test_futures2 test.test_asyncio.test_locks
test.test_asyncio.test_pep492
test.test_asyncio.test_proactor_events
test.test_asyncio.test_protocols test.test_asyncio.test_queues
test.test_asyncio.test_runners
test.test_asyncio.test_selector_events
test.test_asyncio.test_sendfile test.test_asyncio.test_server
test.test_asyncio.test_sock_lowlevel test.test_asyncio.test_ssl
test.test_asyncio.test_sslproto test.test_asyncio.test_streams
test.test_asyncio.test_subprocess
test.test_asyncio.test_taskgroups test.test_asyncio.test_tasks
test.test_asyncio.test_threads test.test_asyncio.test_timeouts
test.test_asyncio.test_transports
test.test_asyncio.test_unix_events test.test_asyncio.test_waitfor
test.test_asyncio.test_windows_events
test.test_asyncio.test_windows_utils test__xxinterpchannels
test__xxsubinterpreters test_asyncgen test_clinic test_cmd_line
test_concurrent_futures test_contextlib_async test_ctypes
test_curses test_dbm_gnu test_dbm_ndbm test_devpoll test_doctest
test_docxmlrpc test_dtrace test_embed test_epoll test_faulthandler
test_fcntl test_file_eintr test_fork1 test_ftplib test_gdb
test_grp test_httplib test_httpservers test_idle test_imaplib
test_interpreters test_ioctl test_kqueue test_launcher test_lzma
test_mmap test_msilib test_multiprocessing_fork
test_multiprocessing_forkserver test_multiprocessing_main_handling
test_multiprocessing_spawn test_nis test_openpty test_ossaudiodev
test_pdb test_peg_generator test_perf_profiler test_poll
test_poplib test_pty test_pwd test_queue test_readline
test_regrtest test_repl test_resource test_select test_selectors
test_smtplib test_smtpnet test_socket test_socketserver test_spwd
test_ssl test_stable_abi_ctypes test_startfile test_subprocess
test_sys_settrace test_syslog test_tcl test_telnetlib test_thread
test_threadedtempfile test_threading test_threading_local test_tix
test_tkinter test_tools test_ttk test_ttk_textonly test_turtle
test_urllib2 test_urllib2_localnet test_urllib2net test_urllibnet
test_venv test_wait3 test_wait4 test_webbrowser test_winconsoleio
test_winreg test_winsound test_wmi test_wsgiref test_xmlrpc
test_xmlrpc_net test_zipfile64 test_zipimport_support
test_zoneinfo
0:25:49 load avg: 10.37
0:25:49 load avg: 10.37 Re-running failed tests is not supported with --python host runner option.

Total duration: 25 min 49 sec

Click to see traceback logs
Traceback (most recent call last):
  File "/opt/buildbot/bcannon-wasm/3.x.bcannon-wasm.emscripten-node-dl/build/Lib/test/string_tests.py", line 341, in test_find_periodic_pattern
    self.checkequal(reference_find(p, text),
  File "/opt/buildbot/bcannon-wasm/3.x.bcannon-wasm.emscripten-node-dl/build/Lib/test/string_tests.py", line 66, in checkequal
    self.assertEqual(
AssertionError: -1 != 0

@arhadthedev

This comment was marked as duplicate.

@erlend-aasland
Copy link
Contributor

Post-merge review: LGTM

@indygreg: please don't force-push; it does not play well with reviews on GitHub. If you haven't already, please take a look at the devguide, specifically Life of a Pull Request

carljm added a commit to carljm/cpython that referenced this pull request May 15, 2023
* main: (29 commits)
  pythongh-101819: Fix _io clinic input for unused base class method stubs (python#104418)
  pythongh-101819: Isolate `_io` (python#101948)
  Bump mypy from 1.2.0 to 1.3.0 in /Tools/clinic (python#104501)
  pythongh-104494: Update certain Tkinter pack/place tests for Tk 8.7 errors (python#104495)
  pythongh-104050: Run mypy on `clinic.py` in CI (python#104421)
  pythongh-104490: Consistently define phony make targets (python#104491)
  pythongh-67056: document that registering/unregistering an atexit func from within an atexit func is undefined (python#104473)
  pythongh-104487: PYTHON_FOR_REGEN must be minimum Python 3.10 (python#104488)
  pythongh-101282: move BOLT config after PGO (pythongh-104493)
  pythongh-104469 Convert _testcapi/float.c to use AC (pythongh-104470)
  pythongh-104456: Fix ref leak in _ctypes.COMError (python#104457)
  pythongh-98539: Make _SSLTransportProtocol.abort() safe to call when closed (python#104474)
  pythongh-104337: Clarify random.gammavariate doc entry  (python#104410)
  Minor improvements to typing docs (python#104465)
  pythongh-87092: avoid gcc warning on uninitialized struct field in assemble.c (python#104460)
  pythonGH-71383: IDLE - Document testing subsets of modules (python#104463)
  pythongh-104454: Fix refleak in AttributeError_reduce (python#104455)
  pythongh-75710: IDLE - add docstrings and comments to editor module (python#104446)
  pythongh-91896: Revert some very noisy DeprecationWarnings for `ByteString` (python#104424)
  Add a mention of PYTHONBREAKPOINT to breakpoint() docs (python#104430)
  ...
carljm added a commit to carljm/cpython that referenced this pull request May 15, 2023
* main: (204 commits)
  pythongh-101819: Fix _io clinic input for unused base class method stubs (python#104418)
  pythongh-101819: Isolate `_io` (python#101948)
  Bump mypy from 1.2.0 to 1.3.0 in /Tools/clinic (python#104501)
  pythongh-104494: Update certain Tkinter pack/place tests for Tk 8.7 errors (python#104495)
  pythongh-104050: Run mypy on `clinic.py` in CI (python#104421)
  pythongh-104490: Consistently define phony make targets (python#104491)
  pythongh-67056: document that registering/unregistering an atexit func from within an atexit func is undefined (python#104473)
  pythongh-104487: PYTHON_FOR_REGEN must be minimum Python 3.10 (python#104488)
  pythongh-101282: move BOLT config after PGO (pythongh-104493)
  pythongh-104469 Convert _testcapi/float.c to use AC (pythongh-104470)
  pythongh-104456: Fix ref leak in _ctypes.COMError (python#104457)
  pythongh-98539: Make _SSLTransportProtocol.abort() safe to call when closed (python#104474)
  pythongh-104337: Clarify random.gammavariate doc entry  (python#104410)
  Minor improvements to typing docs (python#104465)
  pythongh-87092: avoid gcc warning on uninitialized struct field in assemble.c (python#104460)
  pythonGH-71383: IDLE - Document testing subsets of modules (python#104463)
  pythongh-104454: Fix refleak in AttributeError_reduce (python#104455)
  pythongh-75710: IDLE - add docstrings and comments to editor module (python#104446)
  pythongh-91896: Revert some very noisy DeprecationWarnings for `ByteString` (python#104424)
  Add a mention of PYTHONBREAKPOINT to breakpoint() docs (python#104430)
  ...
# for free to join this conversation on GitHub. Already have an account? # to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants