Skip to content

The os.sched_yield() implementation in posixmodule.c doesn't release the GIL #96078

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom
Closed
FilipDepta opened this issue Aug 18, 2022 · 2 comments
Closed

The os.sched_yield() implementation in posixmodule.c doesn't release the GIL #96078

FilipDepta opened this issue Aug 18, 2022 · 2 comments
Labels
3.10 only security fixes 3.11 only security fixes 3.12 only security fixes extension-modules C modules in the Modules dir type-bug An unexpected behavior, bug, or error

Comments

@FilipDepta
Copy link

I've reviewed the main branch and the GIL isn't released: https://github.com/python/cpython/blob/main/Modules/posixmodule.c#L7060

The forum thread: https://discuss.python.org/t/does-the-os-sched-yield-release-the-gil/18301

Expected: The GIL is released before sched_yield() and acquired after that.
@FilipDepta FilipDepta added the type-bug An unexpected behavior, bug, or error label Aug 18, 2022
@eryksun eryksun added extension-modules C modules in the Modules dir 3.11 only security fixes 3.10 only security fixes 3.12 only security fixes labels Aug 18, 2022
@eryksun
Copy link
Contributor

eryksun commented Aug 18, 2022

Interestingly, os.sched_yield() was added 11 years ago, and no one has ever mentioned this oversight. I guess the users, if any1, are satisfied that it can only yield execution to ready threads that aren't waiting to acquire the GIL. The behavior was never documented, so I'm tagging this as a bug in 3.10+.

Footnotes

  1. The man page for sched_yield() in Linux says the behavior is "unspecified" if the process isn't using a deterministic realtime scheduler such as SCHED_FIFO (first in, first out) or SCHED_RR (round robin). I'm guessing it's not common for Python applications to use realtime scheduling. ↩

@FilipDepta
Copy link
Author

FilipDepta commented Aug 19, 2022 •
edited
Loading

[...] using a deterministic realtime scheduler such as SCHED_FIFO (first in, first out) or SCHED_RR (round robin). I'm guessing it's not common for Python applications to use realtime scheduling. ↩

That is exactly what I do - using SHED_FIFO with two threads I sometimes call os.sched_yield() to ensure that the second thread was executed as freshly as possible.

Maybe it's a good idea to add an annotation in docs for older cpythons that os.sched_yield() doesn't release the GIL?

@bedevere-bot bedevere-bot mentioned this issue Oct 6, 2022
Merged
corona10 added a commit to corona10/cpython that referenced this issue Oct 6, 2022
@corona10
pythongh-96078: os.sched_yield release the GIL while calling sched_yi…
5f150a8 
…eld(2).
corona10 added a commit to corona10/cpython that referenced this issue Oct 6, 2022
@corona10
pythongh-96078: os.sched_yield release the GIL while calling sched_yi…
40c5725 
…eld(2).
carljm added a commit to carljm/cpython that referenced this issue Oct 6, 2022
@carljm
Merge branch 'main' into dictwatch
e5d2169 
* main:
  fixes pythongh-96078: os.sched_yield release the GIL while calling sched_yield(2). (pythongh-97965)
  pythongh-65961: Do not rely solely on `__cached__` (pythonGH-97990)
  pythongh-97850: Remove the open issues section from the import reference (python#97935)
  Docs: pin sphinx-lint (pythonGH-97992)
  pythongh-94590: add signatures to operator itemgetter, attrgetter, methodcaller (python#94591)
  Add Pynche's move to the What's new in 3.11 (python#97974)
  pythongh-97781: Apply changes from importlib_metadata 5. (pythonGH-97785)
  pythongh-86482: Document assignment expression need for ()s (python#23291)
  pythongh-97943: PyFunction_GetAnnotations should return a borrowed reference. (python#97949)
  pythongh-94808: Coverage: Test that maximum indentation level is handled (python#95926)
This was referenced Oct 7, 2022
Merged
Merged
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Oct 7, 2022
@corona10 @miss-islington
fixes pythongh-96078: os.sched_yield release the GIL while calling sc…
920e025 
…hed_yield(2). (pythongh-97965)

(cherry picked from commit b9d2e81)

Co-authored-by: Dong-hee Na <donghee.na@python.org>
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Oct 7, 2022
@corona10 @miss-islington
fixes pythongh-96078: os.sched_yield release the GIL while calling sc…
4a38944 
…hed_yield(2). (pythongh-97965)

(cherry picked from commit b9d2e81)

Co-authored-by: Dong-hee Na <donghee.na@python.org>
miss-islington added a commit that referenced this issue Oct 7, 2022
@miss-islington @corona10
fixes gh-96078: os.sched_yield release the GIL while calling sched_yi…
e39b511 
…eld(2). (gh-97965)

(cherry picked from commit b9d2e81)

Co-authored-by: Dong-hee Na <donghee.na@python.org>
miss-islington added a commit that referenced this issue Oct 7, 2022
@miss-islington @corona10
fixes gh-96078: os.sched_yield release the GIL while calling sched_yi…
11945f2 
…eld(2). (gh-97965)

(cherry picked from commit b9d2e81)

Co-authored-by: Dong-hee Na <donghee.na@python.org>
carljm added a commit to carljm/cpython that referenced this issue Oct 8, 2022
@carljm
Merge branch 'main' into typewatch
415ed49 
* main:
  pythonGH-97002: Prevent `_PyInterpreterFrame`s from backing more than one `PyFrameObject` (pythonGH-97996)
  pythongh-97973: Return all necessary information from the tokenizer (pythonGH-97984)
  fixes pythongh-96078: os.sched_yield release the GIL while calling sched_yield(2). (pythongh-97965)
  pythongh-65961: Do not rely solely on `__cached__` (pythonGH-97990)
  pythongh-97850: Remove the open issues section from the import reference (python#97935)
  Docs: pin sphinx-lint (pythonGH-97992)
  pythongh-94590: add signatures to operator itemgetter, attrgetter, methodcaller (python#94591)
  Add Pynche's move to the What's new in 3.11 (python#97974)
  pythongh-97781: Apply changes from importlib_metadata 5. (pythonGH-97785)
  pythongh-86482: Document assignment expression need for ()s (python#23291)
  pythongh-97943: PyFunction_GetAnnotations should return a borrowed reference. (python#97949)
mpage pushed a commit to mpage/cpython that referenced this issue Oct 11, 2022
@corona10 @mpage
fixes pythongh-96078: os.sched_yield release the GIL while calling sc…
0c970bd 
…hed_yield(2). (pythongh-97965)
@wbolster wbolster mentioned this issue Mar 7, 2024
Open
@socketpair socketpair mentioned this issue Jul 8, 2024
Closed
# for free to join this conversation on GitHub. Already have an account? # to comment
Assignees
No one assigned
Labels
3.10 only security fixes 3.11 only security fixes 3.12 only security fixes extension-modules C modules in the Modules dir type-bug An unexpected behavior, bug, or error
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@eryksun @FilipDepta