setproctitle crashes after a fork on MacOS #113
Comments
|
After some tests, the issue was introduced in version 1.3.0 (version 1.2.3 works). |
|
Despite appearances this issue isn't directly related to #111. It is a general issue common to all languages on macOS. See for example this SO question. The best explanation I saw can be found here. Does the official workaround: setting environment variable Note however that:
|
|
Yes, I feel like setting this environment variable is just a way to hide the problem, not a real fix. This error exists because there is a dangerous behavior that could cause other problems. Also using this would hide all the errors related to forking, not just the ones from your library. The fact that this behavior was introduced recently makes me feel like there could be a workaround, although I don't know enough about setproctitle and these low level APIs to really provide help. I found an article discussing some workarounds but I don't know if it's relevant. |
|
Well the usable advice from that article basically says "use spawn instead of fork or initialize everything before fork" which is exactly the workarounds you noticed in the comments of the repro code (e.g. calling The issue is that only the author of the final top level python code can (maybe) say whether the combination of threading and forking is safe in this particular case. In your repro code it is, but it is easy to inadvertently change this by adding some innocuous code or calling into some module. |
|
Calling |
|
Wouldn't it be possible to initialize everything on import rather than having to call I will look into switching to spawn but I think this would break many things, for now calling |
|
Good point. Also note this from the module README
So there is already a need for doing so for other reasons. @dvarrazzo should the module just call |
dvarrazzo
added a commit
that referenced
this issue
Aug 10, 2022
|
@gershnik it is probably a reasonable workaround, yes. Trying to reproduce the issue and then fix it the way suggested. |
|
So, early import caused side effects about which people moaned about in the past (see #45, #46 for instance). So it was a precise choice to make sure they wouldn't happen anymore - see 485a6a5 However the side effects only manifest on Linux, so I think it's reasonable to restore early init on macOS only. I'm preparing a MR... |
dvarrazzo
added a commit
that referenced
this issue
Aug 10, 2022
|
Looks great to me! Thank you! |
|
What is the procedure to install from source ? I trust you for the fix anyway. |
Looks like |
|
Thank you, I can confirm that this fixes my issue. |
|
setproctitle 1.3.2 released. Have fun! |
|
Hi, I believe this issue is still happening on 1.3.2, although you need to do something more on the forked process to trigger the crash. I've found that doing a call with requests library consistently triggers the segfault on the child. This particular snippet works on 1.3.1, but made it crash too with other code (e.g. using requests_kerberos) ... import os
import sys
import requests
import setproctitle # this import makes the child crash on 1.3.2
print(setproctitle.__version__)
pid = os.fork()
if not pid:
print('HTTP GET goole.com: ')
print(requests.get('https://google.com'))
sys.exit()
os.wait() |
|
@glic3rinu I have added your test in the CI but I don't see it failing there: 8d48f94 @gershnik are you able to advise about the issue, and in general about the stability of this feature? |
|
@dvarrazzo I am unable to reproduce this either in a clean environment or in the tests. Without more details it is impossible to say what it is and whether it is even caused by setproctitle. @glic3rinu Can you provide more details about what you observe and your environment? Do you see the same error that opened this bug ("... may have been in progress in another thread when fork() ...") or just plain segfault? Which Python version, MacOS etc. are you using? |
|
Hey @gershnik @dvarrazzo thanks for looking into this. Posting the full error report from Mac OS with all the details of the crash: |
|
@glic3rinu Thank you for the full crash report. On the first blush this doesn't seem to be anything related to setproctitle, (except that its presence or absence likely nudges another issue to sporadically manifest). There seems to be some issue with your Python installation - some shared library is broken or invalid. The code fails on loading of some shared library - unfortunately the crash report doesn't say which. |
|
yep, what you are saying @gershnik makes a lot of sense. I will do further testing on a clean python installation. |
|
Unfortunately this is still happening after reinstalling python3 and purging all site-packages: export PYTHONPATH=
rm -rf /usr/local/lib/python3.10/site-packages/*
brew install python3 # upgraded from 3.10.7 to 3.10.8
pip3 install requests setproctitleI believe requests imports a few c-extensions along. It is gonna be hard to find out which one it is causing the crash, for one will have to find out a way to reproduce it without requests :P. |
|
For what it worth I tried with Homebrew Python too (I usually use the natively installed one) and still no repro. |
|
It seems like Apple changed something in macOS 13.2 because what used to work perfectly and / or with Here's a minimal reproducer: import faulthandler
import os
import urllib.request
import setproctitle
faulthandler.enable()
if os.fork() == 0:
urllib.request.getproxies()Leads to: Commenting out setproctitle fixes it.
I've tried:
Do you want me to open a new ticket? |
|
Maybe we should just make this package no-op on macOS? |
|
that would be fine with me |
|
It reproduces for me too on 13.2. According to this the issue isn't limited to setproctitle and you might hit it with other libraries. However the workaround in that thread: running the Python interpreter with no_proxy=\* python3 test.pyfixes it for me for the repro you provided. @dvarrazzo the issue isn't specifically with py-setproctitle but with he fact that Python on Mac doesn't really support |
|
I am not a macOS user, and personally I think nobody should. I have no problem in releasing a package dropping every functionality from macOS and just leave it working on Linux. Is there any mode of work on macOS that is salvageable - detecting fork and make it no-op in that case - or should we go @hynek thank you for your repro. I think it would be better to create a new issue, yes. |
|
I suspect the number of people who rely on fork with no exec in Python on MacOS is tiny compared to regular folks who simply have many Python processes running and would benefit from this package. 😉 |
Unfortunately this can be difficult, when setproctitle is used in a 3rd party library for instance. I would prefer being able to disable it based on an environment variable, so that the end user can more easily control the behavior if needed. Also as a MacOS user I wouldn't mind always disabling it if it is too complicated to fix / maintain, however I would find it disappointing since the current version works for most people. |
|
Do we have a way to detect the "thin-ice condition", by looking at process pid/ppid etc? Maybe storing the pid on import into a static var, and avoid to operate if getpid() <> stored_pid? (I would prefer to avoid throwing the baby with the bathwater too). |
|
The problem is that all the "damage" is already done by So it appears that the only safe (now and in any future version of macOS) way of using the library together with plain fork would be either:
Or
None of this is unique to setproctitle. Use another library that touches Apple libraries in the parent prior to fork and you will have the same issue after. |
|
With regards to a scenario of another library using setproctitle without host process control I don't think this is a realistic one or one that should be catered to. A library is a guest in a host process house. It has no business changing the process title on its own any more than a guest painting the host's house a different color unasked. 😄 |
|
If forking and not exec'ing is a thing documented not to do on macOS, and a program does a thing that shouldn't do, and doing so it explodes... is this genuinely a setproctitle issue? If it is, either we fix it, or we rip off the feature. Otherwise, no action. |
|
It is kinda documented but not in any place a regular Python (or Node, or whatever not Apple) developer would think to look at or be aware of. It is also kinda documented for Python but again in not very visible places. See for example: https://docs.python.org/3/library/multiprocessing.html
My point is precisely that it is not a setproctitle issue. Beyond documenting that it is unsafe if used before fork (and potentially making it friendlier by not doing anything on import) I don't think any action is needed here. |
# for free
to join this conversation on GitHub.
Already have an account?
# to comment
Using this code, setproctitle crashes after a forked process is started in a thread on Mac OS.
I get these logs:
Python version: 3.7.8
Setproctitle version: 1.3.1
Mac OS version: 12.5
Maybe #111 is related ?
The text was updated successfully, but these errors were encountered: