Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

Support for eventlet Python event loop #15923

Closed
gszpura opened this issue Jul 3, 2018 · 5 comments
Closed

Support for eventlet Python event loop #15923

gszpura opened this issue Jul 3, 2018 · 5 comments
Labels
disposition/stale kind/enhancement lang/Python priority/P3

Comments

@gszpura
Copy link

gszpura commented Jul 3, 2018 •
edited
Loading

grpc calls block eventlet's (0.22+) greenthreads. I think that's because libevent isn't supported by grpc.

What version of gRPC and what language are you using?

python 2.7.13+
grpc 1.12.1

What operating system (Linux, Windows, …) and version?

Linux (Ubuntu 14.04, 16.04)

What runtime / compiler are you using (e.g. python version or version of gcc)

python 2.7.13+

What did you do?

I've modified original helloworld example:
https://github.com/grpc/grpc/tree/master/examples/python/helloworld

The purpose was to run 2 greenthreads: one handling grpc call, second doing some computation
in the same time.

Simple client modification:

import eventlet
eventlet.monkey_patch()

import grpc
import helloworld_pb2
import helloworld_pb2_grpc


def run():
    channel = grpc.insecure_channel('localhost:50051')
    stub = helloworld_pb2_grpc.GreeterStub(channel)
    response = stub.SayHello(helloworld_pb2.HelloRequest(name='you'))
    print("Greeter client received: " + response.message)


def increment_thread():
    cnt = 0
    while True:
        eventlet.sleep(1)
        print("Incrementing", cnt)
        cnt += 1


eventlet.spawn(run)
eventlet.spawn(increment_thread)


while True:
    eventlet.sleep(1)

In server I've added some sleep to delay the response.

What did you expect to see?

I wanted to have those 2 threads running in the same time in non-blocking manner.

What did you see instead?

gRPC greenthread blocked the other thread.

Anything else we should know about your project / environment?

eventlet 0.22+
@kpayson64 kpayson64 assigned mehrdada and unassigned kpayson64 Jul 26, 2018
@bastova
Copy link

bastova commented Aug 1, 2018 •
edited
Loading

Hi, I have a very similar setup and seeing similar errors when trying to use grpc with gevent. From quick online search it seems like it's coming from the incompatibility between grpc and greenlet (even though initial experimental support for gevent was introduced to grpcio half a year ago).
My error stack:

2018-08-01 08:24:08,146 ERROR [root] request_id=None AuthMetadataPluginCallback "<google.auth.transport.grpc.AuthMetadataPlugin object at 0x152a040fe610>" raised exception!
Traceback (most recent call last):
  File "/usr/lib64/python2.7/site-packages/grpc/_plugin_wrapping.py", line 77, in __call__
    callback_state, callback))
  File "/usr/lib/python2.7/site-packages/google/auth/transport/grpc.py", line 77, in __call__
    callback(self._get_authorization_headers(context), None)
  File "/usr/lib/python2.7/site-packages/google/auth/transport/grpc.py", line 65, in _get_authorization_headers
    headers)
  File "/usr/lib/python2.7/site-packages/google/auth/credentials.py", line 122, in before_request
    self.refresh(request)
  File "/usr/lib/python2.7/site-packages/google/oauth2/service_account.py", line 322, in refresh
    request, self._token_uri, assertion)
  File "/usr/lib/python2.7/site-packages/google/oauth2/_client.py", line 145, in jwt_grant
    response_data = _token_endpoint_request(request, token_uri, body)
  File "/usr/lib/python2.7/site-packages/google/oauth2/_client.py", line 106, in _token_endpoint_request
    method='POST', url=token_uri, headers=headers, body=body)
  File "/usr/lib/python2.7/site-packages/google_auth_httplib2.py", line 116, in __call__
    url, method=method, body=body, headers=headers, **kwargs)
  File "/usr/lib64/python2.7/site-packages/newrelic-2.50.0.39/newrelic/api/external_trace.py", line 103, in dynamic_wrapper
    return wrapped(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/httplib2/__init__.py", line 1609, in request
    (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
  File "/usr/lib/python2.7/site-packages/httplib2/__init__.py", line 1351, in _request
    (response, content) = self._conn_request(conn, request_uri, method, body, headers)
  File "/usr/lib/python2.7/site-packages/httplib2/__init__.py", line 1307, in _conn_request
    response = conn.getresponse()
  File "/usr/lib64/python2.7/site-packages/newrelic-2.50.0.39/newrelic/hooks/external_httplib.py", line 65, in httplib_getresponse_wrapper
    return wrapped(*args, **kwargs)
  File "/usr/lib64/python2.7/httplib.py", line 1132, in getresponse
    response.begin()
  File "/usr/lib64/python2.7/httplib.py", line 453, in begin
    version, status, reason = self._read_status()
  File "/usr/lib64/python2.7/httplib.py", line 409, in _read_status
    line = self.fp.readline(_MAXLINE + 1)
  File "/usr/lib64/python2.7/socket.py", line 480, in readline
    data = self._sock.recv(self._rbufsize)
  File "/usr/lib64/python2.7/site-packages/gevent/_sslgte279.py", line 457, in recv
    return self.read(buflen)
  File "/usr/lib64/python2.7/site-packages/gevent/_sslgte279.py", line 318, in read
    self._wait(self._read_event, timeout_exc=_SSLErrorReadTimeout)
  File "src/gevent/_hub_primitives.py", line 265, in gevent.__hub_primitives.wait_on_socket
  File "src/gevent/_hub_primitives.py", line 266, in gevent.__hub_primitives.wait_on_socket
  File "src/gevent/_hub_primitives.py", line 245, in gevent.__hub_primitives._primitive_wait
ConcurrentObjectUseError: This socket is already used by another greenlet: <bound method Waiter.switch of <gevent.__waiter.Waiter object at 0x152a03bfe9f0>>

I am using:
grpcio==1.12.1
gevent==1.3.5
greenlet==0.4.14

@mehrdada
Copy link
Member

mehrdada commented Aug 1, 2018

@gszpura There are no current plans to extend support to other event loops like eventlet.

@bastova gevent support has been in alpha status and there are known issues (does not support libuv engine, for example), but if there are bugs using libev please file them separately.

@mehrdada mehrdada changed the title [gRPC python] Support for eventlet Support for eventlet Python event loop Aug 1, 2018
@bastova
Copy link

bastova commented Aug 1, 2018

I see, so mine is a different issue. Ok will force the 'libev' engine and open separate issues if it doesn't work. Thanks

@mehrdada mehrdada removed their assignment Aug 1, 2018
@vijaykramesh
Copy link

is there any option for using, e.g., google-cloud-spanner in a django app running with eventlet + gunicorn? I don't really want to have to change my entire app server just to support calling an API... but setting GOOGLE_CLOUD_DISABLE_GRPC doesn't seem to work for me

@stale
Copy link

stale bot commented Sep 4, 2019

This issue/PR has been automatically marked as stale because it has not had any update (including commits, comments, labels, milestones, etc) for 180 days. It will be closed automatically if no further update occurs in 1 day. Thank you for your contributions!

@stale stale bot closed this as completed Sep 5, 2019
@lock lock bot locked as resolved and limited conversation to collaborators Dec 4, 2019
Akrog added a commit to Akrog/grpc that referenced this issue Feb 18, 2020
@Akrog
Add compatibility for Python's Eventlet library
140d072 
Trying to use the gRPC library in a Python program that uses the
Eventlet library doesn't work as expected, and we end up with deadlocks
and blocked threads.

This patch adds a custom eventlet IO manager to provide compatibility
between the Eventlet and gRPC libraries.

The code includes 2 workarounds for existing eventlet bugs, but this
code is compatible with the proposed PRs to solve them, so there should
be no problem if/when implements a fix for them:

 - eventlet/eventlet#508
 - eventlet/eventlet#599

An extra import and call is necessary for applications to use gRPC with
eventlets.  This call must be done after eventlet has monkey patched the
Python libraries.

The code would look something like this:

    import eventlet
    eventlet.monkey_patch()  # noqa

    import grpc
    from grpc.experimental import eventlet as grpc_eventlet

    if __name__ == '__main__':
        grpc_eventlet.init_eventlet()

Resolve grpc#15923
Akrog added a commit to Akrog/grpc that referenced this issue Feb 19, 2020
@Akrog
Add compatibility for Python's Eventlet library
14d70c0 
Trying to use the gRPC library in a Python program that uses the
Eventlet library doesn't work as expected, and we end up with deadlocks
and blocked threads.

This patch adds a custom eventlet IO manager to provide compatibility
between the Eventlet and gRPC libraries.

The code includes 2 workarounds for existing eventlet bugs, but this
code is compatible with the proposed PRs to solve them, so there should
be no problem if/when implements a fix for them:

 - eventlet/eventlet#508
 - eventlet/eventlet#599

An extra import and call is necessary for applications to use gRPC with
eventlets.  This call must be done after eventlet has monkey patched the
Python libraries.

The code would look something like this:

    import eventlet
    eventlet.monkey_patch()  # noqa

    import grpc
    from grpc.experimental import eventlet as grpc_eventlet

    if __name__ == '__main__':
        grpc_eventlet.init_eventlet()

Resolve grpc#15923
Akrog added a commit to Akrog/grpc that referenced this issue Feb 26, 2020
@Akrog
Add compatibility for Python's Eventlet library
78087b1 
Trying to use the gRPC library in a Python program that uses the
Eventlet library doesn't work as expected, and we end up with deadlocks
and blocked threads.

This patch adds a custom eventlet IO manager to provide compatibility
between the Eventlet and gRPC libraries.

The code includes 2 workarounds for existing eventlet bugs, but this
code is compatible with the proposed PRs to solve them, so there should
be no problem if/when implements a fix for them:

 - eventlet/eventlet#508
 - eventlet/eventlet#599

An extra import and call is necessary for applications to use gRPC with
eventlets.  This call must be done after eventlet has monkey patched the
Python libraries.

The code would look something like this:

    import eventlet
    eventlet.monkey_patch()  # noqa

    import grpc
    from grpc.experimental import eventlet as grpc_eventlet

    if __name__ == '__main__':
        grpc_eventlet.init_eventlet()

Resolve grpc#15923
Akrog added a commit to Akrog/grpc that referenced this issue Mar 9, 2020
@Akrog
Add compatibility for Python's Eventlet library
3023219 
Trying to use the gRPC library in a Python program that uses the
Eventlet library doesn't work as expected, and we end up with deadlocks
and blocked threads.

This patch adds a custom eventlet IO manager to provide compatibility
between the Eventlet and gRPC libraries.

The code includes 2 workarounds for existing eventlet bugs, but this
code is compatible with the proposed PRs to solve them, so there should
be no problem if/when implements a fix for them:

 - eventlet/eventlet#508
 - eventlet/eventlet#599

An extra import and call is necessary for applications to use gRPC with
eventlets.  This call must be done after eventlet has monkey patched the
Python libraries.

The code would look something like this:

    import eventlet
    eventlet.monkey_patch()  # noqa

    import grpc
    from grpc.experimental import eventlet as grpc_eventlet

    if __name__ == '__main__':
        grpc_eventlet.init_eventlet()

Resolve grpc#15923
# for free to subscribe to this conversation on GitHub. Already have an account? #.
Assignees
No one assigned
Labels
disposition/stale kind/enhancement lang/Python priority/P3
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add compatibility for Python's Eventlet library Akrog/grpc
6 participants
@vijaykramesh @mehrdada @bastova @gszpura @kpayson64 @srini100