Functions Framework does not work with the Pub/Sub emulator

[RomikimoR]

Hello,

I am trying to emulate the behavior of my application.
To do so I use the Pub/Sub Emulator and the Google cloud function emulator.
The cloud function I created looks like the image following and is supposed to be triggered by an event :

Both emulators are communicating without issue.
I followed the example of this repository: https://github.com/GoogleCloudPlatform/functions-framework-nodejs to link them.

When, following the example, I use this command line:

curl -d "@mockPubsub.json" -X POST
-H "Ce-Type: true"
-H "Ce-Specversion: true"
-H "Ce-Source: true"
-H "Ce-Id: true"
-H "Content-Type: application/json"
http://localhost:8080

On my Cloud function emulator I do receive this :

So this is exactly what I want.
Yet, I do not want to use the command line to send messages but I want the cloud function to be triggered by a Pub/Sub event.

Here you can see the methods creating:
I. a pull subscription
II. a push subsciption
III. publishing messages

However When I want to publish messages, the Pub/Sub emulator detect the connection:

[pubsub] INFO: Detected HTTP/2 connection.

But then the Cloud Function emulator does not receive the data, although it detects an event occurred:

I think it is probably because POSTING to a Gcloud Function needs some headers with the 'Ce-' prefix.
But searching a bit on how to add headers to my push subscriber I found out this wasn't possible.
It's possible I totally missed something...

I hope this is understandable.
Thank you

[di]

Thanks for the report, I'll take a look at this and see what's going on.

[wojtek-viirtue]

+1 on this. I can confirm I'm seeing the same exact result for the same exact use case.

[isaldana]

It looks like when setting Ce-Type, Ce-Specversion, Ce-Source, and Ce-Id the request body is not JSON parsed. Unfortunately, when they are not set, only the data portion of the request are passed on the first parameter to the function. According to the samples for Pub/Sub-triggered functions, the first parameter should be a hash with the data and attributes fields. As is, it looks like attributes could not be accessed. I used the following command to test:

curl --header "Content-Type: application/json" --request POST --data '{"data":"This is a message", "attributes":{"k1":"v1"}}' http://0.0.0.0:8091/

on the following function

def main(event, context):
    print(event, context)

with the following output

127.0.0.1 - - [01/Mar/2020 01:40:42] "POST / HTTP/1.1" 200 -
This is a message {event_id: , timestamp: , event_type: , resource: }

[yurkovoznyak]

I've made fork of this repo, that supports events from Pub/Sub. Not the cleanest/best solution, but it works for my needs. Hope this may help
https://github.com/uraniun/functions-framework-python

[RomikimoR]

I will try the fork!! Thank you a lot I really needed a patch even not the cleanest for this !

[grant]

Thanks for the report and workaround. As Dustin stated, we'll look into this (it may be a similar issue with other frameworks).

[di]

Just a quick update here: we've decided to add support for these legacy-style Pub/Sub events. Hang tight while we roll this out, and thanks for your patience.

[dror-g]

Hi, not sure but I think it's related:
When using the Python function code example from Pub/Sub Tutorial
I get an error when using curl to emulate an event (from nodejs framework, same as OP).

main.py", line 17, in hello if 'data' in event: TypeError: a bytes-like object is required, not 'str'

cheers

[di]

@dror-g It looks like your function expects a Pub/Sub message but you're calling it with headers that indicate that it's a binary CloudEvent instead.

Try calling it like so:

$ curl -d '{"data": {"hi": "there"}}' -X POST -H "Content-Type: application/json" http://localhost:8080

Where the value of data is what eventually becomes the event object in your function.

[dror-g]

Hi @di , yes, like OP I am trying to emulate Pub/Sub triggered function, and we both tried to post a mock.json file.
functions-framework seems to be working as long as you BASE64 encode the body (as it is in the cloud) and enclose it in {"data": {"data": "$B64_string"} }.

Tried this function:

from json import loads
from base64 import b64decode


def wak(event, context):
    print("EVENT", event)
    if 'data' in event:
        data = loads(b64decode(event['data']).decode('utf-8'))
        print('data: ', data)
    else:
        raise RuntimeError('no data in event')
    return

What works:

• started with functions-framework --target wak --signature-type event
  calling with: DATA=$(printf '{"some": "thing"}'|base64 -w 0) && curl -d '{"data": {"data": "'$DATA'"}}' -X POST -H "Content-Type: application/json" http://localhost:8080

• Deploying to the cloud and sending a message from Pub/Sub web UI (message body: {"some": "thing"} ).

• calling from gcloud cli with following command:
  ` DATA=$(printf '{"some": "thing"}'|base64 -w 0) && gcloud functions call testFunc --region europe-west2 --data '{"data": "'$DATA'"}'

• created notification on a bucket, uploaded file to bucket (gsutil notification create -t test-topic -f json gs://test-func-bucket2).

What doesn't:
started with functions-framework --target wak --signature-type event

mock.json:

{
  "message": {
    "attributes": {
      "key": "value"
    },
    "data": "SGVsbG8gQ2xvdWQgUHViL1N1YiEgSGVyZSBpcyBteSBtZXNzYWdlIQ==",
    "messageId": "136969346945"
  },
  "subscription": "projects/myproject/subscriptions/mysubscription"
} 

• curl -d "@mock.json" -X POST -H "Content-Type: application/json" http://localhost:8080 -
  EVENT is null, getting:

RuntimeError: no data in event
127.0.0.1 - - [21/Apr/2020 20:36:30] "POST / HTTP/1.1" 500 -

• curl -d '{"hi": "there"}' -X POST -H "Content-Type: application/json" http://localhost:8080 -
  EVENT is null, getting:

RuntimeError: no data in event
127.0.0.1 - - [21/Apr/2020 20:36:30] "POST / HTTP/1.1" 500 -

• curl --data '{"data": {"hi": "there"}}' -X POST -H "Content-Type: application/json" http://localhost:8080
  Returns:

EVENT {'hi': 'there'}
RuntimeError: no data in event
127.0.0.1 - - [21/Apr/2020 20:50:45] "POST / HTTP/1.1" 500 -

[27Bslash6]

Thanks for the report and workaround. As Dustin stated, we'll look into this (it may be a similar issue with other frameworks).

I had this same issue working with the nodejs pubsub client for cloud functions

[mikebridge]

Just so I'm clear, is this an issue with the functions framework, the pubsub emulator, someone's understanding of the correct payload format, or the (adapted) documentation around the technique for connecting the emulator to the functions framework? I'd like to try using the functions framework as part of an end-to-end test with the pubsub emulator but I'm not clear whether this would be a blocking issue.

Or, after reading @di 's comment, does this only affect legacy triggers?

[mikebridge]

Google's "getting started" documentation describes the event as a python dict, but I can see that when posting via curl with the OP's test script will pass the event as a str. So is the bug then with the http testing endpoint?

Maybe the easiest workaround is to write a wrapper function just for testing that json.loads the event and passes it to the function?

def hello_pubsub_test_wrapper(event_raw, context):
    return hello_pubsub(json.loads(event_raw), context)

def hello_pubsub(event: dict, context):
    """Background Cloud Function to be triggered by Pub/Sub.
    Args:
         event (dict):  The dictionary with data specific to this type of
         event. The `data` field contains the PubsubMessage message. The
         `attributes` field will contain custom attributes if there are any.
         context (google.cloud.functions.Context): The Cloud Functions event
         metadata. The `event_id` field contains the Pub/Sub message ID. The
         `timestamp` field contains the publish time.
    """
    import base64

    print("""This Function was triggered by messageId {} published at {}
    """.format(context.event_id, context.timestamp))
    if 'data' in event:
        name = base64.b64decode(event['data']).decode('utf-8')
    else:
        name = 'World'
    print('Hello {}!'.format(name))

functions-framework --target=hello_pubsub_test_wrapper --signature-type=event

echo -n "test" | base64 
=>
dGVzdA==

mockPubsub.json:

{
  "data": "dGVzdA==",
  "messageId": "136969346945"
}

curl -d @mockPubsub.json -X POST \ 
    -H "Ce-Type: true" \
    -H "Ce-Specversion: true" \
    -H "Ce-Source: true" \
    -H "Ce-Id: true" \
    -H "Content-Type: application/json" \
    http://localhost:8080

This yields: Hello test!.

[mikebridge]

The more I read the google documentation, the more frustrated and confused I get. Am I mistaken or is it full of errors? As far as I can tell from running the example I just posted, the data payload is just a plain old string, and it's delivered to my function in a Python dict.

But here it says "When using JSON over REST, message data must be base64-encoded. ", but then nowhere in the examples does it do anything with base64 encoding. And when using the functions-framework, I actually see no reason to just base64-encode the value in the key/value pair---it just gets passed through as a string. Similarly, when I publish the function above, it's just a string. It gets worse---the example encodes the data as a byte-string:

future = publisher.publish(
        topic_path, data=data.encode("utf-8")  # data must be a bytestring.
    )

Soooo... this example uses is a sequence of bytes, not a base64-encoded string. Is there something that's base64-encoding them behind the scenes? I have no idea, because there seems to be no single source of truth.

Anyway, I removed base64 encoding from that example and it seems to work fine when I deploy it to GCP and send it an un-base64-encoded string:

gcloud functions call hello_pubsub --project=my-project --data '{"data": "Not Base64 Encoded"}'

The functions-framework seems to pass it along too. I have no idea if I've done any of this correctly or not, but it seems to work.

[jacobg]

I'm also having an issue connecting the emulator to functions-framework. I'm publish a message with data={"foo": "bar"}, and the functions-framework shows the request comes in with the header Content-Type: application/json and body of:

{"subscription":"projects\\/myproject\\/subscriptions\\/mysubscription","message":{"data":"eyJmb28iOiAiYmFyIn0=","messageId":"52","attributes":{}}}

And yet when my function is called:

def receive_event(event, context=None):
    print('receive_event(event={}, context={}'.format(event, context))
    print('receive_event event type {}'.format(type(event)))

It prints:

receive_event(event=, context={event_id: , timestamp: , event_type: , resource: }
receive_event event type <class 'str'>

Is the posted json above using this legacy format you're referring to? From the snippets pasted in earlier comments on this issue thread, it seems that perhaps data and attributes are no longer embedded in a message attribute? Still, there is no attributes key referenced in the functions framework.

I am publishing using an old version of google-cloud-pubsub version 0.27.0, because I'm publishing from a Python 2 GAE app in local development. It's the last version that supports that environment, since the subsequent version uses grpc. See this issue:
googleapis/google-cloud-python#5670 (comment)

I'm probably going to just fork the functions-framework to make it work in this environment.

[jacobg]

Editing the _run_legacy_event function as follows seems to fix the issue:

def _run_legacy_event(function, request):
    event_data = request.get_json()
    if not event_data:
        flask.abort(400)

    ###############
    # Need to add this code to make pubsub events work.
    if 'message' in event_data:
        if 'data' not in event_data:
            message = event_data['message']
            event_data['data'] = {
                'data': message.get('data'),
                'attributes': message.get('attributes')
            }
    ###############

    event_object = _Event(**event_data)
    data = event_object.data
    context = Context(**event_object.context)
    function(data, context)

[gabriel-young]

@di, any update on this? Looks like @jacobg's comment may be the solution for this bug?

[lrodorigo]

I can confirm that the bug is still there.
Trying to use functions-framork to emulate PubSub events coming from Google IoT Core.
The content of the event parameter is COMPLETELY DIFFERENT between the functions framework and the runtime.
While using the functions-framework only the content of the event parameter is only the value of the data key set in the request (base64-encoded string), while in the actual 'true' runtime data is a dict containing the both, device attributes dict AND the data key.

If I try to put the following headers on the functions-framework request:

Ce-Type: false
Ce-Specversion: true
Ce-Source: true
Ce-Id: true

then, I get also the attribute, but it is not a dict (like in the true environment) but it is a serialized-json instead... so without modifying the function framework sources, the only workaround I found is to add the following, and send the curl request with the previous headers:

    if not isinstance(event,dict):
        event = json.loads(event)

Totally unusable. I expected better, really better from Google.

[anguillanneuf]

Thanks all @RomikimoR @wojtek-viirtue @isaldana @Uraniun @dror-g @27Bslash6 @mikebridge @jacobg @gabriel-young @lrodorigo for your patience!

Functions Frameworks is now working with Pub/Sub Emulator! 🎉

Something like below will work:

# Terminal 1
gcloud beta emulators pubsub start --project=abc --host-port=localhost:8080

# Terminal 2
functions-framework --target=hello --debug --port=8082

# Terminal 3
export PUBSUB_PROJECT_ID=abc
$(gcloud beta emulators pubsub env-init)
git clone https://github.com/googleapis/python-pubsub.git
cd samples/snippets/
python publisher.py abc create may
python subscriber.py abc create-push may five http://localhost:8082
python publisher.py abc publish may

Thanks again @matthewrobertson.

[frankie567]

Thanks @anguillanneuf for the added documentation about this.

However, I'm a bit confused here. The examples you show suppose the Cloud Function expects a request argument, which sounds to be a Flask Request:

def hello(request):
    return "Hello world!"

However, unless there is something I miss here, the official way of implementing a Pub/Sub Cloud Function tells us to define function with two arguments: event_data, a dictionary containing Base64-encoded data, and context, a google.cloud.functions.Context object:

def hello_pubsub(event, context):
    import base64

    print("""This Function was triggered by messageId {} published at {} to {}
    """.format(context.event_id, context.timestamp, context.resource["name"]))

    if 'data' in event:
        name = base64.b64decode(event['data']).decode('utf-8')
    else:
        name = 'World'
    print('Hello {}!'.format(name)))

If I try to debug those kind of functions using with the framework and the Pub/Sub emulator, I get this kind of error:

TypeError: argument of type 'Request' is not iterable

Obviously because we get a Request object instead of a dictionary.

So, what is the right way to debug a Pub/Sub Cloud Function?

[anguillanneuf]

@frankie567 Oh no, this issue was closed and I did not see your comment until now! Part of what you describe sounds like a feature request for Functions Frameworks (FF). I opened a new issue #141 and cc'ed you there.

So, what is the right way to debug a Pub/Sub Cloud Function?

For local testing, the Pub/Sub emulator and FF should suffice if you are okay with using request. (TODO: add a help section in README that shows how to adapt request to event, context).
For remote testing, the official guide you linked is up to date. I see why you are confused because it does mean one needs to update their locally tested code when deploying the function.
Alternatively, you may consider writing some unit and system tests, as described here.
Once the function is deployed, the UI offers a lot actually. I always do a test in Cloud Console under "Testing" just to make sure.

[di]

Reopening this until we can update the example to use a background function instead of a HTTP function per #141.