Bytes request payload

[Aleksandar1932]

Summary

This PR introduces functionality for request payload to be bytes. This is especially useful when sending requests for inference from applications which work with signal processing, i.e. vision or audio.

Example

in preprocessing.py, the body can be converted to an image with:

img = Image.open(io.BytesIO(body)).convert("RGB")

and later inference can be done with sending requests similar to:

curl --request POST \
  --url '<serving>/serve/<endpoint>?=<version>' \
  --header 'Content-Type: image/png' \
  --data '<bytes>' 

Test Plan

1. Build the docker image for serving, as per README
2. Create model preprocessing which expects bytes and converts them to the underlying data structure (ex. Image)
3. Send request

[bmartinn]

Thanks @Aleksandar1932
Looks good to me, let's just add a quick example, wdyt?

from typing import Any

import numpy as np
from PIL import Image, ImageOps


from clearml import StorageManager


# Notice Preprocess class Must be named "Preprocess"
class Preprocess(object):
    def __init__(self):
        # set internal state, this will be called only once. (i.e. not per request)
        pass

    def preprocess(self, body: bytes, state: dict, collect_custom_statistics_fn=None) -> Any:
        # we expect to get a stream of encoded image bytes
        try:
            img = Image.open(io.BytesIO(body)).convert("RGB")
        except Exception:
            raise ValueError("Image could not be decoded")

        image = ImageOps.grayscale(image).resize((28, 28))
        return np.array([np.array(image).flatten()])

    def postprocess(self, data: Any, state: dict, collect_custom_statistics_fn=None) -> dict:
        # post process the data returned from the model inference engine
        # data is the return value from model.predict we will put is inside a return value as Y
        if not isinstance(data, np.ndarray):
            # this should not happen
            return dict(digit=-1)

        # data is returned as probability per class (10 class/digits)
        return dict(digit=int(data.flatten().argmax()))

[Aleksandar1932]

Thanks @Aleksandar1932 Looks good to me, let's just add a quick example, wdyt?

from typing import Any

import numpy as np
from PIL import Image, ImageOps


from clearml import StorageManager


# Notice Preprocess class Must be named "Preprocess"
class Preprocess(object):
    def __init__(self):
        # set internal state, this will be called only once. (i.e. not per request)
        pass

    def preprocess(self, body: bytes, state: dict, collect_custom_statistics_fn=None) -> Any:
        # we expect to get a stream of encoded image bytes
        try:
            img = Image.open(io.BytesIO(body)).convert("RGB")
        except Exception:
            raise ValueError("Image could not be decoded")

        image = ImageOps.grayscale(image).resize((28, 28))
        return np.array([np.array(image).flatten()])

    def postprocess(self, data: Any, state: dict, collect_custom_statistics_fn=None) -> dict:
        # post process the data returned from the model inference engine
        # data is the return value from model.predict we will put is inside a return value as Y
        if not isinstance(data, np.ndarray):
            # this should not happen
            return dict(digit=-1)

        # data is returned as probability per class (10 class/digits)
        return dict(digit=int(data.flatten().argmax()))

I agree this would be very useful. Extended the current MNIST examples to cover this use case.