model training and prediction

[ssetty]

Assume a deep learning model is built using google tensorflow for image classification. For prediction gpu is required. How can Hazelcast JET be used for real time image classification? Model runs on gpu address memory space while Hazelcast JET in-memory works on normal CPU RAM?

[viliam-durina]

Jet can trigger a processing anywhere, on CPU or GPU. You can use mapUsingContextAsync to do the computation. We have sample using tensorflow to execute ML models, see https://github.com/hazelcast/hazelcast-jet-code-samples/tree/3.1-maintenance/integration/tensorflow.

[ssetty]

Then Jet just used as trigger mechanism. This will not serve any purpose. I can trigger using HTTP rest service. Question is assume model built using GPUs.- I guess JET cannot be used for training say thousands of images, trained model will be deployed only on CPU based machine for real time image classification (without GPU) - then JET serves no purpose except trigger mechanism for model prediction.
In short JET is not suitable for deep learning models which are computation intensive GPU based.

[viliam-durina]

Jet is a Stream processing engine. It basically moves data around, in a distributed and fault-tolerant way and is very efficient at doing this. It connects to sources, can transform, regroup, split, join, enrich and aggregate the data and write them to a destination, you define each step in a pipeline.

Image classification is a case of enrichment: an input image is classified and sent to output with the classification result. The classification itself is just triggered by Jet, as you said, and it can run either on CPU or GPU or FPGA or anywhere.

In your case you can use Jet to parallelize the classification in a cluster and to make it fault-tolerant. You can also use it to collect and prepare input for the classification from multiple sources. But the classification itself is done by an external library and Jet only calls it.