README.md

Flow 101

Under Construction: Check back soon

Due to ongoing changes to Fn core, parts of this tutorial 
may not function as described. Check back for updates.

This tutorial is based on Matthew Gilliard's "Flow 101" blog post.

Fn Project was released in October 2017. Chad Arimura explained the motivation and structure of the project in a good amount of detail in his post "8 Reasons why we built the Fn Project", with one of the major components being Fn Flow. Flow allows developers to build high-level workflows of functions with some notable features:

• Flexible model of function composition. Sequencing, fan out/in, retries, error-handling and more.
• Code-driven. Flow does not rely on massive yaml or json descriptors or visual graph-designers. Workflows are defined in code, expressed as a function, naturally.
• Inspectable. Flow shows you the current state of your workflow, allows you to drill into each stage to see logs, stack traces and more.
• Language agnostic. The initial Flow implementation which this post will use is in Java but support for other langauges has already started including Python, Go and JS.

What is a Flow?

A Flow is a way of linking together functions, and incidentally provides a way to define those functions inline if you need to. It's a FaaS-friendly way of saying stuff like

Start with this, then do that, then take the result and do these things in parallel then when they've all finished do this one last thing, and if there's any errors then do this to recover

Where all the this and that are FaaS functions.

A simple Flow function looks like this:

    public String handleRequest(int x) {

        Flow flow = Flows.currentFlow();

        return flow.completedValue(x)
                   .thenApply(i -> i+1)
                   .thenCompose( s -> Flow.invokeFunction("./isPrime", s) )
                   .get();
    }

If you've used a promises-style API before then this will be familiar. The closest analogue in core Java is the CompletionStage API which was even called Promise in a pre-release draft.

Anyway it's easy to tell the stages of what's going to happen:

• Start with a value provided by the user
• Apply some transformation i -> i+1
• Pass that to an external function called ./isPrime
• Then return get the result and return it

Internally the Flow class submits each stage in this workflow to the Flow Server. You'll meet it soon. The Flow Server will then orchestrate each stage as an individual call to Fn. Flow Server is responsible for working out which stages are ready to be called, calling them, handling the results and triggering any following stages until you reach the point where there's no more work to do.

This example could easily be written without Flow but it's good to start simple.

Before you begin

Currently FnProject is available to download, to experiment with, and to run on your private cloud. A managed service by Oracle is in the works. To play with Flow at the moment you will need to run everything locally, but it's not hard. We need fn, the Fn server, the Flow Server and not necessary but nice-to-have is the Flow Server UI. These run on ports 8080, 8081 and 3002 respectively so you might need to configure firewalls to allow access.

As you make your way through this tutorial, look out for this icon . Whenever you see it, it's time for you to perform an action.

Setting up

Install the fn CLI tool:

curl -LSs https://raw.githubusercontent.com/fnproject/cli/master/install | sh

Then start the Fn server:

fn start

The output looks something like the following. The version number below is old. You should see the latest version number in your case.

...
time="2017-10-11T13:12:44Z" level=info msg="Serving Functions API on address `:8080`"
        ______
       / ____/___
      / /_  / __ \
     / __/ / / / /
    /_/   /_/ /_/
        v0.3.119

The Flow Server needs to know how to call the Fn server, so ask Docker which IP address to use.

FNSERVER_IP=$(docker inspect --type container -f '{{.NetworkSettings.IPAddress}}' fnserver)

Start the Flow Server:

docker run --rm -d \
     -p 8081:8081 \
     -e API_URL="http://$FNSERVER_IP:8080/r" \
     -e no_proxy=$FNSERVER_IP \
     --name flowserver \
     fnproject/flow:latest

Then start the Flow UI:

FLOWSERVER_IP=$(docker inspect --type container -f '{{.NetworkSettings.IPAddress}}' flowserver)

docker run --rm -d \
      -p 3002:3000 \
      --name flowui \
      -e API_URL=http://$FNSERVER_IP:8080 \
      -e COMPLETER_BASE_URL=http://$FLOWSERVER_IP:8081 \
      fnproject/flow:ui

Now, everything's set so lets crack on!

A simple Flow function

Create a new function:

fn init --runtime=java simple-flow

Change directory:

cd simple-flow

Flow has a comprehensive test framework, but lets concentrate on playing with the code for the time being:

rm -rf src/test   ## yolo

Make peace with yourself after that, then let's get the code in shape.

Change HelloFunction.java to look like this:

package com.example.fn;

import com.fnproject.fn.api.flow.Flow;
import com.fnproject.fn.api.flow.Flows;

public class HelloFunction {

    public String handleRequest(int x) {

	Flow fl = Flows.currentFlow();

	return fl.completedValue(x)
                 .thenApply( i -> i*2 )
	         .thenApply( i -> "Your number is " + i )
	         .get();
    }
}

Then deploy this to an app which we call flow101 on the local Fn server.

fn deploy --app flow101 --local

Then configure the function to talk to the Flow Server.

fn config app flow101 COMPLETER_BASE_URL "http://$FLOWSERVER_IP:8081"

You can now invoke the function using fn call:

echo 2 | fn call flow101 /simple-flow

The output looks something like the following:

Your number is 4

Alternatively, you can now invoke the function using curl:

curl -d "2" http://localhost:8080/r/flow101/simple-flow

The output looks something like the following:

Your number is 4

Exploring the Flow UI

Browsing to http://localhost:3002 you should see something like this:

Which is showing us 3 function invocations:

• The main flow function, in blue
• .thenApply for the code i -> i*2
• .thenApply for the code i -> "Your number is " + i

Click on any of these and see the detail for each one expanded at the bottom of the screen.

The blue function is shown as running for the whole time that the thenApply stages are. Why? Because we are calling .get() at the end, so this is synchronously waiting for the final result of the chain. Exercise: Try removing the .get() from the code (you'll need to return a different String, and don't forget to re-deploy). Now it will look like:

This shows that Flow is well-suited for asynchronous functions which result in a side-effect (posting to slack, for example).

Why we made Flow

Consider how else you could have achieved what we did in 4 lines of Java above. First of all there are actually 3 different functions at play, each of which would need its own codebase and entrypoint. Then, consider that Flow chains values through them while preserving type information. We could have written an overarching "Call function A using an HTTP client, call function B the same" type orchestrator function (ugh), or even hard-coded each function to call the next one in the chain (ugh ugh). Perhaps we could have used an Enterprise Service Bus to glue it all together (multiple ughs).

We think Flow hits a very sweet spot of allowing sophisticated stateful apps defined in code, and maintaining the promised benefits of FaaS a la Serverless Manifesto. We think you'll like it too.

Summary

So, congratulations - we've covered a lot! You've got a Flow function running, seen how to use the API to compose simple transformations and run things in parallel.

Learn more

Go to the Flow 102 tutorial to take your Flows to the next level.