-
Notifications
You must be signed in to change notification settings - Fork 262
Functional Reactive Programming
Edit: Flexx started out with an "event system" based on RP, but quite soon it became obvious that this does not work well for UI toolkits, and in particular breaks down for visualization tools. Flexx changed to a more classic event system with v0.4.
TL;DR; In Reactive Programming (RP) data flows through your app via asynchronous signals. There are nodes that transform and/or combine these signals and optionally produce a new signal. The output signal of a node is cached to avoid unnecessary recalculation. In Functional RP nodes can be easily added to e.g. map, reduce, and filter signals. Changes in signals automatically propagate downstream. This makes a dev's live easier by avoiding callback spaghetti. Jump to the pseudo code.
Reactive programming (and Functional Reactive Programming in particular) are "a thing" right now. Lots of fuzz, but its rather unclear what it is. There are several implementations, but many follow different principles, or different aspects of FRP. On this page I try to write up a sort of summary of FRP in general, how it could look in Python, and how it can be useful.
The idea of FRP is around for decades. If it's so great, why is not used everywhere? A presentation (about pandas3d), explains: poor implementations, not integrated in useful domains, lingering implementation issues, swallows applications (all or nothing proposition), does not always simplify things.
General resources:
- http://www.reactivemanifesto.org/ - is a call to make apps "reactive".
- Some people say it's overrated.
- This provides a useful overview.
- Good read
Implementations of FRP:
- Escher (a Julia lib) uses it too, through reactive.jl.
- Shiny has a lesson on how to use reactivity, with code, and there is an overview.
- pandas3d, a OSS 3D game engine supports FRP. See e.g. this or this.
- The Facets package claims to be reactive, although it does not provide many the features common in RP.
- Trellis is a dead project that supports FRP in Python. Builds atop Peak and STM.
- The Lusmu project claims to be reactive, but I think its more dataflow-programming and lazy evaluation (rather than following the observer pattern).
The reactive manifesto wants apps to be responsive, resilient, elastic, message driven, asynchronous. The aim is to make apps scale better and have a pleasant user experience even if a system is under stress. No details on what it looks like though. Also the part on message driven vs message driven is rather vague.
Some quotes then:
- "Reactive programming is programming with asynchronous data streams"
- Escher: "Reactive programming is a style of talking about event-driven programs in terms of streams of data."
- Where event-driven programming is about reacting to things that happen, RP is about staying up to date with changing signals.
A central principle of RP seems to be that different components in an application (which are possibly distributed on different processes or even machines) communicate with asynchronous messages, and these messages represent streams of data. They are also referred to as signals, which have a value that changes over time. An example can be the mouse position, but also the key being pressed, some widget state, or the size of a widged; it can be anything.
Thinking about these things as signals/streams is a bit of a paradigm shift; it does bear similarities with event-driven principles, but there are some profound differences:
- An event object usually carries several pieces of information. A signal represents one "value" (though this can be a tuple or some sort of structure).
- Events not handled by a component, are usually propagated to the parent component. This is usually not the case in RP.
- In RP, signals can be chained and combined easily. Doing so in event-driven needs significant boilerplate code.
Thinking of messages as signals quickly leads to having some sort of nodes that operate on the signal, and produce a new signal. Or a node that listens to multiple incoming signals and combines this into a new signal:
A--->-- D--->--E
/
B-->--C
In the graph above, the signals/streams are the lines between the nodes. The nodes are "operators" on the signals, usually in the form of a function (either predefined or custom). These nodes are also called "conductors", "reactive expressions" (Shiny), "rules" (Trellis), "action", or ..., maybe "reactors"?
This is also referred to as dataflow programming, although dataflow programming is not necessarily RP.
The "Functional" in the term FRP refers to applying functional programming concepts to these signals: e.g. map, reduce, accumulate, filter, etc.
For the sake of keeping things clear, here's some terminology (note that different sources may use different terms):
- signal: a value that changes over time (an edge in the graph).
- reactor: an object that subscribes other reactors to receive their signals and may produce a new signal (a node in the graph).
- reactor function: the function that does the processing of a reactor. Each reactor has exactly one function.
- input: a reactor that has no signal input but a settable value. The reactor function is in this case used to validate and clean the input value.
Note: using the name "signal" in an API seems easier / less frightning than "reactor". The object is also used to refer to signals. Though it is not technically correct. Not sure what to do...
Lifting means to turn a common function or expression into a node that can operate on signals (to produce a new signal). i.e. Wrapping a function up in a reactor.
In most implementations, nodes cache their value, so that they are not unnecessarily recalculated. Shiny has a great example of that. I think this is an important feature, as it easily leads to better performance.
In some implementations, when signal A changes, it will automatically update D, and then E. This is also an important feature IMO, because it means a developer does not have to think about all the things that can happen that will require an update to E. It saves callback-spaghetti.
In event-driven language, connecting functions to an event is usually called "binding/connecting callbacks". In RP it's really about connecting nodes.
Some implementations also allow automatic binding. You just write a function in which you use certain signals, and whenever one of these signals changes, the function is rerun. Shiny is a good example. Escher does not support this.
I think this can be a powerful feature, though it might feel a bit too magical. It might also reduce flexibility if there is no manual way to connect nodes (or maybe not, need to look into it more). It saves a developer from specifying callbacks, but I think automatic downstream notification would already help a lot.
Also: without explicit input signals to a signal-processor-function, we cannot do functional RP.
In Shiny signals seem to be addressed by name. Not sure how this works if apps become big. I see no reason why you cannot address by "signal objects".
Some implementations deliberately allow peaking at the history of a signal. In other implementations this is deliberately not supported. Seems like it can be useful to at least know the previous value...
If the implementation supports functions, you can always make a previous signal.
Dynamic dependency graphs. In other words, signal X yields a value that is a signal, and you subscribe to that. X is then a selector of what signal you listen too. Can be powerful. Gui example: subscribe to self.parent.size, and don't worry about the parent changing. Can be done with the join operator? Or simply by smart signal addressing.
There are several resources (see above) that show what FRP can look like. Here is a sort of summary in Python pseudo-code. The exact API can be different, of course, it's about the general ideas.
Imagine an app that fetches data from the web, depending on the value of a slider. If a certain checkBox is checked, a correction is applied on the data. Finally the data is displayed.
Let's write the hypothetical app for an event-driven system:
def fetch_data(self, event): # takes a while
self._data = query_data_from_the_web(sliderWidget.value())
show_data(None) # manually "propagate" the "signal"
def show_data(self, event):
data = self._data
if checkBox.value():
data = apply_correction(data)
plotWidget.plot(data)
checkBox.connect(show_data)
sliderWidget.connect(fetch_data)Now we use signals. The checkBox and Slider are nodes that produce a signal, and the two functions are nodes that consume a signal, one function also produces a new signal:
@react
def fetch_data(): # takes a while
return query_data_from_the_web(sliderWidget.signal)
@react
def show_data():
data = fetch_data() # fetch_data() is a node with signal
if checkBox.signal:
data = apply_correction(data)
plotWidget.plot(data)
checkBox.connect(show_data)
sliderWidget.connect(fetch_data)
fetch_data.connect(show_data.connect)In this example, we have several nodes: the slider, the checkbox, and also the two functions that we defined; they are made a node (i.e. 'lifted') by decorating them.
In the above example we manually set-up the connections between all nodes. This can also be done automatically (as in Shiny). Alternatively, specifying connections can be made easier:
@react(sliderWidget)
def fetch_data(): # takes a while
return query_data_from_the_web(sliderWidget.signal)
@react(fetch_data, checkBox)
def show_data():
data = fetch_data() # fetch_data() is a node with signal
if checkBox.signal:
data = apply_correction(data)
plotWidget.plot(data)The functions do not accept any arguments, because their inputs are simply defined by using signals directly. The return value of these functions signifies their signal (show_data is an end-node).
An alternative is to have the function arguments match with incoming signals (as in reactive.jl). This is a prerequisite to have functional RP, otherwise you cannot generally apply pre-defined functions. In other words, having function inputs match the incoming signals helps code re-use:
@react(fetch_data, checkBox)
def show_data(data, checked):
...
# Now we can do:
not_checkbox = reactions.not(checkbox)
# And thus also:
@react(fetch_data, reactions.not(checkBox))
def show_data(data, checked):
...
If the checkbox changes: show_data() is called, which fetches the signal of fetch_data, which would return the cached value, or retrieve a new value if the slider was changed.
If the slider changes: the fetch_data node will notify its downstream nodes (i.e. show_data), which will call the function, and update the plot.
A good reactive system has at least these features:
- Data flows through an application as signals/streams, and is processed/modified in "nodes".
- Signals must retain their last value to avoid recalculation.
- A signal changing value should automatically propagate the change down the chain. This avoids having to setup a spaghetti of callbacks.
What we need for binding (either of these):
- Implicit binding: Very convenient, though a bit magical and maybe less flexible. Also not functional then.
- Explicit binding, with support to connect to multiple signals at the same time. (e.g. Facets allows this too), e.g. with a decorator.
More ideas:
- Adding support for functional programming concepts can make a system even more useful.
- Signal input type checking like traits/props. Not a FRP thing, but maybe it makes sense to combine the ideas into one unified system.