Implement a mechanism for sending a MetaEvent from Statechart Code Bl…

[trulede]

Hi,

this is a commit that helps support Distributed Statecharts. Please let me know
if you need any further details.

Tim.

Implement a mechanism for sending a MetaEvent from Statechart Code Blocks.

meta("metadata", datum="foo", value="bar")

To support Distributed Statecharts, where one Statechart would like to
communicate Meta-Information to other Statecharts which are possibly
running as separate Systems (i.e running in other Processes, Containers
or Servers).

For Distributed Statecharts these MetaEvents are routed to "Adapters"
which implement the Property Statechart Interface. These Adapters are
attached to the Statechart Interpreter in the normal way. The
MetaEvents are intercepted by an Adapter and communicated to the other
Statecharts accordign to the methods defined in each particular Adapter.

The MetaEvents are not sent to the internal Event Queue and they should
not interfear with the existing operation of Property Statecharts as
these MetaEvents are identifed in Property Statecharts as
"meta event sent" rather than "event sent" (as seen when send() is used).

[AlexandreDecan]

Thanks for the PR!

I've a question. In my mind, MetaEvent are not supposed to be "known" by the statechart: those are events that are automatically generated by the interpreter for the purpose of runtime monitoring of properties. Could you please provide more information about why you want to rely on MetaEvent for the communication between statecharts/components, instead of using the currently implemented binding system (ie. interpreter.bind(adapter) and then send('metadata', datum='foor', value='bar') in a code block)?

Thank you!

[trulede]

Hi,

Yes, I think this is exactly the case, I would like these MetaEvents to not be known by the Statechart because these MetaEvents are actually intended for systems outside of the Statechart function. So, I agree with your existing design, and see this as an extension to the concept to also let the Statechart generate a MetaEvent. By using the meta() function, we can make sure that the MetaEvent is generated in a predictable manner and can be handled by any attached Property Statechart.

Using send() puts these Events directly into the Statechart, and perhaps that will cause problems down the line, particularly with other bound Statecharts (Property or otherwise). It certainly is "Hacky" trying to extract these MetaData Events from the stream of events coming from the StateChart ... when a MetaEvent class already exists which could help.

OK, the Adapters I've created are pretty Hacky too! I will try to get permission to send you one, at least you might find it interesting, what we are doing.

We really put a Sismic Statechart into a much larger System, its very cool to see it working, and that success has generated/validated a few new conceptual ideas. For instance, we are also very interested in generating Statecharts and Models in real-time, so that the Statechart can adapt to a changing System Capability. So for us, Meta Events are probably going to be significantly important, not just for sharing Data across system boundaries, but perhaps also dynamic reconfiguration.

Cheers, Tim.

[AlexandreDecan]

I'm not sure I fully understood the use case. You want to have the ability to send specific events from within the statechart (as with send) but in a way such that only other statecharts or components will receive them (in the current implementation, events that are sent using send are also sent to the statechart that fired them as InternalEvent)? Is it right? The purpose being to have the ability to notify some "monitoring component" (bound as a property)?

[AlexandreDecan]

I've started looking at the code ;) But we still need to discuss about the use case. Perhaps there's a better way to support communication between components (and I'll be happy to implement it either ;-)

[AlexandreDecan]

I suggest to use event_klass instead of event_class for consistency with, e.g., evaluator_klass used in Interpreter.
I also think that the type of event_class should be something like Callable[(str, ...), Event] or simply Callable[..., None] for consistency with the return type of create_send_function.

Finally, L57 should be removed as the "right" event is created on L58, right?

[trulede]

Hi,

Almost, I would like the ability to send specific Events from within the statechart (as with send) but in such a way that only bound Property Statecharts (or components implementing the Property Interface) will revive them. Therefore, I would like to differentiate from an InternalEvent (and ExternalEvent), both of which deal with the operation of the Statechart (or bound Statecharts), and send only MetaEvents via the existing Property Statechart mechanism.

The code sending the events to the bound Property Statecharts:

for event in sent_events:
            if isinstance(event, InternalEvent):
                self._raise_event(event)
                self._notify_properties('event sent', event=event)
            elif isinstance(event, MetaEvent):
                self._notify_properties('meta event sent', event=event)

differentiates between the InternalEvents (send()) and MetaEvents (meta()) in such a way that the consuming "monitoring component" can easily process each. In my current implementation the InternalEvents are distributed/published without much thought to a Redis Pub/Sub (and may end up being consumed by another Statechart) ... where as the MetaEvents have a specific handling.

Since we can detect if these Events came from the meta() call by the name "meta event sent" it is possible to implement fairly robust and simple processing. Our "monitoring component" also processed the other MetaEvents based on their name. So that all works great actually!

Thanks, Tim.

[AlexandreDecan]

Ok, got it, thanks ;-)

Is there any specific reason why you bound components using the bind_property method (i.e. as monitoring component) instead of the `bind`` method (i.e. as dialoging component)? I'm asking this because I've always considered that statecharts should be "passively" monitored (i.e. a statechart shouldn't know it's being monitored, and shouldn't interact with the component that monitors it).

I just want to be sure it's "the way to do it" before merging the PR ;-) Don't worry :)

[trulede]

Sure, actually, I'm also monitoring the Statechart, so the bind_property mechanism is exactly perfect. I write the State and Events into Redis too. Then, other software systems use that Redis information to know what state the Statechart is in (and the system function it represents). They can also react to the InternalEvents raised by the Statechart ... so its a kind of distributed bind() function too.

We have this distributed use case, which I guess is more complex than normal, and the ability to get Events into that distributed "fabric" allows us to implement some interesting mechanisms. We have currently Sismic coordinating with an application written in C via a Redis Distributed Memory service.

I guess as we explore this some more, other mechanisms will come up. Perhaps there is a possibly to develop an Adapter interface, for this distributed system, similar to the Property Statecharts (probably a subset) but task specific. Thats something we are still learning about ... with Python I think we get some interesting possibilities for linking the Statechart to a distributed "fabric".

[AlexandreDecan]

Thanks for the answer.

Could you please have a look at https://github.com/AlexandreDecan/sismic/tree/trulede-metaevent ?
I made some modifications to the code:

• The function is called notify instead of meta. It's a little bit more explicit and uses the same terminology as the one used in the Interpreter;
• The meta-events sent to the bound property statecharts (or monitoring components) are exactly the ones that were created using notify (in your code, sending a meta-event generates a new meta-event whose name was "meta event sent"). This allows more flexibility, and makes the implementation less specific to the use case. It also makes a clear distinction that meta-events sent from within the statechart are not considered as "events" like other kind of events that can be sent from within the statechart.
• As a consequence, the parameters of the meta-event received by the bound properties are directly available (in your implementation, it was required to use event.event.x to access x. Now, event.x is enough).

If you agree with these changes, it's very likely that I'll merge them early next week (it will be release 1.2.0).

[trulede]

Hello,

yes, I think notify() is perfect!

Do you think it might be an idea to create a separate Event Class for the notify() events?
class NotifyEvent(Event)
So that its possible to differentiate between a Meta and Notify event. When I process the Meta Events previously, both those generated by Sismic and those from send()/meta(), I extracted the Events I needed by their name (i.e. event sent, meta event sent) and "discarded" the rest.

Should I make that change against the branch you created?

[AlexandreDecan]

I prefer not to create a subclass. Events sent with notify are only propagated to bound properties, while other events (sent with send) are only propagated to bound interpreters. That means that only meta-events can be received by bound properties, so there is no need to "filter" the other events.

As the name of the meta-events sent using notify can be freely chosen, it's quite easy to filter them based on their name in a bound properties, isn't it? (e.g. notify('notify event', x=1, y=2, ...))

[trulede]

Yes, thats fine. The notify() function is perfect, the internal mechanism can always be changed if necessary.

We (royal "we" at the moment) are working on two concepts with Sismic, for our distributed systems. An Adapter mechanism, similar to PropertyStatecharts, that lets Sismic Statecharts participate in a distributed system without explicit knowledge of the other system components. And the dynamic generation and execution of Sismic Statecharts, which might even extend to modifying a running Statechart so that it matches the capability of the distributed system (which is variable).

A notify() function will be immensely helpful in keeping all of that synchronised!

Thanks, Tim.

[AlexandreDecan]

It's good to see that Sismic is used outside academic context ;) Let me know if you need help to use Sismic or if you need other features.

I've some (minor) other changes to do, but you can expect version 1.2.0 of Sismic to be released soon (Monday or Tuesday) with the notify() function and related changes.

[AlexandreDecan]

Release 1.2.0 will be available on PyPI in a few minutes.

[trulede]

Thanks, I will start working from it tomorrow.

And, I will hopefully be able to push a few other things, as we progress, or at least ask a few questions on the mechanics of a few features.

Cheers, Tim.