fedsd utility updated to make the RTI optional and support enclaves visualization

[ChadliaJerad]

This PR updates fedsd utility to not require an RTI trace file.
This is per @erlingrj 's request, so that it supports enclaves visualization.

[ChadliaJerad]

@erlingrj and @edwardalee, this PR is intended to adapt fedsd to visualize the interactions between enclaves.
To this end, it derives the RTI tracepoints and draws horizontal lines for the communications.
The figure below showcases the output from a federation visualization, without using rti.lft.

Does this make sense?

[edwardalee]

Ah, cool!But there is still the issue that all the trace points will go into one file.It also could be a real can of worms if we try to visualize federates with enclaves.Edward---------Edward A. LeeEECS, UC ***@***.***://eecs.berkeley.edu/~ealOn Jun 27, 2023, at 7:11 PM, Chadlia Jerad ***@***.***> wrote: @erlingrj and @edwardalee, this PR is intended to adapt fedsd to visualize the interactions between enclaves. To this end, it derives the RTI tracepoints and draws horizontal lines for the communications. The figure below showcases the output from a federation visualization, without using rti.lft. Does this make sense? —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>

[erlingrj]

Hi @ChadliaJerad, thanks alot for looking at this. I am currently getting a diagram that looks like this:

Based on the following 2 trace files (renamed from .lft to .txt to enable upload to Github):
EnclaveTracing_wrapper_src.txt
EnclaveTracing.txt

It would be amazing if we could get the the TAGGED_MSG event to go directly between the two enclaves. I think it is quite doable. All the messages that go to/from the RTI have dest/src -1 while this TAGGED_MSG (T_MSG) has src and dest set to the enclave ids

[erlingrj]

I added two minor changes.

1. Improve the exeception handling in fedsd
2. Make the fedsd script use the trace_to_csv utility that is within the same bin directory as fedsd. It was currently using whichever was on the path and for me there was another trace_to_csv executable on the path

[erlingrj]

@edwardalee initially we will just use it draw diagrams of the enclaves coordination. I haven't even thought about Diagrams for enclaved federations yet. I think it could make sense to have one process per enclave and one process for the RTI process within each enclave. Those RTI processes will be the ones exchanging federate-level NETs and LTCs

[ChadliaJerad]

Ah, cool!But there is still the issue that all the trace points will go into one file.It also could be a real can of worms if we try to visualize federates with enclaves.

Indeed. Currently, the diagrams that make sense are either for federations or enclaves.
As mentioned by @erlingrj, the adaptation is just a step to help understand enclaves interactions.
Enclaved federates need to be thought toughly. This can only be done once the coordination federates/enclaves remote_rti/local_rti are set.

It would be amazing if we could get the the TAGGED_MSG event to go directly between the two enclaves. I think it is quite doable. All the messages that go to/from the RTI have dest/src -1 while this TAGGED_MSG (T_MSG) has src and dest set to the enclave ids

This is straightforward, since you can you tracepoint_federate_to_federate()...
The call will be done in the enclave implementation.

[ChadliaJerad]

Ah, cool!But there is still the issue that all the trace points will go into one file.It also could be a real can of worms if we try to visualize federates with enclaves.

Indeed. Currently, the diagrams that make sense are either for federations or enclaves. As mentioned by @erlingrj, the adaptation is just a step to help understand enclaves interactions. Enclaved federates need to be thought toughly. This can only be done once the coordination federates/enclaves remote_rti/local_rti are set.

BTW @edwardalee, there is a trace file per enclave.

[erlingrj]

I am using federate_to_federate but still the arrow goes to the RTI.

[ChadliaJerad]

I am using federate_to_federate but still the arrow goes to the RTI.

I see why. I will fix it right now (and check with your trace files) and let you know.

[ChadliaJerad]

@erlingrj, here is the diagram after the fix (using the trace files you provided):

[erlingrj]

Thats great! Very useful!

[erlingrj]

LGTM!