-
Notifications
You must be signed in to change notification settings - Fork 5
NEED Description Language
While the specification to be consumed by the DeploymentGenerator needs to be given in the specification format described in Topology Description, we also provide a less verbose, more human-readable domain-specific language to define experiments, the NEED Description Language. It allows to define all parts of a NEED experiment and has some semantics on top of the ones in the topology specification format.
The following is a NDL document specifying a simple experiment:
bootstrapper need:2.0
service server image=nginx:latest tags=frontend app share
service api image=api:latest tags=backend app reuse
service client image=client:1.0 command=['80', '0']
service db image=postgres:latest tags=backend app
service dashboard image=need-dashboard:1.0 supervisor port=8088
bridges s1 s2
bridge s3 tags=inactive
link server--s1 latency=9.1 up=1Gb down=800Mb jitter=0.2
link api--s1 latency=5.1 up=1Gb symmetric
link s1--s2 latency=0.11 up=1Gb symmetric drop=0.001
link client--s1 latency=23.4 up=50Mb down=1Gb
link db--s2 latency=8.0 up=1Gb symmetric
at 0s api join 3
at 0s db join 3
at 0s server join 5
at 30s client join
from 30m to 3h30m server churn 40% replace 50%
from 30m to 3h30m client churn 2
at 4h20m server--s1 leave
at 4h25m server--s1 join
from 10h2m to 10h6m api--s1 flap 0.93s
from 12h to 24h tags=backend leave 60%
from 15h to 15h20s server disconnect 1
at 18h20m api--s1 set latency=10.2 jitter=1.2 drop=0.002
The order of components is the same: First the bootstrapper, then services, then bridges, links and dynamic events.
For service declarations, there are a few shorthands. share is equal to share="true" in the specification format; reuse and supervisor work analogously. Services, as well as bridges and links, can be given tags, which we explain in more detail later.
For bridges, since the only mandatory attribute is a name, if no tags are appointed, the specification can be much less verbose: bridges s1 s2 is equal to:
<bridges>
<bridge name="s1" />
<bridge name="s2" />
</bridges>
Links are specified like this: instead of always providing a source and dest attribute, two names can simply be linked by--. The symmetric keyword has the same function as writing download="x" in the specification format, where x would be the same value as in upload.
The most improvements of the NEED Description Language over the XML-based specification format is in its expressions of dynamic events. There are some keywords that do not have a 1:1 equivalent in the specification format, but are rather translated into a series of these.
We enrich the XML-based specification format in three dimensions, essentially: By adding interval events that happen over time; by supporting tags to group network components together, and by a number of high-level concepts we automatically translate into specification format primitives.
The time of a dynamic event, be it instant-based or taking place over an interval of time, is expressed as one or two points in time like this: <x>d<y>h<z>m<t>s, where d, h, m and s mean days, hours, minutes and seconds, respectively, and , , are integers and is an integer or a float. There has to be at least one of those units present, but none is compulsory, ie. these are all valid: "5d", "10.25s", "3m15s", "20h3.5s".
For events that happen exactly once at a specified point in time, the syntax is at <time>. For interval events, it is from <time> to <time>.
join, leave and disconnect events can span an interval of time. In the case of join and leave, the specified number of instances joins/leaves at uniformly randomly distributed points in time within the interval. In the case of disconnect, the service disconnects for the specified period.
Tags are a means to group elements together based on real-world criteria. For example, one of the most common causes for network "failures" is the distributed roll-out of software upgrades, e.g. for routers.
Tags help to capture groups of devices sharing network status-relevant characteristics,for example driver versions that could be updated at the same time. Tags could also be used to map services to data centers or to logical parts of a distributed system (frontend, backend).
In the NEED Description Language, tags are expressed like this: tags=tag1 tag2 tag3 <etc.>. There is no limit to tags that can be put on any component.
Actions that do not target a name or a ---separated pair of names (i.e. a link) but one or more tags target all network components that have this tag.
Churn is the failing of a set of nodes over time and their possible replacement. Churn can be specified for an absolute number of instances, such as in churn 2, or relative to the number of instances with that tag at the start time of the event, as in churn 40%. Internally, if a percentage is given, the absolute number of possible targets at that time is calculated based on all preceding dynamic events. Then, the percentage is calculated. For the resulting number of instances, random points in time within the specified interval are chosen.
If in addition to the churn keyword, a replace-ment rate is given, then immediately after shutting down a service, another instance of that service is started with probability equal to replace.
The flap keyword can be used to describe link flapping, the fast disconnecting and reconnecting of a link as observed in real systems. The syntax is straightforward: <link> flap <time>, where is the period between a disconnect and the next reconnect or vice-versa. At the end of a flap event, the link by definition always reconnects.
Events that take place over a period of time, such as join, leave, or churn, rely on randomness to determine the points in time of the discrete events they describe. We provide a standard seed, so experiments are inherently reproducible. However, users can, by means of the -s flag, also provide their own seed.
After installing the python packages as described in Installation and Usage, compile NDL files to XML specification format files like this:
$NDLTranslator /path/to/input-file.ndl > /path/to/output-file.xml
Or, if you provide your own random seed, like this:
$NDLTranslator /path/to/input-file.ndl -s 1234567890 > /path/to/output-file.xml