Testing MQTT

Testing the performance of the MQTT protocol for COMP3310 Computer Networks at the ANU.

Requirements

This MQTT network analyser was implemented in Rust. See the Installation section of The Rust Programming Language book for installation steps.

The project uses the following crates:

• rumqttc: For MQQT library.
• tokio: For aysnchronous runtime and task spawning.
• csv: For saving data to .csv files.
• chrono: For date-time functionality.
• debug_print: For print functions which only trigger in debug mode.

The MQTT network analyser was primarily tested locally using the Mosquitto broker on an Ubuntu Linux virtual machine. See the Mosquitto Download page for further details on setting up the Mosquitto broker. We used the below Mosquitto configuration file to prevent the broker from saving previous experiments.

# Place your local configuration in /etc/mosquitto/conf.d/
#
# A full description of the configuration file is at
# /usr/share/doc/mosquitto/examples/mosquitto.conf.example

persistence false
persistence_location /var/lib/mosquitto/

log_dest file /var/log/mosquitto/mosquitto.log

include_dir /etc/mosquitto/conf.d

The project uses Python to graph the experiment results. See the Python Download page for installation steps. Our Python scripts requires the pandas, and matplotlib libraries which can be installed using the pip package manager.

Usage

The usage for the program is:

mqqt [-h <hostname>] [-p <port>] [-n <npublishers>] [-i <instancecount list>] [-q <qos list>] [-d <delay list>]

Where

• -h specifies the hostname of the MQTT broker
• -p specifies the port that the MQTT broker is on
• -n specifies the number of publishers
• -i specifies the instancecounts to use during testing as a comma seperated list
• -q specifies the quality-of-service levels (0, 1, and 2) to use during testing as a commas seperated list
• -d specifies the delay to use during testing as a comma seperated list

with default values hostname=localhost, port=1883, npublishers=5, instancecount list=1,2,3,4,5, qos list=0,1,2, and delay list=0,1,2,4.

To run the program in debug mode use

cargo run -- [-h <hostname>] [-p <port>] [-n <npublishers>] [-i <instancecount list>] [-q <qos list>] [-d <delay list>]

in the root directory. In debug mode, the program will print additional information and error messages. To run the program in release mode use

cargo run --release -- [-h <hostname>] [-p <port>] [-n <npublishers>] [-i <instancecount list>] [-q <qos list>] [-d <delay list>]

This will only print an overview of each experiment. Once all the experiments have finished running, the results will be saved to the experiment-results folder. The topic-results.csv file contains the statistics collected and computed by the analyser. The sys-results.csv file contains statistics received from the Mosquitto broker $SYS/# measurements.

Experiments

Our experiment results and figures are stored in the experiment-results and figures directories respectively.

To run the normal experiments on a local Mosquitto broker we used the following command.

cargo run --release

To run the many experiments with 10 publishers, qos=0, and delay=0ms we used the following command.

cargo run --release -- -n 10 -i 1,2,3,4,5,6,7,8,9,10 -q 0 -d 0

To run the online experiments on broker.hivemq.com with delay=0ms we used the following command.

cargo run --release -- -h broker.hivemq.com -p 1883 -n 5 -d 0

Scripts

We have included several scripts in the scripts folder to ease usage with the Mosquitto broker. Full details of their usage can be found in the scripts themselves. The start.sh and stop.sh scripts should be used to start and stop the Mosquitto broker between tests.

The plot_normal.py, plot_many.py, and plot_online.py Python scripts were used to visual the data generated by the normal, 10 publishers, and online broker experiments respectively. All the scripts have the same usage

python3 scripts/<script>.py (-t topic-results.csv) (-s sys-results.csv)

Project Structure

├── Cargo.lock
├── Cargo.toml
├── experiment-results
│   ├── many-sys-results.csv
│   ├── many-topic-results.csv
│   ├── normal-sys-results.csv
│   ├── normal-topic-results.csv
│   ├── online-sys-results.csv
│   └── online-topic-results.csv
├── figures
│   ├── many
│   │   ├── avg-heap-size-plot.png
│   │   └── message-rate-plot.png
│   ├── normal
│   │   ├── avg-heap-size-plot.png
│   │   ├── loss-rate-plot.png
│   │   ├── median-inter-message-plot.png
│   │   ├── message-rate-plot.png
│   │   ├── nconnected-clients-plot.png
│   │   ├── npub-msgs-dropped.png
│   │   ├── npub-msgs-recv.png
│   │   ├── npub-msgs-sent.png
│   │   └── out-of-order-rate-plot.png
│   └── online
│       ├── loss-rate-plot.png
│       └── message-rate-plot.png
├── README.md
├── scripts
│   ├── delete-log.sh
│   ├── log.sh
│   ├── plot_many.py
│   ├── plot_normal.py
│   ├── plot_online.py
│   ├── start.sh
│   ├── status.sh
│   └── stop.sh
└── src
    ├── analyser.rs
    ├── experiment.rs
    ├── main.rs
    ├── mqtt_helper.rs
    └── publisher.rs