Compute Node

Compute node receives data from Collector node(s) and runs chosen modules for different ml-models which classify data as malicious or benign.

Dependencies

• nats-py
• pyspark
• ijson
• scapy
• torch
• PyYAML
• joblib
• pandas
• scikit-learn
• matplotlib

AND

• Docker
• Docker Compose
• tshark

Installation

In order to do the following:

1. Initialize VM
2. Install python requirements
3. Start NATS broker in Docker
4. Run main.py

All you gotta do is:

sudo make

Usage

Environment variables:

To set the URL to the NATS broker:

export BROKER_URL=<url>

To set the verbosity level of the programs output (defaults to INFO):

export LOG_LEVEL=<DEBUG|INFO|WARNING|ERROR|CRITICAL>

Creating New Modules

New modules can created by inheriting from the BaseModel class. Its dependencies should be added to the requirements.txt file.

Module Configuration

Each module has a corresponding config file (<module-name>.conf) where you can edit the paths for the files/programs needed for that specific module, and set constants such as alert_threshold.