Clarification on the Source of JSON Files in fig_8/spec and workload Folders

[prouzan]

Explanation of Topology and Workload Data in the m3 Project

Summary

I have been exploring the open-source project and had some questions regarding the details of the topology and workload data that used in m3 project. Dr. Li has provided valuable information on the origin and generation process of these data. This issue summarizes his responses for future reference.

Details

Sources or Methodologies for Topology Data:

To gather topology data, you can run m3/parsimon-eval/expts/fig_8/src/main.rs and ns-3. This will call the translate_topology function in m3/parsimon-eval/expts/fig_8/src/ns3.rs and write the topology information into a text file.

Process or Tools for Workload Data:

Similarly, running m3/parsimon-eval/expts/fig_8/src/main.rs and ns-3 will call the gen_flows function in m3/parsimon-eval/expts/fig_8/src/experiment.rs, which writes the flow information into a text file.

Generating JSON File

The processes of generating topology and workload data are based on the JSON files located in the parsimon-eval/expts/fig_8/spec folder.
Dr. Li explained that the script m3/parsimon-eval/expts/fig_8/src/bin/gen_mixes.rs is used to generate the JSON files such as all_dctcp.mix.json file. This script requires a configuration file for the parameter sample space and the number of scenarios as input, and it outputs the required JSON file.

Example command:

cargo run --bin gen_mixes -- --input spec/all_dctcp.mixspace.json --count 192 --output spec/all_dctcp.mix.json

Origin of Referenced JSON and TXT Files

Dr. Li provided the following information about the origin of the JSON files in the spatials folder and the TXT files in the parsimon-eval/workload folder, which are referenced in the all_dctcp.mixspace.json file:

1. parsimon-eval/workload/spatials Folder:

The JSON files in the spatials folder define the spatial parameters used for generating the workload and topology configurations.

2. parsimon-eval/workload/distributions/facebook Folder:

The TXT files in the facebook folder contain data from real-world traces and workloads, specifically from Facebook's data center operations.

The source files (workloads, traffic matrices, etc.) are derived from the Meta paper-"Inside the Social Network's (Datacenter) Network. InProceedings of ACM SIGCOMM". The Parsimon team collected these files, which m3 used directly to simulate the Meta workloads and traffic patterns.

Dr. Li's detailed explanation helps clarify the data generation process and the sources of these important files.