This folder contains scripts to process raw results into a series of output forms, described below
Iteration times are extract from DL logs by the pytorch_iter_time_processing script. Truncate options are provided to remove iterations conducted without competing traffic (for example, if the load file finishes before the DL task). This is an important particularly at high load to ensure accurate statistics are produced. Block processing can be performed using the scripts provided in the scripts folder.
Bandwidth values are extracted from logs of tcpdump monitoring of the source side server using the tcpdump_processing script. Processing extracts data arriving in a window length and divides it by time elapsed in that length, overlapping windows in a moving average manner. A window filter size of around 30 works well for the range of loads between 0 and 25 Gbps. This can be adjusted to produce longer or shorter averaging. Block processing can be performed using the scripts provided The '--all_results' option returns every averaged point to the command line to support the construction of histograms or time dependent results.
When using iperf as a load, bandwidth values can also be extracted from logs captured directly from the iperf tool's output, using the iperf_bw_processing script. The number of parallel threads must be specified in order to account for formatting changes.
In order to compare tcpdump results from different positions within the network, the individual flows must be synchronized relative to each other. The sync_instant_tcpdump_processing script takes 3 separate tcpdumps of the same event, overlapping in time, and synchronizes the time axis of the output, allowing each output file to be plotted against the others in the same set accurately. A moving average filter of default window size 30 is applied, defined as a parameter, to produce an instantaneous packet injection rate estimate.
To produce a CDF, raw iteration time data is fed into the CDF_processing file. The option is provided to combine several shorter runs together for a better picture of tail behavior. AS per the iteration time script, truncate options are provided to remove iterations conducted without competing traffic (for example, if the load file finishes before the DL task). This is an important particularly at high load.
For comparison purposes, always verify the achieved bandwidth is close to or equal the expected target bandwidth, as a discrepancy here will lead to inaccurate like for like comparisons between DL models or cross traffic types.
Histograms can be generated by passing the output of the '--all_results' option of the bandwidth script into the histogram_processing MATLAB script. This produces both sample graphs and writes histogram data directly to file.
Verification and/or testing of the behavior of different loads generated by the flumenequi load tool in 'src/network' can be performed utilizing the scripts in 'bandwidth_testing' folder. config_burst_analysis is designed to operate on load '*.config' files, and takes a filter width input for its moving average filter. tcp_burst_analysis operates in much the same way, but takes line by line packet outputs from the tcpdump tool as input instead. In this way, the expected flows requested and resultant packets observed on the link can be compared, although it is important to note that synchronization between the two is not provided.