Benchmark results v4.0.3

[david-chapela]

CC (@marlondumas)

Run Simod version 4.0.3 with the following configurations (benchmark_configs.zip):

• Pooled resources (each pool is a group of resources that share calendar and performance, but more than one pool can perform the same activity).
• Differentiated resources.
• Differentiated resources + extraneous delays
• Fuzzy resources
• Fuzzy resources + extraneous delays

Case arrival rate

Not big differences, as expected, the impact of the resource model here is indirectly through resource contention, but mostly due to the estimated inter-arrival distribution, which is shared.

Only difference in BPIC 2017, where extraneous improve the arrival. Probably because the inter-arrival distribution underestimates the inter-arrival times (the arrivals are closer to each other than they should), and the extraneous delays (delaying the first activity of each trace too) move the arrival closer to its real apparition.

Circadian Event Distribution

Extraneous delays worsen the circadian metric. Makes sense up to some error limit, as they displace the events with delays.

Fuzzy considerably better than differentiated in Academic Credentials (log with many occasional resources), and close in BPICs.

Relative Event Distribution

Extraneous always improve the version without them.

The combination of Differentiated resources + extraneous delays outperforms the other in all cases.

Absolute Event Distribution

A bit of a mess, but Differentiated + Extraneous and Fuzzy + Extraneous seem to be the best ones.

Cycle Time Distance

Quite similar to Relative Event Distribution (as expected).

Differentiated + Extraneous give the best results with difference. Only outperformed by Fuzzy + Extraneous in the BPIC 2012.

2-gram Distance

As expected, similar performances in control-flow as the optimizations share these parameters,