Trying to make sense of startup benchmarks

[martinbonnin]

Running the startup benchmarks, I get this:

ColdStartupBenchmark_startupNoCompilation
timeToFullDisplayMs   min   745.2,   median   803.2,   max 1,124.2
timeToInitialDisplayMs   min   287.1,   median   302.0,   max   317.8
ColdStartupBenchmark_startupFullCompilation
timeToFullDisplayMs   min 654.0,   median 738.9,   max 877.8
timeToInitialDisplayMs   min 283.2,   median 301.0,   max 310.1
ColdStartupBenchmark_startupBaselineProfile
timeToFullDisplayMs   min 403.8,   median 742.3,   max 824.9
timeToInitialDisplayMs   min 267.0,   median 273.4,   max 345.4
ColdStartupBenchmark_startupBaselineProfileDisabled
timeToFullDisplayMs   min 670.2,   median 715.4,   max 792.3
timeToInitialDisplayMs   min 254.7,   median 271.9,   max 290.2

Looking at the median for timeToFullDisplayMs, looks like the best case is with startupBaselineProfileDisabled which sounds counterintuitive. Why would disabling the baseline profile be faster? Any clue what could happen here?

[SimonMarquis]

I reproduced similar values on Pixel 6 Pro:

StartupBenchmark_startupNoCompilation
timeToFullDisplayMs   min 858.1,   median 909.3,   max 958.9
timeToInitialDisplayMs   min 257.7,   median 272.8,   max 306.4
StartupBenchmark_startupFullCompilation
timeToFullDisplayMs   min 644.5,   median 709.8,   max 845.7
timeToInitialDisplayMs   min 264.2,   median 283.4,   max 292.2
StartupBenchmark_startupBaselineProfile
timeToFullDisplayMs   min 722.5,   median 773.7,   max 886.9
timeToInitialDisplayMs   min 241.6,   median 261.9,   max 271.2
StartupBenchmark_startupBaselineProfileDisabled
timeToFullDisplayMs   min 619.0,   median 644.5,   max 780.3
timeToInitialDisplayMs   min 228.0,   median 241.1,   max 253.9

One thing to note is that startupBaselineProfileDisabled is executed with warmupIterations = 1, which means the app is ran at least once, and this is probably enough to optimize this scenario enough 🤷