Questions about the usage of the Simulator to get best size

[adriacabeza]

Hi! I recently added a logger into my workload so that I could capture the traces. In my case I decided to use the LIRS format as it was very easy to get. The reason why I'd like to use the simulator is because it allows me to choose between different maxSizes and see the hit rate. My current cache is experiencing a lot of misses and I suspect it is due thrashing so I'd like to see which would be a better optimal size.

Given that the simulator contains a lot of policies and options, I'd like to know which would be your recommendation here. My goal is not comparing different policies to see if Caffeine (aka TinyLFU) is the best policy for my use case, I want to see which would be the best params for the cache (from what I've read a bunch of the tuneable parameters are already adapted with HillClimbing, that's awesome!).

Which would be the application.conf file that would be the most similar to the current Caffeine implementation? I tried some simulations and the hit rate metrics were not similar to the ones reported by recordStats for any of the policies (using the resources.conf` as it is).

Thanks in advance!

PS: Adding as well one of the small trace file that I captured to this message in case it is helpful.
1.zip

[ben-manes]

The simulator includes caching products as well, as you probably noticed, so you can generate a chart like below. Since the hit rate is so high and the maximum size is small, I'd guess that in production you have expiration causing it to evict more aggressively? Unfortunately that is not currently supported by the simulator. It might be faster to just run your own script to calculate with, using FakeTicker to adjust the logical time based on the event's timestamp.

To add expiration to the simulator wouldn't be hard, just time consuming:

1. Make it the responsibility of the PolicyActor to track and delete, rather than each algorithm.
2. Add a removal method to Policy, a timestamp to the AccessEvent, and any relevant metrics in PolicyStats
3. Update all of the implementations to allow for external deletions
4. Add a trace reader that supports the timestamped log format

Chart

./gradlew simulator:simulate -q \
  -Dcaffeine.simulator.files.paths.0=lirs:/Users/ben/Downloads/1.zip \
  --maximumSize=10,20,30,40,50,60,70,80,90,100 \
  -Dcaffeine.simulator.policies.0=opt.Clairvoyant \
  -Dcaffeine.simulator.policies.1=product.Caffeine \
  -Dcaffeine.simulator.policies.2=product.Guava \
  --title=Experiment

[ben-manes]

for fun, here is a frequency histogram of your data sample.

cat ~/Downloads/1.log \
  | sort \
  | uniq -c \
  | sort -hr \
  | head -n 25 \
  | awk '{printf "%-15s ", $2; for (i = 0; i < $1/10000; i++) printf "*"; print ""}'

-427125702      **************************************************************************************
439283015       **************
651699012       ****
-416970270      ****
-1466723613     ****
659027692       ****
-2124870682     ***
1328399074      ***
-431198752      ***
1573935389      ***
664201264       **
-1849809819     **
1207888672      **
-738926869      **
1827183881      **
-1985175728     **
738193143       *
2122036442      *
-1258062283     *
1033892603      *
37772915        *
1345948885      *
974446421       *
1299928157      *
1176115861      *

[ben-manes]

I fixed the chart to improve its focus on the range of interest. Caffeine is a little wobbly compared to a static W-TinyLFU, so its adaptive search has a worst case -2.5% impact, which is acceptable. I think because the size is so tiny it struggles to reach a good steady state and gets stuck in a worse configuration longer, as the step size is 6.25% of capacity for a sampling periods of 10x capacity, so e.g. its shifting by 2 units in 400 events which is too limited and noisy. Its a minor flaw where we can't get truly optimal at minuscule caches, but it hasn't been worth solving for as users just bump up to something like 512 capacity as extremely small caches don't often make a ton of sense anyway.

Anyway for your actual problem, if you use expiration then review refreshAfterWrite. Or if you are using weak/soft references then you are subject to the GC and if weak keys then since we use identity equality you should ensure that the canonical key is being used or you'll have extra mappings.