Faq
A pinned entry is one that cannot be removed by an eviction policy. This is useful when the entry is a stateful resource, like a lock, that can only be discarded after the client has finished using it. In those cases the behavior of evicting an entry and recomputing it would cause a resource leak.
An entry can be excluded from maximum size eviction by using weights and evaluating the entry to a weight of zero. The entry then does not count towards the overall capacity and is skipped by the maximum size eviction. A custom Weigher must be defined that can evaluate if the entry is pinned.
An entry can be excluded from expiration by using a duration of Long.MAX_VALUE, or roughly 300 years. A custom Expiry must be defined that can evaluate if the entry is pinned.
The weight and expiration are evaluated when the entry is written into the cache. This can be accomplished using cache.asMap().compute to pin and unpin the entry.
A load, computation, or callbacks performed inside of an atomic operation may not write into the cache. These recursive writes are not allowed by ConcurrentHashMap and may result in a livelock (Java 8) or an IllegalStateException (Java 9).
A workaround is to instead store a lookup that will lazily perform the computation outside of the cache's atomic scope. In the example below, Guava's memoize decorator will execute the lambda function, store the results, and return it on subsequent calls. As the supplier is evaluated outside of the cache's atomic scope it does not recursively write into the underlying hash table. Thanks to Guava's use of double-checked locking, the value is computed once and subsequent reads are lock-free.
LoadingCache<Integer, Supplier<Integer>> cache = Caffeine.newBuilder()
.build(x -> Suppliers.memoize(() -> (x == 1) ? 1 : x * factorial(x - 1)));
int factorial(int x) {
var memoizer = cache.get(x);
return memoizer.get();
}A similar approach may be implemented using AsyncCache where the entry maps to a CompletableFuture. This may still deadlock if an unordered dependency chain occurs or if the executor exhausts its thread pool. To minimize this scenario, a Executors.newCachedThreadPool() should be set on Caffeine.executor. The penalty is that each step in the dependency chain requires an additional thread.
A case where Caffeine may suffer from contention is when the number of entries currently being computed is similar to or greater than the maximum number of entries that the cache has ever contained. This corresponds to the currently computing entries being close to the total capacity of the underlying ConcurrentHashMap, which blocks resizing the map until the loading functions complete.
This is expected to happen while the cache is warming up (although likely not at all). It may be more prevalent in small caches, where the number of ongoing computations is similar to the cache's capacity. If you are observing contention due to an issue like this (manifesting as threads making different requests blocking on the same lock in ConcurrentHashMap), consider increasing the initial capacity to your expected maximum concurrency to compensate, or use an async cache.
A good rule of thumb is described in ConcurrentHashMap's internal documentation,
Lock contention probability for two threads accessing distinct elements is roughly 1 / (8 * # of elements) under random hashes.
