Faq
A pinned entry is one that cannot be removed by an eviction policy. This is useful when the entry is a stateful resource, like a lock, that can only be discarded after the client has finished using it. In those cases the behavior of evicting an entry and recomputing it would cause a resource leak.
An entry can be excluded from maximum size eviction by using weights and evaluating the entry to a weight of zero. The entry then does not count towards the overall capacity and is skipped by the maximum size eviction. A custom Weigher must be defined that can evaluate if the entry is pinned.
An entry can be excluded from expiration by using a duration of Long.MAX_VALUE, or roughly 300 years. A custom Expiry must be defined that can evaluate if the entry is pinned.
The weight and expiration are evaluated when the entry is written into the cache. This can be accomplished using cache.asMap().compute to pin and unpin the entry.
A load, computation, or callbacks performed inside of an atomic operation may not write into the cache. These recursive writes are not allowed by ConcurrentHashMap and may result in a livelock (Java 8) or an IllegalStateException (Java 9).
A workaround is to instead store a lookup that will lazily perform the computation outside of the cache's atomic scope. In the example below,
an AsyncCache is used for convenience to store a CompletableFuture which is completed by the calling thread.
AsyncCache<Integer, Integer> cache = Caffeine.newBuilder().buildAsync();
int factorial(int x) {
var future = new CompletableFuture<Integer>();
var prior = cache.asMap().putIfAbsent(x, future);
if (prior != null) {
return prior.join();
}
int result = (x == 1) ? 1 : (x * factorial(x - 1));
future.complete(result);
return result;
}Caffeine offers two implementation strategies for loading multiple entries in a single call using Cache#getAll.
A synchronous cache, Caffeine#build(), piggybacks on ConcurrentHashMap computations so that a single load behaves identically to using computeIfAbsent. This locks through the hash table which supports only one computation per caller. Therefore a bulk load cannot block concurrent loads for the same entries and must replace existing mappings when it completes.
An asynchronous cache, Caffeine#buildAsync(), establishes the hash table mappings immediately where an entry holds a CompletableFuture. This ensures that subsequent calls for an entry obtains an existing future, which may be in-flight, and the bulk operation completes the futures with its result. A synchronous cache with this feature can be emulated by having the AsyncCacheLoader return a completed future and using the synchronous view, but comes at the cost of some minor linearizability characteristics. These approach are preferable for bulk loading due to protecting against a cache stampede.
A case where Caffeine may suffer from contention is when the number of entries currently being computed is similar to or greater than the maximum number of entries that the cache has ever contained. This corresponds to the currently computing entries being close to the total capacity of the underlying ConcurrentHashMap, which blocks resizing the map until the loading functions complete.
This is expected to happen while the cache is warming up (although likely not at all). It may be more prevalent in small caches, where the number of ongoing computations is similar to the cache's capacity. If you are observing contention due to an issue like this (manifesting as threads making different requests blocking on the same lock in ConcurrentHashMap), consider increasing the initial capacity to your expected maximum concurrency to compensate, using an async cache, or following the approach described for recursive computations.
A good rule of thumb is described in ConcurrentHashMap's internal documentation,
Lock contention probability for two threads accessing distinct elements is roughly 1 / (8 * # of elements) under random hashes.
