Caching
suggest changeCaching stores frequently accessed or expensive computation results in fast in-memory storage to decrease latency and offload downstream systems (such as databases and remote REST APIs). In Java, caching ranges from built-in standard library structures like LinkedHashMap and ConcurrentHashMap to high-performance libraries like Caffeine.
Built-in LRU Cache with LinkedHashMap
Java's LinkedHashMap can maintain elements in access order (most recently accessed to least recently accessed). By overriding its removeEldestEntry() hook, you can create a simple Least-Recently-Used (LRU) cache:
import java.util.Collections;
import java.util.LinkedHashMap;
import java.util.Map;
public class SimpleLruCache<K, V> extends LinkedHashMap<K, V> {
private final int maxEntries;
public SimpleLruCache(int maxEntries) {
// initialCapacity, loadFactor, accessOrder = true (orders by access, not insertion)
super(maxEntries + 1, 1.0f, true);
this.maxEntries = maxEntries;
}
@Override
protected boolean removeEldestEntry(Map.Entry<K, V> eldest) {
// Automatically evicts the oldest entry when size exceeds maxEntries
return size() > maxEntries;
}
public static void main(String[] args) {
// Synchronize if accessed by multiple concurrent threads
Map<String, String> cache = Collections.synchronizedMap(new SimpleLruCache<>(3));
cache.put("A", "Apple");
cache.put("B", "Banana");
cache.put("C", "Cherry");
// Accessing "A" moves it to the most recently used position
cache.get("A");
// Inserting "D" exceeds capacity (3); least recently used entry ("B") is evicted
cache.put("D", "Date");
System.out.println("Contains B: " + cache.containsKey("B")); // false
System.out.println("Contains A: " + cache.containsKey("A")); // true
System.out.println("Cache keys: " + cache.keySet()); // [C, A, D]
}
}
Atomic Caching with ConcurrentHashMap
For simple concurrent applications without expiration requirements, ConcurrentHashMap.computeIfAbsent() provides thread-safe, atomic computation that prevents the thundering herd problem (duplicate computations for the same key):
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
public class UserService {
private final Map<Integer, String> userCache = new ConcurrentHashMap<>();
public String getUserProfile(int userId) {
// The mapping function runs atomically per key
return userCache.computeIfAbsent(userId, id -> {
System.out.println("Cache miss: loading user " + id + " from database...");
return "Profile for User #" + id;
});
}
}
Production In-Memory Caching with Caffeine
Caffeine is the industry-standard, high-performance in-memory caching library for modern Java, utilizing the Window TinyLFU eviction algorithm to achieve optimal hit ratios.
Add Caffeine dependency:
<dependency>
<groupId>com.github.ben-manes.caffeine</groupId>
<artifactId>caffeine</artifactId>
<version>3.3.1</version>
</dependency>
LoadingCache with Expiration and Size Limits
import com.github.benmanes.caffeine.cache.Caffeine;
import com.github.benmanes.caffeine.cache.LoadingCache;
import java.time.Duration;
public class ProductCatalog {
// LoadingCache automatically executes loader function on cache miss
private static final LoadingCache<String, String> productCache = Caffeine.newBuilder()
.maximumSize(10_000) // Evicts least valuable entries when limit is reached
.expireAfterWrite(Duration.ofMinutes(10)) // Entries expire 10 minutes after insertion
.expireAfterAccess(Duration.ofMinutes(5)) // Idle entries expire after 5 minutes without reads
.recordStats() // Track hit/miss statistics
.build(sku -> fetchProductFromDatabase(sku));
private static String fetchProductFromDatabase(String sku) {
return "ProductDetails(" + sku + ")";
}
public static void main(String[] args) {
// Automatic population on miss
String details1 = productCache.get("SKU-100");
String details2 = productCache.get("SKU-100"); // Cache hit
System.out.println("Details: " + details1);
System.out.println("Hit rate: " + productCache.stats().hitRate());
}
}
Caching Best Practices and Pitfalls
- Always Set a Maximum Size: Never use unbounded collections (
new HashMap<>()) as caches; under traffic spikes, unbounded memory growth inevitably causesOutOfMemoryError. - Handle Cache Stampedes: When a frequently accessed key expires, multiple concurrent requests might compute the replacement value simultaneously. Use
LoadingCacheorcomputeIfAbsent()to guarantee single-flight computation. - Invalidation Strategy: Provide clear invalidation mechanisms (e.g.
cache.invalidate(key)) whenever state changes in write operations.