Caching

suggest change

Caching stores frequently accessed or expensive computation results in fast in-memory storage to decrease latency and offload downstream systems (such as databases and remote REST APIs). In Java, caching ranges from built-in standard library structures like LinkedHashMap and ConcurrentHashMap to high-performance libraries like Caffeine.

Built-in LRU Cache with LinkedHashMap

Java's LinkedHashMap can maintain elements in access order (most recently accessed to least recently accessed). By overriding its removeEldestEntry() hook, you can create a simple Least-Recently-Used (LRU) cache:

import java.util.Collections;
import java.util.LinkedHashMap;
import java.util.Map;

public class SimpleLruCache<K, V> extends LinkedHashMap<K, V> {
    private final int maxEntries;

    public SimpleLruCache(int maxEntries) {
        // initialCapacity, loadFactor, accessOrder = true (orders by access, not insertion)
        super(maxEntries + 1, 1.0f, true);
        this.maxEntries = maxEntries;
    }

    @Override
    protected boolean removeEldestEntry(Map.Entry<K, V> eldest) {
        // Automatically evicts the oldest entry when size exceeds maxEntries
        return size() > maxEntries;
    }

    public static void main(String[] args) {
        // Synchronize if accessed by multiple concurrent threads
        Map<String, String> cache = Collections.synchronizedMap(new SimpleLruCache<>(3));

        cache.put("A", "Apple");
        cache.put("B", "Banana");
        cache.put("C", "Cherry");

        // Accessing "A" moves it to the most recently used position
        cache.get("A");

        // Inserting "D" exceeds capacity (3); least recently used entry ("B") is evicted
        cache.put("D", "Date");

        System.out.println("Contains B: " + cache.containsKey("B")); // false
        System.out.println("Contains A: " + cache.containsKey("A")); // true
        System.out.println("Cache keys: " + cache.keySet());         // [C, A, D]
    }
}

Atomic Caching with ConcurrentHashMap

For simple concurrent applications without expiration requirements, ConcurrentHashMap.computeIfAbsent() provides thread-safe, atomic computation that prevents the thundering herd problem (duplicate computations for the same key):

import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;

public class UserService {
    private final Map<Integer, String> userCache = new ConcurrentHashMap<>();

    public String getUserProfile(int userId) {
        // The mapping function runs atomically per key
        return userCache.computeIfAbsent(userId, id -> {
            System.out.println("Cache miss: loading user " + id + " from database...");
            return "Profile for User #" + id;
        });
    }
}

Production In-Memory Caching with Caffeine

Caffeine is the industry-standard, high-performance in-memory caching library for modern Java, utilizing the Window TinyLFU eviction algorithm to achieve optimal hit ratios.

Add Caffeine dependency:

<dependency>
    <groupId>com.github.ben-manes.caffeine</groupId>
    <artifactId>caffeine</artifactId>
    <version>3.3.1</version>
</dependency>

LoadingCache with Expiration and Size Limits

import com.github.benmanes.caffeine.cache.Caffeine;
import com.github.benmanes.caffeine.cache.LoadingCache;
import java.time.Duration;

public class ProductCatalog {
    // LoadingCache automatically executes loader function on cache miss
    private static final LoadingCache<String, String> productCache = Caffeine.newBuilder()
            .maximumSize(10_000)                        // Evicts least valuable entries when limit is reached
            .expireAfterWrite(Duration.ofMinutes(10))   // Entries expire 10 minutes after insertion
            .expireAfterAccess(Duration.ofMinutes(5))   // Idle entries expire after 5 minutes without reads
            .recordStats()                              // Track hit/miss statistics
            .build(sku -> fetchProductFromDatabase(sku));

    private static String fetchProductFromDatabase(String sku) {
        return "ProductDetails(" + sku + ")";
    }

    public static void main(String[] args) {
        // Automatic population on miss
        String details1 = productCache.get("SKU-100");
        String details2 = productCache.get("SKU-100"); // Cache hit

        System.out.println("Details: " + details1);
        System.out.println("Hit rate: " + productCache.stats().hitRate());
    }
}

Caching Best Practices and Pitfalls

  1. Always Set a Maximum Size: Never use unbounded collections (new HashMap<>()) as caches; under traffic spikes, unbounded memory growth inevitably causes OutOfMemoryError.
  2. Handle Cache Stampedes: When a frequently accessed key expires, multiple concurrent requests might compute the replacement value simultaneously. Use LoadingCache or computeIfAbsent() to guarantee single-flight computation.
  3. Invalidation Strategy: Provide clear invalidation mechanisms (e.g. cache.invalidate(key)) whenever state changes in write operations.

Feedback about page:

Feedback:
Optional: your email if you want me to get back to you:



Table Of Contents