Grab’s Transition to Time-Aware LRU Caching Improves Android App Performance

To optimize image caching in its Android application, Grab engineers moved from a traditional Least Recently Used (LRU) cache to a Time-Aware Least Recently Used (TLRU) cache. This change allows for more effective storage reclamation without negatively impacting user experience or increasing server costs.

The Challenge with Traditional LRU Caching

The Grab Android app utilizes Glide, an open-source image loading and caching framework, to reduce network requests, improve loading times, and lower server expenses. Glide employs an LRU cache to store images locally. However, a 100 MB LRU cache presented limitations. It frequently filled up quickly for some users, leading to performance issues, although other users had images cached for extended periods, wasting storage space.

Introducing Time-Aware LRU (TLRU)

To address these shortcomings, Grab engineers implemented a TLRU cache, extending the LRU cache with time-based expiration. TLRU utilizes three key parameters:

• Time To Live (TTL): Determines when a cache entry is considered expired.
• Minimum Cache Size Threshold: Ensures essential images remain cached even after expiration if the cache is not full.
• Maximum Cache Size: Enforces the upper storage limit.

Implementation Details

Rather than building a TLRU cache from scratch, the Grab team opted to fork Glide and extend its DiskLruCache implementation. DiskLruCache is a widely adopted disk-based caching solution known for its robustness and handling of complex scenarios like crash recovery and thread safety. Extending DiskLruCache involved adding support for tracking last-access time, implementing time-based eviction logic, and creating a migration mechanism for existing user caches.

Assigning last-access timestamps to existing LRU entries during migration proved challenging. Since filesystem APIs lacked a reliable timestamp source, Grab engineers assigned a uniform migration timestamp to all entries, ensuring bidirectional compatibility with the original LRU implementation.

Optimizing Configuration and Results

Determining optimal configuration values for TLRU involved controlled experiments. The team aimed for a cache hit ratio decrease of no more than 3 percentage points during the transition.

The results were significant: 95% of users experienced a 50MB reduction in cache size, with the top 5% seeing even greater savings. Grab estimates that this approach can reclaim terabytes of storage across devices while maintaining acceptable cache hit ratios and avoiding increased server costs.

Key Takeaways

• TLRU caching offers a more efficient approach to image caching compared to traditional LRU.
• Extending existing, well-established libraries like Glide’s DiskLruCache can accelerate development and ensure robustness.
• Careful configuration and experimentation are crucial for optimizing TLRU performance.
• TLRU can lead to substantial storage savings and improved app performance without increasing server costs.