International Edition
Latest News
Technology

Uber Cuts Translation Cache Memory 90% With Hybrid Disk Design

Uber redesigned its Go-based translation cache to cut its initial memory footprint by more than 90% while keeping p99 cache miss latency below one millisecond, InfoQ reported. Memory profiling across Uber's Go services revealed two major operational bottlenecks.…

Uber Cuts Translation Cache Memory 90% With Hybrid Disk Design

Uber redesigned its Go-based translation cache to cut its initial memory footprint by more than 90% while keeping p99 cache miss latency below one millisecond, InfoQ reported.

Memory profiling across Uber’s Go services revealed two major operational bottlenecks. Garbage collection consumed more than 20% of CPU utilization in certain services, and translations accounted for approximately 70% of allocated objects, according to InfoQ.

The engineering team first replaced Go’s sync.Map with a conventional map[string]string protected by an RWMutex. This workload was read-heavy with infrequent updates and no individual deletions, which removed object allocations and improved CPU utilization by more than 30% in affected services.

To solve overall memory consumption, Uber adopted a hybrid storage design that retains translation keys and frequently accessed values in memory while storing remaining values on disk. The system preserves the existing Get() and Put() API while serving millions of translations per second with a cache hit rate above 99.9%.

Translation Cache Performance Metrics
Metric Before Redesign After Redesign
Dataset Memory ~800 MB ~40 MB (metadata)
Active Memory Range ~2.7 GB 750 MB
Garbage Collection CPU >20% Improved by 30%
Cache Hit Rate >99.9% >99.9%

The implementation divides data across 256 shards, each equipped with its own read-write mutex, in-memory entries, and disk file. Six-byte hash-based keys reduce metadata overhead, backed by a fallback mechanism to handle collisions.

Uber, founded in 2009 after its co-founders paid $800 for a private driver on New Year’s Eve, has grown into the world’s largest ridesharing platform. The company launched publicly in San Francisco in 2011 and now operates in approximately 70 countries, coordinating an average of 42 million trips and delivery orders per day with over 202 million monthly active users. Since its inception in 2010, Uber has coordinated 72 billion trips and delivery orders worldwide.

A time-based global least recently used policy manages in-memory entries. During cache refreshes, the system preloads translations accessed within the previous 15 minutes to prevent a cold cache.

About the author: Technology Desk

Technology Desk at Archynewsy.