# Cloudflare Transitions Quicksilver to Tiered Caching Architecture
The engineering team at Cloudflare recently detailed their transition of Quicksilver, their internal global key-value store, to a tiered caching architecture. This involved moving from storing all data on every server to a distributed caching system, improving storage efficiency while maintaining consistency and low-latency reads at the edge.
Over the past few years, Cloudflare evolved Quicksilver from version 1, where all data resided on each server, to version 2, a tiered caching system where only a select number of servers store data. This evolution is documented in a two-part blog series, outlining the migration of hundreds of thousands of live databases while handling billions of requests per second.
Quicksilver was originally developed by cloudflare as a fast,globally replicated,low-latency key-value store for its data centers. Initially designed for configuration distribution, it has become the foundational storage system for many Cloudflare services, including DNS, CDN, and WAF.
Quicksilver v1’s practice of storing the complete dataset on every server became unsustainable as the 1.6TB dataset grew by 50% in a single year,threatening storage capacity. The transition began with v1.5, introducing proxy and replica server roles and achieving a 50% reduction in disk usage.
The latest v2 implementation utilizes a multi-level caching strategy. This architecture includes local per-server caches, data center-wide sharded caches, and full dataset replicas on specialized storage nodes, enhanced by reactive prefetching to distribute cache misses. cloudflare opted for persistent storage using RocksDB over memory-based caching to address memory usage and cold cache issues, employing the engine’s compaction filters for evictions.
Source: Cloudflare blog
According to systems engineers Anton Dort-Golts and Marten van de Sanden, backward compatibility and sequential consistency were crucial:
“Quicksilver has, from the start, provided sequential consistency to clients (…) We have experienced [issues] when relaxing these guarantees.”