Building a GraphQL Cache That Doesn’t Have to Fit in Memory

GraphQL clients have a wonderful property: once you’ve fetched an object, every part of your application can reference the same copy.

Instead of storing entire query responses, modern GraphQL clients use normalized caches. Each entity is stored exactly once, and queries become references into that graph.

For most applications, this works extremely well. Until the cache gets big.

Macro is a workspace for teams that brings documents, email, chat, tasks, CRM, notes, and files into one connected system. That means the application is not just rendering a few screens. It is constantly navigating a large object graph: documents connected to projects, messages connected to threads, tasks connected to people, properties connected to entities, and search surfaces connected to all of the above.

As we migrated more of that data fetching from REST to GraphQL, we realized the cache itself was becoming infrastructure. A user can easily accumulate hundreds of thousands of objects over time: documents, email threads, messages, tasks, properties, comments, and relationships between them.

Traditional GraphQL caches were not designed for that shape of data.

The problem with today’s GraphQL caches

Libraries like Graphcache, Apollo Cache, and Relay all store normalized entities in memory.

Persistence usually works by serializing that in-memory cache into IndexedDB so it survives a page refresh. Unfortunately, persistence does not actually solve the memory problem.

Memory
  ↓
Serialize
  ↓
IndexedDB
  ↓
Reload
  ↓
Read everything back into memory

The cache survives a restart, but it still has to fit into browser memory. As datasets grow, so do memory usage, garbage collection pauses, startup time, and the amount of work required to hydrate the cache.

For applications with relatively small datasets, that is perfectly reasonable. For a workspace like Macro, where the cache may eventually contain hundreds of thousands of interconnected objects, it becomes a bottleneck.

Thinking of the cache as a database

The key insight was that a GraphQL cache does not actually need to behave like an in-memory JavaScript object. It can behave more like a database.

Instead of keeping every entity resident in memory, we split the cache into two tiers.

Hot entities  → Memory
Cold entities → Persistent storage

The memory tier uses an LRU eviction policy. Recently accessed entities remain in memory. Older entities are serialized to disk. When they are needed again, they are transparently loaded back into memory.

The important property is that memory usage stays bounded regardless of how much data exists overall.

Instead of this:

500,000 entities → JavaScript heap

We get this:

500,000 entities total
 15,000 hot  → Memory
485,000 cold → Disk

The cache starts looking much more like virtual memory than a traditional frontend cache.

Why Rust?

The cache engine is written in Rust and compiled to WebAssembly. Rust gives us three advantages.

Deterministic memory management

The entire point of this architecture is explicit control over memory. Rust lets us reason precisely about when records are allocated, evicted, serialized, and dropped. That makes it much easier to build predictable eviction behavior than relying on JavaScript’s garbage collector.

Compact persistence

Instead of serializing cache records as JSON, we serialize them using Postcard, a compact binary format. Binary serialization reduces storage requirements, which matters when persisting very large normalized graphs.

Systems programming tools

Normalization, dependency tracking, serialization, persistence, and eviction are fundamentally systems problems. Rust is an excellent language for implementing storage engines.

Moving work to compile time

A normalized cache needs to understand the GraphQL schema. It has to answer questions like:

• How is each entity identified?
• Which fields reference other entities?
• Which values should remain embedded?
• Which union members are possible?

Most GraphQL caches compute this information while the application is running. Instead, we generate schema metadata during compilation and embed it directly into the cache engine.

The runtime no longer needs to inspect the schema. It already knows how every entity should be normalized. Besides reducing runtime work, this also lets us ship less JavaScript.

One cache, multiple windows

The browser implementation runs inside a Shared Worker. Rather than every browser tab maintaining its own cache, every window communicates with the same cache engine.

That gives us a single source of truth while keeping normalization work off the main thread. Browsers without Shared Worker support fall back to dedicated workers synchronized through BroadcastChannel.

On desktop, we take this one step further. Instead of running inside WebAssembly, the same Rust cache engine runs natively inside the application and persists to SQLite. The browser and desktop versions share the same core implementation while using storage that is appropriate for each platform.

A local object store

The interesting part of this project is not that it is written in Rust. It is that we are starting to think about the client-side GraphQL cache as a storage engine.

Instead of an in-memory object graph, it becomes a local object store with:

• normalized records
• bounded memory
• persistent storage
• offline support
• compile-time schema metadata
• cross-window synchronization

As applications continue moving toward local-first architectures, this becomes an increasingly useful abstraction. The cache is not just somewhere to keep yesterday’s network responses.

It is becoming the local database your application runs on.