GCP Memorystore (Managed Redis & Memcached) ⚡🧠

Google Cloud Memorystore is a fully managed, in-memory data store service supporting open-source Redis and Memcached. Delivering sub-millisecond data access for high-frequency caching, session state management, gaming leaderboards, and real-time streaming, Memorystore eliminates the operational burden of clustering, automated failover, and OS patching.

---

Architecture & Mental Model

Memorystore for Redis: Standard HA vs. Cluster Architecture

┌────────────────────────────────────────────────────────────────────────┐
│               1. Standard Tier (Regional High Availability)            │
│                                                                        │
│   Zone: us-central1-a (Primary Node) ──► Zone: us-central1-b (Replica) │
│   [Handles Reads & Writes]              [Synchronous/Async Replication]│
│                                         [Auto-promoted on failure <30s]│
│   99.9% SLA • Automatic Failover • Up to 300 GB Memory                 │
├────────────────────────────────────────────────────────────────────────┤
│               2. Memorystore for Redis Cluster (Massive Scale)         │
│                                                                        │
│   Shard 1 (Primary + Replica)  ──► Hash Slots: 0 - 5460                │
│   Shard 2 (Primary + Replica)  ──► Hash Slots: 5461 - 10922            │
│   Shard 3 (Primary + Replica)  ──► Hash Slots: 10923 - 16383           │
│                                                                        │
│   Up to 250 Shards • 100+ TB RAM • 60,000,000 QPS • 99.99% SLA        │
└────────────────────────────────────────────────────────────────────────┘

---

Core Concepts

1. Basic Tier vs. Standard Tier

Feature	Basic Tier	Standard Tier (HA)
High Availability	None (Single node in one zone)	Dual-Zone HA (Primary + Read Replica)
Failover Support	No failover (data lost or unavailable during maintenance)	Automated failover in < 30 seconds
SLA	No uptime SLA	99.9% Uptime SLA
In-Flight Data Replication	None	Replicated asynchronously to replica
Production Feasibility	Dev / Testing only	Mandatory for all production applications

2. Eviction Policies (maxmemory-policy)

When Memorystore memory is full, the configured maxmemory-policy determines how Redis behaves:

Policy	Behavior	Production Recommendation
allkeys-lru	Evicts least-recently-used keys out of all keys	Default Best Practice for Caches
volatile-lru	Evicts least-recently-used keys that have an explicit TTL expiration	Best if mixing persistent session state with temporary cache
allkeys-lfu	Evicts least-frequently-used keys (frequency counter)	Best for long-tail access patterns
noeviction	Never evicts keys: Returns OOM command not allowed error on writes	Dangerous for caching: Use only if Redis is a strict state store

3. The maxmemory-gb & maxmemory-reserve Buffer

Redis requires overhead RAM to manage client connection buffers, background snapshots (RDB), and replication backlogs:

• The OOM Risk: If application data consumes 100% of instance memory, Redis cannot allocate memory for replication sync, causing the standby replica to disconnect.
• maxmemory-reserve (Best Practice): Reserve 10% to 20% of instance memory for replication buffers and fragmentation overhead.

---

Production gcloud CLI Commands

1. Provisioning a Production Standard Tier Redis HA Instance

gcloud redis instances create prod-cache-cluster \
  --size=10 \
  --region=us-central1 \
  --zone=us-central1-a \
  --alternative-zone=us-central1-b \
  --tier=STANDARD_HA \
  --redis-version=redis_7_0 \
  --network=projects/my-prod-project/global/networks/prod-vpc \
  --connect-mode=PRIVATE_SERVICE_ACCESS \
  --redis-config="maxmemory-policy=allkeys-lru" \
  --transit-encryption-mode=SERVER_AUTHENTICATION \
  --enable-auth

• --enable-auth: Enables Redis password authentication (AUTH string generated by GCP).
• --transit-encryption-mode=SERVER_AUTHENTICATION: Enforces TLS encryption for in-transit data.

2. Testing Manual Failover (Disaster Recovery Simulation)

# Initiate an automated zero-downtime failover to the standby replica:
gcloud redis instances failover prod-cache-cluster \
  --region=us-central1

3. Provisioning a Memorystore for Redis Cluster

gcloud redis clusters create prod-redis-cluster \
  --shard-count=3 \
  --replica-count=1 \
  --region=us-central1 \
  --network=projects/my-prod-project/global/networks/prod-vpc \
  --transit-encryption-mode=SERVER_AUTHENTICATION

---

Quotas & Limits

Parameter	Limit	Production Notes
Max instance size (Standalone)	300 GiB	Use Redis Cluster for larger sizes
Max cluster size (Cluster Mode)	250 shards (100 TB RAM)	Scales to 60 Million QPS
Failover transition time	< 30 seconds	Client SDKs must support automated retry
Network throughput ceiling	Scales with instance size	16 Gbps on large tiers

---

References

• Memorystore for Redis Overview: https://cloud.google.com/memorystore/docs/redis
• Redis High Availability Guide: https://cloud.google.com/memorystore/docs/redis/high-availability
• Redis Cluster Architecture: https://cloud.google.com/memorystore/docs/redis/cluster-overview
• Pricing: https://cloud.google.com/memorystore/pricing

---

Pricing Examples

Scenario 1: Standard High Availability Caching Tier

• 10 GB Standard Tier Redis instance (STANDARD_HA) running across 2 zones in us-central1.
• Standard HA hourly rate: ~$0.049 / GB-hour.
• Total cost: 10 GB × $0.049\times 730hrs=\ast \ast$357.70 / month**.
• High Availability standby replica and automated failover included.
• Total Monthly Bill: ~$357.70 / month.

Scenario 2: High-Scale Distributed Redis Cluster

• 5-Shard Redis Cluster (each shard has 13 GB RAM + 1 replica node = 130 GB total RAM).
• Capable of handling over 1,500,000 QPS with sub-millisecond response times.
• Cluster node pricing: ~$0.035 / GB-hour.
• 130 GB × $0.035\times 730hrs=\ast \ast$3,321.50 / month**.
• Total Monthly Cost: ~$3,321.50 / month (Serving real-time user sessions for tens of millions of concurrent gamers).

---

Nuggets & Gotchas

1. Basic Tier Maintenance Wipes All Data: The Basic Tier of Memorystore is a single node with no persistent disk or replication. During monthly Google Cloud infrastructure maintenance or unexpected host failure, the instance reboots and 100% of stored keys are completely wiped! Never use Basic Tier for session state or data that cannot be cleanly recomputed from the primary database.
2. Cluster Mode Requires Cluster-Aware Client SDKs: If you deploy Memorystore for Redis Cluster, your application code cannot use a standard legacy Redis client (e.g. standard redis-py or node-redis). The client SDK must be cluster-aware (e.g. redis.cluster.RedisCluster or ioredis), capable of handling MOVED and ASK hash slot redirects.
3. Private Services Access IP Subnet Exhaustion: Memorystore connects to your VPC via Private Services Access (a peered Google tenant network). If the allocated IP range (e.g. /24) is exhausted by Cloud SQL and other managed services, creating a new Memorystore instance will fail with IP_SPACE_EXHAUSTED. Plan a /20 or /21 range for Private Services Access.
4. Connection Pool Dropped on Failover: During a Standard HA failover (~15–30 seconds), the virtual IP is transferred to the replica. Existing TCP connections to the old primary are abruptly terminated. Client libraries that do not handle reconnections with exponential backoff will enter crash loops. Always configure client-side connection retry policies.
5. Slow Queries Block the Single-Threaded Event Loop: Redis is single-threaded. Running commands like KEYS *, FLUSHALL, or heavy Lua scripts on a large database will block the Redis event loop for seconds. During this freeze, health checks fail and Memorystore can trigger an unnecessary failover! Always use SCAN instead of KEYS *.

Across the wiki

• Amazon ElastiCache — in-memory caches (AWS)
• Azure Cache for Redis Architecture, Clustering, and Enterprise Tiers — in-memory caches (Azure)
• Redis — in-memory caches (Architecture)
• Caching — in-memory caches (Architecture)