2.4 Reliability Requirements
Official task mapping
SAP-C02 task 2.4: Decoupling and quotas knowledge; resilient scaling and failure recovery. This lesson applies the official task to an original design exercise; it is not an exhaustive exam syllabus.
Prerequisites
Review Auto Scaling, serverless integration and database failover. Be able to explain the relevant failure boundary and access/cost model before choosing a service.
Decoupling and scaling
SQS buffers work; SNS fans out notifications; EventBridge routes events; Step Functions coordinates workflows. Standard and FIFO options differ: FIFO ordering is scoped to message groups, not a universal global sequence. SNS FIFO also supports documented archive/replay features, so “SNS never persists messages” is inaccurate.
Design consumers for idempotency and visibility-timeout/retry behavior. Dead-letter queues need ownership and a controlled replay process. Queue depth alone can mislead; backlog per worker and age of oldest message often better express load. Service quotas and FIFO throughput depend on configuration and Region.
Auto Scaling lifecycle hooks pause instances at lifecycle transitions for preparation or cleanup, not necessarily before the operating system starts. Health checks, grace periods, warm-up and downstream capacity must agree. Lambda payload limits differ between synchronous, asynchronous and streaming modes; consult the relevant current quota rather than repeating one 6-MB rule for all invocations.
Event-driven service comparison
| Service | Pattern | Ordering/replay concern |
|---|---|---|
| SQS Standard | Work queue | At-least-once delivery, best-effort order; idempotent consumers |
| SQS FIFO | Ordered queue | Message-group ordering and documented deduplication; business side effects still need idempotency |
| SNS Standard/FIFO | Fan-out | FIFO behavior and archive/replay differ from Standard |
| EventBridge | Event routing | Archive/replay requires configuration; do not assume ordered processing |
| Kinesis Data Streams | Partitioned stream | Per-shard ordering, retention and consumer checkpoints |
| Step Functions | Workflow | Workflow type, retries, timeouts and execution history |
Choose persistence/retention according to recovery requirements. Fixed throughput numbers can differ by Region, batching and high-throughput mode; consult owning quotas rather than treating 300/3,000 TPS as universal.
API Gateway throttling can protect supported API paths but is not an absolute downstream resource/cost cap. REST caching is a specific feature/configuration, not every API product. Lambda proxy and AWS service integrations require target permission/error contracts.
A queue with 120,000 orders after a surge drains in 15 minutes at net 8,000/minute if arrivals stop; with 4,000/minute continuing arrivals it takes 30 minutes. Define backlog admission and customer wait-time promises from net drain rate. Ask what happens when replay meets side effects already committed.
Scenario and rationale
A ticketing site receives a ten-minute surge of 20,000 orders/minute, while fulfillment can safely process 8,000/minute. A queue absorbs a 120,000-order backlog if ingress is accepted throughout the surge. After the surge, capacity must exceed continuing arrivals to drain it; scale based on an acceptable wait time, not only CPU.
Reject synchronous retries against the database: they can amplify overload. Reject “FIFO means no duplicate side effects”: consumer failures still require business idempotency. Define whether strict ordering is actually necessary before accepting its constraints.
Practice drill
Calculate backlog and drain time under two worker capacities. Simulate a worker failing after a charge but before acknowledgement and specify the deduplication key. Design visibility timeout, retry/DLQ policy and overload admission controls. Check quota requests and failure-domain capacity without executing production load tests.
References
- Official Domain 2 task guide
- SQS at-least-once delivery
- SNS FIFO archive and replay
- Auto Scaling lifecycle hooks
- Lambda quotas
Return to Domain 2. The drill is proposed study work, not a claim of a completed deployment.