Multi-Region Failover — Route 53 Health Checks and Routing Policies

Multi-region failover via Route 53 is one of the highest-frequency Domain 3 topics on DOP-C02. The exam tests when to pick failover routing vs latency-based routing vs weighted routing, how endpoint health checks, calculated health checks, and CloudWatch alarm health checks combine to drive failover decisions, the role of Application Recovery Controller (ARC) for explicit recovery readiness and routing controls, and the trade-offs of DNS TTL during failover events.

This guide assumes you know what an A record and a CNAME are. The DOP-C02 focus: failover primary/secondary records, calculated health check trees, CloudWatch alarm health checks for metrics like RDS replica lag, ARC routing controls that flip routing without DNS propagation delay, latency-based routing for active-active multi-region, weighted routing for canary, multi-value answer, geolocation and geoproximity for regulatory routing, DNS TTL as the lower bound of failover RTO, alias records to AWS resources, and private hosted zones for internal failover.

Why Multi-Region Failover Matters on DOP-C02

DOP-C02 explicitly lists "Enabling cross-Region solutions where available (for example, Amazon DynamoDB, Amazon RDS, Amazon Route 53, Amazon S3, Amazon CloudFront)" and "Testing failover of Multi-AZ and multi-Region workloads" under Domain 3. Community pass reports cite DR strategy questions as one of the most frequently flagged categories: candidates know failover routing exists but trip on the layered checks (calculated health check combining children) and on the difference between Route 53 health-check-driven failover and ARC-driven failover.

Real-world DR scenarios drive the exam: "Active-passive 2-region setup; primary fails; the secondary must take over with RTO under 1 minute" - failover routing + low TTL + calculated health check that detects upstream + downstream failure + ARC routing control for explicit fail-open. "Active-active routing with regulatory constraints (EU users to EU region)" - geolocation routing primary + latency fallback. "Canary new version in one region" - weighted routing 95/5. The exam expects you to assemble these patterns from primitives.

Plain-Language Explanation: Route 53 Failover

Route 53 mechanics map cleanly to dispatcher-and-fallback patterns in non-software domains. Three angles cover endpoint health checks, the failover routing model, and ARC.

Analogy 1: The Hospital Emergency Department Diversion System

A regional health network has multiple EDs. Route 53 is the central dispatcher routing patients (DNS queries) to the right ED. Endpoint health checks are the dispatcher's continuous calls to each ED checking "are you accepting patients" - if 3 of 5 calls in the last 30 seconds report "diverting", the ED is marked unhealthy.

Failover routing is the primary/backup ED designation - all patients go to the regional Level 1 trauma center; if it diverts, dispatch sends to the secondary Level 2. Latency-based routing is the patient-residence-based dispatch - patients are sent to the nearest functioning ED. Weighted routing is the load-balanced dispatch - 70 percent to ED-A, 30 percent to ED-B.

Calculated health check is the dispatcher's composite assessment - "ED is healthy only if (radiology AND lab AND staffing) report green". A child check fails (lab is offline), the parent fails, dispatcher reroutes.

CloudWatch alarm health check is the operational metric trigger - "if average wait time > 90 minutes, mark ED unhealthy" - turns metric-based degradation into routing decisions.

Application Recovery Controller is the manual override switchboard - the chief of medicine has an explicit "divert all to backup" toggle that does not depend on automatic health checks. Useful when the chief sees a problem the automated checks have not caught yet.

TTL is the cache time at local ambulance dispatch radios - if the central dispatcher updates the assignment, ambulances refresh their radio every TTL seconds. Lower TTL = faster pickup of changes, more radio traffic.

Analogy 2: The Airline Aircraft Routing System

An airline routes flights between hubs. Route 53 is the flight operations center assigning aircraft to slots. Endpoint health checks are the continuous radio polls checking each aircraft "are you operational" - 3-of-5 healthy means the aircraft is in service.

Failover routing is the primary/backup hub assignment - all flights go to JFK; if JFK closes, reroute to Newark. Latency-based routing is the distance-optimal hub selection - passengers in Detroit go to ORD, passengers in Atlanta go to ATL. Weighted routing is the gradual hub migration - shift 10 percent of traffic to the new terminal weekly.

Calculated health check is the composite hub readiness - "hub is operational only if (runways open AND ATC online AND ground crew staffed)". One child fails, hub is marked closed.

Application Recovery Controller is the operations director's manual control - explicit "shut JFK, redirect to Newark" toggles that work even if the automated systems disagree. ARC's readiness checks are the pre-flight verification that the backup hub actually has fuel, gates, and crew before the toggle is flipped.

Multi-value answer routing is the load-balance-with-fallback dispatch - return up to 8 healthy IPs for the client to choose, no preference, just exclude the unhealthy.

Analogy 3: The Multi-Branch Bank Fault Tolerance

A bank network has branches across cities. Route 53 is the call center that directs customer phone calls to the right branch. Endpoint health checks are the phone heartbeats to each branch's customer service line.

Failover routing is the city primary/backup branch pair - all calls to Branch-A; if Branch-A's phone is down, divert to Branch-B. Geolocation routing is the legal jurisdiction routing - EU customers to the EU branch (regulatory), US customers to the US branch.

Calculated health check is the composite branch readiness - "branch is up only if (network online AND tellers staffed AND vault accessible)". CloudWatch alarm health check is the service-level metric trigger - "if average call wait > 5 minutes, route to backup branch".

Application Recovery Controller is the bank president's emergency control - regardless of the automated checks, the president can manually flip routing. ARC's safety rules prevent dangerous toggles (e.g., "you cannot mark all 4 regional centers as failed simultaneously").

Routing Policies in Detail

Route 53 supports eight routing policies, each tested in DOP-C02 scenarios.

Failover Routing

Two records of the same name with RoutingPolicy=FAILOVER: one Primary, one Secondary. Both must have health checks. When the primary's health check is unhealthy, Route 53 returns the secondary; otherwise primary. If neither is healthy, Route 53 returns the primary as a last resort (better to try than to NXDOMAIN).

For active-passive multi-region, this is the canonical pattern: primary points to ALB in us-east-1, secondary to ALB in us-west-2.

Latency-Based Routing

Returns the record from the AWS region with lowest measured latency to the resolver's IP. AWS maintains a global latency database. Best for active-active multi-region serving end-user-facing traffic.

Combines with health checks: if the lowest-latency region's record is unhealthy, the next-lowest is returned.

Weighted Routing

Multiple records of the same name with weight values; Route 53 returns each in proportion to its weight. Common patterns:

• 90/10 canary: 10 percent of traffic to the new version, 90 percent to old.
• 50/50 active-active.
• 0-weight a record to drain it without deleting (no traffic returned).

Geolocation Routing

Returns records based on the resolver's geographic location (country or US state, with a default catch-all). Use for regulatory constraints ("EU users to EU servers") or content localization.

Geoproximity Routing

Like geolocation but with bias - shifts traffic toward or away from regions with a Bias value (-99 to +99). Requires Route 53 Traffic Flow.

Multivalue Answer Routing

Returns up to 8 healthy records randomly chosen, with no preference. Health-check-aware. Lightweight load distribution without the cost of an ELB.

IP-Based Routing

Returns records based on the resolver's IP being in a configured CIDR block. Useful for ISP-aware routing or large enterprises with known egress IPs.

Simple Routing

One record per name. No health checks (you cannot attach one). The default and the simplest.

Health Check Types

Three primary health check types compose for complex monitoring.

Endpoint Health Check

Probes an HTTP/HTTPS/TCP endpoint from Route 53's global checker fleet (15+ regions). Evaluates 3-of-N healthy thresholds:

• Request interval: 10 or 30 seconds.
• Failure threshold: number of consecutive failed checks (default 3).
• String matching (for HTTP/HTTPS): require a specific substring in the body for healthy.
• HTTPS with SNI: support modern TLS endpoints.
• Latency graphs: optional, adds latency monitoring.

Endpoint health checks see what an external client sees - reachability and content. They cannot evaluate internal metrics like queue depth.

Calculated Health Check

A parent check that combines child checks with logical expressions. The simplest form: health-of-child-1 AND health-of-child-2. More complex: at-least-2-of (child-1, child-2, child-3, child-4).

Useful for "site is healthy only if frontend AND backend AND DB replica are all healthy" patterns.

CloudWatch Alarm Health Check

Watches a CloudWatch alarm. When the alarm transitions to ALARM, the health check is unhealthy. Use cases:

• "Mark route unhealthy if RDS replica lag > 30 seconds" - trigger failover before clients hit stale data.
• "Mark route unhealthy if 5xx error rate > 1 percent over 5 minutes" - cover application-layer failures the endpoint check might miss.
• "Mark route unhealthy if queue depth > 10000 messages" - cover backpressure scenarios.

DNS TTL and Failover RTO

The minimum failover RTO from a Route 53 health check failure is bounded below by:

• Health check detection: 3 consecutive 30-second failures = 90 seconds (or 30 seconds with 10-second interval) for endpoint checks.
• DNS propagation: TTL (resolver caches the old answer until TTL expires).

For "failover within 60 seconds" requirements, set TTL to 30 seconds and use 10-second-interval health checks. Be aware that some resolvers (corporate DNS, some ISPs) ignore TTL and cache longer than instructed.

For sub-30-second failover, DNS-based failover alone is insufficient - you need ARC routing controls or load-balancer-level failover.

Application Recovery Controller (ARC)

ARC adds two key capabilities for multi-region recovery beyond Route 53 health checks.

Readiness Checks

Continuously evaluate whether a backup region is ready to take over:

• Capacity match between regions (ASG sizes, RDS replica capacity).
• Configuration match (security group rules, VPC peering).
• Replication lag (DynamoDB global tables, RDS cross-region replicas).

Readiness Check returns a status (READY, NOT_READY) per resource group. The signal lets operators avoid failing over to a backup that is not actually ready.

Routing Controls and Clusters

Routing controls are explicit On/Off switches that flip Route 53 records without depending on health-check evaluation. Each routing control is paired with a Route 53 health check that watches the control state.

ARC creates a cluster of 5 control endpoints across regions. Operators flip routing controls via the cluster's regional endpoints; the cluster ensures any flip is durable across all 5 regions before acknowledging. The cluster delivers 99.999 percent availability for control-plane operations - higher than the application's data plane.

Safety rules at the routing control group level prevent dangerous combinations - "no more than 2 routing controls can be Off simultaneously" prevents accidentally failing over everything.

When ARC Beats Route 53 Health Checks Alone

• When the failure mode is upstream of the application (e.g., a dependency is degraded but the endpoint check still passes).
• When operators must manually approve a failover.
• When you need 99.999 percent control-plane availability for the failover trigger.
• When you want game-day testing without modifying health checks.

Active-Active vs Active-Passive Patterns

Two canonical multi-region patterns:

Active-Passive

Primary region serves all traffic; secondary is provisioned but receives nothing. RTO and RPO depend on:

• DNS TTL (failover delay).
• Health check detection (90 seconds typical).
• Backup region warm-up time (if scaled-down).
• Data replication lag (RPO).

Lower steady-state cost, higher RTO. Failover routing in Route 53 is the canonical mechanism.

Active-Active

Both regions serve traffic continuously. Latency-based routing distributes traffic to the closest healthy region. Failover is automatic and near-instant - if one region's records become unhealthy, traffic flows to the other.

Higher steady-state cost (running double capacity), lower RTO (no warm-up), and the active load itself validates the backup is ready.

Common Pitfalls (常考陷阱)

1. Attaching health checks to Simple routing records: not supported. Switch to Failover or Multivalue.
2. Treating DNS TTL as guaranteed: some resolvers cache longer; design for some clients seeing the old answer past TTL.
3. Forgetting INSUFFICIENT_DATA is treated as healthy by CloudWatch alarm health checks: explicitly handle missing data.
4. Using only endpoint checks for application-layer issues: endpoint check sees reachability, not error rate. Combine with CloudWatch alarm health checks.
5. Manual failover via DNS edits: too slow; use ARC routing controls or weighted-routing-with-traffic-flow.
6. Latency-based routing as a failover mechanism: latency routing returns the lowest-latency healthy region; for explicit primary/secondary, use Failover policy.
7. Forgetting that the management/billing-account context for ARC is regional: ARC clusters span regions but routing controls are scoped to the records they target.

FAQ

Q1: What is the fastest possible failover with Route 53?

Health check 10-second interval with 1 failure threshold = 10 seconds detection. TTL=10 seconds = 10 seconds DNS propagation. Best case ~20-30 seconds, but real-world resolvers may cache longer. For sub-second failover, use ALB target group failover within a region or Global Accelerator with cross-region failover.

Q2: When should I use ARC over Route 53 health checks alone?

When you need: explicit operator-controlled failover, readiness checking before failover, 99.999 percent control-plane availability, or game-day testing without affecting health checks. ARC adds explicitness and safety on top of Route 53 health checks - you typically use both.

Q3: How does Global Accelerator differ from Route 53 multi-region routing?

Global Accelerator uses anycast IPs from AWS's edge locations to route traffic over the AWS backbone, with sub-second failover when an endpoint becomes unhealthy. Route 53 uses DNS - bound by TTL caching. Global Accelerator is the better choice for latency-sensitive global apps; Route 53 is cheaper and works for all workloads.

Q4: Can Route 53 health checks evaluate metrics from other AWS accounts?

CloudWatch alarm health checks reference an alarm in the same account as the health check. For cross-account scenarios, use a CloudWatch composite alarm in the health-check-owning account that aggregates from a cross-account metric stream, or use ARC routing controls flipped by an EventBridge rule that spans accounts.

Q5: How do I test Route 53 failover?

Methods: (1) manually mark the primary's health check unhealthy in the Route 53 console; (2) deliberately break the endpoint (stop the ALB target); (3) flip an ARC routing control in a test cluster. Run game days quarterly to validate RTO assumptions.

Q6: What is the maximum number of records per failover policy pair?

A failover routing policy supports exactly two records: one primary, one secondary. For more than two regions in priority order, layer policies (e.g., primary returns one record, that record is itself a Latency policy across multiple regions).

Q7: Why do my Route 53 health checks show flapping?

Common causes: (1) endpoint behind an ALB without /healthz exclusion, where the ALB itself is overloaded; (2) string-matching failures on dynamic content; (3) TLS certificate or SNI issues for HTTPS checks; (4) resolver-level DNS issues for the health check target. Add CloudWatch alarms on the HealthCheckPercentageHealthy metric to spot patterns.

Wrap-Up

Route 53 multi-region failover is the composition of routing policies, health checks, DNS TTL, and (for high-stakes scenarios) Application Recovery Controller. Pick failover routing for active-passive, latency-based for active-active, weighted for canary, geolocation for regulatory. Compose endpoint health checks with calculated and CloudWatch alarm health checks to capture both reachability and application-layer health. Use ARC for explicit operator control, readiness verification, and 99.999 percent control-plane availability. Memorise the TTL bound on RTO, the 3-of-N endpoint check evaluation, the INSUFFICIENT_DATA semantics for CloudWatch alarm checks, and the 8 routing policy taxonomy. With those, multi-region failover scenarios resolve quickly.