Storage and Database Selection

The Storage Continuum: Choosing the Right Home for Your Data

In the GCP PCA exam, storage and database selection is one of the most high-stakes areas. Choosing the wrong storage type can lead to performance bottlenecks, data corruption (due to consistency issues), or massive cost overruns.

• Object Storage: Cloud Storage. For unstructured data (images, backups, logs).
• Relational (SQL): Cloud SQL (Regional) and Cloud Spanner (Global/Scalable).
• NoSQL: Bigtable (High-throughput time-series) and Firestore (Document-based mobile/web).
• Block Storage: Persistent Disk (for VMs).
• File Storage: Filestore (NFS).

The "Optimal" choice depends on the Volume, Velocity, Variety, and Consistency requirements of the data.

Plain-Language Explanation: Storage and Database Selection

Choosing a storage service is like choosing how to store items in your life.

Analogy 1 — The Infinite Cardboard Box (Cloud Storage)

Cloud Storage is like an infinite supply of cardboard boxes in a giant warehouse. You can throw anything inside—a photo, a long document, a backup of your hard drive. You don't need to organize what's inside the box for the warehouse to store it. It's cheap, it's huge, and it's perfect for things you don't need to search through constantly.

Analogy 2 — The Bank's Ledger (Cloud SQL & Spanner)

Relational databases are like a bank's ledger. Every entry must follow a strict format (Schema), and the math must always add up (ACID compliance).

• Cloud SQL is like a local bank branch. It works perfectly for your town, but if millions of people from around the world try to enter the branch at once, it gets crowded and slow.
• Cloud Spanner is like a magical, global bank ledger that updates instantly everywhere in the world at the same time. It's more expensive to run, but it's the only way to handle global transactions safely.

Analogy 3 — The High-Speed Ticker Tape (Bigtable)

Bigtable is like a Wall Street ticker tape. It's designed to record millions of tiny pieces of information (like stock prices or sensor readings) every single second. It doesn't care about complex relationships; it just cares about speed and volume. You use it when you need to write data as fast as a firehose.

Deep Dive: Cloud Storage (Object)

• Standard: Frequently accessed data.
• Nearline: Accessed < 1 time per month (e.g., monthly reports).
• Coldline: Accessed < 1 time per quarter (e.g., legal archives).
• Archive: Accessed < 1 time per year (e.g., data kept for 10 years).
• Lifecycle Management: Automatically move data to cheaper tiers or delete it based on age.

Cloud Storage Class Decision Tree

Choosing the wrong Cloud Storage class is one of the most common — and quietly expensive — mistakes on the PCA exam. The four classes (STANDARD, NEARLINE, COLDLINE, ARCHIVE) trade storage cost against retrieval cost, minimum storage duration, and early-deletion fees. Pricing is broadly an inverse curve: Archive is roughly 17x cheaper than Standard for storage but charges higher per-GB retrieval, and deletes incurred before the minimum duration are billed as if the object had lived the full period.

Decision Tree by Access Frequency

• Multiple times per month, latency-sensitive (web, CDN origin, hot analytics): STANDARD. No minimum duration, lowest retrieval cost, can be paired with Cloud CDN or Media CDN for egress savings.
• Roughly monthly access (monthly reports, recent backups within 30 days): NEARLINE. 30-day minimum storage duration.
• Quarterly access (DR copies, compliance reads): COLDLINE. 90-day minimum.
• Rare, regulated, long-tail (10-year audit logs, raw lab data): ARCHIVE. 365-day minimum. First-byte latency is still milliseconds — it is NOT tape.

Location Type (orthogonal to class)

• Region (e.g., asia-east1): cheapest, single-region durability (11 nines), tightest latency for co-located compute.
• Dual-region (e.g., asia1, or custom predefined pairs): synchronous geo-redundancy with turbo replication option for ≤15 min RPO.
• Multi-region (e.g., US, EU, ASIA): geo-redundant across a continent, best for global read distribution.

Automating Tier Movement

Use Object Lifecycle Management rules (JSON), evaluated daily:

{ "lifecycle": { "rule": [
  { "action": {"type": "SetStorageClass", "storageClass": "NEARLINE"},
    "condition": {"age": 30, "matchesStorageClass": ["STANDARD"]} },
  { "action": {"type": "SetStorageClass", "storageClass": "COLDLINE"},
    "condition": {"age": 90} },
  { "action": {"type": "SetStorageClass", "storageClass": "ARCHIVE"},
    "condition": {"age": 365} },
  { "action": {"type": "Delete"}, "condition": {"age": 2555} }
]}}

Combine with Autoclass when access patterns are unpredictable — Autoclass moves objects between classes automatically based on observed access, removing the need to model the rules yourself. Reference: https://cloud.google.com/storage/docs/lifecycle

Deep Dive: Relational Databases (SQL)

• Cloud SQL: Managed MySQL, PostgreSQL, and SQL Server. Best for standard web apps and regional workloads. Supports HA with failover replicas.
• Cloud Spanner: The world's first "NewSQL" database. It combines the scalability of NoSQL with the ACID compliance of SQL. It is the "Optimal" choice for global inventory or financial systems.

Deep Dive: NoSQL Databases

• Bigtable: Wide-column store. Use for high-throughput IoT, AdTech, or Financial telemetry. It is the backbone of Search and Gmail.
• Firestore: Document database. Best for mobile/web app backends, user profiles, and real-time syncing. It scales to millions of users automatically.

Block and File Storage

• Persistent Disk (PD): Attached to Compute Engine or GKE.
  • Balanced: Best for general purpose.
  • SSD: Best for high-performance databases.
  • Extreme: For the most demanding I/O.
• Filestore: Managed NFS for applications that need a shared file system (e.g., legacy media editing tools).

Persistent Disk and Hyperdisk: Picking the Right Block Storage

GCP's block-storage portfolio has expanded well beyond classic Persistent Disk. Hyperdisk decouples capacity from IOPS and throughput so you can tune them independently — a major architectural shift from PD, where performance scales linearly with size.

Classic Persistent Disk (PD)

• pd-standard (HDD): Cheapest, sequential throughput, ideal for cold boot disks and log archives.
• pd-balanced (SSD): Default choice — SSD-class latency at a lower price than pd-ssd. Use for most application servers and mid-tier databases.
• pd-ssd: Lower latency and higher IOPS-per-GB ceiling than pd-balanced. Sweet spot for OLTP databases (Postgres, MySQL self-managed) up to ~30 TB.
• pd-extreme: Custom-provisioned IOPS up to 120,000 per disk. Use for the largest single-disk workloads (SAP HANA, large MS SQL, in-memory caches with persistence).

Hyperdisk Family (next-gen)

• Hyperdisk Balanced: Successor to pd-balanced. Independent IOPS / throughput tuning. The new default on C3, N4, H3 machine families.
• Hyperdisk Extreme: Up to 350,000 IOPS, 5,000 MB/s. Replaces pd-extreme on supported families. Use for tier-0 OLTP and SAP HANA.
• Hyperdisk Throughput: Optimized for sequential scans (cost-effective for log ingestion, Hadoop scratch, Kafka tiered storage). Up to 600 MB/s per disk at HDD-class pricing.
• Hyperdisk ML: Read-mostly disk shared across many GPU/TPU VMs (up to 2,500 concurrent attach). Designed for ML training datasets and serving model weights with massive aggregate read throughput.

Choosing Quickly

Reference: https://cloud.google.com/compute/docs/disks

Filestore Tiers: Picking a Managed NFS

Filestore is GCP's managed NFSv3 service. Tier selection drives both performance ceiling and minimum capacity:

• Basic HDD: 1 TiB – 63.9 TiB. Low cost, suitable for dev/test home directories, content management, and small CMS deployments.
• Basic SSD: 2.5 TiB – 63.9 TiB. SSD-class latency for general-purpose workloads needing shared file access (GKE shared volumes via the Filestore CSI driver, legacy NFS apps).
• Zonal (formerly High Scale): 10 TiB – 100 TiB. Higher IOPS and throughput scaling, supports backups. Good for HPC scratch space and rendering farms.
• Enterprise: 1 TiB – 10 TiB. Regional (multi-zone) HA with synchronous replication — the only tier with a regional SLA. Required for shared file systems that need to survive a zonal failure (e.g., SAP shared volumes, EDA).

For workloads needing POSIX semantics at petabyte scale (HPC, life-sciences pipelines), step up to Parallelstore (Lustre-compatible) instead of Filestore.

Relational and NoSQL Selection: Cloud SQL vs Spanner vs Firestore vs Bigtable

The four flagship databases solve different problems. Map workload shape to the right service before considering anything else.

Cloud SQL

• Managed MySQL, PostgreSQL, SQL Server.
• Regional, vertically scaled (largest instance ~128 vCPU / 864 GB), HA via synchronous standby in another zone.
• Read scaling: up to 10 read replicas (cross-region allowed).
• Picks: classic web app backends, WordPress, GitLab, regional OLTP up to a few TB.

Cloud Spanner

• Horizontally scalable, globally distributed, externally consistent (TrueTime).
• Strong consistency across regions, ACID transactions, SQL surface (GoogleSQL or PostgreSQL dialect).
• Granular billing: Processing Units (100 PU = 1/10 of a node) make it viable at small scale (~$65/month entry).
• Picks: global inventory, financial ledgers, multi-region SaaS where reads/writes happen worldwide.

Firestore (Native Mode)

• Serverless document database, automatic multi-region replication.
• Strong consistency within a document, eventual across queries by default; supports ACID across up to 500 documents per transaction.
• Real-time listeners (onSnapshot) push changes to clients — killer feature for mobile/web sync.
• Picks: mobile/web app backends, user profiles, chat, collaborative editing state.

Bigtable

• Wide-column NoSQL (HBase API compatible), single-digit-millisecond reads at millions of QPS.
• No SQL, no secondary indexes (other than the row key) — schema design is row-key design.
• Picks: time-series (IoT, financial tick data), AdTech profiles, large operational analytics serving layer feeding Looker or Data Studio.

Quick Selection Cheat Sheet

Reference: https://cloud.google.com/architecture/database-design-guidelines

AlloyDB for PostgreSQL Workloads

AlloyDB for PostgreSQL is GCP's managed Postgres-compatible database tuned for enterprise workloads that outgrow Cloud SQL but don't need Spanner's global writes. It's positioned as Google's response to Amazon Aurora and SQL Server-grade workloads on Postgres.

Architectural Highlights

• Decoupled compute and storage: Log-structured storage layer with regional triple-redundant log writes. Adding a read replica does not duplicate the dataset — a major cost win over Cloud SQL read replicas.
• Columnar accelerator: An in-memory columnar engine sits next to the row store and auto-selects which columns to keep hot, accelerating analytic queries on the same primary database (HTAP). Cloud SQL has no equivalent.
• Read pool nodes: Up to 20 read replicas with sub-second lag.
• pgvector and AlloyDB AI: Built-in vector search, model endpoint management for embedding generation via google_ml. Useful for RAG over operational data without a separate vector DB.

When to Pick AlloyDB Over Cloud SQL

• Postgres workload > 10 TB or > 64 vCPU on a single primary.
• Mixed OLTP + analytic queries on the same dataset (HTAP) — would otherwise need a Postgres + BigQuery pipeline.
• Need fast read scaling with low replication cost.
• Vector search co-located with transactional data.

When NOT to Pick AlloyDB

• MySQL or SQL Server workloads (AlloyDB is Postgres-only — use Cloud SQL).
• Need multi-region active-active writes (use Spanner with PostgreSQL dialect).
• Tiny workloads where Cloud SQL Postgres at < $50/month is sufficient.

Reference: https://cloud.google.com/alloydb/docs/overview

Memorystore for Redis and Valkey

Memorystore is GCP's managed in-memory caching service. As of 2026 it supports three engines, and engine choice matters more than ever now that Redis has relicensed.

Engines

• Memorystore for Redis: Original offering, fully managed Redis (latest supported version per release notes). Redis Cluster mode and standalone HA tiers. Picks: legacy compatibility, existing Redis ecosystem.
• Memorystore for Valkey: Open-source fork of Redis 7.2 maintained by the Linux Foundation. Wire-compatible with Redis clients. GCP added Valkey to give customers an OSI-licensed long-term option after Redis Inc.'s license change. Picks: new builds wanting BSD-licensed engine, lower per-instance cost.
• Memorystore for Memcached: Pure multi-tier cache, no persistence, no replication. Picks: session stores, simple cache-aside for read-heavy web apps.

Tier Selection (Redis / Valkey)

• Basic Tier: Single node, no replica. Cheap, no HA. Dev/test only.
• Standard Tier: Primary + replica with automatic failover, regional. Optional read replicas up to 5.
• Cluster mode: Sharded across multiple shards (up to 250) for capacity > 300 GB or > 100,000 ops/sec needs.

Common PCA Patterns

• Session cache for stateless Cloud Run / GKE services → Memorystore for Redis (Standard).
• Leaderboard / rate limiter → Redis sorted sets via Memorystore.
• Sidecar cache for Cloud SQL hot rows → Memorystore for Redis with read-through pattern from the app tier.

For asynchronous queues (vs. caches), prefer Pub/Sub or Cloud Tasks instead of pushing message-queue semantics into Redis.

Reference: https://cloud.google.com/memorystore/docs

BigQuery vs Cloud Storage Data Lake Patterns

PCA scenarios often pit a BigQuery-centric warehouse against a Cloud Storage-based data lake. The optimal answer depends on access pattern, schema stability, and downstream tooling.

Pattern 1 — BigQuery as the Primary Store

• Data lands in BigQuery via Storage Write API, Datastream (CDC), or Dataflow.
• Tables are partitioned (by ingestion time or DATE column) and clustered on high-cardinality filter columns.
• Storage is BigQuery-managed (columnar Capacitor format), and you pay separate storage + compute (slots).
• Picks: SQL-first analytics teams, BI dashboards, well-known schemas.

Pattern 2 — Cloud Storage Data Lake + BigQuery External Tables

• Raw data lands in gs:// buckets as Parquet, ORC, Avro, or JSONL, partitioned by date prefix.
• BigQuery queries the data via BigLake tables — external tables with fine-grained IAM, row/column-level security, and metadata caching.
• Same files can be read by Dataproc (Spark), Dataflow, and Vertex AI training jobs without duplication.
• Picks: multi-engine access, ML training datasets, schema-on-read workflows, very cold data kept cheap on Cloud Storage.

Pattern 3 — Open Lakehouse with Iceberg

• Cloud Storage holds Apache Iceberg tables; BigQuery reads/writes them via BigLake managed tables for Apache Iceberg.
• Gives ACID transactions, time travel, and schema evolution on a Cloud Storage lake, with BigQuery's SQL engine on top.
• Picks: organizations standardizing on open table formats while keeping BigQuery as the query engine.

Decision Heuristic

• Schema is stable, queries are exclusively SQL → BigQuery managed storage.
• Need cheap raw retention plus ML training plus SQL → Cloud Storage + BigLake.
• Want open table format, multi-engine, ACID on a lake → BigLake Iceberg.

Reference: https://cloud.google.com/bigquery/docs/biglake-intro

Summary of Optimal vs. Viable Decisions in Storage

FAQ — Storage Selection Strategy

Q1. When should I choose BigQuery over Bigtable?

Use Bigtable for real-time, low-latency operational data (serving an app). Use BigQuery for analytical data (running long reports and SQL queries on petabytes of history).

Q2. Is Cloud Spanner worth the cost?

If your business depends on global data integrity (no double-selling, no incorrect balances), then yes. The cost of a Spanner instance is much lower than the business cost of a major data corruption event.

Q3. Can Cloud SQL scale globally?

Cloud SQL can have Cross-region Read Replicas, but writes must go to the primary region. This makes it "Globally Readable" but not "Globally Writeable" like Spanner.

Q4. What is "Multi-regional" Cloud Storage?

It stores your data in at least two geographic locations separated by at least 100 miles. This ensures that even if a whole region (like us-east1) is destroyed, your data is still safe in another region.

Q5. What is the limit of Firestore?

Firestore is incredibly scalable for document-sized data. However, for massive, high-speed time-series data, Bigtable is more "Optimal" because of its wide-column structure and lower cost per write.

Final Architect Tip

"Schema vs. Speed." If your data has a strict schema and needs transactions, go SQL (Cloud SQL/Spanner). If your data is messy, massive, and needs to be written fast, go NoSQL (Bigtable/Firestore). If you need to store a 10GB file, go Object (Cloud Storage). Memorize this decision tree, and the PCA exam will be a breeze.