Encryption and Key Management

Introduction to Encryption on GCP

For a Professional Cloud Architect, data protection is non-negotiable. Google Cloud encrypts all customer data at rest and in transit by default. However, many enterprises require higher levels of control to meet regulatory or compliance standards.

Understanding the spectrum of key management—from Google-managed to Customer-managed—is critical for passing the PCA exam.

Plain-Language Explanation: Encryption & Key Management

Encryption is like the difference between hiding a secret in your head and putting it in a high-tech safe.

Analogy 1 — The Russian Matryoshka Doll (Envelope Encryption)

Envelope Encryption is like a Matryoshka doll. You put your secret (Data) inside a small doll (Data Encryption Key - DEK). But the small doll is vulnerable. So, you put the small doll inside a bigger, much stronger doll (Key Encryption Key - KEK). You keep the bigger doll in a safe (Cloud KMS). To get the secret, you must first have permission to open the big doll.

Analogy 2 — The Hotel Safe (CMEK)

Google-Managed Keys are like a safe provided by the hotel. They have the master key, and they take care of everything. Customer-Managed Encryption Keys (CMEK) are like bringing your own High-Tech Padlock to the hotel safe. The hotel provides the safe (Storage), but you hold the only key to the padlock. If you lose the key or throw it away, even the hotel staff can't open the safe.

Analogy 3 — The Armored Vault (Cloud HSM)

Cloud HSM (Hardware Security Module) is like moving your gold from a standard bank safe to an underground, armored vault that is physically protected against tampering. If someone tries to drill into the vault, it self-destructs or locks down instantly. It's for when you need the highest level of physical security and certification (FIPS 140-2 Level 3).

Encryption At Rest: The Three Tiers

GCP offers three ways to manage your encryption keys:

1. Google-Default Encryption

• Managed by: Google.
• Effort: Zero.
• Cost: Free.
• Use Case: Default for almost all customers. Google handles key generation and rotation.

2. Customer-Managed Encryption Keys (CMEK)

• Managed by: You (via Cloud KMS).
• Effort: Low (You must create the key and grant permissions).
• Use Case: Regulatory compliance (e.g., HIPAA, PCI-DSS). You can disable the key to instantly "shred" your data (Crypto-shredding).
• Integration: Works with GCS, BigQuery, Compute Engine (PD), and many more.

3. Customer-Supplied Encryption Keys (CSEK)

• Managed by: You (Outside of GCP).
• Effort: High (You must provide the key with every API call).
• Use Case: Extreme security requirements where the key must never touch a Google disk in unencrypted form.
• Limitation: Only supported by Compute Engine and Cloud Storage.

Core KMS Concepts for Architects

1. Hierarchy of Keys

• Key Ring: A logical grouping of keys for organizational purposes.
• CryptoKey: A named resource representing a logical key.
• CryptoKey Version: The actual material used for encryption. A key can have multiple versions (for rotation).

2. Envelope Encryption (DEK & KEK)

• DEK (Data Encryption Key): Fast, used to encrypt the actual data. Stored with the data but encrypted by the KEK.
• KEK (Key Encryption Key): Used to encrypt the DEK. Stored in Cloud KMS.

3. Key Rotation

• Cloud KMS supports Automatic Rotation (e.g., every 90 days).
• Old versions are kept so that old data can still be decrypted, but all new data is encrypted with the newest version.

External Key Manager (EKM)

For organizations that need to keep their keys physically off the cloud, Google offers Cloud EKM.

• How it works: Cloud KMS communicates with an external key manager (e.g., Thales, Fortanix) via a secure bridge.
• Architect Tip: This is the "ultimate" control, but it adds latency and creates a dependency on your external provider. If the EKM is down, your GCP services cannot access the data.

Cloud HSM-Backed Keys and FIPS 140-2 Level 3

Cloud HSM is a managed HSM service where keys never leave a cluster of FIPS 140-2 Level 3 certified hardware modules operated by Google. It is exposed through the same Cloud KMS API surface — you pick protectionLevel: HSM when creating the CryptoKey instead of SOFTWARE.

When HSM is required, not just nice-to-have

• Regulated workloads: PCI-DSS, FedRAMP High, and many financial regulators explicitly require Level 3 modules for keys that protect cardholder data or sovereign records.
• Tamper-resistance guarantee: Level 3 means the module physically detects intrusion attempts and zeroizes the key material. Software-protected keys in Cloud KMS are still FIPS 140-2 Level 1 only.
• Attestation: HSM-backed keys return a signed attestation statement you can verify offline to prove a given key was generated inside a real Cloud HSM device.

Practical differences from software-protected keys

• Latency: HSM operations add roughly 10-30 ms per encrypt/decrypt call compared with software keys. For high-throughput streaming pipelines, always rely on envelope encryption so that only the DEK is wrapped by the HSM-protected KEK.
• Algorithm coverage: Cloud HSM supports AES-256-GCM symmetric, plus RSA 2048/3072/4096 and EC P-256/P-384 asymmetric keys. Some newer KMS algorithms (e.g., RAW AES) are software-only.
• Cost: HSM keys are billed per active key-version per month plus per operation — meaningfully more than software keys. Reserve them for the KEK tier, not the DEK tier.

Key Access Justifications (KAJ) and Provable Control

Key Access Justifications layers on top of Cloud EKM (and select EKM-via-VPC deployments) to give you cryptographic proof of why Google asked your external key manager to unwrap a key.

How KAJ works

1. A GCP service (e.g., BigQuery) needs to decrypt data protected by a CMEK that lives in your external EKM.
2. Cloud KMS forwards the unwrap request to your EKM with a signed justification code — values like CUSTOMER_INITIATED_ACCESS, GOOGLE_INITIATED_SYSTEM_OPERATION, or THIRD_PARTY_DATA_REQUEST.
3. Your external KMS policy engine inspects the justification and either approves or denies the unwrap. The decision is logged on your side, outside of Google's blast radius.

Why architects care

• Insider risk control: You can configure your EKM to deny any GOOGLE_INITIATED_* reason during a sensitive compliance window, meaning even a rogue Google operator with valid IAM cannot read the data.
• Regulatory evidence: KAJ records constitute auditable proof of denial for regulators that require "the cloud provider must not be able to access plaintext."
• Combined with Assured Workloads: KAJ is one of the building blocks behind Assured Workloads for regulated regions (e.g., eu.gcp.example.com sovereign controls), giving Bring-Your-Own-Key plus Bring-Your-Own-Decision.

Asymmetric vs Symmetric Keys in Cloud KMS

Cloud KMS supports three key purposes — pick the right one or you will hit INVALID_ARGUMENT at runtime.

Symmetric encryption (ENCRYPT_DECRYPT)

• Algorithm: AES-256-GCM.
• Use the same key to encrypt and decrypt. This is what every integration (CMEK on GCS, BigQuery, PD, Spanner, Pub/Sub) actually uses under the hood.
• Supports automatic rotation: new versions are created on a cron, old versions stay usable for decryption.
• API: projects.locations.keyRings.cryptoKeys.encrypt / decrypt.

Asymmetric encryption (ASYMMETRIC_DECRYPT)

• Algorithms: RSA-OAEP 2048/3072/4096 with SHA-256 or SHA-512.
• The public key is exported and used by external clients to encrypt; the private key never leaves KMS and is used through asymmetricDecrypt.
• Use case: a partner uploads files encrypted to your public RSA key; only services holding cryptoKeyDecrypter on the KMS resource can read them.
• No automatic rotation — you must mint a new key version and re-publish the public key.

Asymmetric signing (ASYMMETRIC_SIGN)

• Algorithms: RSA-PSS, RSA-PKCS1, EC P-256, EC P-384.
• Use case: signing JWTs for service-to-service auth, signing software artifacts (Binary Authorization attestations), or signing container images for supply-chain integrity.
• The asymmetricSign API returns a signature; verifiers fetch the public key from cryptoKeyVersions.getPublicKey and check it locally — no second KMS round-trip per verification.

MAC keys (MAC)

• Algorithm: HMAC-SHA-256.
• Use case: token validation, webhook signature verification. Both macSign and macVerify happen server-side in KMS.

Confidential VM and Confidential GKE Nodes

CMEK protects data at rest. Confidential Computing closes the remaining gap: data in use inside RAM.

Confidential VM

• Built on AMD SEV (Secure Encrypted Virtualization) or Intel TDX on supported n2d, c2d, c3, and n2d-confidential machine types.
• The hypervisor cannot read guest RAM — every page is encrypted by a per-VM key embedded in the AMD/Intel hardware.
• Performance overhead is typically 2-7%, mostly on memory-bound workloads.
• Combine with CMEK on the Persistent Disk and CMEK on Cloud Storage so that data is encrypted with customer keys end-to-end: at rest on disk, in transit via TLS, and in use in RAM.

Confidential GKE Nodes

• Set --enable-confidential-nodes on the node pool. Every node becomes a Confidential VM; pods scheduled there inherit the protection transparently — no application changes.
• Works with Workload Identity and CMEK-protected node boot disks to give the regulators-favorite combo: hardware memory isolation + customer-controlled disk encryption + workload-scoped IAM.
• Limitation: not all machine families support Confidential Nodes; GPU node pools and some accelerators are excluded.

When to require it on the PCA exam

• "Multi-tenant SaaS processing competitor data must not be visible to the cloud provider in memory" → Confidential VM.
• "ML training on patient genomes where the data sponsor must verify hardware isolation" → Confidential GKE Nodes + CMEK on training-data bucket + Assured Workloads.
• "Reduce memory exposure during a single short batch job" → Confidential VM is overkill if the data is encrypted client-side before upload; prefer client-side AEAD via Tink.

CMEK for BigQuery Column-Level Encryption

Standard CMEK encrypts an entire BigQuery dataset or table. For finer control over individual sensitive fields, BigQuery offers AEAD encryption functions that consume keysets minted by Cloud KMS (or by Tink).

How column-level CMEK works

1. Create a Cloud KMS symmetric key, e.g., projects/p/locations/us/keyRings/pii/cryptoKeys/ssn-col.
2. Grant roles/cloudkms.cryptoKeyEncrypterDecrypterViaDelegation to the BigQuery service agent in the project that holds the data.
3. Use SQL functions like AEAD.ENCRYPT(KEYS.KEYSET_CHAIN(...), plaintext, aad) and AEAD.DECRYPT_STRING(...) in queries. The keyset is fetched lazily from KMS at query compile time.
4. Combine with policy tags (Data Catalog) so that only IAM principals with fineGrainedReader on the tag can call AEAD.DECRYPT_STRING — everyone else sees ciphertext or NULL.

Patterns architects should know

• Deterministic columns: use DETERMINISTIC_ENCRYPT so that the same plaintext always yields the same ciphertext — preserves the ability to JOIN on the encrypted column without ever revealing plaintext.
• Crypto-shredding per tenant: store each tenant's keyset under a different KMS key. Disabling a tenant's key (gcloud kms keys versions disable ...) instantly renders that tenant's PII unreadable across every BigQuery table, materialized view, and BI cache — without rewriting a single row.
• Performance: column-level AEAD adds noticeable per-row CPU cost. For very wide fact tables, prefer table-level CMEK + DLP de-identification rather than encrypting every column.

Security and IAM for KMS

• Separation of Duties: The "Security Admin" should manage the keys, but the "Storage Admin" should only be able to use the keys.
• Granting Access: You must grant the service account of the resource (e.g., the GCS service account) the roles/cloudkms.cryptoKeyEncrypterDecrypter role.
• Audit Logs: Cloud KMS logs every time a key is used, providing a detailed trail for compliance.

FAQ — Key Management and Encryption

Q1. What is "Crypto-shredding"?

It is the process of making data unreadable by deleting the encryption key used to protect it. Even if the encrypted data still exists on disk, it is useless without the key.

Q2. Does CMEK slow down my application?

Generally, no. Because of Envelope Encryption, the KEK in KMS is only used to decrypt the DEK. Once the DEK is in memory, encryption/decryption happens at local speeds.

Q3. Can I use my own HSM with GCP?

Yes, via Cloud EKM, you can link GCP to your own on-premises HSM.

Q4. What happens if I lose my CSEK (Customer-Supplied Key)?

The data is gone forever. Google does not store a copy of your CSEK. There is no recovery process.

Q5. Is data encrypted in transit within GCP?

Yes. Google automatically encrypts data in transit between Google data centers and between your users and Google services using TLS/SSL.

Final Architect Tip

For the PCA exam, focus on the CMEK vs. CSEK distinction. Remember that CMEK is the standard enterprise recommendation for compliance. If a question mentions "FIPS 140-2 Level 3," the answer is Cloud HSM. If it mentions "total control over keys physically residing on-premises," the answer is Cloud EKM. Also, always verify that the Service Account of the calling service has permission to use the KMS key.