Incident Response and Forensics

Introduction to Incident Response (IR) in GCP

In the cloud, an incident can range from a misconfigured bucket to a full-scale credential theft. For a Professional Cloud Security Engineer (PSE), Incident Response is the process of detecting, containing, and recovering from these events. Forensics is the specialized discipline of gathering evidence from the cloud environment to understand the how, who, and what of the breach.

GCP provides specific tools to "freeze" an environment so it can be analyzed without alerting the attacker or losing volatile data.

白話文解釋（Plain English Explanation）

1. The Crime Scene Tape (Forensic Snapshots)

When a crime happens, the first thing police do is put up yellow tape to preserve the scene exactly as it is. In GCP, creating a Forensic Snapshot of a disk is that yellow tape. It creates a point-in-time copy of everything on the disk so you can look for evidence (malware, logs, deleted files) later.

2. The Black Box Recorder (Cloud Audit Logs)

Think of Cloud Audit Logs as the "Black Box" on an airplane. It records every pilot action (API call). If the plane crashes (a security incident), you check the black box to see who changed the firewall rule or who downloaded the sensitive dataset.

3. The Security Camera (Packet Mirroring)

Imagine a security camera watching the hallway of a building. It doesn't stop people from walking, but it records their every move. Packet Mirroring is that camera for your VPC network. It copies all network traffic and sends it to an IDS or a collector for analysis, allowing you to see exactly what data an attacker was sending out.

Incident Response Lifecycle in GCP

Google Cloud follows the NIST IR framework:

1. Preparation: Hardening the environment, setting up logging, and creating IR playbooks.
2. Detection & Analysis: Identifying the incident via SCC, Cloud Monitoring, or Audit Logs.
3. Containment, Eradication, & Recovery: Isolating affected resources, removing the threat, and restoring services.
4. Post-Incident Activity: Analyzing the root cause and improving defenses.

Forensic Data Collection

When an instance is compromised, you must collect evidence immediately.

1. Compute Engine Disk Snapshots

• Method: Create a standard snapshot of the persistent disk.
• Best Practice: Create the snapshot before you shut down the VM. Shutting down can trigger "kill scripts" left by attackers that delete evidence.
• Storage: Export the snapshot to a dedicated, locked-down "Forensics Project" to ensure the chain of custody.

2. Exporting Disks for Offline Analysis

• You can export a disk snapshot to Cloud Storage as a .tar.gz image.
• This allows you to download the image and analyze it using traditional forensic tools like Sleuth Kit or EnCase on your local workstation.

3. GKE Forensics

• Container Snapshots: Use kubectl cp to pull logs or files from a running container.
• Node Analysis: Since GKE runs on Compute Engine, you can snapshot the underlying GKE node disk to find container escape artifacts.

Network Forensics with Packet Mirroring

To investigate network-level exfiltration or lateral movement:

• Packet Mirroring clones traffic from specific VM instances or entire subnets.
• It sends the mirrored traffic to a load balancer that sits in front of a pool of inspection VMs (running tools like Zeek, Suricata, or Wireshark).

Responding to Compromised Service Accounts

Service account theft is a high-priority incident.

1. Identify: Use Cloud Audit Logs to see which IPs are using the service account keys.
2. Contain: Disable the service account immediately in the IAM console.
3. Eradicate: Delete any leaked JSON keys. Rotate the keys if the account is still needed.
4. Audit: Look for "Identity pivoting"—did the attacker use the SA to create other users or change permissions?

Automated Remediation with Cloud Functions

A PSE should automate response to common incidents.

• Flow: SCC Finding → Pub/Sub → Cloud Function.
• Example: If SCC detects a "Public Bucket," a Cloud Function can be triggered to automatically remove the allUsers member from the bucket's IAM policy.

Using Chronicle for Security Analytics

For large-scale incidents, use Chronicle.

• Search: Search across petabytes of logs in seconds to find an IP address or file hash.
• Timeline: Chronicle creates a visual timeline of an attacker's actions across multiple GCP projects and services.
• Intelligence: It automatically matches your logs against known threat intelligence feeds.

Incident Response Playbooks

An IR Playbook is a step-by-step guide for a specific scenario.

• Scenario: Ransomware on VM.
  1. Snapshot the disk.
  2. Isolate the VM (remove all network tags/firewall rules).
  3. Identify the entry point (Audit logs).
  4. Restore from a known good backup.

Communication and Reporting

During a breach, communication is critical.

• Communication Channels: Use a dedicated, out-of-band communication channel (e.g., a separate Slack workspace) in case your primary corporate email is compromised.
• Reporting: The final step of any IR is the Post-Mortem. Document what happened, why it happened, and how to prevent it.

Security Best Practices for PSE

1. Least Privilege for IR Teams: Give your IR team a specific "Forensics" role that allows them to take snapshots but not delete resources.
2. Enable Audit Logs: You cannot perform forensics without logs. Ensure Admin Activity and Data Access logs are on.
3. Immutable Storage: Store forensic evidence in GCS buckets with Object Retention Locks to prevent attackers from deleting the evidence of their crime.
4. Practice: Conduct "Tabletop Exercises" regularly to ensure everyone knows their role in the IR playbook.

PSE Exam Scenarios

Scenario 1: Preserving Evidence for Legal Action

"A company suspects an internal employee is stealing data from a specific Compute Engine instance. They need to preserve the evidence in a way that is valid for a legal investigation. What should they do?" Answer: Create a Disk Snapshot of the instance. Immediately export that snapshot to a separate, restricted GCP project and set a Retention Policy on the Cloud Storage bucket to ensure the data cannot be modified or deleted.

Scenario 2: Investigating a Network Breach

"An attacker has compromised a web server and is using it to scan other internal servers. You need to see exactly what commands they are sending over the network. VPC Flow Logs do not provide enough detail. What is the solution?" Answer: Enable Packet Mirroring on the compromised VM's subnet. Mirror the traffic to a pool of inspection instances running a tool like Wireshark or Zeek to perform deep packet inspection (DPI).

Summary Checklist

• List the four stages of the Incident Response lifecycle.
• Explain the benefit of taking a snapshot before shutting down a VM.
• Differentiate between VPC Flow Logs and Packet Mirroring.
• Describe how to isolate a compromised service account.
• Understand the role of Chronicle in long-term incident investigation.