Cloud IDS and Network Threat Detection

Introduction to Cloud IDS

While firewalls (L4) and Cloud Armor (L7) block known bad traffic, Cloud IDS (Intrusion Detection System) is designed to "listen" to the traffic that is already inside your network. It provides deep packet inspection (DPI) to identify malicious activity such as malware, spyware, and command-and-control (C2) communications that might bypass other defenses.

For a Professional Cloud Security Engineer (PSE), Cloud IDS is the "Surveillance Camera" of the VPC. It doesn't block traffic (it is not an IPS), but it provides critical visibility into lateral movement and data exfiltration attempts, powered by industry-leading threat intelligence from Palo Alto Networks.

白話文解釋（Plain English Explanation）

1. The Surveillance Camera (IDS)

VPC Firewalls and Cloud Armor are like locked doors and security guards. Cloud IDS is the Surveillance Camera in the hallway. It doesn't stop someone from walking through the door, but it records exactly what they are doing once they are inside. If it sees someone trying to pick a lock on a safe (a server), it alerts the security team.

2. The Wiretap (Packet Mirroring)

Imagine you suspect a spy is operating within your building. Instead of stopping everyone, you Wiretap the phone lines (Packet Mirroring). You send a copy of every conversation to a team of experts (Cloud IDS) who listen for code words or suspicious instructions. The callers don't know they are being recorded, and their conversation is not delayed.

3. The Blood Test (Threat Analysis)

Think of Cloud IDS as a Blood Test for your network. It analyzes the "flow" of data for signs of infection (Malware) or foreign agents (Spyware). It doesn't stop the infection directly, but it tells the doctor (the PSE) exactly what is wrong so they can prescribe the right treatment (Firewall rule or incident response).

Architecting Cloud IDS for VPC Security

Cloud IDS is built on a "service-producer" model using Private Service Connect (PSC).

Key Components:

• IDS Endpoint: A managed resource in a Google-owned project that is peered with your VPC.
• Packet Mirroring: A VPC feature that clones traffic from selected VMs or subnets and sends it to the IDS endpoint.
• Threat Detection Engine: Powered by Palo Alto Networks, it analyzes the mirrored traffic against thousands of known threat signatures.

Managing IDS Endpoints and Packet Mirroring

IDS Endpoint Setup:

1. Enable the Cloud IDS API.
2. Create a Private Service Connect (PSC) connection to the IDS service.
3. Create an IDS Endpoint in a specific region.

Packet Mirroring Policy:

• Mirrored Sources: Can be specific subnets, network tags, or individual VM instances.
• Collector: The IDS Endpoint you created.
• Traffic Filter: You can choose to mirror all traffic, or only specific protocols/ports to save on processing costs.

Analyzing Threats: Malware, Spyware, and C2

Cloud IDS detects a wide range of threats using signatures.

• Malware: Detects files being downloaded that match known malicious hashes.
• Spyware: Identifies traffic patterns indicative of data being exfiltrated to suspicious domains.
• Command and Control (C2): Detects "Beaconing" traffic where a compromised VM is calling back to an attacker's server for instructions.
• Vulnerability Scans: Identifies internal or external actors trying to scan your network for open ports or known bugs.

Integration with Security Command Center (SCC)

Cloud IDS does not exist in a vacuum. Its findings are automatically sent to Security Command Center (SCC).

• Findings Dashboard: Provides a centralized view of all IDS alerts across multiple projects.
• Severity Levels: Alerts are categorized as Informational, Low, Medium, High, or Critical.
• Next Steps: SCC provides recommendations for mitigating the threat (e.g., "Isolate this VM using a firewall rule").

Performance Impact and Throughput Considerations

Because Cloud IDS uses packet mirroring, there is zero latency impact on your application.

• Throughput: Each IDS endpoint has a specific throughput capacity (e.g., 5 Gbps).
• Scalability: If you have extremely high traffic, you may need multiple IDS endpoints and multiple packet mirroring policies.

Multi-VPC and Shared VPC Inspection

Cloud IDS works seamlessly in complex network topologies.

• Shared VPC: The IDS endpoint can be created in the Host Project, and packet mirroring can be applied to VMs in any Service Project.
• VPC Peering: You must create an IDS endpoint in each VPC where you want to inspect traffic, as packet mirroring does not cross VPC peering boundaries by default.

IDS Rule Profiles and Threat Severity

You can choose different Threat Exception profiles to reduce noise.

• Default Profile: Detects all critical and high-severity threats.
• Custom Exceptions: If you have a legitimate tool that triggers an IDS alert (e.g., a vulnerability scanner), you can "Silence" that specific signature in the IDS policy.

Alerting and Investigation Workflows

When Cloud IDS finds a threat:

1. Alert: An entry is created in Cloud Logging and SCC.
2. Notification: Use Pub/Sub or SCC Notifications to send an alert to your SOC (Security Operations Center) or via PagerDuty/Slack.
3. Investigation: Use Cloud Logging to view the raw packet metadata (source IP, destination IP, port, protocol, and the specific threat signature ID).

Comparing Cloud IDS with Partner Solutions

Why choose Cloud IDS over a third-party NVA (Network Virtual Appliance) like a Fortinet or Check Point VM?

• No Management: Cloud IDS is fully managed; no need to patch or scale the underlying VMs.
• Native Integration: Integrated with GCP billing, IAM, and SCC.
• No Bottleneck: Because it's out-of-band, it can't crash your network.

Automated Response to IDS Findings

While Cloud IDS doesn't block traffic, you can automate the response using Cloud Functions.

• Workflow: IDS Find → SCC Alert → Pub/Sub → Cloud Function → Update Firewall Rule to DENY source IP.
• This provides a "Closed-Loop" security system that reacts to threats in seconds.

CLI Commands for Cloud IDS

Creating an IDS Endpoint

gcloud ids endpoints create my-ids-endpoint \
    --network=my-vpc \
    --region=us-central1 \
    --severity=MEDIUM

Creating a Packet Mirroring Policy

gcloud compute packet-mirrorings create my-mirroring \
    --region=us-central1 \
    --network=my-vpc \
    --collector-ilb=ids-endpoint-forwarding-rule \
    --mirrored-subnets=my-subnet

Security Best Practices for PSE

1. Mirror "East-West" Traffic: Don't just look at traffic coming from the internet. Mirror traffic between your subnets to detect lateral movement.
2. Filter Mirroring: Only mirror traffic you care about (e.g., skip internal backup traffic) to stay within throughput limits.
3. Use SCC Premium: SCC Premium provides much deeper analysis and historical context for IDS findings.
4. Regularly Review Exceptions: Ensure that "Silenced" alerts are still valid and haven't become blind spots.

Troubleshooting Scenarios

Scenario: IDS is not showing any threats despite known attacks

Diagnosis: Check the Packet Mirroring Policy. Ensure the source VMs are correctly targeted and that the traffic isn't being dropped by a VPC firewall before it can be mirrored. Fix: Verify the Packet Mirroring status and ensure the traffic matches the mirroring filters.

Scenario: IDS Endpoint creation is failing with a "Network" error

Diagnosis: You might be missing the Private Service Access range or the peering between your VPC and the Google IDS project is not established. Fix: Configure Private Service Access for your VPC before creating the IDS endpoint.

PSE Exam Scenarios

Scenario 1: Detecting Lateral Movement

"A security engineer suspects that a compromised VM in the 'web' subnet is attempting to scan the 'database' subnet. What is the most effective way to confirm this without impacting network performance?" Answer: Deploy Cloud IDS. Create a packet mirroring policy that mirrors all traffic from the 'web' subnet and send it to an IDS endpoint for analysis.

Scenario 2: Regulatory Compliance for DPI

"A financial regulation requires Deep Packet Inspection (DPI) for all traffic containing PII. How can this be implemented in a managed way?" Answer: Use Cloud IDS. It provides managed DPI and integrates with SCC to report findings, satisfying the regulatory requirement for intrusion detection.

Summary Checklist

• Describe the difference between IDS (Detection) and IPS (Prevention).
• Explain how Packet Mirroring works with Cloud IDS.
• List the steps to create an IDS endpoint using PSC.
• Identify the threat categories detected by Cloud IDS (Malware, C2, etc.).
• Understand the role of SCC in managing IDS findings.