Audit Logging and Transparency

Introduction to Audit Logging and Transparency

In a multi-tenant cloud environment, "Who did what, where, and when?" is the most critical question for security operations. For a Professional Cloud Architect, designing an audit strategy involves more than just enabling logs—it requires balancing data volume, cost, regulatory retention, and the ability to detect unauthorized access, even from the cloud provider itself.

Google Cloud provides three pillars of visibility: Cloud Audit Logs, Access Transparency, and Access Approval.

白話文解釋（Plain English Explanation）

Visibility is the "Black Box Flight Recorder" of your cloud environment.

Analogy 1 — The Building Security Camera (Audit Logs)

Think of Cloud Audit Logs as the Security Cameras in your office building. Admin Activity Logs are the cameras at the front door and server room—they record every time someone enters or changes a lock. Data Access Logs are the cameras inside the files—they record every time someone opens a drawer to read a document.

Analogy 2 — The Landlord's Key (Access Transparency)

Imagine you rent an apartment. Usually, you don't know if the Landlord (Google) uses their master key to enter while you're at work. Access Transparency is a system where the landlord MUST leave a note on your table every time they enter, explaining exactly why they were there (e.g., "Fixing a leaky pipe"). It gives you visibility into the provider's actions.

Analogy 3 — The Doorbell Camera with Remote Lock (Access Approval)

Access Approval is taking the landlord analogy one step further. It's like having a smart lock where the landlord can't even use their master key unless you press "Approve" on your phone. You see their request, see their reason, and only then let them in.

Types of Cloud Audit Logs

Google Cloud categorizes audit logs into four main types:

1. Admin Activity Logs: Record API calls or other actions that modify the configuration or metadata of resources. (e.g., Creating a VM, changing IAM roles). Enabled by default, No cost.
2. Data Access Logs: Record API calls that create, modify, or read user-provided data. (e.g., Reading a BigQuery table, downloading a file from GCS). Disabled by default (except for BigQuery), Can be high cost/volume.
3. System Event Logs: Record Google Cloud administrative actions that are not triggered by a direct user action (e.g., A system-triggered VM live migration).
4. Policy Denied Logs: Record when a user or service account is denied access because of a security policy violation (e.g., VPC Service Controls).

Admin Activity vs Data Access vs System Event vs Policy Denied: Retention and Enablement

Each of the four audit log streams behaves very differently with respect to default retention, cost, and the IAM permission required to read them. Architects who blur these distinctions inevitably either over-pay for storage or fail an audit because the wrong stream was active.

Admin Activity Logs — 400 days, free, always on

These capture every write API call (SetIamPolicy, Insert, Delete, Update) and arrive in the cloudaudit.googleapis.com/activity log name. The _Required bucket retains them for 400 days at no charge and you cannot disable them. They are the primary evidence stream for SOC 2 CC6.1 and ISO 27001 A.12.4.1 (event logging) controls. Reading them requires roles/logging.privateLogViewer or the more specific roles/logging.viewer plus logging.privateLogEntries.list.

Data Access Logs — 30 days, opt-in, expensive

These capture DATA_READ, DATA_WRITE, and ADMIN_READ sub-types. Outside BigQuery (where DATA_READ is on by default), you must explicitly enable them per-service in the IAM Audit Config (auditConfigs[]). They land in the _Default bucket with a 30-day default retention and they generate the bulk of an enterprise's Cloud Logging bill. A single chatty Cloud Storage bucket can produce tens of GiB per day. Use exempted members (exemptedMembers) to skip noisy service accounts such as Dataflow workers.

System Event Logs — 400 days, free, Google-triggered

These come from Google's own automation: VM live migrations, instance auto-restarts, Cloud SQL maintenance restarts. They share the 400-day free retention of Admin Activity. They are invaluable for explaining "Why did my VM reboot at 03:14?" during a post-mortem.

Policy Denied Logs — 30 days, charged

When VPC Service Controls, Org Policy, or IAM Conditions block an action, a Policy Denied entry is written. Architects pipe these to Pub/Sub and into Chronicle or a SIEM because they are the canary for exfiltration attempts and misconfigured service accounts.

Access Transparency: Seeing Google's Admin Access to Your Content

Audit Logs answer "what did MY users do." Access Transparency (AXT) answers "what did GOOGLE'S engineers do." When a Cloud Customer Care agent opens a support session and reads your bucket metadata, or when an on-call SRE accesses a hypervisor that hosts your VM, an Access Transparency log entry is written within minutes.

Required entitlement and supported services

Access Transparency is only available on Enterprise / Premium Support or Assured Workloads tiers. Supported services include Compute Engine, GKE, Cloud Storage, BigQuery, Cloud SQL, Spanner, Persistent Disk, Cloud Logging, and dozens more—but not every API. Always verify against cloud.google.com/assured-workloads/access-transparency/docs/supported-services for the current list before promising a customer "100% Google access visibility."

Anatomy of an AXT entry

Each entry contains a justification reason code such as CUSTOMER_INITIATED_SUPPORT, GOOGLE_INITIATED_SERVICE, or THIRD_PARTY_DATA_REQUEST, plus the office location of the accessor (e.g., "US"), the resource accessed, and the action performed. The accesses[].methodName and accesses[].resourceName fields let you correlate the access to a specific user-impact event.

Routing AXT logs

Access Transparency entries appear in Cloud Logging under the resource type audited_resource with logName: projects/PROJECT/logs/cloudaudit.googleapis.com%2Faccess_transparency. Most regulated customers route them via a folder-level log sink into a centralized security project and alert on any entry whose justificationReason is GOOGLE_INITIATED_SERVICE outside an expected maintenance window.

Access Approval: Requiring Google to Ask Before Touching Your Data

Where Access Transparency is a "rear-view mirror," Access Approval (AXA) is a "front-door buzzer." When a Google engineer needs to access your covered resources, the request is held in a pending state and an approval_request Pub/Sub notification is published. Until a designated approver clicks Approve in the Console or calls gcloud access-approval requests approve, the access does not happen.

Approver workflow

You configure one or more approver groups (typically a security distribution list) via gcloud access-approval settings update --enrolled_services=all [email protected]. Approvers receive an email and a Pub/Sub event containing the requested resource, the justification, and an expiration timestamp (default 24 hours). If no one approves in time, access is denied and Google support cannot proceed with that ticket.

Excluded scenarios

Access Approval does not block:

• Read-only access by automated systems for service health (covered by Access Transparency but not gated).
• Legal process requests where Google is compelled by court order.
• Customer-initiated emergencies where you have opened a P1 support case with the "auto-approve" flag.

Architectural integration

The PCA-exam pattern is to wire the Access Approval Pub/Sub topic into a Cloud Function that posts to a private Slack channel and creates a ServiceNow ticket. The approval action itself stays inside the Cloud Console with strong 2FA so that approval cannot be social-engineered.

Log Sinks to Cloud Storage with Bucket Lock: Tamper-Evident Audit Retention

For seven-year and ten-year retention horizons (HIPAA §164.316(b)(2)(i), SEC 17a-4(f), FINRA 4511), the cheapest and most defensible target is a Log Sink to Cloud Storage with Bucket Lock.

Pipeline design

1. Create an aggregated sink at the Organization level: gcloud logging sinks create org-audit-archive storage.googleapis.com/audit-archive-bucket --organization=ORG_ID --include-children --log-filter='logName:"cloudaudit.googleapis.com"'.
2. Grant the sink's writer identity roles/storage.objectCreator on the destination bucket.
3. On the bucket, apply a retention policy of 2555 days (7 years) and then lock it: gcloud storage buckets update gs://audit-archive-bucket --retention-period=2555d followed by gcloud storage buckets update gs://audit-archive-bucket --lock-retention-policy.
4. Enable Object Versioning and Bucket Lock so that even a project owner with storage.objects.delete cannot remove objects before the retention period expires.

Storage class and cost

Use Archive storage class for the sink destination—$0.0012/GB-month with millisecond access. A typical 50 GiB/day audit feed at 7-year retention costs roughly $150/month at Archive versus $4,500/month at Standard. Lifecycle rules are unnecessary because logs are write-once.

Why Bucket Lock matters

Without Bucket Lock, a compromised Organization Admin could gcloud storage rm the entire archive. With Bucket Lock, the retention policy itself is immutable—not even Google support can shorten it. This satisfies SEC 17a-4's "non-erasable, non-rewritable" (WORM) requirement.

BigQuery Log Analytics: SQL on Petabytes of Audit Data

When the question shifts from "store the logs" to "find the needle," route audit logs into BigQuery.

Two routing options

• Log Analytics-upgraded Log Bucket: Cloud Logging's 2023+ feature that lets you run SQL directly against a Log Bucket without exporting. Cheaper for ad-hoc queries because there is no storage duplication.
• Classic BigQuery sink: Export via bigquery.googleapis.com/datasets/audit_logs with partitioned tables (one per day). Better when you need to join audit data with non-log data (HR systems, asset inventories) or feed Looker dashboards.

Useful queries

-- Detect IAM role grants to external (non-corporate) identities
SELECT
  timestamp,
  protopayload_auditlog.authenticationInfo.principalEmail AS actor,
  protopayload_auditlog.resourceName,
  protopayload_auditlog.serviceData
FROM `security-project.audit_logs.cloudaudit_googleapis_com_activity_*`
WHERE protopayload_auditlog.methodName = 'SetIamPolicy'
  AND NOT REGEXP_CONTAINS(
    TO_JSON_STRING(protopayload_auditlog.serviceData),
    r'@mycompany\.com'
  )
  AND _TABLE_SUFFIX BETWEEN
    FORMAT_DATE('%Y%m%d', DATE_SUB(CURRENT_DATE(), INTERVAL 7 DAY))
    AND FORMAT_DATE('%Y%m%d', CURRENT_DATE());

Cost control

Always partition by _PARTITIONTIME and require partition filters on the dataset (require_partition_filter = true). A single un-partitioned SELECT * over a year of audit data can scan multiple TB and cost hundreds of dollars per query.

Alerting on IAM Policy Changes and Other High-Risk Events

The single highest-signal log-based alert in any GCP environment is "someone changed IAM." It is the most common precursor to data exfiltration.

Log-based metric

Create a counter metric on the filter:

logName:"cloudaudit.googleapis.com/activity"
protoPayload.methodName:("SetIamPolicy" OR "google.iam.admin.v1.CreateServiceAccountKey")

Alert policy

In Cloud Monitoring, build an alert that triggers on any non-zero value over a 1-minute window, with a notification channel to PagerDuty for roles/owner or roles/iam.securityAdmin grants, and a softer Slack channel for lower-privilege grants. Include the actor's principal email in the notification template so the on-call can immediately verify legitimacy.

Additional must-alert filters

• protoPayload.methodName="google.cloud.kms.v1.KeyManagementService.DestroyCryptoKeyVersion" — someone is destroying a KMS key.
• protoPayload.methodName="storage.buckets.update" AND protoPayload.serviceData.policyDelta involving allUsers — a bucket is being made public.
• protoPayload.methodName="compute.firewalls.insert" with sourceRanges:"0.0.0.0/0" — wide-open firewall created.

Audit Log Exclusion Filters: Cost Control Without Losing Compliance

Data Access logs from chatty services (Dataflow, Cloud Run, GKE control plane) can dominate your logging bill. The Log Router supports two reductive tools.

Sink exclusions

On the _Default sink, add exclusions[] filters that drop entries before they are billed:

gcloud logging sinks update _Default \
  --add-exclusion=name=dataflow-workers,filter='resource.type="dataflow_step"
    protoPayload.authenticationInfo.principalEmail:"dataflow-service-account@"' \
  --add-exclusion=name=health-check-noise,filter='httpRequest.userAgent:"GoogleHC"'

Excluded entries are not stored, not billed, not searchable. Use this only for entries you are 100% sure have no compliance value.

IAM Audit Config exempted members

For Data Access logs specifically, mark service accounts as exemptedMembers so their reads/writes never produce a log entry in the first place. This is finer-grained than a sink exclusion because it prevents log generation rather than dropping after the fact.

What you must never exclude

• Anything in cloudaudit.googleapis.com/activity (Admin Activity) — these are the compliance baseline.
• Access Transparency entries — auditors will ask for an unbroken trail.
• Policy Denied entries — the cost is tiny and the security value is enormous.

Compliance Evidence Collection for Auditors

The PCA exam frequently presents a scenario where an external auditor (PwC, Deloitte, a HIPAA assessor) asks for "evidence" of controls. Translating that into GCP artifacts is the architect's job.

Standard evidence packages

Automation pattern

Build a scheduled Cloud Run job that, on the first of every quarter, runs the relevant BigQuery queries and writes the result as a signed-URL PDF to a compliance-evidence GCS bucket with its own multi-year Bucket Lock. The auditor is given read access to that bucket—they pull evidence themselves, eliminating ad-hoc "please send me logs" emails.

Log Routing and Storage (Log Router)

Architects must decide where logs go based on the use case:

• Cloud Logging (Log Bucket): Good for quick searching and short-term analysis (30 days default).
• BigQuery (Sink): Best for complex SQL analysis, security forensics, and dashboards.
• Cloud Storage (Sink): Cheapest for long-term cold storage (e.g., 7-year retention for HIPAA).
• Pub/Sub (Sink): Used for real-time streaming to third-party SIEMs (like Splunk or Chronicle).

Access Transparency and Access Approval Summary

These features are essential for highly regulated industries (Banking, Government).

• Access Transparency: Provides near real-time logs of actions taken by Google personnel when they access your content (e.g., during a support ticket).
• Access Approval: Allows you to require explicit approval before Google personnel can access your data. This is the highest level of data sovereignty.

Retention and Immutability

• Immutability: Once an audit log is written, it cannot be changed or deleted by any user, including the owner.
• Retention Policies: You can set custom retention periods on Log Buckets or GCS buckets to meet regulatory requirements (e.g., SEC 17a-4).

Security vs. Transparency vs. Approval

FAQ — Audit Logging and Transparency

Q1. Why are Data Access logs disabled by default?

Because they generate massive amounts of data. For a high-traffic database, logging every "Read" operation can lead to significant storage costs and impact performance if not managed correctly.

Q2. Can I delete Audit Logs to save money?

No. Admin Activity logs are immutable and cannot be deleted within their retention period. To manage costs, filter out unnecessary Data Access logs using the Log Router.

Q3. How long are Admin Activity logs kept?

By default, they are kept for 400 days at no charge. You can export them to GCS or BigQuery if you need longer retention.

Q4. Does Access Transparency cover all Google Cloud services?

Most major services (Compute, SQL, GKE, Storage) are covered, but you should check the official documentation for the latest list of supported services.

Q5. What happens if I don't respond to an Access Approval request?

If you do not approve the request within the specified window, the Google personnel is denied access, and they cannot perform the task (e.g., they cannot troubleshoot your underlying hardware issue).

Final Architect Tip

On the PCA exam, pay attention to the Identity of the actor. If the actor is a "User" or "Service Account," look for Cloud Audit Logs. If the actor is a "Google Support Engineer," look for Access Transparency. Also, remember that for compliance audits, you often need to aggregate logs from multiple projects into a Centralized Logging Project using a "Log Sink" at the Folder or Organization level.