Cloud Build for CI/CD

Introduction to Cloud Build CI/CD

Cloud Build is Google Cloud's serverless build platform. You hand it a YAML file (or a Dockerfile) and a source location, and it runs each step inside a container, in order, on infrastructure that Google manages. There are no Jenkins agents to patch, no GitHub runners to autoscale, no Buildkite VMs to size. The unit of work is the step; the unit of billing is the build minute.

For the GCP Professional Cloud Developer (PCD) exam, Cloud Build sits at the intersection of three big areas: continuous integration, container packaging, and supply chain security. The exam tests whether you can wire up a trigger correctly, fetch a secret without leaking it, pick the right pool for a network-bound build, gate production with manual approval, and read SLSA provenance to prove what shipped. This study note covers each of those, plus the IAM permissions that make or break a real team's pipeline.

Plain-Language Explanation: (Plain English Explanation)

Before diving into YAML syntax and IAM bindings, three analogies. The point is not to memorize feature names but to feel what each Cloud Build moving part actually does.

Cloud Build as a factory assembly line

A cloudbuild.yaml is the blueprint for an assembly line. Each step is a station. The first station might unpack the crate (git clone, already done by Cloud Build's fetcher), the next station runs unit tests, another station builds a container image, another pushes it to Artifact Registry, and the final station deploys to Cloud Run. Every station is staffed by a specialist robot, which in Cloud Build is just a Docker container with the tool installed (the gcr.io/cloud-builders/* images, or your own).

The conveyor belt is the /workspace volume that travels between steps. Anything one station drops on the belt is visible to the next station. If you need parallel work (run unit tests and lint at the same time), you tell two stations they can start as soon as the previous one finishes by using waitFor. If you build everything sequentially without waitFor, the line runs single-file and slow.

Build triggers as a doorbell

A trigger is a doorbell wired to a specific door. Push to main rings the production doorbell. Open a pull request and the PR doorbell rings. Tag a release as v1.2.3 and the release doorbell rings. The same factory answers every doorbell, but each ring sends a different cloudbuild.yaml down the line, with different substitution variables. The doorbell does not care who pushed the button (well, it does for IAM); it cares which door rang and what the visitor brought.

A manual trigger is a doorbell that only the owner can ring on purpose, from the Cloud Console or gcloud builds triggers run. A webhook trigger is a doorbell wired directly to the street, so anything that can POST to the URL with the right secret can ring it. Useful for Jira, Linear, or any SaaS that does not speak GitHub.

Private pools as a private workshop

The default Cloud Build pool is a shared public workshop on the internet. Fast to spin up, but the workshop has a public address. If your build needs to reach a Cloud SQL instance over a private IP, an on-prem artifact server over a Cloud VPN, or a self-hosted GitLab behind a corporate firewall, the public pool cannot get there.

A private pool is your own private workshop, dropped inside a VPC peering with your environment. Builds run there with a static egress and access to private resources. You pay more (you reserve capacity) but you can build against private networks, lock down egress, and pick a beefier machine for slow Bazel or Gradle builds without queuing behind every other shared-pool customer.

cloudbuild.yaml Anatomy and Build Steps

The cloudbuild.yaml file is the spine of Cloud Build. Everything else (triggers, IAM, provenance) is configuration around this YAML. Understand it deeply and the rest falls into place.

Top-level keys

A typical build config looks like:

steps:
  - name: 'gcr.io/cloud-builders/docker'
    args: ['build', '-t', 'us-central1-docker.pkg.dev/$PROJECT_ID/app/api:$SHORT_SHA', '.']
  - name: 'gcr.io/cloud-builders/docker'
    args: ['push', 'us-central1-docker.pkg.dev/$PROJECT_ID/app/api:$SHORT_SHA']
images:
  - 'us-central1-docker.pkg.dev/$PROJECT_ID/app/api:$SHORT_SHA'
options:
  logging: CLOUD_LOGGING_ONLY
  machineType: E2_HIGHCPU_8
timeout: 1200s

steps is the ordered list of build steps. images is a convenience field: any image you list there is automatically pushed to Artifact Registry (or Container Registry) at the end of a successful build. options controls global build behavior (machine type, logging mode, substitution option, source provenance hashing). timeout sets the overall job timeout; default is 10 minutes, max is 24 hours.

Builders and step containers

Each step's name is a container image. Cloud Build provides a curated set of cloud-builders (gcloud, docker, kubectl, gsutil, mvn, npm, yarn, go, gradle, gke-deploy, helm, bazel, and friends). You can also point name at any image on Artifact Registry, Docker Hub, or GitHub Container Registry. The build runs docker run <name> <args> for each step.

The entrypoint field overrides the container's default entrypoint, useful when you want to run a shell rather than the image's main binary. The env field injects environment variables. The dir field changes the working directory inside /workspace. The script field (newer) lets you write inline shell instead of marshaling everything through args.

Parallel steps with waitFor

By default, each step waits for the previous one. To run in parallel, give each step an id and use waitFor to point at an earlier id (or - for "run immediately"):

steps:
  - id: 'install'
    name: 'node:20'
    entrypoint: 'npm'
    args: ['ci']
  - id: 'test'
    name: 'node:20'
    waitFor: ['install']
    entrypoint: 'npm'
    args: ['test']
  - id: 'lint'
    name: 'node:20'
    waitFor: ['install']
    entrypoint: 'npm'
    args: ['run', 'lint']

Both test and lint start once install finishes, in parallel. The build's wall-clock time becomes max(test, lint) instead of test + lint.

Substitutions and Build Variables

Substitutions are the variables Cloud Build resolves before running a step. They make the same YAML reusable across branches, environments, and triggers.

Default substitutions

Cloud Build injects a fixed set: $PROJECT_ID, $BUILD_ID, $PROJECT_NUMBER, $LOCATION, $TRIGGER_NAME, $COMMIT_SHA (full), $SHORT_SHA (first 7 chars), $REVISION_ID, $REPO_NAME, $BRANCH_NAME, $TAG_NAME. These cover most of what you need to tag an image or write a deployment label.

User-defined substitutions

User-defined substitutions start with an underscore: _REGION, _ENV, _ARTIFACT_REPO. Define them in the trigger configuration or pass them via gcloud builds submit --substitutions=_REGION=us-east1. The convention of the leading underscore is enforced; without it the build will fail to validate.

substitutions:
  _REGION: us-central1
  _ENV: dev
steps:
  - name: 'gcr.io/cloud-builders/gcloud'
    args: ['run', 'deploy', 'api-${_ENV}', '--region=${_REGION}', '--image=us-central1-docker.pkg.dev/$PROJECT_ID/app/api:$SHORT_SHA']

Defaults inside the YAML can be overridden by the trigger or the CLI invocation. The options.substitution_option: ALLOW_LOOSE flag relaxes the rule that all referenced substitutions must exist; useful but a footgun, because a typo silently becomes an empty string.

Substitution scope and quoting

Substitutions are pure string replacement, so they need careful quoting in YAML. A substitution like _TAGS: 'a,b,c' survives a single substitution but breaks if you try to pass it to a step expecting a YAML list. For complex values, prefer Secret Manager (for secrets) or a config file in the repo (for structured data).

Secret Manager Integration

Hardcoding a secret in cloudbuild.yaml is the cardinal sin of CI/CD. Cloud Build's first-class answer is the availableSecrets block, which pulls secrets from Secret Manager at runtime.

The availableSecrets pattern

availableSecrets:
  secretManager:
    - versionName: projects/$PROJECT_NUMBER/secrets/npm-token/versions/latest
      env: 'NPM_TOKEN'
    - versionName: projects/$PROJECT_NUMBER/secrets/docker-hub-password/versions/3
      env: 'DOCKER_PASS'
steps:
  - name: 'gcr.io/cloud-builders/npm'
    entrypoint: 'bash'
    args: ['-c', 'echo "//registry.npmjs.org/:_authToken=$$NPM_TOKEN" > ~/.npmrc && npm ci']
    secretEnv: ['NPM_TOKEN']

The availableSecrets block declares which Secret Manager versions Cloud Build can fetch. Each step opts into a specific secret via secretEnv. The secret value is injected as an environment variable inside that step only, and the value is masked from build logs.

The $$NPM_TOKEN syntax (double dollar sign) tells Cloud Build "do not substitute this; the shell will resolve it at step runtime." A single dollar sign would attempt YAML-time substitution, which would expose the secret in the substituted YAML.

Permissions for Secret Manager access

The Cloud Build service account (<project-number>@cloudbuild.gserviceaccount.com by default, or your custom service account) needs secretmanager.secretAccessor on each secret it reads. Pin to specific versions in production rather than latest so a secret rotation does not silently change build behavior.

Alternatives and anti-patterns

The older _SECRET_KEY substitution-based pattern (encrypting with KMS, pasting ciphertext into YAML) still works but is harder to rotate and audit. New work should always use availableSecrets. Never put a real secret in a substitution variable; trigger config is readable by anyone with cloudbuild.builds.editor.

Private Worker Pools

The default Cloud Build pool runs in a Google-managed VPC. Private pools (the renamed "Cloud Build private pools" service, formerly "private worker pools") give you a dedicated pool inside your project's VPC.

When to use a private pool

Pick a private pool when at least one of these is true:

• The build needs to reach a private endpoint (Cloud SQL with private IP only, internal load balancer, on-prem service over Cloud VPN or Interconnect, self-hosted GitLab).
• You need a static egress IP for a third-party allowlist.
• You want larger machines (E2_STANDARD_32, N1_HIGHCPU_32) without queuing on the shared pool.
• You need predictable concurrency and dedicated capacity for an enterprise tier of builds.
• You require VPC Service Controls coverage for builds.

Creating a private pool

gcloud builds worker-pools create my-pool \
  --project=my-project \
  --region=us-central1 \
  --worker-machine-type=e2-standard-8 \
  --worker-disk-size=200GB \
  --no-public-egress \
  --peered-network=projects/my-project/global/networks/build-vpc

The pool peers with a VPC you specify. --no-public-egress blocks workers from reaching the public internet (you must then provide a NAT or Cloud Build-aware Private Google Access for any external dependencies). The pool is regional; pick the region closest to the resources the build talks to.

Routing a build to the pool

Reference the pool in the build YAML's options:

options:
  pool:
    name: 'projects/my-project/locations/us-central1/workerPools/my-pool'

Or set it on the trigger so every build of that trigger uses the pool.

Cost trade-offs

Private pools bill per build-minute at a higher rate than the shared pool, and you can also reserve capacity. For low-volume teams, the math favors the shared pool plus a tighter security model on Secret Manager. For high-volume or compliance-driven shops, private pools pay for themselves in VPC reach and predictable performance.

Build Triggers: GitHub, GitLab, Bitbucket, Manual, Webhook

A trigger is what turns a code event into a build. Cloud Build supports several trigger sources, each with its own setup and quirks.

GitHub triggers (Cloud Build GitHub App)

The recommended GitHub integration is the Cloud Build GitHub App, installed on the repo or org. It pushes events to Cloud Build via Pub/Sub and supports push triggers, pull request triggers, and tag triggers. The PR trigger is the killer feature: each PR builds independently, and the build status posts back to GitHub as a check. Branch filters use regex (^main$, ^feature/.*, ^(?!main$).* to exclude main).

GitLab and Bitbucket triggers

For GitLab.com and self-hosted GitLab, Cloud Build supports an integration via Secure Source Manager or the older webhook flow. Bitbucket Cloud and Bitbucket Server work via host connections (Generation 2) or webhook triggers (Generation 1). The trend is toward Generation 2 (host connections) which gives PR status, branch protection, and richer event filtering.

Manual triggers

A manual trigger has no source event; it only fires when someone clicks "Run" in the console or runs gcloud builds triggers run my-trigger --branch=main. Use manual triggers for promotion gates ("deploy this image to prod after QA signoff") or one-off ops tasks (rotate a service account key).

Webhook triggers

A webhook trigger exposes a URL that accepts a POST with a shared-secret header. Anything that can POST JSON can fire it: Jira automation, Linear webhooks, custom Slack slash commands, an internal admin tool. The trigger uses the request body as substitution variables. Useful for hybrid workflows where the source of truth is not Git.

Cloud Source Repositories

If your repo lives in Cloud Source Repositories (Google's hosted Git), triggers run natively without an app install. CSR is mostly used by teams already deep in GCP that want IAM-managed Git access instead of a SaaS Git provider.

Build Provenance and SLSA

Supply chain security is now table stakes. Cloud Build generates SLSA (Supply-chain Levels for Software Artifacts) provenance for every build, attesting what source produced what artifact.

What provenance contains

A Cloud Build SLSA provenance attestation includes the source URI (Git repo + commit SHA), the builder identity (https://cloudbuild.googleapis.com/...), the build trigger, the build configuration hash, the input materials (other artifacts pulled in), and the output artifact digest. The attestation is signed by Google and stored alongside the image in Artifact Registry.

Reading provenance

gcloud artifacts docker images describe \
  us-central1-docker.pkg.dev/my-project/app/api@sha256:abc... \
  --show-provenance

The output is JSON conforming to the SLSA v1.0 in-toto schema. Tools like Sigstore's cosign verify-attestation and Google's binary-authorization policy engine consume this to gate deployments.

SLSA build levels

Cloud Build advertises SLSA Build Level 3 for builds running in private pools with default settings, which is the current ceiling on shared CI services. Level 3 means hermetic, parameterless, isolated builds with non-falsifiable provenance. Public pool builds typically reach Level 2.

Binary Authorization integration

Binary Authorization is the natural consumer. A policy that says "only deploy images with Cloud Build provenance attesting the source repo is github.com/myorg/api and the trigger was prod-trigger" gives you a verifiable supply chain. The policy is enforced at GKE, Cloud Run, and Cloud Functions deployment.

Artifact Uploads to GCS and Artifact Registry

Cloud Build's last act on a successful build is uploading artifacts.

The images field

The simplest artifact upload is the images top-level field. Any image listed gets pushed to its registry (Artifact Registry or the deprecated Container Registry) and recorded in the build's metadata. The image must already exist locally (from a previous docker build step), and the registry path must be in the same project or one the build service account can push to.

The artifacts field for non-image outputs

For non-image artifacts (JARs, ZIPs, source maps, SBOMs), use the artifacts block:

artifacts:
  objects:
    location: 'gs://my-build-artifacts/$BUILD_ID/'
    paths: ['target/*.jar', 'sbom.json']

Files matching the glob get uploaded to Cloud Storage after all steps succeed. Cloud Build records each object's MD5 and timestamp in the build metadata.

Artifact Registry advantages

Artifact Registry replaced Container Registry in 2023 and is now the default. It supports Docker, Maven, npm, Python, apt, yum, and generic formats. Each repo has its own IAM, VPC Service Controls coverage, and regional or multi-regional placement. For Docker, the URI shape is <region>-docker.pkg.dev/<project>/<repo>/<image>:<tag>. Container Registry's gcr.io/<project>/<image> URIs are now backed by Artifact Registry under the hood, but new repos should use the explicit pkg.dev paths.

Build Approvers and Manual Approval Gates

Some builds should not fire automatically, even when the trigger source event matches. Cloud Build's approval feature pauses the build at start and waits for a human to click "Approve."

Configuring approval on a trigger

Enable require approval on the trigger. When the trigger fires, the build enters PENDING state and a notification goes to the approvers (configured via Pub/Sub or email channel). An approver with cloudbuild.builds.approver IAM role can approve or reject. Approval records the approver's identity in build metadata.

Approver IAM role

The role roles/cloudbuild.builds.approver is separate from the editor role. A senior engineer can approve prod deployments without being able to edit the YAML. This separation is exam-relevant: PCD scenarios often ask which role to grant for a "deploy gatekeeper" persona.

Approval workflows in practice

Common pattern: PR triggers run automatically with no approval (fast feedback). The main branch CI trigger runs automatically and builds + pushes an image. A separate "deploy to prod" trigger is approval-gated and only fires on tag push or manual invocation, taking the already-built image and rolling out to Cloud Run. This splits "build" (cheap, automated) from "deploy" (gated, slow) and lets approval focus only on the deploy step.

Rejected builds

A rejected build records the rejecter and the rejection note in metadata, and never runs the steps. Useful audit trail: "we rejected the v1.4.7 deploy on Friday at 5pm because change freeze."

Multi-Stage Docker Builds and Build Cache

Most Cloud Build pipelines produce a Docker image. The image build itself is where most of the wall-clock time and most of the cost go.

Multi-stage Dockerfiles

A multi-stage Dockerfile has multiple FROM lines. Each stage is a build environment; the final stage is what ships. A typical Go service:

FROM golang:1.22 AS build
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -o /out/api ./cmd/api

FROM gcr.io/distroless/static-debian12
COPY --from=build /out/api /api
ENTRYPOINT ["/api"]

The build stage has the full Go toolchain (large, mutable). The final stage is distroless/static-debian12 (5MB, no shell, no package manager). The runtime image is tiny and the attack surface is minimal.

Docker layer caching

Cloud Build can use Docker's layer cache to skip layers whose inputs have not changed. Two main approaches:

The Kaniko cache: build with gcr.io/kaniko-project/executor and pass --cache=true --cache-ttl=24h. Layer cache lives in Artifact Registry. Slower first build, much faster subsequent builds.

Docker buildx with cache-to/cache-from: use docker buildx build --cache-to=type=registry,ref=<repo>:cache --cache-from=type=registry,ref=<repo>:cache. Works the same way at the buildx level.

Cloud Build's source archive cache

For non-Docker dependency caching (node_modules, .m2, .gradle), use a Cloud Storage bucket as a manual cache:

steps:
  - name: 'gcr.io/cloud-builders/gsutil'
    args: ['-m', 'rsync', '-r', 'gs://my-cache/node_modules', 'node_modules']
  - name: 'node:20'
    entrypoint: 'npm'
    args: ['ci']
  - name: 'gcr.io/cloud-builders/gsutil'
    args: ['-m', 'rsync', '-r', 'node_modules', 'gs://my-cache/node_modules']

Cheap, simple, but you own cache invalidation. For Node, the lockfile-based npm ci is usually fast enough without bucket-level caching for small projects.

Concurrency and machine type

Cloud Build's default machine is E2_MEDIUM. For Docker builds that compile Rust or large Java apps, the build can be ten times faster on E2_HIGHCPU_8 or E2_HIGHCPU_32. Bump via options.machineType and benchmark; you usually pay roughly the same total because the higher-rate machine finishes proportionally faster.

Build IAM and Service Accounts

The most failure-prone part of any Cloud Build setup is IAM. The PCD exam pulls this thread often.

The default Cloud Build service account

Each project gets a default Cloud Build service account: <PROJECT_NUMBER>@cloudbuild.gserviceaccount.com. Builds run as this account by default. It has cloudbuild.builds.builder by default, which is wide-ish (it can read secrets and push to Artifact Registry only after you grant those individual roles).

Per-trigger custom service accounts

Best practice for production: assign a custom service account per trigger via the serviceAccount field. The prod-deploy trigger uses [email protected] with narrow roles (deploy to Cloud Run, read prod secrets). The CI trigger uses [email protected] with build-only roles (push to Artifact Registry, read non-prod secrets). Separation of duties without separating projects.

Key IAM roles to know

• roles/cloudbuild.builds.editor: create, update, delete triggers and builds. Wide; treat as developer-level write.
• roles/cloudbuild.builds.viewer: read-only access to builds and triggers. Safe for auditors.
• roles/cloudbuild.builds.approver: approve or reject approval-gated builds. Distinct from editor.
• roles/cloudbuild.builds.builder: the role granted to the default service account, lets a service account run builds.
• roles/iam.serviceAccountUser on the build service account: needed for a user or trigger to "act as" that account during a build.

The trap: granting cloudbuild.builds.editor lets the user change which service account a trigger runs as, which is effectively privilege escalation. Treat editor as a privileged role.

Common Pitfalls and Production Lessons

A few patterns that bite teams in production, exam-relevant because the PCD scenarios mirror them.

The shared service account everything-can-do-anything trap

Convenient on day one, painful when the first incident happens. A single Cloud Build service account with Editor on the project will happily deploy anywhere, read every secret, and rotate every key. Split early.

The substituted secret leak

_DB_PASSWORD: 'hunter2' in a trigger config is readable by anyone with editor on the trigger. Substitutions are not secrets; they are visible in build logs unless masked. Always route real secrets through Secret Manager + availableSecrets.

Triggers without branch filters

A trigger with no branch filter runs on every push to every branch. A Dependabot PR that bumps a transitive dep just kicked off a prod-shaped build. Always set a filter.

Long builds timing out at 10 minutes

The default timeout is 600 seconds. A first build that pulls a fresh Maven cache can run 15 minutes. Set timeout: 3600s for safety, then tune downward as you optimize.

Manual approval bypass via direct API

If a user has cloudbuild.builds.editor, they can submit a build directly via gcloud builds submit and skip the approval-gated trigger. Approval gates protect the trigger path, not the underlying build API. Lock down editor and use Binary Authorization to gate the deployment side.

Frequently Asked Questions (FAQ)

Can I run Cloud Build locally for debugging?

Yes. The cloud-build-local tool (legacy but still functional) simulates the Cloud Build environment on your machine using Docker. It is useful for iterating on cloudbuild.yaml without burning cloud build minutes, but the local emulation does not perfectly match Cloud Build's networking or service account context. Many teams now skip local and rely on a fast PR trigger instead.

How do I deploy to multiple regions from a single build?

Add multiple deploy steps, one per region, in parallel using waitFor. Each step calls gcloud run deploy --region=... with a different region. The image is pushed once to a multi-region Artifact Registry repository so all regions pull from the same source. If regions span continents, the image pull latency dominates; a regional repo per region with images replicated by a separate job can be faster.

What is the difference between Cloud Build and Cloud Deploy?

Cloud Build is the build engine: source-to-artifact. Cloud Deploy is the progressive delivery engine: artifact-to-environment, with promotion pipelines, approval gates, and rollback. The PCD exam expects you to know they are complementary: Cloud Build produces the image, Cloud Deploy rolls it through dev > staging > prod with verification steps.

How do I cache Docker layers across builds?

Use Kaniko (gcr.io/kaniko-project/executor) with --cache=true, or Docker buildx with --cache-to=type=registry,ref=<repo>:cache. Both store layer cache in Artifact Registry. The first build is normal-speed; subsequent builds that change only the top layers are dramatically faster. Avoid relying on the build VM's local Docker cache; build VMs are ephemeral and the cache is gone after each build.

Can Cloud Build pull from a private GitLab on my corporate network?

Yes, with a private pool peered to a VPC that has a Cloud VPN or Interconnect to the corporate network. The pool's workers reach GitLab over the private connection. A webhook trigger then handles the event side, or you set up the Generation 2 GitLab integration if your GitLab is on GitLab.com.

What is the cheapest way to gate a deployment with manual approval?

Use a manual or approval-gated trigger for the deployment step only, keep CI fully automated. The build that produces the image runs free of approval. The deploy trigger requires cloudbuild.builds.approver on a small group. Combined with Binary Authorization on the runtime side, you get a verifiable, low-friction gate.

How is SLSA Build Level 3 different from Level 2?

Level 2 requires version-controlled source, scripted builds, and authenticated provenance. Level 3 adds non-falsifiable provenance and isolated builds where the user cannot inject build-time arbitrariness (no SSH into a builder, no arbitrary environment manipulation). Cloud Build private pools meet Level 3; shared public pool builds typically meet Level 2. The exam may ask which Cloud Build configuration achieves Level 3.

Related Topics

• Artifact Registry covers the destination for most Cloud Build outputs, including IAM, vulnerability scanning, and Docker/Maven/npm format support.
• Deployment Manager and Terraform pairs with Cloud Build for IaC-driven environment provisioning where the build executes terraform apply.
• Secret Manager and KMS is the source of build-time secrets surfaced via availableSecrets.

Further Reading

• Google Cloud, Cloud Build configuration file schema: https://cloud.google.com/build/docs/build-config-file-schema
• Google Cloud, Private pools overview: https://cloud.google.com/build/docs/private-pools/private-pools-overview
• Google Cloud, Using secrets from Secret Manager: https://cloud.google.com/build/docs/securing-builds/use-secrets
• Google Cloud, Creating and managing build triggers: https://cloud.google.com/build/docs/automating-builds/create-manage-triggers
• Google Cloud, View build provenance and SLSA: https://cloud.google.com/build/docs/securing-builds/view-build-provenance
• Google Cloud, Cloud Build service account: https://cloud.google.com/build/docs/cloud-build-service-account