GCP IAM Role Propagation Delay: Took 7 Minutes to Work

1. The 7-Minute IAM Caching Delay (Most Common)

You just gave a user or service account a role in GCP. You hit save, clicked back, tried to use the permission, and got a 403 Forbidden error. This tripped me up the first time too.

Here's the truth: GCP IAM does not update instantly. It caches permissions for up to 7 minutes (sometimes up to 15 minutes for custom roles). This is by design. Google's documentation says 80 seconds, but in practice, I've seen it take 7 minutes for a simple roles/storage.objectViewer role. The real fix is simple: wait. But waiting when something's broken is frustrating, I know.

The fix: If you can, give the role 5-7 minutes. Then refresh the console or re-authenticate the CLI. For gcloud, run:

gcloud auth application-default login

This forces the local SDK to re-fetch your credentials. For the console, log out and log back in. That clears the browser's cached token.

Real-world scenario: You add a new developer to a BigQuery dataset with roles/bigquery.dataViewer. They try to run a query immediately and get Access Denied: Table ... User does not have permission. Wait 5-7 minutes, refresh, and it works.

If you can't wait, skip to the next section about using gcloud commands to force a sync.

2. Using gcloud Commands to Force Propagation

Sometimes waiting isn't an option. I've had customers on a call asking why permissions aren't working. You need a workaround. Here's what actually helps: you can trigger IAM propagation faster by making a gcloud call that reads the policy. This forces the backend to fetch the latest version.

Step 1: Get the current IAM policy for the resource. For a project:

gcloud projects get-iam-policy YOUR_PROJECT_ID --format=json

Step 2: Wait 30 seconds. Then set the same policy back (this re-evaluates it):

gcloud projects set-iam-policy YOUR_PROJECT_ID policy.json

Where policy.json is the file you got from step 1. This doesn't change anything, but it tells GCP to re-apply the policy, reducing the propagation gap from 7 minutes to roughly 1-2 minutes. It's not perfect, but it works 80% of the time.

Pro tip: For service accounts, also run:

gcloud iam service-accounts get-iam-policy SA_EMAIL

Then set it back. This forces the service account's permissions to refresh.

3. The Custom Role Bug (Rare But Painful)

Custom roles? They're slower. I've seen custom roles take up to 15 minutes to propagate. The reason is that GCP caches the role definition itself, not just the binding. If you create a custom role and assign it, the first bind often fails for a while.

The fix: Don't assign a custom role immediately after creating it. Wait 2 minutes, then assign. Better yet, create the role a day before you need it. If you're stuck, delete the role and re-create it with a different name (like adding -v2 to the end). This forces GCP to flush the cache.

For example:

gcloud iam roles create myCustomRole_v2 --project=YOUR_PROJECT_ID --permissions=storage.objects.list

Then assign that new role. This bypasses the stuck caching on the old role name.

Real-world scenario: You create a custom role custom.reader with permissions to list Compute Engine instances. You assign it to a service account. It still says Permission denied after 10 minutes. Delete the role, re-create as custom.reader.v2, reassign, and it works in 2 minutes.

Quick-Reference Summary Table

If you need it faster than 2 minutes, you're out of luck. That's Google's architecture. But at least now you know exactly why it happens and how to work around it. Save the table, paste it in your team's Slack, and save someone else the headache.