Cloud NAT Gateway 64k Port Limit Exhaustion Fix

Yeah, this is frustrating. Let's fix it.

You're seeing connections drop, timeouts, or TCP RSTs. If you're on Azure, AWS, or GCP, and using a Cloud NAT gateway, you've probably hit the 64,000 source port limit per NAT IP. This happens when too many outbound connections use the same public IP. The gateway can't assign new source ports, so it drops new connections. Here's the fix.

The Real Fix: Add More NAT IPs

On all three major clouds, the easiest fix is adding more public IP addresses to your NAT gateway. Each extra IP gives you another 64k ports. So if you have 100k concurrent connections, you need 2 IPs (64k + 64k = 128k ports).

On Azure:

1. Go to your NAT gateway resource in the portal.
2. Under Outbound IP, add more public IP addresses or IP prefixes.
3. Save. That's it. No downtime.

On GCP:

1. Edit your Cloud NAT gateway.
2. Under Public IP addresses, add more IPs.
3. Check the Manual port allocation option if you want control per VM.
4. Save.

On AWS:

1. You can't add IPs to a single NAT gateway. You must create multiple NAT gateways, each in a different AZ. Then split your outbound traffic across them using route tables.
2. AWS also lets you use NAT gateways with Elastic IPs – but you still have the 64k limit per gateway.

Why Adding IPs Works

What's actually happening here is that each NAT IP has 64,000 TCP source ports available. When a VM makes an outbound connection, the NAT gateway picks a source port from that pool. Once all 64,000 are in use (waiting for replies), the gateway can't create new connections. Adding another IP gives you another pool of 64k ports. Simple math.

The reason step 3 works on GCP (manual port allocation) is it lets you assign specific IPs to specific VMs. This stops one busy VM from hogging all ports on a shared IP. On Azure, you can't do that directly – but you can use multiple NAT gateways per subnet.

Less Common Variations of the Same Issue

Not all exhaustion looks the same. Here are a few:

1. Idle timeout too long

If your NAT gateway keeps ports open for 4 minutes (Azure default) after a connection closes, you'll exhaust ports faster. Shorten the idle timeout to 2 minutes or 60 seconds if your app doesn't need long idle connections. On Azure, set idle-timeout to 4 minutes minimum – but you can go to 2 minutes with the CLI:

az network nat gateway update --name myNat --resource-group myRg --idle-timeout 2

2. Health probes or monitoring traffic

Load balancer health probes, monitoring agents, and cron jobs can eat ports silently. I've seen a single VM with 500 monitoring probes hit 30k ports. Check your outbound connection logs. On Azure, use NSG flow logs. On GCP, use VPC flow logs. Filter by src_port in the ephemeral range (1024–65535).

3. DNS queries going through NAT

If your VMs use external DNS servers (not the cloud's internal resolver), each DNS query uses a source port. High DNS traffic can exhaust ports fast. Move to the cloud's internal DNS – on Azure it's 168.63.129.16, on GCP it's the metadata server. That bypasses the NAT entirely.

Prevention

Stop it before it starts. Here's my opinionated checklist:

• Monitor port usage – On Azure, use NAT gateway metrics (SNATConnectionCount). Set an alert at 80% of the limit (51,200 ports). On GCP, enable Cloud NAT metrics. On AWS, use CloudWatch for PacketsOutToSource.
• Use multiple NAT IPs – Start with 2–4 IPs even if you think you don't need them. It's cheap and saves headaches.
• Shorten idle timeout – 2 minutes works for most web apps. Don't use the default 4 unless you have long-lived connections.
• Use private connectivity where possible – VPC peering, Private Service Connect, or VPNs keep traffic off the NAT entirely.
• Scale out with multiple NAT gateways – On AWS, always put a NAT gateway in each AZ and split traffic. On Azure, put a NAT gateway per subnet if you have high traffic.
• Check your app's connection patterns – Apps that open and close connections repeatedly (HTTP/1.0, some databases) burn ports fast. Use connection pooling where possible.

One last thing: if you're on Azure and using a Standard Load Balancer with outbound rules, a NAT gateway is cheaper and simpler. Don't mix them – use one or the other.

That's it. Add IPs, set alerts, and you'll stop seeing those drops.