You're seeing 503 Service Unavailable from your Application Load Balancer, but only sometimes. Not a constant outage. The targets are passing health checks one minute, failing the next, then recovering. This usually happens right when traffic spikes or when you deploy a new version of your app. The ALB logs show 503 with target_status_code of - and unhealthy in the load balancer metrics.
I know this error is infuriating because it's intermittent—hard to pin down. But once you understand what's actually happening, the fix is straightforward. The root cause is almost always one of three things: the health check path is too strict, the response time is too slow, or the target itself doesn't handle concurrent health checks well.
What's Actually Happening
The ALB sends a health check request to each target at a set interval (default is 30 seconds). If the target doesn't return a 200 within the timeout (default is 5 seconds), it's marked unhealthy. After two consecutive failed checks, it's taken out of rotation. But if the next check passes, it's brought back. This flapping uses up your target's resources and causes the ALB to return 503 when it has no healthy targets to route to.
The trigger is often a moment of high CPU or memory usage on your EC2 instances—like when your app does a heavy query or a cron job kicks off. The health check request comes in, the app is busy, and the response takes longer than 5 seconds. Boom. Unhealthy. Then the app frees up, passes the next check, and the cycle repeats.
Step-by-Step Fix
1. Simplify Your Health Check Path
Start by looking at what path you're checking. If it's /health and that endpoint does a database query or calls an external API, you're asking for trouble. The health check should be a lightweight, in-memory responder. Create a dedicated endpoint like /ping that returns a simple 200 without any dependencies. I've seen this solve 80% of intermittent 503s.
# In your app (Node.js example)
app.get('/ping', (req, res) => res.status(200).send('ok'))2. Increase the Health Check Timeout and Interval
Go to your target group in the AWS Console. Under Health checks, set the timeout to 10 seconds (max is 30) and the interval to 30 or 60 seconds. The default timeout of 5 seconds is too aggressive for most apps, especially under load. Also set healthy threshold to 3 and unhealthy threshold to 3—so one slow response doesn't cause an immediate eviction.
I'm not a fan of the default 5-second timeout. It's fine for a static website, but for any dynamic app, bump it up. You're trading a slightly longer failover time for stability.
3. Check Your Security Group and Network ACLs
Make sure the security group on your targets allows the ALB's health check requests. The ALB uses the source of the target group's VPC CIDR. If you have a stateful firewall, it can drop health checks intermittently. Add a rule allowing inbound traffic on the health check port from the VPC CIDR. This tripped me up once—everything looked right, but a strict NACL was blocking half the health checks.
4. Look at Your App's Concurrency
If your app runs on a single-threaded runtime (like a small Node.js instance), a health check request can queue behind other requests. Use a process manager like PM2 or increase your instance size. Better yet, serve the health check from a different process—like nginx directly returning 200 for /ping without hitting your app at all. That way, even if your app is overloaded, the health check stays green.
# nginx config
location = /ping {
return 200;
}If It Still Fails
You've done the above and the 503s are still popping up. Now get into the logs. Enable ALB access logs and check the target_status_code for the 503 responses. If it's -, the ALB didn't send the request to any target—likely because all targets were unhealthy. If it's a 5xx from the target, then the health check is passing but the actual request is failing. That's a different beast—look at your app's error rate and response time during those windows.
Also, check the CloudWatch metrics for the target group: HealthyHostCount and UnhealthyHostCount. See if the unhealthy host count is fluctuating. If yes, it's the health check. If no, it's your app's response to real traffic.
One more thing—if you have multiple targets and they're all in different AZs, make sure your subnets are healthy. A flapping NAT gateway or a misconfigured route table can cause the ALB to lose connection to some targets intermittently. I've seen a case where one AZ's route table pointed to the wrong NAT, causing every other target to fail health checks.
The real fix is to make your health check as boring as possible. If it's a pure 200 response with no logic, your targets will stay healthy unless they're truly down. Then your 503s will vanish, and you can sleep through deployments.