Auto Scaling Group Launch Failure – Fix When EC2 Instances Won't Start

When This Error Actually Happens

You see it in the AWS console or CloudWatch: your Auto Scaling group (ASG) launches a new EC2 instance, it shows as 'InService' for a few seconds, then suddenly goes 'Unhealthy' and gets terminated. This cycle repeats over and over. I had a client last month — their web app was down for 20 minutes because the ASG kept spinning up instances that couldn't pass the ELB health check. The trigger? They updated the AMI but forgot to include the web server binary.

Another common trigger: someone changes a security group rule, or the load balancer's health check path changes. The ASG itself works fine, but the new instances never become healthy, so they get killed.

Root Cause — Plain English

The ASG's job is to keep a certain number of healthy instances running. When it launches a new instance, it waits for the ELB health check to pass (if you have an ELB attached). If that check fails after a timeout (default 300 seconds), the ASG tags the instance as unhealthy and terminates it. So the cycle keeps going: launch, fail, terminate, launch again.

There are three main reasons this happens:

1. Bad AMI — The machine image you're using doesn't have the right software, or the software isn't starting up properly. For example, Apache or Nginx might be missing, or the app server doesn't start on boot.
2. Security group blocks traffic — The instance's security group doesn't allow inbound traffic from the ELB on the health check port (usually port 80 or 443).
3. Health check config mismatch — The ELB health check path (like "/health") doesn't exist on the instance, or the response code isn't 200.

The Fix — Step by Step

Here's what I do every time. Skip the generic AWS docs — this is the real fix.

Step 1: Check the ASG Activity History

Go to the Auto Scaling Groups console, select your group, and look at the "Activity" tab. You'll see a reason like "Instance failed ELB health checks". Click on it. AWS gives you a description like "Instance i-abc123 failed health check on port 80". This tells you which port is failing.

Step 2: Check the Security Group

Make sure the security group attached to the ASG launch configuration allows inbound traffic from the ELB's security group on the health check port. If you're using an ALB, the ELB's security group is usually something like "default" or "elb-sg".

Inbound rule example:
Type: HTTP (80)
Source: The ELB's security group ID (like sg-12345678)

I've seen this 5 times this year alone — someone adds a new rule but forgets to allow the health check port. Easy fix.

Step 3: Verify the AMI

Launch a single test instance from the same AMI manually (not through the ASG). Connect to it via SSH. Check if your web server is running:

sudo systemctl status nginx   # or httpd, apache2, etc.
curl http://localhost/health

If the service isn't running, you need to enable it to start on boot:

sudo systemctl enable nginx
sudo systemctl start nginx

Then create a new AMI from that instance. That's the one you use in your ASG.

Step 4: Check the Health Check URL

If your ELB health check is set to "/health" but your app only responds to "/", change one of them. In the ELB target group settings, you can set the health check path to "/" if that's simpler. Or add a simple health endpoint to your app.

For a simple Nginx health check, create a file at /usr/share/nginx/html/health with content "OK"

Step 5: Look at the Launch Template or Launch Configuration

Go to the launch template or launch config used by your ASG. Check if there's a user data script. If there is, run it manually on a test instance to see if it errors out. User data scripts often fail silently, leaving the instance without the required software.

Example user data that might fail:
#!/bin/bash
yum install -y httpd   # if yum isn't available or repo is broken
systemctl start httpd

If the script fails, the instance boots but the web server never starts. The ASG then kills it.

What to Check If It Still Fails

If you've done all the above and new instances still fail health checks, here are the hidden gotchas:

• ELB security group itself — The ELB's security group might have outbound rules that block traffic to the instance's port. Make sure outbound is allowed to the instance's security group.
• VPC route tables — If your instances are in a private subnet without a NAT gateway or internet gateway, the health check might time out because the ELB can't reach them. Verify the subnet route table has a route to the NAT gateway (for private subnets) or to the internet gateway (for public subnets).
• Network ACLs — Network ACLs (NACLs) are stateless. They can block health checks if the inbound or outbound rules don't allow the ephemeral ports. Check both directions.
• Instance state — Sometimes the instance launches but stops responding due to memory pressure or CPU. Check the CloudWatch metrics for the instance. If it's maxed out, you need a larger instance size.

One last thing: if you're using a custom AMI with a pre-installed agent (like the CloudWatch agent or SSM agent), make sure that agent doesn't interfere with the health check. I had a case where an old SSM agent version was causing a delay in starting the web server. Updating the AMI to the latest agent fixed it.

That's it. No fluff. Check the AMI, the security group, and the health check path. Nine times out of ten, one of those is the problem.