Elastic Load Balancing Fundamentals

A payments API runs on 6 EC2 instances split across 2 Availability Zones. A deploy goes out, one instance comes up with a broken configuration, and for the next 40 minutes about a sixth of all requests return errors. Nobody was paged, because nothing was down. The instance was reachable, it answered on port 8080, and it returned HTTP 500 for every call.

A load balancer would have pulled that instance out of rotation in under a minute. That is what Elastic Load Balancing does for you: not only spreading requests, but continuously asking every target whether it is fit to receive them, and routing around the ones that are not. This lesson builds the model of how that distribution works. The next one covers what to do when the health check itself is the thing that is broken.

A load balancer is a set of nodes, not a box

The first thing to unlearn is the picture of a single appliance sitting in front of your fleet. When you enable an Availability Zone for a load balancer, Elastic Load Balancing creates a load balancer node in that zone. Enable 3 zones and you get 3 nodes, each with its own IP address, each able to receive client traffic.

Clients find those nodes through DNS. Your load balancer gets a name such as my-lb-1234567890abcdef.elb.us-east-2.amazonaws.com, and Amazon DNS answers a query for it with the IP addresses of the nodes. The record carries a time to live of 60 seconds, which is short on purpose: it lets AWS add nodes as traffic grows, and remove a node whose zone is in trouble, without waiting on client caches.

dig +short my-lb-1234567890abcdef.elb.us-east-2.amazonaws.com
3.13.20.147
18.222.9.88

Two consequences follow immediately, and both show up on the exam.

First, a load balancer needs subnets in the zones you want it to serve. An Application Load Balancer requires at least 2 Availability Zones; a Network Load Balancer will run with one, though a single zone gives you no zonal redundancy. If you register targets in a zone you never enabled, those targets get no traffic at all.

Second, the client picks which node IP address to use. AWS does not steer the choice. So the shape of your traffic depends on how many nodes exist and which targets each node is allowed to reach, which is exactly the cross-zone question later in this lesson.

One more distinction before the types. An internet-facing load balancer has nodes with public IP addresses and needs public subnets with a route to an internet gateway. An internal load balancer has only private addresses. Both reach their targets over private IP addresses, so your instances never need public IPs to sit behind either one.

Three load balancers, three depths of inspection

AWS offers 3 current generation load balancers. The temptation is to memorize a feature matrix. The faster model is to ask how deep into the packet each one is allowed to look, because that single fact determines everything it can decide.

An Application Load Balancer operates at layer 7. It parses the HTTP request, so it can route on the hostname, the path, a header, the method, a query string parameter, or the source IP. It can also terminate TLS, authenticate users against an identity provider, return a fixed response, and issue a redirect without ever touching a target.

A Network Load Balancer operates at layer 4. It never parses HTTP. It picks a target using a flow hash over the protocol, source IP and port, destination IP and port, and TCP sequence number, then pins that whole TCP connection to the chosen target for its lifetime. Because it does less per packet, it handles very high connection rates with low added latency, and because it does not need to rewrite anything, it can hand your target the client's real source IP.

A Gateway Load Balancer operates at layer 3. It listens for all IP packets on all ports and forwards them to virtual appliances, firewalls, intrusion detection systems, deep packet inspection tools, using the GENEVE protocol on port 6081. Traffic reaches it through a Gateway Load Balancer endpoint that you set as the next hop in a route table, so the appliance fleet becomes a transparent bump in the wire.

The Classic Load Balancer still exists and still appears in older material, but AWS recommends migrating away from it, and SOA-C03 scenarios are built around the current generation.

How an Application Load Balancer decides where a request goes

Four objects sit between a client and an instance, and every ALB troubleshooting question depends on knowing which one owns which decision.

A listener checks for connections on a protocol and port, HTTP or HTTPS, port 1 to 65535. A listener rule has a priority, one or more conditions, and one or more actions. A target group collects targets that share a protocol, a port, and one set of health check settings. A target is the instance, IP address, or Lambda function that finally answers.

Walk one request through it. A client sends GET https://shop.example.com/api/orders/1042. The HTTPS listener on port 443 terminates TLS using the ACM certificate attached to it. It then evaluates its rules in priority order, lowest number first:

priority 10   IF path-pattern = /api/*        THEN forward to tg-api
priority 20   IF host-header  = img.example.com THEN forward to tg-images
default       (cannot be deleted)             forward to tg-web

Rule 10 matches, so the request goes to tg-api. Rules are first-match-wins, and the default rule is always evaluated last and cannot be deleted, so it functions as the catch-all. If your new path rule sits at priority 60000 behind a broad /* rule at priority 5, it will never fire, and that ordering mistake is a common exam stem.

tg-api then picks one target using its routing algorithm and forwards the request over a backend connection, HTTP/1.1 by default. The load balancer adds X-Forwarded-For, X-Forwarded-Proto, and X-Forwarded-Port so the application can still see who the client was, since at layer 7 the packet the target receives comes from the load balancer, not from the client.

Two settings on this path are worth committing to memory. The connection idle timeout defaults to 60 seconds and can be set anywhere from 1 to 4000. And the load balancer allows 10 seconds to establish a TCP connection to a target before it gives up. Both of those numbers become error codes in the next lesson.

Target types: what you register decides what you can reach

A target group has one target type, chosen at creation and never changeable afterwards.

• instance registers by instance ID. Traffic goes to the primary private IP of the primary network interface. Simple, and it is what an Auto Scaling group attaches to automatically. It cannot reach instances in a peered VPC.
• ip registers by IP address. Allowed ranges are the target group VPC subnets plus 10.0.0.0/8, 100.64.0.0/10, 172.16.0.0/12, and 192.168.0.0/16. Publicly routable addresses are rejected. This is how you reach a peered VPC, a database, or an on-premises server over Direct Connect or Site-to-Site VPN, and it also lets several containers on one instance use the same port.
• lambda registers a single function, which the load balancer invokes rather than connects to. Request and response bodies are capped at 1 MB.

The tempting mistake is to assume instance is always the safe default. It is the one target type that cannot cross a VPC peering boundary, so any hybrid or multi-VPC scenario points at ip.

How a Network Load Balancer decides, and why client IP matters

There are no rules to evaluate at layer 4. The node hashes the flow, picks a target, and keeps that connection there. Everything interesting is in the connection handling.

Client IP preservation is the setting that surprises people. When it is on, your target sees the client's real source address, which means your security groups must allow the client range, not the load balancer. When it is off, the target sees the private IP of the load balancer node. The defaults are not uniform:

If preservation is off and you still need the caller's address, turn on proxy protocol v2, which prepends a binary header carrying the connection details. Make sure the application can parse it first: a target that chokes on the header will start failing health checks, because the header is added to health check connections too.

The idle timeouts are fixed points worth knowing. TCP flows default to 350 seconds and can be set from 60 to 6000. TLS listeners are fixed at 350 seconds. UDP flows are fixed at 120 seconds, after which the next packet is treated as a new flow and can land on a different target.

Cross-zone load balancing, with the arithmetic

This is the single most testable behavior in the topic, and the numbers make it stick better than the definition does.

Take a load balancer with 2 enabled zones. Availability Zone A holds 2 targets, Availability Zone B holds 8. DNS hands out both node addresses about evenly, so each node receives 50% of client traffic.

Cross-zone off. Each node can only reach targets in its own zone. The node in zone A splits its 50% between 2 targets, so each gets 25%. The node in zone B splits its 50% across 8 targets, so each gets 6.25%. Two targets are doing 4 times the work of the other 8.

Cross-zone on. Each node can reach all 10 targets, so every target receives 10%.

The defaults differ by type, and the reason is architectural rather than arbitrary. An Application Load Balancer is always cross-zone at the load balancer level and cannot be turned off there, though an individual target group can set load_balancing.cross_zone.enabled to false and become zonal. Network Load Balancers and Gateway Load Balancers default to off, which keeps each zone a self-contained failure domain and avoids cross-zone data transfer charges.

That charge is the practical catch. Turning cross-zone on for a Network Load Balancer means traffic crosses Availability Zone boundaries, and EC2 data transfer charges apply. Turning it off means you must plan enough capacity in every zone, because a zone with too few targets will overload them while another zone sits idle.

There is one trap in the off configuration. When cross-zone is off, an empty or all-unhealthy zone is a real outage for the clients that resolved to it. For an Application Load Balancer this produces HTTP 503 responses for every request entering that zone, which is why AWS warns against creating empty subnets.

The target group attributes that change production behavior

Health check settings get their own lesson. These 4 attributes decide how traffic reaches a healthy target, and each one exists to solve a specific failure.

Deregistration delay (deregistration_delay.timeout_seconds, default 300 seconds, range 0 to 3600). When a target is deregistered it enters the draining state and stops receiving new requests while in-flight ones finish. If the delay expires while a request is still running, the client gets a 502. If the target has no in-flight requests and no active connections, deregistration completes immediately, though the console keeps showing draining until the timer runs out.

Routing algorithm (load_balancing.algorithm.type, default round_robin). Round robin assumes similar requests hitting similar targets. Least outstanding requests sends each request to the target with the fewest in-progress requests, which is the right answer when request cost or target capacity varies. Weighted random distributes randomly and is the only algorithm that supports Automatic Target Weights anomaly mitigation. Neither of the last 2 can be combined with slow start.

Slow start (slow_start.duration_seconds, default 0, meaning disabled, range 30 to 900). A newly healthy target normally receives its full share of traffic the instant it passes one health check. For an application that needs to warm a cache or a JIT compiler, that full share arrives as a wall of latency. Slow start ramps the target up linearly over the configured window instead.

Stickiness (stickiness.enabled, default false). Duration-based stickiness makes the load balancer issue an encrypted AWSALB cookie that pins a client to a target; application-based stickiness follows a cookie your application sets. Both are configured per target group, with a duration from 1 second to 7 days. Two constraints catch people out: stickiness requires cross-zone load balancing to be on, and it does not work with the weighted random algorithm.

It is tempting to treat stickiness as the fix for a session problem. It is a workaround, and it costs you even distribution, because a scale-out event leaves old clients pinned to old targets. The durable fix is to move session state out of the instance, into ElastiCache or DynamoDB, which is where a later topic in this domain picks the thread back up.

Exam tips

• "Route based on URL path or hostname" is always an Application Load Balancer. "Static IP address", "Elastic IP", "millions of requests per second", or a protocol that is not HTTP points at a Network Load Balancer. "Firewall appliance", "intrusion detection", or "third-party inspection" points at a Gateway Load Balancer.
• "The targets must terminate TLS themselves" rules out an HTTPS listener on an ALB. The answer is a Network Load Balancer with a TCP listener on port 443.
• Uneven load across instances with cross-zone in the stem is the 25% versus 6.25% arithmetic. Reach for the target count per zone before reading the options.
• If a question mentions cost as a reason to leave a Network Load Balancer setting alone, it is talking about cross-zone data transfer charges.
• Requests dropped during a deploy point at deregistration delay. New instances slow for their first minute point at slow start. The 2 are opposites and their names do not make that obvious.
• A target group registered by instance ID cannot reach a peered VPC. If the stem mentions peering, hybrid, or on-premises targets, the target type must be ip.
• "The application needs the client's real IP address" is X-Forwarded-For on an Application Load Balancer, and client IP preservation or proxy protocol v2 on a Network Load Balancer. Note the default is disabled for TCP and TLS ip target groups.

The idea worth carrying forward is that a load balancer routes only to targets it believes are healthy, and belief is built entirely from health check settings you chose. Every distribution behavior in this lesson assumes those checks are telling the truth. When they are not, targets go missing from rotation for reasons that have nothing to do with the application, and that is where the next lesson starts.