Nginx Load Balancing Setup to Distribute Traffic

June 2026 · 7 min read

Load Balancing is a technique that distributes incoming traffic from users across multiple servers simultaneously, allowing your system to handle more concurrent users, reduce downtime, and prevent any single server from becoming overloaded. In this article, we'll show you how to configure Nginx to function as a Load Balancer and Reverse Proxy, distributing requests to multiple backend servers using various algorithms.

Architecture Used in This Guide

In this example, we'll work with a total of 3 servers:

• Load Balancer — IP: 10.0.0.1 (Nginx is installed here, receives requests from the internet)
• Backend 1 — IP: 10.0.0.11 (runs the web application)
• Backend 2 — IP: 10.0.0.12 (runs the web application)

Install Nginx on the Load Balancer

Connect to your load balancer and install Nginx using the following commands:

# Update package list
sudo apt update && sudo apt upgrade -y

# Install Nginx
sudo apt install nginx -y

# Check version
nginx -v

# Enable Nginx to start automatically
sudo systemctl enable nginx
sudo systemctl start nginx

Understanding the 3 Load Balancing Algorithms

1. Round Robin (Default)

Nginx sends requests to each backend server in a rotating fashion: server 1 → server 2 → server 1 → server 2. This is best suited for backends with similar specifications and where each request takes approximately the same time to process.

2. Least Connections

Nginx sends each new request to the backend server with the fewest active connections at that moment. This is ideal for workloads where requests have varying processing times, such as file uploads or APIs with inconsistent response times.

3. IP Hash

Nginx calculates a hash from the client's IP address and always routes that client to the same backend server. This enables sticky sessions, where a user always connects to the same server, suitable for applications that store session data locally on the server.

Configure Round Robin Load Balancing

Create a new configuration file for the load balancer:

sudo nano /etc/nginx/conf.d/loadbalancer.conf

Insert the following configuration:

upstream backend_pool {
    server 10.0.0.11:80;
    server 10.0.0.12:80;
}

server {
    listen 80;
    server_name yourdomain.com;

    location / {
        proxy_pass http://backend_pool;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

Then verify the configuration and reload Nginx:

sudo nginx -t
sudo systemctl reload nginx

Configure Least Connections

Simply add least_conn; directive inside the upstream block:

upstream backend_pool {
    least_conn;
    server 10.0.0.11:80;
    server 10.0.0.12:80;
}

Configure IP Hash

Add ip_hash; instead:

upstream backend_pool {
    ip_hash;
    server 10.0.0.11:80;
    server 10.0.0.12:80;
}

Assign Weights to Backend Servers

If your backend servers have different specifications, you can assign weight values so that more powerful servers handle more traffic:

upstream backend_pool {
    server 10.0.0.11:80 weight=3;   # Receives 3 parts
    server 10.0.0.12:80 weight=1;   # Receives 1 part
}

In this example, Backend 1 will receive 3 out of 4 requests, while Backend 2 will receive 1 out of 4 requests.

Configure Health Check and Failover

Nginx can mark failed backend servers as "offline" temporarily using the max_fails and fail_timeout directives:

upstream backend_pool {
    least_conn;
    server 10.0.0.11:80 max_fails=3 fail_timeout=30s;
    server 10.0.0.12:80 max_fails=3 fail_timeout=30s;
}

• max_fails=3 — If connection fails 3 consecutive times, that backend is considered down
• fail_timeout=30s — The time period Nginx waits before attempting to send requests to that backend again (30 seconds)

Additionally, you can add a backup server to use when all primary backends fail:

upstream backend_pool {
    server 10.0.0.11:80;
    server 10.0.0.12:80;
    server 10.0.0.99:80 backup;   # Used when both primary backends fail
}

Add Timeout and Buffer Settings

Adjust the proxy timeout values to match your application's requirements in the location / block:

location / {
    proxy_pass http://backend_pool;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;

    proxy_connect_timeout 10s;
    proxy_send_timeout    60s;
    proxy_read_timeout    60s;

    proxy_buffering       on;
    proxy_buffer_size     16k;
    proxy_buffers         4 16k;
}

Enable HTTPS with Load Balancer

Install Certbot to obtain an SSL certificate from Let's Encrypt:

sudo apt install certbot python3-certbot-nginx -y
sudo certbot --nginx -d yourdomain.com

Certbot will automatically modify your configuration, add HTTP to HTTPS redirection, and set up the SSL certificate.

Test Load Balancing

Make each backend return different values to verify that traffic is actually being distributed:

# On Backend 1 (10.0.0.11)
echo "Response from Backend 1" | sudo tee /var/www/html/index.html

# On Backend 2 (10.0.0.12)
echo "Response from Backend 2" | sudo tee /var/www/html/index.html

Then test from a client machine using curl:

for i in {1..6}; do
    curl -s http://yourdomain.com/
done

Expected output (Round Robin):

Response from Backend 1
Response from Backend 2
Response from Backend 1
Response from Backend 2
Response from Backend 1
Response from Backend 2

Check Logs and Monitor

View the access log to verify which backend each request was sent to:

sudo tail -f /var/log/nginx/access.log

Add a custom log format to show which upstream server was selected:

# Add to http {} block in /etc/nginx/nginx.conf
log_format upstream_log '$remote_addr - $upstream_addr [$time_local] '
                        '"$request" $status $body_bytes_sent '
                        '"$http_referer" "$http_user_agent"';

Then specify the format in the access_log directive:

access_log /var/log/nginx/upstream.log upstream_log;

Summary

Configuring Nginx load balancing enables your system to handle more users and tolerate failures of individual backend servers. Here's a checklist of steps before going live:

1. Install Nginx on the load balancer and create configuration file in /etc/nginx/conf.d/
2. Choose an algorithm that suits your workload — Round Robin for uniform requests, Least Connections for varying response times, IP Hash for sticky sessions
3. Configure max_fails and fail_timeout for automatic failover
4. Enable HTTPS using Certbot with SSL Termination at the load balancer
5. Test with curl and verify logs show traffic distributed to all backends
6. Move session storage to Redis or a central database to reduce reliance on IP Hash

As your system grows, you can add new backend servers to the upstream block and reload Nginx without service interruption. A single reload is all that's needed to start serving traffic to the new backend.

Verify Load Balancing is Working Correctly

# Check Nginx config syntax
nginx -t

# Test that traffic is distributed to both backends
for i in {1..6}; do curl -s http://yourdomain.com/; done
# Should alternate between Backend 1 and Backend 2

# Watch upstream log in real-time
tail -f /var/log/nginx/upstream.log

# Check active backends
curl http://localhost/nginx_status   # if stub_status is enabled

Common Issues & Troubleshooting

Error: "502 Bad Gateway" from load balancer

The backend server is not responding or Nginx cannot connect to it. Check:

# Verify the backend service is running
curl -s http://10.0.0.11/   # Test directly on the backend

# View Nginx error log
tail -50 /var/log/nginx/error.log

# Check firewall on backend
# Backend must allow port 80 from load balancer IP

Error: Traffic going only to one backend, not distributing

Usually caused by browser cache or Nginx proxy cache directing old requests to the same backend:

# Always test with curl instead of browser
for i in {1..4}; do curl -s -o /dev/null -w "%{http_code} %{time_total}\n" http://yourdomain.com/; done

# Check if proxy_cache is enabled
grep -r "proxy_cache" /etc/nginx/conf.d/

Error: Session lost when user is routed to different backend

The application stores sessions locally on each server, so different backends don't recognize sessions:

# Short-term solution: use IP Hash (sticky session)
upstream backend_pool {
    ip_hash;
    server 10.0.0.11:80;
    server 10.0.0.12:80;
}

# Long-term solution: move sessions to central Redis
# WordPress: install WP Redis Object Cache plugin
# Laravel: set SESSION_DRIVER=redis in .env

Error: One backend failed but load balancer still sends traffic to it

Configure max_fails and fail_timeout correctly in upstream:

upstream backend_pool {
    server 10.0.0.11:80 max_fails=3 fail_timeout=30s;
    server 10.0.0.12:80 max_fails=3 fail_timeout=30s;
}
# After reloading Nginx, it will mark backend as down after 3 failures
# and retry after 30 seconds

Error: "upstream timed out (110: Connection timed out)"

The backend is slower than the configured timeout. Increase the read timeout or debug why the backend is slow:

location / {
    proxy_pass http://backend_pool;
    proxy_read_timeout 120s;    # Increase from default 60s
    proxy_connect_timeout 10s;
}

Frequently Asked Questions

How many VPS servers do I need for load balancing?

Minimum: 2 servers (1 load balancer + 1 backend). However, for true high availability: 3 servers (1 load balancer + 2 backends). If one backend fails, the other continues serving. For production environments with no single point of failure, you should have 2 load balancers (in HA Proxy pair configuration).

What does SSL termination at the load balancer mean?

SSL termination means the load balancer decrypts SSL traffic from clients and sends plain HTTP to backends. This way, you only need to manage SSL certificates on the load balancer—backends don't need certificates. This reduces operational complexity and saves backend CPU resources from SSL handshakes.

Do I need to restart Nginx when adding a new backend server?

No, just reload. Add the new IP to the upstream block and run nginx -t && systemctl reload nginx — reload doesn't kill existing connections, unlike restart which would drop all active connections.