I have Nginx set up as a load balancer with cookie-based stickiness. The logic is:

1. If the cookie is NOT there, use round-robbing to choose a server from cluster.

2. If the cookie is there, go to the server that is associated with the cookie value. Server is then responsible for setting the cookie.

   What I want to add is this:

3. If the cookie is there, but server is down, fallback to round-robbing step to choose next available server.

So actually I have load balancing and want to add failover support on top of it.

I have managed to do that with the help of error_page directive, but it doesn't work as I expected it to.

The problem: 504 (and the fallback associated with it) fires only after 30s timeout even if the server is not physically available.

So what I want Nginx to do is fire a 504 (or any other error, doesn't matter) immediately (I suppose this means: when TCP connection fails). This is the behavior we can see in browsers: if we go directly to server when it is down, browser immediately tells us that it can't connect. Moreover, Nginx seems to be doing this for 502 error: if I intentionally misconfigure my servers, Nginx fires 502 immediately.

Configuration (stripped down to basics):

http {
    upstream my_cluster {
        server 192.168.73.210:1337;
        server 192.168.73.210:1338;
    }
    map $cookie_myCookie $http_sticky_backend {
        default 0;
        value1   192.168.73.210:1337;
        value2   192.168.73.210:1338;
    }
    server {
        listen 8080;

        location @fallback {
            proxy_pass http://my_cluster;
        }

        location / {
            error_page 504 = @fallback;

            # Create a map of choices
            # see https://gist.github.com/jrom/1760790
            set $test HTTP;
            if ($http_sticky_backend) {
                set $test "${test}-STICKY";
            }

            if ($test = HTTP-STICKY) {
                proxy_pass http://$http_sticky_backend$uri?$args;
                break;
            }
            if ($test = HTTP) {
                proxy_pass http://my_cluster;
                break;
            }

            return 500 "Misconfiguration";
        }
    }
}

Disclaimer: I am pretty far from systems administration of any kind, so there may be some basics that I miss here.

EDIT: I'm interested in solution with standard free version of Nginx, not Nginx Plus. Thanks.