Re: [PATCH] : Prevent redispatcher from selecting the same server

On Wed, Dec 26, 2007 at 02:45:17AM +0100, Krzysztof Oledzki wrote:
> [RFC]: Prevent redispatcher from selecting the same server
>
> When haproxy decides that session needs to be redispatched it chose a server,
> but there is no guarantee for it to be a different one. So, it often
> happens that selected server is exactly the same that it was previously, so
> a client ends up with a 503 error anyway, especially when one sever has
> much bigger weight than others.
>
> This RFC quality patch fix RR and PH algos, so when a redispatching
> occures assign_server() never selects previously assigned server again.
>
> TODO: SH/UH, test PH, more tests

IMHO, this should only be done for non-deterministic algorithms such as roundrobin, and later leastconn. But people who use hashes really expect that the same client or request will be sent to the same server (or to an equivalent backup server when that may happen).

And in fact, in the PH code, if you find the same server, you return NULL, which will immediately cause a 503. In this case, we'd probably prefer to still return the same server and perform a last connection attempt to it in the hope that it finally accepts the connection.

Basically, this would mean that non-deterministic algorithms should try to evict the faulty server from the algorithm before choosing one (which is what you do in case of RR), but others should just ignore it. If the server is already down, another one will be reported. And if it's still up, it is said and documented that no other server should get the request.

But generally, the idea seems good. I do also find it frustrating that when you only have two servers, you have 50% chances that a redispatch will get back to the faulty server. And it's in fact even harder to explain to people, so it seems that it doesn't appear logical :-)

This reminds me about a feature I've wanted to add for a long time now: two different check intervals. The normal one (current "inter") and a fast one, used during transitions (eg: "fastinter"), used to speed up state transition when a failed check has been detected. The idea is that once this is in place, it should be relatively simple to switch a server to fastinter as soon as it experiences a redispatch. This will make it possible to detect server failures very quickly even with slow checks.

For instance, if we consider that a server is checked every 10 seconds and has 3 retries ("fall 3"), 30 seconds will be necessary to detect a failure. Now if we use "inter 10s fastinter 500ms", we will need 11 seconds to detect a failure, or 1.5 second after the first redispatch occurs. In this example, this is 20 times faster than what we can currently achieve.

Regards,
Willy Received on 2007/12/26 10:34