Re: HAProxy and FreeBSD EADDRINUSE

Hi James,

On Sun, Sep 21, 2008 at 01:26:58AM -0700, James Satterfield wrote:
> So it seems that setting the source address in the proxy definition is
> the culprit. At least, I've gone about 3 hours now without a
> connection failure since removing the source address definition.

That's amusing because I experienced the opposite on OpenBSD. I noticed the connection failures went away when I forced the source address.

> The explanation I've come up with makes sense to me, however I'm not a
> coder and I don't know C. So I'm presenting this as my understanding
> of what's going on, not as a statement of fact. Please correct me if
> I'm wrong.
> Setting the source address for the server or the proxy causes haproxy
> to call bind() to set the IP we've requested on the fd. haproxy sets
> sockopt SO_REUSEADDR. This succeeds but periodically the connect()
> fails with an EADDRINUSE.
> The reading I've done suggests that SO_REUSEADDR allows you to reuse
> local addresses but the 5 tuple must still be unique. So if haproxy
> reuses a socket currently in TIME_WAIT with web1:80 to connect() to
> web1:80 it will fail with EADDRINUSE.

OK, that would make sense. It's a bit annoying that the OS does not let you re-use a port you recently released. It would be nice to find if there is an equivalent of the Linux's tcp_tw_reuse flag which allows you to reuse the TIME_WAIT sockets.

It does not explain why not binding works though. Because in that case, the OS must find itself one free ip:port couple. If it works, it should also when you just bind to ip: and let the OS choose the port.

There is something else you can do. In proto_tcp.c, you can duplicate the line with SO_REUSEADDR in tcpv4_bind_socket(), and replace SO_REUSEADDR with SO_REUSEPORT. Then check if it works better or not. I know that both FreeBSD and OpenBSD let you bind multiple processes on same IP:ports when SO_REUSEPORT is set, so it is possible that it would help here. Something like this :

        setsockopt(fd, SOL_SOCKET, SO_REUSEADDR, (char *) &one, sizeof(one));
        setsockopt(fd, SOL_SOCKET, SO_REUSEPORT, (char *) &one, sizeof(one));

If it works, we'll add this for FreeBSD.

Regards,
Willy Received on 2008/09/23 07:06