Cannot bind to source address before connect() for server

From: Simon Yeo <simon#meebo-inc.com>
Date: Fri, 15 May 2009 07:09:58 -0700

Has anyone see this error in haproxy logs?

Somewhere between 65k and 70k sessions, haproxy starts spewing the error and things start dying from that point. CPU(s) get pegged at 100%, healthchecks start failing, etc.

Some relevant information:
-- incoming bandwith is no more than 50Mbps at 65k sessions

• Centos 5.2 (kernel 2.6.18-92.1.22.el5)
• haproxy-1.3.14.11

Config details:

        maxconn 400000
        mode http
        balance roundrobin
        option httpchk HEAD / HTTP/1.0
        option forwardfor except 127.0.0.1
        option httpclose

each server gets "check inter 3s source [ip-address]", ip-address varies

sysctl stuff:

net.ipv4.ip_local_port_range = 1024 65000
net.ipv4.tcp_max_syn_backlog = 400000
net.ipv4.tcp_synack_retries = 3

fs.file-max = 1048576

net.core.somaxconn = 400000
net.core.netdev_max_backlog = 400000
net.ipv4.tcp_keepalive_time = 1800
net.ipv4.tcp_max_tw_buckets = 1440000

thanks in advance,
--simon Received on 2009/05/15 16:09