Re: Sky is the limit with HAProxy?

From: Marcus Herou <marcus.herou#tailsweep.com>
Date: Thu, 25 Sep 2008 11:14:10 +0200


Hi.

Here is my sysctl.conf please give input if you find it necessary.

# Tcp memory

net.core.rmem_max=16777216
net.core.wmem_max=16777216
net.ipv4.tcp_rmem=4096 87380 16777216
net.ipv4.tcp_wmem=4096 65536 16777216

# Increase the number of incoming connections that can queue up before dropping
net.core.somaxconn = 262144

# Big queue for the network device
net.core.netdev_max_backlog=30000

# Apache Scaling suggests 1000 ?

net.ipv4.tcp_max_orphans = 262144
net.ipv4.tcp_max_syn_backlog = 262144
net.ipv4.tcp_synack_retries = 2
net.ipv4.tcp_syn_retries = 2

# Lots of local ports for connections
net.ipv4.tcp_max_tw_buckets = 1000000
net.ipv4.ip_local_port_range = 1024 65000
net.ipv4.tcp_sack = 1
net.ipv4.tcp_keepalive_time = 300
net.ipv4.tcp_synack_retries = 0

# These ensure that TIME_WAIT ports either get reused or closed fast. net.ipv4.tcp_fin_timeout = 1
net.ipv4.tcp_tw_recycle = 1

# Security
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_rfc1337 = 1

# Disables IP source routing

net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.lo.accept_source_route = 0
net.ipv4.conf.eth0.accept_source_route = 0
net.ipv4.conf.eth1.accept_source_route = 0
net.ipv4.conf.default.accept_source_route = 0

# Decrease the time default value for tcp_fin_timeout connection net.ipv4.tcp_fin_timeout = 30

# Tuning the FS
fs.file-max = 5049800

# Tuning the VM - According to
http://kb.pert.geant2.net/PERTKB/ApacheScaling vm.min_free_kbytes = 204800
vm.page-cluster = 20
# Apache suggests 200 but most say 0.... vm.swappiness = 10

On Thu, Sep 25, 2008 at 9:36 AM, Christian Wiese <morfoh#opensde.org> wrote:

> Hi Marcus,
>
> just my 2 cents regarding your hardware. I think that it
> will be more than perfectly ok.
> At work we built a haproxy based failover-loadbalancing
> solution for a customer, using Linux running on an Intel(R)
> Core(TM)2 Quad CPU 2.4GHz, with 2GB of RAM.
> Currently this haproxy handles traffic between 500-600
> MBit/s at peak times.
> You might have seen Willy's report while testing 10G NICs
> (http://haproxy.1wt.eu/10g.html), which shows quite well,
> that the overall performance depends on quite some things,
> and hardware has to be wisely choosen and you should
> definetly do some tests to find a reasonable setup for your
> use case.
> Definitely you have to get your hands a bit dirty while
> tweaking kernel settings via sysctl, and special care has
> to be taken if you have netfilter's conntrack stuff
> enabled, because you can easily run out of space of the
> conntrack table if you are using the default settings. (At
> least it happened to me, but sysctl is your friend ;)
>
> I have to admit that I became I quite big fan of haproxy,
> and want to take the opportunity to thank Willy and all
> other people working on improving haproxy ;)
>
> Cheers,
> Christian
>
>
> Thu, 25 Sep 2008 07:34:52 +0200 "Marcus Herou"
> <marcus.herou#tailsweep.com> wrote:
>
> > Hi. Thanks that you took the time to answer.
> >
> > Dirty Harry is my second name :)
> >
> > Jokes aside, tweaking and tuning is what I like and
> > performance is my high. I do not believe in silver
> > bullets. I used to work for a huge company as a
> > consultant and even though how much money they poured
> > into the arch it never performed since they always needed
> > everything to solve everything. For instance having
> > Apache load all modules possible and turn on all options,
> > having all java webapps in one BIG container etc. Stupid
> > and waste of money and resources and in the end user
> > experience.
> >
> > The requirements of our load balancing is quite simple
> > since the nature of our architecure is as well. Basically
> > all dynamic requests (which needs LB) should go to
> > script.tailsweep.com and static ones to
> > media.tailsweep.com and soon to come static.tailsweep.com.
> >
> > I will test haproxy on Ubuntu Hardy 64bit, 8GB, quad-core
> > 2.4MHZ, RAID-1and apply a sysctl.conf which I normally
> > use on our webservers. Is Linux bytw OK for the job ?
> > Know some people do not like the schedulers in Linux.
> > Want to have a look at the sysctl.conf ?
> >
> > I think I used 1.3.XX (perhaps .12 since it ships with
> > Ubuntu these days). The webapp only had about 20 reqs/sec
> > so it worked just perfectly. The main reason why I look
> > at HAProxy is because my friends and some former
> > colleagues recommends the product.
> >
> > Kindly
> >
> > //Marcus
> >
> >
> >
> >
> >
> > On Thu, Sep 25, 2008 at 5:52 AM, Jeffrey 'jf' Lim
> > <jfs.world#gmail.com>wrote:
> >
> > > On Thu, Sep 25, 2008 at 3:01 AM, Marcus Herou
> > > <marcus.herou#tailsweep.com> wrote:
> > > > Hi.
> > > >
> > > > <snip>
> > > >
> > > > The question is: Is HAProxy an alternative for us ?
> > > >
> > > > I'm thinking like this; Buy a couple of servers tuned
> > > > for webserving and
> > > try
> > > > HAProxy on them and if that does not turn out well
> > > > make them regular webservers and buy a hardware
> > > > loadbalancer like the ones from loadbalancer.org
> > > >
> > > > I'm a geek at tuning systems on many levels and hate
> > > > to waste money in unneeded infrastructure so I would
> > > > really like to find that HAProxy meet
> > > my
> > > > criterias.
> > > >
> > > > I would as well rather have many smaller
> > > > loadbalancers tweaked at serving different content
> > > > than having a monster serving everything.
> > > >
> > > > Anyone have any input to guide me?
> > > >
> > >
> > > haproxy can be fast - a lot of it will depend on both
> > > how you tweak it.. and the platform on which you run
> > > it. Do you have any special requirements for your
> > > load-balancing? I find it helps for you to actually get
> > > your hands dirty before you find out whether a piece of
> > > software is suitable for you. I'd be curious about the
> > > version of haproxy that you used in the past - which
> > > version was it? and how was your experience with it?
> > >
> > > -jf
> > >
> >
> >
> >
>

-- 
Marcus Herou CTO and co-founder Tailsweep AB
+46702561312
marcus.herou#tailsweep.com
http://www.tailsweep.com/
http://blogg.tailsweep.com/
Received on 2008/09/25 11:14

This archive was generated by hypermail 2.2.0 : 2008/09/25 11:17 CEST