Okay Willy. I stand corrected. Sorry about the confusion.
Thanks for clearing that up.
-Dan
Willy Tarreau wrote:
> Hi Guys,
>
> On Tue, Dec 18, 2007 at 11:13:52AM -0700, Dan Zubey wrote:
>> We're going through some of the same thinking. BGP looks like a route we
>> may have to take soon as well.
>
> Please see at the end of this mail for an explanation.
>
>> John Marrett wrote:
>>
>> <snip>
>>
>>> - What is this nbsrv you refer to? All I can find are references to
>>> some kind of worm that uses that name?
>> nbsrv is an option in the haproxy config file that specifies how many
>> haproxy processes to fork. Unless you have special needs, one works fine
>> for almost all loads. The down side of using more than one is that the
>> statistics page only shows stats from the reporting thread, not all of
>> them.
>
> No Dan, it's not the same. You're talking about nbproc in the global section,
> while John is talking about the nbsrv acl verb. This one may be used to
> match depending on the number of available servers in a backend :
>
> acl enough_static nbsrv(static) ge 3
> acl enough_dynamic nbsrv(dynamic) ge 5
>
> Then you can decide to direct traffic to one backend or the other depending
> on the number of available servers. But that was not the original goal, see
> below.
>
>>> - Are there existing open source tools that will allow me to handle
>>> intra site health checking and DNS updates? Our current plans are
>>> rolling a perl script and using dynamic DNS to update DNS information
>>>
>> Not sure. I, for one, would like to hear of what's available as well.
>> And so would several thousand of our users.
>
> Anyway, DNS is horribly bad for high availability. I recently read a very
> good article on the subject that comforted me in this feeling I've been
> having for a long time. Basically, the problem find its roots in caches.
> DNS is only good to spread multiple IP addresses which are almost never
> updated. Use BGP the find where the IPs are located.
>
>>> - Currently we aren't quite ready for it, but it would be very
>>> interesting to take BGP information, and use it to refer customers to
>>> the closest site, any ideas on this subject?
>> Same. This would be the holy grail of scalability options for us.
>
> If you announce multiple IP addresses with your DNS, and if all those
> addresses are available on all sites, BGP will ensure that your customers
> will reach them on the closest site for them.
>
>>> Any advice that can be provided on the subject of open source tools and
>>> GSLB will be greatly appreciated!
>> We're looking at spread/wackamole (http://www.backhand.org/wackamole/),
>> and Linux High Availability as well.
>>
>> Spread/wackamole won't do GSLB (so far as I know) but it swaps IPs
>> around, and can effectively do the same function. Having spread
>> installed opens a few more doors for inter-node communication. But I
>> drift off topic.
>>
>> One of my dream feature requests for Willy is to have Haproxy listen to
>> a unix socket and accept changes to the servers on the fly. Haproxy has
>> a real powerful internal architecture.
>
> It's in progress :-)
> We already need to be able to enable/disable a server on the fly, but this
> will still be limited. I think that when I will start version 2.0 (in quite
> some time), the idea will be to have a load-balancing daemon which only does
> its load-balancing work, and a user-space client to feed its configuration.
> It will then be a lot easier to support on-the-fly reconfiguration.
>
> OK, now that that's said, here's the initial idea behind nbsrv. For a major
> customer, we have a situation in which we can basically find the following
> elements chained in this order :
>
> [clients] --- [ cisco router ] --- [ alteon ] --- [ haproxy ] --- [ servers ]
>
> Now imagine that the site is replicated with a link between the routers :
>
> [clients] --- [ cisco router ] --- [ alteon ] --- [ haproxy ] --- [ servers ]
> |
> |
> [clients] --- [ cisco router ] --- [ alteon ] --- [ haproxy ] --- [ servers ]
>
> It is possible for a given client to reach the same application on two sites,
> depending on where he's routed. The routing decision must be based on service
> availability. How can the availability be measured ?
>
> Well, here's what we had before 1.3.14 :
> - haproxy checks the servers and uses only those which worked ;
> - alteon checks that haproxy responds to the "monitor" requests, and enables
> or disables a VIP if all of its haproxy real servers are dead.
> - alteon announces its available VIPs to the cisco routers via BGP
> - the client routes through a BGP-enabled router, and is ensured to find
> at least one alteon with an available VIP which can reach a haproxy.
>
> But... If all servers are dead behind the haproxy, it's pointless. So we have
> developped the conditionnal monitor request. It is now possible for haproxy
> to tell the alteon that it's unavailable if there are not enough servers
> behind it. You would declare it like this :
>
> acl enough_static nbsrv(static) ge 3
> acl enough_dynamic nbsrv(dynamic) ge 5
>
> monitor-uri /haproxy_alive
> monitor fail unless enough_static enough_dynamic
>
> Note that the alteon here is not required. Starting with Cisco IOS 12.4,
> the SLA monitor allows many tricks. For instance, you can put conditions
> on routes. So it is completely possible to have as many static routes on
> your router as you have service addresses on haproxy, and put a condition
> on them so that they stop being announced if the SLA monitor check returns
> an error. The SLA monitor would simply query haproxy's monitor URI and by
> this way ensure that there are enough servers behind it.
>
> The whole idea of availability propagation from the servers to the clients
> is very interesting because it allows many strange architectures where you
> can cascade your equipments. For instance, it's completely possible to have
> multiple level of haproxies, each checking the others to ensure they have
> enough servers behind them to process some specific traffic.
>
> Some people have suggested to me to see how we could combine this with Quagga.
> In fact, it should not be too hard, we would just need an external script to
> regularly check the stats page on the UNIX socket, and update Quagga's routing
> table or announces depending on the frontend or backend's states. This way,
> you could have your load balancer doing BGP for a very low cost (but some
> work still).
>
> I don't know if my explanation was clear enough.
>
> Regards,
> Willy
>
>
Received on 2007/12/19 02:58