I was treating the total as the total number of successful sessions.
This site isn't live and some things were intentionally broken for
testing. After putting the site back into working order I can confirm
that the total sessions counter works as you've described.
This is actually a php driven site but for anyone who finds the mongrel reference via a search, go check out Passenger as a much better alternative.
Thanks again for the explanation Willy.
Ryan
(Missed the Reply All button the first time. Sorry for the duplicate Willy.)
On 09/23/2009 01:20 PM, Willy Tarreau wrote:
> On Tue, Sep 22, 2009 at 05:49:33PM -0700, Ryan Schlesinger wrote:
>
>> I've obtained permission to post my config file.
>>
> wow, you're having a BIG network problem. Look at the connection
> errors and the retries. You'll see than when you sum them up, you
> almost get the number of sessions per server.
>
> The number of sessions per server indicates how many connect
> attempts have really been emitted at the server. It was necessary
> to account for this because some people sometimes asked why they
> saw more log lines on their servers or firewalls than they saw on
> haproxy. So the total number of sessions per server is defined as
> the number of connect attempts, not the number of successes.
>
> Having 75% connection retry is a very big issue. Your site must
> be almost unresponsive. There are two possibilities :
>
> - either you have a flaky network equipment somewhere (eg: poor
> cable, duplex mismatch on switch/server, ...)
> - or your server does not accept enough connections
>
> I'd be tempted to rule out first point because there are no response
> errors, which means that once a connection succeeds, it's correctly
> served. A network issue would also cause this column to grow.
>
> So the other possibility is that your server is saturated. I see from
> the "sessions/max" column that you never went above 2 concurrent
> connections. Let me guess : wouldn't this be a mongrel server ? If so,
> that would explain everything. Mongrel only accepts one connection.
> This means that as long as a user is being served, nobody else can
> connect! To workaround this poor design, people who use it in
> production tend to install tens to hundreds of instances of it on a
> machine and perform the load-balancing between all those instances.
>
> A second thing to do when your server has such extreme limits is
> to set the limit on the server line in the haproxy's configuration,
> here :
>
>
>> balance roundrobin
>> server backup 127.0.0.1:81 backup
>>
> You should use this :
>
> server backup 127.0.0.1:81 backup maxconn 1
>
> That way, instead of trying to establish a connection, wait for TCP
> retransmits and drop packets, haproxy will queue excess requests and
> serialize them. It's already efficient when there is only one server,
> but it's even more when there are multiple servers, because when you
> reach tens to hundreds of instances, haproxy+mongrel behave like a
> normal server since statically there's always the capacity to accept
> new connections. Don't forget to set "timeout queue 30s" or higher,
> otherwise the default timeout equals the connect timeout.
>
> You will also have to enable health checks when you deploy more
> instances. Since the health checks will sometime have to wait for a
> free slot to establish a connection, I suggest only using them in TCP
> mode, or using them on a cheap HTTP status page and allowing large
> timeouts.
>
> If you're really running on mongrel or a similar product, please check
> the web (and the links at the bottom of haproxy's web page), as there
> are very detailed articles on the net explaining how to tune such a
> setup to work well in production.
>
> I hope this helps,
> Willy
>
>
Received on 2009/09/23 22:38
This archive was generated by hypermail 2.2.0 : 2009/09/23 22:45 CEST