Re: RFC: continous counters (fwd)

On Sun, 4 Nov 2007, Willy Tarreau wrote:

> Hi Krzysztof,
>
> On Sun, Nov 04, 2007 at 08:09:39PM +0100, Krzysztof Oledzki wrote:
>> Hello,
>>
>> Currently counters get increased when session finishes. It works quite well
>> when serving small objects, but with big ones (for example large images or
>> archives) or with A/V streaming, a grap generated from haproxy counters
>> looks like a hedgehog. ;)
>
> Yes, I've already observed that too. Interestingly, this only happens with
> moderate loads (few users with large objects). Because as soon as you have
> thousands of users, you reach a point where your stats become smooth.

If you can only have a thousands of users. When you are going to stream A/V content the limit of simultaneous users are not going to be high enough to make stats smooth, especially when an average session may last for >1h.

>> This RFC-quality patch implements a contcnt (continous counters) option,
>> when set makes counters incremented continuously during the whole session.
>> As recounting touches a hotpath directly I decided to make it optional, per
>> a frontend.
>
> I agree with not making it mandatory, for the same reason. I thought this
> should become a stats option (after all, its only purpose is only to get
> continuous stats). But the fact is that stats are often reported by different
> proxies than the ones you want to monitor.
>
> However, there's something which puzzles me. I think you should not update
> the stats as long as the backend has not been assigned or the session closed
> (flags & BE_ASSIGNED I believe). The reason is that with current patch, the
> backend will never be updated for the size of the request headers.

This is somehow different from what I observed: with first call "flags & BE_ASSIGNED" indeed returns 0, but at the same time s->req->total is still 0. If I understand it right it gets incremented only after a request get delivered to a server. So, changing:

         if (s->fe->options & PR_O_CONTCNT)
into:

         if ((s->fe->options & PR_O_CONTSTATS) && (s->flags & SN_BE_ASSIGNED)) does not change anything from counter's point of view, hovewer it prevents this empty call so I trust you that this is better.

>> BTW: It think that PR_O_* defines seems to be in the wrong place. We should
>> move everyting from types/backend.h to types/proxy.h, shouldn't we?
>
> We could. The defines are the only remains of what has moved from backend.h
> to proxy.h. What I wanted first was to separate proxy options between
> frontend-specific and backend-specific. It would still make sense given that
> we use nearly all bits now. But this is still a big change.
>
> BTW, I think that your option should recall the "stats" keyword rather than
> "counters". Maybe "contstats" or something like this ? The more options we
> have, the harder it is to remind them all, and to remember how to write them.
> That's why I'd prefer avoiding something like "cnt" for "counters".

Right, "contstats" looks much better.

Thank you.

Best regards,

                                 Krzysztof Olêdzki Received on 2007/11/05 01:00