Re: 1.5 badly dies after a few seconds

Thank you for the heads up,

Managed to put it in a single listen block, and worked! Temporarily! ;( Was fine on testing environment, but after putting it into production, haproxy gone wild after 40mins, and then after 20mins in the next round. 'Wild' being it returner Error instead of serving up files from an other block (symptom being static files missing from site), not even the one being rate restricted, and a few mins later it completely 'died' not serving anything just seemingly loading endlessly.

If you could let me know what debug to save or look at, I'd be happy to do so. Meanwhile I've rolled back to 1.3 (exact same config except for rate limiting), being a reliable provider for months.

Compiled from latest source, by make -f Makefile.bsd REGEX=pcre DEBUG= COPTS.generic="-Os -fomit-frame-pointer" (no mgnu)

Whole config attached:
global

         log     127.0.0.1       daemon  debug
         maxconn 1024
         chroot /var/chroot/haproxy
         uid 99
         gid 99
         daemon
         quiet
         pidfile /var/run/haproxy-private.pid

defaults
         log     global
         mode    http
         option  httplog
         option  dontlognull
         option redispatch
         retries 3
         maxconn 3000
         contimeout      4000
         clitimeout      1000
         srvtimeout      200000
         stats enable
         stats scope Static-farm
         stats scope mySite-webfarm
         stats scope ease-up
         stats uri     /admin?stats
         stats realm   Haproxy\ Statistics
         stats auth    user:pass

listen mySite-webfarm 82.136.11.111:80
     option forwardfor
     option httpchk /!healthcheck.php
     option httpclose

     balance roundrobin
     server host1 192.168.0.4:80 check inter 5000 rise 2 fall 3
     server host2 192.168.0.3:80 check inter 5000 rise 2 fall 3

     contimeout  6000
     clitimeout  2000

     errorfile 503 /usr/local/etc/503error.html

     ### (d)dos protection ###
     # check master 'banned' table first
     stick-table type ip size 1m expire 5m store gpc0,conn_rate(10s)

     acl source_is_abuser src_get_gpc0 gt 0
     use_backend ease-up if source_is_abuser
     tcp-request connection track-sc1 src

     # values below are specific to the backend
     tcp-request content  track-sc2 src
     acl conn_rate_abuse  sc2_conn_rate gt 30

     # abuse is marked in the frontend so that it's shared between all sites
     acl mark_as_abuser  sc1_inc_gpc0 gt 0
     #tcp-request content reject if conn_rate_abuse mark_as_abuser
     use_backend ease-up if conn_rate_abuse mark_as_abuser


backend ease-up
     mode http
     errorfile 503 /usr/local/etc/503error_dos.html


listen Static-farm 82.136.11.114:80
     balance roundrobin
     option forwardfor
     option httpclose
     option httpchk
     server stat1 192.168.0.8:80 check inter 3000 rise 3 fall 2
     server stat2 192.168.0.7:80 check inter 3000 rise 3 fall 2 backup

Thank you,
Joe

Idézet (Willy Tarreau <w#1wt.eu>):

> On Wed, Sep 15, 2010 at 07:17:32AM +0200, R.Nagy József wrote:
>> My bad, most likely.
>> After killing haproxy process completely -instead of just config
>> reloads-, and restarting it, problem can't be reproduced anymore
>> without rate limiting config.
>
> OK, thanks for this clarification.
>
>> So most likely it was simply rejecting the request where it seemed to
>> be serving 'random' blank pages due to config not being reloaded
>> properly.
>
> indeed.
>
>> Number of denied reqs in the stats is 0 all along though. Bug?
>
> No it's expected if you drop at the connection level. Only sessions
> are accounted right now in the stats. A session is defined as a
> connection that has been accepted. The difference is important for
> analyzing what causes the drops. More counters should be added, but
> there will probably be some more general work on the stats first.
>
>> Let me mod the question then though:
>> All I'm trying to achieve is a simple rate limiting config against
>> (d)dos attacks.
>> Need to:
>> - Serve custom 503 page when client is banned (never give blank page)
>> - Ban with over 30reqs/10secs, temp ban for 10mins then
>>
>> Based on "better rate limiting" and docs, I came up with the config
>> below, but problem is, the rate limiting does not take place with
>> "use_backend ease-up if conn_rate_abuse mark_as_abuser" in the
>> backend, while it does _reject_ the page if I use "tcp-request content
>> reject if conn_rate_abuse mark_as_abuser" in there (but I need custom
>> 503 as stated above).
>
> In my opinion your config is OK for this and I see no reason why it
> should not work (however you have "src_get_gpc0(http)" instead of
> naming the correct frontend, but I assume that's because you renamed
> the frontend before sending the conf).
>
>> By the way: to achieve this with as simple config as possible, could 2
>> stick-tables config be put under a single listen block (don't need
>> separate frontend/backend blocks for anything but this)?
>
> Yes, you could even have the same stick-table for this and store two
> different data. The fact that the Stackoverflow's config makes use of
> two stick tables is because they wanted to measure the request only
> on some backends.
>
> If you want to store both gpc0 and conn_rate over 10 seconds, simply
> declare it this way :
>
> stick-table type ip size 200k expire 10m store gpc0,conn_rate(10s)
>
> Regards,
> Willy
>
> ---
Received on 2010/09/15 10:17