Re: Server response time discrepancy.

On Friday 20 June 2008 08:15:01 Unai Rodriguez wrote:
> Dear Willy,
>
> > These times are huge (and unacceptable). What I don't understand is
> > why changing the location of the *client* has any impact on the
> > *server* response time.
> >
> > Ah, maybe I have an idea. Are those requests posts with large data ?
> > In that case, the time for the client to post all data to the server
> > will be included in the response time. If this is the case, you may
> > like to log the request's Content-length header using :
> >
> > capture request header content-length
> >
> > If this is not the problem, then I have no idea right now. BTW, the
> > total time seems huge too (several minutes). Are you sure that
> > "option httpclose" is set ? It should not impact server's response
> > time, but it would be good to ensure that timers are correct.
>
> Some of the full log lines that correspond to the "suspicious" requests:
>
> -------------------------
> Jun 13 7:11:59 localhost haproxy[24367]: 204.11.203.211:52926
> [13/Jun/2008:07:11:02] DPSOP-UAT 10.123.12.112 3/0/3/56757/56883 200 23092
> - - CDVN 1/1/2001 0/0 POST /DPDataServices2/ProductWebService.asmx HTTP/1.0
> -------------------------
> Jun 13 7:12:24 localhost haproxy[24367]: 204.11.203.211:52973
> [13/Jun/2008:07:11:44] DPSOP-UAT 10.123.12.112 4/0/1/40364/40580 200 23092
> - - CDVN 0/0/0 0/0 POST /DPDataServices2/ProductWebService.asmx HTTP/1.0
> -------------------------
> Jun 13 7:14:07 localhost haproxy[24367]: 204.11.203.211:53017
> [13/Jun/2008:07:12:25] DPSOP-UAT 10.123.12.112 3/0/4/102493/102616 200
> 23092 - - CDVN 0/0/0 0/0 POST /DPDataServices2/ProductWebService.asmx
> HTTP/1.0
> -------------------------
> Jun 13 1:27:27 localhost haproxy[24367]: 204.11.203.211:58106
> [13/Jun/2008:01:25:21] DPSOP-UAT 10.123.12.112 4/0/3/125353/125487 200
> 25003 - - CDVN 1/1/2001 0/0 POST /DPDataServices2/OrderWebService.asmx
> HTTP/1.0
> --------------------------
>
> The configuration can be found here: http://pastebin.ca/1051753
>
> I am troubleshooting with the developers now, it seems that the client
> detects some sort of network error and does not reply anymore to the
> server. That is why we are getting the CDxx termination state, I guess
> (correct me if I am wrong, please).
>
> As for the amount of data being transfer, OrderWebService.asmx calls
> usually generate a lot of data from the client to the server and vice versa
> (ie the client needs to pass many parameters to the server) WHILE
> ProductWebService.asmx calls usually require a low amount of data from
> client to server, but long response from the server.
>
> Regarding the size of the requests, would that be a concern? I mean, do
> long request make the application less robust or not necessarily (ie
> http/tcp is very reliable)?
>
> I will explore the capture feature ASAP, and update the findings here.
>
> Thank you so much!!!
> unai

Unai,

what kernel version are you running and what is the output of "cat /proc/sys/net/ipv4/tcp_frto"?

I had a similar problem running haproxy on ubuntu 8.04 (standard ubuntu kernel 2.6.24-19-server). One client reported frequent timeouts when going over the loadbalancer. Some debugging with tcpdump revealed that in situations with moderate packet loss the tcp connection to the client could not recover proberly. It looked like the retransmit timeouts increased until the connection completely stalled.
Setting /proc/sys/net/ipv4/tcp_frto to 0 solevd this problem (Originally the value was 2).

Best regards,
-Rainer Received on 2008/06/24 13:52