HA Proxy as Amazon S3 Content Switch

From: John Singleton <jsinglet#gmail.com>
Date: Sat, 27 Sep 2008 16:46:56 -0700

Hi All,

I've been using HA Proxy as a straight load balancer for some time now and decided to check out the Layer 7 features in version 1.3. Specifically I am trying to get content switching working with Amazon S3 as the backend. I've tried just about everything but I can't seem to get it going right and I suspect the reason has something to do with the way S3 handles http.

The setup:

global

         log 127.0.0.1   local0
         log 127.0.0.1   local1 notice


         #log loghost    local0 info

         maxconn 4096
         chroot /usr/share/haproxy
         uid 99
         gid 99
         daemon


         #debug

         #quiet


defaults
         log     global
         mode    http
         option  httplog
         option  dontlognull
         retries 3
         redispatch
         maxconn 2000
         contimeout      5000
         clitimeout      300000
         srvtimeout      300000


##

## HTTP Only Frontend
##

frontend app X.X.X.X:80

         mode http
         acl  image_content url_sub images #simple example to trigger  
the static cdn
         use_backend static_content if image_content
         default_backend dynamic_content

backend static_content
         mode http
         option httpclose
          balance roundrobin
         reqrep  ^([^\ ]*)\ /(.*) \1\ /my-content-cache/web/\2

	server CDN  s3.amazonaws.com:80

Here's what SHOULD happen.

1) Url for /images/foo-image.jpg is requested
2) HA Proxy forwards the request to the backend, static_content
3) Rewrite rule rewrites request from /images/foo-image.jpg -> /my- 

content-cache/web/images/foo-image.jpg
4) File is served!

However, I don't get that result. I simply get a "could not find bucket" error from S3. S3 is very picky about bucket naming, (ie, / web//images/image.jpg is not the same thing as /web/images/image.jpg) but I don't think that's the problem. Testing this setup against a regular apache server works fine---any idea what I am doing wrong?

Perhaps someone can shed some light on this?

All Best
JLS Received on 2008/09/28 01:46