Deploying a Set of HAProxy Servers as EC2 Instances

If you want high availability in EC2, one option is to deploy a couple of load balancers in front of a bunch of application servers, all in the same security group. I really like HAProxy. It's fast, and very configurable. I remember we got a couple of F5's back in the day and I wanna say that was like 60k for two. Which is a lot if you're a startup. Not that you can deploy them in EC2 but it's just kinda cool how all this stuff trickles on down to the cloud.

I think it makes complete sense. Anyway, here's how I did it recently. Since you can't get a floater IP in EC2, or more than one IP per instance, I setup DNS round robin for the proxy addresses. So, in this case we have two dedicated ec2 instances, running CentOS. Each one will have haproxy installed and configured. So, once you get them spun up, you'll want to get an elastic IP for each, then configure DNS to point to both.

www.domain.com { 1.2.3.4, 1.2.3.5 }

Each IP is the public address of each of your HAProxy servers. To test, you can just setup a dummy hostname, like test.domain.com pointing to those IP's and do the cut over when you're sure you're happy with the setup.

So, next, login to each instance and run:

yum -y install haproxy

HAProxy supports two modes, tcp and http. You can't do SSL in http mode so, this deployment was in tcp mode. HTTP mode has some really cool and interesting features with HAProxy's recent acl additions. Google around for HAProxy and ACL. You can get super granular on which app server handles what kind of traffic to include or where to direct certain kinds of request.. for example: go here for SSL, here for dynamic content and here for static HTML and images. It's really pretty cool and new in 1.3, I think. Maybe I'll try all that out someday.

Next, edit /etc/haproxy/haproxy.cfg and enter the following:

# HA Proxy Configuration
defaults
balance roundrobin

global
    chroot      /var/lib/haproxy
    pidfile     /var/run/haproxy.pid
    maxconn     20000
    user        haproxy
    group       haproxy
    daemon

defaults
    mode        tcp
    log         127.0.0.1       local0
    log         127.0.0.1       local1 notice
    option      dontlognull
    option      redispatch
    timeout connect 2000 # default 2 second time out if a backend is not found
    timeout client 300000
    timeout server 300000
    maxconn     60000
    retries     3

# This is what we're listening on.
frontend haproxy *:443
   mode tcp
   maxconn 20480
   default_backend app_servers

# This is who we send requests to
backend app_servers
   mode tcp
   server app1 10.19.127.30:443
   server app2 10.19.127.49:443
   server app3 10.19.127.21:443
   server app4 10.19.127.19:443

So, in this example we have 4 app servers. I feel like it's so simple and self explanatory that you can just get in there and edit and test it out. Both HAProxy instances have the exact same configuration file - assuming you've deployed everything to the same security group.

The only other thing i did was to add a snippit to /etc/syslog-ng/syslog-ng.conf to log all HAProxy's messages via syslog. That's here /etc/syslog-ng/syslog-ng.conf:

source s_udp {
       udp(ip(127.0.0.1) port(514));
};
destination d_haproxy { file("/var/log/haproxy"); };
filter f_local0 { facility(local0); };
log { source(s_udp); filter(f_local0); destination(d_haproxy); };

Now, just fire it all up:

service haproxy start
chkconfig haproxy on
service syslog-ng restart

HAProxy has a stats interface which I haven't enabled here. If I do, I'll edit the above with the stat's info config.