High Availability using Round Robin DNS

A couple of months ago I wrote about using Nginx as a load balancer to ensure uptime. However this still leaves a single point of failure e.g. if the load balancer goes down this means the client cannot reach your application servers. One solution to this is to have a secondary load balancer. How would such a setup work?

Round Robin DNS

To make use of this we would need to have multiple A records for a particular domain. For example say our endpoint is example.com we would need to have an A record pointing to load balancer 1's IP address and another A record pointing to load balancer 2's IP address. When a user types the domain name into the browser, DNS will return a list of IP addresses that point to the said resource, most Http clients will try the first IP in the list, if this is unavailable it will try the second one and so on. So if the first IP is unresponsive they will be directed to the second IP, all this of course is transparent to the user.

Sample Round Robin DNS Setup

Gotchas

Sometimes some http clients will not attempt to try the second IPs unless the user does a page refresh, other times because of a DNS cache the browser may not have the secondary IP available. DNS propagation is also something to consider. This method will not work for application level errors on your load balancer, thus it is important to have monitoring for the application health of your load balancers. Something else to consider is that you now have multiple servers to manage. For example now when generating SSL certificates you'll need to make them available to both load balancers.

Are there other techniques you are using to ensure HA for your applications?