Tuesday, September 11, 2012

Client Side Load Balancing and Failover with Javascript and Cookies


Today there is many kind of infrastructure to make load balancing and fail over on the web :

- load-balancing using DNS : a domain name server that dynamically change the ip address of the requested host, by using metrics like traffic load, or Geoip .
References :
  • Big IP GTM 
  • Using Opensource project like Powerdns combined with geoip, and a script that get health of servers and write them into a mysql dns record to maintain an ordered list of servers by using health metrics.

- load balancing using network :

  • BGP loadbalancing, but you need to have an Autonomous system.

- load balancing using proxies, best when used with dynamic dns loadbalancing between cluster of server on different location ( datacenters ) :
  • Haproxy, that can also check back-end web-servers availability 
  • Varnish, If you use it like Haproxy.
and many others....

The main problem today occurs when the browser needs to get static content from static servers and that you use classic load-balancing and fail-over solution. The browser will get the data to the targeted static server however you may experience that your server is down and you can't deliver static content to the browser.                 

 So, the only way to make sure that the content can be delivered to the client browser, it's to declare a multiple IP address list for a specific resource and let the browser handle the failover by using available ressource in that list.
Proof of Concept

How it works ?

We need two things :
  •  Dynamic web server, that serves transparently a dynamic array of static servers into cookie when a page is get.
  • JavaScript code, this code reads the cookie and make an on demand fail-over and transparent load balancing.
If the same document is requested by the client browser it could result of a HTTP code status code 304 "Not modified", but the cookie is always transferred. it save in bandwidth to do not dynamically change static servers list into the body of the document.

You can see a fully working proof of concept here : here .

Apache2 and mod_rewrite module will make it possible.

An example of Apache2 vhost configuration :

#Enable Rewrite Engine
   RewriteEngine On

#Add the RewriteMap source file "cookmap.txt", it contain an array of Key / Value, if this file is updated  Apache server updates it and delivers it for the next client request

   RewriteMap cookmap txt:/opt/apache2/cookmap.txt

# We define a rule, we say here that for any url, deliver cookie to the client with a value from the RewriteMap file, we call the  "static_server"  key.

        RewriteRule /* - [CO=static_server:${cookmap:static_server}:]

The content of RewriteMap file cookmap.txt :

static_server www2.example.com,100|,100|,10

key : static_server
value : www2.example.com,100|,100|,100;

Each value are separated by a "|" (pipe) and have two other values separated by ",", the first one is the static server address and the other the maximum accepted loading time accepted for this server.

www2.example.com         100

note : don't set low loading time, because of network latency on ISP (Dsl,cable,mobile).

How to handle the fact that the resource is correctly loaded ?

The Javascript offer HTML DOM events that can be used in JavaScript, now
i would like to load a picture and use events to know if the picture is loaded or not.
So we need to make a function that is called from the img html attribute when the picture won't to load, for that we have onload, onerror and onabort.

  • Onerror : if the picture canno't be load, call loadbalancer function to switch to an another ip from the cookie.
  • Onabort : if the picture loading is aborted by any event, it call loadbalancer function to switch to an another ip from the cookie.
  • Onload : when the picture is correctly loaded unhide the picture and display it to the user because it was hidden by CSS stylesheet style.

    The IMG HTML tag :

    <img src='' onload='lb_ready(this.id);' onabort='go(this.id=get_ts(),this.src)' onerror='go(this.id=get_ts(),this.src);' style='visibility:hidden' >

    This call a picture test.jpg  on the web-server,  this server don't have the picture so, the onerror event catch it and call JavaScript function with a unique id based on timestamp.

    You can see  style='visibility:hidden' is necessary to hide picture before the picture load.

    A trick to force browser to call onerror event  directly :

    <img src='data:image/png;base64,' title='/testa.jpg' onload='lb_ready(this.id);'  onabort='go(this.id=get_ts(),this.title)' onerror='go(this.id=get_ts(),this.title);'  style='visibility:hidden' >

    by doing src='data:image/png;base64,' title='/testa.jpg , the browser want to load encoded base64 picture but it's empty, so onerror event catch it and call JavaScript failover function with title as requested file.

    Look on the commented source code here
What about now?

Without restarting Apache2, we can modify the file that contain value of cookie.


How to make list of static servers dynamically, there is many possibilities, it depend of your infrastructure. Having  daemon  that run on each static server  and deliver health to the dynamic server that choose how to order the list of static server into the cookie.

So how can i deliver status of static servers to the dynamic one?

It depends of the choice, it could be technological choice, strategic choice or an economic one.

You can with Apache2 and GeoIp module, deliver a cookie in function of the client IP address location.

You can also add  a score and a weight for each static server.
For example : sort static servers by bandwidth quota in order to fetch data from the cheapest servers.

You can also use round robin to share, or to loadbalance.

There are many possibilities, it's up to you.

Server side web applications can also handle a list of static servers and set it into an Apache rewrite-map file.

It could  be also integrated with BIG IP like load balancer used for dynamic servers,
It will send the cookie list of static servers.

what's next ?

Make a library that can be integrated in any code, and do the trick automatically.

- Fix some bugs on IE and others. making it compatible to almost client browser.

- Find a way to do the same with ajax calls, get json. shard data across multiple static servers.

Conclusions :

Mixing a server list from cookies and web client side application ensure a better reliability for web applications content like pictures and others, it let the possiblity to control traffic, failover, loadbalancing  on client and server side

Reference :

No comments:

Post a Comment