We recently discovered that our cloud provider’s load-balancing layer does not support HTTP Keep-Alive requests. HTTP Keep-Alive allows an HTTP client to reuse an HTTP connection, which removes the overhead of TCP and SSL handshakes; this is especially critical for clients that poll for weather, which make up a substantial fraction of our API customers.
Our cloud provider did not plan to update their load-balancing layer to support Keep-Alive requests, and so we have taken the step of running our own load-balancing layer in-house. As of 4PM today, this load-balancing layer is live, and HTTP Keep-Alive requests are supported. Clients that poll our API and enable this HTTP feature should see substantially reduced overhead.
Additionally, this new load-balacing layer is being carefully monitored at multiple levels, closing the open downtime incident that we reported about on 28 Sep 2016.