How much do you know about HTTP? I’m sure you know it’s at the front of every URL, whether you type it or not, but do you know what version of HTTP your site uses or the difference in performance by switching to HTTPS?
I started this series by mentioning the critical path and how the focus of this series is on the first two steps of the path that result in the first byte of data being received from the server. I covered the first step, the DNS lookup and caching DNS records, over the last two weeks and now it’s time to move to the second step in the path, the HTTP request.
For the next few weeks I want to talk about HTTP. I’ll start today with HTTP/1.1 and some of the performance issues in the protocol and I’ll continue with strategies and tactics to overcome these issues.
Next week I’ll talk specifically about HTTP/1.1 caching and the following week I’ll finish up by talking about HTTP/2, how it overcomes many of the performance issues of HTTP/1.1, and then how we can optimize for performance under HTTP/2. I’ll include some thoughts about HTTPS as well.
Ideal HTTP Performance
In doing research for this series I came across an article by Mark Nottingham about ideal HTTP performance. I recommend reading the article in full, but here’s a summation in Mark’s own words.
a page load should send the minimal amount of data to the server and download the minimal amount of data possible needed from it in the least possible number of round trips.
In other words we don’t want the client or the server to send any extra information and we want the necessary information to be sent across the network in as few trips as possible. With that summary of the goal, let’s look at HTTP/1.1.
Performance Issues in HTTP/1.1
HTTP/1.1 is the hypertext transfer protocol most sites have run over for the last 20 years. It’s a good protocol, but it does come with some downsides, specifically in regards to performance. The three main issues are:
- The additional round trips needed
- The repetition of information sent for each request
- The head-of-line blocking
Your browser sends a request for a web page. Included in the request is user-agent information, the type of content that can be accepted, the type of language expected, cookie information, etc.
The server hopefully sends back the first packet of the HTML file and your browser begins to render the page, discovering that it also needs one or more CSS files, Javascript files, external resources and so on. Your browser then sends additional requests for these other resources, each adding another trip worth of network latency to the load time of the page.
Each of the requests repeats a lot of the same information. Each includes the same user-agent information and accepted content information, etc. so more than the minimal amount of data is being sent to the server with each request. The request headers are considered verbose because they all repeat information the server should know after the first request.
HTPP/1.1 also uses what’s called head-of-line blocking, which means that each request has to be completed in full and in the order they are received before the next request can be processed. This serial process has led to things like CSS sprites in which multiple resources are combined and delivered as a single resource to minimize HTTP requests.
That probably makes HTTP/1.1 sound pretty bad for performance, but it does include some positive things. Caching for example, allows you to serve a fresh local copy instead of having to send requests over the network and it provides conditional requests, which allow you to use older cached items at times.
Web developers have also come up with techniques over the years to minimize the performance issues I just mentioned.
Performance Strategies for HTTP/1.1
The general strategy to improve performance over HTTP/1.1 is to send less data across the network. You want to send fewer requests and you want the requests you do send to send as little data as possible.
While there are many techniques, such as image sprites and concatenation of files, there are three things we can do with HTTP itself.
- Persistent Connections reduce the need to open a new connection for each and every HTTP request.
- Gzip Compression reduces the size of data before it’s sent across the network.
- HTTP Caching techniques help avoid requests and make use of resources that have already been sent across the network.
I’ll talk about the first two today and save caching for next week.
Persistent HTTP Connections Using Keep-Alive
To understand how a persistent connection will help performance, let’s consider what happens with an HTTP request without a persistent connection.
First a client, say a browser, opens a new connection to a web server and they performa what’s called a TCP handshake. The browser then requests an HTML file using the connection it just opened. When the file has been sent and received the connection is terminated.
The browser checks the HTML file for other resources, like CSS and Javascript files or images and other media, that are necessary for the page to load as intended. One at a time your browser will open a new connection to the server, request and receive a file, close the connection, open another connection, request and receive a different file, and so on.
That’s a lot of connections for most of today’s websites. I think you’d agree if we can make multiple requests using the same TCP connection, that would be a better solution. This is where the Keep-Alive HTTP header comes in.
Enabling Keep-Alive creates a persistent connection. It allows the same TCP connection to be used for an HTTP conversation instead of requiring a new connection with every new request. That means you can download more than one file at a time instead of needing a new connection for each file requested.
Fewer connections reduces both network latency as well as CPU usage as TCP connections use a lot of server resources.
Odds are Keep-Alive is enabled by default on your server as it was on mine, but you can check here. If not, you’ll want to enable it.
The way to do that is through the HTTP Connection header and you want it set to Connection: Keep-Alive. How you do that depends on your server software and the type of access you have.
Since the odds say your site currently runs on Apache, here’s how you can enable Keep-Alive in an .htacess file.
[code type=html]
Header set Connection keep-alive
[/code]
Here’s another way for Apache 2.4.
[code type=html]
SetEnv KeepAlive On
[/code]
Note the two different Apache mods being used. You’ll probably opt for the first way using mod_headers.c, but I wanted to present both options.
There are two additional headers you probably want to set or modify, MaxKeepAliveRequests and KeepAliveTimeout. The former sets the maximum number of requests for each Keep-Alive connection. The more resources your HTML file requires, the larger (within reason) you want this to be. A value of 100 is probably good enough for most sites.
KeepAliveTimeout tells the server how long to wait for another request before closing the connection. The default is 15 seconds, which is probably too high. A higher setting means your server is waiting on every open connection for longer amounts of time, which requires greater resources. On the other hand, if you set the value of KeepAliveTimeout too low, it defeats the purpose of Keep-Alive as the connection closes too fast.
You can set both in your .htaccess file in the same place you enable Keep-Alive.
[code type=html]
Header set Connection keep-alive
Header set Keep-Alive timeout=5,max=100
[/code]
or
[code type=html]
SetEnv KeepAlive On
SetEnv KeepAliveTimeout 5
SetEnv MaxKeepAliveRequests 100
[/code]
If you have access to the Apache config file, you can also set the values there. These would generally be included inside a <VirtualHost> directive.
[code type=html]
KeepAlive On
MaxKeepAliveRequests 100
KeepAliveTimeout 5
[/code]
Since I’m not currently familiar with other server software I won’t attempt to show you how to set Keep-Alive on them. However, do know you can set these values outside of Apache. Nginix uses a keepalive_disable directive to disable Keep-Alive and you would want to remove it to enable Keep-Alive, if present.
Litespeed servers also have keep-alive enabled by default, but it may also default to smart keep-alive, which you may or may not want depending on how much traffic you receive.
Either of the two articles below will give you more information for enabling Keep-Alive in Apache and NGinx. The first article also mentions Litespeed and the second article mentions IIS servers.
HTTP Compression with GZip
Before sending data over the network it makes sense to compress it so that what’s delivered is as small in file weight as possible. Gzip is the most common way to compress file data before sending it from server to client.
I’ve seen several sites suggest GZip can save up to 70% of file weight, which is quite a bit. It’s a good idea to have GZip turned on. Like Keep-Alive it probably is, but you can check either of these sites to make sure. I’m glad I checked, because I didn’t have GZip on even though I thought I did.
On Apache you can enable GZip in your .htaccess file using either mod_gzip in Apache 1.3 or mod_deflate in Apache 2.0. You should opt to use mod_deflate if you have the option.
[code type=html]
# BEGIN GZIP
AddOutputFilterByType DEFLATE text/text text/html text/plain text/xml text/css application/x-javascript application/javascript
# END GZIP
[/code]
You can also write each file type on a different line.
[code type=html]
# BEGIN GZIP
AddOutputFilterByType DEFLATE text/text
AddOutputFilterByType DEFLATE text/html
AddOutputFilterByType DEFLATE text/plain
AddOutputFilterByType DEFLATE text/xml
AddOutputFilterByType DEFLATE text/css
AddOutputFilterByType DEFLATE application/x-javascript
AddOutputFilterByType DEFLATE application/javascript
# END GZIP
[/code]
Note that these are not the only file types you may want to compress.
To enable Gzip on NGinx you can add the following code to your nginx.conf file.
[code type=html]
gzip on;
gzip_disable "MSIE [1-6]\.(?!.*SV1)";
gzip_vary on;
gzip_types text/plain text/css text/javascript image/svg+xml image/x-icon application/javascript application/x-javascript;
[/code]
Again as I’m not too familiar with Nginx, I’ll point you to some other resources for better information.
- Module ngx_http_gzip_module
- Enabling Gzip on Nginx servers (including Laravel Forge)
- Enable GZIP Compression on nginx Servers
Closing Thoughts
HTTP/1.1 has been a pretty good protocol, which is why it’s been in use for the past 20 years. However, it does have a few performance issues in that it requires many round trips between client and server and it repeats information for each request. Its head-of-line blocking also means that one request must be completed in full before the next one can begin.
Optimizing HTTP/1.1 for performance involves sending fewer requests and minimizing the amount of data sent across the network. Persistent connections can help with the former and data compression can help with the latter.
Next week I want to look at another way to improve performance over HTTP/1.1. I’ll talk about HTTP caching and how we can avoid some requests and use local cache to serve resources instead.
Download a free sample from my book, Design Fundamentals.