Performance Testing—Tools And Waterfall Diagrams

Not all performance improvements are equal. If your CSS file loads in 100ms, but your images take 20 seconds to load, it makes more sense to focus on the images before trying to squeeze extra performance out of an already performant CSS file. In order to know where to spend your time to get the most in return for your effort, you need to know how your site is currently performing.

Last week I talked about performance budgets as part of a first series on improving site performance. I began with an introduction and followed that by talking about all the steps that take place from the moment you request a web page until it loads in your browser.

Today I want to look at performance testing, specifically through WebPageTest.org. I’ll check the home page of this site and talk about the results and I’ll talk about some of the things WebPageTest reports.

I also want to walk you through reading the waterfall diagrams that most performance testing tools present. Hopefully by the end of this post, you’ll have a better sense of which are the metrics you want to pay attention to and use to set performance goals for your site.

Testing with WebPageTest

I’ll link to some other sources of performance testing at the end this post, but for most of it I want to focus on the WebPageTest.org site. The site is an open source project with Google as the primary developer.

WebPageTest offers one metric, SpeedIndex, that you can’t get anywhere else. I’ve mentioned SpeedIndex a few times in this series and I spent some time talking about what it is last week. You can think of it as a way to measure the perceived performance of a web page.

If you navigate to the WebPageTest site you’ll find a form like the one in the image above. You can type whatever URL you want to test and then click the button to start the test. It’s what I did to generate the results I’ll use in this post. However, note that you can make some changes to the defaults. You can change the test location and the browser to test how your site performs around the world.

If you click open the Advanced Settings (see image above) you’ll be able to tweak additional options and configure the test how you like. You’ll probably leave most at their default, but I’ll point out a couple of things that might be useful to explore.

On the Test Settings tab you can test against different types of connections. The default is a cable modem, but you can change it to test against DSL, FIOS, and mobile connections like LTE and 3G. The Chrome tab will let you test using a few different mobile browsers, though the selection is somewhat limited.

Again, I stuck to the defaults for this post. I typed in the URL for the home page of this site and clicked the button. If you do similar when the test finishes you’ll be presented with a couple of thumbnail images, one leading to results for the initial view and one leading to results for repeat views where some resources will already be loaded into cache. The results I’m presenting here are those for the initial view test.

Results from WebPageTest

Before I get to the waterfall diagram, let me point out a few things at the top of the results page (see the image above this section). First you’ll notice six letter grades in the top right. Two Fs and a C suggest I have some work to do. Each is something I should, no doubt, fix or improve, though none are probably part of any performance budget you might set.

Below the grades are a series of times for things like the load time, the time to first byte, and the time to start render. These are milestone timings that possibly and probably are a part of your performance budget.

SpeedIndex is one of the metrics among this group and you may remember from last week’s post it makes for a good metric to measure the overall perception of the performance of your page.

These timings are numbers you’ll want to improve, but you need more details to know how best to improve them. The waterfall diagram will show the details.

How to Read Waterfall Diagrams

First what is a waterfall diagram, now that I’ve mentioned them a few times? Waterfall diagrams are graphical views, like the one above, of the resources that are loaded by a web browser in order to render the web page being tested.

The diagram presents each resource in the order it loads, along with the time each took to load and also, in the case of Javascript, execute. It gives you a graphical view to make it easier to see which resources are causing performance bottlenecks.

Each row in the diagram represents a different resource or unique request made by the browser. One obvious way to improve performance is to make fewer requests. You can see my home page required 38 different resources to completely load, which seems like more than it should need. In time, I’ll remove unnecessary resources and look to combine the others into fewer files.

The horizontal bars on the graph are color coded with the keys directly above the diagram. Notice that the two-tone colors are for any resource that needs to be downloaded. The first tone shows the time to first byte to download the resource and the second tone shows how much time it took to finish. The solid colors are for connections and the execution of code.

Take a look at the first row. There’s a bit of teal for the DNS lookup, some orange as my browser makes a connection with the server, and then a lot of blue as my browser downloads the HTML for the page. The total for all three is 690ms.

The rows below the first one are for various CSS files to download followed by quite a few Javascript files and then some images. Finally there are a few 3rd party Javascript files that need to download.

In addition to all the horizontal bars, you might notice there are some vertical lines, with keys for these lines above the graph. For example, the thick pink vertical line shows the time for DOM Content Loaded. Here are definitions for each of the six milestones.

start render indicates when content starts to display and a visitor sees more than a blank screen.
msFirstPaint is when the browser thinks it started the paint process, which could still be a blank screen to the visitor.
DOM Interactive is when the DOM begins to process user events for the document (clicking a Javascript button, for example).
DOM Content Loaded is when the initial HTML document has been completely loaded and parsed, without waiting for stylesheets, images, and subframes to finish loading.
On Load is the time from the start of the initial navigation until the beginning of the window load event.
Document Complete is the time it takes for most resources to render in the browser. It’s the point where the webpage has loaded all the initial components of the HTML DOM and you can start interacting with the page, though images may still need to load.

The last of these (Document Complete) is more important to developers than visitors to the page, but each of the others is a metric you may want to pay attention to depending on the specifics of your page.

One of the nice things about the way these milestones are shown is once you know where the vertical line is for one of them, you can see all the resources that had to load before the page started to render or the DOM content loaded. If you want to improve one of the milestones, it makes sense to focus on the resources to the left of the vertical line showing that milestone.

How Waterfall Charts Suggest Performance Improvements

Using the information presented in the waterfall diagram gives you three basic approaches to optimizing performance.

Reduce the number of requests.
Reduce the time it takes to download resources.
Optimize the order of resources to get to start render sooner.

The first two approaches are probably obvious. If you see a long vertical waterfall chart, you probably make more requests than necessary on the page. If any resources shows long horizontal bars, then that resource likely needs a reduction in file weight.

The reason the order is important is because some resources are visually necessary and some aren’t. For example, some javascript files that load on my home page are to help the shopping cart and contact form function.

That doesn’t make sense as you can’t directly enter the cart or leave a comment on the home page. The images have to wait for the javascript to download before they can load so I should probably move the scripts to the end of the HTML file instead of the beginning or better yet not include them on the home page.

The colors on the chart can also provide quick suggestions about what you need to do. For example, the orange in the diagram shows the TCP connection. Only the first few resources should need to make a connection. If more require a connection you probably aren’t using keep-alive to keep the connections persistent.

Another example could be too much time to get to the first byte of each resource. You would see this with too much of the lighter tones in these results. This could be an indication of latency problems, which might mean it’s time to look for a new host or perhaps think about using a CDN (Content Delivery Network)

One last point before I show you to some different tools. The color conventions on WebPageTest are conventions only on that site. There’s no reason that orange has to represent the connection or blue the HTML. Understand that different tools use their own color schemes. The metrics will be the same, but the colors used to show them may be different.

Tools and Resources

Here’s a list of tools you can use to get waterfall diagram results starting with WebPageTest that I used for the results shown in this post. Some let you run online tests and some offer tools you can download.

This is an interesting and definitely simple tool created by Lea Verou. It does something different than the other tools here. You enter URLs for two different pages and watch as they both load. It’s a good test of perceived performance.

These four sites offer monitoring services so you can keep track of the performance of your site. GTmetrix also lets you run free tests.

Finally this last group contains articles about reading waterfall diagrams. I read through all of them while researching for this series. There’s certainly overlap in all of them, but each offers some unique thoughts that can help you understand waterfall diagrams more.

Closing Thoughts

Writing this brief series has helped make a few things about site performance clearer to me. When I’ve looked into performance the last few years I could easily find testing tools and checklists of recommendations, but not as much in the way of explanation for why I should make any of the changes or what the results of the tests really meant.

Understanding more about the critical path and performance budgets helped clarify for me what I should be looking for in waterfall diagrams. They’ve given me a better sense of what the data is telling me and helped make waterfall diagrams more accessible.

I hope the series has helped you as well.

That sets the landscape. I’ll come back to this series in a few weeks and talk about actually making performance improvements.

« Prev PostNext Post »

Download a free sample from my book, Design Fundamentals.