Performance at Velocity Conference

A marked change has occurred since the first Velocity Conference a year ago, and while the effects are not yet obvious, they will be. The web is still slow, but we have something now, that we didn’t a year ago: business metrics. This was the year we quantified the impact of performance choices on our businesses, and the results were astounding.

Those of us who worked in the field had a gut feeling that users want a fast web experience, but most of the studies done previously were lacking something, either in experiment design or reliability of the data. They were all strong indicators that more research needed to be done, but they weren’t damning enough to provide real certainty. This year we found a real correlation between websites speed and its ability to establish and keep relationships with visitors. Not everyone could attend, so I’d like to share with you some of the key moments of an amazing conference. Please feel free to add others in the comments.

David Artz at AOL

David Artz from AOL presented their findings from a study which measured page views per visit against performance. They divided users into buckets based on response time and plotted it against PV. The results were startling. Across six AOL sites there was a clear inverse correlation.

The take away: AOL

Users who had a slower experience view far fewer pages.

Goog and Bing sitting in a tree, K-I-S…

Goog and Bing got together (whoa!) to do a study looking at search behavior when performance is worsened over very narrow increments. This study was unique particularly because it followed the same users over a period of time. The data can be used to determine the threshold at which clicks, refined searches, revenue, satisfaction, and time to click are likely to be impacted by features which slow a website. Their methodologies were a bit different, but the conclusions were remarkably similar. A 50MS delay seemed to have no impact, but as little as 200-500MS changed user behavior across the board. Revenue, clicks, and time to first click were most profoundly impacted.

The Take Away: Bing

One key point was that users seem to lose their focus if you make them wait too long. Progressive rendering and flushing the header (which are also recommended by Yahoo!) can help. Bing had this to say:

Notice that as the delays get longer the Time To Click increases at a more extreme rate (1000ms increases by 1900ms). The theory is that the user gets distracted and unengaged in the page. In other words, they’ve lost the user’s full attention and have to get it back.

~ Google & Bing

We’ve all experienced that. We open a new tab and run a search. Multitasking fools that we are, we flip to a new tab or open our email if the results take too long to load.

The Take Away: Google

The most interesting data to come out of the Google tests took place long after the experiment had finished. As much as five weeks later, some users, especially those who saw delays greater than 400MS, were still searching less than before. Performance is a feature users want. Fail them, and they may never come back.

The % change recorded was very small. For instance, a half a second delay caused a -1.2% loss of revenue per user. What does that mean? We need to think big, and simultaneously work on incremental and profound ways to make the web faster.

Shopzilla – Profound improvements

Shopzilla also presented their (profound) performance improvements. They decreased their response time by around 3.5 seconds and the data showed their conversions increased by 7-9% and their page views skyrocketed 25%. This is good stuff. This is how we go to business and make the case that performance is an important feature that deserves attention, not a band-aide that you stick on afterwards. Dave Artz has more details.

JavaScript versus CSS versus Network Latency: Which is killing our sites?

In a separate session with Mike Belche from the Chrome team, he discussed his experiments which tested total time spend on executing JavaScript, rendering the page (CSS), and network latency. He said the vast majority of the time is being spent on network latency. There was a subtle flaw in his methodology, because his rendering time included only one full rendering and painting, because all resources were already in cache and no JavaScript was used, there would be no unnecessary reflows.

The Take Away: Reflows

This got me thinking about images and other fixed dimension media. We should always set height and width of images to avoid reflows being caused when the resource is finally downloaded and available.

I agree with him that, except in extreme cases (and a lot of selector/reflow experiments have been too extreme to really reflect reality), rendering will be much less important that network latency. It is much more important to keep page weight and HTTP requests as low as possible. Over-complicating our CSS selectors to reduce render time would be a mistake. Browsers are really good at parsing selectors, we need to be really good at writing the minimum number we actually need. This is clearly missing not handled correctly in the current suite of testing tools such as Page Speed.

My talk included (not yet released) suggestions for coding performant selectors. More on that later. ;)

Further Reading

• Aladdin Nassir spoke about linking performance and business metrics via Performance-Based Design.
• Lindsey Simon spoke about reflows and an open source tool he is building to better measure these things. The methods for accurately measuring reflows are still a WIP, and the numbers are fuzzy, but that makes this a really interesting project to get involved in.