Our Thoughts

Performance benchmarking is one of the main areas we are focusing on at TNG in 2012. The value of a benchmark for determining program growth and success is immeasurable. Not only are performance benchmarks good for measuring where you have been, they are also critical to determining where you want to be. Setting yourself a random 5% growth goal is not so great if you've had 10% growth for the past 24 months. Benchmarks help us imagine realistically where we can go based on where we have been.

One hard part about performance monitoring is knowing when performance is unusual—either good or bad. If you have 50,000 visitors to your website in March, is that normal? You might rightfully ask what the benchmark is. What if the benchmark is 40,000 visitors a month? Then a number like 50,000 looks pretty good. That's 25% growth in one month! However, even comparing to the benchmark could be misleading because we are not taking variability into account.

Variability describes how spread out or closely clustered a set of data is. When you look at variability in website traffic, you might find that twice in the last year your website had more than 50,000 visitors in a single month. This puts things in a little better perspective. However, it's not always a good idea to compare against high and low values—these can be misleading especially in low-volume situations.

When doing performance benchmarking, standard deviation is a good way to balance the highs and lows against the average and come up with a "normal" range. When we apply standard deviation to website traffic, we are looking for spikes and dips that fall outside the normal range. When you find these exceptional data points, you can make recommendations to investigate what was going on.

Showing Standard Deviation in Performance Charts

Overlaying standard deviation in a chart is very handy for presentations and pretty easy to do. Here is one method that takes less than five minutes.

1. Export the past 12 months of visitor data into Excel.
2. Create a new column named "Average." Average the number of visitors, then paste the value into each month.
3. Create a new column named "Std. Dev." Find the standard deviation of the visitors, and then paste the value into each month.
4. Create a "High" column. Add the standard deviation number to the average. Copy this formula for each month.
5. Create a "Low" column. Subtract the standard deviation number from the average. Copy this formula for each month.
6. Select it all and insert an overlapping line chart.
7. View "Select Data" from the chart tools and remove everything except the High, Low and Visitors series.

Here you see that even though this website has low monthly traffic, we can tell what is normal and where certain months have been exceptionally good and others have been lower than normal.

I like to clean up the default Excel charts and add high/low/current data labels.

If you want to account for seasonality and do "same time last year" comparisons, then make sure you rearrange your data so the correct month numbers are lined up.

Now, even thought the benchmark may be set for the average—855 in this case—we can see that a "normal" range for traffic is really anywhere between 625 and 1,084. The two-month decline from October's high numbers may be a bummer, but it is really just an adjustment back to normal from an exceptionally high month.

If we take it one step further and put in a linear trend line, we see that the visitor trend is improving significantly. In fact, the trend is strongly toward the high end of normal—a very good sign indeed. December traffic can then be understood as in the normal range, but below the current trend.

Benchmarking with standard deviation is a great way to enhance the usefulness of performance benchmarks. Once you know what KPIs you need to track and what performance benchmarks you need set, try using standard deviation.