News reading metrics: One big game of “telephone” and average errors

I am building a presentation on mobile metrics, so most any data-related tidbit catches my eye. This from Mathew Ingram crossed my stream on Saturday:

The post linked to and lead me to re-read a piece by Google’s Hal Varian from 2010. I was shocked to be reminded how little time people spend reading news online:

…visitors to online newspaper sites don’t spend a lot of time there. The average amount of time looking at online news is about 70 seconds a day, while the average amount of time spent reading the physical newspaper is about 25 minutes a day.

But as I tweeted last week:

Basically, digital metrics are as much art as science. However, print metrics have been around a lot longer and are even more ‘artsy.’ Not to mention, never trust anything that reports an ‘average’ without explaining the details.

So, where does the ’70 seconds vs 25 minutes’ data come from?

The Slideshare deck from Varian’s talk includes a bit more detail on the digital numbers:

Online for month of June 2009

Unique audience: 70 million

Web page views: 3.2 billion

Sessions: 600 million

Pages per person: 49

Sessions per person: 8.5

Time per person: 38 min

Somewhat surprisingly, one of Varian’s sources for the statistics is a story by Martin Langeveld on NiemanLab.org in 2009. Langeveld was making a specific point (as evidenced by the headline Print is still king: Only 3 percent of newspaper reading happens online) that despite the hype around digital – a lot of people still like print. Fair enough.

Luckily, he does provide some context on the print readership numbers:

How much time was spent with printed newspapers?  NAA doesn’t offer a study providing an average, nor can I find one elsewhere, but I’m going to use 25 minutes Monday-Saturday and 35 minutes on Sunday. Multiplying this out, we get 96.5 billion minutes per month spent with printed newspapers.

In the comments and a postscript, Langeveld points out that the numbers can be disputed. But he links out to a 2006 report on the Newspaper Association of America website to support the print time-spent estimates.

Now that we have a source, the problem is a bit clearer. The NAA data is the result of field work by Scarborough Reseach done in late 2005 using a 23-minute questionnaire of 4,594 respondents. Seventy-four percent were regular (once weekly minimum) newspaper readers, 26% were not.

And that is the catch. The study (as far as I can tell from the PowerPoint) did not measure daily readership – it analyzed weekly engagement. So, among the 74% of respondents who read the paper at least ONCE in the previous week, 45 percent spent more than 30 minutes and 21 percent spent as little as one minute during their most recent session.

Among 4,594 repondents, 3,400 read a newspaper in the past 7 days
Among readers:
1,530 spent 30 minutes or more
1,156 spent 16-30 minutes
714 spent between 1 and 15 minutes

Since we have no idea how often these 3,400 people from 2005 read the paper we have quite a range of “time-spent” per day. The calculations are largely pointless, but we can easily generate averages from 2.5 to 40 minutes daily. A realistic range is anywhere from 4 to 30 minutes per day – among those who read newspapers regularly.

So while the original research methods were legitimate, they were not intended as a measurement of monthly time-spent.  And even as far as their intended purpose, the NAA numbers are still just a guess. There really is no way to track how many people (the pass-along rate is important) read the printed paper, how many pages they read, how many stories on each page they read, and how long they spend reading each story.

Plus, that was 2005.

So, lesson learned. Readership metrics are open for interpretation. And once a rumor gets started like an old game of telephone: from Scarborough in 2005 to NAA in 2006 to NiemanLab in 2009 to Hal Varian in 2010 to a random blog post in 2012 – the original meaning gets a bit lost.

Of course, a fair question might be – does using old/unintended numbers really change anything in the Web’s favor readership-wise? Not really. Print is still very popular and the news media possess only a small portion of the digital audience. But more importantly, we still have a three percent problem. That challenge has more to do with competing against Google and Facebook for attention, not against print for advertising revenues.

To end on a happier note, when you are measuring Web traffic, and reporting averages — you are going to end up with very low averages compared to actual reader behavior. That’s because 40-50% of visits are probably from fly-by consumers arriving via a Twitter or Facebook or Google. They are not your customers, and while they drive up page view and unique visitor numbers, they drive down average time-spent and ad click-through-rates.

So, if the NAA/Scarborough print engagement study can filter out non-customers why don’t we do the same on the Web? Go check out the numbers for your loyal visitors (10+ visits per month) and see what they look like. Of course, unless you are Facebook you are not going to end up with 30 minute reading sessions on average, but then in reality neither does print.