I’m still digesting the TVNewser.com story pointing out a dispute FNC has with Nielsen over the size of its DVR panel. Some very interesting tidbits in the story:
Another complaint is the small sample size when it comes to calculating viewership. Nielsen spokesperson Gary Holmes tells TVNewser that 14,000 households make up the national sample for Nielsen. Because “25% of the population has DVR,” Holmes estimates that 25% of the Nielsen sample is DVR-capable as well, putting the figure at approximately 3,500 households. “3,500 is an adequate sample size,” says Holmes. “3,500 is not too small.”
Cheatwood disagrees. The 3,500 sample size equates to approximately 65-100 DVR households watching cable news on a given night, and as few as 8-10 households making up the time-shifted ratings, says Cheatwood. “The sample is so small it just defies any application of credibility,” says Cheatwood. “It was one piece of information that confirmed our greatest fears. It just doesn’t pass the test at all. First be transparent, secondly, please, so much rides on the info you put out there, please take the time to make it credible.”
I’m sure I’ll have more thoughts on this tomorrow after the World Series and Monday Night Football. Bill might have some sooner. Click on over to the full article on TVNewser.com.
Update from Bill: We were getting both Live and Live+SD cable news data last fall (but now get just Live+SD), and have lots of charts/tables from about a year ago with O’Reilly and Olbermann Live and Live+SD numbers for comparison. While I cannot be certain of the Live numbers today, a year ago a tiny % of O’Reilly’s numbers were Same Day DVR viewing (<5%). Olbermann had a far higher (10-20%) typical Same Day DVR viewership. That could be explained by the younger skew of Olbermann’s audience and also by the fact that O’Reilly was (and still is) rerun at 11pm each weekday. A year ago, I don’t believe Olbermann was rerun at 10pm like he is now, but I’d love for a commenter who knows for sure to let me know. Today, because Olbermann is rerun at 10pm, his share of Same Day DVR viewers may be smaller.
Definitions:
Time Shifted Viewing – Program ratings for national sources are produced in three streams of data – Live, Live+Same Day (Live+SD) and Live+7 Day. Time shifted figures account for incremental viewing that takes place with DVRs which are currently in approximately 24.4% of all U.S. TV households. Live+Same Day (Live+SD) include viewing during the same broadcast day as the original telecast, with a cut-off of 3:00AM local time when meters transmit daily viewing to Nielsen for processing. Live+7 Day ratings include incremental viewing that takes place during the 7 days following a telecast.
First of all, Nielsen is planning on increasing to a sample size of 37,000 HHs by 2011, so they are working on this already. Second, why do they care so much about DVR? Do cable nets have more to gain from from DVR viewing? And lastly, could it possibly just be that 1) with older viewers, you’re going to get less DVRs and 2) most people don’t find it necessary to DVR the news?
I think it’s mostly the politics of PR — and in this case isn’t about the news itself but about the “opinion” shows like O’Reilly and Olbermann (apparently the whole thing started because of something O’Reilly said on air). I further suspect that KO has more benefit from the LIVE+SD numbers as a percentage increase than O’Reilly does (and what you suggest about age probably plays into that).
But I am just speculating. I think it’s a tempest in a teapot but something about it not being a big enough sample at the level of 8-10 homes does resonate with me and I need to think about it more. my interest here isn’t with the news channels, but rather so many shows, especially on cable that have ratings roughly the size of some of the shows on the cable news channels.
Robert, I routinely defend Nielsen and their sampling, I think it is reasonably accurate for the major nets. However, I do think the sample size can cause questions and errors for the small cable nets. With 200+ cable options, its seems possible that the ratings for some shows could come from one or two people.
Which is probably why Nielsen has been increasing the sample size so steadily. The numbers I have are:
2002: 5,000 HHs
2008: 12,000 HHs (this is probably last season, since this article says 14,000)
2011: 37,000 HHs
And it was 2,500 before they increased to 5,000.
To be clear, I’m talking about measuring viewers for the really small nets only available with the highest priced packages, not major cable nets like TNT.
Julia, I’m not really arguing against Nielsen’s and I know they are increasing their sample size precisely to address this. I’m just not entirely sure that the system last year or this year would accurately measure the viewership for a show airing at 2 in the afternoon on “Fine Living” or “Speed Network”.
You’re probably right. But I do wonder if there’s any way to really accurately measure them.
Probably not without having an insanely large sample size that would be cost prohibitive and produce too much data.
Holly…Nielsen already uses an insanely large sample size on the more profitable sides of their business. There might be 40,000 or more scanners in HH’s to register product usage, but the TV portion is a disaster. Theya nnounced yesterday that they donot have a handle on Universe estimates for second tier cable nets. They are having a really hard time with the explosion of tiers in both cable and sat. And their time shifted data is wrought with errors that can be explained away, but they are still errors. Also, the wild swings in ratings from live to live +SD to live +7 to C3 commercial ratings ae alos problematic. Nilesen is trying to do everything that is aasked of them, but they cannot accomplish half of what is actually needed.
BTW, someone asked a couple of posts ago about Olbermann and a repeat. He has at times over the last year had a live 8pm show and a repeat at 11. then MSNBC would periodically go to their crime shows at 11, and then they changed back and now KO is at 1000pm on the repeat.
In effect what Cheatwood is saying is not that the sample is too small but that the Fox News audience is too small for the ratings to be credible.
Interesting position for a FNC exec to take.
I find it interesting that the more highly-rated cable new network is the net in which the questioning originated here.
In any case, Nielsen does have far too small a sample size to accurately measure audience fractionation. Reference: Charter Cable Company’s set-top-box data and “The Jade Channel” in the L.A. area. I don’t have a link to the article handy as it was a few years ago. The point was that, when STB data was used, The Jade Channel was among the top 10 most-viewed channels on Charter Cable in L.A. However, due to Nielsen’s small sample size, The Jade Channel wasn’t even factored into analysis.
I really like what TNS and DirecTV are doing in the DirecTView system; they have 100,000 households in a market of 17.1 million. That’s a pretty good sample size, and a pretty good way to gage what’s happening on the smaller cable/satellite and “exclusive” channels. I just wish they were more open to the public with their results.