Comments on: Early Ratings: Sunday Night Football, Desperate Housewives are Winners

By: clutz12001

clutz12001 — Tue, 07 Oct 2008 01:01:12 +0000

Holly,
Nielsen is the only game in town, for now, and they’re doing their best regarding accuracy. As for who pays for what, if I were an advertiser, I’d demand that Nielsen provide gage R&R evidence before I’d agree to pay them for anything. This all might exist already, just not free to the public. Still, I have to prove the validity of my research in my career (chemist) – why shouldn’t they? And Sweeps simply provides a bigger collective sample using different equipment (diaries). It’s not five repeated identical studies, compared for accuracy. As for discrepancies in diaries versus metered markets – they do exist. They’re the reason for the zero-cell phenomena.

Back in April advertising firm Starcom/MediaVest signed on with the set-top box data provided by TNS and DirecTV – it’s called DirecTView. Apparently, TNS ran a similar program with Charter Cable. Based on the encouraging results they saw, Starcom bought into DirecTView. DirecTView measures 100,000 households via set-top box data. At this point, it is only measuring DirecTView customers – as the pilot program measured only Charter Cable. Still, to me this proves that advertisers are questioning Nielsen’s accuracy and level of detail. Starcom was in search of something more. Only time will tell if others will follow.

(And no, I was not joking about Friday Night Lights. I’m curious to see if Nielsen can find anyone, anywhere watching that show. I’d love to compare their data to TNS/DirecTView data on the topic. I cannot pay for the data, but I’d bet DirecTV and NBC would!)

By: Holly

Holly — Mon, 06 Oct 2008 21:06:00 +0000

clutz, Basically, the sweeps period does your study for you. It adds 5 times as many subjects into the sample. If there was a significant difference between the results during sweeps between the metered markets and the paper diaries, it would show that there is a problem with the sample. If not, it shows that the information gathered from the sample is accurate.

By: Holly

Holly — Mon, 06 Oct 2008 21:01:48 +0000

Hey look, Nielsen’s answers for us… “We collect information from approximately 25,000 metered households…In addition, we collect and process data from 1.6 million handwritten paper diaries from households across the country during sweep periods.”
http://www.nielsenmedia.com/nc/portal/site/Public/menuitem.55dc65b4a7d5adff3f65936147a062a0/?show=%252FFilters%252FPublic%252Ftop_tv_ratings%252Fcable_tv&vgnextoid=4f5247f8b5264010VgnVCM100000880a260aRCRD&selOneIndex=1

By: Bill Gorman

Bill Gorman — Mon, 06 Oct 2008 20:58:41 +0000

Holly, I did a bit more digging and here’s a release (although almost a year old) showing Nielsen’s people meter roll out schedule and some numbers:

http://tvbythenumbers.com/wp-content/uploads/2008/10/lpm_all_markets_announcement.pdf

By: Bill Gorman

Bill Gorman — Mon, 06 Oct 2008 20:52:53 +0000

Yes Holly, that’s my understanding.

By: Holly

Holly — Mon, 06 Oct 2008 20:40:47 +0000

So they have between 12,000 and 37,000 people meters in addition to however-many diaries in the smaller markets?

By: Bill Gorman

Bill Gorman — Mon, 06 Oct 2008 20:33:42 +0000

Holly, there were 12,000 Nielsen people meters in the US in October, 2007. Since then they were to be installed in three to eight more markets before the end of 2008, but I’m not sure where they are in that process. By the end of 2008 the intended the eighteen largest markets to be completely measured by people meters (as opposed to diaries as in the smaller markets).

Nielsen has announced plans to expandthe total to 37,000 meters by 2011.

By: Holly

Holly — Mon, 06 Oct 2008 19:48:58 +0000

The thing is, the people for whom the ratings REALLY matter (the ad companies and the networks) DO have the money, but haven’t started any studies or a different audience measurement system. If they were really concerned that Neilsen’s was inaccurate, wouldn’t they do something about it?

By: Julia

Julia — Mon, 06 Oct 2008 19:43:19 +0000

Holly, if I had the money, I would be. My dream when I’m a billionaire is to set up an independent audience measurement group to confirm Nielsen’s numbers. Non-profit of course.

By: Holly

Holly — Mon, 06 Oct 2008 19:39:21 +0000

clutz, Are you willing to PAY for that study?

By: Julia

Julia — Mon, 06 Oct 2008 19:37:07 +0000

A simpler way that they can prove accuracy to me – tell me how many people watched the premiere of “Friday Night Lights” on DirecTV’s the101 channel. I am very curious to see how the success of FNL will be gagued.

Um… What? You do mean that as a joke, right?

By: clutz12001

clutz12001 — Mon, 06 Oct 2008 19:34:25 +0000

I’ve yet to see a Nielsen gage R&R study. Such a study would prove their accuracy to me. Nielsen would need triple the sample size they have now, but split into three “identical” groups. The television ratings – nightly, weekly, cable, broadcast, etc… would be run in triplicate for an entire season. The ratings would be calculated independently in each sample. Then, the three samples’ ratings would be compared to one another. If they are all within 5% error, then I would be convinced that Nielsen sampling methodology is accurate.

By: Holly

Holly — Mon, 06 Oct 2008 19:09:04 +0000

I’d also like to add that Neilsens doesn’t just use one constant sample. There may be 5,000 households (I don’t know the real number, so I’m using this arbitrary number for the example) at any given time, but each household only participates for a limited amount of time (used to be two-weeks, but I don’t know if that’s changed). So, while only 5,000 are part of the sample any given day, they have gathered data from many, many more. The fact that you do not see major changes as the sample changes actually indicates that the results are rather accurate. If you saw ratings jump up and down a lot, that could indicate that there was a problem with the sampling.

By: Julia

Julia — Mon, 06 Oct 2008 18:54:39 +0000

Thanks, Holly. Trying to convince non-believers that sampling is accurate is difficult. But I’m one of the converted who used to go on about how inaccurate Nielsen numbers are, so it can happen!

By: Holly

Holly — Mon, 06 Oct 2008 18:38:57 +0000

Gusar, Not necessarily. If you have a base population of 1,000,000 people and you survey a sample of 5,000, you will have a remarkably accurate view of how all 100,000 people would respond. In fact, you would need less than 2,000 people to have results that would be statistically accurate (99% confidence level, +/- 3). Increasing the sample size would improve accuracy, but not significantly.

By: Gusar

Gusar — Mon, 06 Oct 2008 18:20:11 +0000

Julia, that does not comply at all. No matter where and how much blood you take, the content will be the same. But if you sample 5000 people in one region, then switch and sample different 5000 people, the results will be different.

By: Julia

Julia — Mon, 06 Oct 2008 17:19:28 +0000

I’ve been waiting for the accuracy question to come up, too, since I read a quote from A.C. Nielsen, Sr. last week. “f you don’t believe in sampling theory, next time you go to the doctor and he wants to take a little blood, tell him to take it all.”

By: clutz12001

clutz12001 — Mon, 06 Oct 2008 17:12:04 +0000

Johnthemon,

Fans of the now-cancelled CBS drama “Jericho” certainly question not only accuracy of the Nielsen numbers, but the validity of the system as a whole in the on-demand television age.

Nielsen’s sample is somewhere between 5,000 and 25,000 “families,” depending on who you ask. Families have meters attached to their sets, or log their entries into paper diaries. Nielsen does claim they are an accurate representation of the population based on census data. However, I’ve seen no proof of their diaries being verified for accuracy, and their meters have encountered a “zero-cell” phenomenon where data must be assumed. Zero-cell means a TV set is turned on, but no matching personal info has been entered as to who’s watching what.

Jericho was always a top TiVo’d program, top DVR +7 program (in percentage increase over live viewers), and a top iTunes download. Yet it was expected to obtain 10 million Nielsen-caluclated viewers as it aired at 10 p.m., after dismally low-rated “Big Brother.” CBS dramas “Moonlight” and “Shark” fell to the same fate – cancellation based on universally accepted Nielsen ratings.

My apologies for the rant. I haven’t seen the accuracy question in a while, and it was cathartic to vent those old feelings again

By: RViewer

RViewer — Mon, 06 Oct 2008 17:06:49 +0000

The Nielsen system may not be perfect but every tv show are judged by it. Can it count everyone who watch tv? No. What it count is a percentage of the tv audience.

DH lower numbers might be due to the season premiere effect, all shows sees an upsurge then lose viewers the next week, happened to almost all shows this season.

By: chinafan

chinafan — Mon, 06 Oct 2008 16:28:28 +0000

I guess DH got 16 million viewers, still not bad.

By: dennis

dennis — Mon, 06 Oct 2008 16:19:39 +0000

Hmm DH has weirdly low numbers … I guess it pulled only 15 million viewers.

By: Johnthemon

Johnthemon — Mon, 06 Oct 2008 16:10:11 +0000

Doesn’t anyone question the accuracy of the Nielson numbers?