The whole point of free speech is not to make ideas exempt from criticism but to expose them to it.

Thursday, September 17, 2009

Don't bank on the IBD/TIPP Doctors poll, people.

Nate Silver spells it out pretty clearly. The same pollsters who had McCain beating Obama in the YOUTH VOTE 74%-22% are also responsible in the Investors Business Daily poll. The long and short of it is, the poll may not just be an "outlier," it looks like it's probably an "out and out liar." Tsk. This isn't going to set well with my pals over at SDWC, I'm afraid. They were just so sure they were onto something with the Docs not liking Obama's ideas on health care. Sorry fellas.

Excerpt:

1. The survey was conducted by mail, which is unusual. The only other mail-based poll that I'm aware of is that conducted by the Columbus Dispatch, which was associated with an average error of about 7 percentage points -- the highest of any pollster that we tested.

2. At least one of the questions is blatantly biased: "Do you believe the government can cover 47 million more people and it will cost less money and th quality of care will be better?". Holy run-on-sentence, Batman? A pollster who asks a question like this one is not intending to be objective.

3. As we
learned during the Presidntial campaign -- when, among other things, they had John McCain winning the youth vote 74-22 -- the IBD/TIPP polling operation has literally no idea what they're doing. I mean, literally none. For example, I don't trust IBD/TIPP to have competently selected anything resembling a random panel, which is harder to do than you'd think.

4. They say, somewhat ambiguously: "Responses are still coming in." This is also highly unorthodox. Professional pollsters generally do not report results before the survey period is compete.

5. There is virtually no disclosure about methodology. For example, IBD doesn't bother to define the term "practicing physician", which could mean almost anything. Nor do they explain how their randomization procedure worked, provide the entire question battery, or anything like that.


4 comments:

Michael Sanborn said...

This is why all polls are suspect. There's no reason to believe this one.

Bill Fleming said...

Mike there are good polls and bad polls. This is a bad one, for the reason Nate Silver explains.

You remember who Nate Silver is, right?

He predicted the Obama win to 49 of 50 states, based on his custom composite model of all polls in the fields and his unique ability to rank and weigh their results.

Also if you ever want to bet on sports, ask Nate first.

http://en.wikipedia.org/wiki/Nate_Silver

Michael Sanborn said...

After my beloved Wildcats from Kansas State lost to Louisiana Lafayette, I'm not bettin' on nothin' for a while!

Taunia Adams said...

Nice work, BF.

Remind me to bury all my deep dark secrets a little deeper and not to piss you off.