Poll Discrepancies

I noted something interesting in an Associated Press piece that appeared on the MSNBC website.

The piece reports widely different, perhaps I should say even conflicting, results achieved by pollsters on the question of whether or not American citizens continue to support our nation's war efforts in Afghanistan.

The first poll, by the Pew Research Center, was conducted May 18 to June 16, with a margin of error in most countries of 3 to 4 percentage points. (Please note the "most countries" tag is due to this poll actually involving reports of public opinion within several nations.) The results? 57 percent of American respondents favored keeping U.S. troops in Afghanistan while 38 percent said they should be withdrawn.

The second poll, by AP-Gfk, An AP-GfK poll, was conducted July 16-20, with a margin of error of plus or minus 3.1 percentage points. The results? 44 percent favor the war and 53 percent oppose it.

So in a period of just two months, these polls report a shift away from firm majority support to a slim plurality of opposition. Even if one takes the margins of error into account, this reported change is not explained.

There was the time period difference of when the sampling was taken. If one looks at when the polls were started, this difference is as large as two months. Has anything been happening during that time that would cause a wild swing of public opinion? There has been the relentless reporting of casualties in Afghanistan, with the death toll rapidly rising. As the Associated Press piece reports:

The new U.S. emphasis on Afghanistan has raised the level of fighting — and in turn, the number of casualties. July is already the deadliest month of the war for both U.S. and NATO forces with 63 international troops killed, including 35 Americans and 19 Britons.

However during this time political leadership within America has been speaking out in support of our war efforts in Afghanistan with little political opposition coming from either of the two major parties.

Now I understand that within two months, some people are going to change their mind. In all wars, over time, support tends to slip as the public starts to become war weary. But we have already been at this war for some time, and I do not think the passage of an additional two months alone explains thing.

I understand that polls can constructed so as to be push polls. The polling questions can be constructed so that they shape your opinion even as you are asked to give your opinion. However these types polls normally are being conducted by organizations who have something to gain by a certain result. I am not aware of the two organizations which conducted the above polls having ulterior motives on the above issues or for having the reputation for engaging in push polls.

I am aware that poll researchers report greater difficulty in polling due to changes within society. As more and more citizens shift to cell phones, it is getting harder and harder for pollsters to achieve representative samples of society. I know that it is extremely rare for my own family members to be consulted for our opinions on polls. Perhaps the explanation in our case is that we use an answering machine to screen our calls even though our family remains firmly dependent on a land line.

Whatever the explanation for the disparate results, I think these results point to at least one thing. The method of coming up with the claimed margin of error on polls needs to be adjusted. While I am not an expert on statistics, I believe I understand the justification for the claimed margin of error. As long as your sampling size is large enough, it should become increasingly statistically unlikely the sampling is wrong as the number sampled increases.

Perhaps the formula for determining margins of error was arrived at by analyzing poll results taken back in the good ole days. Back before cell phones and when it was still highly unlikely for citizens to even have an answering machine to screen calls. Back in those days, pollsters were probably more likely to be rewarded with a representative sample of society for their efforts. Nowadays, because along with advances in technology making it more difficult to contact many segments of society, citizens have become jaded with the advent of large scale telemarketing which even the "Do not call list" hasn't solved. I would imagine that back in ancient times, people were more willing to engage with a pollster and perhaps even flattered that their opinion was being asked for. Nowadays, citizens are more apt to be skeptical. When someone asks for their opinion, the defenses go up with the question "OK, what are you trying to sell me?" being in the back of their head.

I am at least more wary of believing the results of polls when I hear them. The results from the above two polls helps to explain why.