To Get Useful Answers, Ask Correct Questions

It’s all in how you ask the question.

The Judy Ford Wason Center for Public Policy at Christopher Newport University has done a pre-General Assembly poll testing various issues that may dominate the 2019 session.  The headlines are driven by the favorable and unfavorable rankings (ask me about President Tariff Man this morning as I survey my portfolio) and the apparent openness of Virginians to ending restrictions on gambling.

The poll’s authors took a dive into the complex world of tax policy, as well, seeking to tease out how voters view various moves related to the windfall tax conformity revenue.  It could have been more useful.  To start the discussion, here is the question they used:

Q8: Virginia is expected to receive as much as $600 million in additional tax revenue as a result of the recent federal tax reform. There are several ideas about what to do with this additional money. I’m going to read two of them and I’d like you to tell me if you support it or oppose each one.

  1. Provide an across-the-board tax cut to all Virginians who pay state income taxes.
  2. Provide a fully refundable tax credit to low and moderate-income Virginians regardless of how much they pay in state income taxes.
  3. If only one of these options could be done, which one would you most prefer to see done, an across-the-board tax cut to all Virginians who pay state income taxes or a fully refundable tax credit to low and moderate-income Virginians regardless of how much they pay in state income taxes?

The results were ambivalent, with a healthy portion of voters liking either approach.  There were predictable partisan divides, with Republicans preferring the idea of a broad-based cut for all taxpayers and the Democrats leaning towards a tax credit targeted to the lower and middle income.  But 59 percent of Democrats were positive on Q1 and 49 percent of Republicans were positive on Q2.  Forced to choose by Question 3, the partisan divide appeared again.

The problem is those are not the choices, at least based on the discussions so far.

First, missing from the mix was the idea which may yet prevail, taking no steps to return the money.  A fair additional option to give the poll respondents would have been: “Retain the money to increase the state’s financial reserves and spend it on other pressing state priorities.”  Listing especially popular priorities would have pumped up the positives on that question.

That is the biggest and most important question:  does the General Assembly keep the money or give it back?  They didn’t ask it.

Second, nobody has proposed an across the board tax cut to all those who pay the income tax.  We at the Thomas Jefferson Institute have come closest to that, with a proposal to double the standard deduction that might reach 70 percent of taxpayers, and we may tout the Wason result as supporting our case.  But that is not a tax cut for everybody who pays, so they didn’t poll our idea.

Third, I doubt if more than a handful of people polled know what a “fully refundable” tax credit is (or a partially refundable one, for that matter).  It sure sounds nice; everybody loves a refund – the word by itself may inject a bit of question bias.  Which of course is why it has always been used to describe the Earned Income Tax Credit grant payments.

But imagine the answer to this question, which more accurately describes the proposal around the Earned Income Tax Credit: “Provide an annual cash payment to low- and moderate-income Virginians who do not owe any income tax but are still struggling to meet the needs of their families.”

There still would have been positive responses to that, but how many?  Would it have proven to be as popular a choice as a general tax cut?  More popular?  We will never know.   Will we hear repeatedly in the coming weeks that this or that idea has been “strongly supported in a poll”?  Probably.

To borrow a line used about modeling, all polls are wrong, but some polls are useful.  This poll unfortunately is not very useful because it left off the main choice – keep it or give it back – and didn’t really describe the two choices for giving it back getting the most attention.

I do commend the CNU center for releasing the full text, cross-tabs and demographics of their sample.  Absent those, nobody should believe any poll result featured in the media or in campaign materials.

Share this article


(comments below)


(comments below)


9 responses to “To Get Useful Answers, Ask Correct Questions”

  1. Lawrence Hincker Avatar
    Lawrence Hincker

    Agree with you on this one, Steve. Another way to put the question would have been, “Would you agree to a tax increase on one segment of the Virginia population to transfer monies to the low income segment of the state’s population” through a welfare program called the Earned Income Credit.

    1. Steve Haner Avatar
      Steve Haner

      Well, that might have a bit too much question bias in the other direction! 🙂 I did throw in the “struggling families” line, putting a bit of a thumb on the scale. Truly neutral wording is hard to find sometimes.

  2. Great analysis, Steve. The framing of the question can heavily influence the outcome. Asking unbiased poll questions is difficult. One must strike a balance between brevity and acknowledgement of the complexity of issues. I think the Wason Center tries to do an honest job, but unconscious biases can easily creep into the wording — as you have clearly demonstrated.

    It would be make an interesting experiment to conduct a poll framing the question the way Wason did, then poll another group framing it the way you did, and poll yet another group framing it the way Larry H. did. I’d love to see those results!

  3. djrippert Avatar

    They may have missed the mark to some extent in how they framed the question. However, they put forth an important concept – redistribute wealth or not. Apparently, Virginians are divided on this.

  4. The extra money is not just appearing like magic…Virginia is apparently proposing to increase taxes on middle class itemizers.

    I just purchased TurboTax to try to refine my prior estimates (per my Virginia Gothic article) but foiled again, as the Virginia state tax forms are not ready yet.

    1. Steve Haner Avatar
      Steve Haner

      TBill – Yep, I figure this next state return will be done by hand in March or April. I can do the federal and file it electronically earlier. I’m not throwing away my receipts in case you prevail and I get to itemize at the state level still.

      DJ – reading that poll the way a political consultant would, I think a large set of voters were comfortable with either approach (hence: ambivalent.) Neither position is likely to prove fatal in an election, if framed the way you want. Framed the way your opponent would want (note TBill simmering) things might get interesting.

    2. LarrytheG Avatar

      The software companies are stopped in their tracks until actual definitive law is voted.

      And these days – those who think they can do the “paper” are going to find that to be a tedious task.

      What could be done would be for the various players – say like the Thomas Jefferson folks – to provide an online tax calculator that would help taxpayers understand the choices.

      What we’re probably going to get instead is the usual backroom foolishness such that few will actually know the real changes until the tax software is updated which – probably will be weeks/months after General Assembly action.

      I think the window for changes is pretty much gone and trying to rush through changes at the last minute often results in bad software and unintended consequences.

      Like I said – the GOP and Conservatives KNEW in 2017 that changes needed to be made and they waited until now to try to do it. Other states got on it right away. We just goofed off because the folks in the GA and related players are stuck in a mindset of when the GA can do “stuff” …. besides they were busy with Dominion-friendly legislation…keep their profits, no tax rebate, get a slush fund for grid “modernization”… etc.

  5. LarrytheG Avatar

    The other thing to keep in mind about the “credit” is that it’s called “earned income” because you don’t get the credit unless you work.

    For single parent families – where Mom is trying to make a living while raising kids – this money can go to pay for child care, health insurance and retirement – things that can save taxpayers money if Mom decides to stay home and get TANF.

  6. […] weeks ago, I complained that a poll from the Judy Wason Ford Center at Christopher Newport University didn’t ask the […]

Leave a Reply