Writing Good Surveys, Part 3: More Question Basics

Doug Peterson HeadshotPosted By Doug Peterson

In part 2 of this series, we looked at several tips for writing good survey questions. To recap:

  • Make sure to ask the right question so that the question returns the data you actually want.
  • Make sure the question is one the respondent can actually answer, typically being about something they can observe or their own personal feelings, but
    not the thoughts/feelings/intentions of others.
  • Make sure the question doesn’t lead or pressure the respondent towards a certain response.
  • Stay away from jargon.
  • Provide an adequate rating scale. Yes/No or Dislike/Neutral/Like may not provide enough options for the respondent to reply honestly.

In this installment, I’d like to look at two more tips. The first is called “barreling”, and it basically refers to asking two or more questions at once. An example might be “The room was clean and well-lit.” Clearly the survey is trying to uncover the respondent’s opinion about the atmosphere of the training room, but it’s conceivable that the room could have been messy yet well-lit, or clean but dimly lit. This is really two questions:

  • The room was clean.
  • The room was well-lit.

I always look for the words “and” and “or” when I’m writing or reviewing questions. If I see an “and” or an “or”, I immediately check to see if I need to split the question out into multiple questions.

The second tip is to keep your questions as short, as clear, and as concise as possible. Long and complex questions tend to confuse the respondent; they get lost along the way. If a sentence contains several commas, phrases or clauses inserted with dashes – you know, like this – or relative or dependent clauses, which are typically set off by commas and words like “which”, it may need to be broken out into several sentences, or may contain unneeded information that can be deleted. (Did you see what I did there?)

In the next few entries in this series, we’re going to take a look some other topics involved in putting together good surveys. These will include how to construct a rating scale as well as some thoughts about the flow of the survey itself. In the meantime, here are some resources you might want to review:

Problems with Survey Questions” by Patti J. Phillips. This covers much of what we looked at in this and the previous post, with several good examples.
Performance-Focused Smile Sheets” by Will Thalheimer. This is an excellent commentary on writing level 2 and level 3 surveys.
Correcting Four Types of Error in Survey Design” by Patti P. Phillips. In this blog article, Patti give a quick run-down of coverage error, sampling error, response rate error, and measurement error.
Getting the Truth into Worplace Surveys” by Palmer Morrel-Samuels in the February 2002 Harvard Business Review. You have to register to read the entire article, or you can purchase it for $6.95 (registration is free).

If you are interested in authoring best practices, be sure to register for the 2014 Questionmark Users Conference  in San Antonio, Texas March 4 – 7. See you there!

Leave a Reply