When it comes to projects that require straightforward input from either small or large cohorts of people, surveys and questionnaires are tough methods to beat. They’re a super efficient way to gather and analyze quantitative and qualitative data, are relatively easy to administer, easily facilitate data analysis, and can be extremely cost-efficient.
There are a few tried and true techniques you should employ as you start to plan and build your survey. As with any information gathering method, surveys also come with challenges. But if you’re aware of them, you can confidently approach your survey knowing you’re doing your best to overcome them and produce data that is insightful and meaningful.
In this article, we’ll take a look at the essentials of surveying and some of the challenges you need to be aware of.
Clarity
Effective communication is the key to survey success. You must ensure that your questions are structured and phrased in a way that is clear and concise. If respondents cannot clearly understand your questions, you risk inaccurate results that will skew your data and insights.
The language you use to build out your questions must be simple and easy to understand. Know your audience and the type of language that they will likely be familiar with. We’ll explore this more in a bit when we discuss how to recognize and avoid response bias.
Avoid using long, complex words and words that may have multiple meanings. Keeping language simple and easy to understand avoids confusion and allows respondents to more accurately answer your questions. This will, ultimately, provide you with rich data and valuable insights.
Strategic Placement of Questions
When building your survey, remember that question structure can make a big difference in response rate and completion. Approach question structure and survey design as if you’re having a conversation with someone. Usually, the conversation starts with lighter questions ( such as “hey, how are you?”) and gradually moves into deeper topics (maybe “what do you plan to do with your career?”, for example).
Close-Ended Questions
Ask close-ended questions at the beginning, as they seem less intimidating and easier to answer. Close-ended questions are composed of pre-populated answers for respondents to select; think of multiple choice and true/false type questions.
Open-Ended Questions
Place any open-ended questions closer to the end of the survey, since these types of questions require respondents to provide input in their own words, and can feel like more effort or seem more daunting.
Personal Questions
Include any personal information questions near the end of the survey as well. Respondents will likely feel more open to answering them since they’ve already spent time answering the survey and providing their input.
Structuring your questions this way can help ease respondents into the survey and encourage them to continue on to completion as they work their way through it.
Participation Incentives
Don’t take this personally, but not everyone will want to complete your survey. I know, the truth hurts. But again, it’s not personal. Think of this as a classic “it’s not you, it’s them” scenario.
People are busy and are caught up in their own tasks and day-to-day roles. Your survey – while necessary and likely there to help them, in the long run – just isn’t a priority for many. But you can help offer a little encouragement to your respondents.
Offering incentives is a common way to increase survey responses and boost your respondent population. But be strategic with your incentives. Offer one that is enticing enough to elicit responses and completions, but not so impressive that it causes a response bias. Too big an incentive can lead to respondents wanting to be agreeable and completing your survey with answers that aren’t completely honest. This is called demand characteristics bias, and it can happen when respondents try to figure out the goal of your survey, and want to provide you with “the right” answers.
Recognizing and Avoiding Survey Bias
One of the biggest challenges in surveying is recognizing and overcoming bias. While surveys are a fantastic way to collect data, especially for a large population of people, the integrity of the data they produce relies on honesty in respondents’ answers. Response bias occurs when respondents don’t tell you the full truth when answering questions in surveys.
There are a number of reasons why survey bias occurs. Sometimes it is the respondent’s fault:
- They want to be a “good participant” and provide the “correct” answers for your survey goal.
- They may experience response fatigue and just check off answers that don’t really reflect how they feel or what they think.
- If respondents become bored with your survey, they may select answers just to get through it.
But most often, response bias occurs as a result of something the surveyor did – or didn’t do!
- The questions or answers may have been phrased unclearly or be too complex.
- The wording of the questions may be leading and cause respondents to choose a certain answer.
- The survey design or format may be confusing or not engaging.
- Sometimes even the surveyor’s demeanor can influence bias.
Here are a few ways that you can help avoid response bias in your surveys and stand a better chance of gathering accurate results that provide valuable data and insight.
Phrase Questions Carefully
Be clear in the way that you phrase your questions, answers, and instructions, and don’t make assumptions about respondents. Ensure that you are asking qualifying questions and pointed follow-up questions when necessary.
For example, if you’re conducting a survey about which snack foods to stock in the office kitchen, and you’d like to know if chocolate bars should be stocked, and if so, which brands. Qualify with a question that clarifies whether the respondent enjoys chocolate bars or not. Then, if they answer yes, move on to ask which chocolate bars they enjoy. Like this:
Question 1: Do you like chocolate bars? If no, please skip to question 4.
Question 2: Which flavors of chocolate bar do you like? (Choose All That Apply)
This will help you when analyzing data and gathering insights from the audience who has a preference or opinion on specific snack foods in the company kitchen. It will also ensure that people who don’t like chocolate bars (yes, they do exist) are not choosing which flavors to keep stock of in the kitchen.
Provide Enough Answer Options
Ensure that you’re offering enough answer options so respondents aren’t forced to choose something that closely fits what they truly want to answer, but isn’t really an accurate representation of the truth. If you know you cannot include all possible answers, add an “Other” option with an open text field so respondents can type or write in their answer (if appropriate).
Do Your Research
Make sure that you are knowledgeable about your topic and your respondent audience ahead of time. This will give you a better idea about the right topics to cover and the necessary questions that need to be asked.
This also helps you understand the terminology that is appropriate to use when creating survey questions. For example, if you’re conducting a survey to gather insight on the cost-effectiveness of a certain type of software, you likely won’t be distributing the same version of the survey to end-users and developers. The version you provide to end-users will very likely contain less technical terminology in its questions and will more likely focus on the features that they use, ease of use, and other front-end factors. However, with the developer cohort, you may also want to understand how easy the software is to maintain, update, and build upon, and you’ll likely want to ask some more technical questions that wouldn’t be relevant to the end-user group.
Anonymize Your Survey, When Possible
If you can, make your survey anonymous. This drastically increases the likelihood that respondents will be honest when answering your questions, especially if you’re asking questions about personal beliefs and behaviors. The chances that respondents will answer incorrectly in a survey increase if the possibility of backlash is high.
If you are asking people to identify themselves on a survey that is measuring customer satisfaction, workplace culture, or employee feedback, honest responses require assurance that there is no chance of judgment as a result of their answers.
Recognize Your Own Bias
Just as respondents will sometimes enter into a survey with bias, you may also have ideas or certain beliefs that could get in the way of learning the truth. Recognizing your assumptions or hypotheses right from the get-go will help you ensure that you’re asking all of the necessary questions and that you aren’t leading respondents to certain answers or outcomes.
Some Final Thoughts
I hope this helps you tackle some of the common challenges that inherently come with the survey method. While surveys can be extremely effective in gathering larger amounts of data and insights from many people, Business Analysts need to be aware of both the benefits and the obstacles that come with this method – and, really, any method of information gathering.
Check out my course Understand and Elicit Requirements (at a special discounted price!) for more insight into how you can effectively approach surveys and other elicitation techniques.
Happy learning!
– Written by Jeremy Aschenbrenner, The BA Guide
