top of page

Data Disasters and Dumb Survey Design

Market research is ultimately a means for companies to communicate with their customers and potential customers. Whether conducting customer satisfaction research or testing a new product design, researchers are communicating with respondents and again with the research sponsor, each step of the way striving to maintain the integrity of valuable information.

The top priorities in any research project are to carefully choose one’s words and to attentively listen. If respondents aren’t forthright, the information yielded won’t be safely projectable to a broader population. If researchers don’t pay close attention to the formulation of their questions, or to the answers to those questions, their analysis will prove muddy at best and dishonest at worst.

Communication can break down at any step in the research process, turning the project into a game of telephone. Amidst the confusion, the actual opinions and sentiments of the target market are lost, undermining the entire premise of the research. With this in mind, it’s important to apply the same conventions for high quality communication in each interaction during the process.

Unfortunately, in the world of online surveys, this imperative is sometimes overlooked at a particularly important step along the way: the survey questionnaire itself.

When time and budget are limited, it may seem tempting to accept a questionnaire as soon as the right questions are down on the page. If you know what you’re trying to learn, how hard can it be to craft your questions and put them in a coherent order? But this sort of impetuousness may result in miscommunication from the very onset, potentially snowballing into something downright untrue by the time the message has made its way around the room in that proverbial game of telephone.  

Understanding that survey respondents are an audience too, and that the survey is a message as much as it is a question, is paramount to collecting sound data. This is why crafting a questionnaire is more than simply cooking up the best way to ask your particular questions.

We must ask ourselves: who will be taking this survey? On what kind of device will they be taking it? How, in terms of attitude, will they approach answering our questions? For instance, if the answers to those first two questions are: 1. People ages 18 to 24 will be taking the survey, 2. Most likely on their mobile phones, then the answer to the third question is probably “impatiently.” (This is not intended as an affront to the attention spans of young people, but rather an indictment of clueless survey design in which consideration of one’s audience never takes place.)

If we don’t consider our audience, if we don’t spend a good deal of time interrogating the probable ways in which our audience will be engaging with our survey, then the data will suffer.

Respondents will drop out; “Boring!” You might hear them shout in frustration after the 20th redundant question. Data will lose integrity; “If I just race through these questions without reading the paragraphs of instructions, I can still enter the drawing for the Macbook at the end.”

Market research is all about communication, and the oldest rule of communication is “Know your audience.” It may not be possible to know the ins and outs of our survey audience ahead of time (if we did, why would we be surveying them?), but it is possible, and extremely important, to entertain hypotheticals, conduct some secondary research on the target demographic, really spend time thinking about how our audience will receive our questions and in turn go about answering them.

If we skip this crucial step, we’re paving the way for a mere illusion of insight. If we skip this step, we’ll likely wind up with the hurried, apathetic responses of disinterested respondents.


bottom of page