Getting meaningful data from customer or user surveys are not as easy as many believe. Samples sizes, survey instruments, poorly-designed questions and research assumptions all affect the validity of results. Word chioce is another common source of error.
Rank order questions sometimes help to discover relative priorities. Consider information technology professional responses to a survey on endpoint management. This set of responses does show some differentiation of “ability to manage” various elements of the IT environment.
Those results are not a normal probability distribution. As often is the case with self reporting, all the organizations perform “above average.” If capabilities really are described by a normal probability distribution, that cannot be correct.
Sometimes researchers are looking for indicators of “most important/least important” opinions. In such cases, forcing respondents to choose can better highlight opinions about what is “most important.”
Harder to control are researcher frames of reference. Any multiple-choice survey must necessarily embed assumptions about a few variables researchers believe are relevant. That bias is among the most-common I encounter as someone who gets asked to take surveys.
No comments:
Post a Comment