Journalists Should Be Suspicious Of Our Results: Survey Monkey Boss

The US head of research at Survey Monkey agrees that journalists need to be as suspicious of stories and research drawn from its online surveys as they would be when handling any other story.

Jon Cohen, Survey Monkey’s vice president of survey research, agreed that people who used the platform to do “legitimate research” too often prejudiced the results with loaded or directed questions to a converted audience. He agreed that these “results” – often doled out to journalists as press releases with screaming headlines – can form the basis of articles that hold little journalistic worth.

Cohen was in Australia recently presenting as Sydney’s Ad:Tech conference and admitted to B&T that a lot of groups and organisations that used its survey tools need to be better trained in how to do it.

“Anyone can put a survey up and get back data that’s completely inappropriate for what you meant to ask,” Cohen agreed. “We acknowledge that and we’re trying to put in guide posts to try and steer the user on how to write questions that don’t produce bias.”

Cohen admitted that journalists were increasingly publishing Survey Monkey results in articles without questioning the calibre of the findings as well as they should.

“All numbers are not created equal,” he said. “I’d certainly implore journalists to tick through when they encounter a number and they need to be just as sceptical as they are when they interview a person as they are when they see a number.”

What Survey Monkey was best at doing, Cohen said, was providing insights, and those insights could be used for a host of things including an agency’s creative process.

He said that agencies often sought external opinions only at the end of a six-month pitch when Survey Monkey could prove an invaluable tool throughout the process, particularly at the start.

“There’ll always be a natural tension between creatives and marketers and their bosses. And the tension is going to happen when it all comes from very diverse sectors, but I just think that conflict is much worse at the end,” Cohen said.

“It comes when there’s six months of work on the line, people’s jobs are on the line; you’ll always have conflict. But if you have the information throughout the process it (tensions) will be smaller and more contained and you can debate it and test and you’ll be fighting smaller fires than you will at the end,” he said.

This sort of data, Cohen said, can be valuable in getting existing work heading in the right direction or getting it started when you have no starting point at all. Survey Monkey didn’t necessarily give you the data required, rather the insights to get something up and moving.

“I don’t think anyone wants more numbers and this is why we keep bringing the debate back to insight,” he said. “Sometimes you can have so much data you can’t separate the signal from the noise. I think what we do have now is the capacity to collect this information in a structured way and then formalise and rationalise that process.

“You can have too much data, just as you can have too little. The idea is about asking the right question when the answer is going to be useful for the decisions you have to make,” he said.