As billed, the first day of the inaugural NEXT conference has given us glimpses into what’s happening in the world of research and more importantly where we could/should/might hopefully be going. Wanting to attend multiple presentations at the same time but hampered by my inability to physically be in two or three places at once (damn you, physics), I was forced to make choices, as did the rest of the attendees, so I won’t even try to cover it all here. But I’ll try to pull out some highlights, themes, and food for thought as the conference progresses.
After keynotes that pushed us to think in new ways and inspired us to approach research with a different mentality, four companies were brave enough to enter the Shark Tank and have their ideas vetted by the MRX sharks and ultimately the whole audience. Protobrand ultimately took the crown with Anders Bengtsson’s compelling presentation of their Meta4Insight tool, which uses images to engage system-one thinking of research participants. I’m pretty certain the red pants had some impact on his victory. CoolTool presented their approach to automation-fueled neuroscientific research. Vital Insights explored InTask, their novel approach to engaging physicians for research with a version of online patient interaction simulation. And Mission Field shared how they approach shopper research by engaging with people in the context of their shopping behaviors, directly in retail stores. While the pitches varied greatly in terms of methodology, what they found in common was an underlying drive to get at what’s “real,” doing research in an improved way that can better understand human motivations, choices, and behaviors. Certainly such innovations are needed – and most should be welcomed – but it has to leave those committed to more traditional survey research wondering what the future brings for them and calls them to think about how to test and adopt innovative techniques.
While the conference as a whole and several of the sessions in particular focused on new advancements in research, for those of us with one foot in innovation and the other in more traditional survey work, it was great to see concerted focus on how to improve some of the tried and true approaches (read: online surveys) that form the core of our industry.
Pete Cape’s presentation on gamification across cultures was a practical and compelling take on how simple efforts taken by SSI to gamify can improve both data quality and participant experience. Something as simple as changing wording and encouraging participants to “take up to 60 seconds to name up to 10 brands” can yield positive results. I left thinking, “why would anyone not try to make their survey more engaging in such a simple but powerful way?”. Seriously, why not? (Yes, tracker owners, I know why you’ll raise a stink.) It’s also important to heed Pete’s call to take inspiration from those outside the research industry – both academic and non-academic –who seek to understand human behavior from different perspectives.
Cecile Carre and Nancy Brigham presented a wealth of information from research they conducted on mobile (or device agnostic) survey design with a focus on LOI and specific question types. One of the more striking findings was that Ipsos has moved from drawing the line for what demarcates an intolerable survey on a smartphone from over 15 minutes to over 20 minutes. While they were careful to caveat that 20 minutes isn’t recommended, but is doable, it still gives me pause as I know clients will take license to push that limit. I’ve personally had clients ask me flat out what is the longest survey I can create that you’ll still send to mobile participants. After the steam stops coming out of my ears I usually tell them (sometimes even a little bit nicely) that that is the totally wrong way to approach research. That said, things change. And consistent research on survey best practices, specifically those focused on mobile, continue to be invaluable for survey researchers.
As the day came to a close with a very thought provoking panel put on by the Global Research Business Network (GRBN), my favorite and most intriguing comment came from audience member Kelsy Saulsbury of Schwan’s. When pressed on what clients are doing to resist the horrible surveys that plague our industry, she mentioned that she includes her email address at the end of all surveys she writes. Participants then can – and do – contact her directly about their experiences. That’s ownership! And so I ask of everyone attending the conference (and observing from afar), would you put your name on the line for the surveys you put out into the world? So far, I’ve seen plenty of inspiration and practical guidance on how to do better research. Now it’s up to us to take what we learn back into the world and put it into action.
I had the privilege of chairing the Thought Leadership track on day 2 of the NEXT conference, which meant I was forced to stave off my ADD and predilection for jumping between tracks. While I heard that I missed a lot of good stuff in the other tracks, focusing on one track proved rewarding. Our presenters and panelists tackled some of the most pressing topics in the research industry and I consistently wanted to continue the discussion for 10 – or 30 – more minutes. However, my job was to keep things on time, so I had to play the guy who quells lively discussion in the name of order. And I’ll have to invoke that role again to reign myself in from trying to comment on everything in the track.
If I was able to magically extend a session, I certainly would have done it for the panel discussion on data quality led by Chuck Miller (DM2). He, along with Lisa Wilding-Brown (Innnovate), Ken Berry (Jibunu), and Garrett Gil de Rubio (P2Sample) candidly discussed some of the things that many of us in the industry have been seeing but are not often raised in a public forum. First, and importantly, the group maintained that bad survey taking behaviors are not equivalent to fraud. I certainly concur as I’ve seen plenty of examples where data is thrown out, deemed fraudulent, or otherwise maligned because a participant, for example, straightlines through a grid, the last in a line of many they have been forced to answer in a thirty-minute survey. Yes, I’m placing some blame here on the questionnaire designer and away from the participant in this case, but that doesn’t mean that survey takers are never to blame. It’s important, however, to distinguish lack of attentiveness or other things we don’t like with fraud. While I’m in the business of throwing shade, I’ll further echo the sentiments of the panelists in reinforcing the importance of good screeners. Memo to people including Yes/No questions in screeners: just don’t. Research design set aside, the industry has been and continues to be impacted by fraud from savvy people (or bots) intent on scamming their way to incentives. There are a number of markers of fraudulent behavior, varying from IP address spoofing to inconsistent answer selection. Chuck walked through several of these indicators and how well they worked, which I won’t recount here, but I know he’d be happy to share. Suffice it to say that panel companies are taking data quality more seriously than ever and employing various techniques to prevent fraud. No traps or safeguards are perfect, however, and should you encounter bad data, it’s worth approaching your data provider with a level of understanding and engage in a candid and open conversation about how to make things right.
The day was brought to a close with a session about the pressure of speed on qualitative research, deftly led by Jim Bryson (20 I 20). Zach Simmons (Discuss.io), Katrina Noelle (Scoot Insights), Stephen Cribbett (Dub), and John Joseph White (Feelr/Dub) grappled with the increasing pressures to conduct research faster. As agile research has become both voguish and more commonplace, those in the qual space have devised ways to meet these needs. The panel agreed that agile or quick turn qual is often intense, both for the research provider and the client – and that’s something that clients need to be made aware of. As on the quant side, technology has made qual research faster (and better), increasing speed of recruitment and analysis/reporting. That said, there is still a crunch for researchers to devote themselves intently to sometimes taxing schedules to meet deadlines, something the panel acknowledged needed to be considered along with providing researchers a requisite reprieve when necessary. The burnout struggle is real. Having been on the qual side when there was much less technology available, I lived through some of the inefficient and time-consuming approaches to conducting qual research. Compared with our quant colleagues five or ten years ago, it was fairly easy to say that qual suffered from a lack of process and was overly manual; and that was pretty much true. Times have changed however, and the companies on the panel as well as other players in the space have worked hard to bring speed to qual research without sacrificing – and I’d argue in many cases significantly improving – quality.
Spending the better part of the day hearing about ways in which we, as an industry, are working to improve some of the fundamental aspects of how we conduct research, brought me back to thinking about Anthony Tasgal’s keynote on storytelling. He pointed out that researchers need to be fluent in behavioral economics and to truly grasp what drives people – both the people who participate in research and the people who research is shared with. Ultimately, he made a compelling case for why storytelling is critical and provided practical tips on how to tell better stories. He’s right, of course. I’ve heard a lot about storytelling in market research the past few years – as have you if you’ve been listening. We should be focused on storytelling, but we simultaneously need to focus on consistently improving how we collect data and approach research. Research on research and thought leadership discussions are essential. If we’re not improving data quality and innovating, we’ll have much less believable and compelling stories to tell.
NewsTechniqueRecapping the New NEXT ConferenceRoddy Knowles – Research Now