Oh the power of the internet.
56 seconds.
Yes, that’s all. It took just 56 seconds to disrupt thousands of scientific studies online. Imagine, the validity of your data — gone in under a minute.
Months of work … out the window.
But, let’s back up, just a bit, to hear how the drama unfolded. Florida teen, Sarah Frank, filmed a short TikTok about side hustles she recommends trying. One online platform, Prolific, had caught her eye, because she could get paid for taking surveys.
So, she made a little video. That’s where the trouble started. See, if you’re familiar with TikTok’s algorithm, you know that it’s pretty easy to go viral.
Her “little” video got 4.1 million+ views in weeks.
The aftermath is scary.
You see, Prolific had no screening tools in place. This meant they had no way to balance their consumer panel — if thousands of people decided to sign up.
Yikes.
I think you can guess what happens next.
Within weeks, scientists using the Prolific platform start seeing a big jump in US participants. The change in panelists are starting to skew their results. Why?
Well, they effectively doubled the size of the panel.
Uh oh.
Virtually overnight, 85% of participants became 21-year-old women. That meant all the demographic balancing they’d had in place to ensure accurate data – disappeared. Any study using the new group of participants, became highly skewed … and inaccurate.
How many you ask?
More than 4,600 studies were impacted by Frank’s viral video.
Now, I’d like to point out that young women’s opinions matter, of course.
But, you can’t base an entire panel on them, unless you’re only selling to a very small group of market researchers. That’s why a demographic skew this big is pretty scary.
Especially for Prolific, who had to refund researchers.
Since then, the company has implemented new screening tools and re-structured. They have a team in place dedicated to finding and fixing problems ahead of time.
So, what’s the lesson?
Choose your consumer panel provider wisely. Make sure your studies are protected by quotas and screeners. Better yet, make sure they use Fair Trade Data®. That’s where all consumers know exactly what they’re signing up for. So, you can trust the data.
Here’s an example.
This research database has 1,699+ consumer insights. It’s data that is collected every week from 10 million daily consumer journeys. All of the data is demographically balanced before it’s added into the pool. So, you can sort by brand, category, or time.
Check it out, it’s completely free.
Oh, and if you need help on a project, just send us an email.
References:
