Blog

Read our latest articles for tips, tricks, advice and news about research techniques and solutions.

How to Obtain an Online National Sample Stratified by Wealth

Aaron Moss, PhD
October 10, 2018

How to Obtain an Online National Sample Stratified by Wealth

A Case Study From a Recent JESP Article A new study appearing in the Journal of Experimental Social Psychology suggests Americans strongly believe in economic mobility because they fail to appreciate how vast wealth inequality really is. In this blog,...

Read More >
The Universal Exclude List: A Tool for Blocking Bad Workers

Aaron Moss, PhD
October 4, 2018

The Universal Exclude List: A Tool for Blocking Bad Workers

By now, even casual users of MTurk have heard about recent concerns of “bots” or low quality data. We’ve written about the topic here and laid out evidence that suggests “bots” are actually foreign workers using tools to obscure their true location (here). Perhaps most importantly, we’ve created two tools to help keep these workers out of your studies. In this blog, we introduce a third tool: the Universal Exclude List.

Read More >
After the Bot Scare: Understanding What’s Been Happening With Data Collection on MTurk and How to Stop It

Aaron Moss, PhD
September 18, 2018

After the Bot Scare: Understanding What’s Been Happening With Data Collection on MTurk and How to Stop It

In this blog, we review recent concerns about bots, along with general data quality issues, on Mechanical Turk and report the results of a study we conducted to investigate the problem.

Read More >
CloudResearch Tools to Help Combat Responses from Suspicious Geolocations

Aaron Moss, PhD
August 15, 2018

CloudResearch Tools to Help Combat Responses from Suspicious Geolocations

Last week, the research community was struck with concern that “bots” were contaminating data collection on Amazon’s Mechanical Turk (MTurk). We wrote about the issue and conducted our own preliminary investigation into the problem using the CloudResearch database. In this blog, we introduce two new tools CloudResearch is launching to help researchers combat suspicious activity on MTurk and reiterate some of the important takeaways from this conversation so far.

Read More >
Concerns About Bots on Mechanical Turk: Problems and Solutions

Aaron Moss, PhD
August 10, 2018

Concerns About Bots on Mechanical Turk: Problems and Solutions

When researchers collect data online, it’s natural to be concerned about data quality. Participants aren’t in the lab, so researchers can’t see who is taking their survey, what those participants are doing while answering questions, or whether participants are who they say they are. Not knowing is unsettling.

Read More >
MicroBatch Is Now a Pro Feature

Leib Litman, PhD
January 11, 2018

MicroBatch Is Now a Pro Feature

CloudResearch is announcing a change in our pricing for the MicroBatch feature. MicroBatch is now included as a Pro feature, with a fee of 2 cents + 5% per complete. This will also provide users with access to all other...

Read More >
Conducting Research: How Online Research Tools Can Help

Leib Litman, PhD
December 22, 2017

Conducting Research: How Online Research Tools Can Help

When researchers learn about conducting research online, it can sometimes be difficult to understand quite how all the tools that are available can actually be applied to a project. This post is about how specific research ideas can be carried...

Read More >
New Feature: Exclude Highly Active Workers

Leib Litman, PhD
December 15, 2017

New Feature: Exclude Highly Active Workers

Some workers on MTurk are extremely active, and take the majority of posted HITs. This can lead to many issues, some of which are outlined in our previous post. Although MTurk has over 100,000 workers who take surveys each year,...

Read More >
Best Recruitment Practices: Working With Issues of Non-Naivete on MTurk

Leib Litman, PhD
December 8, 2017

Best Recruitment Practices: Working With Issues of Non-Naivete on MTurk

It is important to consider how many highly experienced workers there are on Mechanical Turk. As discussed in previous posts, there is a population pool of active workers in the thousands, but this is far from exhaustible. A small group...

Read More >
Are MTurk Workers Who They Say They Are?

Leib Litman, PhD
December 1, 2017

Are MTurk Workers Who They Say They Are?

The internet has the reputation of being a place where people can hide in anonymity, and present as being very different people than who they actually are. Is this a problem on Mechanical Turk? Is the self-reported information provided by...

Read More >

SUBSCRIBE TO RECEIVE UPDATES

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.