On MTurk, iPad responses are different (sort of)

That’s the click bait finding from a new study by Findlater and colleagues. They noted all the research showing Turkers perform similarly to lab populations (validity wise) but wondered if this held true when looking at Turkers doing work on mobile  devices with touchscreens. . They argued since mobile devices are the most popular form of access to the Internet, there may be more variability due to posture of the respondent and in movement of the device.

So, they compared 30 people in a lab to 303 people on MTurk.  People had to do a number of tasks involving the interface that they were working on such as dragging things across a screen. Findings show:

“(1) separately analyzing the crowd and lab data yields different study conclusions-touchscreen input was significantly less error prone than mouse input in the lab but more error prone online;

(2) age-matched crowdsourced participants were significantly faster and less accurate than their lab-based counterparts, contrasting past work;

(3) variability in mobile device movement and orientation increased as experimenter control decreased–a potential factor affecting the touchscreen error differences. ”

The study itself (which you can download from the link) shows that the mobile device used was an iPad, and there were specific instructions on how and where to place the device.

I think the abstract somewhat overstates the findings, since this was an interaction task and not a task like a survey which requires fewer manipulation skills and this isn’t really that clear in the abstract. But kind of interesting.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s