Human Workers, Managed by an Algorithm

document Cloud Labor

Human Workers, Managed by an Algorithm
Foreign recruits are the newest cogs in the crowdsourcing machine.

Antonio Regalado , July 31, 2012

Global workforce: Remote digital workers earned $0.32 each for producing these self-portraits. MobileWorks

Stephanie Hamilton is part of something larger than herself. She's part of a computer program. The 38-year-old resident of Kingston, Jamaica, recently began performing small tasks assigned to her by an algorithm running on a computer in Berkeley, California.

That software, developed by a startup called MobileWorks, represents the latest trend in crowdsourcing: organizing foreign workers on a mass scale to do routine jobs that computers aren't yet good at, like checking spreadsheets or reading receipts. By assigning such tasks to people in emerging economies, MobileWorks hopes to get good work for low prices. It uses software to closely control the process, increasing accuracy by having multiple workers perform every task. According to company cofounder Anand Kulkarni, the aim is to get the crowd of workers to "behave much more like an automatic resource than like individual and unreliable human beings." The best-known crowd marketplace is Mechanical Turk, which Amazon launched in 2005. The website operates as an online odd-job market where humans earn a few pennies at a time by carrying out low-level, repetitive tasks that machines have difficulty with, such as identifying objects in photographs. Amazon's marketplace was a revolutionary idea. But it has proved difficult to use effectively. A 2010 study published by the online reviews company Yelp found that "Turkers" were less accurate than a simple machine learning program at the task of categorizing businesses as a restaurant or a clothing store (see "AI Defeats the Hivemind"). "If you put something on Mechanical Turk, it's easy to get cheated," says Luis Von Ahn, a crowdsourcing expert at Carnegie Mellon University. The main problem is low pay. He estimates that Turkers, who are based mostly in the United States, make only $1 or $2 per hour. That's a big incentive to submit nonsense or poorly researched answers. Even so, many large companies remain intensely interested in crowdsourcing, Von Ahn says: "They all want to lower the cost of something."

Now several startups, including CrowdFlower and CrowdSource, have written software that works on top of Mechanical Turk, adding ways to test and rank workers, match them up to tasks, and organize work so it gets double- or triple-checked. "In the past [crowdsourcing] has been more experimental than a real enterprise solution," says Stephanie Leffler, the founder of CrowdSource. "The reality is that it's tough to do at any kind of scale." MobileWorks has its own workflow software, but it's also trying to solve the incentive problem by recruiting workers overseas, in developing nations like India, where low payments can still add up to meaningful income. Kulkarni, who founded the company in 2010 with fellow graduate students from the University of California, Berkeley, says the value of tasks is set so that workers can reasonably earn $2 to $4 an hour; payments are on a sliding scale, with lower rates for poorer countries. "Even though they are acting as agents of a computer program, we are creating an opportunity for them," he says. MobileWorks charges its clients rates starting at $5 per hour for workers' time. Hamilton began doing microtasks about a month ago in Kingston. That is when MobileWorks struck a deal with Jamaica's government to promote this type of work, saying it could create 1,000 jobs in three months. Jamaica, which suffers from 14 percent unemployment, has the third-largest number of native English speakers in the Americas. Hamilton isn't among Jamaica's jobless. She works for the government port authority as a procurement agent. That's also where she has been logging on to do microtasks. "I do it in my free time and when I have low amounts of work. I do it at my job," she says.

So far, she's carried out tasks like collaborating with other remote workers to double-check a spreadsheet of admissions contacts for several thousand U.S. schools. Workers can rate data as correct, provide a different answer, or flag it for a supervisor. Her first 16 completed tasks, some of which took up to 10 minutes, have netted her a total of 85 cents. Some of the work is a little shady. One person describing himself as a sociologist paid Hamilton a few cents to create a "believable" Twitter account. Her creation, Luke Lynch, who uses the handle @luke_daredevil, says he is "the eldest of four children" and a fan of rugby and pizza. For her character, whose photo she plucked off Google, Hamilton was required to post one believable tweet, choosing "The performance at BET Awards was popping." The experience left Hamilton "wondering what that was all about." That fake Twitter account appears to be part of a wider spam network established by people associated with the social network View.io, possibly to recruit members. Unknown others have since take over the Luke Lynch identity, which has emitted a series of raunchy comments and stock quotes in Indian rupees. (View.io's founder, Felix Chan, didn't respond to a request for comment.) "Ugh!" said Kulkarni after hearing how his MobileWorks software was being used. "We do try to police the spambots." He says inexperienced workers might not recognize "spammy" jobs, calling it a "cyberliteracy issue." But the problem isn't new. Two years ago, researchers at New York University estimated that 41 percent of all jobs posted to Mechanical Turk were for generating spam, generating clicks on ads, or influencing search engine results (see "How Mechanical Turk Is Broken").

Despite these abuses, overseas crowds have become essential to legitimate businesses. Consider inDinero, a three-year-old San Francisco Web startup whose software helps small businesses track their finances. Businesses can e-mail or upload scanned receipts (including handwritten ones) that then need to get logged—so inDinero sends the images to MobileWorks, which in turn farms them out to workers in India or the Philippines, who transcribe the receipt amounts into online forms. Jessica Mah, inDinero's CEO, says she couldn't easily run her business without a crowd of assistants. "The alternative would be to have highschool students do it ... but there's no redundancy, and there's no built-in workflow management," she says. "The price of having MobileWorks have five people look over a receipt is cheaper than the cost of hiring a high-school student at minimum wage to do the task once."

Summary
Foreign recruits are the newest cogs in the crowdsourcing machine.

Description
Stephanie Hamilton is part of something larger than herself. She's part of a computer program. The 38-year-old resident of Kingston, Jamaica, recently began performing small tasks assigned to her by an algorithm running on a computer in Berkeley, California.That software,developed by a startup called MobileWorks,represents the latest trend in crowdsourcing: organizing foreign workers on a mass scale to do routine jobs that computers aren't yet good at,like checking spreadsheets or reading receipts. According to company cofounder Anand Kulkarni,the aim is to get the crowd of workers to"behave much more like an automatic resource than like individual and unreliable human beings."

Tags: Business, business impact, business report, Crowdsourcing, Mechanical Turk, MobileWorks

Posted by Adam Frazer

Jul 31, 2012 10:26 pm GMT Viewed 1003 times

Author: Antonio Regalado Source

Contact us if you are owner of this content and want to make changes.

Download Flag This

Events

Human Workers, Managed by an Algorithm

Comments

Crowdsourcing.org

Categories

Content

Crowdsourcing.org, an initiative by massolution®

Events

Related Content

Human Workers, Managed by an Algorithm

Comments

Crowdsourcing.org

Categories

Content

Crowdsourcing.org, an initiative by massolution®