News just in - crowdsourced articles are better than single-author efforts
By Peter Farquhar, Technology Editor news.com.au February 07, 2011 11:22AM
Why pay one writer to compile 100 facts for a story when you can pay 100 writers next to nothing to deliver one fact each? Picture: Getty Images Source: Getty Images Article broken down into mini tasks Written by 36 different authors Rated higher quality than single-author pieces
JOURNALISTS, consider this a warning. The crowd doesn't care for your craft anymore. It can do it better and it can do it cheaper. Or so researchers at Carnegie Mellon Univeristy claim, after proving single articles can be written by dozens of people with no discernible drop in quality or accuracy. They've come up with a framework they call CrowdForge, which breaks down complex tasks into simple microtasks and combines it all into a meaningful result. For example, to prepare an encyclopedia article, CrowdForge would find several people to write an outline. As a quality control measure, different workers would vote for the best outline or combine the best parts of each outline into a master outline. Subsequent sub-tasks might include collecting one fact for a topic in the outline. Finally, another worker would collect the facts and turn them into a paragraph, or combine several paragraphs in proper order for an article. It sounds like a lot of work for little result, but that's exactly what crowdsourcing is all about - and it's catching on quickly.
Two weeks ago, Victoria's Swinburne University joined the University of Bonn in Germany in putting out a call to amateur astronomers to crowdsource the definition of a galaxy. Another project - NASA's BeAMartian - sees millions of volunteers performing tasks such as identifying landforms on Mars. To test CrowdForge's ability to write articles, CMU tasked it with preparing five articles on New York City. Each article would require 36 sub-tasks - 36 authors adding a sliver of information for roughly 10c a pop. It would cost an average of $3.26 to prepare a 650-odd word piece. CMU crowdsourced its work through Amazon's Mechanical Turk (MTurk), an online marketplace for work. Employers can post simple, self-contained tasks that "turkers" complete in return for a few cents. Eight writers were then paid $3.05 to write articles on the same five subjects. The average length came out at 393 words. Here's the bad news for journalists - when 15 people compared the articles, they rated the crowdsourced articles of higher quality than those produced by individuals. In a further blow to the writers, the crowdsourced articles were rated about the same quality as a Wikipedia entry on the topic.
The variability - the range from the best to the worst article - was also lower for the crowdsourced articles. "This is exciting because collaborative crowdsourcing could change the future of work," assistant professor in CMU's Human-Computer Interaction Institute, Aniket Kittur, said. "We foresee a day when it will be possible to tap into hundreds of thousands or millions of workers around the globe to accomplish creative work on an unprecedented scale." In fact, it worked so well, a couple of San Francisco-based science journalists - Jim Giles and MacGregor Campbell - have created a blog, www.mybossisarobot.com, to explore using CrowdForge for preparing science news articles based on research reports. "We were surprised at how well CrowdForge worked," Dr Kittur said. "Admittedly, none of these articles is going to win any awards. "But the ratings weren't bad considering that the work of dozens of people had to be coordinated to produce these pieces." Source: EurekaAlert
Crowdsourced articles are better than single-author efforts. Peter discusses the statement through a clear example:
researchers at Carnegie Mellon Univeristy claim, after proving single articles can be written by dozens of people with no discernible drop in quality or accuracy.
The crowd doesn't care for your craft anymore. It can do it better and it can do it cheaper.
Journalists need to be aware that crowdsourcing is slowly changing the phase of journalism and in other forms of writing.