2,958 crowdsourcing and crowdfunding sites
Disclosure: CrowdComputing Systems is a client of Massolution / Crowdsourcing.org. This is a sponsored post.
Our increasingly digitized world has unlocked complex information and data sets that we could not access in the past. The rise of ‘big data’ is an important development, and many governments, organizations, and enterprises are taking notice of power of organizing, understanding, and creating large data sets.
Trudging through the data, however, is a challenging task, and many companies are experimenting with different approaches to identify the best one. One method that has been suggested is crowdsourcing. Humans can easily identify patterns that computers may not readily recognize, and tapping into a large pool of workers to sort through massive data sets is logical.
Splitting the work into manageable microtasks, managing the workforce, and quality controlling the work, however, still requires significant overhead, which has made crowdsourcing less appealing to large enterprises.
The company CrowdComputing Systems (CCS) thinks it has a practical solution that combines crowdsourcing, automation, and artificial intelligence (AI). The company (along with our sister firm Massolution) has written a white paper on the subject, and we spoke with CCS’s executives to learn more about the role of AI and automation in their approach to crowdsourcing,.
Gartner defines ‘big data’ as “high-volume, -velocity and -variety information assets that demand cost-effective, innovative forms of information processing for enhanced insight and decision making.” This definition captures both the promise of big data (the ability to enhance insight and decision making), as well as the current problem with the concept (the need for economical and innovative ways to process the data). CCS calls its approach to realizing the promise and solving the problem “crowd computing,” and the company believes it’s the most innovative and cost-effective solution to big data.
CCS is the brainchild of Max Yankelevich and Andrew Volkov, who were experimenting with using crowdsourcing for fraud detection at MIT. Through their research, they discovered that adding an AI layer between a manager and the crowdsourced workers for recruitment, management, and quality assurance made the process much more efficient. Since then, they’ve advanced that concept and have built CCS on two guiding principles: human-guided AI and AI-guided humans.
“AI is responsible for two major things: managing a very large, crowdsourced workforce through by breaking down work into small bits and optimizing the work, and, the main thing, by quality-controlling work,” Yankelevich, cofounder and CEO, told Crowdsourcing.org. “We call that AI-guided humans.”
“Human-guided AI is having machine learning actively participating in the process, as a manager, but also standing over your shoulder and watching you perform these tasks, and eventually taking over more and more work, if it’s repetitive enough,” he continued. “This is the ability to not only manage the human-based processes, but also to take over some of the more repetitive tasks.”
This AI layer is the key to CCS’s processes and is the reason why the company positions itself as creator of a SaaS product, rather than a more traditional crowdsourcing platform.
When a client begins using crowd computing, about 80 percent of the work is human-based, and the other 20 percent is automated, Yankelevich said.
“We end up with the customer over a couple of years or months is a flipped ration – 20 percent human-based, and 80 percent is automated,” he explained. “Humans are responsible for mainly processing the exceptions.”
CCS taps into a number of labor pools to source its workers, including Amazon Mechanical Turk, Elance, and oDesk, and a company’s employees can also be included in the labor pool as subject matter experts (SMEs). Yankelevich estimates that the total number of workers adds up to over 20 million individuals. To ensure the quality of such a large workforce, the software uses various AI techniques such as fuzzy-logic, statistical analysis, and inference engines in order to judge whether an answer is correct.
“If crowdsourced workers are constantly getting the work wrong or are making mistakes, the implication there is that the pay isn’t high enough and it’s not attractive to higher-skilled workers, or that the process is wrong,” said VP of Marketing Adam Devine. “The AI will increase the amount of pay, and recruit workers, and apply a higher caliber of quality control to the process.”
Yankelevich explained the concept further:
“This is a pretty challenging thing because you would think that it’s pretty much black and white – either a worker is contributing high quality work, or he or she is not. But it turns out there’s really a lot of gray – it’s probably 99 percent gray territory. It’s really easy to find out who’s a spammer, but it’s really difficult to understand what to do when someone is submitting great quality work maybe 20 percent of the time, and the other 80 percent of the time, he or she is not submitting good work. We had to teach our algorithms to essentially work with all these different shades of gray and be able to address them as a human manager would.”
Ultimately, Devine said, CCS’s software enables managers to spend less of their time defining and assigning work, managing the workers, checking for quality, and handling payments. Instead, they managers are able to focus on more interesting and complex tasks. He mentioned one of the company’s youngest customers, is using the software to oversee a crowd of 500 workers: “He’s probably got the biggest team of any 24-year-old on the planet, but it’s like driving a freight train on cruise control. The software does most of the managing.”
Combining the best of the crowdsourcing with automation not only allows large enterprises to sort through and understand big data sets, but it also frees up existing employees to pursue more valuable work and to create new products. We’re excited to see what CCS and its enterprise customers can build on top of the power of crowdsourcing. To find out more about crowd computing and read several use cases, make sure to download the white paper, which can be found here.