EIGHT ESSENTIALS OF CROWDSOURCING

document Distributed Knowledge, Tools

Eight Ways to Launch Higher Quality Applications and Get to Market Faster with the Crowd
WHITEPAPER: February, 2009

CROWDSOURCING

Table of Contents
The Emergence of Crowdsourcing…………………………………………………………………………………………….……………………..……………….……….… Today’s Staffing Options for Software Development and Testing……………………………………………………………….. Driving Forces Behind Crowdsourcing…………………………………………………………………………………………………………..……………………. Key Benefits from Crowdsourcing………………………………………………………………………………………………………………………………………….. The Eight Essentials: Launch Better Apps Faster Through Crowdsourcing………………………………………………….. 1. Build your dream team……………………………………………………………………………………………………………………………………..…………………… 2. Independent thought is the lifeblood of creativity…………………………………………………………………….…………………… 3. Aggregate small bits of information into a knowledge base………………………………………………………………….. 4. Let their reputation precede them……………………………………………………………………………………………………………………………. 5. Use crowd diversity to mirror real-world conditions…………………………………………………………….………………………. 6. Gain more than you bargained for……………………………………………………………………………………………………….……………………… 7. Not all crowds are created equal………………………………………………………………………………………………………………………………….. 8 You still call the shots……………………………………………………………………………………………………………………………………..………………………. 2 2 2 3 4 4 4 4 5 5 5 6 6

Dig Deeper & Take Action…………………………………………………………………………………………………………………………………………………………………… 7 About uTest……………………………………………………………………………………………………………………………………………………………………………………………………. 7

“Peer production is about more than sitting down and having a nice conversation. It's about harnessing a new mode of production to take innovation and wealth creation to new levels.” - Eric Schmidt CEO, Google

WHITEPAPER: EIGHT ESSENTIALS OF CROWDSOURCING

1

The Emergence of Crowdsourcing
Today’s Staffing Options for Software Development and Testing Software executives and corporate IT managers have a number of options for staffing and sourcing the various tasks of software development. Design, development and testing can be accomplished through a combination of three staffing methods: • In-house staff • Outsourcing firms • Crowdsourcing communities These three options have historically been viewed by some as an “either/or” choice. But because of technology and innovative business models, this is no longer the case. This shift has empowered progressive software company execs to make the best decisions on a per-project basis. Software teams can now build in-house expertise while maintaining the operational flexibility that comes with external providers. The emergence of crowdsourcing has triggered this increased flexibility. The relative pros and cons of standard outsourcing – which typically involves the displacement of large-scale project resources, lack of accountability, and hidden management costs – are well known to most execs. In this whitepaper, we focus instead on how crowdsourcing can complement in-house resources at a practical, tactical level. The task and opportunity for executives today is to get the most bang for their buck by building in-house strengths around key employees and core competencies, while also tapping into variable resources like crowdsourcing when they can augment in-house personnel and reduce fixed costs. Driving Forces behind Crowdsourcing Crowdsourcing enables firms to tap into the creativity and diversity of a global community of skilled professionals. The trend has been adopted across many industries, including graphic design, development, testing, usability and R&D. Much of the inspiration grew out of the success of the Open Source movement. So it’s no surprise that software enterprises have successfully adopted crowdsourcing for the development and testing of commercial applications for web, desktop and mobile platforms. “Because crowdsourcing is built from the open source template, any company looking to leverage computer professionals has the advantage of an audience already familiar with the basic idea of community production.”

In some cases, including mySQL and RedHat, the pure opensource model has been adopted for commercial purposes. Even more interestingly, online communities such as Guru, - Jeff Howe or CrowdSpring (for UI and graphic design), TopCoder or Journalist, Wired.com Rent-a-Coder (for app development) and uTest (for app testing and usability) have opened the crowdsourcing model to a wide range of software applications.

WHITEPAPER: EIGHT ESSENTIALS OF CROWDSOURCING

2

By bringing in diverse and talented professionals on an on-demand, pay-per-performance basis, your applications can reach market faster, with higher levels of quality and at a lower cost. Communities of skilled professionals who bring fresh eyes and ideas, and who care about their online reputations, can be used (and are being used) to complement in-house resources to build, test and launch world-class desktop, web and mobile apps. Key Benefits from Crowdsourcing The numerous benefits that can be achieved when adopting crowdsourcing for software development and testing include: • Cost containment: Crowdsourcing allows managers to utilize lower-cost outside support without being tied down by long-term commitments. The fundamental structure of online communities creates competition, making it more accountable and cost-effective than traditional outsourcing. • Dealing with increased complexity: Significant quality improvement is achieved when development and testing is done across a wider set of platforms, locations and languages. Crowdsourced developers and testers can be easily recruited for each OS, browser, plugin, mobile handset model and language. • Faster time to market: Crowdsourcing enables you to ramp up or down to meet your needs at any given time. This eliminates delays and staffing constraints that often stifle companies around peak release times. This is especially useful for companies that are working with tight budgets and employing short-cycle development processes like Agile. • Meeting expectations for higher quality: Today’s customers expect applications to be complete and fully functional from day one. The era of ‘beta-testing’ as a substitute for QA is long gone. Companies that believe that “good enough” is good enough when it comes to quality and usability do so at their own peril. • Keeping your customers (and converting your prospects): The cost of having bugs or subpar usability has never been higher. Rest assured, if your product has problems, or doesn’t meet expectations, it’s easier than ever for your customers (and prospective customers) to find your competition. • Empowering the user: Users that feel that they have a hand in the success of a product will have a stronger emotional commitment to the product and the company. Crowdsourcing blurs the line between software builders (designers, developers and testers) and software users, creating powerful connections and an army of fans.

WHITEPAPER: EIGHT ESSENTIALS OF CROWDSOURCING

3

The Eight Essentials: Launching Better Apps with Crowdsourcing
The beauty of crowdsourcing is that it does not require an “either/or” commitment. Software company executives can easily incorporate and experiment with a community on a trial basis, and strike a balance between in-house personnel and community staffing. The following list of tips describes how to maximize the benefits of crowdsourcing for app development and testing. 1. Build your dream team The best online communities of professionals serve as a perfect complement to in-house teams. Build a team that blends the strengths of your full-time employees (brand, IP, reputation) with those of the "uTest found two "showcommunity (flexibility, cost-effectiveness, testing coverage stopper" bugs that would have across locations, language and platforms.) killed our release. In just 12 Having this blend of staffing allows you to scale your resources up and down in a fluid manner, meeting tight deadlines during peak periods of development and testing, while controlling costs during slow periods. And the end result of this scalable community-driven workforce is faster time-to-market. hours we had the feedback we needed to recreate and fix the bugs and release a stable product that captured significant market share."

- Sagi Richberg 2. Independent thought is the lifeblood of creativity CEO, Group Gain With a diverse community of professionals that transcends location and background, you can avoid the group-think that often stifles internal teams. When a community is following a lead, no matter how large the crowd, it will simply reflect the one leadership voice. Often, the consensus view from inside your company can drown out objections and alternate points of view. This is often an unintended byproduct of strong leadership. Again, a global community brings diverse opinions and experience (as well as fresh eyes), which can result in creative development solutions and more complete testing coverage. 3. Aggregate small bits of information into a knowledge base The information and feedback that is generated by a crowd of professional testers can be aggregated into high-value business knowledge. Software firms of all sizes can use data tagging and statistical aggregation to identify trends, which is a goldmine for management decision making. In addition to guiding your product design and development roadmap, companies can expose this information to their customers and prospects. By showing the number of users, platform types, geographic coverage that your product serves, you instantly establish credibility in the market.

“When it comes to real-world application testing, the crowds in Camden Market are preferable to the elite residents of Buckingham Palace. Diversity yields world-class QA coverage.” - Doron Reuveni CEO, uTest

WHITEPAPER: EIGHT ESSENTIALS OF CROWDSOURCING

4

4. Let their reputation precede them To get the most out of crowdsourcing, find communities that enable community members to build their online reputation. This increases accountability and performance by rewarding good actors and punishing bad behavior. Performance ratings and recognition levels enable your firm to choose the right “There are two types of currency professionals, and incent the desired behavior with members in an online community: money of the community. and reputation. Of course, money talks, but the transparency of 5. Use crowd diversity to mirror real-world conditions crowdsourcing motivates people Forget the fact that you are paying these people to write to build strong reputations. This code or find bugs. While that’s the primary job to be done, a is the only way they can earn secondary (but powerful) benefit is that your product is being future work and the respect of validated in the hands and minds of real-world users in realworld conditions. Select the members of the community that their peers.” best match your customer base: by location, language, expertise and technology platform. The community that works on your app matches the users you hope to target. - Matt Johnston VP Marketing, uTest

Homogenous internal teams – even those comprised of smart, hungry, talented people – are often less effective at generating new product ideas and offering objective feedback than crosspollinating groups with a variety of backgrounds and perspectives. Marketers and product managers can use this feedback as another way to solicit ideas and drive usability and new feature design. Of course, the day-to-day tasks of banging out code and testing applications cannot be ignored. But if this is your only goal, you may be missing an opportunity to gain even more from the crowd. 6. Gain more than you bargained for Companies who view crowdsourcing as just another way to outsource will often experience results similar to those of traditional outsourcing. In and of itself, this isn’t a bad thing, as it can still provide tremendous cost savings, increased efficiencies and enable your in-house teams to focus on areas that are mission critical. But the potential of crowdsourcing is much greater for those firms that ‘get it’. Companies that truly engage their online communities and tap into the passion and creativity within them often find that their crowd can become an engine of new ideas, buzz and overall awareness.

“The uTest model lets us scale correctly. I don’t have to hire and manage a whole QA staff. We never miss a step, since we start a test cycle when we leave for the weekend, and test results are waiting for us when we come back in on Monday.” - James McElhiney CTO, Second Rotation

WHITEPAPER: EIGHT ESSENTIALS OF CROWDSOURCING

5

7. Not all crowds are created equal When selecting a community to work on your products, choose wisely. Some communities tend to be noisy, with lots of opinionating, but not much actual productive work. Choose a community that has proven success and exposes the past performance and reputation of community members. Also look for those communities that can point to clear, demonstrable wins – other customers that they’ve worked with to solve real business problems. An important measurement for community sites is whether they include tools that help you manage the ongoing process. It is easy to build a marketplace website that connects an employer with a freelance worker, but this alone does not provide ongoing community-building. It is simply a fancy employment agency. 8. You still call the shots Crowdsourcing does not change a fundamental truth of software design, development and testing: Effective, detailed communication and project management is key to any successful project. This is true in managing in-house resources or outsourced partners, and crowdsourcing is no exception. So assign an internal project owner to keep the information flowing and manage the process. While you don’t have to micro-manage the crowd in terms of tactical execution and idea generation, strong management and executive buy-in enables processes, plans and deadlines to still remain firm when using crowdsourcing.

WHITEPAPER: EIGHT ESSENTIALS OF CROWDSOURCING

6

Dig Deeper & Take Action
Want to capture the power of crowds to launch higher quality apps faster? • Design: Need help with UI or graphic design? o Guru – www.guru.com o CrowdSpring – www.crowdspring.com Development: Want to get your app built with the help of the crowd? o TopCoder – www.topcoder.com o Rent-A-Coder – www.rentacoder.com Testing & Usability: Want to make sure your new app or version is ready for primetime? o uTest – www.utest.com

•

•

Other inspirations for this paper and valuable resources about crowdsourcing: • Wired Magazine – Crowdsourcing article by Jeff Howe • Jeff Howe – Crowdsourcing blog • ReadWriteWeb – Crowdsourcing review • BNet – Crowdsourcing review • Crowdsourcing Links – Crowd-driven companies & news

About uTest
Headquartered near Boston, uTest is the world's largest marketplace for software testing services. The company provides real-world QA services through its community of 15,000+ professional testers from over 150 countries around the world. To date, more than 400 companies - from web start-ups to enterprise software firms - have signed up to get their apps tested by the uTest community. uTest enables companies to launch higher quality products; get their desktop, web and mobile applications to market faster; and control the cost of testing. Customers build their virtual QA team based upon tester experience, location, language, OS and browser. Companies pay only for those bugs that they approve. A brief online demo is available at www.utest.com/demo. uTest can be contacted at: uTest, Inc. 153 Cordaville Road Southborough, MA 01772 p: 1.800.445.3914 e: info@utest.com w: www.utest.com

WHITEPAPER: EIGHT ESSENTIALS OF CROWDSOURCING

7

Summary The whitepaper explains the basic benfits of crowdsourcing application development and explains how companies are using crowdsourcing to design, build and test better applications / products, faster.

Description This paper decribes the emergence of crowdsourcing in the area of applications developing and testing and how its powercan be used to leverage the creativity and diversity of a global community.

Tags: uTest

Events

EIGHT ESSENTIALS OF CROWDSOURCING

Comments

Crowdsourcing.org

Categories

Content

Crowdsourcing.org, an initiative by massolution®

Events

Related Content

EIGHT ESSENTIALS OF CROWDSOURCING

Comments

Crowdsourcing.org

Categories

Content

Crowdsourcing.org, an initiative by massolution®