uTest: The Alternative to Outsourced Software Testing
How crowdsourced testing has changed the game for innovative software companies
A Free eBook from uTest
The Problem with Outsourcing
T
wice the headache for half the price. This is how a growing segment of the software industry has come to regard
outsourced or “offshore” testing. Tired of being burned by time zone challenges, language barriers, inflexible contracts, hidden costs and a loss of control and quality, software execs are abandoning these offshore firms at a time when comprehensive testing coverage has never been more important. For many companies, their alternative is crowdsourced testing with uTest. As the world’s largest marketplace for software testing services, uTest offers companies of any size a more efficient way to get their web, mobile and desktop applications tested: On-demand, through a global community of professional software testers. Who benefits? This eBook offers a side-by-side comparison of crowdsourcing vs. outsourcing in the following areas: •Cost Containment: Why pay for testing services you don’t need and won’t use? Learn about uTest’s on-demand pricing. •Quality: How do you know you’re getting the best testers and their best efforts? It’s easy with uTest’s global community. •Communication: Each customer gets a dedicated uTest project manager who works closely with your in-house QA team. •Control: Don’t compromise accountability or visibility. Discover more about how uTest works. •Flexibility: What happens when your development schedule or scope changes? See how quickly and easily uTest scales.
A Free eBook from uTest
Page 1
Cost Savings
The problem with outsourcing: Companies that seek to save big with an offshore testing firm are likely to pay big in the long run. While the contract may have a fixed price, the hidden costs don’t remain hidden for long. Consider the true cost of an inflexible testing contract. Suppose your testing priorities change before your contract ends – think of how much time and effort you’ll spend to renegotiate the scope of work and try to get the offshore team re-focused. Are you willing to pay for downtime and other wasted offshore QA resources? Now that they have a choice, the answer for many firms is a resounding, “NO”. The uTest advantage: Founded and staffed by former QA managers and executives (who’ve experienced the true costs of offshore testing first hand), uTest employs an on-demand pricing model, where companies pay only for the services they need. With a minimum commitment of one test cycle (“packs” of test cycles can also be purchased), companies now have a risk-free way to outsource their software testing that doesn’t involve any hidden costs. By offering global testing coverage at a lower price than offshoring (see sidebar for more details), uTest is an ideal solution for software companies looking to reduce overall costs without sacrificing quality. On cost savings: “We wanted the ability to test across browsers and platforms in a way that could save us time and money. uTest enabled us to do just that.” - Allen Lin, QA Manager, Internet Brands
What’s Included In A Test Cycle From uTest?
Functional testing of a web application, for example, would consist of: *Customized testing team - Location - Languages spoken - Hardware & software - Industry expertise *Cross-browser testing - IE / Firefox / Chrome / Safari *Cross-OS testing -Windows / Mac / Linux *Bug-tracking system integration *Dedicated project manager - Test planning - Selecting testing team - Managing testing team *Comprehensive bug report *Project management scorecard
Inquire about pricing >>
A Free eBook from uTest
Page 2
Quality
The problem with outsourcing: If you plan on contracting with an offshoring firm, be prepared to spend a significant amount of time dealing with the transfer of knowledge. Offshoring firms are often hampered by extremely low retention rates, meaning the testers originally assigned to your team will likely not be there when you need them most. This naturally diminishes the overall quality of testing, as more senior members are routinely swapped out with newer team members who are unfamiliar with your application.
Maximum Coverage
Through our platform, any company can build a virtual QA team and test their application across: •Locations •Languages spoken •Operating systems •Browsers •Plug-ins and anti-virus programs •Handset makers, models and wireless carriers (for mobile applications)
The uTest advantage: With more than 20,000 testers in 150+ countries, there’s certainly no shortage of talent in our community. And unlike our offshoring counterparts, uTest enables its customers to know (i.e. handpick) which testers are active on their release at all times – ensuring both transparency and accountability through an easy-to-use online platform. This means that you get the testers that best match your testing requirements and you can invite the same testers back again and again.
Now you can ensure the quality of your application across multiple operating systems, browsers, languages and locations. No matter what type of testing will improve quality (functional, usability or load & performance) uTest has QA professionals specializing in that field. On increasing testing coverage: “uTest provided us with much better OS and browser coverage and they did it in a much shorter time frame than a traditional outsourcing firm could offer.” – Peter Clegg, Director of Product Management, Move Networks
Learn More >>
A Free eBook from uTest
Page 3
Communication
The problem with outsourcing: For companies that are accustomed to communicating in a fluid, real-time manner, offshore testing has proven to be highly problematic. Time zone differences and language barriers are just two of the factors contributing to the lack of effective communication that plagues offshoring firms. While causing needless delays and making it hard to adapt to on-the-fly changes in the testing cycle, such factors tend to have a spill-over effect into other areas product development as well. The uTest advantage: In addition to having a dedicated project manager and customer support (in multiple time zones), uTest enables the in-house QA manager to contact testers directly through our online platform - greatly improving the communication process. If a bug or survey response seems vague or out-of-scope, or if an urgent issues arises, you’ll no longer have to wait for the team liaison to get to the bottom of it. The benefits of this feature have proven to be particularly valuable for customers that adhere to the agile development methodology, where tight deadlines must be met in order to achieve their stated goals. On ‘real time’ testing: “We’d heard about getting bug reports back in as little as 48 hours with crowdsourcing, but until you see it actually happen, it can be hard to envision. We were very impressed with the process.” – Travis McElfresh, VP of Technology, 1Cast
Consider This Before Deciding To Offshore:
“If your project or application is at all open-ended, fluid or ambiguous, then offshoring will prove to be very problematic. You’ll have an extremely difficult time adapting to the way these firms do business.”
-Daniel Merriman, QA exec with 10+ years of experience
A Free eBook from uTest
Page 4
Control
The problem with outsourcing: Software companies that contract with an outsourcing firm often find themselves on the outside looking in when it comes to control of the day-to-day testing process. If you are comfortable handing over responsibility to the team liaison – the person who will be selecting testers, writing test scripts, setting deadlines, etc. – this may not be such a problem. But for the other 99% of us who want to maintain control and visibility into the testing process, an offshoring firm might not be the right fit.
The uTest advantage: While our project managers are there to assist you every step of the way, it is the uTest customer that calls the shots during the QA process - from uploading and writing test cases, to defining the scope of work, adding your list of known bugs and reviewing bugs. And for those companies who don’t have all of this QA documentation, uTest will even help you create it! The uTest platform was specifically designed with the in-house QA manager in mind, so if you’re looking for an on-demand extension of your in-house testing efforts, look no further. On calling the shots: “With uTest, I specified the exact type of testing I needed, approved the bugs myself and moved on to the next release. I was in control the entire time.” – Cosma Dinca, QA Manager, IMinent
uTest Works With You
We’ve integrated our platform with popular bug-tracking systems like Jira, Rally and Bugzilla, so we’re an ideal complement to inhouse QA teams looking to ramp up during peak release periods.
Learn More >>
A Free eBook from uTest
Page 5
Adaptability
The problem with outsourcing: When confronted with sudden changes in their development cycle, companies face an even greater challenge when contracting with an outsourcing firm. When the scope or direction of testing begins to deviate from the original vision – something that happens to most agile shops and start-ups – the outsourcing contract and team does not. This makes it difficult for offshore firms to adapt to your evolving testing needs. So unless your app development and testing proceed like clockwork every time, exercise caution when considering an outsourcing firm.
The uTest advantage: Since uTest employs an on-demand model, companies can alter their testing requirements at a moment’s notice. And because uTest already has a wide-range of software testing expertise – with 20,000+ QA professionals from around the globe – we can easily accommodate these changes without any additional costs or delays. The flexibility of crowdsourced software testing has helped hundreds of companies to ramp up (or down) their testing when it’s needed most by enabling them to purchase test cycles, rather than committing to a long-term contract. Simply put, uTest was designed for present-day software companies. On uTest’s adaptability: “With uTest being able to put such a large group of testers on our release, we were able to meet a deadline that would have otherwise been impossible.” – Paul Schuytema, Extension Specialist, University of Illinois
What We Test
Thinking of producing a mobile version of your popular web or desktop application? Our community is the best place to find mobile testers to validate your app across location, language, handsetmaker and carrier – global coverage for a global user base.
Read About Mobile Testing >>
A Free eBook from uTest
Page 6
uTest Case Study: Internet Brands
As a leading online media company, Internet Brands operates 150+ community and e-commerce websites, attracting more than 49 million visitors per month for a total of 800 million unique page views. After searching for a cost-effective way to increase testing coverage, the company discovered uTest - performed their due diligence - and signed up as a customer in early 2009. Being an agile development shop – “Our average test cycle is less than a week,” said QA Manager Allen Lin – Internet Brands needed the ability to locate testers in specific regions for assignments on extremely short notice. Since he oversees the QA process for all of the company’s 150+ sites, Allen also needed the ability to temporarily supplement existing in-house QA resources by allocating testing coverage to various projects. For those reasons, he said, uTest made perfect sense. Speaking of cents, Allen was fully aware of the costs of testing. So when he called on uTest to provide functional testing on new websites and major releases, he freed up the company’s in-house QA resources for test case creation, advanced testing tasks, and scripting for automated test tools. In fact, Allen figured out that in less than 12 hours, the uTest community can perform testing that would take a member of his in-house team an entire week for roughly the same cost. On global testing coverage: "With uTest, we are able to test projects for specific regions and countries. We’ve been very pleased with the testing coverage uTest has provided us."
Customer Spotlight:
Location: El Segundo, CA Industry: Web properties Company Size: 600+ employees Methodology: Agile; weekly sprints
Testing Project @ a Glance
Testers: 50-100 per test cycle Testing Type: Functional App Type: Web Browsers: Across all types & versions (IE, FF, Chrome, Safari, etc.)
Allen Lin, QA Manager, Internet Brands
For more uTest case studies, click here. For a sample list of uTest’s customers, click here.
A Free eBook from uTest
Page 7
Summary
Software companies have come to realize the pitfalls of outsourcing are no longer a necessary evil. Because of no clear-cut alternative, there was a time when most businesses reluctantly accepted the hidden costs, poor communication and other problems of outsourced testing, but that time is rapidly coming to an end.
With uTest, software companies of all sizes now have the ability to: - Achieve Real-World Testing Coverage: Build your own virtual testing team - Get to Market Faster: Complete test cycles in as little as 48 hours - Increase App Quality: Test your app by OS/browser, location, language, etc. - Control Costs: uTest’s customers pay only for completed test cycles
uTest in the News
Click on the logos below to read some of our latest media coverage:
Unfortunately, far too many companies still accept the idea that software testing cannot be as lean, agile and flexible as other parts of their business. They continue to view traditional outsourcing as the only alternative to an inhouse QA team and lab. And they’re paying a hefty price for this misconception in terms of money, time and effort.
To learn more about leveraging uTest to supercharge your testing efforts, you can contact one of our QA gurus or call us at 1-800-445-3914.
A Free eBook from uTest
Page 8