Like it or not, today’s end users expect their applications to work perfectly from the moment they are downloaded. Naturally, these heightened expectations have placed a great deal of additional stress on in-house QA and engineering teams, who are now tasked with finding and fixing more bugs – on tighter deadlines – across a wider range of environments.
Consider, for instance, the mobile testing matrix. With thousands of possible device, carrier, and platform combinations, companies are quickly discovering that it’s almost impossible (not to mention expensive) to achieve adequate coverage with internal resources alone.
For this reason (among others), there’s been a huge shift to crowdsourced testing in the last few years. With this method, certain testing functions are conducted by members of a global community, resulting in greater coverage, faster time-to-market, and lower QA costs.
While the results of crowdsourced testing are generally positive, there are a few things you’ll want to avoid. With that in mind, here are four tips for testing your apps with the crowd:
- Tip#1 - Know Your End Users: Perhaps the greatest advantage of crowdsourced testing is that it enables companies to test their applications with people who closely resemble their typical end users. So if, for instance, your audience is shoppers in the UK with an iPhone, why would you want Blackberry users from Brazil testing your product? Or developers with emulators in a California lab? The key takeaway here is to understand your target demographic before your project begins. This way, you can receive bugs that your real end users would likely encounter.
- Tip#2 – Identify Your QA Gaps: Another key benefit of crowdsourced testing is that it helps companies solve very specific problems, particularly in terms of testing coverage. Whether it’s location, language, operating system, or some other criteria, it’s extremely important to know the shortcomings of your internal QA team. Without this, your bugs and feedback will lack the precision that crowdsourced, In-The-Wild Testing can offer. Take a look at some of your most recently reported bugs and make a list of the similarities. Knowing where gaps exist is a major advantage when considering crowdsourced testing as an option.
- Tip#3 – Call the Shots: Crowdsourcing does not change a fundamental truth of software design, development, and testing: Effective, detailed communication and project management are key to any successful project. This is true in managing in-house resources or outsourced partners, and crowdsourcing is no exception. So assign an internal project owner to keep the information flowing and manage the process. While you don’t have to micromanage the crowd in terms of tactical execution and idea generation, strong management and executive buy-in enables processes, plans, and deadlines to still remain firm when using crowdsourcing.
- Tip#4 – Be Specific (or not): Many times, a company delving into crowdsourcing will have very specific tasks it needs completed. While crowdsourcing is a great way to achieve test case execution on a large scale, companies also find it beneficial to let the crowd explore an application at their own discretion. With a diverse community of professionals that transcends location and background, you can avoid the groupthink that often stifles internal teams. Homogenous internal teams – even those comprised of smart, hungry, talented people – are often less effective at discovering new issues. Again, a global community brings diverse opinions and experience (as well as fresh eyes), which can result in creative development solutions and more complete testing coverage.
Companies are quickly realizing that the only way to launch apps that consistently work in the hands of users and meet their expectations – apps that are functional, reliable, secure, and intuitive from the very first download – is to move a portion of testing into the wild via crowdsourcing. Make sure to keep these tips in mind for your next crowdsourced testing project.
Today’s post is contributed by Doron Reuveni. As CEO and co-founder of uTest, Reuveni is the driving force behind one of the fastest-growing tech companies in America. Although his idea of “in-the-wild” software testing through a crowdsourcing model was a radical notion in 2007, it has been enthusiastically adopted as an integral part of the software development process for 1,000+ companies worldwide. With more than 25 years experience, Reuveni has become an internationally-recognized expert in software development and testing, as well as startup formation and entrepreneurship.
Doron Reuveni, firstname.lastname@example.org, @doronr
uTest provides in-the-wild testing services that span the entire software development lifecycle – including functional, security, load, localization, and usability testing. The company’s community of 60,000+ professional testers from more than 190 countries put web, mobile and desktop applications through their paces by testing on real devices under real-world conditions.
Thousands of companies – from startups to industry-leading brands – rely on uTest as a critical component of their testing processes for fast, reliable, and cost-effective testing results.