It's a good time to be a quality assurance engineer. Last month, CareerBliss released the results of its annual job satisfaction survey for 2014, where survey participants were asked to rate their jobs in areas like work-life balance, internal office relationships, growth opportunities, job control, compensation and more. Coming in at number two, just behind database administrators, are QA engineers.
QA is a tricky but crucial consideration for startups who want to retain customers and invest less time and money on improving products post-launch, while also keeping their development team as lean as possible. Here are a few critical questions that startups without dedicated QA engineers need to consider.
What is the best QA approach for my company?
This is a broad question, but the simple answer is an agile QA process that anticipates user requirements will evolve and ensures your team can implement changes quickly with minimal disruption to the product and customers. SaaS technology executive Andrea Corey, most recently the VP of Quality Assurance at Eloqua (Oracle), advises startups on their QA processes and she has some valuable tips for getting started.
"The key thinking that needs to [happen at the outset] is to determine what areas need the most test coverage - a process that is core to the product's value proposition needs to be covered well by tests; a simple administrative screen that is used infrequently does not need to be well-covered," says Corey.
She recommends brushing up on processes described by software testing specialists like Brian Marick in 2003, also adopted by professionals like Lisa Crispin, which outlines four quadrants of focus that are divided into two dimensions - the first is whether the tests aim to support the development process, versus whether the tests evaluate the product; the second dimension is whether the tests are business or technology-focused. Corey describes the four quadrants as follows:
Q1 - Technical tests that support the development team and process, such as component and unit tests.
Q2 - Business-facing tests such as discrete functional tests or more elaborate story tests that exercise a business process.
Q3 - Business-facing tests that describe the quality of the product from a user standpoint - usability tests, exploratory tests. These are primarily manual tests and typically require a dedicated QA team member who has experience in these areas.
Q4 - Technical tests that evaluate the limits of the product - such as performance and load tests, as well as security tests. These also require specialized skills and dedicated focus.
How extensive should my QA process should be?
To properly utilize resources, startups usually conduct manual testing on their dev server/QA environment, which involves testing per issue before testing integration of the changes with the entire application. Automation is also an option, but that usually involves writing a separate program to conduct the tests, which can distract the development team from putting the product first. Automation becomes a more feasible option once you scale out your development team. When you scale out your dev team to include dedicated QA, you should have one QA engineer for every two developers.
When should I start doing QA?
Deciding when you should start implementing a process for QA depends on the size and nature of the organization, but definitely once you have your first paying customer. Even if you don't have the bandwidth for a dedicated QA engineer, you should be doing daily testing for bugs on any product that will be delivered to the hands of customers. The most successful companies ingrain QA processes into their company culture - that way everyone is building higher quality products from the get-go, rather than playing catch up when it's time to launch.