Once upon a time we had a finicky client manager with no understanding of testing logic. He just wanted us [a team of 4] to execute thousands of test cases in a week, that too maintaining quality. Is it possible in any real-world? No, right? There was a tiff between the Test Manager and Client Manager and that’s when I got the real-time exposure to Risk-based Testing, a win-win idea!
What is Risk-based testing?
As the term goes, Risk-based testing is nothing but mitigating the risks or threats to software product. Simple analogy – It’s like a heart patient will follow regular exercise regime to mitigate any risk for a heart attack. One of the popular Test strategy or approach, risk-based testing prioritize the testing effort based on the risk analysis. Risk-based testing is getting more popular now-a-days since there might not be sufficient time to test all the functionality, i.e. prioritize & test the functionality in order which has the highest impact and probability of failure.
- Risk = Probability * Impact
Risk-based testing involves prioritizing the features, modules and functions to be tested based on impact and likelihood of failures. It involves assessing the risk based on the complexity, business criticality, usage frequency, visible areas, defect prone areas, etc.
Risk-based Testing Process
As you might have guessed risk-based testing involves,
- Identify all functional modules of AUT (Application under test), analyze and review the requirements (SRS, FRS, Use cases)
- The first step to solving a problem is identifying it. Identify & Prioritize project, software & module risks – how do you prioritize? Based on the probability of occurrence and subsequent impact. Evaluating critical business modules is a first step in prioritizing tests.
- Test based on the above risk-analysis exploring each risk priority-wise. E.g. Module 2 & 5 should be tested first and rest can be taken care of depending on the time left.
- Mitigate risk or prepare a contingency plan
- Redo risk analysis and re-adjust the test efforts
Note: Risk identification & prioritization can be done through risk workshops, checklists, brainstorming, interviewing, Delphi technique, cause and effect diagrams, lessons learned from previous projects, root cause analysis, contacting domain experts and subject matter experts.
Mitigation and Contingency
- Mitigate – How do you alleviate or lessen the risk? I.e. taking steps to reduce the likelihood of risk outcome or adverse effects by accepting or avoiding or controlling or transferring the risk.
- Contingency – Say a particular risk cannot be mitigated. What now? We can at least prepare for it, right? I.e. identify work-around or back-up plan to risks or missed defects in order to minimize the impact.
- Risk Management should be part of the overall testing process, hence it should start early in the life cycle – Guide Test planning, preparation & execution efforts based on the risk-analysis
- Risk-based testing does not ‘eliminate’ risk – just follow best practices in risk management to achieve a project outcome that balances risks with quality, features, budget and schedule.
- The most common situation to utilize Risk-based testing – in case of time, resource or budget constraints as in many cases for QA 🙂 to get the best possible testing done.
- Risk-based testing can be used at every level of testing, e.g. component, integration, system, and acceptance testing
- Risk-based testing can also help in identifying proactive opportunities to remove or prevent defects even before actual test execution starts.
- The development team’s input is very important, i.e. what might need additional verification
- Always maintain traceability between risk items, tests that cover them, test results, and subsequent defects.
- Risks can be of different types – Business or Operational, Technical, External or E-business failure-mode related
- Depending upon the industry, the risk factor may change but the concept remains the same; except, of course, if you are in the pace-maker industry and you have zero risk-taking ability.
- Testing strategy, goals and directions should be focused and continuously adjusted against problem areas throughout the duration of test cycle by continuously monitoring the risks.
Benefits of Risk-based testing
- Firstly, it prioritizes tests against deadlines i.e. higher priority areas / critical functions of the application are tested first thus leading to improved quality.
- In case of limited cost, time and resources – Risk-based testing is a better way to accelerate the testing effort while still managing the risk. Efforts are not wasted on non-critical or low risk functions.
- Testing becomes a much more targeted and organized activity since less but more efficient test cases can be specified.
- Improved Market opportunity (Time to market) and on time delivery.
- Continuous risk monitoring and assessment throughout the project’s entire lifecycle helps in identification and resolution of risks and address the issues that could endanger the achievement of overall project goals and objectives.
- Problem areas are discovered early. Test cases can be reduced and focused on the most critical areas. Preventive activities can be started immediately.
- It also helps reduce schedule slippage, helps define severity of defects, and is a great way to flag test cases for regression testing.
- Risk-based testing is a great tool for prioritizing test execution and selecting candidates for automation.
As you might have guessed – what if not all risks have been identified? Or not including right personnel for risk assessment? Incorrect prioritization? Lastly, what if a low risk becomes a reality and causes a problem in the future?
One interesting fact is that unknowingly testers have always utilized risk-based testing but they have done it in ad-hoc fashion based on their personal expertise. Somehow over a period of time, we as testers get to know the inherent priority features of the application-under-test. But still we don’t utilize formal risk assessment methods to derive priority.
One of the seven Testing principle is “Exhaustive Testing is not possible”, i.e. you can always find more to test. Thus the goal of Software Testing is to find the most important defects first, and to find as many such defects as possible. In today’s era when IT industry is running agile and the majority of stakeholders are looking for quicker solutions – Risk-based Testing is a scientific & practical test approach to prioritize the efforts with limited resources, ensuring optimum quality from a business perspective.