The world is driven by beliefs. It may be in God or Technology. Over a period of time every society lay down a set of philosophies or values for better organization & operation. Similarly, any technology has its own fundamental principles which have been proven right as the time elapsed. ‘Software Testing’ is no different – it also has a set of 7 fundamental principles that are proven right over the time.
1. Testing shows the presence of defects
In simple terms – Software Testing is the activity of finding defects. Flipping the coin, it doesn’t mean post-testing the application is defect-free. As we say in life – ‘There is always a scope for improvement’, similarly even after rigorous testing there is always a scope of defect occurring or further optimizing the application. The goal of testing is clear – to identify ‘as many’ defects as possible.
Software Testing is not a proof of correctness, it’s all about the assurance that most of the defects have been identified (and rectified) thus reducing the probability of undiscovered defects that might occur in live environment. Why do you think companies don’t sign a contract for a software to be defect-free after testing OR why do they have warranty period even after the release?
Please note this doesn’t mean you can use this principle as an excuse for poor quality, instead focus on designing the test cases that have maximum coverage and identify ‘maximum’ defects!
2. Exhaustive testing in not possible
Exhaustive as in complete or in-depth testing is not possible. Why? You don’t have all the money, time & resources at your perusal – the budget is limited, timelines are defined, resource count is fixed, P&C are quite huge, etc. Even a simple text-box with 5 possible values need data tests, UI tests, security tests, compatibility tests, etc. Keeping different factors in mind, it is not possible to test all the possible combinations, flows, data & scenarios.
What’s the way out then? Prioritize based on the Risk analysis! Use different Test Design techniques to optimize test cases. Focus is to derive optimal amount of testing required based on the risks & priorities, such that it uncovers maximum defects focusing on the most important aspects first.
Caution: Your quality of testing depends upon the risk-analysis, prioritization & techniques used.
3. Early testing
Say client asks to develop a login page for his application. Developers build a page with Username & Password fields and deploy it for testing. During testing you observe that there is no ‘Forgot Password’ link provided. Come on, it’s obvious! What do you do? Simple – raise a defect >> client says it’s obvious >> requirements are detailed >> developer changes the code to fix it >> you retest it & close.
Now imagine Testing team is involved from the project start itself. What do you do? As soon as requirements are received, you raise an issue with business team about the ambiguous, missing or confusing requirements >> Business team details out the requirements clearly. Result – It is easier & cheaper to fix the issue if identified earlier in the cycle. Hope you got the point 😉
Software Testing should start as early as possible in the Software Development Life Cycle. E.g. it is much cheaper to change an incorrect requirement than having to change a functionality in a large system that is not working as requested or as designed! That’s why we have requirement analysis, unit tests, design discussions, impact analysis, reviews, etc.
4. Defect clustering
Ever heard about the ‘Pareto principle’, also known as the 80–20 rule? It states that for many events, roughly 80% of the effects come from 20% of the causes.
- 80% of your sales come from 20% of your clients
- 80% of a company’s sales come from 20% of its products
- 80% of the traffic occurs during 20% of the time
Many natural phenomena (population, wealth, sports, business, technology, etc.) have been shown empirically to exhibit such a distribution. Applying this principle to Software Testing – 80% of the defects are found in 20% of the modules. Yeah! That’s roughly true. And how do you identify such modules? It comes with experience my boy 😉 (Or girl)
Trivia: Management consultant Joseph M. Juran suggested the principle and named it after Italian economist Vilfredo Pareto. Pareto noticed that 80% of Italy’s land was owned by 20% of the population. He then carried out surveys on a variety of other countries and found to his surprise that a similar distribution applied.
5. The pesticide paradox
Trivia: The pesticide paradox states that applying pesticide to a pest may end up increasing the abundance of the pest if the pesticide upsets natural predator–prey dynamics in the ecosystem, i.e. the target pest has a naturally occurring predator that is equally affected by the pesticide.
Or in other words, if you keep running the same set of tests over and over again the software gets immune to testing, i.e. as the system evolves, many of the previously reported defects will have been fixed and the old test cases cannot find any new defects.
What’s the way out then? To overcome this “Pesticide Paradox”, the test cases need to be regularly reviewed & revised, adding new & different test cases to help find more defects.
6. Testing is context dependent
Context as in the environment, i.e. application type. All the developed software’s are not identical. You might use different methodologies, approach, method, techniques and types of testing depending upon the application type, whether it’s a Game, Mobile App, Cloud-based, Web-based, desktop application, embedded system, etc. After all testing an ATM machine will be different from testing Flipkart.
Click here to understand the difference using an example.
7. Absence of errors fallacy
This is a bit tricky 😉 First of all don’t confuse this principle with 1 & 2. In simple terms:
- Testing is to identify defects, not to confirm that there are no defects.
- Even after complete testing, defects might be present in the system.
Coming to absence of errors fallacy, imagine you developed an e-commerce system & tested it completely. All the identified defects have been fixed & retested. 99% of the defects have been rectified. Management is pretty sure about the product quality w.r.t. defects. Now when the system is demonstrated to client, what if you get a feedback saying “Though it is so-called defect-free, but still this is not what I wanted. I wanted a simple UI that can handle the user load”?
Yeah! Everybody in the Test team was confident about the product ‘quality’ (absence of errors) but at the end it proved to be false (fallacy) – the system is not usable, it doesn’t fulfill client’s expectations 🙁
The lesson: Client’s requirement & expectations are as important as product quality. And it might change over time. Why do you think Agile methodology is so popular now-a-days? It allows continuous interaction with business users and rapid prototyping (minimum marketable feature or MMF) prevents unhappy clients at the end.
Now that you know the 7 fundamental principles of software testing, don’t forget the key take-away:
- Focus on designing the test cases that have maximum coverage and identify ‘maximum’ defects.
- Your quality of testing depends upon the risk-analysis, prioritization & techniques used.
- Software Testing should start as early as possible in the Software Development Life Cycle.
- 80% of the defects are found in 20% of the modules. Yeah! That’s roughly true. And how do you identify such modules? It comes with experience.
- The test cases need to be regularly reviewed & revised, adding new & different test cases to help find more defects.
- You might use different methodologies, approach, method, techniques and types of testing depending upon the application type.
- Client’s requirement & expectations are as important as product quality.