Most of the QA professionals are among the smartest people on this earth who often boast advanced degrees in engineering, mathematics or computer science. In some ways, we are like Superheroes – capable of breaking down a whole application and defeating an army of programmers with our defects, testing complex functions, juggling numerous technologies, ensuring customer ideas morphed into working software, all the while not breaking a sweat. So how is it that despite such technical savvy, programming prowess and quality driven attitude, we are defeated at test estimation front? And it’s when I was forced to ask this question to myself –
Why some super-testers produce dumb estimates?
The problem of test estimation is not straightforward; however, the bigger part of the problem with software testing is doing “estimates” for software that is yet to be designed or incorrectly estimating evolving software. I believe the BIGGEST MYTH about test estimations is:
“Using estimation tool or techniques, perfect estimation can be achieved”
Estimation tool or technique tries to improve the existing estimation or provide an approximate idea of estimation but none can help you with exact estimation. The common misconception is greater dependency on tool and technique than your own calculation based upon real time factors. Test Estimation tools / techniques are a guide, but they are not the whole solution, and should be adjusted to the project characteristics.
Hidden issues leading to Test Estimation failures
I am herewith listing the major issues which lead to test estimation failures.
- Problem 1: Do Companies collect and use empirical data for the purpose of estimation of V&V effort?
- Problem 2: What are the current practices in effort and cost estimation?
- Problem 3: How to address verification, validation, and testing estimations?
- Problem 4: A very common mistake that we often make when planning out work is only looking at the brightest of days.
- Problem 5: How can we improve the current trends in estimations of V&V effort?
- Problem 6: What does the quality of experiments, simulations, and current industry data indicate about the evidence based software cost effort estimation in testing?
After comparing the results obtained from various studies and project learning’s I found that there exists practice for empirical evidence based verification, validation, and testing cost/effort estimations. Also that test estimation follows the same pattern as software development project estimates. It was observed that:
- All the companies prepare separate estimates for test efforts,
- Empirical data is commonly used to estimate test efforts, and
- Test estimation error seems to be closely correlated with development effort estimation error.
Solving the problems to effective Test Estimation
Using my 12 years of testing experience, I would like to offer solutions to above test estimation issues and make a few suggestions to achieve better and realistic test estimation;
- Solution 1: Find best practices of verification, validation, and testing using empirical evidence resulting in practical and realistic test efforts
- Solution 2: Find about overall effort cost estimation scenario.
- Solution 3: Always be aware about the latest research and practical practices going on for estimation processes for test efforts
- Solution 4: Keep an eye on the measures suggested to improve current effort estimations in testing.
- Solution 5: Act on the real insights into overall quality of testing estimations based on evidence based software engineering
- Solution 6: Try avoiding estimation to meet someone else’s rushed deadline and come up with the genuine numbers and prices.
- Solution 7: Take the time you really need to work with your team – and the customer, if appropriate – to estimate the work properly and come up with the realistic numbers and prices.
Also do consider following personal and team factors in accordance with estimation tools / techniques:
- Factor 1: Technical/Domain expertise of testing resource.
- Factor 2: Experience of the test resources
- Factor 3: Dev – QA relationship and compatibility.
- Factor 4: Types of testing to be conducted – Manual/Automated /Performance/Security/Etc.
- Factor 5: Keep into account the unplanned risks – Sudden leaves, unseen emergency/patch release, attrition, resource movement etc.
- Factor 6: Reality vs Assumption balance for man and machine.
- Factor 7: Projects are rarely a piece of cake to walk. Once the design gets going, there are always complexities and missing requirements that come up.
Moving from Theory to Reality
Verification & Validation Testing cost effort estimations is not new in research or industry or among academicians, but since last few years as we started to migrate from Traditional testing towards Agile, it became quite evident and widespread. We have to accept the fact that test estimation is just a theory; as and when we proceed, we realize that whether we over or underestimated cost, time, resource availability and probably a few other things that would adversely affect the project just because we were relying on everything to go “just right”.
Not everyone is good at estimating. Try experimenting yourself – Estimate work before you ask your team members to provide an estimate. See how close you are, and then discuss the differences. You might find you need to work on it, or you might find you’re pretty good at it.
Keep Estimating and please share your thoughts on Test estimation best practices in the comments section below as I learn just as much from you as you do from me.
This is a Guest post by Prashant Kumar | Having 12+ years IT experience as Java Developer, Business Analyst, Consultant and finally – QA professional, Blogger, Mentor. Worked with major IT firms, Investment banks and start-ups across US/UK/Switzerland/Germany/India to deliver quality product/services. Connect at LinkedIn for any QA related mentoring or consulting.