Now that’s a tricky one. Many a times we face these kind of one-off defect 🙁 which peep-out and then hide somewhere. “It was a one-off bug and now not reproducible – so what can I do?” Wrong! Though one-off but still it is present somewhere in the software and as a Tester it is our responsibility to investigate it. How?
- ‘Think’ of the exact steps and then try to replicate.
- Environment plays a vital role. Think of the environment, what has changed since then.
- Logs! Yes logs are a great source of narrowing down on the specific module.
- Collaborate with a developer to validate the specific code.
- Test Data can be a root-cause.
- Identifying if there is a pattern to it. Say, after every 3 attempts.
Sometimes these one-off bugs can prove costly in the live environment. So it is important to analyse carefully. What say? How do you tackle the one-off bugs?
Note: Invest your time in a one-off bug depending upon its severity. It doesn’t make sense to work on a low one-off defect for a complete day 😉
Rajeev Vaidya | Software QA veteran. Aspiring data scientist.
Rightly said. You may want to create a forum internally in the team to get help on reproducing the defect. Also, document how such intermittent defects got reproduced and keep as the reference in your repository.
Jaimin Shah | QA Engineer at Lodestone Software Services
If a bug is tricky and is reproducing at random, I’ll simply jot down the steps and test data with all other info and then communicate with developer to see if any changes in code had actually happened. Also will debug entire flow. Hope I’d covered all possible points. If not please rectify me, so I can have better understanding.
Amaresh Kumar Rout | Software Engineer at Innodata India
One should able to read log from the console with the specified time the error occurred and try with similar kind of data to replicate it again.
Deepanshu Arora | Senior Quality Assurance Executive at Exclusife
We can use Screen Recording Software, which may help in backtrack that bug, and reproduce again.
Ramdas Krishna Baliga | Staff Software Engineer in Test at Intuit
One of the things I suggest to my colleagues is taking notes – (since my domain is mobile app) – OS version, device info, even minute details like did u put app to background, any interruptions like message, call, mark when u cover important flow, data that you enter…with this you don’t really need to try to reproduce, just continue with regular testing.. when ever it comes up again you would have some data to work on.
Gerry Isaacson, MBA | Entrepreneur, President and Contract QA Consultant
When you say “if a defect is not reproducible” your are saying I don’t have a detailed script that concludes with the defect, and I don’t have the test data exactly defined then the tester should not waste everyone’s time by calling it a defect. If testing is done correctly and the results are documented correctly then the tester and/or developer will be able to duplicate the defect. Anything less is a waste of everyone’s time.
Jeff Nyman | Quality and Test Specialist
But what about helping avoid these situations? Testers should, for example, know about correlation IDs.
Basically, I might have multiple services collaborating to perform some operation as part of the same logical transaction. So it’s necessary to understand the “call journey” through the call stack.
A correlation ID is passed through all calls through the system. Each service passes along this ID as part of its operations. When this done via logging, now your logs become part of your data strategy. This also becomes a way to design real-time call graphs of your system: documentation.
So now, as a tester, we’ve done a couple things: worked with development to get a discovery mechanism in place; create a data strategy; and provide some documentation as well. This kind of overall technique is very helpful for not just mitigating transient or intermittent bugs but also being able to better understand how they can lurk in the system to begin with.
Harsh S. Kulshrestha | Application Developer at ThoughtWorks
One of the reasons why one can’t reproduce the bug easily may be because of the fact that the bug surfaces only when the system or it’s memory is at a uniquely high utilization. May be there was a memory leak in the code, or may be the main thread spawned too many child processes.
One way to identify such bugs could be to run automated test suites in such scenarios. Let them create a load that the system is usually not accustomed to. Who knows the bug may pop up right there!
Jeff Nyman | Quality and Test Specialist
A key thing also is to have good heuristics for looking for intermittent bugs and transient bugs. Those are two very different categories when it comes to conditions for bugs. Some bugs of these sorts can be “spatial” (meaning occurring in relation to other components of the system) or “temporal” (meaning occurring in relation to time) or both.
But often they can leave traces. So, just like a historian or an archaeologist, testers have to sometimes look for the evidence that bugs leave behind. You mention logs, which is certainly one of those.