We all had a nightmare with “intermittent” bugs at some point in our testing careers. Seasoned testers call them wittily “Heisenbugs“.
Context
We know that these intermittent/elusive issues appear and disappear unpredictably, making them difficult to diagnose and fix. Forget about diagnosing and repairing; politically speaking, the responsibility of proving that the “bug really exists” is on the tester. And when you try to reproduce, you can’t. Sounds familiar?
Enter Generative AI (GenAI), a cutting-edge tool with immense potential for software testers. In this post, we will discuss how GenAI can revolutionize the way we approach intermittent bugs, complete with a practical example you can try using the free version of ChatGPT.
Problem/Solution
Intermittent bugs are challenging to catch because they do not consistently appear under the seemingly same conditions. GenAI can help by simulating a wide range of conditions and user behaviors, increasing the likelihood of reproducing these hidden bugs.
By leveraging GenAI to generate diverse scenarios, testers can systematically and efficiently identify the conditions that trigger these bugs that play hide-and-seek with us.
Situation
Imagine you are testing a web application in which testers/users occasionally experience a server-side crash when uploading PDF files. Let us say that traditional debugging methods have failed to reproduce the issue, and even testers are not able to reproduce it consistently. Yes, this can be classified as an “intermittent bug”. Developers pray that it does not occur again, while testers want to prove it can. Or someone wearing both hats is truly upset!
GenAI Solution with ChatGPT (or your choice of GenAI tool)
Here are 4 steps that you can repeat step-by-step in most cases of analytically reproducing intermittent bugs.
1. Describe the Intermittent Bug
Clearly outline the observed behavior, environment, and any known conditions of the intermittent bug.
(For ChatGPT)
—-
“Intermittent bug observed – The server-side process crashes occasionally when users upload PDF files through the web application. This happens randomly and is not consistently reproducible. The upload process involves validating the PDF, extracting data, and storing it in the database.“
—-
Note: Of course, you can be more descriptive, and I wrote the shortest yet sensible line for the quick demo.
2. Craft the Prompt for ChatGPT
Create a detailed prompt for ChatGPT to generate a wide range of test scenarios.
(For ChatGPT)
—-
“Generate a variety of test scenarios to reproduce an intermittent bug that causes the server-side process to crash when uploading PDF files. Include variations in file content, file size, user behavior, network conditions, and system load.“
—-
3. Interact with ChatGPT
Combine the narrations/instructions in #1 and #2 above:
(Enter in ChatGPT)
—-
“Intermittent bug observed – The server-side process crashes occasionally when users upload PDF files through the web application. This happens randomly and is not consistently reproducible. The upload process involves validating the PDF, extracting data, and storing it in the database.”
“Generate a variety of test scenarios to reproduce an intermittent bug that causes the server-side process to crash when uploading PDF files. Include variations in file content, file size, user behavior, network conditions, and system load.”
—-
Please enter all of the above colored instructions (Prompt) into ChatGPT (or your favorite GenAI tool). You will now get a detailed response. I am not pasting what I received here, but for sure you will see that The response will ideate you with the below possibilities:
- File Content Variations – Different content types, encryption settings, and corrupted files.
- File Size Variations – Various sizes, extremely large files, and multiple simultaneous uploads.
- Uncommon User Behavior – Rapid uploads, navigation interruptions, and different browsers/devices.
- Network Conditions and System Load – Varying speeds, network interruptions, peak usage, and high server load.
At this point, you can enhance your prompt and even get more refined possibilities that could lead you to reproduce the intermittent bug.
4. Execute and Analyze
Needless to say, use the generated scenarios to systematically test the feature(PDF uploads) again. Document any crashes and conditions under which they occur. Send any associated artifacts(PDF files and sequences/configurations in this case) to the development team.
Think of the broad range of possibilities, both GenAI provided and your own, and you will naturally increase the chances of reproducing the intermittent bug.
Takeaway
Intermittent bugs are challenging but not unbeatable. By leveraging GenAI tools like ChatGPT, we can systematically explore a wide range of conditions, significantly improving the chances of reproducing and ultimately fixing them.
Try it out yourself! Open ChatGPT and use the above provided prompts or customize them to fit your testing scenario. Observe how ChatGPT assists you in generating diverse scenarios to uncover the conditions that trigger the hidden bugs.
