My boss asked me if there was a way to have Eggplant, or any product, test an application without having to write a specific script for each scenario/bug. The way a human tester would just tinker with a new program plopped in from of them. I figured this was bonkers but it occurred to me last week that there is a very good way to fake it at least but it’s messy.
As anyone ever played with this? Created a script that just keeps clicking randomly or semi-randomly at a program till there is a crash to document?
One approach to this is to model your application as a state machine (various windows, panels, etc). Then do some randomization of the tasks, even if they don’t follow what would be considered a “use case”. We recommend this approach because many bugs aren’t plain crashes and so a model that shows clicking “X” should take you to “Y” and validating that much is helpful.
You can do some web searches on Model Based Testing for more information.
Along those lines, I’ve used the random function in Sensetalk to choose the course of my script. You can also assign weights to what path you’d like to test, to increase the probability of something happening.
Here’s a simple example of randomly choosing 3 paths, and looping 1000 times to pick one of the three. Path 1’s odds are 1 in 3, Path 2 is 1 in 2, and Path 3 is 1 in 6:
set rndMax to 100
set rndMin to 0
set path1 to 1/3 * rndMax
set path2 to path1 + (1/2 * rndMax)
set path3 to path2 + (1/6 * rndMax)
repeat 1000
put random(rndMin, rndMax) into foo
if foo is less than path1 then
# run test path 1
add 1 to p1sum
else if foo is less than path2 then
# run test path 2
add 1 to p2sum
else
# run test path 3
add 1 to p3sum
end if
end repeat
log p1sum / 1000 && p2sum / 1000 && p3sum / 1000
And if I run this I get something like:
0.349 0.502 0.149
Is that something approaching what you were thinking?
no, but that is definitely something worth adding! I was going to essentially use NESTED IF ELSEs to look for certain buttons or menus in our application. If it sees one it will click on it… if it sees another recognizable button or menu item it will continue down any of a variety of decision trees or “Rabbit Holes” as I like to call them.
I would add repeats to it like yours as well to keep the thing running ad naseum.
So that you end up with a script that just keeps banging away all night. If it sees a crash it takes a picture, runs a clean-up script, and moves on.
I’ve also been trying to develop a sense of Model Based testing and apply it to Eggplant but what I’ve read so far seems needlessly heady: