How can I click (automate) random elements?

This is a gaming application. Please see the attachment.

Let me explain how users would play…

  1. User gets Start Button.

  2. User would see the screen similar to attachment. This attachment has some jewels and some fish

  3. This screen will show up for very brief time and converted like rest of the fish. In this few seconds, user has to remember which of the fish has Jewels underneath.

  4. These fish will move randomly for few seconds and stop.

  5. Now the user’s job is click the correct Fish that has Jewels underneath.

Could you please tell me How I can automate such kind of dynamic application?

If it is too complex, then if I use QA Cheats, the Fish with Jewels are visible. If we can automate that then it would be great.

I am OK with QA Cheats so we can see the fish with Jewels (please see With_QA_Cheats attachmetn)

It is possible to just click on a jeweled fish. If that is the goal of the automation then with EP get a SUT connection and capture mode the screen. Capture the representative image you will use, set the tolerance properly, and then do the job.

Moving objects can be a challenge. I have written automation to drive and demonstrate games like breakout and pong before with EP. So doing what you want is possible, but likely a bit of challenge due to the transparency of the fish and changing (likely) background colors.

So, grab as little as possible of the jewel itself, and use that as your target to search for. If the objects are not moving too fast, you likely can then use the ‘get EveryImageLocation(fishjewel)’ and then randomly click on any of the locations of the jewels.

set the imageSearchTime to 5 seconds  -- max time to find an element
repeat forever
	get EveryImageLocation(ImageName:jewelcore, Tolerance:60)
	if it is an empty list then exit repeat
	Click any item in it
end repeat

I found that the tolerance of 60 found all instances of your jewel in the fish presented in the first photo. I have also found a precise setting of tolerance found all of your jewel images in the second snap, where all the jewels are out from behind/inside the fish. In this case, a best practice is to grab as small rectangle as possible of ONLY the jewel portions, nothing else. To find out the tolerances you can use Eggplant’s search facility in the ImageDoctor, or write your own loop as follows…

set the scriptLogging to off  -- for our purposes, no loggin' is good loggin'
set the imageSearchTime to 1 second  -- something small
repeat with t = 100 down to 1
	set r.(number of items in EveryImageLocation(ImageName:"jewelcore", tolerance: t)) to t
	if t mod 10 is zero then write "[" & t & "]"
	write "."  -- give some feedback
end repeat
put return & "Found number of images at tolerance levels: " & r

Which in our case gives the following feedback and summary information:

Thu, 5/22/08 7:52:32 PM	START		Running Main.script
Found number of images at tolerance levels: ("5":"1", "6":"53", "7":"56")
Thu, 5/22/08 7:52:39 PM	SUCCESS		Execution Time 0:00:06 Main.script

I’m assuming also that at some point the number of jewels goes to zero and that we can find them all with a proper image collection set of grabs (1 or more images that guarantee all finds in all situations). Without the game in front of me that is hard to tell you if this will work. The idea is close to plausible though in most situations of gaming and collecting items. If the jewels are displayed on the screen say in a ‘basket’ area, then you would want to limit the searchRectangle to the location of game play only.

Is this game online for the public to see? I can easily have fun with this and show you a solution in real time with a suite and movie showing it working if so.

Is this game Treasure Cove from the Learning Company?

Thanks Todd for your response…

hmmm…that looks very complicated to me. I will try to postpone investigating this game after I done with other games. BTW, they are not avalable online at this moment…

PS: I have posted another game in a different post…