Code working on one mac doesnt work on another

Nikhil_Gore · March 22, 2007, 7:23pm

Hi,

I am trying to automate navigating around in Safari and am using the following code to figure out where to click to get to the Tabs options under Safari Preferences (assuming that the Tabs section is not currently selected)

Click (Text"Tabs", TextSize"11", TextPlatform"MacOSX_10.4")

This works fine on my MBP Core 2 Duo. However when I give this code to my colleague to run on his Mac Mini this code does not work. However if he adds a tolerance option then things work fine

Click (Tolerance "60", Text"Tabs", TextSize"11", TextPlatform"MacOSX_10.4")

This seemed wrong to me so I did a ImageLocation for each of our codes and got the following on my machine

Thu, 3/22/07 40319 PM START Running Selection from Untitled.script
040319 Start handler 1 untitled
put ImageLocation (Text"Tabs", TextSize"11", TextPlatform"MacOSX_10.4")
Thu, 3/22/07 40319 PM imagelocation Text Tabs at (246, 330)
(246,330)
040319 End handler 1 untitled (normal exit)
Selection Executed in 00000

…and the following on his machine (without tolerance)…

Thu, 3/22/07 41827 PM START Running Selection from Untitled.script
041827 Start handler 1 untitled
put ImageLocation (Text"Tabs", TextSize"11", TextPlatform"MacOSX_10.4")
Thu, 3/22/07 41828 PM imagelocation Text Tabs at (205, 28)
(205,28)
041828 End handler 1 untitled (normal exit)
Selection Executed in 00000

… and then with tolerance…

Thu, 3/22/07 40534 PM START Running Selection from Untitled.script
040534 Start handler 1 untitled
put ImageLocation (Tolerance “60”,Text"Tabs", TextSize"11", TextPlatform"MacOSX_10.4")
Thu, 3/22/07 40534 PM imagelocation Text Tabs at (246, 330)
(246,330)
040534 End handler 1 untitled (normal exit)
Selection Executed in 00000

So as you can see if I do not put the image tolerance parameter it finds some random location which is the main safari window title bar. Could someone please explain to me what I am doing wrong?

Nikhil_Gore · March 22, 2007, 7:50pm

Upon running more tests I see that a few of the TIGS goto random locations

Thu, 3/22/07 44654 PM START Running Untitled.script
044654 Start handler 1 untitled
put ImageLocation (Text"General", TextSize"11", TextPlatform"MacOSX_10.4")
Thu, 3/22/07 44654 PM imagelocation Text General at (199, 26)
(199,26)
put ImageLocation (Text"Appearance", TextSize"11", TextPlatform"MacOSX_10.4")
Thu, 3/22/07 44654 PM imagelocation Text Appearance at (138, 339)
(138,339)
put ImageLocation (Text"Bookmarks", TextSize"11", TextPlatform"MacOSX_10.4")
Thu, 3/22/07 44654 PM imagelocation Text Bookmarks at (211, 338)
(211,338)
put ImageLocation (Text"Tabs", TextSize"11", TextPlatform"MacOSX_10.4")
Thu, 3/22/07 44654 PM imagelocation Text Tabs at (205, 28)
(205,28)
put ImageLocation (Text"RSS", TextSize"11", TextPlatform"MacOSX_10.4")
Thu, 3/22/07 44654 PM imagelocation Text RSS at (309, 337)
(309,337)
put ImageLocation (Text"AutoFill", TextSize"11", TextPlatform"MacOSX_10.4")
Thu, 3/22/07 44654 PM imagelocation Text AutoFill at (354, 338)
(354,338)
put ImageLocation (Text"Security", TextSize"11", TextPlatform"MacOSX_10.4")
Thu, 3/22/07 44655 PM imagelocation Text Security at (407, 338)
(407,338)
put ImageLocation (Text"Advanced", TextSize"11", TextPlatform"MacOSX_10.4")
Thu, 3/22/07 44655 PM imagelocation Text Advanced at (464, 338)
(464,338)
044655 End handler 1 untitled (normal exit)
Thu, 3/22/07 44655 PM SUCCESS Execution Time 00000 Untitled.script

jruuska · March 23, 2007, 1:53am

Hi Nikhil,

It looks like the text you are rendering is somewhat small on a gray background. Here are three things to think about:

Sometimes using the text-image generator for small fonts on same colored backgrounds (such as black text on gray) requires a different tolerance setting as you were describing.
Make sure that each Mac has the same font-smoothing setting if possible (this goes for Windows too).
You may need to specify the search type as “text” as well.

Please let us know what combination works for you!

viztester · March 23, 2007, 10:04am

This can happen when you have different display settings on different machines, hence the TIG generates and looks for different text. For example, Apple does sub-pixel alignment on their LCD screens, and this can cause what appears to be black text to really contain no black at all. Look very closely at what you gen’d, and what’s on the screen. This issues gets me when I change test machines and forget to observe the display settings (and it got me last week, hence, it’s fresh in my mind).

I’d follow Jay’s suggestions, and in particularly difficult tests, the two techniques often used are:

Make a handler tolerantFind, that inside a repeat loop increases tolerance by 5 (find what works for you), and does an anyImageFound, relying on the fact that if you go from none found in one loop to multiple finds in the next loop, you probably haven’t found what you’re looking for (if it is in fact, unique on the screen). Hopefully, you only find one, and if multiples, report that as none.
If you know roughly where the text will be found, you can ask for just a section of the screen by setting your search rectangle. For example the upper left quartile of the remote screen (your fractions may vary):

set mySearchRect to (0,0, item 1 in remoteScreenSize() * 1/2, item 2 in remoteScreenSize() * 1/2)

You could add this in the repeat loop above, and if you find more than one, try to narrow the location.

Good luck.

Todd · March 26, 2007, 3:56am

Just something I tossed together like a fresh salad with Eggplant Using a known window title bar of OS X, find the tolerance of the text when it will uniquely show. The TIG image should be unique on the SUT for this function to work properly.

-- TIG tolerance determinator.
put toleranceFinder (text:"About This Mac", \
  textStyle:"WindowTitle", \
  textSize:11)

to toleranceFinder  tig, sTol, eTol, aStep

  if tig is empty then return empty
  if sTol is not between 1 and 100 then set sTol to 1
  if eTol is not between 1 and 100 then set eTol to 100
  if eTol is not greater than sTol then return empty
  if aStep is not between 1 and 100 then set aStep to 1
  
  if ImageFound(tig) then
    repeat with n = sTol to eTol step aStep
      set the tolerance of tig to n
      get number of items in EveryImageLocation(tig)
      if it = 1 then
        put "The tolerance should be about " & n
        exit repeat
      else if it > 1 then
        throw "Image was not found uniquely, " \
          "please guarentee that one instance of the TIG '" \
          & tig & "' is viewable/visible at a time."
      end if
    end repeat
  end if
  
  return n
  
end toleranceFinder

Nikhil_Gore · March 26, 2007, 10:01pm

Thanks for all the suggestions everyone. I guess I shouldve been clear that safari is maximized and the word “Tabs” does not appear anywhere else on the screen. The location that eggplant says that the text exists in the incorrect case is the titlebar of the blank page.

jruuska
I understand that problem occur with font smoothing however neither option 2 nor 3 worked for me. I understand the case you make in point 1 however I think its not valid here since I get an image found. If it returned a false on ImageFound I would be all set.

viztester
You mention that I should do an ImageFound and increase the tolerance every time however this means that I find all images on the screen that eggplant finds. I just checked this and it find 98 images so this approach will not work.
Setting the search rectangle is an option I considered however I cannot control where the preferences dialog pops up.

Nikhil_Gore · March 26, 2007, 10:33pm

I am going to file this under a hack. What I did was insert two spaces before and after the word Tabs and it worked. Could someone explain this to me as it does not fit into the various theories that were mentioned (at least not to me)?

put ImageFound(Text:" Tabs ", TextSize:“11”, TextPlatform:“MacOSX_10.4”)

Mon, 3/26/07 7:30:50 PM START Running Selection from Untitled.script
Mon, 3/26/07 7:30:50 PM imagefound Text: Tabs found at (474, 181)
true
Selection Executed in 0:00:00

jruuska · March 27, 2007, 1:03am

Nikhil,

This is working reliably on my machine. Try it and see if it works for you:

repeat 100 times
	moveto (200, 75) -- moveTo is optional but sort of fun to watch
	moveto (Text:"Tabs", TextSize:"11", TextBackgroundColor: "244,244,244",  TextPlatform:"MacOSX_10.4", tolerance:42) 
end repeat

Is this a separate SUT or a virtual SUT on your Mac? You might want to double-check that you are specifying the right background color as well.

Please be sure to let us know what solution works for you.

SenseTalkDoug · March 27, 2007, 12:39pm

Nikhil Gore wrote:
"What I did was insert two spaces before and after the word Tabs and it worked. Could someone explain this to me as it does not fit into the various theories that were mentioned (at least not to me)? "

Text on the screen (particularly in Mac OS X) is not rendered consistently at the pixel level. If you magnify a word such as “Tabs” and look closely, you may be astonished (we certainly were!) to see how many different ways it will be drawn depending on where it appears on the screen or other factors. The individual pixels may be different almost every time it is drawn (Mac OS X uses a very dynamic screen drawing system).

Eggplant’s “Text” search type uses sophisticated matching techniques to enable it to reliably compare a captured (or generated) image containing text in order to find text on the screen that may be rendered differently. Unfortunately, in some situations this text matching technique may occasionally produce “false positives”, in which it identifies an image on the screen in the wrong place. This is especially likely when the image being matched is relatively small.

In your case, I suspect that your colleague’s Mac either had slightly different settings, or he simply had other things showing on the screen that more closely approximated the pixel pattern of the word “Tabs” in an 11 point font. This led to a false positive match at an incorrect location (note that generated text images use the “Text” search type by default).

To correct this problem, one option would be to adjust the tolerance, as has been suggested, but in this case you might need to reduce the tolerance since the problem is not that you aren’t finding the image, but that you’re finding it in too many places. Reducing the tolerance of text image is tricky, though, because Eggplant calculates the tolerance setting automatically based on the amount of contrast present in the image (so it’s not the same for all text images).

The approach that you’ve discovered, of adding spaces before and after the text, may be working by making the image a little wider and including more blank space (although Eggplant automatically trims off excess background color around generated text images as part of the image creation process, so adding extra spaces won’t necessarily change the image size. You can disable the trimming by including a Trim:NO property).

You may need to experiment a little to determine what works best in your situation. In any case, I hope this explanation helps you to better understand what’s going on here!