Clever OCR pipe-dream

Eggplant Functional
1

Does anyone know a clever way to make OCR do the following:

I have a list of 10 links. I want to click on 5 of them. I want to do this with a repeat loop that will repeat for each item of ListOfThingsToClick.

set ListOfThingsToClick to [
“Apple”,
“Red Apple”,
“Banana”,
“Orange”,
“Frisbee”
]

My links look like this:
Green Apple
Apple
Red Apple
Banana
Orange
Frisbee
Cat
House
Pirate
Dog

The obvious issue is that if I tell it to repeat for each item ListOfThingsToClick its going to find “Apple” in three places. And if I tell it to just click the first item of everyimagelocation I’m going to get “Green Apple” twice. Even if I set “first item” to a counter, I still won’t get the clicks I want.

Best I can think of is to tell it to OCR the stuff ‘nearby’ and make sure there are no other characters in the vicinity. But that’s tricky of course because there will generally be other artifacts on any given page. It would also be time-consuming.

Let’s assume I don’t want to take image captures of my links.

I feel like someone probably knows a clever way to make OCR ‘care’ about what’s around it, but maybe I’m expecting too much. Just thought I’d check. I’ll probably have to convert everything to images and deal with the outliers.

Thanks,

Logan Craig
IBM | Senior Software QA Engineer
Digital Transformation Center (DTC) | OIT
C: 480.516.8322

2

Hey Logan,

Sensetalk Pattern for Text is the answer:

repeat for each item in ListOfThingsToClick
	put "Word to find:" && it
	set mypattern to <not preceded by  (1 to 2 characters) , it, not followed by (many  characters) >
	put imagelocation(text:mypattern)
end repeat

How to “read” it: Find the word apple this has not any one or two characters before and any number of characters behind.

Here is another example for Sensetalk pattern for an unknown version number on screen.

Here is the docu for sensetalk patterns. Its essentially a more readable regular expression.

put Mypattern // prints the regular expression (?<!(?:.{1,2}?))Apple(?!(?:.+))

Sensetalk pattern for text searches are powerful. Just takes some training to master them. Trust me its worth the effort.

Cheers,
Karsten

2 Likes
3

Thanks! This looks like exactly the sort of thing I was hoping for.
Logan

1 Like