OCR Rectangle Search - Can we use predetermined areas in ReadText / SearchRectangle?

Creating a Suite of scripts where the areas for OCR to take place is always divided into four “Panels”. Is there a way to define these from the start, and then use the commands ReadText and SearchRectangle utilizing these?

ReadText(“Panel A”)
SearchRectangle (“Panel C”)

Thanks in advance.

You could define the areas as global variables, as much as globals work within eggplant (constantly needing to redefine in every script / function you want to use them in, as they are not “global” by default as with other languages) …

–// a globals file somewhere
Global PanelA
Set PanelA to [1,1,1920,100]

–// your script / function
global g_PanelA
ReadText g_PanelA

Would that cover your scenario?

1 Like

That is perfect. I wasn’t sure whether the command allowed for variables to be used. This will make my work go a lot smoother.

Thank you for taking the time!

You might also find this helpful. We can talk about ways to apply it quickly through Suite Snippets on our next call.

//function to divide the screen into parts to reduce search times
function ScreenPart Portion	
// Set up an easy-to-maintain property list that stores the necessary multipliers for the specified screen portion parameter value
	set tmpRWI to the remoteWorkInterval
	log "The default remote work interval is"&&tmpRWI
	setoption remoteWorkInterval, .01
	log "The current remote work interval is"&&(the remoteworkinterval)
	// The CurrentPopup screen part was developed specifically for Epic Hyperspace
	// It traces a precise rectangle around a popup window using trangulation of three points (upper left corner, upper right corner, and bottom right corner)
	// If you want to use this screen part for other UIs, follow these steps:
	// 1. Create a screen part below called CurrentPopupUL using coordinates where the upper left corner of a popup would typically be located ex: CurrentPopupUL: ((.12,.026),(.35,.35))
	// 2. Capture images for the upper left, upper right, and bottom right corners of your popup. The height of the UR image must be the same as UL and the width of BR needs to be less than or equal to that of UR
	// 3. Substitute the names of your UL, UR, and BR images for HyperspacePopupBox_UL, etc. below
	if Portion = "CurrentPopup" then
		set tempSearchAreaUL to HandlerGarage.ScreenPart("CurrentPopupUL")
		If ImageFound(imageName:"HyperspacePopupBox_UL", waitFor:2,SearchRectangle:tempSearchAreaUL) is false then
			LogWarning "No popup can be located"
			setoption remoteWorkInterval,tmpRWI
			exit handler
		End If
		log "The previous UL was located @ " & tempPreviousUL -- addresses issues of popup windows opening over other popup windows. tempPreviousUL holds the value from the previous run
		if ImageRectangle("HyperspacePopupBox_UL", SearchRectangle:tempSearchAreaUL) = tempPreviousUL then
			Log "Process is waiting for original window to finish closing."
			Wait 1
		end if
		set tempPreviousUL to ImageRectangle("HyperspacePopupBox_UL", SearchRectangle:tempSearchAreaUL) -- stores location of current UL corner of popup to be used for subsequent runs of this handler
		log "The new UL corner is located @ " & tempPreviousUL
		set UL_Rectangle to ImageRectangle(image:"HyperspacePopupBox_UL",SearchRectangle:tempSearchAreaUL)
		Log "The value for UL_Rectangle is"&&UL_Rectangle
		//Begins search for UR corner 15 pixels to the right of the UL
		set HyperspacePopupBox_UR_SearchAreaX1 to right(UL_Rectangle)+15
		log "The value for UR_X1 is " & HyperspacePopupBox_UR_SearchAreaX1\
		//Begins search for UR corner at the top of the UL
		set HyperspacePopupBox_UR_SearchAreaY1 to top(UL_Rectangle)
		log "The value for UR_Y1 is " & HyperspacePopupBox_UR_SearchAreaY1
		//Ends search for UR corner at the right margin of the screen
		set HyperspacePopupBox_UR_SearchAreaX2 to width of the RemoteScreenRectangle
		log "The value for UR_X2 is " & HyperspacePopupBox_UR_SearchAreaX2
		//Ends search for UR corner 5 pixels below the bottom of the UL
		set HyperspacePopupBox_UR_SearchAreaY2 to bottom(UL_Rectangle)+5
		log "The value for UR_Y2 is " & HyperspacePopupBox_UR_SearchAreaY2
		//Uses the 4 points calculated above to create a search rectangle to find to UR corner
		set HyperspacePopupBox_UR_SearchArea  to ((HyperspacePopupBox_UR_SearchAreaX1,HyperspacePopupBox_UR_SearchAreaY1),(HyperspacePopupBox_UR_SearchAreaX2,HyperspacePopupBox_UR_SearchAreaY2))
		log "The value for UR Search Area is " & HyperspacePopupBox_UR_SearchArea
		//Locates the UR corner using the search rectangle calculated above
		set UR_Rectangle to ImageRectangle(image:"HyperspacePopupBox_UR",SearchRectangle:HyperspacePopupBox_UR_SearchArea)
		Log "The value for UR_Rectangle is"&&UR_Rectangle
		//Begins search for the BR corner 2 pixels to the left of the UR
		set HyperspacePopupBox_BR_SearchAreaX1 to left(UR_Rectangle) - 2
		log "The value for BR_X1 is " &HyperspacePopupBox_BR_SearchAreaX1
		//Begins search for the BR corner 125 pixels below the UR
		set HyperspacePopupBox_BR_SearchAreaY1 to top(UR_Rectangle)+125
		log "The value for BR_Y1 is " & HyperspacePopupBox_BR_SearchAreaY1
		//Ends search for the BR corner 2 pixels to the right of the UR
		set HyperspacePopupBox_BR_SearchAreaX2 to right(UR_Rectangle) + 2
		log "The value for UR_X2 is " & HyperspacePopupBox_BR_SearchAreaX2
		//Ends search for the BR corner 800 pixels below the UR
		set HyperspacePopupBox_BR_SearchAreaY2 to bottom(UR_Rectangle)+800
		log "The value for UR_Y2 is " & HyperspacePopupBox_BR_SearchAreaY2
		//Uses the 4 points calculated above to create a search rectangle to find to BR corner
		set HyperspacePopupBox_BR_SearchArea  to ((HyperspacePopupBox_BR_SearchAreaX1,HyperspacePopupBox_BR_SearchAreaY1),(HyperspacePopupBox_BR_SearchAreaX2,HyperspacePopupBox_BR_SearchAreaY2))
		log "The value for BR Search Area is " & HyperspacePopupBox_BR_SearchArea
		//Locates the BR corner using the search rectangle calculated above
		set BR_Rectangle to ImageRectangle(image:"HyperspacePopupBox_BR",searchrectangle:HyperspacePopupBox_BR_SearchArea)
		Log "The value for BR_Rectangle is"&&BR_Rectangle
		//Calculates the 4 fractions necessary to create a screen part
		//Values will be referenced by the screen part array CurrentPopup:((PopupUL_X,PopupUL_Y),(PopupBR_X,PopupBR_Y))
		set PopupUL_X to (left(UL_Rectangle)/right(RemoteScreenRectangle()))
		put PopupUL_X rounded to 3 places into PopupUL_X
		log "The first value for the CurrentPopup screen part is"&&PopupUL_X
		set PopupUL_Y to (top(UL_Rectangle)/bottom(RemoteScreenRectangle()))
		put PopupUL_Y rounded to 3 places into PopupUL_Y
		log "The second value for the CurrentPopup screen part is"&&PopupUL_Y
		set PopupBR_X to (right(BR_Rectangle)/right(RemoteScreenRectangle()))
		put PopupBR_X rounded to 3 places into PopupBR_X
		log "The third value for the CurrentPopup screen part is"&&PopupBR_X
		set PopupBR_Y to (bottom(BR_Rectangle)/bottom(RemoteScreenRectangle()))
		put PopupBR_Y rounded to 3 places into PopupBR_Y
		log "The fourth value for the CurrentPopup screen part is"&&PopupBR_Y
	end if
	//The PreciseSidebar screen part is specific to Epic Hyperspace and does not have equivalents outside of that UI
	//Remove the If statement below if using with non-Epic customers
	if Portion = "PreciseSidebar" then
		If ImageFound(imageName:"HyperspaceSidebarExpand", waitFor:2, SearchRectangle:HandlerGarage.ScreenPart("RightHalf")) then click FoundImageLocation() -- allows for the possibility that the sidebar is currently collapsed
		WaitFor 10, "HyperspaceSidebarCollapse" -- waits until the sidebar is expanded
		//Calculates the 4 fractions necessary to create a screen part
		//Values will be referenced by the screen part array PreciseSidebar:((SidebarUL_X,SidebarUL_Y),(SidebarBR_X,SidebarBR_Y))
		//Calculates X1 as being the right side of the sidebar collapse icon
		set SidebarUL_X to (right(ImageRectangle("HyperspaceSidebarCollapse"))/right(RemoteScreenRectangle()))
		put SidebarUL_X rounded to 2 places into SidebarUL_X
		log "The first value for the PreciseSidebar screen part is"&&SidebarUL_X
		//Calculates Y1 based on the sidebar's typically beginning 420 pixels above the flyout on a 1920x1080 implementation
		set SidebarUL_Y to (top(ImageRectangle(image:"HyperspaceSidebarCollapse"))-420)/bottom(RemoteScreenRectangle())
		put SidebarUL_Y rounded to 2 places into SidebarUL_Y
		log "The second value for the PreciseSidebar screen part is"&&SidebarUL_Y
		//Calculates X2 as being the right margin
		set SidebarBR_X to 1
		log "The third value for the PreciseSidebar screen part is"&&SidebarBR_X
		//Calculates Y2 as being the bottom of the screen
		set SidebarBR_Y to 1
		log "The fourth value for the PreciseSidebar screen part is"&&SidebarBR_Y
	end if
	set screenPortions to {
		TopHalf: ((0,0),(1,.5)),
		BottomHalf: ((0,.5),(1,1)),
		LeftHalf: ((0,0),(.5,1)),
		RightHalf: ((.5,0),(1,1)),
		HorizontalMiddle: ((0,.25),(1,.75)),
		Sidebar:((.8,0),(1,1)), -- Epic specific
		HyperspaceToolbar:((0,.022),(1,.044)), -- Epic specific
		HyperspaceHorizontalTabbar:((0,.056),(1,.086)), -- Epic specific
		HyperspaceTabbar:((0,0),(.065,1)), -- Epic specific
		HyperspacePatientHeader:((0,.082),(1,.165)), -- Epic specific
		HyperspaceVerticalMiddle:((.35,0),(.65,1)), -- Epic specific
		CurrentPopup:((PopupUL_X,PopupUL_Y),(PopupBR_X,PopupBR_Y)), -- Epic specific, but can be modified for other UIs
		CurrentPopupUL: ((.12,.026),(.35,.35)),
		PreciseSidebar:((SidebarUL_X,SidebarUL_Y),(SidebarBR_X,SidebarBR_Y)), -- Epic specific
		CoveragePayorPlanSummaryBox:((.222,.192),(.663,.30)), -- Epic specific
		ClinicalDocumentationArea:((.13,.145),(.81,.95)), -- Epic specific
		MessageViewerSegmentArea: ((.077,.15),(.11,.55)), -- Epic specific
		Storyboard: ((0,.09),(.14,.96)), -- Epic specific
		ChartReviewLeftPanel: ((.13,.21),(.45,.91)), -- Epic specific
		ChartReviewRightPanel: ((.45,.21),(.77,.91)), -- Epic specific
		NavigatorList: ((.119,.146),(.23,.96)), -- Epic specific
		TabbedNoteTypes:((.12,.18),(.80,.216)), -- Epic specific
		HyperspaceReportButtons:((0,.095),(1,.158)) -- Epic specific

	set multiplier to screenPortions.(Portion) // Retrieves the value for the key in the property list. The name of the key is passed as a parameter into the function.
	// Error handling is needed in the event that an unrecognized parameter is passed to the function
	if multiplier is empty then
		throw "Parameter error", Portion && "is not a recognized screen portion."
		setoption remoteWorkInterval, tmpRWI
	end if
	setoption remoteWorkInterval, tmpRWI
	log "The current remote work interval is"&&(the remoteworkinterval)
	return (the remoteScreenSize,the remotescreenSize) * multiplier
End ScreenPart

Hope this helps,

1 Like

That is brilliant!.. Thanks Dave. This is even more specific to what I’m doing and much more “UI Update” proof. All the best to you and yours, sir.

1 Like