SikuliXRobotLibrary Keyword Documentation

SikuliXRobotLibrary

Introduction

SikuliXRobotLibrary is a gui recognition testing library for Robot Framework that leverages the SikuliX version 1.1.0 methods to control matching PSMRL (pattern, string, match, region or location). See http://sikulix-2014.readthedocs.org/en/latest/index.html for more information on SikuliX. It should work in most modern OS.

Before running tests

The following must be installed:

Java # Version 6, 7 or 8 (preferably 7, version 6 is deprecated)
Jython # Version 2.7.0
robotframework # Installed via jython setup.py install (jybot must be setup in the environment variable)
SikuliX v1.1.0 # The sikuli script (sikulixapi.jar) must be installed together with the Tesseract based OCR features in order to read texts on screen. The Settings.OcrTextSearch and Settings.OcrTextRead must be set to True. see http://sikulix-2014.readthedocs.org/en/latest/)

Set the Environment Variables The following must be included in the environment variables:

CLASSPATH = <sikulixapi.jar’s directory> # Must be setup in the environment variable in the test PC or Node
JYTHONPATH = <sikulixapi.jar’s directory>/Lib # Must be setup in the environment variable in the test PC or Node

Installing the SikuliXRobotLibrary

1. Copy the SikuliXRobotLibrary to a local directory.
2. cd to the SikuliXRobotLibrary where setup.py is located
3. Open a terminal or command prompt then install via jython setup.py install.

Verify Installation

Once you have installed SikuliXRobotLibrary it is a good idea to verify the installation. To verify installation start jython and then at the Jython prompt type:

>>> import SikuliXRobotLibrary >>>

If the jython command line interpreter returns with another prompt (‘>>>’ as shown above) then your installation was successful.

Locating or specifying elements

All keywords in SikuliXRobotLibrary that need to find a pattern or any of the PSMRL on the screen take an argument that serves as the locator (for more information about PSMRL, see http://sikulix-2014.readthedocs.org/en/latest/) . locator is a string that describes how to locate an element using a syntax specifying a pattern to search for in a screen, which is a representation of the element.

PNG format must be used for reference patterns for better image resolution and better image detection.

Using locators By default, when a locator value is provided, it is checked if a png with image sensitivity otherwise, it is treated as a string. When a locator is a string, the OCR (Optical Character Recognition) setting must be set to True.

For example:

Keywords Locators or PSMRL Description
Click Pattern pattern.png = 0.90 # Locator is a PNG image format with image sensitivity set to 0.90
Click Pattern pattern.png # Locator is a PNG image format with image sensitivity set to the default value 0.70.
Click Pattern Password # Locator as a string or non-PNG images are matched against any string or image on screen.

Importing

Arguments Documentation
sikulix_timeout=10.0, sikulix_scanrate=3.0,screenshot_directory=None, target_screen=Screen 0,run_on_failure=Capture Screenshot Of Active App SikuliXRobotLibrary can be imported with optional arguments.

sikulix_timeout is the default timeout used to wait for all waiting actions. Default sikulix_timeout value is 3 seconds. It can be later set with Set SikuliX Timeout.

sikulix_scanrate is the default search rate for the given Visual rate times per second until pattern is found or the timeout is reached. Default sikulix_scanrate value is 3. It can be later set with Set SikuliX Scanrate.

target_screen is the default screen for multi-monitor environments. For more information on multi-monitor environments, see https://sikulix-2014.readthedocs.org/en/latest/screen.html

run_on_failure specifies the name of a keyword (from any available libraries) to execute when a SikuliXRobotLibrary keyword fails. By default Capture Screenshot Of Active App will be used to take a screenshot of the Active application in focus. Using the value “Nothing” will disable this feature altogether. See Register Keyword To Run On Failure keyword for more information about this functionality.

screenshot_root_directory specifies the default root directory that screenshots should be stored in. If not provided the default directory will be where robotframework places its logfile.

Examples:

Library | SikuliXRobotLibrary | 15 # Sets default sikulix_timeout to 15 seconds
Library | SikuliXRobotLibrary | 0 | 5 # Sets default sikulix_timeout to 0 seconds and default sikulix_scanrate to 5 seconds
Library | SikuliXRobotLibrary | 5 | run_on_failure=Capture Screenshot Of Active App # Sets default timeout to 5 seconds and runs Log Source on failure
Library | SikuliXRobotLibrary | sikulix_timeout=5 | run_on_failure=Log Source # Sets default implicit_wait to 5 seconds and runs Log Source on failure
Library | SikuliXRobotLibrary | sikulix_scanrate=10 | run_on_failure=Nothing # Sets default sikulix_scanrate to 10 seconds and does nothing on failure

Shortcuts

App Get Name · App Get Process ID · App Get Window · App Has Window · Application Is Running · Assert Pattern Is Not Visible In Active App · Assert Pattern Is Not Visible In Region · Assert Pattern Is Visible In Active App ·Assert Pattern Is Visible In Region · Capture Screenshot · Capture Screenshot Of Active App · Check And Open Application · Click Nth Pattern In Active App · Click Pattern · Click Pattern At Coordinates · Click Region · Close Application ·Confirm Env OS · Double Click Nth Pattern In Active App · Double Click Pattern · Double Click Pattern At Coordinates · Double Click Region · Drag And Drop Element · Drag And Drop From Nth Pattern · Drag Nth Pattern In Active App ·Drag Pattern · Drag Pattern In Coordinates · Drop At Nth Pattern In Active App · Drop At Pattern · Drop Pattern At Coordinates · Get Active App Coordinates · Get Active App Region · Get Active Screen Coordinates · Get Active Screen Region ·Get Application Coordinates · Get Application Region · Get Env OS · Get Env OS Type And Version · Get Env OS Version · Get Pattern Count In Active App · Get Reference Pattern Coordinates · Get Reference Pattern Region ·Get Text In Nth Pattern · Get Text In Pattern · Get Text In Search Region · Highlight Nth Pattern In Active App · Highlight Pattern · Highlight Region · Hover At Nth Pattern In Active App · Hover At Pattern · Hover At Pattern At Coordinates ·Mouse Button Down · Mouse Button Up · Move Mouse At Coordinates · Open Application · Paste String · Paste Text In Nth Pattern In Active App · Paste Text In Pattern · Paste Text In Pattern At Coordinates · Press Key N Times ·Press Keyboard Key · Register Keyword To Run On Failure · Right Click Nth Pattern In Active App · Right Click Pattern · Right Click Pattern At Coordinates · Right Click Region · Run Command · Scroll From Pattern ·Scroll From Pattern At Coordinates · Scroll X Steps · Set Application Focus · Set Image Library · Set New Search Region In Active App · Set New Search Region In Application · Set New Search Region In Target Screen · Set Screenshot Directory· Set Search Region To Active App · Set Search Region To Application · Set Search Region To Target Screen · Set Sikulix Scanrate · Set Sikulix Timeout · Switch Application Focus · Type String · Type Text In Nth Pattern In Active App ·Type Text In Pattern · Type Text In Pattern At Coordinates · Wait For Pattern To Be Visible · Wait For Pattern To Vanish · Wait In Seconds · Wait Until Pattern Is Visible · Wait Until Pattern Vanish

Keywords

Keyword Arguments Documentation
App Get Name app_name Returns the application’s short name as show in the process list.

See also App Has Window, App Get Process ID and App Get Window.

Example:

App Get Name Calculator # Returns calc.exe if Calculator app is running in windows.
App Get Process ID app_name Returns the application’s process ID as number if app is running, -1 otherwise.

See also App Has Window, App Get Name and App Get Window.

Example:

App Get Process ID Calculator # Returns a PID number if Calculator app is running in windows, else -1.
App Get Window app_name Returns the title of the frontmost window of the application as specified in app_name, might be an empty string

See also App Has Window, App Get Process ID and App Get Name.

Example:

App Get Window Calculator # Returns Calculator if Calculator app is running in windows.
App Has Window app_name Returns True if application’s window or dialog as specified in app_name is open, else, returns False.

See also App Get Process ID, App Get Name and App Get Window. Example:

App Has Window Calculator # Returns True if Calculator app is running in windows, else False.
Application Is Running app_name Returns True if application as specified in app_name is running, else, returns False.

See also Check And Open Application, Close Application, and Open Application.

Assert Pattern Is Not Visible In Active App pattern Assert that pattern is not visible in active app. See introduction for details about locating elements.

Note this keyword automatically searches on region of the app in focus and may be paired with the keyword Set Application Focus.

See also Assert Pattern Is Not Visible In Active App, Assert Pattern Is Not Visible In Region, and Assert Pattern Is Visible In Region.

Example:

Set Application Focus Calculator
Assert Pattern Is Visible in Active App equalsbutton.png

# Asserts that pattern is visible in the new search region in active app as defined by offset values.

Assert Pattern Is Not Visible In Region pattern Assert that pattern is not visible in search region. See introduction for details about locating elements.

This keyword must be paired with any of the following keywords: Set Search Region To Target Screen, Set New Search Region In Active App, Set Search Region To Application, Set New Search Region In Target Screen or Set New Search Region In Application.

See also Assert Pattern Is Not Visible In Active App, Assert Pattern Is Visible In Active App and Assert Pattern Is Visible In Region.

Example 1:

Set New Search Region In Active App 10, 60, -20, -270
Assert Pattern Is Not Visible in Region pattern.png

# Asserts that pattern is visible in the new search region in active app as defined by offset values.

Example 2:

Set New Search Region In Active Screen 10, 60, -20, -270
Assert Pattern Is Not Visible in Region pattern.png

# Asserts that pattern is visible in the new search region in active screen as defined by offset values.

Assert Pattern Is Visible In Active App pattern Assert that pattern is visible in search region See introduction for details about locating elements.

Note this keyword automatically searches on region of the app in focus and may be paired with the keyword Set Application Focus.

See also Assert Pattern Is Not Visible In Active App, Assert Pattern Is Not Visible In Region, and Assert Pattern Is Visible In Region.

Example:

Set Application Focus Calculator
Assert Pattern Is Visible in Active App equalsbutton.png

# Asserts that pattern is visible in the new search region in active app as defined by offset values.

Assert Pattern Is Visible In Region pattern Assert that pattern is visible in search region See introduction for details about locating elements.

This keyword must be paired with any of the following keywords: Set Search Region To Target Screen, Set New Search Region In Active App, Set Search Region To Application, Set New Search Region In Target Screen or Set New Search Region In Application.

See also Assert Pattern Is Not Visible In Active App, Assert Pattern Is Not Visible In Region, and Assert Pattern Is Visible In Active App.

Example 1:

Set New Search Region In Active App 10, 60, -20, -270
Assert Pattern Is Visible in Region pattern.png

# Asserts that pattern is visible in the new search region in active app as defined by offset values.

Example 2:

Set New Search Region In Active Screen 10, 60, -20, -270
Assert Pattern Is Visible in Region pattern.png

# Asserts that pattern is visible in the new search region in active screen as defined by offset values.

Capture Screenshot target,filename=sikuli-screenshot-{index}.png Takes a screenshot of the target application, screen or pattern and embeds it into the log.

filename argument specifies the name of the file to write the screenshot into. If no filename is given, the screenshot is saved into file sikuli-screenshot-{index}.png under the directory where the Robot Framework log file is written into. The filename is also considered relative to the same directory, if it is not given in absolute format. If an absolute or relative path is given but the path does not exist it will be created.

If filename contains {index} characters, it will be automatically replaced with running index. The running index is unique for each different filename. The absolute path of the saved screenshot is always returned and it does not depend does the filename contain {index}. See example 1 and 2 for more details.

The {index} is replaced with the actual index by using Python’s str.format method, and it can be formatted using the standard format string syntax. The example 3 shows this by setting the width and the fill character.

If there is a need to write literal {index} or if filename contains { or } characters, then the braces must be doubled.

Example 1:

${file1} = Capture Screenshot activeapp
File Should Exist ${OUTPUTDIR}${/}sikuli-screenshot-1.png
Should Be Equal ${file1} ${OUTPUTDIR}${/}sikuli-screenshot-1.png
${file2} = Capture Screenshot activeapp
File Should Exist ${OUTPUTDIR}${/}sikuli-screenshot-2.png
Should Be Equal ${file2} ${OUTPUTDIR}${/}sikuli-screenshot-2.png

Example 2:

${file1} = Capture Screenshot activeapp ${OTHER_DIR}${/}other-{index}-name.png
${file2} = Capture Screenshot activeapp ${OTHER_DIR}${/}some-other-name-{index}.png
${file3} = Capture Screenshot activeapp ${OTHER_DIR}${/}other-{index}-name.png
File Should Exist ${OTHER_DIR}${/}other-1-name.png
Should Be Equal ${file1} ${OTHER_DIR}${/}other-1-name.png
File Should Exist ${OTHER_DIR}${/}some-other-name-1.png
Should Be Equal ${file2} ${OTHER_DIR}${/}some-other-name-1.png
File Should Exist ${OTHER_DIR}${/}other-2-name.png
Should Be Equal ${file3} ${OTHER_DIR}${/}other-2-name.png

Example 3:

Capture Screenshot activeapp ${OTHER_DIR}${/}sc-{index:06}.png
File Should Exist ${OTHER_DIR}${/}sc-000001.png
Capture Screenshot Of Active App Takes a screenshot of the application in focus.
Check And Open Application path, app_name Checks if application is running and sets the focus to the application, otherwise, opens application matching the given app_name and path.

Example:

Open Application My Awesome App C:/Program Files (x86)/Awesome App/awesomeapp.exe # Opens My Awesome App, if app is already running, sets the focus to the app

See also Close Application, Open Application, and Application Is Running

Click Nth Pattern In Active App pattern,pattern_index Perform a mouse click on the click point using the left button.

Click point is at the center of the nth element identified by pattern. This Keyword is useful when multiple patterns on screen is present. Search region or setROI() value is set to the application in focus as default. Pattern sensitivity must be set at maximum level (0.99) in order to avoid false positives during image detections. Search for matching patterns is set to top > down > left > right.

Example:

Click Nth Pattern In Active App pattern.png = 0.99 3 # Clicks the third instance of an element identified by pattern.png
Click Pattern pattern Perform a mouse click on the click point using the left button. Click point is at the center of the element identified by pattern.

Key attributes for arbitrary patterns are images which is advisably in PNG format or strings. See introduction for details about locating elements.

Click Pattern At Coordinates pattern, xoffset,yoffset Perform a mouse click on the click point using the left button. Click point is at x/y coordinates of the element identified by pattern.

Key attributes for arbitrary patterns are images which is advisably in PNG format or strings. See introduction for details about locating elements.

Click Region Left-clicks the center x,y coordinate of a specified ROI or search region.

This keyword must be paired with any of the following keywords: Set Search Region To Target Screen, Set New Search Region In Active App, Set Search Region To Application, Set New Search Region In Target Screen or Set New Search Region In Application.

See also Double Click Region, Right Click Region, Get Text In Search Region and Highlight Region.

Example:

Set New Search Region In Active App 10, 60, -20, -270 # Sets offset values to a region of active app.
Click Region # Clicks the center of the specified ROI or search region.
Close Application app_name Checks if the application matching the given app_name is running then closes it.

See also Check And Open Application, Open Application and Application Is Running

Confirm Env OS env_OS Returns True or False based on the env_OS version where the script is executed.
Double Click Nth Pattern In Active App pattern,pattern_index Perform a mouse double-click on the click point using the left button.

Click point is at the center of the nth element identified by pattern. This Keyword is useful when multiple patterns on screen is present. Search region or setROI() value is set to the application in focus as default. Pattern sensitivity must be set at max level (0.99) in order to avoid false positives during image detections. Search for matching patterns is set to top > down > left > right.

Example:

Double Click Nth Pattern In Active App pattern.png = 0.99 5 # Double-clicks the fifth instance of an element identified by pattern.png
Double Click Pattern pattern Perform a mouse double-click on the click point using the left button. Click point is at the center of the element identified by pattern.

Key attributes for arbitrary patterns are images which is advisably in PNG format or strings. See introduction for details about locating elements.

Double Click Pattern At Coordinates pattern, xoffset,yoffset Perform a mouse double-click on the click point using the left button. Click point is at x/y coordinates of the element identified by pattern.

Key attributes for arbitrary patterns are images which is advisably in PNG format or strings. See introduction for details about locating elements.

Double Click Region Double-clicks the center x,y coordinate of a specified ROI or search region.

This keyword must be paired with any of the following keywords: Set Search Region To Target Screen, Set New Search Region In Active App, Set Search Region To Application, Set New Search Region In Target Screen or Set New Search Region In Application.

See also Click Region, Right Click Region, Get Text In Search Region and Highlight Region.

Example:

Set New Search Region In Active App 10, 60, -20, -270 # Sets offset values to a region of active app.
Double Click Region # Double-clicks the center of the specified ROI or search region.
Drag And Drop Element pattern1, pattern2 Drag and drop an element identified by pattern1 to another element identified by pattern2.

Example:

Drag and drop pattern1.png = 0.90 pattern2.png = 0.90 # Drag element identified by pattern.png
Drag And Drop From Nth Pattern pattern1,pattern1_index,pattern2,pattern2_index Drag and drop an nth element identified by pattern1 to another nth element identified by pattern2.

This Keyword is useful when multiple patterns on screen is present. Search region or setROI() value is set to the application in focus as default. Pattern sensitivity must be set at max level (0.99) in order to avoid false positives during image detections. Search for matching patterns is set to top > down > left > right.

Examples:

Drag and drop pattern1.png = 0.90 1 pattern2.png = 0.90 2 # Drag and drop the first instance of pattern1.png to the 2nd instance of pattern2.png
Drag and drop pattern.png = 0.90 3 pattern.png = 0.90 4 # Drag and drop the third instance to the fourth instance of pattern.png
Drag Nth Pattern In Active App pattern,pattern_index Drag the nth element identified by pattern. Drag Nth Pattern In Active App and Drop At Nth Pattern In Active App keywords must be used in pairs to complete a drag and drop action.

This Keyword is useful when multiple patterns on screen is present. Search region or setROI() value is set to the application in focus as default. Pattern sensitivity must be set at max level (0.99) in order to avoid false positives during image detections. Search for matching patterns is set to top > down > left > right.

Examples:

Drag Nth Pattern In Active App pattern.png = 0.99 1 # Drags the first instance of an element identified by pattern.png
Drop At Nth Pattern In Active App pattern.png = 0.99 13 # Drops at the 13th instance of an element identified by pattern.png
Drag Pattern pattern Drag Pattern and Drop At Pattern keywords must be used in pairs to complete a drag and drop action.

Examples:

Drag Pattern pattern.png = 0.90 # Drag element identified by pattern.png
Drop At Pattern pattern.png = 0.90 # Drag at element identified by pattern.png
Drag Pattern In Coordinates pattern, xoffset,yoffset Drag element at x/y coordinates of the element identified by pattern.
Drop At Nth Pattern In Active App pattern,pattern_index Drop at the nth element identified by pattern. Drag Nth Pattern In Active App and Drop At Nth Pattern In Active App keywords must be used in pairs to complete a drag and drop action.

This Keyword is useful when multiple patterns on screen is present. Search region or setROI() value is set to the application in focus as default. Pattern sensitivity must be set at max level (0.99) in order to avoid false positives during image detections. Search for matching patterns is set to top > down > left > right.

Examples:

Drag Nth Pattern In Active App pattern.png = 0.99 13 # Drags the 13th instance of an element identified by pattern.png
Drop At Nth Pattern In Active App pattern.png = 0.99 1 # Drops at the first instance of an element identified by pattern.png
Drop At Pattern pattern Drag Pattern and Drop At Pattern keywords must be used in pairs to complete a drag and drop action.

Examples:

Drag Pattern pattern.png # Drag element identified by pattern.png
Drop At Pattern pattern.png # Drag at element identified by pattern.png
Drop Pattern At Coordinates pattern, xoffset,yoffset Drop or release mouse button element at x/y coordinates of the element identified by pattern.
Get Active App Coordinates Returns the coordinates of the application in focus. Keyword must be combined with Set Application Focus.

Examples:

Set Application Focus My Awesome App # Sets the focus to My Awesome App
Get Active App Coordinates |# Gets the coordinates of My Awesome App
Get Active App Region Returns the region of the application in focus.
Get Active Screen Coordinates target_screen Returns the coordinates of the screen as specified in target_screen.
Get Active Screen Region Returns the region of the active screen.
Get Application Coordinates app_name Returns the coordinates of the application in focus. Keyword must be combined with Set Application Focus.

Examples:

Set Application Focus My Awesome App # Sets the focus to My Awesome App
Get Active App Coordinates |# Gets the coordinates of My Awesome App
Get Application Region app_name Returns the region of the application as specified in app_name.
Get Env OS Returns the Operating System type of test pc. Example of Operating System types: WINDOWS, MAC, LINUX
Get Env OS Type And Version Returns the Operating System type and version of test pc.
Get Env OS Version Returns the Operating System version of test pc.
Get Pattern Count In Active App pattern Returns the count of elements identified by pattern in the region of an active application.

Search region or setROI() value is set to the application in focus as default. Pattern sensitivity must be set at max level (0.99) in order to avoid false positives during image detections. Search for matching patterns is set to top > down > left > right.

Get Reference Pattern Coordinates pattern Returns the coordinates of the element identified by pattern.

Example:

Get Reference Pattern Coordinates pattern.png = 0.90 # Gets the coordinates of pattern.png
Get Reference Pattern Region pattern Returns the region of the element identified by pattern.
Get Text In Nth Pattern pattern,pattern_index,search_location Return the text in nth element identified by pattern.

This Keyword is useful when multiple patterns on screen is present. Search region or setROI() value is set to the application in focus as default. Pattern sensitivity must be set at max level (0.99) in order to avoid false positives during image detections. Search for matching patterns is set to top > down > left > right.

Examples:

Get Text In Nth Pattern At Location And Area pattern.png 1 None # Returns the text of the first instance of pattern.png
Get Text In Nth Pattern At Location And Area pattern.png 1 left = 300 # Returns the text within 300px left of the first instance of pattern.png
Get Text In Nth Pattern At Location And Area pattern.png 2 right = 400 # Returns the text within 400px right of the second instance of pattern.png
Get Text In Nth Pattern At Location And Area pattern.png 3 above = 500 # Returns the text within 500px above of the third instance of pattern.png
Get Text In Nth Pattern At Location And Area pattern.png 5 below = 600 # Returns the text within 600px below of the fifth instance of pattern.png
Get Text In Pattern pattern,search_location Returns the text in an area at a spatial location in reference to an element identified by pattern

Examples:

Get Text In Pattern pattern.png None # Returns the text within 300px left of element identified by pattern.png
Get Text In Pattern pattern.png left = 300 # Returns the text within 300px left of element identified by pattern.png
Get Text In Pattern pattern.png right = 400 # Returns the text within 400px right of element identified by pattern.png
Get Text In Pattern pattern.png above = 500 # Returns the text within 500px above of element identified by pattern.png
Get Text In Pattern pattern.png below = 600 # Returns the text within 600px below of element identified by pattern.png
Get Text In Search Region search_location Returns the text from a specified search location.

This keyword must be paired with any of the following keywords: Set Search Region To Target Screen, Set New Search Region In Active App, Set Search Region To Application, Set New Search Region In Target Screen or Set New Search Region In Application.

See also Click Region, Double Click Region, Right Click Region, and Highlight Region.

Examples:

Set New Search Region In Active App 10, 60, -20, -270 # Sets offset values to a region of active app.
Get Text In Search Region region # Returns the text within the specified search region.
Get Text In Search Region left 300 # Returns the text within the specified search region.
Get Text In Search Region right 400 # Returns the text within the specified search region.
Get Text In Search Region above 500 # Returns the text within the specified search region.
Highlight Nth Pattern In Active App pattern,pattern_index,highlight_duration Highlights the nth element identified by pattern for a specified highlight_duration in seconds.

This Keyword is useful when multiple patterns on screen is present. Search region or setROI() value is set to the application in focus as default. Pattern sensitivity must be set at max level (0.99) in order to avoid false positives during image detections. Search for matching patterns is set to top > down > left > right.

Example:

Highlight Nth Pattern In Active App pattern.png = 0.99 6 2

# Highlights the sixth instance of an element identified by pattern.png for 2 seconds.

Highlight Pattern pattern,highlight_duration Highlight element identified by pattern for a specified highlight_duration in seconds.

Key attributes for arbitrary patterns are images which is advisably in PNG format or strings. See introduction for details about locating elements.

Example:

Highlight Pattern pattern.png = 0.90 2 This is a sample text. # Type a text in element identified by pattern.png
Highlight Region highlight_duration Highlights the specified ROI or search region for a specified highlight_duration in seconds.

This keyword must be paired with any of the following keywords: Set Search Region To Target Screen, Set New Search Region In Active App, Set Search Region To Application, Set New Search Region In Target Screen or Set New Search Region In Application.

See also Click Region, Double Click Region, Right Click Region and Get Text In Search Region.

Example:

Set New Search Region In Active App 10, 60, -20, -270 # Sets offset values to a region of active app.
Highlight Region 2 # Highlights the specified ROI or search region. for 2 seconds.
Hover At Nth Pattern In Active App pattern,pattern_index Hovers mouse pointer at the center the nth element identified by pattern.

This Keyword is useful when multiple patterns on screen is present. Search region or setROI() value is set to the application in focus as default. Pattern sensitivity must be set at max level (0.99) in order to avoid false positives during image detections. Search for matching patterns is set to top > down > left > right.

Example:

Hover At Nth Pattern In Active App pattern.png = 0.99 7 # Hovers mouse pointer the seventh instance of an element identified by pattern.png
Hover At Pattern pattern Hover mouse pointer at center of element identified by pattern.

Key attributes for arbitrary patterns are images which is advisably in PNG format or strings. See introduction for details about locating elements.

Hover At Pattern At Coordinates pattern, xoffset,yoffset Hover mouse pointer at x/y coordinates of the element identified by pattern.

Key attributes for arbitrary patterns are images which is advisably in PNG format or strings. See introduction for details about locating elements.

Mouse Button Down mouse_button Press the left mouse button down. Must be paired with Mouse Button Up keyword.
Mouse Button Up mouse_button Release the mouse button previously pressed. Must be paired with Mouse Button Down keyword.
Move Mouse At Coordinates xoffset, yoffset Move mouse cursor coordinates at x/y coordinates of the screen.
Open Application application_path opens the application matching the given application_path.

See also Check And Open Application, Close Application and Application Is Running

Paste String string_param Pastes a string as specified by srtring_param

See also Type String.

Example:

Paste String A quick cat fox jumps over the mat. # Types the specified string
Paste Text In Nth Pattern In Active App pattern,pattern_index, text Paste a text at x/y coordinates of the element identified by pattern.

Example:

Type Text In Pattern pattern.png = 0.90 This is a sample text. # Type a text in element identified by pattern.png
Paste Text In Pattern pattern, text Paste a text in element identified by pattern.

Example:

Paste Text In Pattern pattern.png = 0.90 This is a sample text. # Paste a text in element identified by pattern.png
Paste Text In Pattern At Coordinates pattern, text,xoffset, yoffset Paste a text at x/y coordinates of the element identified by pattern. Example:

Paste Text In Pattern pattern.png = 0.90 30 30 This is a sample text. # Paste a text in x/y coordinates of element identified by pattern.png
Press Key N Times keyboard_key, count Simulates pressing a key multiple times as specified by count value

Examples:

Press Key N Times BACKSPACE 5 # Press Backspace key five times
Press Key N Times A 8 # Press A eight times
Press Keyboard Key keyboard_keys Simulates pressing keyboard keys. Single key; 2-key combination example: Ctrl + Shift; 3-key combination example: Ctrl + Shift + Delete

Examples:

Press Keyboard Key DELETE # Presses the Delete button
Press Keyboard Key CTRL + A # Presses the keys Ctrl + A
Press Keyboard Key CTRL + Shift # Presses the keys Ctrl + Shift
Press Keyboard Key CTRL + Shift + Delete # Presses the keys Ctrl + Shift + Delete
Register Keyword To Run On Failure keyword Sets the keyword to execute when a Selenium2Library keyword fails.

keyword is the name of a keyword (from any available libraries) that will be executed if a Selenium2Library keyword fails. It is not possible to use a keyword that requires arguments. Using the value “Nothing” will disable this feature altogether.

The initial keyword to use is set in importing, and the keyword that is used by default is Capture Page Screenshot. Taking a screenshot when something failed is a very useful feature, but notice that it can slow down the execution.

This keyword returns the name of the previously registered failure keyword. It can be used to restore the original value later.

Example:

Register Keyword To Run On Failure Log Source # Run Log Source on failure.
${previous kw}= Register Keyword To Run On Failure Nothing # Disables run-on-failure functionality and stores the previous kw name in a variable.
Register Keyword To Run On Failure ${previous kw} # Restore to the previous keyword.

This run-on-failure functionality only works when running tests on Python/Jython 2.4 or newer and it does not work on IronPython at all.

Right Click Nth Pattern In Active App pattern,pattern_index Perform a mouse click on the click point using the right button.

Click point is at the center of the nth element identified by pattern. This Keyword is useful when multiple patterns on screen is present. Search region or setROI() value is set to the application in focus as default. Pattern sensitivity must be set at max level (0.99) in order to avoid false positives during image detections. Search for matching patterns is set to top > down > left > right.

Example:

Right Click Nth Pattern In Active App pattern.png = 0.99 4 # Right-clicks the fourth instance of an element identified by pattern.png
Right Click Pattern pattern Perform a mouse click on the click point using the right button. Click point is at the center of the element identified by pattern.

Key attributes for arbitrary patterns are images which is advisably in PNG format or strings. See introduction for details about locating elements.

Right Click Pattern At Coordinates pattern, xoffset,yoffset Perform a mouse click on the click point using the right button. Click point is at x/y coordinates of the element identified by pattern.

Key attributes for arbitrary patterns are images which is advisably in PNG format or strings. See introduction for details about locating elements.

Right Click Region Right-clicks the center x,y coordinate of a specified ROI or search region.

This keyword must be paired with any of the following keywords: Set Search Region To Target Screen, Set New Search Region In Active App, Set Search Region To Application, Set New Search Region In Target Screen or Set New Search Region In Application.

See also Click Region, Double Click Region, Get Text In Search Region and Highlight Region.

Example:

Set New Search Region In Active App 10, 60, -20, -270 # Sets offset values to a region of active app.
Right Click Region # Right-clicks the center of the specified ROI or search region.
Run Command command Runs a command, script or application path as specified in command.

Example:

Run Command control appwiz.cpl # Opens the Windows Control Panel > Programs and Features window.
Scroll From Pattern pattern, scroll Scroll up or down as specified by the scroll step at center of the element identified by pattern. Key attributes for arbitrary patterns are images which is advisably in PNG format orstrings. See introduction for details about locating elements.

Examples:

Scroll From Pattern Up = 10 # Scroll up 10 steps.
Scroll From Pattern Down = 10 # Scroll down 10 steps.
Scroll From Pattern At Coordinates pattern, scroll,xoffset, yoffset Scroll up or down as specified by the scroll step at x/y coordinates of the element identified by pattern.

Key attributes for arbitrary patterns are images which is advisably in PNG format or strings. See introduction for details about locating elements.

Examples:

Scroll From Pattern At Coordinates pattern.png = 0.80 Up = 10 # Scroll up 10 steps.
Scroll From Pattern At Coordinates pattern.png = 0.80 Down = 10 # Scroll down 10 steps.
Scroll X Steps scroll Scroll up or down as specified by the scroll step.

This keyword must be paired with Hover At Pattern in order to execute scrolling action within the region. Examples:

Hover At Pattern pattern.png # Hover at element identified by pattern.png
Scroll X Steps Up = 10 # Scroll up 10 steps.
Scroll X Steps Down = 10 # Scroll down 10 steps.
Set Application Focus app_name Sets focus to the open application matching the given app_name.

Example:

Set Application Focus My Awesome App # Sets the focus to My Awesome App
Set Image Library path Set the default image library path where all image references will be accessed. This must be used during setup in order to set the image library before the test cases are executed.
Set New Search Region In Active App offsets Sets new ROI or the search area to a specified offsets based on original coordinate values of active application in focus.

See also Set Search Region To Target Screen, Set Search Region To Active App, Set Search Region To Application, Set New Search Region In Target Screen.

Example:

Set New Search Region In Active App 10, 60, -20, -270

# Offsets x, y, height, width to 10, 60, -20, -270 pixels respectively.

Set New Search Region In Application app_name, offsets Sets new ROI or the search area to a specified offsets based on original coordinate values of application as specified in app_name.

See also Set Search Region To Target Screen, Set Search Region To Active App, Set Search Region To Application, and Set New Search Region In Target Screen.

Example:

Set New Search Region In Application Calculator 10, 60, -20, -270

# Offsets x, y, height, width to 10, 60, -20, -270 pixels respectively.

Set New Search Region In Target Screen offsets,target_screen Sets new ROI or new search area to a specified offsets based on original coordinate values of target screen.

See also Set Search Region To Target Screen, Set New Search Region In Active App, Set Search Region To Application, Set Search Region To Active App and Set New Search Region In Application.

Example:

Set New Search Region In Target Screen 10, 60, -20, -270 Screen 0

# Offsets x, y, height, width of the Primary Monitor to 10, 60, -20, -270 pixels respectively.

Set Screenshot Directory path, persist=False Sets the root output directory for captured screenshots.

path argument specifies the absolute path where the screenshots should be written to. If the specified path does not exist, it will be created. Setting persist specifies that the given path should be used for the rest of the test execution, otherwise the path will be restored at the end of the currently executing scope.

Set Search Region To Active App Sets the ROI or the search area to the application in focus. Searching for pattern match is faster if the search region is smaller.

Example:

Set Search Region To Active App # Sets the search region to the application in focus

See also Set Search Region To Target Screen, Set New Search Region In Active App, Set Search Region To Application, Set New Search Region In Target Screen and Set New Search Region In Application.

Set Search Region To Application app_name Sets the ROI or the search area to the application as specified in app_name. Searching for pattern match is faster if the search region is smaller.

Example:

Set Search Region To Application Calculator # Sets the search region to the Calculator application.

See also Set Search Region To Target Screen, Set New Search Region In Active App, Set New Search Region In Target Screen and Set New Search Region In Application.

Set Search Region To Target Screen target_screen Sets the ROI or the search area to a specified target_screen. Searching for pattern match is faster if the search region is smaller.

For more information on multi-monitor environment, see https://sikulix-2014.readthedocs.org/en/latest/screen.html

See also Set Search Region To Active App, Set New Search Region In Active App, Set Search Region To Application Set New Search Region In Target Screen, and Set New Search Region In Application.

Example:

Set Search Region To Target Screen Screen 0 # Sets ROI to the Primary Monitor
Set Search Region To Target Screen Screen 1 # Sets ROI to Additional Monitor 1
Set Search Region To Target Screen Screen 2 # Sets ROI to Additional Monitor 2
Set Search Region To Target Screen Screen 3 # Sets ROI to Additional Monitor 3
Set Sikulix Scanrate scanrate Sets the sikulix scanrate value.

See introduction for details about sikulix_scanrate.

Set Sikulix Timeout timeout Sets the sikulix timeout value.

See introduction for details about sikulix_timeout.

Switch Application Focus app_name Switches focus to the open application matching the given app_name.

Example:

Set Application Focus My Awesome App # Switches the focus to My Awesome App application
Switch Application Focus My Very Awesome App # Switches the focus to My Very Awesome App
Type String string_param Types a string as specified by srtring_param

See also Paste String.

Example:

Type String A quick cat fox jumps over the mat. # Types the specified string
Type Text In Nth Pattern In Active App pattern,pattern_index, text Type a text in nth element identified by pattern.

This Keyword is useful when multiple patterns on screen is present. Search region or setROI() value is set to the application in focus as default. Pattern sensitivity must be set at max level (0.99) in order to avoid false positives during image detections. Search for matching patterns is set to top > down > left > right.

Example:

Type Text At Nth Pattern In Active App pattern.png = 0.90 3 This is a sample text. # Type a text in third element identified by pattern.png
Type Text In Pattern pattern, text Type a text in element identified by pattern.

Example:

Type Text In Pattern pattern.png = 0.90 This is a sample text. # Type a text in element identified by pattern.png
Type Text In Pattern At Coordinates pattern, text,xoffset, yoffset Paste a text at x/y coordinates of the element identified by pattern.

Example:

Type Text In Pattern pattern.png = 0.90 30 30 This is a sample text. # Type a text in x/y coordinates of element identified by pattern.png
Wait For Pattern To Be Visible pattern Waits until pattern appears on application in focus. Fails if pattern is not immediately visible on application in focus. SikuliX’s default timeout is 3 seconds.

See also Wait In Seconds, Wait Until Pattern Is Visible, Wait For Pattern To Vanish and Wait Until Pattern Vanish.

Wait For Pattern To Vanish pattern Waits until pattern disappears on application in focus. Fails if pattern is not immediately hidden on application in focus. SikuliX’s default timeout is 3 seconds.

See also Wait In Seconds, Wait For Pattern To Be Visible, Wait Until Pattern Is Visible, and Wait Until Pattern Vanish.

Wait In Seconds timeout Waits until timeout expires.

See also Wait For Pattern To Be Visible, Wait Until Pattern Is Visible, Wait For Pattern To Vanish and Wait Until Pattern Vanish.

Wait Until Pattern Is Visible pattern, timeout Waits until pattern appears on application in focus at specified timeout.

timeout value may be set to a float or to FOREVER.

Using FOREVER as timeout will execute the script to infinity unless the pattern appears

Otherwise, Fails if timeout expires before the pattern appears.

See also Wait In Seconds, Wait For Pattern To Be Visible, Wait For Pattern To Vanish and Wait Until Pattern Vanish.

Wait Until Pattern Vanish pattern, timeout Waits until pattern disappears on application in focus at specified timeout.

timeout value may be set to a float or to FOREVER.

Using FOREVER as timeout will execute the script to infinity unless the pattern appears

Otherwise, Fails if timeout expires before the pattern appears.

See also Wait In Seconds, Wait For Pattern To Be Visible Wait For Pattern To Be Visible, Wait For Pattern To Vanish.

Sikuli Script To Screenshot Multiple Monitors

import shutil

print getNumberScreens()

print Screen(0).getBounds()
print Screen(1).getBounds()
s0 = (SCREEN.getX(), SCREEN.getY(), SCREEN.getW(), SCREEN.getH())
s1 = (Screen(0).getX(), Screen(0).getY(), Screen(0).getW(), Screen(0).getH())
s2 = (Screen(0).getW(), 0, Screen(1).getW(), Screen(0).getH())
print s0
print s1
print s2

img1 = capture(*s0)
shutil.copy(img1, “/home/user/Desktop/Screenshots/SCREEN.png”)
img2 = capture(*s1)
shutil.copy(img2, “/home/user/Desktop/Screenshots/Screen(0).png”)
img3 = capture(*s2)
shutil.copy(img3, “/home/user/Desktop/Screenshots/Screen(1).png”)

How to install SikuliX 1.1.0

Prerequisites:

  • Supported Java versions are installed (6, 7 or 8)

Steps:

  1. Download sikulix-1.1.0.jar.
  2. Put the file in an accessible directory, example C:\SikuliX
  3. Double-click the jar file.
  4. Select the package that you want to install (Note: selecting all three of the options seems to give broken jar files, I tried selecting just the first two and it turned out OK, but I may need to rerun since I need the OCR).
  5. If successful, the following files are created: sikulix
  6. The previously 1.0.0 sikuli-script.jar is now named as sikulixapi.jar and the sikuli-ide.jar is named sikulix.jar.
  7. Use sikuix.jar if you need to run scripts in IDE. I’m using sikulixapi.jar for my command line based scripts for robot framework.

How to setup Jenkins Dumb Slave Node via Java Web Start

Prerequisites:

  • Virtual Box 4.3 or supported version is installed (we will use virtual box to setup the slave server)
  • Jenkins is installed

Steps:

A: Add, configure and start a Dumb Slave node (Note: you may do this in either the slave or master machine as long as you type the correct URL i.e. http://JenkinsHostName:port).

  1. Go to ‘Manage Jenkins’ > ‘Manage Nodes’.
  2. Click ‘New Node’.
  3. Select ‘Dumb Slave’.
  4. Fillout the form:

    dumbSlave

  5. Make sure to select

    Launch slave agents via java web start

    as Secondary launch method.

  6. Save the settings then set the node online.

B: Start the slave agent in the slave machine or vm(Note that this steps should be executed in the slave server thus use the master server’s IP or domain for JenkinsHostName instead of localhost)

  1. Open a web browser then download ‘slave.jar’ from

    http://JenkinsHostName:port/jnlpJars/slave.jar

  2. Run the following command in the terminal or command prompt to start the slave agent.

    .java -jar slave.jar -jnlpUrl http://JenkinsHostName:port/computer/nodeName/slave-agent.jnlp

References:

https://wiki.jenkins-ci.org/display/JENKINS/VirtualBox+Plugin

https://wiki.jenkins-ci.org/display/JENKINS/Launch+Java+Web+Start+slave+agent+via+Windows+Scheduler

https://wiki.jenkins-ci.org/display/JENKINS/Distributed+builds#Distributedbuilds-LaunchslaveagentviaJavaWebStart

https://wiki.jenkins-ci.org/display/JENKINS/Windows+slaves+fail+to+start+via+DCOM