Automate what you see on a computer monitor
Latest version is SikuliX 1.1.1
What is it? For what is it? Get it? Use it? Get help? Contribute?
Sikuli is God's Eye
… in Huichol Indian culture: the power to see and understand things unknown.
(Thanks for the picture showing a Sikuli)
Sikuli was started somewhen in 2009 as an open-source research project at the User Interface Design Group at MIT by Tsung-Hsiang Chang and Tom Yeh.
Both left the project at Sikuli-X-1.0rc3 during 2012, when I decided to take over development and support and name it SikuliX.
What is SikuliX?
SikuliX automates anything you see on the screen of your desktop computer running Windows, Mac or some Linux/Unix. It uses image recognition powered by OpenCV to identify and control GUI components. This is handy in cases when there is no easy access to a GUI's internals or the source code of the application or web page you want to act on.
SikuliX supports as scripting languages
- Python language level 2.7 (supported by Jython)
- running RobotFramework text-scripts is supported (see docs)
- Ruby language level 1.9 and 2.0 (supported by JRuby)
- JavaScript (supported by the Java Scripting Engine)
Though SikuliX is currently not available on any mobile device, it can be used with the respective emulators on a desktop computer or based on VNC solutions. A solution for Android devices based on ADB (Android Debugging Bridge) is in an early experimental stage,
Besides locating images on a screen SikuliX can run the mouse and the keyboard to interact with the identified GUI elements. This is available for multi monitor environments and even for remote systems with some restrictions.
SikuliX comes with basic text recognition (OCR) and can be used to search text in images. This feature is powered by Tesseract.
SikuliX is a Java application, that works on Windows XP+, Mac 10.6+ and most Linux/Unix systems. For Windows, Mac and Ubuntu 12.04+ it is complete and should normally work out of the box. For Linux/Unix systems there are a few prerequisites to be setup.
SikuliX is the follow up of Sikuli Script - last version 2011 as Sikuli-X-1.0rc3. Though it is no longer supported: here you find valuable information for the installation.
— back to top —
When is SikuliX useful?
You want to automate some repetitive tasks in
- daily usage of applications or web pages
- playing games
- administration of IT systems and networks
You want to test applications or web pages, that are under development.
You want to create usage documentation or training material, that run live on the addressed application or web page.
You already have tools and workflows for that, but you want to add the specific features of SikuliX to enhance your approach and improve efficiency and outcome.
Using SikuliX means doing WYSIWYS: What You See Is What You Script ;-)
— back to top —
How can I get SikuliX?
Before thinking about using SikuliX, you have to assure, that you have a valid Java JRE installation (preferably version 7 or 8 latest). Though normally the SikuliX application can be started by just double clicking the respective SikuliX elements (either jar files or command scripts), it helps to be familiar with the usage of a command line window on your system for some more sophisticated usage cases.
But before downloading and fiddling around,
it is highly recommended to read about the gory details in QUICKSTART.
SikuliX can be downloaded from SikuliX's Launchpad page. What you download from there is sikulisetup-X.Y.Z.jar (where X.Y.Z is the version number - currently 1.1.x).
You should put it in a separate folder in a prominent place, since it is intended to stay there for ever ;-) together with other SikuliX artefacts you will get later on, either by will or because SikuliX creates them on the fly. It is not recommended to put any scripts, images or other user stuff into that folder.
To finally get the ready-to-use SikuliX elements you have to run the setup.
— back to top —
How do I use SikuliX?
In any case you should have read through QUICKSTART.
With SikuliX there are two rather distinct usage scenarios:
Scripting
One sets up and runs more or less complex SikuliX workflows based on searching images and acting on GUI elements with mouse and keyboard. For this one uses the so called SikuliX IDE, that brings some basic support for editing and running scripts.
In the Scripting usage scenario, you handle SikuliX scripts named you-name-it.sikuli, which are folders containing the script file and the images you need for the workflow. You either run the scripts from within the SikuliX IDE or from the command line.
To get on the road you should at least once scan the docs.
Programming
In a mature IDE (IntelliJ IDEA, Eclipse, NetBeans, ...) and/or some testing environment one integrates SikuliX as a feature library using the available API's while programming in Java or Java aware/based languages (Jython, JRuby, Scala, Clojure, ...).
If you work in this usage scenario, then
- you should at least once scan the docs
- you might have a look at the Java docs
- ... and this might be helpful
— back to top —
Where can I get help?
The first place to get help is to visit SikuliX's Launchpad page. You might ask a question in the Answers section or scan it for anything helpful. You might check the FAQ section. Even the Bugs section might shed light on your current problem.
… and even just asking Google, might reveal valuable information from elsewhere in the net.
More information on how to get help and insights.
— back to top —
How can I contribute?
The sources and hence the development base is on GitHub, where you will find everything, that you need to know:- current: SikuliX-2014
- development: SikuliX2
Any contribution is welcome and highly appreciated.
Use any of the available channels, to get in contact.
— back to top —