Sikulix : Compare a image and a screen of website

Question

I use sikulix and I want to check if a region of a website is red or green.

So I have taken a picture of the part of the website when is red (negatif.png) A another region for the check of the color. But it doesn't work.

img_negatif = "negatif.png"
profit = Region(1342,212,175,21)
if profit == img_negatif:
    click(Location(3406,1420))
else:
    click(Location(23,1420))

Everytime sikulix take the else.

Answer 1

You cannot compare regions strictly by color. You can see my answer here (this is from years back but I think it's still relevant).

Sikuli compares the regions with pixel to pixel precision (you can set a similarity level if the images slightly differ but that's not the case). So having selected an area with just color and no other pattern will not work.

Answer 2

Wait for 3 second for negatif and if appear ...

if exists(img_negatif, 3):
  click(Location(3406,1420))
else:
  click(Location(23,1420))

Answer 3

You can search a region for color with Brobot, an open-source automation framework for Java that wraps SikuliX functions. Brobot references OpenCV for its color functionality. The Brobot documentation gives more detailed information on the framework. Additionally, there is a page in the documentation on color searches. For a summary of Brobot's color functionality, you can read this StackOverflow answer.

Finding the Red Region

public Matches find() {
    ActionOptions actionOptions = new ActionOptions.Builder()
            .setAction(FIND)
            .setFind(COLOR) 
            .build();
    return action.perform(actionOptions, redRegionState.getRedArea());
}

Elements of this code block:

• Matches, the return value, holds information about the individual matches and the action.
• ActionOptions is used to configure the action. Here, we tell it we want a FIND operation and we want to find by COLOR.
• The Action class is the Brobot class that performs actions. The result of an action is always a Matches object. The function action.perform requires as parameters (1) the action configuration and (2) the object(s) to act on. Here, we pass in our actionOptions variable and our image.
• The image “negatif.png” is stored in a Brobot variable called redArea, located in the state RedRegionState, and called with redRegionState.getRedArea().
• The average pixel color of all pixels in “negatif.png” is calculated and this color is searched for on-screen.
• The classes Action and RedRegionState are passed to this class with dependency injection.

The Red Image

You could also declare the Brobot variable containing “negatif.png” locally but one of the main benefits of Brobot is its ability to simulate the environment and test code with mock runs. This requires creating states that group associated images. Inside the state, the image definition would look like this:

@Component
@Getter
public class RedRegionState {
    …
    private StateImageObject redArea = new StateImageObject.Builder()
            .withImage("negatif.png")
            .withSearchRegion(1342,212,175,21)
            .build();
    …
}

Including a search region in the image variable means that the image will be searched for in this region on-screen unless a search region is added to the action configuration (the ActionOptions variable).

Clicking after Searching

public void click() {
    int x = find().isSuccess() ? 3406 : 23;
    action.perform(CLICK, new Location(x, 1420).asObjectCollection());
}

Elements of this code block:

• Each action (FIND, CLICK, DEFINE, etc.) has its own success condition, and the result is stored in the Matches variable. It can be called with the boolean function .isSuccess().
• As above, action.perform takes as parameters the action configuration and the object(s) to act on.
  • Usually an ActionOptions variable is required for the first parameter. However, an enum representing the action (here, CLICK) can be passed directly if you wish to use the action's default options.
  • The second parameter usually takes an ObjectCollection variable. In some cases, as with the image redArea, the object can be passed directly. Here, the object acted on, a Location, is converted to a singleton ObjectCollection and passed as the second parameter.
• When building a more complex automation application, it would make sense to include the Location variables as well in states. Most likely, the Location at (3406, 23) would be in RedRegionState, as this Location has a special meaning when redArea is visible.

Disclaimer: I'm the developer of Brobot. It's free and open source.