automatically detecting the HSV values for known situations

Question

I came across this stack overflow question to recognise pieces of a boardgame and their location (aswell as the dice location etc).

I was quite intrigued so using the picture they provided I played with openCV and adjusted HSV values etc until it found all the player pieces for each color - and the positions they are on (the triangles). I got this working. However obviously this only works for this specific set of colors on this specific board.

If we know that we are looking for a load of round playing pieces of two distinct colours and the triangles they sit on, i.e polygons of 3 sides and circles, what approach would you recommend so that the user doesn't have to fiddle with HSV values to pick up the pieces (lets start with saying "some" level of accuracy and not 100% perfect).

I'm thinking from a user experience perspective, You I don't think can ask the average Joe to fiddle with H and S and V until everything is setup so detecting it automatically seems to some level necessary...

I think it could be fine to ask the user to calibrate against a known setup like a start setup for a board or something and to ask them if its correct - and perhaps click on areas that have been missed or something. I am using gocv for implementation btw but generally its a port of openCV. However to test this i believe i cannot access the colour directly under a click and so I looked into using SelectROI to get the color under the click

    selected := gameWindow.SelectROI(originalImage)
    x := (selected.Max.X + selected.Min.X)/2
    y := (selected.Max.Y + selected.Min.Y)/2
    fmt.Println("x", x, "y", y)
    colorAt := selected.RGBA64At(x, y)
    //r, g, b, _ := colorAt.
    r := colorAt.R
    g := colorAt.G
    b := colorAt.B
    fmt.Println("color is ", colorAt, r, g, b, " HSV", coco.Rgb2Hsv(float64(r), float64(g), float64(b)))

but that returns garbage as far as I can see:

x 309 y 187
color is  {65535 65535 65535 65535} 65535 65535 65535  HSV [0 0 25700]

Any other suggestions as to how i could go about this?

One other question - when colours are shiny/whites/blacks and they differ only slightly from their background is there a "trick" to forcing more differentiation or making them more matt or something to easier detect them and seperate them?

so...

1. Can I auto detect based on a set of rules if I know a calibration configuration?
2. Can I make/use a color picker some how directly from the image
3. can i seperate out somehow similar colours/colours that are bright/white/black?

Answer 1

I'd recommend to you the MatchTemplate function for your purposes. The opencv documentation explains it better and there's even a demo showing how to recognize game peices.