Using python binding for flycapture to retrieve color image

Question

I am working with the CMLN-13S2C-CS CCD camera from PointGrey Systems. It uses FlyCapture API to grab images. I would like to grab these images and do some stuff in OpenCV with them using python.

I am aware of the following python binding: pyflycapture2. With this binding I am able to retrieve images. However, I cannot retrieve the images in color, which is what the camera should be able to do.

The videomode and framerate that the camera is able to handle are VIDEOMODE_1280x960Y8, and FRAMERATE_15, respectively. I think it has something to do with the pixel_format, which I think should be raw8.

Is anyone able to retrieve a color image using this or any existing python binding for flycapture? Note that I am working on Linux.

Answer 1

I'm using the same camera with Matlab and also got an issues with "raw8" format. So, I've chose "rgb8", specifically, "F7_RGB_644x482_Mode1" and all things starts to work (not sure, how it should look at Python).

P.S. At the moment I'm trying to start work with Python and pyflycapture2, let's see, if I would be able to find workaround.

UPD: Okay, now I know the things. :) Your (and mine) issue reasons are buried inside the pyflycapture2 itself, especially "Image" class definition. You can have a look here: https://github.com/jordens/pyflycapture2/blob/eec14acd761e89d8e63a0961174e7f5900180d54/src/flycapture2.pyx

if self.img.format == PIXEL_FORMAT_MONO8:
            dtype = np.dtype("uint8")
            stride[1] = 1
        elif self.img.format == PIXEL_FORMAT_MONO16:
            dtype = np.dtype("uint16")
            stride[1] = 2
        else:
            dtype = np.dtype("uint8")
            stride[1] = self.img.stride/self.img.cols

ANY image will be converted into grayscale, even if it was RGB initially. So, we need to update that file somehow.

Answer 2

You don't need to use the predefined modes. The Context class has the set_format7_configuration(mode, x_offset, y_offset, width, height, pixel_format) method with which you can use your custom settings. Using this you can at least change the resolution of the grabbed image. Usage example:

c.set_format7_configuration(fc2.MODE_0, 320, 240, 1280, 720, fc2.PIXEL_FORMAT_MONO8)

As for the coloring issue. I've so far managed to get a colored image using PIXEL_FORMAT_RGB8 and modifying the Image class in flycapture2.pyx as follows:

def __array__(self):
    cdef np.ndarray r
    cdef np.npy_intp shape[3]  # From 2 to 3
    cdef np.dtype dtype
    numberofdimensions = 2  # New variable
    if self.img.format == PIXEL_FORMAT_MONO8:
        dtype = np.dtype("uint8")
    elif self.img.format == PIXEL_FORMAT_MONO16:
        dtype = np.dtype("uint16")
    elif self.img.format == PIXEL_FORMAT_RGB8:  # New condition
        dtype = np.dtype("uint8")
        numberofdimensions = 3
        shape[2] = 3
    else:
        dtype = np.dtype("uint8")
    Py_INCREF(dtype)
    shape[0] = self.img.rows
    shape[1] = self.img.cols

    # nd value (numberofdimensions) was always 2; stride set to NULL
    r = PyArray_NewFromDescr(np.ndarray, dtype,
            numberofdimensions, shape, NULL,
            self.img.pData, np.NPY_DEFAULT, None)
    r.base = <PyObject *>self
    Py_INCREF(self)
    return r

This code is most likely not flawless (i.e I removed the stride stuff) for the simple reason that I have pretty much 0 experience with C and Cython but this way I at least managed to get a colored frame (now in the process of trying to get the PIXEL_FORMAT_RAW8 working).

And just as a reminder: the flycapture2.pyx is a Cython file so you need to recompile it before you can use it (I just run the pyflycap2 install script again).