Renderscript Colormatrix intrinsic - Help me understand

Question

I think that I must be misunderstanding how the Renderscript intrinsic to apply a colormatrix works, because my results don't turn out as I expect.

So I have an allocation for Renderscript which "overlays" an OpenCV Mat, basically imagine it as a 3 dimensional array full of pixels where every pixel has RGBA (Red,Green,Blue,Alpha) values.

So I want to apply a colormatrix to every pixel like this:

Vector R times Matrix 0.152286f,  1.052583f,  -0.204868f, 0f,
       G              0.114503f,  0.786281f,  0.099216f,  0f,
       B              -0.003882f, -0.048116f, 1.051998f,  0f,
       A              0.000000f,  0.000000f,  0.000000f,  1f

So what I expect to happen is that the new Vector R'G'B'A' will be like this:

R' = R * 0.152286f + G * 1.052583f + B * -0.204868f + A * 0f
G' = as above
B' = as above
A' = R * 0 + G * 0 + B * 0 + A * 1 

So the new R' value will be a combination of the old RGB values, A does not affect RGB. Same behavior for G' and B'. A will always stay the same.

In Code it looks like this:

Matrix4f mProtan = new Matrix4f(new float[]{
                0.152286f, 1.052583f, -0.204868f, 0f,
                0.114503f, 0.786281f, 0.099216f, 0f,
                -0.003882f, -0.048116f, 1.051998f, 0f,
                0.000f, 0.000f, 0.000f, 1f
        });
        scriptIntrinsicColorMatrix.setColorMatrix(mProtan);
        scriptIntrinsicColorMatrix.forEach(inputAllocation, outputAllocation);

So I am already doing this with OpenCV and it works as expected, but it's kinda slow, hence I'd like to use Renderscript, but there my outcome is usually weird, for example this matrix shouldn't really affect anything other than Red, Green and combinations of them (e.g. Red becomes a dark shade of grey/brown, Green becomes a muddy yellow and Purple is Red + Blue so Red goes away and purple becomes only Blue. Even white paper gets a greenish tint).

I also tried direct streaming from the camera feed via Renderscript only and storing information in Bitmaps, but the results end up the same.

Any help would be greatly appreciated! :-)

Answer 1

RS uses column-major format for matrices, so you need to transpose your matrix to get your expected R'G'B'A' values.

https://developer.android.com/guide/topics/renderscript/reference/rs_matrix.html

https://android.googlesource.com/platform/frameworks/rs/+/android-7.0.0_r6/cpu_ref/rsCpuIntrinsicColorMatrix.cpp#820