Issues constructing a Texture2D from a byte[] extracted from an XRCpuImage in Unity

50 Views Asked by At

I am trying to create an AR debugging application that takes information from a server (the phone) and displays it on a client (the computer).

I am using the AROcclusionManager component to get environmentDepthImage information. I am using a native method (TryAcquireEnvironmentDepthCpuImage) that returns an XRCpuImage with information relating to the environmentDepthImage. I'm diving into the (image)Plane property of the XRCpuImage and am extracting the byteArray that is a property of XRCpuImage through XRCpuImage.GetPlane(); . I am putting this information into a custom serializable class and transferring it as metadata through klakNdi. on the other side, once I deserialize the byteArray, it is populated with information, but I am having trouble getting this byte array into a format that I can display in the unity editor. It is showing as DepthUint16 format server side, and it is also showing the R16 format supported, but once I get the information on the other side, I can't put the byte array into any format I can display in the editor.

this is my Class constructor:

    public SerializableDepthImage(XRCpuImage depthImage)
    {
        width = depthImage.width;
        height = depthImage.height;
        format = depthImage.format;
        planeCount = depthImage.planeCount;
        texData = new byte[depthImage.GetPlane(0).data.Length];
        depthImage.GetPlane(0).data.CopyTo(texData);
        depthImage.Dispose();
    }

this is the code where I'm deserializing the SerializableDepthImage:

if (remotePacket.depthImage.texData == null)
{
    Debug.Log("texData is null");
}
else
{
    Debug.Log("texData is not null");
    
    Texture2D newDepthImage = remotePacket.depthImage.
                              ReconstructTexture2DFromSerializableDepthImage(remotePacket.depthImage);
    Debug.Log("serialized depthImage texData byte value: " +
               remotePacket.depthImage.texData[remotePacket.depthImage.texData.Length -100]);
    Debug.Log("newDepthImage width = " + newDepthImage.width + 
              "newDepthImage.height = " + newDepthImage.height + 
              "newDepthImage.format = " + newDepthImage.format);
    
    depthImage.texture = newDepthImage;

but after this assignation, nothing shows in unity. I'm at my wits end... any help would be greatly appreciated!

I was expecting the depthImage.texture = newDepthImage code to correctly assign the Texture2D newDepthImage to the texture property of the RawImage depthImage. It is just showing as black.

0

There are 0 best solutions below