My situation is the following : I run a video stream from an android device running Unity via WebRTC and i want to send a custom render result that i have as a GLES20 RGB/RGBA format texture.
Unfortunately whenever i try to send this texture, WebRTC crashes with a 1282 GL_INVALID_OPERATION error and takes the whole app down with it.
Additionally, calling glReadPixels and then sending a byte buffer works, so the texture is atleast valid by directly accessing it through GLES20, although the format is wrong for streaming. I'd use this approach if it weren't for the very inefficient realtime performance.
Anyway, on to the code :
public void getCameraImage(int textureID, int rotation, long timestamp)
{
this.nTexId = textureID;
if (!isStarted){
isStarted = true;
GLES20.glGenFramebuffers(1, this.genFBO, 0);
}
if (!newFrameReady){
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, genFBO[0]);
GLES20.glFramebufferTexture2D(GLES20.GL_FRAMEBUFFER, GLES20.GL_COLOR_ATTACHMENT0, GLES20.GL_TEXTURE_2D, incomingTexId, 0);
// outputs correct values in RGBA
GLES20.glReadPixels( 0,0, 16, 16, GLES20.GL_RGBA, GLES20.GL_UNSIGNED_BYTE, testBuffer);
//clean up
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, 0);
newFrameReady = true;
mThread.run();
}
}
And there's the code for the thread:
public void setupObserverThread(CapturerObserver capturerObserver) {
this.capturerObserver = capturerObserver;
mThread = new Thread(new Runnable() {
@Override
public void run() {
if (newFrameReady) {
try {
capturerObserver.onTextureFrameCaptured(width, height, this.nTexId, RendererCommon.identityMatrix(), mRotation, mTimestamp);
newFrameReady = false;
UnityPlayer.UnitySendMessage(gameObjectName, callbackFunction, "Captured");
}catch( Exception e ){
Log.e(TAG, "run: GL ERROR ");
}
}
}
});
mThread.start();
}
So my question is - what can i give WebRTC that would satisfy it's format requirements and not crash?
Here's the stracktrace logcat after webRTC crashes. It happens only once it's connected to peer and stream is started (on first frame).
11-24 16:01:59.673: E/org.webrtc.Logging(25509): MediaCodecVideoEncoder: encodeTexture failed
11-24 16:01:59.673: E/org.webrtc.Logging(25509): MediaCodecVideoEncoder: java.lang.RuntimeException: glUseProgram: GLES20 error: 1282
11-24 16:01:59.676: E/org.webrtc.Logging(25509): MediaCodecVideoEncoder: java.lang.RuntimeException: glUseProgram: GLES20 error: 1282
11-24 16:01:59.676: E/org.webrtc.Logging(25509): at org.webrtc.GlUtil.checkNoGLES2Error(GlUtil.java:29)
11-24 16:01:59.676: E/org.webrtc.Logging(25509): at org.webrtc.GlShader.useProgram(GlShader.java:110)
11-24 16:01:59.676: E/org.webrtc.Logging(25509): at org.webrtc.GlRectDrawer.prepareShader(GlRectDrawer.java:195)
11-24 16:01:59.676: E/org.webrtc.Logging(25509): at org.webrtc.GlRectDrawer.drawOes(GlRectDrawer.java:118)
11-24 16:01:59.676: E/org.webrtc.Logging(25509): at org.webrtc.MediaCodecVideoEncoder.encodeTexture(MediaCodecVideoEncoder.java:592)
11-24 16:01:59.939: E/rtc(25509): #
11-24 16:01:59.939: E/rtc(25509): # Fatal error in ../../webrtc/sdk/android/src/jni/native_handle_impl.cc, line 225
11-24 16:01:59.939: E/rtc(25509): # last system error: 11
11-24 16:01:59.939: E/rtc(25509): # Check failed: !jni->ExceptionCheck()
11-24 16:01:59.939: E/rtc(25509): # textureToYUV throwed an exception
11-24 16:01:59.939: E/rtc(25509): #
11-24 16:01:59.940: A/libc(25509): Fatal signal 6 (SIGABRT), code -6 in tid 25838 (EncoderQueue - )