Flickering while using surface view to draw frame by frame bitmap

Question

I am using surface view to render frames after extracted from video, the problem is that there is a flickering effect while rendering, the origin video was smooth, I play the frames in ImageViewer and hold on press NEXT key to switch next and next it was smooth too, only flicker after render them in SurfaceView.

the problem is I have a period between frames drawing, because I want to control the playing frame rate, make it slower or faster via user's choice, once I give up the delay drawing the problem gone, but that's no my intention, I need to make them delay.

I understand that this is due to double/triple buffering problem, even though I went through many posts, including turn to use GLSurfaceView to render, also drawBitmap twice intent to keep front-buffer and back-buffer align, it doesn't help to fix this problem.

I found this Flickering while using surface view post, and try all the solution-like mention inside, but it's not work, the accepted answer mention about dirty rect, remind me to update every pixels if call lockCanvas() without rect specified, but I think I already draw the whole bitmap in the next, imply I updated every pixels, so I get not idea of this.

below are the code and the problem's gif, please take a look at my code and help me get this fixed.

class CustomView(
    context: Context, attrs: AttributeSet?
) : SurfaceView(context, attrs), Runnable {
    private var animationThread: Thread? = null

    @Volatile private var running = false
    private var frameList: List<BitmapFrame1>? = null
    private var index = 0

    fun start(frameList: List<BitmapFrame1>) {
        if (running) return
        running = true
        index = 0
        this.frameList = frameList
        animationThread = Thread(this).apply {
            start()
        }
    }

    override fun run() {
        val surHolder = holder
        var nextFrameTimeMs = 0L
        while (running) {
            if (!surHolder.surface.isValid) continue
            if (SystemClock.uptimeMillis() >= nextFrameTimeMs) {
                val currentFrame = frameList!!.getOrNull(++index)
                if (currentFrame == null) {
                    running = false
                } else {
                    val canvas = surHolder.lockCanvas()
                    canvas.drawBitmap(currentFrame.bitmap, 0f, 0f, Paint())
                    val drawTimestamp = SystemClock.uptimeMillis()
                    surHolder.unlockCanvasAndPost(canvas)
                    nextFrameTimeMs = drawTimestamp + currentFrame.delayMs
                }
            } else {
                // have tried to draw the current frame again before delay time's up,
                // but not effect
                val currentFrame = frameList!![index]
                val canvas = surHolder.lockCanvas()
                canvas.drawBitmap(currentFrame.bitmap, 0f, 0f, Paint())
                surHolder.unlockCanvasAndPost(canvas)
            }
        }
    }
}

Answer 1

You have to use Choreographer and not lock Canvas in "random" moment (this wrong moment could occurs when Surface is swapping from one buffer to another and the Bitmap/Texture is uploaded on wrong buffer) but only after a Choreographer callback event. When this event occurs it's guaranteed that the next frame is uploaded on the right buffer and rendered well in next Drawing call.

Choreographer.getInstance().postFrameCallback(new Callback(){
    @Override
    public doFrame(final long frameTimeNanos) {
        surHolder.lockCanvas();
        canvas.drawBitmap(currentFrame.bitmap, 0f, 0f, paint);
        surHolder.unlockCanvasAndPost(canvas);
    }
});

So if you want to slow down rendering you need to "queue" requests using an Handler and send a delayed Message to trigger the drawing procedure (below it's pseudo code):

private Handler mHandler;
private final Choreographer.Callback mCallback = new Callback() {
    @Override
    public doFrame(final long frameTimeNanos) {
        surHolder.lockCanvas();
        canvas.drawBitmap(currentFrame.bitmap, 0f, 0f, paint);
        surHolder.unlockCanvasAndPost(canvas);
    }
}

public void prepareHandler() {
    mHandler = new Handler(Looper.getMainLooper(), new method() {
        @Override
        private void handleMessage(@NonNull final Message message) {
            switch (message.what) {
                case 1234: {
                    Choreographer.getInstance().postFrameCallback(mCallback);
                    break;
                }
            }
        }
    });
}

private void postponeDraw(final long millis) {
    mHandler.sendEmptyMessageDelayed(1234, 500/*milliseconds*/);
}

Answer 2

You may need to implement Choreographer.FrameCallback to synchronize the device's frame rate apart from playing frame rate.

Some good samples are Google's Grafika. Here is a comment from RecordFBOActivity.java:

We use Choreographer so our animation matches vsync, and a separate rendering thread to keep the heavy lifting off of the UI thread. Ideally we'd let the render thread receive the Choreographer events directly, but that appears to be creating a permanent JNI global reference to the render thread object, preventing it from being garbage collected (which, in turn, causes the Activity to be retained). So instead we receive the vsync on the UI thread and forward it.

They made the RenderThread as a Looper thread. Also prepared a Handler for the thread. Implementing Choreographer.FrameCallback to the SurfaceView, they forward doFrame callback to RenderThread by posting a Handler message.