In python how to synchronize screengrabs with vertical sync

Question

I'm using python v 3.8.9 with windows 7 and grabbing portions of the screen using the mss module.

I'm attempting to capture output of a program that updates with the monitor vertical sync at ~60 FPS. Knowing the relative time (to around 1ms) of the capture with respect to the vsync is also import.

Using a high resolution timer to wait until a time of 1/FPS would have an unknown time, possibly constant, relative to the vsync.

I've made use of DwmFlush from dwmapi.dll, however it requires "desktop composition" to be enabled. When this is enabled, without waiting for the vsync, the frame rate of grabbing drops to 30 fps (from over 500 fps).

How might I determine if a new vsync has occurred?

Might it be possible to cause a screen update to block until vsync using a library such as pygame, opengl, open cv, etc.?

As the frame grab is fast, one solution would be to poll the screen by grabbing and look for a difference the grabbed images, however, I'd prefer a more elegant solution if available.

Ultimately, this wait for vsync could be a feature that is built into mss.

Answer 1

An example that uses glfw to wait for the vsync. This solution requires a glfw context window to be created, which can be hidden. As written it works only on the primary monitor.

import mss
import glfw
import time

CV2_SHOW_GRAB = False

if CV2_SHOW_GRAB :
    import numpy as NP
    import cv2

class Vsync_glfw:
    def __init__(self, useless_window_position=None, HIDE=True):

        if not glfw.init():  # Initialize GLFW
            self.window = None
            return

        # Create a windowed mode window and its OpenGL context
        self.window = glfw.create_window(320,240, title="useless? vsync", monitor=None, share=None)
        if not self.window:
            glfw.terminate()
            return

        if useless_window_position is not None :
            glfw.set_window_pos(self.window, useless_window_position[0], useless_window_position[1])

        glfw.make_context_current(self.window)
        # glfw.iconify_window(self.window)   # if the window is iconified - swap buffer doesn't wait for the vsync
        if HIDE:
            glfw.hide_window(self.window)   # since the window isn't used, it can be hidden and swap buffer still waits
        glfw.swap_interval(1)   # key to double buffering and making vsync work


    def wait(self):  # wait for vsync
        if self.window :
            # glfw.poll_events()   # runs without, but maybe is required
            glfw.swap_buffers(self.window)

    def __del__(self):
        glfw.terminate()


if __name__ == "__main__" :
    sct = mss.mss()
    vsync = Vsync_glfw(useless_window_position=(100, 50), HIDE=True)
    sct_region = dict(top=50, left=50, width=640, height=480)

    for _ in range(0,15) :
        count = 0
        tend = time.perf_counter() + 1
        while time.perf_counter() < tend:
            vsync.wait()
            frame = sct.grab(sct_region)
            count += 1

        print("FPS:%d   cursor pos:%s   window pos:%s" % (count, glfw.get_cursor_pos(vsync.window), glfw.get_window_pos(vsync.window)))

        if CV2_SHOW_GRAB :
            full_region = NP.array(frame)
            half = cv2.resize(full_region, (0, 0), fx=0.5, fy=0.5)
            cv2.imshow("half", half)
            result = cv2.waitKey(1)

    if CV2_SHOW_GRAB:
        cv2.destroyAllWindows()

Using another monitor has seems to require a system delay as the target monitor enters a full-screen mode. The full-screen mode can then be exited. In set_window_monitor when the monitor is none, the refresh rate isn't used.

monitor_list = glfw.get_monitors()
selected_monitor = monitor_list[1]
self.window = glfw.create_window(640, 480, "vsync", selected_monitor, None)
glfw.set_window_monitor(self.window, None, position[0], position[1], 50, 50, refresh_rate=10)  

glfw doc for create window

glfw doc for set window monitor