I've been streaming video from the HoloLens 2 to a FastAPI Python server using the Device Portal API found here: https://learn.microsoft.com/en-us/windows/mixed-reality/develop/advanced-concepts/device-portal-api-reference and it works great for getting the mixed or non-mixed reality view alone. However, I can't figure out a way to get these streams simultaneously, that is when I make a GET request for the stream with holograms, then make a GET request for the stream without, the previous stream stops. I imagine this is because the HoloLens is restricting multiple accesses to the camera. Is there any way around this?
My current workaround has been accessing the first stream, stopping the request, requesting the other stream, and so on. The problem with this is there's a significant latency when starting each new request that is not present when reading from only one stream. I'd like to find a method where two processes can read from each individual stream at once. I've also tried using the PhotoCapture class within Unity, and switching back and forth between holograms being visible, but this too introduces high latency.