Unable to Playback MediaRecorder Blobs Streamed over WebSocket

Question

Unable to Playback MediaRecorder Blobs Streamed over WebSocket

47 Views Asked by lincolnberryiii At 24 March 2024 at 16:54

I am unable to get blobs recorded by an instance of MediaRecorder that are sent over WebSocket to playback on the client machine's video element. Here is an excerpt of my code:

sender.js:

navigator.getUserMedia({
  video: {
      facingMode: 'user'
  },
  audio = true
}).then((stream) => {
    const recorder = new MediaRecorder(stream);
    recorder.ondataavailable((event) => {
        socket.send(event.data); // according to the client console, THIS SENDS A BLOB!!!
    });
    recorder.start(1000);
});

receiver.js:

socket.onmessage = (message) => {
    console.log(message.data); // instanceof Blob
    blob = message.data;
    // tried this  answer but it still does not work: https://stackoverflow.com/a/72772924/1079320
    blob = new Blob([message.data], { // the Blob.type attribute is ignored when sent by WebSocket: https://developer.mozilla.org/en-US/docs/Web/API/WebSocket/send
        type: 'video/webm; codecs=vp9'
    });
    // blob = blob.slice(0, blob.size, 'video/webm; codecs=vp9'); // this answer does not work: https://stackoverflow.com/a/50875615/1079320
    video.srcObject = blob; // under "Usage notes" from https://developer.mozilla.org/en-US/docs/Web/API/HTMLMediaElement/srcObject the URL.createObjectUrl() method is no longer needed!
    video.play(); // const video = document.getElementById('video-element-id');
};

I get no errors, but the video element simply does not play. Testing using the Firefox browser.

Original Q&A

There are 2 best solutions below

**guest271314** · Answer 1 · 2024-03-24T17:01:13.040000

recorder.ondataavailable(event => { should be recorder.ondataavailable = (event) => {} or recorder.addEventListener("dataavailable", (event) => {}). If message.data is a Blob and not an ArrayBuffer you don't have to create a new Blob, just use the Blob defined on the event. Additionally, Blob set as src won't work. You are not setting a MediaStream to srcObject. Pass the Blob to URL.createObjectURL() then set the Blob URL as the HTMLMediaElement src.

socket.onmessage = (message) => {
  video.src = URL.createObjectURL(message.data);
  video.play(); 
};

**lincolnberryiii** · Answer 2 · 2024-03-24T21:49:53.500000

I've finally figured it out: The key was to use the MediaSource API. Example code can be found under "Examples" here:

https://developer.mozilla.org/en-US/docs/Web/API/MediaSource/MediaSource

Firstly, it is important to note that I couldn't make this code work within the body.onload listener I was using before: Instead, I had to put the <script> tag directly in the <body> of the HTML document itself.

Furthermore, I had to pass the raw Blob contents as an arrayBuffer using the blob.arrayBuffer() promise.

That said, here is the revised version of my code:

receiver.js:

let mediaSource = new MediaSource;
video.src = URL.createObjectURL(mediaSource); // const video = document.getElementById('video-element-id');
let sourceBuffer;

socket.onmessage = (message) => {
    console.log(message.data); // remember, this is instanceof Blob
    if (!sourceBuffer) {
        sourceBuffer = mediaSource.addSourceBuffer('video/webm');
        video.play();
    }
    message.data.arrayBuffer().then((arrayBuffer) => {
        sourceBuffer.appendBuffer(arrayBuffer);
    });
};

And BAH-BAM! Everything is working as it should now.

Thanks to "guest271314" for suggesting I look into MediaSource.

Unable to Playback MediaRecorder Blobs Streamed over WebSocket

There are 2 best solutions below

Related Questions in JAVASCRIPT

Related Questions in WEBSOCKET

Related Questions in WEB-MEDIARECORDER

Trending Questions

Popular # Hahtags

Popular Questions