Load image from byte array without onload event

56 Views Asked by At

I want to show a MJPEG stream on a website with angular. I also want to know the FPS and get notified, when the stream is not sending images, so that I can restart it. The problem is, that chrome is not firing an onload event for each frame (image). So I do not know the FPS and chrome is not fireing an event, when the stream stops. For that I have writen a pipe for the HTTP stream in angular, which splits the stream in single images. Each single image is then added to the srcproperty of the image element as an data:image/png;base64 This is working really fine, feel free, to use the code below in your own projects.

import { Observable, Subscriber } from "rxjs";
import { MjpegModel } from "../Models/mjpegModel";

export class MjpegReader {
    private stream: ReadableStreamDefaultReader<Uint8Array>;
    private seperatorBytes: any[] = [13, 10, 13, 10];

    public fps: number;
    public isActive: boolean;
    public imageReceived: (value: Uint8Array) => any;
    private abortController: AbortController;

    public start():Observable<MjpegModel> {
        return this.read();
    }

    private timerTick: Date = new Date();
    private pictureCount: number = 0;

    constructor(stream: ReadableStreamDefaultReader<Uint8Array>, abortController: AbortController) {
        this.stream = stream;
        this.abortController = abortController;
        window.setInterval(() => {
            let timerDelay = new Date().getTime() - this.timerTick.getTime();
            this.fps = (this.pictureCount / timerDelay) * 1000;
            this.isActive = this.fps > 0;
            this.timerTick = new Date();
            this.pictureCount = 0;
        }, 1000);
    }

    private read():Observable<MjpegModel> {
        let bla = 
         new Observable<MjpegModel>((obs) => {  
            this.continueReading(obs);
        });        
        return bla;
    }

    private async continueReading(obs: Subscriber<MjpegModel>){
        let result = await this.stream.read();
        let buffer: any[] = [];
        let header: any[] = [];
        let content: any[] = [];
        let contentLength: number = -1;
        let headerFound: boolean = false;
        while(!result.done) {
            if(result.value) {
                buffer = [].concat(this.copyArrayBuffer(result.value, 0, result.value.length));
                while(buffer
                    && buffer.length > 0){
                    if(!headerFound){                                
                        let headerEnd = -1;
                        headerEnd = this.searchBytes(buffer, this.seperatorBytes);
                        if (headerEnd > 0) {                                
                            headerFound = true;
                            header = header.concat(buffer.splice(0, (headerEnd + this.seperatorBytes.length)));                                

                            let headerString = '';
                            for (let i = 0; i < header.length; i++) {
                                headerString += String.fromCharCode(+header[i]);
                            }
                            contentLength = +(headerString.split('Content-Length:')[1].replace(/\D/g, ''));
                        }
                        else {
                            header = header.concat(buffer);
                            buffer = [];
                        }
                    }

                    if(buffer.length > 0) {
                        let remainingContentLength = contentLength - content.length;
                        if(buffer.length > remainingContentLength){
                            content = content.concat(buffer.splice(0, remainingContentLength));
                        }    
                        else {
                            content = content.concat(buffer);
                            buffer = [];
                        }
                    }

                    if(content.length === contentLength) {
                        this.pictureCount++;
                        this.isActive = true;
                        let mjpeg = new MjpegModel();
                        mjpeg.TimeStamp = new Date();
                        mjpeg.Header = new Uint8Array(header);
                        mjpeg.Value = new Uint8Array(content);
                        mjpeg.ValueAsString = "data:image/png;base64," + window.btoa(String.fromCharCode.apply(null, mjpeg.Value));
                        mjpeg.CurrentFrameRate = this.fps;
                        header = [];
                        content = [];
                        headerFound = false;
                        contentLength=-1;
                        obs.next(mjpeg);
                    }
                }
            }
            result = await this.stream.read();
        }                
    }

    private copyArrayBuffer(source: Uint8Array, offset: number, count: number): any[] {
        let bytes: any[] = [];
        for (let j = 0; j < count; j++) {
            bytes.push(source[j + offset]);
        }
        return bytes;
    }

    private searchBytes(haystack: any[], needle: any[]) {
        var len = needle.length;
        var limit = haystack.length - len;
        for (var i = 0; i <= limit; i++) {
            var k = 0;
            for (; k < len; k++) {
                if (needle[k] != haystack[i + k]) {
                    break;
                }
            }
            if (k == len) {
                return i;
            }
        }
        return -1;
    }

    public stop() {
        if (this.stream) {
            this.stream.cancel();
        }

        if(this.abortController) {
            this.abortController.abort();
        }
    }
}

But there is one problem. Each single frame will result as an event in the network tab of the developer console. There are 20 FPS. When the developer console is open, Chrome will crash after some minutes, because of a memeory exception. When the developer console is closed, the website is running for days without problems, only allocating something about 60 MBs. Is there a way, to load an image from a byte array, without fireing the onload event, so that the developer console will not crash the browser?

enter image description here

0

There are 0 best solutions below