I developed a JavaScript class for streaming audio. It works on desktop browsers, but I'm encountering an issue on iOS mobile browsers. Specifically, the audio plays for the first 2-3 seconds (which I believe corresponds to the first chunk), but then it stops, and I receive an error in the decodeAndPlayAudio function.

I am using websocket to get the chunks of audio data(encoded). The AudioStream class manages the streaming and playback of audio data in chunks. Chunks are decoded and queued, with playback starting once enough data is buffered. The class ensures playback, handling decoding errors and streaming continuity. As chunks are played sequentially, it monitors for the end-of-stream, triggering any final callbacks.

Sometimes this.audioContext.state is suspended on mobiles (it is very inconsistent) but even after forcing it to resume (this.audioContext.resume()) in all the scenarios, the audio is still not streaming and only first chunk plays.

Audio stream class below:

export default class AudioStream {
  constructor(onStreamEndedCallback, bufferThreshold = 2) {
    this.audioContext = new (window.AudioContext || window.webkitAudioContext)();
    this.chunkQueue = [];
    this.isPlaying = false;
    this.onStreamEnded = onStreamEndedCallback;
    this.bufferThreshold = bufferThreshold;
  }

  async resumeAudioContext() {
    await this.audioContext.resume();
  }

  processChunk(chunk, isFinalChunk = false) {
    if (isFinalChunk) {
      this.chunkQueue.push(null);
      this.checkAndPlay();
      return;
    }

    try {
      const decodedChunk = atob(chunk);
      const byteNumbers = Uint8Array.from(decodedChunk, (c) => c.charCodeAt(0));
      this.chunkQueue.push(byteNumbers);
      this.checkAndPlay();
    } catch (e) {
      alert("Error decoding chunk: " + e.message);
      console.error("Error decoding chunk:", e.message);
    }
  }

  checkAndPlay() {
    if (this.chunkQueue.length >= this.bufferThreshold && !this.isPlaying) {
      this.playNextChunk();
    }
  }

  async playNextChunk() {
    if (this.isPlaying || this.chunkQueue.length === 0) {
      return;
    }

    this.isPlaying = true;
    const byteArray = this.chunkQueue.shift();

    if (byteArray === null) {
      if (this.onStreamEnded) {
        this.isPlaying = false;
        this.onStreamEnded();
      }
      return;
    }

    try {
      const buffer = await this.decodeAndPlayAudio(byteArray);
      await this.playAudioBuffer(buffer);
    } catch (e) {
      let detailedMessage = `Error decoding chunk. Details: ${e.message}.`;

      alert("Error with decoding audio data: " + this.audioContext.state + detailedMessage);
      console.error("Error with decoding audio data:", e.message);
      this.isPlaying = false;
      this.playNextChunk();
    }
  }

  async decodeAndPlayAudio(byteArray) {
    try {
      const audioBuffer = await this.audioContext.decodeAudioData(byteArray.buffer);
      return audioBuffer;
    } catch (e) {
      let errorDetails = {
        message: e.message,
        name: e.name,
        stack: e.stack,
      };

      let detailedMessage = `Error decoding chunk. Details: ${JSON.stringify(
        errorDetails
      )}. AudioContext State: ${this.audioContext.state}. Byte Array Length: ${byteArray.length}`;

      console.error("Error decoding audio data:", e);
    }
  }

  async playAudioBuffer(buffer) {
    const source = this.audioContext.createBufferSource();
    source.buffer = buffer;
    source.connect(this.audioContext.destination);
    source.start(0);

    source.onended = () => {
      this.isPlaying = false;
      this.playNextChunk();
    };
  }
}

0

There are 0 best solutions below