Accessing streaming flow of data (server events)

16 Views Asked by At

I have the following function to get a completion from OpenAI in streaming:

   ` const accumulateText = async () => {
      return new Promise((resolve, reject) => {
        res.data.on('data', (data) => {
          const lines = data.toString().split('\n').filter((line) => line.trim() !== '');
          for (const line of lines) {
            const message = line.replace(/^data: /, '');
            if (message === '[DONE]') {
              resolve(accumulatedText); // Resolves the promise with the accumulated text
              return; // Stream finished
            }
            try {
              const parsed = JSON.parse(message);
              accumulatedText += parsed.choices[0].text;

            } catch (error) {
              console.error('Could not JSON parse stream message', message, error);
            }
          }
        });
        res.data.on('error', (error) => {
          reject(error); // Rejects the promise if an error occurs
        });
      });
    };`

This is working code and I am able to access the string after receiving the whole response but I need to access the tokens as they come to be able to start giving an aswer to the customer as soon as possible as I´m building a voice bot and waiting for the whole response is too much time. Also notice that this is s skill for Alexa in Node.js so I am somehow limited.

It also would be enough for me to get 10 tokens of the response each time I to invoke the function (finishing the loop beforehands) but the first time I invoce it it gives me the tokens correctly but the second time it does nothing. I think it´s because it is not possible to access res.data a second time.

I am getting mad with this. If anybody could help me it would be great!!!

If possible working code to solve the problem. Otherwise some advice, please.

0

There are 0 best solutions below