DOM manipulation of proxy response in express node using Cheerio and http-proxy-middleware

Question

I've setup proxy server in express Node, which is working fine.

const express = require('express');
const { createProxyMiddleware } = require('http-proxy-middleware');

const app = express();

// Define your proxy rules here
app.use(
  '/', // Replace with the path you want to proxy
  createProxyMiddleware({
    target: 'https://www.lemonade.com/',
    changeOrigin: true,
  })
);

// Start the Express server on your desired port
const port = process.env.PORT || 3000; // Use an appropriate port
app.listen(port, () => {
  console.log(`Proxy server is running on port ${port}`);
});

Now, I'd like to perform DOM manipulation but it is not working. Below is my code so far:

const express = require("express");
const { createProxyMiddleware } = require("http-proxy-middleware");
const cheerio = require("cheerio");

const app = express();

// Define your proxy rules here
app.use(
  "/", // Replace with the path you want to proxy
  createProxyMiddleware({
    target: "https://www.lemonade.com/",
    changeOrigin: true,
    onProxyRes: (proxyRes, req, res) => {
      let body = [];

      proxyRes.on("data", (chunk) => {
        body.push(chunk);
      });

      proxyRes.on("end", () => {
        body = Buffer.concat(body).toString();

        // Use Cheerio to load the HTML content
        const $ = cheerio.load(body);

        // Now you can manipulate the DOM using Cheerio
        // For example, let's change the title of the page
        $("title").text("Modified Title");

        // Set the manipulated HTML content as the response
        const modifiedHTML = $.html();
        console.log("modifiedHTML:", modifiedHTML);

        // Send the response here, outside of the Cheerio operations
        res.send(modifiedHTML);
      });
    },
  })
);

// Start the Express server on your desired port
const port = process.env.PORT || 3000; // Use an appropriate port
app.listen(port, () => {
  console.log(`Proxy server is running on port ${port}`);
});

I'm getting this error:

Error [ERR_HTTP_HEADERS_SENT]: Cannot set headers after they are sent to the client

It looks like I'm missing something.

Answer 1

You need to add selfHandleResponse: true option because along with your call res.send, res.end is being called automatically, hence the headers already sent error, and this flags prevents it:

 createProxyMiddleware({
    target: "https://www.lemonade.com/",
    changeOrigin: true,
    selfHandleResponse: true,
    onProxyRes: (proxyRes, req, res) => {
        //...

see:

option.selfHandleResponse true/false, if set to true, none of the webOutgoing passes are called and it's your responsibility to appropriately return the response by listening and acting on the proxyRes event

/**
 * IMPORTANT: avoid res.end being called automatically
 **/
selfHandleResponse: true, // res.end() will be called internally by responseInterceptor()

example