CSP works with Next.js prod build but not on dev server

Question

I've been trying to implement CSP on my Next.js application, and I've managed to get it working when running next build and next start, but for some reason when running next dev it doesn't render at all, and the following error is logged on the console:

Uncaught EvalError: Refused to evaluate a string as JavaScript because 'unsafe-eval' is not an allowed source of script in the following Content Security Policy directive: "script-src 'self'".
    at ./node_modules/next/dist/compiled/@next/react-refresh-utils/dist/runtime.js (react-refresh.js?ts=1710261451116:30:26)

I tried following the suggestions in the Next.js docs as well as in this GitHub discussion of generating a nonce using middleware, but I was only able to get CSP working by adding the header to next.config.js. I'm also using the pages router, if that info is helpful. This is what the relevant parts of my next.config.js file looks like:

const cspHeader = `
  default-src 'self';
  script-src 'self';
  style-src 'self' 'unsafe-inline' fonts.googleapis.com;
  font-src 'self' https:;
  img-src 'self' blob: data:;
  object-src 'none';
  connect-src 'self' https:;
  upgrade-insecure-requests;
`

module.exports = {
  ...
  async headers() {
    return [
      {
        source: '/(.*)',
        headers: [
          {
            key: 'Content-Security-Policy',
            value: cspHeader.replace(/\n/g, ''),
          },
        ],
      },
    ]
  },
}

Any idea on how to resolve this issue (preferably without modifying the CSP header just for local development)?