Puppeteer-Cluster consistently using only half of my cores

Question

I'm running a pretty standard puppeteer cluster job, with the following settings:

const cluster = await Cluster.launch({
    concurrency: Cluster.CONCURRENCY_PAGE, // maximize sharing data between jobs
    maxConcurrency: 8,
    monitor: true
}

However, my cpu monitor never shows more than 4 cores spiking.

Forcing maxConcurrency higher shows more workers running, but the cpu utilization is unchanged.

I've read dozens of StackOverflow threads but I can't find anything to addresses this phenomenon, and none of the other tricks to maximize performance (changing the concurrency model, passing custom chrome args, etc) have any effect on the overall cpu utilization.

FWIW, I'm on Mac OS Ventura running node v16.19.0, with an 8-core i7 from 2019. (One theory I have is that there's some OS/node thread affinity thing going on here...)

Answer 1

Aaaaand it turns out this is indeed a platform issue, but it's even lower-level than OS/node!

My octo-core i7 isn't octo-core at all: it uses SMT, to run two threads in parallel on a single core. So I'm actually dealing with four cores, and node's worker threads are maxing them out.

This is described in detail here: https://github.com/denoland/deno/issues/10592

I'm leaving this up, in case someone else runs into the same issue.