Clarify rust rayon nested thread pool worker numbers

170 Views Asked by At

I have tried to use rayon's parallel iterators in a nested way. As a global pool would have a fixed number of threads, I tend to create an outer-pool in main and an inner-pool each in threads the outer-pool spawns.

use rayon::prelude::*;

fn stress(index: u32) {
    println!("Runner {} started.", index);
    let mut _x = 0;
    loop {
        _x += 1;
        _x -= 1;
    }
}

fn sub(outer: u32) {
    let inner_pool = rayon::ThreadPoolBuilder::new().num_threads(1 as usize).build().unwrap();
    inner_pool.install(|| {
        vec![1, 2, 3, 4].into_par_iter().map(|inner| stress(outer * 10 + inner)).collect()
    })
}

fn main() {
    let outer_pool = rayon::ThreadPoolBuilder::new().num_threads(1 as usize).build().unwrap();
    outer_pool.install(|| vec![1, 2, 3, 4].into_par_iter().map(|outer| sub(outer)).collect())
}

As the code tells, I have 16 stress jobs which had been split into a 4x4 group to run, and would create an outer-pool and an inner one both containing only 1 thread. So what I had expected is only 1 job would run, and only one line (possibly Runner 11 started.) would be printed. But I got 3 jobs running in parallel and three lines (11, 31 and 41) printed.

I know rayon tends to cut the jobs in half so I understand why 31 rather than 21 should appear after 11, but I couldn't figure out why multiple jobs can run parallel in a literally 1x1 pool.

Also I had tried different pairs for the pool size, but the results still perplex me.

outer inner load
1 1 3
1 2 6
1 3 9
1 4 12
1 16 12
2 1 4
2 2 8
2 3 12
2 4 16
3 1 4
3 2 8
3 3 12
3 4 16

Can anyone shed the light on me?

0

There are 0 best solutions below