foreach vs ForEach-Object -Parallel in powershell - what is wrong?

101 Views Asked by At

I am confused with ForEach-Object -Parallel. The following code has $blobs array with more than 2000 blobs. With the regular foreach I can print the names of each blob without a problem. Then using the ForEach-Object -Parallel just after the first foreach, nothing is printed. Why ?

foreach ($blob in $blobs) {
           Write-Host    $blob.Name
}

# Use parallel processing to process blobs concurrently
$blobs|ForEach-Object -Parallel {
         param (
             $blob)
    
        Write-Host $blob.Name 
} -ThrottleLimit 300
1

There are 1 best solutions below

0
mklement0 On

The script block passed to a ForEach-Object cmdlet call - whether or not you use -Parallel - receives its (pipeline) input implicitly, via the automatic $_ variable (aka $PSItem). (In other words: there's no need for or point in declaring a parameter):

$blobs |
  ForEach-Object -Parallel {
    $_.Name  # $_ implicitly refers to the pipeline input object at hand
  } 
  • By contrast, the foreach statement requires a self-chosen iterator variable (such as $blob in your example).

    • Perhaps confusingly, foreach is also an alias of the ForEach-Object cmdlet, and it is the syntactic context that decides whether foreach refers to the language statement or the cmdlet.
  • Note that the -ThrottleLimit parameter controls how many threads are allowed at a time. For CPU-bound operations, it doesn't make sense to specify a number higher than the available CPU cores; only for operations that are network-bound, I/O-bound, or wait for events do higher numbers make sense.

  • With respect to the ForEach-Object cmdlet, while $_ is used both with or without -Parallel, there is an important difference with respect to referencing variables from the caller's scope:

    • With -Parallel, you need the $using: scope; e.g.:

      $foo = 'bar'
      # Note the $using:foo reference.
      # *Without* -Parallel, use just $foo
      # -> 'bar1', 'bar2', 'bar3'
      1..3 | ForEach-Object -Parallel { $using:foo + $_ }