Powershell dynamic Start-Job command to execute all powershell scripts in subfolders in parallel

Question

For an important data migration, I would like to execute in parallel multiple scripts in side subfolders :

I did not succeed to do this :

$folders = Get-ChildItem -Directory

foreach($folder in $folders){
    $cmd = "powershell '$folder\script.ps1'"
    $job = start-job -ScriptBlock {Invoke-Expression $cmd}
}

Thank you for your help!

Answer 1

You can use Invoke-Command for concurrent processing. Here's an example that may meet your requirements.

Get-ChildItem -Directory | ForEach-Object {
    $folderPath = $_.FullName
    Start-Job -ScriptBlock { .\script.ps1 $using:folderPath }
}

Get-Job | ForEach-Object {
    $result = Wait-Job $_ | Receive-Job
    Remove-Job $_
    $result
}

Answer 2

Multithreaded in powershell 7:

Get-ChildItem -Directory | foreach-object -parallel {
  & $_\script.ps1
}

Answer 3

js2010's answer is the simplest and best solution for PowerShell (Core) 7+, where ForEach-Object now has a -Parallel parameter for running script blocks in parallel threads.

---

A solution for Windows PowerShell - the legacy PowerShell edition whose latest and last version is v5.1:

Get-ChildItem -Directory |
 ForEach-Object {
   Start-Job { Set-Location -LiteralPath ($using:_).FullName; .\script.ps1 }
 } |
 Receive-Job -Wait -AutoRemoveJob

Note:

• If the script needn't be run with its own directory as the working dir., the following Start-Job call will do:

  Start-Job { & (Join-Path ($using:_).FullName script.ps1) }
  

• If installing a module on demand is an option, consider installing the ThreadJob module, whose Start-ThreadJob offers a thread-based and therefore faster and more lightweight alternative to the slow and resource-intensive child-process-based background jobs that Start-Job creates.

  • Once installed, you can simply substitute Start-ThreadJob for Start-Job in the code above, given that Start-ThreadJob is compatible with all job-related cmdlets, such as Receive-Job.

  • Note that PowerShell (Core) 7+ comes with Start-ThreadJob, though the ForEach-Object -Parallel option - built on the same thread-based infrastructure - is often the simpler choice.

Answer 4

I found a working solution, by using Start-Process, it answer my need. THank you every one