Why do process blocks correctly return a hashtable, instead of an array of hashtables?

Question

I am trying to sort some REST API responses and extract data from them, and found this helpful answer on how to end up with a hashtable at the end.

$res= (Invoke-RestMethod @commonParams -uri "foo" -Method GET -FollowRelLink) | ForEach-Object { $_ } | Select-Object -Property id, filename | ForEach-Object -begin {$h=@{}} -process {$h[$_.filename] = $_.id} -end {$h}

Great, so I thought I'd shorten it a bit by getting rid of the begin/process/end blocks like so:

$res = (Invoke-RestMethod @commonParams -uri "foo" -Method GET -FollowRelLink) | ForEach-Object { $_ } | Select-Object -Property id, filename | ForEach-Object { $h=@{}; $h[$_.filename] = $_.id; $h }

All good, I should now be able to reference each item by it's key value using square brackets, but that doesn't work, only referencing by dot notation.

So I went searching and found this answer, which turned out to be correct:

$res.GetType()

IsPublic IsSerial Name                                     BaseType
-------- -------- ----                                     --------
True     True     Object[]                                 System.Array

But, even changing from the generic hashtable constructor to using the .Add() method didn't help:

$res = (Invoke-RestMethod @commonParams -uri "foo" -Method GET -FollowRelLink) | ForEach-Object { $_ } | Select-Object -Property id, filename | ForEach-Object { $h=@{}; $h.add($_.filename, $_.id); $h }

$res.GetType()

IsPublic IsSerial Name                                     BaseType
-------- -------- ----                                     --------
True     True     Object[]                                 System.Array

I add the process blocks back in:

$res = (Invoke-RestMethod @commonParams -uri "$foo" -Method GET -FollowRelLink) | ForEach-Object { $_ } | Select-Object -Property id, filename | ForEach-Object -begin {$h=@{}} -process {$h[$_.filename] = $_.id} -end {$h}

$res.GetType()

IsPublic IsSerial Name                                     BaseType
-------- -------- ----                                     --------
True     True     Hashtable                                System.Object

And it's all working as expected.

So I'd like to know, if possible, what exactly is happening when I use the process blocks as opposed to just making 'fake' multi-line functions in the pipeline?

And if anyone cares, I actually ended up avoiding the above issue completely by using Group-Object from here.

Answer 1

ForEach-Object emulates how script blocks work in the pipeline. The reason why the first and last snippets work as expected, returning a hashtable is because -Begin and -End not because of -Process which executes by default and is a mandatory parameter. If you don't use -Begin you would simply be creating and outputting a new hashtable per input object.

• Using ForEach-Object

0..10 | ForEach-Object -Begin {
    "executes only once, before the first input object is processed"
} -Process {
    "processes each input object: $_"
} -End {
    "executes only once, after the last input object is processed"
}

• Using a script block

0..10 | & {
    begin {
        "executes only once, before the first input object is processed"
    }
    process {
        "processes each input object: $_"
    }
    end {
        "executes only once, after the last input object is processed"
    }
}

about_Functions_Advanced_Methods explain how these blocks work.

Also your code could be simplified to this:

$h = @{}
Invoke-RestMethod @commonParams -uri "foo" -Method GET -FollowRelLink |
    Write-Output |
    ForEach-Object { $h[$_.filename] = $_.id }

Or using Group-Object -AsHashTable as you have already found:

$h = Invoke-RestMethod @commonParams -uri "foo" -Method GET -FollowRelLink |
    Write-Output |
    Group-Object FileName -AsHashTable

Write-Output in this case may not be needed if the output from Invoke-RestMethod is already enumerated.