Weekend Scripter: A Look at the PoshRSJob Module
Summary: Boe Prox presents a module for working with runspaces.
Honorary Scripting Guy and Cloud and Datacenter Management MVP, Boe Prox, here today filling in for my good friend, The Scripting Guy. Today I finish up my series about using runspaces in PowerShell by showing you a module that I put together that makes working with runspaces a lot easier.
Note This is a four-part series that includes the following posts:
- Beginning Use of PowerShell Runspaces: Part 1 Begin use with runspaces.
- Beginning Use of PowerShell Runspaces: Part 2 Begin use with runspaces.
- Beginning Use of PowerShell Runspaces: Part 3 Use runspace pools for multithreading.
- A Look at the PoshRSJob Module Learn about a module for working with runspaces.
There are some great scripts out there that provide a tool to work with runspaces, such as Invoke-Parallel. I wanted to come up with something that would provide a familiar approach to spinning off multiple commands in parallel—like what we have with the *-Job cmdlets. With that in my mind, I set out to basically copy those cmdlets by using the same naming scheme and parameter names as those cmdlets. But instead of relying on the PSJobs framework, I instead rely on runspaces and runspace pools to provide a way to run commands and other tasks concurrently with the benefits of runspaces.
My module, PoshRSJob, does just that, and it is available to download a number of ways. If you have access to the PowerShellGet module, you can easily download the module by using Save-Module. (This is better for those who want to inspect their downloaded modules before actually installing them.)
Save-Module –Name PoshRSJob –Path C:\ModulesToReview
Or if you want to install it without having to save it first, you can use Install-Module instead:
Install-Module –Name PoshRSJob
Another option is to head over to my GitHub page, proxb/PoshRSJob, and download the zip file. GitHub is where I post my code prior to placing it in the PowerShell Gallery. I am always open to folks who want to make my code better or fix a bug that they may find. so be sure to fork it and submit a pull request if you want to help out.
Now that you have the module available and ready to use, the first thing we should do is look at the available functions that you can use. With the exception of the RS being prepended to each noun, the commands will look pretty familiar to you.
Get-Command –Module PoshRSJob
There are a few commands missing (Suspend, Resume, and Debug), but the core commands are here to work with.
I won’t take you through each command and its parameters, but I can assure you that the parameters are almost identical to the ones that you would find using the *-Job cmdlets. This is definitely by design because I wanted a familiar approach for running the commands. Even the output of the RSJob objects will have a familiar look.
Let’s start with a quick example of how we would use this module. I am going to use a range of numbers and send them to Start-RSJob to create a job for each number. One of the most important things that I can stress about doing this is that you should never use ForEach with Start-RJob because it will create a runspace pools for each iteration of ForEach.
All of the processing of the objects happens within Start-RSJob. You will have to make use of $_ in your script block to represent each item that is piped to Start-RSJob, as shown in the following example:
$Test = ‘test’
$Something = 1..10
1..5|start-rsjob -Name {$_} -ScriptBlock {
[pscustomobject]@{
Result=($_*2)
Test=$Using:Test
Something=$Using:Something
}
}
I also have the ability to include the $Using: variable, which you may have seen before. This allows you to use a variable from the parent scope within the runspace job so you do not have to worry about messing with a Param() block in your script block, and then ensure you have the order correct with the –ArgumentList parameter. I will add that you can still do this if you want to because I wanted to keep that familiar functionality available.
Now, to check if they have completed yet:
Get-RSJob
Looks like they are not only complete, but they also have data available to retrieve. I can do that by using Receive-RSJob and display the output from the RSJobs:
Get-RSJob | Receive-RSjob
This example isn’t really that impressive, but you can see that all of my jobs completed in the background as runspaces in a runspace pool, and I was able to do all of this by using commands that you are familiar with!
I’ll finish up by removing those RSJobs because they are no longer needed”
Get-RSJob | Remove-RSJob
The last example might be a little more useful than simply looking at a collection of numbers. The following example also utilizes a custom function of mine that I want to have available within my runspace pool, so it can be used by all of the runspaces.
Function Convert-Size {
[cmdletbinding()]
Param (
[parameter(ValueFromPipeline=$True,ValueFromPipelineByPropertyName=$True)]
[Alias(“Length”)]
[int64]$Size
)
Begin {
If (-Not $ConvertSize) {
Write-Verbose (“Creating signature from Win32API”)
$Signature = @”
[DllImport(“Shlwapi.dll”, CharSet = CharSet.Auto)]
public static extern long StrFormatByteSize( long fileSize, System.Text.StringBuilder buffer, int bufferSize );
“@
$Global:ConvertSize = Add-Type -Name SizeConverter -MemberDefinition $Signature -PassThru
}
Write-Verbose (“Building buffer for string”)
$stringBuilder = New-Object Text.StringBuilder 1024
}
Process {
Write-Verbose (“Converting {0} to upper most size” -f $Size)
$ConvertSize::StrFormatByteSize( $Size, $stringBuilder, $stringBuilder.Capacity ) | Out-Null
$stringBuilder.ToString()
}
}
Get-ChildItem $PWD -Directory | Start-RSJob -Name {$_.Name} -ScriptBlock {
[int64]$Bytes = (Get-ChildItem -Path $_.FullName -Force -Recurse | Measure-Object -Property length -Sum).Sum
[pscustomobject]@{
Name = $_.Name
Size = Convert-Size -Size $Bytes
Size_bytes = $Bytes
}
Start-Sleep -Seconds (Get-Random -InputObject (1..5))
} -FunctionsToLoad Convert-Size -Throttle 5 | Wait-RSJob -ShowProgress |
Receive-RSJob | Sort-Object -Property Size_bytes -Descending
Get-RSJob | Remove-RSJob
I also make use of the Wait-RSJob function here, which gives me a little progress bar (shown in the following image), so I know the current status of the RSJobs. At the end, I retrieve the data and dispose of the jobs.
In addition to being able to import custom functions, you can import modules by using –ModulesToImport. You can also work with PSSnapins if you need to import one of those.
If you want to know more about this module, be sure to hit up my GitHub page, proxb/PoshRSJob. If you want to help make this module even better, be sure to contribute!
I invite you to follow the Scripting Guy on Twitter and Facebook. If you have any questions, send email to the Scripting Guy at scripter@microsoft.com, or post your questions on the Official Scripting Guys Forum. Until then, see ya!
Boe Prox, Microsoft Cloud and Datacenter Management MVP and Honorary Scripting Guy
Light
Dark
1 comment
Link to other parts are broken.