November 28th, 2015

Beginning Use of PowerShell Runspaces: Part 3

Doctor Scripto
Scripter

Summary: Boe Prox shows us some tips about using runspace pools for multithreading.

Honorary Scripting Guy and Cloud and Datacenter Management MVP, Boe Prox, here today filling in for my good friend, The Scripting Guy.

Note   This is a four-part series that includes the following posts:

After spending the past couple of days working with runspaces and looking at how we can use parameters and arguments to supply variables to our runspaces, we are taking the next step in our journey by looking at runspace pools to do some multithreading with multiple commands. The process for working with runspace pools is similar to working with runspaces. There are a few minor differences in how we use the runspace pools with the PowerShell instance.

By using a runspace pool, we cannot only run multiple commands at the same time (which we could do using multiple runspaces), but we now have the capability to throttle the number of runspaces that are running concurrently. For example, I may have 50 items that I need to scan, but because of resource limits, I can only run six at a time. I queue up all of the objects, but only six will run at a given time. A new one will start when another one ends.

We can create the runspace pools by using the CreateRunspacepool() method from the [runspacefactory] type accelerator. We’ll provide some additional data during this construction that tells the runspace pools the limits for minimum runspaces and the maximum runspaces that can be allowed to run at a time (our throttle).

I am going to set up some parameters because I want to show how you can add that data (just like we did with runspaces):

$Parameters = @{

    Param1 = ‘Param1’

    Param2 = ‘Param2’

}

Next I will build out the runspace pools with a throttle of 10 and open the RunspacePool property so I can start working with it to add my script block.

#region Runspace Pool

[runspacefactory]::CreateRunspacePool()

$SessionState = [System.Management.Automation.Runspaces.InitialSessionState]::CreateDefault()

$RunspacePool = [runspacefactory]::CreateRunspacePool(

    1, #Min Runspaces

    5 #Max Runspaces

)

$PowerShell = [powershell]::Create()

#Uses the RunspacePool vs. Runspace Property

#Cannot have both Runspace and RunspacePool property used; last one applied wins

$PowerShell.RunspacePool = $RunspacePool

$RunspacePool.Open()

#endregion

With that accomplished, I can move forward to start running commands against my systems (or items) by creating a script block with my commands:

$jobs = New-Object System.Collections.ArrayList

1..50 | ForEach {

    $PowerShell = [powershell]::Create()

    $PowerShell.RunspacePool = $RunspacePool

    [void]$PowerShell.AddScript({

        Param (

            $Param1,

            $Param2

        )

        $ThreadID = [appdomain]::GetCurrentThreadId()

        Write-Verbose “ThreadID: Beginning $ThreadID” -Verbose

        $sleep = Get-Random (1..5)

        [pscustomobject]@{

            Param1 = $param1

            Param2 = $param2

            Thread = $ThreadID

            ProcessID = $PID

            SleepTime = $Sleep

        }

        Start-Sleep -Seconds $sleep

        Write-Verbose “ThreadID: Ending $ThreadID” -Verbose

    })

    [void]$PowerShell.AddParameters($Parameters)

    $Handle = $PowerShell.BeginInvoke()

    $temp = ” | Select PowerShell,Handle

    $temp.PowerShell = $PowerShell

    $temp.handle = $Handle

    [void]$jobs.Add($Temp)

    Write-Debug (“Available Runspaces in RunspacePool: {0}” -f $RunspacePool.GetAvailableRunspaces())

    Write-Debug (“Remaining Jobs: {0}” -f @($jobs | Where {

        $_.handle.iscompleted -ne ‘Completed’

    }).Count)

}

There is a bit happening here that I need to explain. First, you probably noticed the ArrayList that I created to handle the jobs. I do this because as I add a new command to run in the PowerShell instance that uses RunspacePool, I need to call BeginInvoke() to kick off the command (if it is being helped due to throttling). That kicks back an Async object that I need to monitor for completion.

Also, I chose the script block that displays a simple object with some threading information. These work in a throttle of 10 items (threads) and will never use additional threads because the default ThreadOption for RunspacePool is ReuseThread.

Now that I have run all of my runspace jobs, I can sit back and wait for them to finish. I can check on them and when they are finished, I call EndInvoke() on each one to return the results of my scan.

#Verify completed

Write-Debug (“Available Runspaces in RunspacePool: {0}” -f $RunspacePool.GetAvailableRunspaces())

Write-Debug (“Remaining Jobs: {0}” -f @($jobs | Where {

    $_.handle.iscompleted -ne ‘Completed’

}).Count)

$return = $jobs | ForEach {

    $_.powershell.EndInvoke($_.handle)

    $_.PowerShell.Dispose()

}

$jobs.clear()

And for fun, I can look at the data to see that I only used a throttle of 10:

$return | Group ProcessID | Select Count, Name

$return | Group Thread | Select Count, Name

($return | Group Thread).Count

As we can see, my throttle of 10 was respected during the run against 50 items. I never went over that limit, and I was able to successfully perform my task.

Tomorrow I will wrap up my series about PowerShell runspaces by showing a module that I put together. It can allow you to use runspaces and runspace pools with ease by giving a familiar look and feel to them. Stay tuned!

I invite you to follow the Scripting Guy on Twitter and Facebook. If you have any questions, send email to the Scripting Guy at scripter@microsoft.com, or post your questions on the Official Scripting Guys Forum. Until then, see ya!

Boe Prox, Windows PowerShell MVP and Honorary Scripting Guy

Author

The "Scripting Guys" is a historical title passed from scripter to scripter. The current revision has morphed into our good friend Doctor Scripto who has been with us since the very beginning.

8 comments

Discussion is closed. Login to edit/delete existing comments.

  • heweiguang

    Intentionally write the code incorrectly?Maybe it’s a webpage revision error

  • Balaji Kithiganahalli

    You say
    Next I will build out the runspace pools with a throttle of 10 and open the RunspacePool property so I can start working with it to add my script block.
    #region Runspace Pool
    [runspacefactory]::CreateRunspacePool()
    $SessionState = [System.Management.Automation.Runspaces.InitialSessionState]::CreateDefault()
    $RunspacePool = [runspacefactory]::CreateRunspacePool(
    1, #Min Runspaces
    5 #Max Runspaces
    )

    But you are setting Max Runspaces to 5. Is Max Runspaces and Throttle are same? If not, Where is...

    Read more
    • Wil Wilder Apaza Bustamante

      this comment has been deleted.

  • Mikel V.

    Is there a way to execute a script from a runspace back to the main console?I want my WPF form to run in different runspace, the form shows a list of commands that I can execute. would be perfect if they launch in main console window to see the messages they throw…

  • Harish

    I seriously want you guys to update this series of articles, seems multiple issues with code.. I don’t think its a friendly blog for a Powershell learner
    “$temp = ” | Select PowerShell,Handle” can you please explain what does this means?

    • Martin C

      Hi, $temp = ” | Select PowerShell,Handle, is way how to create PS object, only I think there should be pair of quotes, not just single quote.
      Will create object having PowerShell and Handle properties.

      • Brecht Gijbels

        I had a deeper look into this and it’s just another way of creating an object. Bot ways are equal:

        $Person = ” | Select-Object SurName, FirstName

        $Person = [PSCustomObject]@{
        SurName = $null
        FirstName = $null
        }

      • Steve Rochford

        Many thanks for this - I was completely mystified!

        I've now been playing with this and it looks like it's just a way to create a kind of user defined object which has any properties you need. For example, you could do:

        $person= '' | select surname,firstname,dob

        and you'll get an object of type Selected.System.String which has members called surname,firstname,dob all of type NoteProperty and currently with null value (check by doing $person | get-member)

        I'm not sure what...

        Read more