Microsoft Compute Cluster now has a PowerShell Provider and Cmdlets

Steve Lee

Today the Compute Cluster group announced the release of their Tool Pack HERE. This includes a monitoring tool, a ping/pong diagnostic tool and (drum roll please……) a PowerShell Snap-in which provides a provider and a set of commands. I really like what they have done. I’ve include the text of their about file so that you can see for yourself. Here is a test – take a look at the file below but don’t read any of the text – just look at the command examples. See if you can figure out what it is doing. Now go back and read it again with the text and ask yourself whether the text really added a lot more to your understanding. Let me be very quick to state that I’m NOT making the case against documentation. Heavens no! Great documentation is super important. What I’m trying to point out is the self-documenting nature of PowerShell scripts. This is why we are so hard core about the naming guidelines. At some point, the mess is going to hit the fan and you are going to be under lots of pressure to figure out what a script is doing and modify it to meet some new requirement. We’ve all been there before, trying to figure out what the heck a script is doing. Now imagine that same scenario but the script is written in PowerShell. We are working our butts off to make sure that that is a good experience for you. BTW (I’m going to keep reminding you of this) you all play a super important role in making this come true. When you see anyone produce Cmdlets or Scripts which don’t conform to the naming guidelines – you should scream! Chaos is the natural state of things and everyone is going to argue that their world is “special” and they need their own terms. If you accept that, then we’ll end up with yet another incoherent mess. What those people should do is to use the standard terms and then define aliases from their “special” terms to the standard terms. Anyway – I love what they have done with PowerShell. It’s another example of a team delivering a great experience using PowerShell. This train keeps gathering steam! SNOOPY DANCE! Jeffrey Snover [MSFT] Windows Management Partner Architect Visit the Windows PowerShell Team blog at: http://blogs.msdn.com/PowerShell Visit the Windows PowerShell ScriptCenter at: http://www.microsoft.com/technet/scriptcenter/hubs/msh.mspx TOPIC CCPPSH

SHORT DESCRIPTION Microsoft Compute Cluster Pack Snap-in for Windows PowerShell

LONG DESCRIPTION

The Compute Cluster Pack (CCP) snap-in lets you view and manage Compute Cluster Pack features in Windows PowerShell. The snap-in adds Compute Cluster Pack cmdlets and a CCP Windows PowerShell provider that lets you view and navigate cluster data in Windows PowerShell.

INSTALL THE SNAPIN

To add the Compute Cluster Pack (CCP) snap-in to Windows PowerShell, at the Windows PowerShell prompt, type:

Add-PSSnapin CCPSnapIn

To display CCP data correctly, load the CCP type file and formatting files, at the Windows PowerShell prompt, type:

Update-TypeData -PrependPath “ccp.ps1xml” Update-FormatData -PrependPath “ccp.format.ps1xml”

These commands affect only the current Windows PowerShell console. To add the files to all Windows PowerShell consoles, add these commands to your Windows PowerShell profile.

1. To determine whether you have a Windows PowerShell profile, at the Windows PowerShell prompt, type:

test-path $profile

“True” means that you have a profile. “False” means that you do not. If “True”, skip to step 3.

2. To create a Windows PowerShell profile, at the Windows PowerShell prompt, type:

new-item -type file -path $profile -force

The $profile variable is a built-in variable that stores the path to your Windows PowerShell profile.

3. To open the profile in Notepad, type:

notepad $profile

4. Copy and paste the commands into your profile. The new profile is effective when you restart Windows PowerShell.

NEW FEATURES

The CCP snap-in adds two features to Windows PowerShell:

— CCPProvider, a Windows PowerShell provider that lets you browse and navigate cluster data. (For information about Windows PowerShell providers, type “get-help about_provider”.)

— A set of CCP cmdlets for Windows PowerShell, including Get-Node, Get-Job, Get-Task, New-Job, and New-Task.

NAVIGATING CCP DATA IN WINDOWS POWERSHELL

You can use the features added by the CCP snap-in to manage CCP cluster in Windows PowerShell.

The following command creates a new Windows PowerShell drive called CCP: and mounts a cluster onto the Windows PowerShell CCP: drive. The cluster is identified by its head node, which is stored in the value of the CCP_SCHEDULER environment variable.

New-PSDrive -name CCP -provider CCPProvider -root $Env:CCP_SCHEDULER

After the drive is created, you can switch to this drive and access the data as though it were located in the drive.

For example, the following command changes to the new CCP: drive. It uses the “cd” alias for the Set-Location cmdlet, and the “dir” alias for the Get-Childitem cmdlet.

PS c:\> cd ccp: PS ccp:\> dir nodes

Name Status Proc Idle —- —— —- —- HPC-HN-01 Pend… 2 2 HPC-CN-01 Ready 2 2 … HPC-CN-08 Ready 1 1

PS ccp:\> dir jobs

ID SubmitBy Name Status Priority — ——– —- —— ——– 96 CCP\USER1 Job 1 Failed Normal

EXAMPLES: USING CCP CMDLETS IN WINDOWS POWERSHELL

The CCP cmdlets lets you manage clusters in Windows PowerShell. All of the CCP cmdlets have a Scheduler parameter that specifies the head node that the command affects.

If the Scheduler parameter is not included in the command, the CCP cmdlets use the value of the CCP_SCHEDULER environment variable ($ENV:CCP_SCHEDULER)as the default head node. You can use the Set-Content cmdlet to change the value of CCP_SCHEDULER.

For help with any cmdlet, at the Windows PowerShell prompt, type:

get-help

For example:

get-help get-node

Get-Node: Gets all of the nodes under the head node.

This command uses the Get-Node cmdlet to list all of the nodes under the head node.

PS ccp:\jobs> get-node

Name Status Proc Idle —- —— —- —- HPC-DEMO-CN-11 Pend… 2 2 HPC-DEMO-CN-12 Pend… 2 2 HPC-DEMO-CN-13 Pend… 2 2 HPC-DEMO-CN-14 Pend… 2 2 HPC-DEMO-CN-15 Pend… 2 2 HPC-DEMO-HN-01 Pend… 2 2

Enable-Node: Approves any pending nodes under the head node. This changes the status of the node from Pending to Paused. Use the Resume-Node cmdlet to make them ready.

The first command in this example uses the Get-Node cmdlet to get all of the nodes under the head node. Then, it uses the pipeline operator (|) to send the results of the Get-Node command to the Enable-Node cmdlet, which approves the pending nodes. As a result, the nodes that were in a Pending status are now in a Paused status.

The second command uses the Get-Node cmdlet to display the updated status of the nodes.

PS ccp:\jobs> get-node | enable-node PS ccp:\jobs> get-node

Name Status Proc Idle —- —— —- —- HPC-DEMO-CN-11 Paused 2 2 HPC-DEMO-CN-12 Paused 2 2 HPC-DEMO-CN-13 Paused 2 2 HPC-DEMO-CN-14 Paused 2 2 HPC-DEMO-CN-15 Paused 2 2 HPC-DEMO-HN-01 Paused 2 2

Resume-Node: Resumes any paused nodes under the head node. This changes their status from Paused to Ready. Because newly-enabled nodes are paused, you need to use this cmdlet to change their status to Ready.

The first command in this example uses the Get-Node cmdlet to get all of the nodes under the head node. Then, it uses the pipeline operator (|) to send the results of the Get-Node command to the Resume-Node cmdlet, which enables any paused nodes. As a result, the nodes that were in a Paused status are now in a Ready status.

The second command uses Get-Node to display the updated status.

PS ccp:\jobs> get-node | resume-node PS ccp:\jobs> get-node

Name Status Proc Idle —- —— —- —- HPC-DEMO-CN-11 Ready 2 2 HPC-DEMO-CN-12 Ready 2 2 HPC-DEMO-CN-13 Ready 2 2 HPC-DEMO-CN-14 Ready 2 2 HPC-DEMO-CN-15 Ready 2 2 HPC-DEMO-HN-01 Ready 2 2

You can save a collection of nodes in a variable and use array notation to view and manage the node objects. (For more information, type “get-help about_shell_variable” and “get-help about_array”.)

This command uses the assignment operator (=) to save the results of the Get-Node command in the $a variable.

PS ccp:\> $a = get-node

This command displays the first node (element 0) in the $a variable.

PS ccp:\> $a[0]

Name Status Proc Idle —- —— —- —- HPC-DEMO-CN-11 Ready 2 2

You can use the Windows PowerShell filtering cmdlets, such as Select-Object, Where-Object, and Sort-Object, to manage CCP data.

The first command uses the Select-Object to assign only the first two nodes in the collection to the $a variable. Because the pipeline operator (|) has precedence over the assignment operator (=), the “get-node | select-object first 2” part of the command is completed first and the results are assigned to the $a variable.

The second command displays the two nodes that are stored in $a.

PS ccp:\> $a = get-node | select-object -first 2 PS ccp:\> $a

Name Status Proc Idle —- —— —- —- HPC-DEMO-CN-11 Ready 2 2 HPC-DEMO-CN-12 Ready 2 2

Suspend-Node: Suspends ready nodes. This changes their status from Ready to Paused.

The following command uses the Select-Object cmdlet and its First parameter to select only the first three nodes returned by Get-Node. The second pipeline operator in the command sends the objects representing the three nodes to Suspend-Node, which pauses the nodes.

The second Get-Node command confirms that Suspend-Node affected only the first three nodes.

PS ccp:\> get-node | select-object -first 3 | Suspend-Node PS ccp:\> get-node

Name Status Proc Idle —- —— —- —- HPC-DEMO-CN-11 Paused 2 2 HPC-DEMO-CN-12 Paused 2 2

Get-Job: Gets all of the jobs on the head node or any nodes under the head node.

This command gets all of the jobs on the default head node, that is, the value of the CCP_SCHEDULER parameter.

PS ccp:\> get-job

ID SubmitBy Name Status Priority — ——– —- —— ——– 96 CCP\USER1 Job 1 Failed Normal

Restart-Job: Restarts jobs and returns them to the queue.

This command gets all of the jobs under the head node. It uses the pipeline operator (|) to send the jobs to the Restart-Job cmdlet, which restarts any jobs that not started. As a result, a job that had failed is now requeued.

Unlike the Start-Job cmdlet, the Restart-Job cmdlets starts only jobs that were started, but failed, were cancelled, or were suspended.

PS ccp:\> get-job | restart-job PS ccp:\> get-job

ID SubmitBy Name Status Priority — ——– —- —— ——– 96 CCP\USER1 Job 1 Queued Normal

Stop-Job: Stops and cancels jobs.

This command gets all of the jobs under the head node. It uses the pipeline operator (|) to send the jobs to the Stop-Job cmdlet, which stops the jobs that it receives. As a result, a job that had been queued is now cancelled.

The second command uses Get-Job to display the cancelled jobs.

PS ccp:\> get-job | stop-job PS ccp:\> get-job

ID SubmitBy Name Status Priority — ——– —- —— ——– 96 CCP\USER1 Job 1 Cancelled Normal

Get-Task: Gets the tasks in one or more jobs.

This command gets the tasks in a job and displays them in order by task ID.

This command uses the Get-Job cmdlet to get the jobs under the head node. It uses array notation to select the second job ([1] in a zero-based array). The pipeline operator (|) sends the job to the Get-Task cmdlet, which gets all of the tasks in the job and sends them to the Sort-Object cmdlet, which sorts them in order by task ID.

By default, tasks are displayed in the order that they are discovered.

PS ccp:\> (get-job)[1] | get-task | sort-object -property id

ID Status Name CommandLine — —— —- ———– 1 Finished CCPDemo 10K Tasks Group 00 echo 1 2 Finished CCPDemo 10K Tasks Group 00 echo 2 … 9998 Queued CCPDemo 10K Tasks Group 09 echo 998 9999 Queued CCPDemo 10K Tasks Group 09 echo 999 1… Queued CCPDemo 10K Tasks Group 09 echo 1000

This command gets the tasks in a selected job and uses the Where-Object cmdlet to select only the tasks with IDs greater than 500 and less than 530.

The Where-Object command uses the “$_.” notation, which refers to the current object in the pipeline. The pipeline operator sends objects down the pipeline one at a time. The “$_.” notation indicates that the operation should be performed on each object as it arrives from the pipeline. When all objects have been processed, the results are sent down the pipeline to the next cmdlet.

PS ccp:\> (get-job)[1] | get-task | where-object { ($_.id -gt 500) ¨Cand ($_.id – lt 530)}

ID Status Name CommandLine — —— —- ———– 521 Queued CCPDemo 10K Tasks Group 00 echo 521 522 Queued CCPDemo 10K Tasks Group 00 echo 522 … 529 Queued CCPDemo 10K Tasks Group 00 echo 529 501 Finished CCPDemo 10K Tasks Group 00 echo 501 502 Finished CCPDemo 10K Tasks Group 00 echo 502 … 520 Finished CCPDemo 10K Tasks Group 00 echo 520

Stop-Task: Stops one or more tasks.

The first command in this series stops any queued tasks in a selected job that have IDs between 500 and 530 (exclusive). It uses the Get-Job cmdlet to get the jobs in the second job in the system (job [1] in a zero-based array) and the Get-Task cmdlet to get the tasks in that job. It uses the Where-Object cmdlet to select tasks with a status of “queued” and a task ID between 500 and 530. It sends the selected tasks to the Stop-Task cmdlet, which stops them.

The second command displays the tasks with task IDs between 500 and 530. The results show that the affected tasks are cancelled.

PS ccp:\> (get-job )[1] | get-task | where-object { ($_.id -gt 500) -and ($_.id – lt 530) -and ($_.status -eq “queued”)} | stop-task

PS ccp:\> (get-job)[1] | get-task | where-object { ($_.id -gt 500) -and ($_.id – lt 530)}

ID Status Name CommandLine — —— —- ———– 501 Finished CCPDemo 10K Tasks Group 00 echo 501 502 Finished CCPDemo 10K Tasks Group 00 echo 502 … 520 Finished CCPDemo 10K Tasks Group 00 echo 520 521 Cancelled CCPDemo 10K Tasks Group 00 echo 521 … 528 Cancelled CCPDemo 10K Tasks Group 00 echo 528 529 Cancelled CCPDemo 10K Tasks Group 00 echo 529

Restart-Task: Restarts (or starts) tasks.

The first command in this series starts the tasks in a selected job that have IDs between 500 and 530 (exclusive) and a status of “Cancelled”. It uses the Get-Job cmdlet to get the jobs in the second job in the system (job [1] in a zero-based array) and the Get-Task cmdlet to get the tasks in that job. It uses the Where-Object cmdlet to select tasks with a status of “cancelled” and a task ID between 500 and 530. It sends the selected tasks to the Restart-Task cmdlet, which starts them.

The second command displays the tasks with task IDs between 500 and 530. The results show that the affected tasks are requeued.

PS ccp:\> (get-job)[1] | get-task | where-object { ($_.id -gt 500) -and ($_.id – lt 530) -and ($_.status -eq “cancelled”)} | Restart-Task PS ccp:\> (get-job)[1] | get-task | where-object { ($_.id -gt 500) -and ($_.id – lt 530)}

ID Status Name CommandLine — —— —- ———– 501 Finished CCPDemo 10K Tasks Group 00 echo 501 … 519 Finished CCPDemo 10K Tasks Group 00 echo 519 520 Finished CCPDemo 10K Tasks Group 00 echo 520 521 Queued CCPDemo 10K Tasks Group 00 echo 521 522 Queued CCPDemo 10K Tasks Group 00 echo 522 … 529 Queued CCPDemo 10K Tasks Group 00 echo 529

New-Job: Creates a new job.

The first command in this series creates a new job and stores it in the $j variable.

The second command names the job “CCPDemo”. It uses the . format to access the Name property of the job object in the $j variable. It uses the assignment operator (=) to assign the “CCPDemo” value to the Name property of the job object in the $j variable.

The third command uses the Format-List cmdlet to display the properties of newly-created job object as a list.

PS ccp:\> $j = new-job PS ccp:\> $j.Name = “CCPDemo” PS ccp:\> $j | format-list

AllocatedNodes : AskedNodes : CreateTime : 12/31/9999 11:59:59 PM EndTime : 12/31/9999 11:59:59 PM ExtendedJobTerms : {} Id : 0 IsBackfill : False IsExclusive : True MaximumNumberOfProcessors : 1 MinimumNumberOfProcessors : 1 ErrorMessage : Name : CCPDemo SubmittedBy : Priority : Normal Project : RunUntilCanceled : False Runtime : Infinite StartTime : 12/31/9999 11:59:59 PM SoftwareLicense : Status : NotSubmitted SubmitTime : 12/31/9999 11:59:59 PM Tasks : {} User : TaskCount : 0 Timestamp :

This command limits the CCPDemo job to four processors. It uses the . format to access the MaximumNumberOfProcessors property of the job object in the $j variable, and an assignment statement to assign a value of “4” to the property.

PS ccp:\> $j.MaximumNumberOfProcessors = 4

New Task: Creates a new task in a job.

The first command in this series uses the New-Task cmdlet to creates a new task. Then, it stores the new task in the $t1 variable.

The second command uses the . format to access the Name property of the new task object in the $t variable. Then, it assigns the “t1” name to the task.

The third command uses the object>. format to access the CommandLine property of the task object in the $t1 variable. Then, it assigns the “t1cmd” value to the CommandLine property.

PS ccp:\> $t1 = new-task PS ccp:\> $t1.Name = “t1” PS ccp:\> $t1.CommandLine = “t1cmd”

This series of commands is much like the previous one. It creates a new task and stores it in the $t2 variable; it assigns the name, “t2” to the task and the value “t2cmd” to the CommandLine property.

PS ccp:\> $t2 = new-task PS ccp:\> $t2.Name = “t2” PS ccp:\> $t2.CommandLine = “t2cmd”

Add-Task: Adds a task to a job. The task and the job must exist before they are associated.

The first command uses the Add-Task cmdlet to add the tasks stored in the $t1 and $t2 variables to the job stored in the $j variable. It uses the Job parameter to specify the job.

The second command uses the Format-List cmdlet to display the properties of job object in a list. It shows that the maximum number of processors was changed to 4, and that the new tasks are added.

PS ccp:\> $t,$t2 | add-task -job $j PS ccp:\> $j | format-list

AllocatedNodes : AskedNodes : CreateTime : 12/31/9999 11:59:59 PM EndTime : 12/31/9999 11:59:59 PM ExtendedJobTerms : {} Id : 0 IsBackfill : False IsExclusive : True MaximumNumberOfProcessors : 4 MinimumNumberOfProcessors : 1 ErrorMessage : Name : CCPDemo SubmittedBy : Priority : Normal Project : RunUntilCanceled : False Runtime : Infinite StartTime : 12/31/9999 11:59:59 PM SoftwareLicense : Status : NotSubmitted SubmitTime : 12/31/9999 11:59:59 PM Tasks : {t1, t2name} User : TaskCount : 2 Timestamp :

Start-Job: Submits the job for execution.

The first command starts the job stored in the $j variable by piping the variable to the cmdlet. After it is started, the status is “queued”.

The second command gets the jobs under the head node. In this case, the job stored in the $j variable is the only one.

PS ccp:\> $j | start-job PS ccp:\> get-job

ID SubmitBy Name Status Priority — ——– —- —— ——– … 6 DOMAIN1\User1 CCPDemo Queued Normal

This command uses the Get-Task cmdlet to display the tasks in the newly started job. It shows that the tasks in the job are queued when the job is started.

PS ccp:\> get-job)[5] | get-task ID Status Name CommandLine — —— —- ———– 1 Queued t1 t2cmd 2 Queued t2 t2cmd

Using Variables and Arrays with CCP Data

This command saves the result of a Get-Node command in the $a variable. As a result, $a contains objects representing all of the nodes under the head node.

PS ccp:\> $a = get-node PS ccp:\> $a

Name Status Proc Idle —- —— —- —- HPC-DEMO-CN-11 Ready 2 2 HPC-DEMO-CN-12 Ready 2 2 HPC-DEMO-CN-13 Ready 2 2 HPC-DEMO-CN-14 Ready 2 2 HPC-DEMO-CN-15 Ready 2 2 HPC-DEMO-HN-01 Ready 2 2

The Get-Node command actually gets an array of objects, each of which represents one node. You can use standard array notation to access the items in the array.

This command uses array notation to access the first node object in the $a variable.

PS ccp:\> $a[0]

Name Status Proc Idle —- —— —- —- HPC-DEMO-CN-11 Ready 2 2

You can use the cmdlets that come with Windows PowerShell to manipulate the CCP data.

This command uses the Select-Object cmdlet and its First parameter to select only the first two nodes returned by Get-Node. The results are stored in the $a variable.

The parentheses in this command have no effect. They are added to show that the Get-Node and Select-Object commands are completed before the result is stored in the $a variable. (The pipeline operator takes precedence over the assignment operator.)

PS ccp:\> $a = (get-node | select-object -first 2) PS ccp:\> $a

Name Status Proc Idle —- —— —- —- HPC-DEMO-CN-11 Ready 2 2 HPC-DEMO-CN-12 Ready 2 2

This command uses the Suspend-Node cmdlet, which pauses ready nodes.

The command uses the Select-Object cmdlet and its First parameter to select only the first three nodes returned by Get-Node. The second pipeline operator in the command sends the objects representing the three nodes to Suspend-Node, which pauses the nodes.

The second Get-Node command confirms that Suspend-Node affected only the first three nodes.

PS ccp:\> get-node | select-object -first 3 | Suspend-Node # node pause PS ccp:\> get-node

Name Status Proc Idle —- —— —- —- HPC-DEMO-CN-11 Paused 2 2 HPC-DEMO-CN-12 Paused 2 2 HPC-DEMO-CN-13 Paused 2 2 HPC-DEMO-CN-14 Ready 2 2 HPC-DEMO-CN-15 Ready 2 2 HPC-DEMO-HN-01 Ready 2 2

This command uses the Where-Object cmdlet to select only the paused nodes from the array that Get-Node gets. The paused nodes are sent to the Resume-Node cmdlet, which changes their status from “paused” to “ready.”

The pipeline operator sends objects down the pipeline one at a time. The Where- Object command uses the “$_.” symbol, which refers to the current object sent down the pipeline. The operation that it specifies is performed on each object as it arrives. When all objects have been processed, the results are sent to the next cmdlet.

PS ccp:\> get-node | where-object {$_.status -eq “paused”} | Resume-Node PS ccp:\> get-node

Name Status Proc Idle —- —— —- —- HPC-DEMO-CN-11 Ready 2 2 HPC-DEMO-CN-12 Ready 2 2 HPC-DEMO-CN-13 Ready 2 2 HPC-DEMO-CN-14 Ready 2 2 HPC-DEMO-CN-15 Ready 2 2 HPC-DEMO-HN-01 Ready 2 2

SEE ALSO For information about individual Cmdlets, use the help alias (for the get-help Cmdlet) followed by the Cmdlet name. For example, to view information about the get-node Cmdlet, Enter the following command:

help get-node

To get a full list of Cmdlets for CCP, enter the following command:

get-command -pssnapin CCPSnapIn

To change the value of the environment set CCP_SCHEDULER, use this command:

set-content env:CCP_SCHEDULER your_scheduler

0 comments

Comments are closed. Login to edit/delete your existing comments