Gathering real-time Perfmon Counters in a cluster

Developer Support

Premier Development Consultant Kurt Schenk demonstrates how to gather real-time performance metrics in a cluster.


Performance (aka. Perfmon) Counters are critical to understanding the health of and diagnosing issues on Windows. In recent performance and scalability testing of a solution built on top of Azure Service Fabric, we collected Perfmon Counters across 15 VMs and sent them to Log Analytics. This was very helpful; however, there were times when we wanted them to be real time. In our case, we wanted to verify core OS Perfmon Counters like Disk, CPU, Memory, Exceptions, and since we were driving load over WebSockets, how many connections were established. In order to enable this, I created a PowerShell script that can easily be configured for the particular Perfmon Counters you want to collect, and machines you want monitor.  Ran this script in PowerShell ISE:

# uncomment the line below and run by itself to help you find the correct name for the Perfmon Counters you are interested in
#(get-counter -listset ".NET CLR Memory").paths
 
$listOfMetrics = @(
      "\LogicalDisk(c:)\% Free Space"
      "\LogicalDisk(d:)\% Free Space",
      "\LogicalDisk(d:)\Free Megabytes",
      "\processor(_Total)\% processor time",
      "\Memory\Available MBytes",
      "\TCPv4\Connections Established",
      "\.NET CLR Exceptions(_global_)\# of Exceps Thrown / sec",
      "\.NET CLR Memory(_global_)\% Time in GC"
)
 
$vms2 = "zero000000", "zero000001", "zero000002", "zero000003", "zero000004", "zero00000J", "zero00000K", "zero00000L", "zero00000M", `
"zero00000N", "zero00000O", "zero00000X", "zero00000Y", "zero00000Z", "zero000010"
 
Get-counter -Counter $listOfMetrics -SampleInterval 1 -MaxSamples 1 -ComputerName $vms2 | select -expand countersamples | `
Out-GridView #Format-Table -AutoSize # | Sort-Object Path | Tree

Which displays following.

Machine generated alternative text: Out-GridView #Format-Table -AutoSize # I Sort-Object Path I Tree Filter Add criteria Path \\zer000000k\\logicaldisk(d:)\% free space \\zer0000001\\logicaldisk(d:)\% free space \\zer000000m\\logicaldisk(d:)\% free space \\zer000000n\\logicaldisk(d:)\% free space \\zer0000000\\logicaldisk(d:)\% free space \\zer000000x\\logicaldisk(d:)\% free space \\zer000000y\\logicaldisk(d:)\% free space \\zer000000z\\logicaldisk(d:)\% free space \\zer0000010\\logicaldisk(d:)\% free space \\zer0000000\\logicaldisk(d:)\free megabytes \\zer0000001\\logicaldisk(d:)\free megabytes \\zer0000002\\logicaldisk(d:)\free megabytes \\zer0000003\\logicaldisk(d:)\free megabytes \\zer0000004\\logicaldisk(d:)\free megabytes \\zer000000j\\logicaldisk(d:)\free megabytes \\zer000000k\\logicaldisk(d:)\free megabytes \\zer0000001\\logicaldisk(d:)\free megabytes \\zer000000m\\logicaldisk(d:)\free megabytes \\zer000000n\\logicaldisk(d:)\free megabytes \\zer0000000\\logicaldisk(d:)\free megabytes \\zer000000x\\logicaldisk(d:)\free megabytes \\zer000000y\\logicaldisk(d:)\free megabytes \\zer000000z\\logicaldisk(d:)\free megabytes \\zer0000010\\logicaldisk(d:)\free megabytes \\zer0000000\\processorCtotal)\% processor time \\zer0000001\\processorCtotal)\% processor time \\zer0000002\\processorCtotal)\% processor time \\zer0000003\\processorCtotal)\% processor time \\zer0000004\\processorCtotal)\% processor time \\zer000000j\\processor(_total)\% processor time \\zer000000k\\processor(_total)\% processor time x InstanceName d: d: d: d: d: d: d: d: d: d: d: d: d: d: d: d: d: d: d: d: d: d: d: d: total total total total total total total CookedValue 22.49 43.73 34.66 42.91 44.19 34.86 44.89 45.97 14.70 16,938.00 30,005.00 28,565.00 28,081.00 22,742.00 20,806.00 14,736.00 28,655.00 22,717.00 28,121.00 28,960.00 22,846.00 29,417.00 30,123.00 9,631.00 1.61 0.33 0.71 1.01 0.79 0.57 1.53 10:02 PM 11/21/2018

And with the PowerShell grid, it is easy to sort and filter (for example by machine name below).

Machine generated alternative text: Administrator: Windows PowerShell ISE File Edit View Tools Debug Add-ons Help Get-Counte Out-GridView #Format-Table -AutoSize # I Sort-Object Path I Tree x 1 2 4 5 6 7 8 9 10 11 12 13 14 15 16 Svm 17 " ze 18 19 • \usel PS C. • \usel PS C. • \usel PS C. Completed zer000000m Add criteria Path \\zer000000m\\logicaldisk(c:)\% free space \\zer000000m\\logicaldisk(d:)\% free spa... \\zer000000m\\logicaldisk(d:)\free mega.. \\zer000000m\\processor(_total)\96 proc.. \\zer000000m\\memory\available m bytes \\zer000000m\\tcpv4\connections establ... \\zer000000m\\.net clr exceptions(_glob.. \\zer000000m\\.net clr InstanceName c: d: d: total global global CookedValue 77.45 60.96 39,951.00 2.67 10,670.00 135.00 0.00 0.07 Ln4 Col 1 100%

Keep in mind that this results are a snapshot. To update run the script again.

0 comments

Discussion is closed.

Feedback usabilla icon