How PowerShell Formatting and Outputting REALLY works

<WIZARD WARNING>

Dreeschkind posted a question in the Microsoft.Public.Windows.Server.Scripting newsgroup about how PowerShell formatting worked with Select. He saw some behavior that he thought was a bug. Here is what he saw:

PoSh C:\>gps | sort starttime | select name,starttime
Sort-Object : Exception getting “StartTime”: “Zugriff verweigert”
At line:1 char:11
+ gps | sort <<<< starttime | select name,starttime
Select-Object : Exception getting “StartTime”: “Zugriff verweigert”
At line:1 char:30
+ gps | sort starttime | select <<<< name,starttime

Name
—-
Idle
System
SMSS
CSRSS
WINLOGON
SERVICES
<…>

Notice the lack of a column for StartTime. It appears that we’ve lost information. We haven’t and as Wei Wu’s response shows, you can easily get what you want by asking for it explicitly (by piping it to FT and specifying the properties you want):

Name   starttime
—-   ———
Idle
System 12/31/1600 4:00:00 PM
smss   4/28/2006 9:17:44 PM
csrss 4/28/2006 9:17:47 PM
…

This is a good setup to drill into how formatting works. This is a black belt dive into formatting and outputting and not for everyone so don’t continue unless you want to know how things REALLY work.

The vast majority of PowerShell is implemented by System.Management.Automation.dll. PowerShell.Exe is an unmanaged executable which determines the environment that is required and dynamically loads the correct CLR and managed code PowerShell host (Microsoft.PowerShell.ConsoleHost.dll). Control is passed to this DLL which provides the user experience. This is a relatively small DLL providing a console experience of the core engine. It calls the engine to get the prompt but then it displays it and collects user input. It then takes that input, concatenates ” | Out-Default” and submits it to the engine for execution. … YES, under the covers every command entered in through the console host is piped to Out-Default.

Out-Default is responsible for figuring out how to format and output the object stream. If the object stream is a stream of strings, Out-Default pipes these directly to Out-Host which calls the appropriate APIs provided by the host (hosts have to provide an interface to the engine, in this case Microsoft.PowerShell.ConsoleHost.dll provides implementations which send the output to the Console). If the object stream does not contain strings, Out-Default inspects the object to determine what to do. First it looks at the object type and determines whether there is a registered VIEW for this object type.

PowerShell defines XML schema and a mechanism (the Update-FormatData cmdlet) where anyone can register views for an object type. You can specify WIDE, LIST, TABLE, or CUSTOM views for any object type. The views specify which properties to display and how they should be displayed. If a view is registered, it defines which FORMATTER to use. So if the registered view is a TABLE view, OUT-DEFAULT streams the objects to “Format-Table | Out-Host”. Format-Table transforms the objects into a stream of Formatting records (driven by the data in the View definition) and Out-Host transforms the formatting records into calls on the Host interface. You can see this for yourself by observing that you’ll get the same display by typing any of the following:

NOTE: Out-String transforms formatting records to a string and then Out-Default passes them unmolested to Out-Host.

You can also see the formatting records by doing the following:

Get-Process |Format-Table |Get-Member

Now, if there is not a registered view for a datatype, then Out-Default looks at the FIRST OBJECT IN THE STREAM to determine how many properties the object has 5 or more properties, it send the ENTIRE STREAM to Format-List, otherwise it sends the ENTIRE STREAM to Format-Table. When it sends the stream to Format-Table, that command needs to generate columns. It does this by looking at the properties of the FIRST OBJECT – those become the columns. If the first Object has 2 properties, you’ll get a 2 column table even if all the other objects in the stream have 10 properties.

This is what is happening to Dreeschkind. Go back and look at the original example and you’ll notice that there was an error accessing StartTime during Sort and Select-Object. What that is there was an ACCESS WAS DENIED error while accessing StartTime for the Idle process. When this occurred, sort put this object at the front of the stream and Select did not create a property for StartTime for this result object. Thus an object that did not have a StartTime property was the first object in the stream. Since there were fewer than 5 properties, the objects where sent to Format-Table which created columns based upon the first object and since it did not have StartTime – there was no column created for it. Had the objects not been sorted by StartTime or if it had been a DESCENDING sort, the first object would have had a StartTime property and everything would have been fine. Alternatively, if you pipe the objects to Format-Table and specify the fields you want, it would have done what you wanted. This creates columns and if the object doesn’t have that field – the row has a blank in that column.

It is interesting to note that if you and selected enough properties, Out-Default would have sent the object stream to Format-list and then each object would have all of its properties displayed.

In the end, the fact that select did not create a property when it encountered a problem is probably a bug and should be fixed. That said, this allowed us to explore the fascinating inner workings of formatting and outputting so something good came of it.

Enjoy
Jeffrey Snover
PowerShell Architect

PSMDTAG:INTERNAL: How formating decides which metadata to use, default formatters and outputters
PSMDTAG:CMDLET:FORMAT: How formating decides which metadata to use, default formatters and outputters

How PowerShell Formatting and Outputting REALLY works

Author

0 comments

Read next

Chewy WMI example

em dash, en dash, dash, dash, dash…