automatically<\/em> creates the code that calls the underlying application. This framework can also handle the output mapping to an object more suitable for the PowerShell environment.<\/p>\nPossibilities in wrapping<\/h2>\n
The aspect that makes this possible is that some commands have consistently structured help that describes how the application can be used. If this is the case, then we can iteratively call the help, parse it, and automatically construct much of the infrastructure needed to allow these native applications to be incorporated into the PowerShell environment.<\/p>\n
First Experiment – Microsoft.PowerShell.Kubectl Module<\/h3>\n
I created a wrapper for to take the output of kubectl api-resources<\/code> and create functions for each returned resource. This way, instead of running kubectl get pod<\/code>; I could run Get-KubectlPod<\/code> (a much more PowerShell-like<\/em> experience). I also wanted to have the function return objects that I could then use with other PowerShell tools (Where-Object, ForEach-Object, etc). To do this, I needed a way to map the output (JSON) of the kubectl<\/code> tool to PowerShell objects. I decided that it was reasonable to use a more declarative approach that maps the property in the JSON to a PowerShell class member.<\/p>\nThere were some problems that I wanted to solve with this first experiment<\/p>\n
\n- wrap
kubectl api-resources<\/code> in a function\n\n- automatically create object output from
kubectl api-resources<\/code><\/li>\n<\/ul>\n<\/li>\n- Auto-generate functions for each resource that could be retrieved (only resource get for now)\n
\n- only support
name<\/code> as a parameter<\/li>\n<\/ul>\n<\/li>\n- Auto-generate the conversion of output to objects to look similar to the usual
kubectl<\/code> output<\/li>\n<\/ul>\nWhen it came to wrapping kubectl api-resources<\/code> I took the static approach rather than auto generation. First, because it was my first attempt so I was still finding my feet. Second, because this is one of the kubectl<\/code> commands that does not emit JSON. So, I took the path of parsing the output of kubectl api-resources -o wide<\/code>. My concern is that I wasn’t sure whether the table changes width based on the screen width. I calculated column positions based on the fields I knew to be present and then parsed the line using the offsets. You can see the code in the function get-kuberesource<\/code> and the constructor for the PowerShell class KubeResource<\/code>. My plan was that these resources would drive the auto-generation of the Kubernetes resource functions.<\/p>\nNow that I have retrieved the resources, I can auto-generate specific resource function for calling the kubectl get <resource><\/code>. At the time, I wanted some flexibility in the creation of these proxy functions, so I provided a way to include a specific implementation, if desired (see the $proxyFunctions<\/code> hashtable). I’m not sure that’s needed now, but we’ll get to that later. The problem is that while the resource data can be returned as JSON, that JSON has absolutely no relation to the way the data is represented in the kubectl get pod<\/code> table. Of course, in PowerShell we can create formatting to present any member of an object (JSON or otherwise), but I like to be sure that the properties seen in a table are properties that I can use with Where-Object<\/code>, etc. Since, I want to return the data as objects, I created classes for a couple resources by hand but thought there might be a better way.<\/p>\nI determined that when you get data from kubernetes, the table (both normal and wide) output is created on the server<\/em>. This means the mapping of the properties of the JSON object to the table columns is defined in the server code. It’s possible to provide data as custom columns, but you need to provide the value for the column using a JSON path expression. So, it’s not possible to automatically generate those tables. However, I thought it might be possible to provide a configuration file that could be read to automatically generate a PowerShell class. The configuration file would need to define the mapping between the property in the table with the properties of the object. The file would include the name of the column and the expression to get the value for the object. This allows a user to retrieve the JSON object and construct their custom object without touching the programming logic of the module but a configuration file. I created the ResourceConfiguration.json<\/code> file to encapsulate all the resources that I had access to and provide a way to customize the object members where desired.<\/p>\nhere’s an example:<\/p>\n
{\r\n \"TypeName\": \"namespaces\",\r\n \"Fields\": [\r\n {\r\n \"PropertyName\": \"NAME\",\r\n \"PropertyReference\": \"$o.metadata.NAME\"\r\n },\r\n {\r\n \"PropertyName\": \"STATUS\",\r\n \"PropertyReference\": \"$o.status.phase\"\r\n },\r\n {\r\n \"PropertyName\": \"AGE\",\r\n \"PropertyReference\": \"$o.metadata.creationTimeStamp\"\r\n }\r\n ]\r\n },\r\n<\/code><\/pre>\nThis JSON is converted into a PowerShell class whose constructor takes the JSON object and assigns the values to the members, according to the PropertyReference<\/code>. The module automatically attaches the original JSON to a hidden member originalObject<\/code> so if you want to inspect all the data that’s available, you can. The module also automatically generates a proxy function so you can get the data:<\/p>\nfunction Get-KubeNamespace\r\n{\r\n [CmdletBinding()]\r\n param ()\r\n (Invoke-KubeCtl -Verb get -resource namespaces).Foreach({[namespaces]::new($_)})\r\n}\r\n<\/code><\/pre>\nThis function is then exported so it’s available in the module. When used, it behaves very close to the original:<\/p>\n
PS> Get-KubeNamespace\r\n\r\nName Status Age\r\n---- ------ ---\r\ndefault Active 5\/6\/2020 6:13:07 PM\r\ndefault-mem-example Active 5\/14\/2020 8:14:45 PM\r\ndocker Active 5\/6\/2020 6:14:25 PM\r\nkube-node-lease Active 5\/6\/2020 6:13:05 PM\r\nkube-public Active 5\/6\/2020 6:13:05 PM\r\nkube-system Active 5\/6\/2020 6:13:05 PM\r\nkubernetes-dashboard Active 5\/18\/2020 8:44:01 PM\r\nopenfaas Active 5\/6\/2020 6:51:22 PM\r\nopenfaas-fn Active 5\/6\/2020 6:51:22 PM\r\n\r\nPS> kubectl get namespaces --all-namespaces\r\n\r\nNAME STATUS AGE\r\ndefault Active 26d\r\ndefault-mem-example Active 18d\r\ndocker Active 26d\r\nkube-node-lease Active 26d\r\nkube-public Active 26d\r\nkube-system Active 26d\r\nkubernetes-dashboard Active 14d\r\nopenfaas Active 26d\r\nopenfaas-fn Active 26d\r\n<\/code><\/pre>\nbut importantly, I can use the output with Where-Object<\/code> and ForEach-Object<\/code> or change the format to list, etc.<\/p>\nPS> Get-KubeNamespace |? name -match \"faas\"\r\n\r\nName Status Age\r\n---- ------ ---\r\nopenfaas Active 5\/6\/2020 6:51:22 PM\r\nopenfaas-fn Active 5\/6\/2020 6:51:22 PM\r\n<\/code><\/pre>\nSecond Experiment – Module KubectlHelpParser<\/h3>\n
I wanted to see if I could read any help content from kubectl<\/code> that would enable me to auto-generate a complete proxy of the kubectl<\/code> command that included general parameters, command specific parameters, and help. It turns out that kubectl<\/code> help is regular enough that this is quite possible.<\/p>\nWhen retrieving help, kubectl provides subcommands that also have structured help. I created a recursive parser that allowed me to retrieve all of the help for all of the available kubectl commands. This means that if an additional command is provided in the future, and the help for that command follows the existing pattern for help, this parser will be able to generate a command for it.<\/p>\n
PS> kubectl --help\r\nkubectl controls the Kubernetes cluster manager.\r\n\r\n Find more information at: https:\/\/kubernetes.io\/docs\/reference\/kubectl\/overview\/\r\n\r\nBasic Commands (Beginner):\r\n create Create a resource from a file or from stdin.\r\n expose Take a replication controller, service, deployment or pod and expose it as a new Kubernetes Service\r\n run Run a particular image on the cluster\r\n set Set specific features on objects\r\n\r\nBasic Commands (Intermediate):\r\n explain Documentation of resources\r\n get Display one or many resources\r\n. . .\r\n\r\nkubectl set --help\r\n\r\nPS> kubectl set --help\r\n\r\nConfigure application resources\r\n\r\n These commands help you make changes to existing application resources.\r\n\r\nAvailable Commands:\r\n env Update environment variables on a pod template\r\n . . .\r\n subject Update User, Group or ServiceAccount in a RoleBinding\/ClusterRoleBinding\r\n\r\nUsage:\r\n kubectl set SUBCOMMAND [options]\r\n\r\nPS> kubectl set env --help\r\n\r\nUpdate environment variables on a pod template.\r\n\r\n List environment variable definitions in one or more pods, pod templates. Add, update, or remove container environment\r\nvariable definitions in one or more pod templates (within replication controllers or deployment configurations). View or\r\nmodify the environment variable definitions on all containers in the specified pods or pod templates, or just those that\r\nmatch a wildcard.\r\n\r\n If \"--env -\" is passed, environment variables can be read from STDIN using the standard env syntax.\r\n\r\n Possible resources include (case insensitive):\r\n\r\n pod (po), replicationcontroller (rc), deployment (deploy), daemonset (ds), job, replicaset (rs)\r\n\r\nExamples:\r\n # Update deployment 'registry' with a new environment variable\r\n kubectl set env deployment\/registry STORAGE_DIR=\/local\r\n . . .\r\n # Set some of the local shell environment into a deployment config on the server\r\n env | grep RAILS_ | kubectl set env -e - deployment\/registry\r\n\r\nOptions:\r\n --all=false: If true, select all resources in the namespace of the specified resource types\r\n --allow-missing-template-keys=true: If true, ignore any errors in templates when a field or map key is missing in\r\nthe template. Only applies to golang and jsonpath output formats.\r\n . . .\r\n --template='': Template string or path to template file to use when -o=go-template, -o=go-template-file. The\r\ntemplate format is golang templates [http:\/\/golang.org\/pkg\/text\/template\/#pkg-overview].\r\n\r\nUsage:\r\n kubectl set env RESOURCE\/NAME KEY_1=VAL_1 ... KEY_N=VAL_N [options]\r\n\r\nUse \"kubectl options\" for a list of global command-line options (applies to all commands).\r\n<\/code><\/pre>\nThe main function of the module will recursively collect the help for all of the commands and construct an object representation that I hope can then be used to generate the proxy functions. This is still very much a work in progress, but it is definitely showing promise. Here’s an example of what it can already do.<\/p>\n
PS> import-module .\/KubeHelpParser.psm1\r\nPS> $res = get-kubecommands\r\nPS> $res.subcommands[3].subcommands[0]\r\n<\/code><\/pre>\nCommand : set env\r\nCommandElements : {, set, env}\r\nDescription : Update environment variables on a pod template.\r\n\r\n List environment variable definitions in one or more pods, pod templates. Add, update, or remove container environment variable definitions in one or more pod templates (within replication controllers or deployment configurations). View or modify the environment variable definitions\r\n on all containers in the specified pods or pod templates, or just those that match a wildcard.\r\n\r\n If \"--env -\" is passed, environment variables can be read from STDIN using the standard env syntax.\r\n\r\n Possible resources include (case insensitive):\r\n\r\n pod (po), replicationcontroller (rc), deployment (deploy), daemonset (ds), job, replicaset (rs)\r\nUsage : kubectl set env RESOURCE\/NAME KEY_1=VAL_1 ... KEY_N=VAL_N [options]\r\nSubCommands : {}\r\nParameters : {[Parameter(Mandatory=$False)][switch]${All}, [Parameter(Mandatory=$False)][switch]${NoAllowMissingTemplateKeys}, [Parameter(Mandatory=$False)][System.String]${Containers} = \"*\", [Parameter(Mandatory=$False)][switch]${WhatIf}\u2026}\r\nMandatoryParameters : {}\r\nExamples : {kubectl set env deployment\/registry STORAGE_DIR=\/local, kubectl set env deployment\/sample-build --list, kubectl set env pods --all --list, kubectl set env deployment\/sample-build STORAGE_DIR=\/data -o yaml\u2026}\r\n<\/code><\/pre>\nPS> $res.subcommands[3].subcommands[0].usage\r\n<\/code><\/pre>\nUsage supportsFlags hasOptions\r\n----- ------------- ----------\r\nkubectl set env RESOURCE\/NAME KEY_1=VAL_1 ... KEY_N=VAL_N [options] False True\r\n<\/code><\/pre>\nPS> $res.subcommands[3].subcommands[0].examples\r\n<\/code><\/pre>\nDescription Command\r\n----------- -------\r\nUpdate deployment 'registry' with a new environment variable kubectl set env deployment\/registry STORAGE_DIR=\/local\r\n. . .\r\n\r\n<\/code><\/pre>\nPS> $res.subcommands[3].subcommands[0].parameters.Foreach({$_.tostring()})\r\n<\/code><\/pre>\n\r\n[Parameter(Mandatory=$False)][switch]${All}\r\n[Parameter(Mandatory=$False)][switch]${NoAllowMissingTemplateKeys}\r\n[Parameter(Mandatory=$False)][System.String]${Containers} = \"*\"\r\n[Parameter(Mandatory=$False)][switch]${WhatIf}\r\n. . .\r\n[Parameter(Mandatory=$False)][System.String]${Selector}\r\n[Parameter(Mandatory=$False)][System.String]${Template}\r\n\r\n<\/code><\/pre>\nThere are still a lot of open questions and details to work out here:<\/p>\n
\n- how are mandatory parameters determined?<\/li>\n
- how do we keep a map of used parameters?<\/li>\n
- does parameter order matter?<\/li>\n
- can reasonable debugging be provided?<\/li>\n
- do we have to “boil the ocean” to provide something useful?<\/li>\n<\/ul>\n
I believe it may be possible to create a more generic framework which would allow a larger number native executables to be more fully incorporated into the PowerShell ecosystem. These are just the first steps in the investigation, but it looks very promising.<\/p>\n
Call To Action<\/h2>\n
First, I’m really interested in knowing that having a framework that can auto-generate functions that wrap a native executable is useful. The obvious response might be “of course”, but how much of a solution is really needed to provide value? Second, I would really<\/em> like to know if you would like us to investigate specific<\/em> tools for this sort of treatment. If it is possible to make this a generic framework, I would love to have more examples of tools which would be beneficial to you and test our ability to handle.<\/p>\nJames Truher\nSoftware Engineer\nPowerShell Team<\/p>\n","protected":false},"excerpt":{"rendered":"
How to wrap native commands to take better advantage of the PowerShell environment<\/p>\n","protected":false},"author":2413,"featured_media":13641,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[3173],"class_list":["post-18450","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-powershell","tag-powershell-crescendo"],"acf":[],"blog_post_summary":"
How to wrap native commands to take better advantage of the PowerShell environment<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/powershell\/wp-json\/wp\/v2\/posts\/18450","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/powershell\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/powershell\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/powershell\/wp-json\/wp\/v2\/users\/2413"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/powershell\/wp-json\/wp\/v2\/comments?post=18450"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/powershell\/wp-json\/wp\/v2\/posts\/18450\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/powershell\/wp-json\/wp\/v2\/media\/13641"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/powershell\/wp-json\/wp\/v2\/media?parent=18450"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/powershell\/wp-json\/wp\/v2\/categories?post=18450"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/powershell\/wp-json\/wp\/v2\/tags?post=18450"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}