{"id":14205,"date":"2018-07-31T08:46:31","date_gmt":"2018-07-31T15:46:31","guid":{"rendered":"https:\/\/blogs.msdn.microsoft.com\/powershell\/?p=14205"},"modified":"2019-02-18T12:37:46","modified_gmt":"2019-02-18T19:37:46","slug":"increased-windows-modules-coverage-with-powershell-core-6-1","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/powershell\/increased-windows-modules-coverage-with-powershell-core-6-1\/","title":{"rendered":"Increased Windows Modules coverage with PowerShell Core 6.1"},"content":{"rendered":"

During the\u00a0<\/span>May 2018 Community Call<\/a>\u00a0<\/span>and a\u00a0<\/span>tweet<\/a>\u00a0<\/span>a few weeks later, we mentioned that PowerShell team was spending significant time in the Windows codebase. We even demoed using the\u00a0<\/span>Active Directory PowerShell Module<\/a>\u00a0<\/span>from PowerShell Core 6 during the PowerShell Community Call.<\/p>\n

We started investigating some of the top requested modules that ship with Windows to see how to get them working in\u00a0<\/span>PowerShell Core 6<\/a>. Our eventual goal is to have near 100% parity with Windows PowerShell for the in-box modules, but we expect this effort to span across multiple releases of Windows as there are many modules and PowerShell team is working with each feature team on an individual basis to port and validate their modules. The good news is that you can pick up newly compatible modules with frequent updates to Windows 10, and Windows Insiders will see them even sooner.<\/p>\n

TL;DR; (Too Long; Didn’t Read)<\/h2>\n

Starting with\u00a0<\/span>Windows 10 Insiders build 17711<\/a>\u00a0<\/span>or newer, you can use many Windows PowerShell modules with PowerShell Core 6.1-Preview.4 and if you see an issue or difference compared to Windows PowerShell 5.1, please open an issue in the\u00a0<\/span>PowerShellModuleCoverage<\/a>\u00a0<\/span>repository.<\/p>\n

The Journey to Porting Windows PowerShell modules<\/h2>\n

Prioritizing Windows PowerShell Modules<\/h3>\n

Windows ships with many PowerShell modules. We created a prioritized list of Windows PowerShell modules to investigate compatibility and portability based on customer and partner requests.\nWe ended up with a list of what we thought would have maximum customer impact. For example, the top requested module was the\u00a0<\/span>Active Directory PowerShell Module<\/a>, so we made sure to partner with the AD team and investigate their module first. After investigation, we also deferred work on certain modules that were infeasible to port to PowerShell Core given our time frame and expertise.<\/p>\n

ApiPort<\/h3>\n

First, we looked at was using\u00a0<\/span>ApiPort<\/a>\u00a0<\/span>to identify APIs used by Windows PowerShell modules that were not available in .NET Core. Depending on the API, we were able to include additional .NET Core compatible assemblies, and in others, we rewrote the code to use a different API that is functionally equivalent. In some cases, the API was simply unavailable, so we could only mark the module as incompatible by setting\u00a0<\/span>CompatiblePSEditions<\/code>\u00a0<\/span>to just\u00a0<\/span>Desktop<\/code>\u00a0<\/span>in the module manifest. The initial investigation showed that >95% of the APIs used by Windows PowerShell modules were available in .NET Standard (or compatible assemblies).<\/p>\n

.NET Standard 2.0<\/h3>\n

.NET Standard<\/a>\u00a0<\/span>is a formal specification of .NET APIs intended to be available on all .NET implementations. For example, .NET Framework 4.6.1+ used by Windows PowerShell and .NET Core 2.0+ used by PowerShell Core 6 both implement APIs specified by .NET Standard 2.0. By rebuilding existing source code for Windows PowerShell modules to target .NET Standard instead of .NET Framework, we would get compile-time errors if an API wasn’t available and shouldn’t be used.<\/p>\n

The Windows build environment is self-contained and did not support building code targeting .NET Standard. The next work item was to work with the folks on the Windows Engineering System team to enable building managed assemblies and target .NET Standard 2.0. They were excellent partners, and together, we built a simple PowerShell module against .NET Standard 2.0 that worked in both Windows PowerShell 5.1 and PowerShell Core 6.<\/p>\n

Windows Compatibility Pack<\/h3>\n

Although the proof of concept to use the Windows build environment to produce a .NET Standard assembly worked, actual Windows PowerShell modules used many APIs beyond just what is in the formal .NET Standard 2.0 specification (and associated reference assembly). To help developers move from .NET Framework to .NET Core, the .NET team published a set of assemblies called the\u00a0<\/span>Windows Compatibility Pack<\/a>\u00a0<\/span>as an extension to .NET Standard. The Windows Compatibility Pack brought back important namespaces like\u00a0<\/span>System.DirectoryServices<\/a>, that Windows PowerShell modules required.<\/p>\n

To make this work with PowerShell Core 6, we\u00a0<\/span>added the Windows Compatibility Pack 2.0.0 assemblies<\/a>\u00a0<\/span>into PowerShell Core 6.1. This also means that any module author who wants to leverage the Windows Compatibility Pack will not need to redistribute the Windows Compatibility Pack assemblies with their modules, as they are now built into PowerShell Core 6.1 (.NET Framework would already have its own implementations of the same APIs).<\/p>\n

Trouble in Paradise<\/h3>\n

Things were moving along getting the infrastructure ready for the porting work. However, we were also up against a strict timeline. We made a tactical decision to minimize risk of not getting our changes into Windows by moving away from targeting .NET Standard 2.0 and Windows Compatibility Pack explicitly. Building against .NET Standard 2.0 and relying on Windows Compatibility Pack meant we had to ship the\u00a0<\/span>Windows Compatibility Pack facade assemblies<\/a>\u00a0<\/span>in Windows so that .NET Framework knew how to find the actual implementation assembly to work correctly. Adding new assemblies in the Windows build environment is easy compared to adding new assemblies into Windows itself. Adding new binaries to Windows require making changes to specific files in the code base and you can only validate it worked successfully when you have a complete build of Windows. Windows code base is\u00a0<\/span>huge<\/strong>\u00a0<\/span>and takes a long time to build on a developer machine. A complete build may not be successful due to other changes happening in the overall code base which is what happened to us.<\/p>\n

In addition, we also hit issues where other assemblies in the Windows code base had a dependency on the Windows PowerShell module assembly (like a graphical user interface sitting on top of PowerShell) and would break the build since the GUI code built with .NET Framework didn’t know how to reference a .NET Standard built assembly.<\/p>\n

Since we could not successfully get a working build of Windows with our assembly additions, we decided to abandon that work and go without .NET Standard compilation.<\/p>\n

Moving Forward<\/h3>\n

Because .NET Standard is an API contract (like a C\/C++ header file), we did not need to formally rebuild Windows PowerShell modules with .NET Standard to have them work in .NET Core and thus PowerShell Core 6. What .NET Standard gives us is some assurance that future additions to Windows PowerShell modules would catch new API usage that is not part of .NET Standard or Windows Compatibility Pack. However, although .NET Standard indicates that an API is available, we found out during testing that even if the API exists, it may throw\u00a0<\/span>PlatformNotSupportedException<\/a>\u00a0<\/span>at runtime which prevents the Windows PowerShell module from working correctly in PowerShell Core 6. As we found these, we’ve had to make code changes in the modules to handle these cases appropriately during runtime whether it’s running within Windows PowerShell or PowerShell Core 6.<\/p>\n

Changes in PowerShell Core 6.1<\/h3>\n

In anticipation of the changes we were making in Windows PowerShell modules, we also needed changes in PowerShell Core 6.1 to make the experience seamless. In mid-June we published an\u00a0<\/span>RFC<\/a>\u00a0<\/span>to get community feedback on the user experience we would enable for using compatible Windows PowerShell modules from PowerShell Core 6.<\/p>\n

Basically, with\u00a0<\/span>PowerShell Core 6.1-Preview.4<\/a>\u00a0<\/span>the PowerShell engine will look in\u00a0<\/span>$env:WinDir\\System32\\WindowsPowerShell\\v1.0\\Modules<\/code>\u00a0<\/span>as part of enumerating available modules from\u00a0<\/span>$env:PSModulePath<\/a>. However, by default, only Windows PowerShell module manifests that explicitly indicate\u00a0<\/span>Core<\/code>\u00a0<\/span>as a supported in the\u00a0<\/span>CompatiblePSEditions<\/code>\u00a0<\/span>property. Windows PowerShell module manifests that don’t have the\u00a0<\/span>CompatiblePSEditions<\/code>\u00a0<\/span>property or only has\u00a0<\/span>Desktop<\/code>\u00a0<\/span>will not show up as an available module by default. The\u00a0<\/span>-SkipEditionCheck<\/code>\u00a0<\/span>switch on\u00a0<\/span>Import-Module<\/code>\u00a0<\/span>can be used to override this behavior. It is important to reiterate that this logic only applies to modules in\u00a0<\/span>$env:WinDir\\System32\\WindowsPowerShell\\v1.0\\Modules<\/code>\u00a0<\/span>path. If you used\u00a0<\/span>Install-Module<\/code>\u00a0<\/span>to install a module from the\u00a0<\/span>PowerShell Gallery<\/a>\u00a0<\/span>and its module manifest either doesn’t contain\u00a0<\/span>CompatiblePSEditions<\/code>\u00a0<\/span>nor includes\u00a0<\/span>Core<\/code>, the PSEdition check logic doesn’t apply and that module will show up as available since it was explicitly installed from PowerShell Core 6.<\/p>\n

Availability of changes in Windows<\/h3>\n

The Windows PowerShell module changes we’ve made will start showing up in\u00a0<\/span>Windows 10 Insiders Build 17711<\/a>. This means that with that build of Windows 10 and PowerShell Core 6.1-Preview.4 you can start using some of the Windows PowerShell modules we’ve identified as Core compatible. A reminder that as we continue to make changes in Windows, they will show up in newer builds of Windows 10 Insiders. Our target is to get >65% of the Windows PowerShell in-box modules compatible with PowerShell Core 6.1 within the next Windows 10 release. We’ll continue to work with Windows partner teams to get the number of compatible modules closer to 100%.<\/p>\n

Community Feedback<\/h3>\n

We work with individual Windows teams who are the owners of their modules to validate our changes as much as possible with PowerShell Core 6. However, due to how they wrote their tests, it’s not as simple as rerunning their existing tests within PowerShell Core 6 and in many cases, we opted to do some limited manual validation. This means that although we’ve made some effort to validate Windows PowerShell modules marked as compatible with\u00a0<\/span>Core<\/code>, we cannot guarantee they are fully compatible (see the note about runtime issues above) and we would treat them as bugs.<\/p>\n

Here the community can help start using Windows PowerShell modules in PowerShell Core 6.1-Preview.4 and if you see an issue or difference compared to Windows PowerShell 5.1, please open an issue in the\u00a0<\/span>PowerShellModuleCoverage<\/a>\u00a0<\/span>repository. We are only using that repository for issue tracking. If someone else has opened an issue on a module (or identified a module they need but isn’t ported yet), please give it a ?? (thumbs up) to up-vote the issue which helps us prioritize the work.<\/p>\n

Note that if there are other Windows PowerShell modules by Microsoft that aren’t in-box in Windows that you think is a priority to port, please open an issue in that repo! PowerShell Core compatibility will help us with engagements with those partner teams if we can show them the demand for it.<\/p>\n

Windows Compatibility Module<\/h3>\n

As we continue this journey porting Windows PowerShell modules which I expect to take multiple Windows releases, you can use any Windows PowerShell module from PowerShell Core 6 without waiting for a port. A significant majority of the Windows codebase is written in C\/C++ (95+%!). A number of teams who are primarily native code developers wrote their PowerShell modules in\u00a0<\/span>Managed C++<\/a>. However, there are currently\u00a0<\/span>no plans to support Managed C++ in .NET Core<\/a>. For these sets of modules and those waiting to be ported, we published the\u00a0<\/span>Windows Compatibility<\/a>\u00a0<\/span>module which leverages implicit remoting from PowerShell Core 6 to Windows PowerShell, so it makes the cmdlets from that module available in PowerShell Core 6 but executes it within Windows PowerShell.<\/p>\n

You can use this today with the stable release of\u00a0<\/span>PowerShell Core 6.0.3<\/a>\u00a0<\/span>and doesn’t require using PowerShell Core 6.1-Preview builds. If you find issues with this module, please open issues in the\u00a0<\/span>Windows Compatibility repo<\/a>.<\/p>\n

Summary<\/h2>\n

The initial goal of the PowerShell Core 6.0 release was to provide a cross-platform automation platform. I expect by end of July, we should hit a new high of\u00a0<\/span>3 million startups<\/a>\u00a0<\/span>of PowerShell Core 6!\nApproximately 80% of the usage is on Linux, so I consider us working with the community to have achieved our first goal.<\/p>\n

The goal of PowerShell Core 6.1 release (targeting end of August for General Availability) is to help move Windows PowerShell users forward to PowerShell Core 6 by providing compatibility with in-box Windows PowerShell modules that they depend upon. It is important to understand that the Windows PowerShell module porting work won’t be complete by the time PowerShell Core 6.1 GA nor when the next version of Windows 10 is released, and we expect to continue this work as needed to eventually get complete coverage.<\/p>\n

Thanks to all the feedback and contributions from the community helping PowerShell move forward!<\/p>\n

Steve Lee\nPrincipal Software Engineering Manager\nPowerShell Team\nAzure Compute\nhttps:\/\/twitter.com\/Steve_MSFT<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

During the\u00a0May 2018 Community Call\u00a0and a\u00a0tweet\u00a0a few weeks later, we mentioned that PowerShell team was spending significant time in the Windows codebase. We even demoed using the\u00a0Active Directory PowerShell Module\u00a0from PowerShell Core 6 during the PowerShell Community Call. We started investigating some of the top requested modules that ship with Windows to see how to […]<\/p>\n","protected":false},"author":685,"featured_media":13641,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[],"class_list":["post-14205","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-powershell"],"acf":[],"blog_post_summary":"

During the\u00a0May 2018 Community Call\u00a0and a\u00a0tweet\u00a0a few weeks later, we mentioned that PowerShell team was spending significant time in the Windows codebase. We even demoed using the\u00a0Active Directory PowerShell Module\u00a0from PowerShell Core 6 during the PowerShell Community Call. We started investigating some of the top requested modules that ship with Windows to see how to […]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/powershell\/wp-json\/wp\/v2\/posts\/14205","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/powershell\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/powershell\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/powershell\/wp-json\/wp\/v2\/users\/685"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/powershell\/wp-json\/wp\/v2\/comments?post=14205"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/powershell\/wp-json\/wp\/v2\/posts\/14205\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/powershell\/wp-json\/wp\/v2\/media\/13641"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/powershell\/wp-json\/wp\/v2\/media?parent=14205"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/powershell\/wp-json\/wp\/v2\/categories?post=14205"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/powershell\/wp-json\/wp\/v2\/tags?post=14205"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}