![\"Assembly](\"https:\/\/devblogs.microsoft.com\/powershell\/wp-content\/uploads\/sites\/30\/2020\/06\/moduleconflict.png\")
<\/a><\/p>\nIn this blog post, we’ll look at some of the ways dependency conflicts can arise in PowerShell, and some of the ways to mitigate dependency conflict issues. Even if you’re not a module author, there are some tricks in here that might help you with dependency conflicts arising in modules that you use.<\/p>\n
<\/a>Contents<\/h2>\nThis is a fairly long blog post, so here’s a table of contents:<\/p>\n
\n- Why do dependency conflicts occur?<\/a><\/li>\n
- PowerShell and .NET<\/a><\/li>\n
- Quick fixes and their limitations<\/a><\/li>\n
- More robust solutions<\/a><\/li>\n
- Solutions for dependency conflicts that don’t work for PowerShell<\/a><\/li>\n
- Solving the issue in PowerShell itself…<\/a><\/li>\n
- Further reading<\/a><\/li>\n
- Final notes<\/a><\/li>\n<\/ul>\n
<\/a>Why do dependency conflicts occur?<\/h2>\nIn .NET, dependency conflicts arise when two versions of the same assembly are loaded into the same Assembly Load Context (this term means slightly different things on different .NET platforms, which we’ll cover later). This conflict issue is a common problem that occurs essentially everywhere in software where versioned dependencies are used. Because there are no simple solutions (and because many dependency resolution frameworks try to solve the problem in different ways), this situation is often called “dependency hell”.<\/p>\n
Conflict issues are compounded by the fact that a project almost never deliberately or directly depends on two versions of the same dependency, but instead depends on two different dependencies that each require a different version of the same dependency.<\/p>\n
For example, say your .NET application, DuckBuilder<\/code>, brings in two dependencies, to perform parts of its functionality and looks like this:<\/p>\n![\"Two](\"https:\/\/devblogs.microsoft.com\/powershell\/wp-content\/uploads\/sites\/30\/2020\/06\/dep-conflict.jpg\")
<\/a><\/p>\nBecause Contoso.ZipTools<\/code> and Fabrikam.FileHelpers<\/code> both depend on Newtonsoft.Json<\/code>, but different versions, there may be a dependency conflict, depending on how each dependency is loaded.<\/p>\n<\/a>Conflicting with PowerShell’s dependencies<\/h3>\nIn PowerShell, the dependency conflict issue is exacerbated because PowerShell modules, and PowerShell’s own dependencies, are all loaded into the same shared context. This means the PowerShell engine and all loaded PowerShell modules must not have conflicting dependencies.<\/p>\n
One scenario in which this can cause issues is where a module has a dependency that conflicts with one of PowerShell’s own dependencies. A classic example of this is Newtonsoft.Json:<\/p>\n
![\"FictionalTools](\"https:\/\/devblogs.microsoft.com\/powershell\/wp-content\/uploads\/sites\/30\/2020\/06\/engine-conflict.jpg\")
<\/a><\/p>\nIn this example, the module FictionalTools<\/code> depends on Newtonsoft.Json<\/code> version 12.0.3<\/code>, which is a newer version of Newtonsoft.Json than 11.0.2 that ships in the example PowerShell. (To be clear, this is an example and PowerShell 7 currently ships with Newtonsoft.Json 12.0.3.)<\/p>\nBecause the module depends on a newer version of the assembly, it won’t accept the version that PowerShell already has loaded, but because PowerShell has already loaded a version of the assembly, the module can’t load its own using the conventional load mechanism.<\/p>\n
<\/a>Conflicting with another module’s dependencies<\/h3>\nAnother common scenario in PowerShell is that a module is loaded that depends on one version of an assembly, and then another module is loaded later that depends on a different version of that assembly.<\/p>\n
This often looks like the following:<\/p>\n
![\"Two](\"https:\/\/devblogs.microsoft.com\/powershell\/wp-content\/uploads\/sites\/30\/2020\/06\/mod-conflict.jpg\")
<\/a><\/p>\nIn the above case, the FictionalTools<\/code> module requires a newer version of Microsoft.Extensions.Logging<\/code> than the FilesystemManager<\/code> module.<\/p>\nLet’s imagine these modules load their dependencies by placing the dependency assemblies in the same directory as the root module assembly and allowing .NET to implicitly load them by name. If we are running PowerShell 7 (on top of .NET Core 3.1), we can load and run FictionalTools<\/code> and then load and run FilesystemManager<\/code> without issue. However, if in a new session we load and run FilesystemManager<\/code> and then FictionalTools<\/code>, we will encounter a FileLoadException<\/code> from the FictionalTools<\/code> command, because it requires a newer version of Microsoft.Extensions.Logging<\/code> than the one loaded, but cannot load it because an assembly of the same name has already been loaded.<\/p>\n<\/a>PowerShell and .NET<\/h2>\nPowerShell runs on the .NET platform, and since we’re discussing assembly dependency conflicts, we must discuss .NET. .NET is ultimately responsible for resolving and loading assembly dependencies, so we must understand how .NET operates here to understand dependency conflicts.<\/p>\n
We must also confront the fact that different versions of PowerShell run on different .NET implementations, which respectively have their own mechanisms for assembly resolution.<\/p>\n
In general, PowerShell 5.1 and below run on .NET Framework, while PowerShell 6 and above run on .NET Core. These two flavours of .NET load and handle assemblies somewhat differently, meaning resolving dependency conflicts can vary depending on the underlying .NET platform.<\/p>\n
<\/a>Assembly Load Contexts<\/h3>\nIn this post, we’ll use the term “Assembly Load Context” (ALC) frequently. An Assembly Load Context is a .NET concept that essentially means a runtime namespace into which assemblies are loaded and within which assemblies’ names are unique. This concept allows assemblies to be uniquely resolved by name in each ALC.<\/p>\n
<\/a>Assembly reference loading in .NET<\/h3>\nThe semantics of assembly loading (whether versions clash, whether that assembly’s dependencies are handled, etc.) depend on both the .NET implementation (.NET Core vs .NET Framework) and the API or .NET mechanism used to load a particular assembly.<\/p>\n
Rather than go into deep detail describing these here, there is a list of links in the Further reading<\/a> section that go into great detail on how .NET assembly loading works in each .NET implementation.<\/p>\nIn this blog post, we’ll refer to the following mechanisms:<\/p>\n
\n- Implicit assembly loading (effectively
Assembly.Load(AssemblyName)<\/code>), when .NET implicitly tries to load an assembly by name from a static assembly reference in .NET code.<\/li>\nAssembly.LoadFrom()<\/code>, a plugin-oriented loading API that will add handlers to resolve dependencies of the loaded DLL (but which may not do so the way we want).<\/li>\nAssembly.LoadFile()<\/code>, a bare-bones loading API intended to load only the assembly asked for with no handling of its dependencies.<\/li>\n<\/ul>\nThe way these APIs work has changed in subtle ways between .NET Core and .NET Framework, so it’s worth reading through the further reading links.<\/p>\n
<\/a>Differences in .NET Framework vs .NET Core<\/h3>\nImportantly, Assembly Load Contexts and other assembly resolution mechanisms have changed between .NET Framework and .NET Core.<\/p>\n
In particular .NET Framework has the following features:<\/p>\n
\n- The Global Assembly Cache, for machine-wide assembly resolution<\/li>\n
- Application Domains, which work like in-process sandboxes for assembly isolation, but also present a serialization layer to contend with<\/li>\n
- A limited assembly load context model, explained in depth here<\/a>, that has a fixed set of assembly load contexts, each with their own behaviour:\n
\n- The default load context, where assemblies are loaded by default<\/li>\n
- The load-from context, for loading assemblies manually at runtime<\/li>\n
- The reflection-only context, for safely loading assemblies to read their metadata without running them<\/li>\n
- The mysterious void that assemblies loaded with
Assembly.LoadFile(string path)<\/code> and Assembly.Load(byte[] asmBytes)<\/code> live in<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n.NET Core (and .NET 5+) has eschewed this complexity for a simpler model:<\/p>\n
\n- No Global Assembly Cache; applications bring all their own dependencies (PowerShell, as the plugin host, complicates this slightly for modules ??? its dependencies in
$PSHOME<\/code> are shared with all modules). This removes an exogenous factor for dependency resolution in applications, making dependency resolution more reproducible.<\/li>\n- Only one Application Domain, and no ability to create new ones. The Application Domain concept lives on purely to be the global state of the .NET process.<\/li>\n
- A new, extensible Assembly Load Context model, where assembly resolution can be effectively namespaced by putting it in a new ALC. .NET Core processes begin with a single default ALC into which all assemblies are loaded (except for those loaded with
Assembly.LoadFile(string)<\/code> and Assembly.Load(byte[])<\/code>), but are free to create and define their own custom ALCs with their own loading logic. When an assembly is loaded, the first ALC it is loaded into is responsible for resolving its dependencies, creating opportunities to create powerful .NET plugin loading mechanisms.<\/li>\n<\/ul>\nIn both implementations, assemblies are loaded lazily when a method requiring their type is run for the first time.<\/p>\n
For example here are two versions of the same code that load a dependency at different times.<\/p>\n
The first will always load its dependency when Program.GetRange()<\/code> is called, because the dependency reference is lexically present within the method:<\/p>\n\nusing<\/span> Dependency<\/span>.Library<\/span>;\r\n\r\npublic<\/span> static<\/span> class<\/span> Program<\/span>\r\n{\r\n public<\/span> static<\/span> List<\/span><int<\/span>> GetRange<\/span>(int<\/span> limit<\/span>)\r\n {\r\n var<\/span> list<\/span> =<\/span> new<\/span> List<\/span><int<\/span>>();\r\n for<\/span> (int<\/span> i<\/span> =<\/span> 0<\/span>; i<\/span> <<\/span> limit<\/span>; i<\/span>++<\/span>)\r\n {\r\n if<\/span> (i<\/span> >=<\/span> 20<\/span>)\r\n {\r\n \/\/<\/span> Dependency.Library will be loaded when GetNumbers is run<\/span>\r\n \/\/<\/span> because the dependency call occurs directly within the method<\/span>\r\n DependencyApi<\/span>.Use<\/span>();\r\n }\r\n\r\n list<\/span>.Add<\/span>(i<\/span>);\r\n }\r\n return<\/span> list<\/span>;\r\n }\r\n}<\/pre>\n<\/div>\nThe second will load its dependency only if the limit<\/code> parameter is 20 or more, because of the internal indirection through a method:<\/p>\n\nusing<\/span> Dependency<\/span>.Library<\/span>;\r\n\r\npublic<\/span> static<\/span> class<\/span> Program<\/span>\r\n{\r\n public<\/span> static<\/span> List<\/span><int<\/span>> GetNumbers<\/span>(int<\/span> limit<\/span>)\r\n {\r\n var<\/span> list<\/span> =<\/span> new<\/span> List<\/span><int<\/span>>();\r\n for<\/span> (int<\/span> i<\/span> =<\/span> 0<\/span>; i<\/span> <<\/span> limit<\/span>; i<\/span>++<\/span>)\r\n {\r\n if<\/span> (i<\/span> >=<\/span> 20<\/span>)\r\n {\r\n \/\/<\/span> Dependency.Library is only referenced within<\/span>\r\n \/\/<\/span> the UseDependencyApi() method,<\/span>\r\n \/\/<\/span> so will only be loaded when limit >= 20<\/span>\r\n UseDependencyApi<\/span>();\r\n }\r\n\r\n list<\/span>.Add<\/span>(i<\/span>);\r\n }\r\n return<\/span> list<\/span>;\r\n }\r\n\r\n private<\/span> static<\/span> void<\/span> UseDependencyApi<\/span>()\r\n {\r\n \/\/<\/span> Once UseDependencyApi() is called, Dependency.Library is loaded<\/span>\r\n DependencyApi<\/span>.Use<\/span>();\r\n }\r\n}<\/pre>\n<\/div>\nThis is a good thing for .NET to do, since it minimizes the memory and filesystem reads it needs to use, making it more resource efficient.<\/p>\n
Unfortunately a side effect of this is that if an assembly load will fail, we won’t necessarily know when the program is first loaded, only when the code path that tries to load the assembly is run.<\/p>\n
It can also set up timing conditions for assembly load conflicts; if two parts of the same program will try to load different versions of the same assembly, which one is loaded depends on which code path is run first.<\/p>\n
For PowerShell this means that the following factors can affect an assembly load conflict:<\/p>\n
\n- Which module was loaded first?<\/li>\n
- Was the cmdlet\/code path that uses the dependency library run?<\/li>\n
- Does PowerShell load a conflicting dependency at startup or only under certain code paths?<\/li>\n<\/ul>\n
<\/a>Quick fixes and their limitations<\/h2>\nIn some cases it’s possible to make small adjustments to your module and fix things with a minimum of fuss. But these solutions tend to come with caveats, so that while they may apply to your module, they won’t work for every module.<\/p>\n
<\/a>Change your dependency version<\/h3>\nThe simplest way to avoid dependency conflicts is to agree on a dependency. This may be possible when:<\/p>\n
\n- Your conflict is with a direct dependency of your module and you control the version<\/li>\n
- Your conflict is with an indirect dependency, but you can configure your direct dependencies to use a workable indirect dependency version<\/li>\n
- You know the conflicting version and can rely on it not changing<\/li>\n<\/ul>\n
A good example of this last scenario is with the Newtonsoft.Json<\/code> package. This is a dependency of PowerShell 6 and above, and isn’t used in Windows PowerShell, meaning a simple way to resolve versioning conflicts is to target the lowest version of Newtonsoft.Json<\/code> across the PowerShell versions you wish to target.<\/p>\nFor example, PowerShell 6.2.6 and PowerShell 7.0.2 both currently use Newtonsoft.Json version 12.0.3. So, to create a module targeting Windows PowerShell, PowerShell 6 and PowerShell 7, you would target Newtonsoft.Json 12.0.3 as a dependency and include it in your built module. When the module is loaded in PowerShell 6 or 7, PowerShell’s own Newtonsoft.Json assembly will already be loaded, but the version will be at least the required one for your module, meaning resolution will succeed. In Windows PowerShell, the assembly will not be already present in PowerShell, and so will be loaded from your module instead.<\/p>\n
Generally, when targeting a concrete PowerShell package, like Microsoft.PowerShell.Sdk or System.Management.Automation, NuGet should be able to resolve the right dependency versions required. It’s only in the case of targeting both Windows PowerShell and PowerShell 6+ that dependency versioning becomes more difficult, either because of needing to target multiple frameworks, or due to targeting PowerShellStandard.Library.<\/p>\n
Circumstances where pinning to a common dependency version won’t work include:<\/p>\n
\n- The conflict is with an indirect dependency, and there’s no configuration of your dependencies that can use the common version<\/li>\n
- The other dependency version is likely to change often, meaning settling on a common version will only be a short-term fix<\/li>\n<\/ul>\n
<\/a>Add an AssemblyResolve event handler to create a dynamic binding redirect<\/h3>\nWhen changing your own dependency version isn’t possible, or is too hard, another way to make your module play nicely with other dependencies is to set up a dynamic binding redirect by registering an event handler for the AssemblyResolve<\/code> event.<\/p>\nAs with a static binding redirect, the point here to is force all consumers of a dependency to use the same actual assembly. This means we need to intercept calls to load the dependency and always redirect them to the version we want.<\/p>\n
This looks something like this:<\/p>\n
\n\/\/<\/span> Register the event handler as early as you can in your code.<\/span>\r\n\/\/<\/span> A good option is to use the IModuleAssemblyInitializer interface<\/span>\r\n\/\/<\/span> that PowerShell provides to run code early on when your module is loaded.<\/span>\r\n\r\n\/\/<\/span> This class will be instantiated on module import and the OnImport() method run.<\/span>\r\n\/\/<\/span> Make sure that:<\/span>\r\n\/\/<\/span> - the class is public<\/span>\r\n\/\/<\/span> - the class has a public, parameterless constructor<\/span>\r\n\/\/<\/span> - the class implements IModuleAssemblyInitializer<\/span>\r\npublic<\/span> class<\/span> MyModuleInitializer<\/span> : IModuleAssemblyInitializer<\/span>\r\n{\r\n public<\/span> void<\/span> OnImport<\/span>()\r\n {\r\n AppDomain<\/span>.CurrentDomain<\/span>.AssemblyResolve<\/span> +=<\/span> DependencyResolution<\/span>.ResolveNewtonsoftJson<\/span>;\r\n }\r\n}\r\n\r\n\/\/<\/span> Clean up the event handler when the the module is removed<\/span>\r\n\/\/<\/span> to prevent memory leaks.<\/span>\r\n\/\/<\/span><\/span>\r\n\/\/<\/span> Like IModuleAssemblyInitializer, IModuleAssemblyCleanup allows<\/span>\r\n\/\/<\/span> you to register code to run when a module is removed (with Remove-Module).<\/span>\r\n\/\/<\/span> Make sure it is also public with a public parameterless contructor<\/span>\r\n\/\/<\/span> and implements IModuleAssemblyCleanup.<\/span>\r\n
<\/a>Why do dependency conflicts occur?<\/h2>\nIn .NET, dependency conflicts arise when two versions of the same assembly are loaded into the same Assembly Load Context (this term means slightly different things on different .NET platforms, which we’ll cover later). This conflict issue is a common problem that occurs essentially everywhere in software where versioned dependencies are used. Because there are no simple solutions (and because many dependency resolution frameworks try to solve the problem in different ways), this situation is often called “dependency hell”.<\/p>\n
Conflict issues are compounded by the fact that a project almost never deliberately or directly depends on two versions of the same dependency, but instead depends on two different dependencies that each require a different version of the same dependency.<\/p>\n
For example, say your .NET application, DuckBuilder<\/code>, brings in two dependencies, to perform parts of its functionality and looks like this:<\/p>\n<\/a><\/p>\nBecause Contoso.ZipTools<\/code> and Fabrikam.FileHelpers<\/code> both depend on Newtonsoft.Json<\/code>, but different versions, there may be a dependency conflict, depending on how each dependency is loaded.<\/p>\n<\/a>Conflicting with PowerShell’s dependencies<\/h3>\nIn PowerShell, the dependency conflict issue is exacerbated because PowerShell modules, and PowerShell’s own dependencies, are all loaded into the same shared context. This means the PowerShell engine and all loaded PowerShell modules must not have conflicting dependencies.<\/p>\n
One scenario in which this can cause issues is where a module has a dependency that conflicts with one of PowerShell’s own dependencies. A classic example of this is Newtonsoft.Json:<\/p>\n
<\/a><\/p>\nIn this example, the module FictionalTools<\/code> depends on Newtonsoft.Json<\/code> version 12.0.3<\/code>, which is a newer version of Newtonsoft.Json than 11.0.2 that ships in the example PowerShell. (To be clear, this is an example and PowerShell 7 currently ships with Newtonsoft.Json 12.0.3.)<\/p>\nBecause the module depends on a newer version of the assembly, it won’t accept the version that PowerShell already has loaded, but because PowerShell has already loaded a version of the assembly, the module can’t load its own using the conventional load mechanism.<\/p>\n
<\/a>Conflicting with another module’s dependencies<\/h3>\nAnother common scenario in PowerShell is that a module is loaded that depends on one version of an assembly, and then another module is loaded later that depends on a different version of that assembly.<\/p>\n
This often looks like the following:<\/p>\n
<\/a><\/p>\nIn the above case, the FictionalTools<\/code> module requires a newer version of Microsoft.Extensions.Logging<\/code> than the FilesystemManager<\/code> module.<\/p>\nLet’s imagine these modules load their dependencies by placing the dependency assemblies in the same directory as the root module assembly and allowing .NET to implicitly load them by name. If we are running PowerShell 7 (on top of .NET Core 3.1), we can load and run FictionalTools<\/code> and then load and run FilesystemManager<\/code> without issue. However, if in a new session we load and run FilesystemManager<\/code> and then FictionalTools<\/code>, we will encounter a FileLoadException<\/code> from the FictionalTools<\/code> command, because it requires a newer version of Microsoft.Extensions.Logging<\/code> than the one loaded, but cannot load it because an assembly of the same name has already been loaded.<\/p>\n<\/a>PowerShell and .NET<\/h2>\nPowerShell runs on the .NET platform, and since we’re discussing assembly dependency conflicts, we must discuss .NET. .NET is ultimately responsible for resolving and loading assembly dependencies, so we must understand how .NET operates here to understand dependency conflicts.<\/p>\n
We must also confront the fact that different versions of PowerShell run on different .NET implementations, which respectively have their own mechanisms for assembly resolution.<\/p>\n
In general, PowerShell 5.1 and below run on .NET Framework, while PowerShell 6 and above run on .NET Core. These two flavours of .NET load and handle assemblies somewhat differently, meaning resolving dependency conflicts can vary depending on the underlying .NET platform.<\/p>\n
<\/a>Assembly Load Contexts<\/h3>\nIn this post, we’ll use the term “Assembly Load Context” (ALC) frequently. An Assembly Load Context is a .NET concept that essentially means a runtime namespace into which assemblies are loaded and within which assemblies’ names are unique. This concept allows assemblies to be uniquely resolved by name in each ALC.<\/p>\n
<\/a>Assembly reference loading in .NET<\/h3>\nThe semantics of assembly loading (whether versions clash, whether that assembly’s dependencies are handled, etc.) depend on both the .NET implementation (.NET Core vs .NET Framework) and the API or .NET mechanism used to load a particular assembly.<\/p>\n
Rather than go into deep detail describing these here, there is a list of links in the Further reading<\/a> section that go into great detail on how .NET assembly loading works in each .NET implementation.<\/p>\nIn this blog post, we’ll refer to the following mechanisms:<\/p>\n
\n- Implicit assembly loading (effectively
Assembly.Load(AssemblyName)<\/code>), when .NET implicitly tries to load an assembly by name from a static assembly reference in .NET code.<\/li>\nAssembly.LoadFrom()<\/code>, a plugin-oriented loading API that will add handlers to resolve dependencies of the loaded DLL (but which may not do so the way we want).<\/li>\nAssembly.LoadFile()<\/code>, a bare-bones loading API intended to load only the assembly asked for with no handling of its dependencies.<\/li>\n<\/ul>\nThe way these APIs work has changed in subtle ways between .NET Core and .NET Framework, so it’s worth reading through the further reading links.<\/p>\n
<\/a>Differences in .NET Framework vs .NET Core<\/h3>\nImportantly, Assembly Load Contexts and other assembly resolution mechanisms have changed between .NET Framework and .NET Core.<\/p>\n
In particular .NET Framework has the following features:<\/p>\n
\n- The Global Assembly Cache, for machine-wide assembly resolution<\/li>\n
- Application Domains, which work like in-process sandboxes for assembly isolation, but also present a serialization layer to contend with<\/li>\n
- A limited assembly load context model, explained in depth here<\/a>, that has a fixed set of assembly load contexts, each with their own behaviour:\n
\n- The default load context, where assemblies are loaded by default<\/li>\n
- The load-from context, for loading assemblies manually at runtime<\/li>\n
- The reflection-only context, for safely loading assemblies to read their metadata without running them<\/li>\n
- The mysterious void that assemblies loaded with
Assembly.LoadFile(string path)<\/code> and Assembly.Load(byte[] asmBytes)<\/code> live in<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n.NET Core (and .NET 5+) has eschewed this complexity for a simpler model:<\/p>\n
\n- No Global Assembly Cache; applications bring all their own dependencies (PowerShell, as the plugin host, complicates this slightly for modules ??? its dependencies in
$PSHOME<\/code> are shared with all modules). This removes an exogenous factor for dependency resolution in applications, making dependency resolution more reproducible.<\/li>\n- Only one Application Domain, and no ability to create new ones. The Application Domain concept lives on purely to be the global state of the .NET process.<\/li>\n
- A new, extensible Assembly Load Context model, where assembly resolution can be effectively namespaced by putting it in a new ALC. .NET Core processes begin with a single default ALC into which all assemblies are loaded (except for those loaded with
Assembly.LoadFile(string)<\/code> and Assembly.Load(byte[])<\/code>), but are free to create and define their own custom ALCs with their own loading logic. When an assembly is loaded, the first ALC it is loaded into is responsible for resolving its dependencies, creating opportunities to create powerful .NET plugin loading mechanisms.<\/li>\n<\/ul>\nIn both implementations, assemblies are loaded lazily when a method requiring their type is run for the first time.<\/p>\n
For example here are two versions of the same code that load a dependency at different times.<\/p>\n
The first will always load its dependency when Program.GetRange()<\/code> is called, because the dependency reference is lexically present within the method:<\/p>\n\nusing<\/span> Dependency<\/span>.Library<\/span>;\r\n\r\npublic<\/span> static<\/span> class<\/span> Program<\/span>\r\n{\r\n public<\/span> static<\/span> List<\/span><int<\/span>> GetRange<\/span>(int<\/span> limit<\/span>)\r\n {\r\n var<\/span> list<\/span> =<\/span> new<\/span> List<\/span><int<\/span>>();\r\n for<\/span> (int<\/span> i<\/span> =<\/span> 0<\/span>; i<\/span> <<\/span> limit<\/span>; i<\/span>++<\/span>)\r\n {\r\n if<\/span> (i<\/span> >=<\/span> 20<\/span>)\r\n {\r\n \/\/<\/span> Dependency.Library will be loaded when GetNumbers is run<\/span>\r\n \/\/<\/span> because the dependency call occurs directly within the method<\/span>\r\n DependencyApi<\/span>.Use<\/span>();\r\n }\r\n\r\n list<\/span>.Add<\/span>(i<\/span>);\r\n }\r\n return<\/span> list<\/span>;\r\n }\r\n}<\/pre>\n<\/div>\nThe second will load its dependency only if the limit<\/code> parameter is 20 or more, because of the internal indirection through a method:<\/p>\n\nusing<\/span> Dependency<\/span>.Library<\/span>;\r\n\r\npublic<\/span> static<\/span> class<\/span> Program<\/span>\r\n{\r\n public<\/span> static<\/span> List<\/span><int<\/span>> GetNumbers<\/span>(int<\/span> limit<\/span>)\r\n {\r\n var<\/span> list<\/span> =<\/span> new<\/span> List<\/span><int<\/span>>();\r\n for<\/span> (int<\/span> i<\/span> =<\/span> 0<\/span>; i<\/span> <<\/span> limit<\/span>; i<\/span>++<\/span>)\r\n {\r\n if<\/span> (i<\/span> >=<\/span> 20<\/span>)\r\n {\r\n \/\/<\/span> Dependency.Library is only referenced within<\/span>\r\n \/\/<\/span> the UseDependencyApi() method,<\/span>\r\n \/\/<\/span> so will only be loaded when limit >= 20<\/span>\r\n UseDependencyApi<\/span>();\r\n }\r\n\r\n list<\/span>.Add<\/span>(i<\/span>);\r\n }\r\n return<\/span> list<\/span>;\r\n }\r\n\r\n private<\/span> static<\/span> void<\/span> UseDependencyApi<\/span>()\r\n {\r\n \/\/<\/span> Once UseDependencyApi() is called, Dependency.Library is loaded<\/span>\r\n DependencyApi<\/span>.Use<\/span>();\r\n }\r\n}<\/pre>\n<\/div>\nThis is a good thing for .NET to do, since it minimizes the memory and filesystem reads it needs to use, making it more resource efficient.<\/p>\n
Unfortunately a side effect of this is that if an assembly load will fail, we won’t necessarily know when the program is first loaded, only when the code path that tries to load the assembly is run.<\/p>\n
It can also set up timing conditions for assembly load conflicts; if two parts of the same program will try to load different versions of the same assembly, which one is loaded depends on which code path is run first.<\/p>\n
For PowerShell this means that the following factors can affect an assembly load conflict:<\/p>\n
\n- Which module was loaded first?<\/li>\n
- Was the cmdlet\/code path that uses the dependency library run?<\/li>\n
- Does PowerShell load a conflicting dependency at startup or only under certain code paths?<\/li>\n<\/ul>\n
<\/a>Quick fixes and their limitations<\/h2>\nIn some cases it’s possible to make small adjustments to your module and fix things with a minimum of fuss. But these solutions tend to come with caveats, so that while they may apply to your module, they won’t work for every module.<\/p>\n
<\/a>Change your dependency version<\/h3>\nThe simplest way to avoid dependency conflicts is to agree on a dependency. This may be possible when:<\/p>\n
\n- Your conflict is with a direct dependency of your module and you control the version<\/li>\n
- Your conflict is with an indirect dependency, but you can configure your direct dependencies to use a workable indirect dependency version<\/li>\n
- You know the conflicting version and can rely on it not changing<\/li>\n<\/ul>\n
A good example of this last scenario is with the Newtonsoft.Json<\/code> package. This is a dependency of PowerShell 6 and above, and isn’t used in Windows PowerShell, meaning a simple way to resolve versioning conflicts is to target the lowest version of Newtonsoft.Json<\/code> across the PowerShell versions you wish to target.<\/p>\nFor example, PowerShell 6.2.6 and PowerShell 7.0.2 both currently use Newtonsoft.Json version 12.0.3. So, to create a module targeting Windows PowerShell, PowerShell 6 and PowerShell 7, you would target Newtonsoft.Json 12.0.3 as a dependency and include it in your built module. When the module is loaded in PowerShell 6 or 7, PowerShell’s own Newtonsoft.Json assembly will already be loaded, but the version will be at least the required one for your module, meaning resolution will succeed. In Windows PowerShell, the assembly will not be already present in PowerShell, and so will be loaded from your module instead.<\/p>\n
Generally, when targeting a concrete PowerShell package, like Microsoft.PowerShell.Sdk or System.Management.Automation, NuGet should be able to resolve the right dependency versions required. It’s only in the case of targeting both Windows PowerShell and PowerShell 6+ that dependency versioning becomes more difficult, either because of needing to target multiple frameworks, or due to targeting PowerShellStandard.Library.<\/p>\n
Circumstances where pinning to a common dependency version won’t work include:<\/p>\n
\n- The conflict is with an indirect dependency, and there’s no configuration of your dependencies that can use the common version<\/li>\n
- The other dependency version is likely to change often, meaning settling on a common version will only be a short-term fix<\/li>\n<\/ul>\n
<\/a>Add an AssemblyResolve event handler to create a dynamic binding redirect<\/h3>\nWhen changing your own dependency version isn’t possible, or is too hard, another way to make your module play nicely with other dependencies is to set up a dynamic binding redirect by registering an event handler for the AssemblyResolve<\/code> event.<\/p>\nAs with a static binding redirect, the point here to is force all consumers of a dependency to use the same actual assembly. This means we need to intercept calls to load the dependency and always redirect them to the version we want.<\/p>\n
This looks something like this:<\/p>\n
\n\/\/<\/span> Register the event handler as early as you can in your code.<\/span>\r\n\/\/<\/span> A good option is to use the IModuleAssemblyInitializer interface<\/span>\r\n\/\/<\/span> that PowerShell provides to run code early on when your module is loaded.<\/span>\r\n\r\n\/\/<\/span> This class will be instantiated on module import and the OnImport() method run.<\/span>\r\n\/\/<\/span> Make sure that:<\/span>\r\n\/\/<\/span> - the class is public<\/span>\r\n\/\/<\/span> - the class has a public, parameterless constructor<\/span>\r\n\/\/<\/span> - the class implements IModuleAssemblyInitializer<\/span>\r\npublic<\/span> class<\/span> MyModuleInitializer<\/span> : IModuleAssemblyInitializer<\/span>\r\n{\r\n public<\/span> void<\/span> OnImport<\/span>()\r\n {\r\n AppDomain<\/span>.CurrentDomain<\/span>.AssemblyResolve<\/span> +=<\/span> DependencyResolution<\/span>.ResolveNewtonsoftJson<\/span>;\r\n }\r\n}\r\n\r\n\/\/<\/span> Clean up the event handler when the the module is removed<\/span>\r\n\/\/<\/span> to prevent memory leaks.<\/span>\r\n\/\/<\/span><\/span>\r\n\/\/<\/span> Like IModuleAssemblyInitializer, IModuleAssemblyCleanup allows<\/span>\r\n\/\/<\/span> you to register code to run when a module is removed (with Remove-Module).<\/span>\r\n\/\/<\/span> Make sure it is also public with a public parameterless contructor<\/span>\r\n\/\/<\/span> and implements IModuleAssemblyCleanup.<\/span>\r\n
DuckBuilder<\/code>, brings in two dependencies, to perform parts of its functionality and looks like this:<\/p>\n<\/a><\/p>\nBecause Contoso.ZipTools<\/code> and Fabrikam.FileHelpers<\/code> both depend on Newtonsoft.Json<\/code>, but different versions, there may be a dependency conflict, depending on how each dependency is loaded.<\/p>\n<\/a>Conflicting with PowerShell’s dependencies<\/h3>\nIn PowerShell, the dependency conflict issue is exacerbated because PowerShell modules, and PowerShell’s own dependencies, are all loaded into the same shared context. This means the PowerShell engine and all loaded PowerShell modules must not have conflicting dependencies.<\/p>\n
One scenario in which this can cause issues is where a module has a dependency that conflicts with one of PowerShell’s own dependencies. A classic example of this is Newtonsoft.Json:<\/p>\n
<\/a><\/p>\nIn this example, the module FictionalTools<\/code> depends on Newtonsoft.Json<\/code> version 12.0.3<\/code>, which is a newer version of Newtonsoft.Json than 11.0.2 that ships in the example PowerShell. (To be clear, this is an example and PowerShell 7 currently ships with Newtonsoft.Json 12.0.3.)<\/p>\nBecause the module depends on a newer version of the assembly, it won’t accept the version that PowerShell already has loaded, but because PowerShell has already loaded a version of the assembly, the module can’t load its own using the conventional load mechanism.<\/p>\n
<\/a>Conflicting with another module’s dependencies<\/h3>\nAnother common scenario in PowerShell is that a module is loaded that depends on one version of an assembly, and then another module is loaded later that depends on a different version of that assembly.<\/p>\n
This often looks like the following:<\/p>\n
<\/a><\/p>\nIn the above case, the FictionalTools<\/code> module requires a newer version of Microsoft.Extensions.Logging<\/code> than the FilesystemManager<\/code> module.<\/p>\nLet’s imagine these modules load their dependencies by placing the dependency assemblies in the same directory as the root module assembly and allowing .NET to implicitly load them by name. If we are running PowerShell 7 (on top of .NET Core 3.1), we can load and run FictionalTools<\/code> and then load and run FilesystemManager<\/code> without issue. However, if in a new session we load and run FilesystemManager<\/code> and then FictionalTools<\/code>, we will encounter a FileLoadException<\/code> from the FictionalTools<\/code> command, because it requires a newer version of Microsoft.Extensions.Logging<\/code> than the one loaded, but cannot load it because an assembly of the same name has already been loaded.<\/p>\n<\/a>PowerShell and .NET<\/h2>\nPowerShell runs on the .NET platform, and since we’re discussing assembly dependency conflicts, we must discuss .NET. .NET is ultimately responsible for resolving and loading assembly dependencies, so we must understand how .NET operates here to understand dependency conflicts.<\/p>\n
We must also confront the fact that different versions of PowerShell run on different .NET implementations, which respectively have their own mechanisms for assembly resolution.<\/p>\n
In general, PowerShell 5.1 and below run on .NET Framework, while PowerShell 6 and above run on .NET Core. These two flavours of .NET load and handle assemblies somewhat differently, meaning resolving dependency conflicts can vary depending on the underlying .NET platform.<\/p>\n
<\/a>Assembly Load Contexts<\/h3>\nIn this post, we’ll use the term “Assembly Load Context” (ALC) frequently. An Assembly Load Context is a .NET concept that essentially means a runtime namespace into which assemblies are loaded and within which assemblies’ names are unique. This concept allows assemblies to be uniquely resolved by name in each ALC.<\/p>\n
<\/a>Assembly reference loading in .NET<\/h3>\nThe semantics of assembly loading (whether versions clash, whether that assembly’s dependencies are handled, etc.) depend on both the .NET implementation (.NET Core vs .NET Framework) and the API or .NET mechanism used to load a particular assembly.<\/p>\n
Rather than go into deep detail describing these here, there is a list of links in the Further reading<\/a> section that go into great detail on how .NET assembly loading works in each .NET implementation.<\/p>\nIn this blog post, we’ll refer to the following mechanisms:<\/p>\n
\n- Implicit assembly loading (effectively
Assembly.Load(AssemblyName)<\/code>), when .NET implicitly tries to load an assembly by name from a static assembly reference in .NET code.<\/li>\nAssembly.LoadFrom()<\/code>, a plugin-oriented loading API that will add handlers to resolve dependencies of the loaded DLL (but which may not do so the way we want).<\/li>\nAssembly.LoadFile()<\/code>, a bare-bones loading API intended to load only the assembly asked for with no handling of its dependencies.<\/li>\n<\/ul>\nThe way these APIs work has changed in subtle ways between .NET Core and .NET Framework, so it’s worth reading through the further reading links.<\/p>\n
<\/a>Differences in .NET Framework vs .NET Core<\/h3>\nImportantly, Assembly Load Contexts and other assembly resolution mechanisms have changed between .NET Framework and .NET Core.<\/p>\n
In particular .NET Framework has the following features:<\/p>\n
\n- The Global Assembly Cache, for machine-wide assembly resolution<\/li>\n
- Application Domains, which work like in-process sandboxes for assembly isolation, but also present a serialization layer to contend with<\/li>\n
- A limited assembly load context model, explained in depth here<\/a>, that has a fixed set of assembly load contexts, each with their own behaviour:\n
\n- The default load context, where assemblies are loaded by default<\/li>\n
- The load-from context, for loading assemblies manually at runtime<\/li>\n
- The reflection-only context, for safely loading assemblies to read their metadata without running them<\/li>\n
- The mysterious void that assemblies loaded with
Assembly.LoadFile(string path)<\/code> and Assembly.Load(byte[] asmBytes)<\/code> live in<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n.NET Core (and .NET 5+) has eschewed this complexity for a simpler model:<\/p>\n
\n- No Global Assembly Cache; applications bring all their own dependencies (PowerShell, as the plugin host, complicates this slightly for modules ??? its dependencies in
$PSHOME<\/code> are shared with all modules). This removes an exogenous factor for dependency resolution in applications, making dependency resolution more reproducible.<\/li>\n- Only one Application Domain, and no ability to create new ones. The Application Domain concept lives on purely to be the global state of the .NET process.<\/li>\n
- A new, extensible Assembly Load Context model, where assembly resolution can be effectively namespaced by putting it in a new ALC. .NET Core processes begin with a single default ALC into which all assemblies are loaded (except for those loaded with
Assembly.LoadFile(string)<\/code> and Assembly.Load(byte[])<\/code>), but are free to create and define their own custom ALCs with their own loading logic. When an assembly is loaded, the first ALC it is loaded into is responsible for resolving its dependencies, creating opportunities to create powerful .NET plugin loading mechanisms.<\/li>\n<\/ul>\nIn both implementations, assemblies are loaded lazily when a method requiring their type is run for the first time.<\/p>\n
For example here are two versions of the same code that load a dependency at different times.<\/p>\n
The first will always load its dependency when Program.GetRange()<\/code> is called, because the dependency reference is lexically present within the method:<\/p>\n\nusing<\/span> Dependency<\/span>.Library<\/span>;\r\n\r\npublic<\/span> static<\/span> class<\/span> Program<\/span>\r\n{\r\n public<\/span> static<\/span> List<\/span><int<\/span>> GetRange<\/span>(int<\/span> limit<\/span>)\r\n {\r\n var<\/span> list<\/span> =<\/span> new<\/span> List<\/span><int<\/span>>();\r\n for<\/span> (int<\/span> i<\/span> =<\/span> 0<\/span>; i<\/span> <<\/span> limit<\/span>; i<\/span>++<\/span>)\r\n {\r\n if<\/span> (i<\/span> >=<\/span> 20<\/span>)\r\n {\r\n \/\/<\/span> Dependency.Library will be loaded when GetNumbers is run<\/span>\r\n \/\/<\/span> because the dependency call occurs directly within the method<\/span>\r\n DependencyApi<\/span>.Use<\/span>();\r\n }\r\n\r\n list<\/span>.Add<\/span>(i<\/span>);\r\n }\r\n return<\/span> list<\/span>;\r\n }\r\n}<\/pre>\n<\/div>\nThe second will load its dependency only if the limit<\/code> parameter is 20 or more, because of the internal indirection through a method:<\/p>\n\nusing<\/span> Dependency<\/span>.Library<\/span>;\r\n\r\npublic<\/span> static<\/span> class<\/span> Program<\/span>\r\n{\r\n public<\/span> static<\/span> List<\/span><int<\/span>> GetNumbers<\/span>(int<\/span> limit<\/span>)\r\n {\r\n var<\/span> list<\/span> =<\/span> new<\/span> List<\/span><int<\/span>>();\r\n for<\/span> (int<\/span> i<\/span> =<\/span> 0<\/span>; i<\/span> <<\/span> limit<\/span>; i<\/span>++<\/span>)\r\n {\r\n if<\/span> (i<\/span> >=<\/span> 20<\/span>)\r\n {\r\n \/\/<\/span> Dependency.Library is only referenced within<\/span>\r\n \/\/<\/span> the UseDependencyApi() method,<\/span>\r\n \/\/<\/span> so will only be loaded when limit >= 20<\/span>\r\n UseDependencyApi<\/span>();\r\n }\r\n\r\n list<\/span>.Add<\/span>(i<\/span>);\r\n }\r\n return<\/span> list<\/span>;\r\n }\r\n\r\n private<\/span> static<\/span> void<\/span> UseDependencyApi<\/span>()\r\n {\r\n \/\/<\/span> Once UseDependencyApi() is called, Dependency.Library is loaded<\/span>\r\n DependencyApi<\/span>.Use<\/span>();\r\n }\r\n}<\/pre>\n<\/div>\nThis is a good thing for .NET to do, since it minimizes the memory and filesystem reads it needs to use, making it more resource efficient.<\/p>\n
Unfortunately a side effect of this is that if an assembly load will fail, we won’t necessarily know when the program is first loaded, only when the code path that tries to load the assembly is run.<\/p>\n
It can also set up timing conditions for assembly load conflicts; if two parts of the same program will try to load different versions of the same assembly, which one is loaded depends on which code path is run first.<\/p>\n
For PowerShell this means that the following factors can affect an assembly load conflict:<\/p>\n
\n- Which module was loaded first?<\/li>\n
- Was the cmdlet\/code path that uses the dependency library run?<\/li>\n
- Does PowerShell load a conflicting dependency at startup or only under certain code paths?<\/li>\n<\/ul>\n
<\/a>Quick fixes and their limitations<\/h2>\nIn some cases it’s possible to make small adjustments to your module and fix things with a minimum of fuss. But these solutions tend to come with caveats, so that while they may apply to your module, they won’t work for every module.<\/p>\n
<\/a>Change your dependency version<\/h3>\nThe simplest way to avoid dependency conflicts is to agree on a dependency. This may be possible when:<\/p>\n
\n- Your conflict is with a direct dependency of your module and you control the version<\/li>\n
- Your conflict is with an indirect dependency, but you can configure your direct dependencies to use a workable indirect dependency version<\/li>\n
- You know the conflicting version and can rely on it not changing<\/li>\n<\/ul>\n
A good example of this last scenario is with the Newtonsoft.Json<\/code> package. This is a dependency of PowerShell 6 and above, and isn’t used in Windows PowerShell, meaning a simple way to resolve versioning conflicts is to target the lowest version of Newtonsoft.Json<\/code> across the PowerShell versions you wish to target.<\/p>\nFor example, PowerShell 6.2.6 and PowerShell 7.0.2 both currently use Newtonsoft.Json version 12.0.3. So, to create a module targeting Windows PowerShell, PowerShell 6 and PowerShell 7, you would target Newtonsoft.Json 12.0.3 as a dependency and include it in your built module. When the module is loaded in PowerShell 6 or 7, PowerShell’s own Newtonsoft.Json assembly will already be loaded, but the version will be at least the required one for your module, meaning resolution will succeed. In Windows PowerShell, the assembly will not be already present in PowerShell, and so will be loaded from your module instead.<\/p>\n
Generally, when targeting a concrete PowerShell package, like Microsoft.PowerShell.Sdk or System.Management.Automation, NuGet should be able to resolve the right dependency versions required. It’s only in the case of targeting both Windows PowerShell and PowerShell 6+ that dependency versioning becomes more difficult, either because of needing to target multiple frameworks, or due to targeting PowerShellStandard.Library.<\/p>\n
Circumstances where pinning to a common dependency version won’t work include:<\/p>\n
\n- The conflict is with an indirect dependency, and there’s no configuration of your dependencies that can use the common version<\/li>\n
- The other dependency version is likely to change often, meaning settling on a common version will only be a short-term fix<\/li>\n<\/ul>\n
<\/a>Add an AssemblyResolve event handler to create a dynamic binding redirect<\/h3>\nWhen changing your own dependency version isn’t possible, or is too hard, another way to make your module play nicely with other dependencies is to set up a dynamic binding redirect by registering an event handler for the AssemblyResolve<\/code> event.<\/p>\nAs with a static binding redirect, the point here to is force all consumers of a dependency to use the same actual assembly. This means we need to intercept calls to load the dependency and always redirect them to the version we want.<\/p>\n
This looks something like this:<\/p>\n
\n\/\/<\/span> Register the event handler as early as you can in your code.<\/span>\r\n\/\/<\/span> A good option is to use the IModuleAssemblyInitializer interface<\/span>\r\n\/\/<\/span> that PowerShell provides to run code early on when your module is loaded.<\/span>\r\n\r\n\/\/<\/span> This class will be instantiated on module import and the OnImport() method run.<\/span>\r\n\/\/<\/span> Make sure that:<\/span>\r\n\/\/<\/span> - the class is public<\/span>\r\n\/\/<\/span> - the class has a public, parameterless constructor<\/span>\r\n\/\/<\/span> - the class implements IModuleAssemblyInitializer<\/span>\r\npublic<\/span> class<\/span> MyModuleInitializer<\/span> : IModuleAssemblyInitializer<\/span>\r\n{\r\n public<\/span> void<\/span> OnImport<\/span>()\r\n {\r\n AppDomain<\/span>.CurrentDomain<\/span>.AssemblyResolve<\/span> +=<\/span> DependencyResolution<\/span>.ResolveNewtonsoftJson<\/span>;\r\n }\r\n}\r\n\r\n\/\/<\/span> Clean up the event handler when the the module is removed<\/span>\r\n\/\/<\/span> to prevent memory leaks.<\/span>\r\n\/\/<\/span><\/span>\r\n\/\/<\/span> Like IModuleAssemblyInitializer, IModuleAssemblyCleanup allows<\/span>\r\n\/\/<\/span> you to register code to run when a module is removed (with Remove-Module).<\/span>\r\n\/\/<\/span> Make sure it is also public with a public parameterless contructor<\/span>\r\n\/\/<\/span> and implements IModuleAssemblyCleanup.<\/span>\r\n