Standard C++20 Modules support with MSVC in Visual Studio 2019 version 16.8
Please see our Visual Studio 2019 version 16.8 Preview 3 release notes for more of our latest features.
It has been some time since our last update regarding C++ Modules conformance. The toolset, project system, and IDE teams have been hard at work to create a first class C++ Modules experience in Visual Studio 2019. There is a lot to share, so let’s get right into it:
What’s new?
/std:c++latestImplies C++ Modules.- Private Module fragments are a new form of API encapsulation for primary Module interfaces.
- Include translation allows for easy adoption of header units without changing existing code.
- Module linkage is a new type of linkage enforced by the back-end and linker.
- Project system changes to enable C++ Module scenarios.
- IntelliSense updates.
/std:c++latest Implies C++ Modules
Since MSVC began down the path of implementing the Modules TS the toolset has always required the use of /experimental:module on any compilation. Since the merge of Modules into the C++20 standard (we can officially say C++20 now!) the compiler has been working towards C++20 Modules conformance until precisely such a time that we can confidently roll Modules into /std:c++latest. That time is now!
There are a few caveats to implying C++ Modules under /std:c++latest:
/std:c++latestnow implies/permissive-. This means that customers currently relying on the permissive behavior of the compiler in combination with/std:c++latestare required to now apply/permissiveon the command line. Note: enabling/permissivealso disables the use of Modules.- Now that Modules are rolled into the latest language mode some code is subject to breakage due to
moduleandimportbeing converted into keywords. We have documented some of the common scenarios. The paper MSVC implements in order to convertmoduleandimportinto keywords has even more scenarios: P1857R1. - The
std.*Modules which ship with Visual Studio will not be available through/std:c++latestalone. The standard library Modules have not yet been standardized and as such remain experimental. To continue using the standard library Modules users will need/experimental:moduleas part of their command line options.
Private Module Fragments
C++20 added a new section to a primary Module interface known as the private Module fragment, [module.private.frag]. Private Module fragments allow authors to truly hide details of a library without having to create a separate C++ source file to contain implementation details. Imagine a scenario where a PIMPL pattern is used in a primary Module interface:
module;
#include <memory>
export module m;
struct Impl;
export
class S {
public:
S();
~S();
void do_stuff();
Impl* get() const { return impl.get(); }
private:
std::unique_ptr<Impl> impl;
};
module :private; // Everything beyond this point is not available to importers of 'm'.
struct Impl {
void do_stuff() { }
};
S::S():
impl{ std::make_unique<Impl>() }
{
}
S::~S() { }
void S::do_stuff() {
impl->do_stuff();
}
And on the import side:
import m;
int main() {
S s;
s.do_stuff(); // OK.
s.get(); // OK: pointer to incomplete type.
auto impl = *s.get(); // ill-formed: use of undefined type 'Impl'.
}
The private Module partition is an abstraction barrier shielding the consumer of the containing Module from anything defined in the purview of the private partition, effectively enabling single-“header” libraries with better hygiene, improved encapsulation, and reduced build system administrivia.
Include Translation
With the introduction of header units comes header include translation, [cpp.include]/7 enables the compiler to translate #include directives into import directives provided the header-name designates an importable header (to MSVC, a header unit is made an importable header through the use of /headerUnit). This switch can be enabled through C/C++ -> All Options -> Additional Options and adding /translateInclude. In future releases, users will have the choice of selecting specific headers that should be subject to include translation, instead of an all-or-nothing switch.
Module Linkage
C++ Modules demands more from the toolset beyond simply parsing (front-end). C++20 introduces a new flavor of linkage, “module linkage” [basic.link]/2.2. A proof-of-concept, using only front-end name mangling, implementation of Module linkage developed during the Modules Technical Specification (TS) era has proven to be imperfect, and inefficient at scale. Starting with Visual Studio 2019 16.8, the compiler and linker work together in order to enforce module linkage semantics (without the front-end name mangling workaround). The new linker work means that users can more freely author code using named Modules without being concerned with possible name collision issues while gaining stronger odr guarantees not offered by any other language facility.
Strong Ownership
The MSVC toolset has also adopted a strong ownership model for the purposes of program linkage. The strong ownership model brings certainty and avoids clashes of linkage names by empowering the linker to attach exported entities to their owning Modules. This capability allows MSVC to rule out undefined behavior stemming from linking different Modules (maybe revisions of the same Module) reporting similar declarations of different entities in the same program.
For instance, consider the following example that is formally left undefined behavior (in practical terms):
m.ixx
export module m;
export
int munge(int a, int b) {
return a + b;
}
n.ixx
export module n;
export
int munge(int a, int b) {
return a - b;
}
libM.cpp Also a header file which forward declares libm_munge
import m;
int libm_munge(int a, int b) {
return munge(a, b);
}
main.cpp
#include "libM.h"
import n; // Note: do not import 'm'.
int main() {
if (munge(1, 2) != -1)
return 1;
if (libm_munge(1, 2) != 3) // Note uses Module 'm' version of 'munge'.
return 1;
}
In practice and in general, one wouldn’t write code purposefully like that, but it is hard to avoid in practice under code migration, evolution, and maintenance. Before strong module ownership semantics, a program such as this would not be possible (with two external linkage names of munge). Strong ownership buys this new odr guarantee. There is a great paper “A Module System for C++” which details rationale behind strong ownership.
Project System
Quite possibly the most essential part of using C++ Modules is having a build system which can cope with the requirements of C++ Modules build while providing an experience for users without a steep learning curve. The VC Project System team has been working closely with the compiler toolset team to bring an experience with automatic Modules and header units support minimizing user work to set them up.
The files with .ixx or .cppm extensions are considered “Module interface” source. But ultimately it is controlled by CompileAs property:
If you want to build a header unit for a .h file, you need to change its item type to be “C/C++ compiler” as by default .h files are in “C/C++ header” group and are not passed to the compiler. “C/C++ compiler” files with .h extension are considered “header units” by default.
The project build will automatically scan the Modules and Header Unit files (according to their Compile As setting), for other Module and Header Units dependencies in the same project, and build them in the correct dependency order.
To reference a Module or a header unit produced by another project, just add a reference to that project. All “public” Modules and header units from referenced projects are automatically available for referencing in code.
A project can control which Modules and headers (including the ones built as header units) are considered “public” by modifying the following properties:
This short video gives a brief look of the workflow. The only manual work done was setting the C++ language standard to /std:c++latest.
Compiler Switch Overhaul
The experimental phase of many /module:* prefixed switches is over so we have moved them into a permanent home under a new name:
| Old | New |
|---|---|
/module:interface |
/interface |
/module:internalPartition |
/internalPartition |
/module:reference |
/reference |
/module:search |
/ifcSearchDir |
/module:stdIfcDir |
/stdIfcDir |
/module:output |
/ifcOutput |
/module:ifcOnly |
/ifcOnly |
/module:exportHeader |
/exportHeader |
/module:showResolvedHeader |
/showResolvedHeader |
/module:validateChecksum[-] |
/validateIfcChecksum[-] |
Build systems and users wishing to use the 16.8 toolset should take note of the new changes.
IntelliSense
Visual C++ would not be… visual without IntelliSense. In 16.8 we’re adding full support for using IntelliSense in modules, both for writing Module interfaces (.ixx) and getting IntelliSense from imported Modules and header units. IntelliSense support for imported Modules will not be available in Preview 3 but we plan to enable it in an upcoming Preview. Please stay tuned for our CppCon demo which will feature IntelliSense usage!
The toolset team has worked hard to ensure that the C++ Module format emitted by the compiler is well-documented and stable for use in the IntelliSense engine. It is MSVC which is responsible for building the IFC file which is then used by the IDE. The ability for other tools to consume the MSVC IFC format is crucial to a healthy ecosystem, and the IntelliSense engine using the IFC output is the first step in that direction.
Closing
We urge you to go out and try using Visual Studio 2019 with Modules. 16.8 Preview 3 will be available through the Visual Studio 2019 downloads page!
As always, we welcome your feedback. Feel free to send any comments through e-mail at visualcpp@microsoft.com or through Twitter @visualc. Also, feel free to follow me on Twitter @starfreakclone.
If you encounter other problems with MSVC in VS 2019 please let us know via the Report a Problem option, either from the installer or the Visual Studio IDE itself. For suggestions or bug reports, let us know through DevComm.
Light
Dark
37 comments
Great to see progress on this awesome feature! I had one question, does the spec explicitly say that modules will add strong ownership to symbols? I may have missed it (or read an outdated spec) but I was under the impression that modules do nothing to solve ODR violations for publicly exported symbols. I thought the only improvement in this area was the new module linkage to guarantee unique symbols for private symbols within each module from colliding with other symbols with the same name.
ODR violations are UB – so strong ownership is one valid way of resolving it, no?
Hi Matthew,
Thank you for the response!
On the subject of Modules and their involvement in odr, indeed the current C++20 standard does provide the flexibility to the implementation for supporting the strong ownership model. As noted in the blog above, strong ownership affords more freedom with integrating outside libraries which may have overlapping names across the entire program. There was a great response on the reddit thread from Gaby about this as well.
Ha, that was my thread on Reddit! Thanks for the response.
What about IFC – any plans of publishing its related source codes / interfaces ?
Yes, we are working on the plans for involving the community on: (1) The IFC format specification; (2) open source tooling around it.
Stay tuned 🙂
This is great! Thank you for working on modules!
Will it be possible to have the binary contents of the module interface/IFC/the neccessary bits appended to the dll itself, to ease distribution?
Coming soon, we are working on it. We want to make sure that we open the IFC spec for the entire C++ community to contribute to, and for the Visual C++ team to be able to address and to reflect those feedback into the MSVC toolset. Furthermore, embedding IFC into the exe or dll has been part of the overall C++ Modules vision — see my CppCon 2015 talk. Such embedding surfaces interesting issues with ABI compatibility that need to be addressed.
That’s great news, thanks 🙂
I thought it was clear now that BMI/CMI/IFC files are just glorified PCH, totally internal to the compiler and build process, not meant to be distributed.
Awesome work! I’ve been itching to get started with modules for years, excited to finally be able to do some real programming using them.
I tried to compile a simple project that uses module partitions, but it does not seem to want to compile. It looks almost identical to code from your blog posts so I am not sure what is going on.
I’ve made a post about it here which includes a zipped sample project: https://developercommunity.visualstudio.com/content/problem/1185466/module-with-partition-wont-compile.html
Hi Quincy,
I have confirmed that this problem is not a compiler bug. It is, most likely, a configuration issue on the build system side. I am coordinating with the project system to see where the core hitch is. Stay tuned.
Thanks for the quick response! Ah, I had such a suspicion, I tried messing around with some of the parameters but couldn’t get it to work myself unfortunately.
I will definitely stay tuned.
We have root caused the issue to an odd interaction between the build system and how the compiler reports Module partition dependencies. Luckily this issue will be fixed in Preview 4!
Please let us know if you have any other problems with using Modules in VS.
Fantastic, thanks! Really appreciate the direct communication, looking forward to preview 4 🙂
Hi Cameron, great work.
I can get it to work with manually adding /reference to the partitions it is supposed to import, however, this doesn’t work at all with pure implementation partitions (without export kw) and even on the CLI side it will only work if the extension is not .ixx – is this intended, in other words, will .ixx be reserved for interface partitions?
This is awesome ! I’m facing an issue though so I’d best ask here:
When naming my module my.module and defining its interface in a my.module.ixx file, it does not work correctly (the module is not found when compiling the module internal partitions).
If I name my module mymodule in a mymodule.ixx file it all works correctly though. Is it a bug or is it by design ? The standard allows dotted identifiers last time I checked.
If it’s caused by something in design, what is the correct way to name the interface file of a dotted module identifier so that VS will pick it up properly ?
Hi Hugo,
Just like Quincy’s response above, this is an interaction issue between the build system and the compiler. The problem will be resolved in Preview 4.
Hello ! Thanks for the answer !
I have another, more general question. I want to use modules but I depend on third party code which I cannot modify, which includes headers from the STL in its own headers. Do you now what is or are the correct way(s) to depend on this code from my modules which also imports std modules, if there are any such ways ? So far I only end up trying ways where I bump into multiple definitions issues in the STL (generating a header unit from this third party code, for example).
Are you asking about how to use “include translation” to turn certain `#include` into `import`? Do you have some repro of your scenario somewhere (like GitHub) to share?
I noticed how modules now implies and requires /permissive-
If I want to port a library L to modules, but an application A which is using L will be updated to Visual Studio 2019 but with /permissive (i.e. not compatible with modules) – is it possible to somehow consume a module from L in A or is modularization of a library dependent on all downstream applications/users also not using /permissive mode?
Hi Mats,
Luckily there is still an avenue for combining /permissive and /permissive- with Modules in these “bridge” modes. Historically, we have had instances with programs needing both /permissive and /permissive- in different parts of a program, and the compiler does preserve ABI between the two. With Modules you must compiler with /permissive-, which is why it is now implied by /std:c++latest, but as long as an individual translation unit does not use any Modules feature the /permissive switch can be used to build the TU.
I hope this helps!
Hi,
Thanks, but I don’t quite understand this. What I need is to _use_ modules (i.e. import module) in my application A which is compiled using /permissive (not /permissive-). So what you’re saying is that I need to compile TUs that need to import a module from library L using /permissive- (while other TUs can continue using /permissive)?
Yes, if you are using a module functionality (e.g. import, export) in a TU, you need to use /permissive- for that TU.
ok – thanks 🙂
Great improvment thanks!
I’ve noticed that linker warning:
LINK : warning LNK4301: ignoring ‘/INCREMENTAL’ because of the use of cxx modules
Is it something temporary or is there an issue by design and we’ll never get incremental link with c++ modules ?
Hi Onduril,
Yes, this linker warning is a known issue and one that is fixed with Preview 4. Stay tuned and thank you for trying out the new feature!
Maybe more of a general question, but what about exporting a function which is defined in an internal partition? The standard doesn’t seem to have too much to say about that, except that it allows it in implementation units. If I do it in a partition, the compiler will generate a slightly different symbol for it (so linkage will fail). I’m not sure if this is effectively prohibited or if it’s something that will change so that the compiler will realize that a function is being defined rather than introducing a new one.
Second, is it possible to expose an internal datatype as an opaque pointer with the same name (eg I have a struct MyData internally in a partition and want to let users use the MyData* syntax)? So far from what I’ve tried, it only works if I use another name and export using.
If the definition in the internal partition has no reachable exported declaration, then that definition is effectively a declaration that has module linkage — so, that is a programmer error.
Regarding your second question, an internal partition is never seen by consumers of the module, so it is not clear what you mean by “expose an internal datatype”. If you just want to export the name of the class without its definition, then: (1) export the declaration of the type without its definition; and (2) put the definition in the private module partition or in a separate source file. See also my CppCon 2019 talk: https://www.youtube.com/watch?v=tjSuKOz5HK4
I’m not sure I understand the reachability of an exported declaration – let’s say I have an
in primary module interface and then in an internal partition I define it. This doesn’t seem to work now – it will not link, even if I `import mod;` (no other way to import the primary?) in the partition. So that export declaration is not reachable – I suppose? Strangely, if I make the export declaration in an interface partition and then import this partition into the partition that defines myfun in order to ensure it is reachable, I will get a compiler error that the internal partition does not contribute to the exported interface, despite only defining the function, not introducing an export declaration.
As for the second one, I think I’ve figured it out. I had an interface partition :E that would introduce the export declaration and an internal partition :A that would define it while importing the declaration from the interface partition :E, but if another internal partition was to import both :A and :E, it wouldn’t have access to the definition. If I just effectively introduce a new declaration in the defining partition :A without importing the “forward” exporting declaration from :E, it works. This might be intended, it just struck me as nonintuitive.
Hi,
The strategy of converting project to module aware depends on implementation defined but very important feature, IMHO. Let’s consider the following case – we change the implementation in module definition file but keep the interface untouched and we don’t want to recompile the other files that import it. That allows us to drop source/header hell. Have you any plans to implement the feature?
See short discussion on stackoverflow .
Thanks.
Are there known issues/remaining work with precompiled headers interacting with modules? Given a main.cpp that is “Not Using Precompiled Headers” and includes precomp.h, and precomp.h has “import MyModule;”, it builds fine; but if I set main.cpp “Precompiled header” to “/Use (Yu)” (and corresponding precomp.cpp), I get error C2230: could not find module ‘MyModule’ from inside that same precomp.h. I totally get if there’s an ordering dependency issue here with the build system (because precompiled headers are generated so early), but the error message is bewildering/misleading, because the .ifc file is definitely already generated in the output directory (and imports fine if not within the PCH).