C++17/20 Features and Fixes in Visual Studio 2019
Visual Studio 2019 version 16.0 is now available and is binary compatible with VS 2015/2017. In this first release of VS 2019, we’ve implemented more compiler and library features from the C++20 Working Paper, implemented more <charconv>
overloads (C++17’s “final boss”), and fixed many correctness, performance, and throughput issues. Here’s a list of the C++17/20 compiler/library feature work and the library fixes. (As usual, many compiler bugs were also fixed, but they aren’t listed here; compiler fixes tend to be specific to certain arcane code patterns. We recently blogged about compiler optimization and build throughput improvements in VS 2019, and we maintain a documentation page about compiler conformance improvements in VS 2019.)
New Features:
- Implemented P1164 from C++20 unconditionally. This changes
std::create_directory
to check whether the target was already a directory on failure. Previously, allERROR_ALREADY_EXISTS
type errors were turned into success-but-directory-not-created codes. - The iterator debugging feature has been taught to properly unwrap
std::move_iterator
. For example,std::copy(std::move_iterator<std::vector<int>::iterator>, std::move_iterator<std::vector<int>::iterator>, int*)
can now engage ourmemcpy
fast path. - The standard library’s macroized keyword enforcement
<xkeycheck.h>
was fixed to emit the actual problem keyword detected rather than a generic message, look for C++20 keywords, and avoid tricking IntelliSense into saying random keywords were macros. - We added LWG 2221‘s
operator<<(std::ostream, nullptr_t)
for writingnullptr
s to streams. - Parallel versions of
is_sorted
,is_sorted_until
,is_partitioned
,set_difference
,set_intersection
,is_heap
, andis_heap_until
were implemented by Miya Natsuhara. - P0883 “Fixing atomic initialization”, which changes
std::atomic
to value-initialize the containedT
rather than default-initializing it, was implemented (also by Miya) when using Clang/LLVM with our standard library. This is currently disabled for C1XX as a workaround for a bug in constexpr processing. - Implemented the “spaceship” three-way comparison operator from P0515 “Consistent comparison”, with partial support for the C++20
<compare>
header as specified in P0768 (specifically, the comparison category types andcommon_comparison_category
type trait, but not the comparison algorithms which are undergoing some redesign in WG21). (Implemented by Cameron DaCamara in the compiler.) - Implemented the new C++20 P1008 rules for aggregates: a type with a user-declared constructor – even when defaulted or deleted so as not to be user-provided – is not an aggregate. (Implemented by Andrew Marino in the compiler.)
- Implemented the
remove_cvref
andremove_cvref_t
type traits from P0550, which are handy for stripping reference-ness and cv-qualification but without decaying functions and arrays to pointers (whichstd::decay
andstd::decay_t
do). - C++17
<charconv>
floating-pointto_chars()
has been improved: shortestchars_format::fixed
is 60-80% faster (thanks to Ulf Adams at Google for suggesting long division), and shortest/precisionchars_format::hex
is complete. Further performance improvements for shortest fixed notation have been implemented and will ship in a future VS 2019 update, along with the decimal precision overloads that will complete the<charconv>
implementation. - C++20 P0941 feature-test macros are now completely supported in the compiler and STL, including
__has_cpp_attribute
implemented by Phil Christensen. As a reminder, the feature-test macros are always active (i.e. defined or not defined, depending on the availability of the feature in question) regardless of the Standard mode option selected, because making them conditional on/std:c++latest
would largely defeat their purpose.
Correctness Fixes:
std::allocator<void>
,std::allocator::size_type
, andstd::allocator::difference_type
have been un-deprecated.- A spurious
static_cast
not called for by the standard that accidentally suppressed C4244 narrowing warnings was removed fromstd::string
. Attempting to callstd::string::string(const wchar_t*, const wchar_t*)
will now properly emit C4244 “narrowing awchar_t
into achar
.” - Fixed
std::filesystem::last_write_time
failing when attempting to change a directory’s last write time. std::filesystem::directory_entry
‘s constructor was changed to store a failed result, rather than throwing an exception, when supplied a nonexistent target path.std::filesystem::create_directory
‘s 2-parameter version was changed to call the 1-parameter version, as the underlyingCreateDirectoryExW
function would performcopy_symlink
when theexisting_p
was a symlink.std::filesystem::directory_iterator
no longer fails when encountering a broken symlink.std::filesystem::space now accepts
relative paths.std::filesystem::path::lexically_relative
is no longer confused by trailing slashes, reported as LWG 3096.- Worked around
CreateSymbolicLinkW
rejecting paths with forward slashes instd::filesystem::create_symlink
. - Worked around the POSIX deletion mode delete function existing on Windows 10 LTSB 1609 but not actually being capable of deleting files.
std::boyer_moore_searcher
andstd::boyer_moore_horspool_searcher
‘s copy constructors and copy assignment operators now actually copy things.- The parallel algorithms library now properly uses the real
WaitOnAddress
family on Windows 8 and later, rather than always using the Windows 7 and earlier fake versions. std::system_category::message()
now trims trailing whitespace from the returned message.- Some conditions that would cause
std::linear_congruential_engine
to trigger divide by 0 have been fixed. - The iterator unwrapping machinery we first exposed for programmer-user integration in VS 2017 15.8 (as described in https://devblogs.microsoft.com/cppblog/stl-features-and-fixes-in-vs-2017-15-8/ ) no longer unwraps iterators derived from standard library iterators. For example, a user that derives from
std::vector<int>::iterator
and tries to customize behavior now gets their customized behavior when calling standard library algorithms, rather than the behavior of a pointer. - The unordered container reserve function now actually reserves for N elements, as described in LWG 2156.
- Many STL internal container functions have been made private for an improved IntelliSense experience. Additional fixes to mark members as private are expected in subsequent releases of MSVC.
- Times passed to the concurrency library that would overflow (e.g.
condition_variable::wait_for(seconds::max())
) are now properly dealt with instead of causing overflows that changed behavior on a seemingly random 29-day cycle (when uint32_t milliseconds accepted by underlying Win32 APIs overflowed). - Exception safety correctness problems wherein the node-based containers like
list
,map
, andunordered_map
would become corrupted were fixed. During apropagate_on_container_copy_assignment
orpropagate_on_container_move_assignment
reassignment operation, we would free the container’s sentinel node with the old allocator, do the POCCA/POCMA assignment over the old allocator, and then try to acquire the sentinel node from the new allocator. If this allocation failed, the container is corrupted and can’t even be destroyed, as owning a sentinel node is a hard data structure invariant. This was fixed to allocate the new sentinel node from the source container’s allocator before destroying the existing sentinel node. - The containers were fixed to always copy/move/swap allocators according to
propagate_on_container_copy_assignment
,propagate_on_container_move_assignment
, andpropagate_on_container_swap
, even for allocators declaredis_always_equal
. std::basic_istream::read
was fixed to not write into parts of the supplied buffer temporarily as part of \r\n => \n processing. This gives up some of the performance advantage we gained in VS 2017 15.8 for reads larger than 4k in size, but efficiency improvements from avoiding 3 virtual calls per character are still present.std::bitset
‘s constructor no longer reads the ones and zeroes in reverse order for large bitsets.- When implementing P0083 “Splicing Maps And Sets”, we managed to overlook the fact that the
merge
andextract
members of the associative containers should have overloads that accept rvalue containers in addition to the overloads that accept lvalue containers. We’ve rectified this oversight by implementing the rvalue overloads. - With the advent of the Just My Code stepping feature, we no longer need to provide bespoke machinery for
std::function
andstd::visit
to achieve the same effect. Removing that machinery largely has no user-visible effects, except that the compiler will no longer produce diagnostics that indicate issues on line 15732480 or 16707566 of<type_traits>
or<variant>
. - The
<ctime>
header now correctly declarestimespec
andtimespec_get
in namespacestd
in addition to declaring them in the global namespace. - We’ve fixed a regression in
pair
‘s assignment operator introduced when implementing LWG 2729 “Missing SFINAE onstd::pair::operator=
“; it now correctly accepts types convertible to pair again. - Fixed a minor type traits bug, where add_const_t etc. is supposed to be a non-deduced context (i.e. it needs to be an alias for
typename add_const<T>::type
, notconst T
).
Header Inclusion Restructuring:
The standard library’s physical design was substantially overhauled to avoid including headers when they are not necessary. A large number of customers want to use standard library containers but don’t want to use the iostreams and locales. However, the C++ standard library has a circular dependency among components:
- Systems that depend on
<locale>
facets want to usestd::string
as part of their underlying implementations. std::string
wants a stream insertion operator, which depends onstd::ostream
, which depends on<locale>
.
- Historically our standard library worked around this problem by introducing a lower level header
<xstring>
, which definedstd::string
, but none of the other contents of<string>
.<xstring>
would be included by both<locale>
components, and by<string>
, restoring the directed dependency graph. However, this approach had a number of problems: #include <string>
needed to drag in all of the iostreams machinery to provide the stream insertion operator, even though most translation units wouldn’t use the stream insertion operator.- If someone included only
<ostream>
they gotstd::basic_string
andstd::ostream
, but they did not getstd::basic_string
‘s stream insertion operator, thestd::string
typedef, or the string literals. Customers found this extremely confusing. For example, if one tried to stream insert astd::basic_string
after including only<ostream>
, the compiler would print an incredibly long diagnostic sayingoperator<<
couldn’t be found, listing 26 unrelated overloads. Also, attempts to usestd::string_literals
,std::to_string
, or other<string>
components, would fail, which is confusing whenstd::basic_string
was otherwise available.
In VS 2019, we resolve the circular reference completely differently. The stream insertion operator now finds the necessary ostream
components using argument-dependent lookup, allowing us to provide it in the same place as string. This restores appropriate layering (of std::string
below <locale>
components), and makes it possible to use <string>
without dragging in the entire mass of iostreams machinery.
If you have lots of .cpp
files that include string
and do something simple, for example:
#include <stdio.h> #include <string> void f(const std::string& s) { puts(s.c_str()); }
In VS 2017 15.9 this program takes 244 milliseconds to compile on a 7980XE test machine (average of 5 runs), while in VS 2019 16.0 it takes only 178 milliseconds (or about 73% of the time).
Moreover, seemingly unrelated headers like <vector>
were pulled into this mess. For example, vector
wants to throw std::out_of_range
, which derives from std::runtime_error
, which has a constructor that takes a std::string
. We already had out-of-line functions for all throw sites, so the spurious include of <stdexcept>
in <vector>
was unnecessary and has been removed. The following program used to take 177 milliseconds to compile in VS 2017 15.9, but now only needs 151 milliseconds (85% of the time):
#include <vector> void f(std::vector<int>& v) { v.push_back(42); }
The one downside of this change is that several programs that were getting away with not including the correct headers may need to add #includes
. If you were saying std::out_of_range
before, you may need to #include <stdexcept>
. If you were using a stream insertion operator, you may now need to #include <ostream>
. This way, only translation units actually using <stdexcept>
or <ostream>
components pay the throughput cost to compile them.
Performance and Throughput Improvements:
if constexpr
was applied in more places in the standard library for improved throughput and reduced code size in thecopy
family, permutations likereverse
androtate
, and in the parallel algorithms library.- The STL now internally uses
if constexpr
to reduce compile times even in C++14 mode. - The runtime dynamic linking detection for the parallel algorithms library no longer uses an entire page to store the function pointer array, as marking this memory read-only was deemed no longer relevant for security purposes.
std::thread
‘s constructor no longer waits for the thread to start, and no longer inserts so many layers of function calls between the underlying C library_beginthreadex
and the supplied callable object. Previouslystd::thread
put 6 functions between_beginthreadex
and the supplied callable object, which has been reduced to only 3 (2 of which are juststd::invoke
). This also resolves an obscure timing bug wherestd::thread
‘s constructor would hang if the system clock changed at the exact moment astd::thread
was being created.- Fixed a performance regression in
std::hash
that we introduced when implementingstd::hash<std::filesystem::path>
. - Several places the standard library used to achieve correctness with catch blocks now use destructors instead. This results in better debugger interaction — exceptions you throw through the standard library in the affected locations will now show up as being thrown from their original throw site, rather than our rethrow. Not all standard library catch blocks have been eliminated; we expect the number of catch blocks to be reduced in subsequent releases of MSVC.
- Suboptimal codegen in
std::bitset
caused by a conditional throw inside anoexcept
function was fixed by factoring out the throwing path. - The
std::list
andstd::unordered_meow
family use non-debugging iterators internally in more places. - Several
std::list
members were changed to reuse list nodes where possible rather than deallocating and reallocating them. For example, given alist<int>
that already has a size of 3, a call toassign(4, 1729)
will now overwrite the ints in the first 3 list nodes, and allocate one new list node with the value 1729, rather than deallocating all 3 list nodes and then allocating 4 new list nodes with the value 1729. - All locations the standard library was calling
erase(begin(), end())
were changed to callclear()
instead. std::vector
now initializes and erases elements more efficiently in certain cases.<variant>
has been refactored to make it more optimizer-friendly, resulting in smaller and faster generated code. Most notably,std::visit
and the inliner have now become good friends.- We’ve applied clang-format to the STL’s headers for improved readability. (There were additional manual changes, e.g. adding braces to all control flow.)
Reporting Bugs:
Please let us know what you think about VS 2019. You can report bugs via the IDE’s Report A Problem and also via the web, at the Developer Community’s C++ tab.
Billy O’Neal, Casey Carter, and Stephan T. Lavavej
32 comments
When will start more C++20 features to pop in? Like ranges, concepts, and especially this thing here:{ auto fsTime = std::filesystem::last_write_time(filename); return decltype(fsTime)::clock::to_time_t(fsTime); }
Calendars and Time Zones is near the top of the list precisely because folks have explicitly asked to move it to the top, but no firm ETA at this time, sorry. It’s probably an even split between that feature and ranges for “biggest and most expensive library feature to implement” voted into C++20.
One point to correct here though, C++20 does not add a to_time_t member to file_time_clock. What it does add is clock_cast, which lets you easily convert from any clock type back into system_clock, which has the to_time_t member.
I got a bad feeling about concepts… Hoping I may be wrong.
Does std::unordered_meow still randomly push items off its table?
I’m unfamilar with unordered_meow randomly doing anything (other than the usual probilistic nature of hashes). Have you filed a repro on Developer Community?
Developer Community does not seem to provide a suitable input method for a repro in this case. What is the appropriate address to send a sample feline?
Sorry, you had me terrified I had broken unordered_Xxx in the shipping product for a moment. There was a period where I broke the heck out of them after I implemented LWG 2156 because we didn’t do as good a job internally documenting the container’s invariants as we should have.
Can you say what the impact of binary compatability for Visual Studio 2019 has? What features and improvements (performance or conformance) are waiting for the next binary incompatible release?
We’ve figured out how to deliver almost all new features in a binary-compatible manner (including Filesystem and other things that need separately compiled code), but some major improvements to existing features are blocked. In a separate branch (that we need to migrate from Team Foundation Version Control to git), we have:
* A rewrite of the multithreading library, fixing many conformance and performance bugs
* A rewrite of iterator debugging machinery throughout the STL, avoiding dynamic memory allocation and quadratic wall-clock complexity (almost complete; deque and vector<bool> remain to be overhauled)
* Removals of old machinery (hash_map/hash_set, tr1, broken and unused floating-point conversion code, /clr:pure support, more in the future)
We’ll be able to further improve the multithreading machinery if we can simultaneously drop support for XP and Vista targeting (not yet decided; we will support Win7 targeting for a very very long time, and being able to assume Win7+ will allow us to unconditionally use the OS’s condition variable support).
There are also many potential compiler improvements that would break ABI, including universal support for the empty base class optimization, potential improvements to RTTI size, fixing name mangling issues, etc.
To be clear: the entire VS 2019 release series (RTW/16.0 and all future 16.x updates) will contain the binary-compatible 19.2x toolset that can be freely mixed with VS 2015/2017. Any binary-incompatible toolset work will appear in a different major version of the IDE, or as an opt-in additional toolset (no specific timeframe planned right now) – you will not be switched over automatically.
So I can compile a static library with vs2019 and deliver to someone who will link with other code compiled by vs2017 and that will work? Any other steps required?
Yes, it will work, with the following restrictions: (1) Link-Time Code-Gen (LTCG, controlled by the /GL compiler option) is incompatible with mixing-and-matching compiler versions, and (2) the linker that performs the final link of object files and static libraries into the output executable can’t be older than the compiler versions used to generate those object files and static libraries. That is, if your static library is built with VS 2017 15.0, and it’s linked into an application that’s otherwise built with VS 2019 16.0, and the final linker is 16.0, that’s fine. Also fine if the static library is compiled with 16.0 and the application is compiled with 15.0, but the final linker is 16.0. However, if your static library is 16.0 and the application is both compiled and linked with 15.0, that is not supported (and will fail to link in certain scenarios). This binary compatibility is mostly meant for applications to be able to upgrade their toolsets without simultaneously upgrading all compiled dependencies, but it isn’t symmetric for third-party library authors. (Note that even though some mixing-and-matching is supported, it is still best for the whole program to be built with the newest version when possible, as that will give you the most correctness and performance fixes.)
This is documented at https://docs.microsoft.com/en-us/cpp/porting/binary-compat-2015-2017?view=vs-2019 (the behavior is the same for 2015/2017/2019).
I really wish you keep the IDE for longer (4-5-6 years) and to update the toolsets. One problem I experienced is that third party extensions don’t get updated as often or at all and as a resault if we cannot upgrade to a newer VS IDE for the life of the project (which may span a decade).
You should really include the binary incompatible toolset before the next major release of VS. There are many projects that can migrate without considering backward compatibility.
I seem to recall statements in the past from the VS dev team (probably in posts or comments on the Visual C++ blog) that said “we cant do xyz in VS 2017 because it is ABI breaking but the next version will be one where we can break the ABI” or something similar. What changed and why did the VS dev team not go ahead with the original plan to make the ABI breaking changes in VS 2019 in the way the dev team had originally indicated was the plan?
Short explanation: the plan wasn’t that simple (and we clearly miscommunicated the subtleties), we had to put this work on the back burner for several reasons, and C++17/20 work has been higher priority.
The plan: we initially envisioned shipping the binary-incompatible branch (“vNext”, aka “WCFB02”) as an opt-in preview alongside the compatible toolset. I’ve personally tried to make this clear by talking about the “next binary-incompatible major version”, but it’s really easy to read that as “the next major version”.
The back burner: We had been spending a fair amount of time on the vNext branch (Billy overhauled atomics/threading, Steve overhauled iostreams and ABI stability, I mostly overhauled iterator debugging), but then other work items took priority (e.g. non-STL library work, the infinite expanse of C++17 charconv, etc.) We also switched from TFS to Git for source control, which has been a massive productivity boon, but also stranded our vNext changes in TFS (extensive manual work will be required to port them to Git, not because of TFS/Git themselves, but because we’ve significantly changed the underlying code).
C++17/20: We’ve gotten better at delivering features in a binary-compatible way, even when they have separately compiled components. The Committee has been relentlessly developing new features, and by spending dev time on keeping up with the pace of Standardization, we can deliver immediate value to customers. This comes at the cost of working on vNext (which is long-range work without an immediate payoff).
While binary compatibility has proven to be extremely valuable to our customers, management is definitely hearing increased customer interest in the significant improvements that require a bincompat break, so the vNext work hasn’t been cancelled in any way – it will simply take longer to arrive than we initially thought.
> P0883 “Fixing atomic initialization”, which changes
std::atomic
to value-initialize the containedT
rather than default-initializing it, was implemented (also by Miya) when using Clang/LLVM with our standard library. This is currently disabled for C1XX as a workaround for a bug in constexpr processing.This is about statics constant initialization by constexpr constructors bug, right? Do you know when it will be fixed? There is not even a comment on the corresponding ticket tha was open more than a half year ago, and `boost::atomic` also suffering because of it.
Rest assured that your standard library implementers are more frustrated than anyone by this bug 🙂
Just a tiniest correction: “Windows 10 LTSB 1609” doesn’t really exist. It’s either 1607, or LTSB 2016 😉
Just curious: Why require braces for all control flow?
It prevents bugs from creeping in during maintenance.
Will there be a concepts enabled standard library? Will it come as a preview first or it will only come as a final product?
We’re going to implement the C++20 Ranges library (designed by Eric Niebler at Facebook and Casey Carter on the Visual C++ Libraries team), which is powered by concepts (and will be the first consumer of the compiler’s concepts implementation). C++20 is nearing completion, so I don’t expect major new concept-powered machinery to appear beyond the <concepts> and <ranges> headers, but C++23 and beyond will certainly be adding more.
When we implement features from the C++ Working Paper, we generally ship them in a complete form, at production quality, but with the understanding that we may need to make source breaking and even binary breaking changes if they evolve before final Standardization (that is, our binary compatibility is “frozen” when a Standard ships, but not for /std:c++latest features). That said, sometimes we do ship partial implementations, as is happening with char8_t and spaceship.
I guess I was wrong. I was under the impression after Concepts are voted in there would be also a concepts implementation of the C++ containers. So there aren’t any such proposals/TSs or proof-of-concept implementations beyound <ranges>.
Is there any news on the preprocessor? There hasn’t been any news since last July.
No news at this time. (I recently asked the compiler front-end dev lead about /experimental:preprocessor.)
Hey there STL, I’m using Visual Studio Enterprise 2019 v16.1.0, Visual C++ 2019 – 00435-60000-00000-AA453, and CMake reports:
[CMake] — The CXX compiler identification is MSVC 19.21.27702.2
here’s the contents of the file to reproduce it. It seems to be some interaction between std::variant and a templatised function. Note the template parameter is note even used in the template function but making it non templatised stops the ICE. I removed main for brevity because the ICE happens before link time so it will still fail:
— file below —
// [CMake] — The CXX compiler identification is MSVC 19.21.27702.2#include <variant>inline void f(int i){}template <typename T>void g_templatised(){std::variant<int> v;f(std::get<int>(v));}
/*Produces:
C:\Users\JessePepper\source\repos\mi-geometry-framework\miAutomaticUVMapping\main.cpp(9): fatal error C1001: An internal error has occurred in the compiler.(compiler file ‘d:\agent\_work\2\s\src\vctools\Compiler\CxxFE\sl\p1\c\convert.cpp’, line 674)To work around this problem, try simplifying or changing the program near the locations listed above.Please choose the Technical Support command on the Visual C++Help menu, or open the Technical Support help file for more information
*/
— end of file —
By the way it would be nice to be able to have a bit of backtick code markups in here.
Hi Jesse, can you report that ICE through Developer Community, so that the compiler team can follow up with you?
Thanks… wasn’t being lazy just didn’t really know where to submit it.
No problem. You can also report bugs through the IDE (Report A Problem).
Hallo ich möchte für meine Schule ein Programm schreiben hat jemand so was vielleicht schonmal gemacht und kann mir Tipps geben ich würde mich über antworten freuen.