I’m Stephan T. Lavavej, and for the last six and a half years I’ve been working with Dinkumware to maintain the C++ Standard Library implementation in Visual C++. It’s been a while since my last VCBlog post, because getting our latest batch of changes ready to ship has kept me very busy, but now I have time to write up what we’ve done!
If you missed the announcement, you can download 2013 Preview right now!
Note: In this post, whenever I say “2013” without qualification, I mean “2013 Preview and RTM”. I will explicitly mention when things will appear between Preview and RTM.
Quick Summary
The STL in Visual C++ 2013 features improved C++11 conformance, including initializer lists and variadic templates, with faster compile times. We’ve also implemented features from the upcoming C++14 Standard, including make_unique and the transparent operator functors.
Compiler Features
As a reminder, here are the C++11 Core Language features that have been added to the compiler in Visual C++ 2013 Preview (in addition to the features in Visual C++ 2012, of course):
* Default template arguments for function templates
* Delegating constructors
* Explicit conversion operators
* Initializer lists and uniform initialization
* Raw string literals
* Variadic templates
They were available in the “Visual C++ Compiler November 2012 Community Technology Preview” (and they were covered in my video walkthrough), but the compiler team has made lots of progress since then. That is, tons and tons and tons of bugs have been fixed – many of which were reported by users of the Nov CTP (thanks!).
As Herb Sutter just announced in “The Future Of C++” at the Build conference (with a companion blog post), more C++11 Core Language features will be implemented in 2013 RTM:
* Alias templates
* Defaulted functions (except for rvalue references v3)
* Deleted functions
* Non-static data member initializers (NSDMIs)
(About that caveat: “rvalue references v3” is how I refer to C++11’s rules for automatically generating move constructors and move assignment operators, plus the rules for suppressing automatically generated copies when moves are declared. There simply isn’t enough time between 2013 Preview and RTM for the compiler team to implement this with RTM-level quality. As a consequence, requesting memberwise move constructors and move assignment operators with =default will not be supported. The compiler team is acutely aware of how important this stuff is, including to the STL, and it is one of their highest priorities for post-2013-RTM.)
Additionally, some C99 Core Language features will be implemented in 2013 RTM:
* C99 _Bool
* C99 compound literals
* C99 designated initializers
* C99 variable declarations
Please see Herb’s announcement video/post for more information, especially a post-2013-RTM C++11/14 conformance roadmap, including (among many other things) rvalue references v3, constexpr, and C++14 generic lambdas.
STL Features
The STL in Visual C++ 2013 Preview has been fully converted over to using the following C++11 features:
* Explicit conversion operators
* Initializer lists
* Scoped enums
* Variadic templates
In 2013 RTM, this list will be extended to:
* Alias templates
* Deleted functions
I say “converted over” because the STL has been faking (I suppose “simulating” would be a more dignified word) some of these features with library-only workarounds, with varying degrees of success, ever since Visual C++ 2008 SP1. Going into further detail:
* In the Core Language, explicit conversion operators are a general feature – for example, you can have explicit operator MyClass(). However, the Standard Library currently uses only one form: explicit operator bool(), which makes classes safely boolean-testable. (Plain “operator bool()” is notoriously dangerous.) Previously, we simulated explicit operator bool() with operator pointer-to-member(), which led to various headaches and slight inefficiencies. Now, this “fake bool” workaround has been completely removed.
* We didn’t attempt to simulate initializer lists, although I accidentally allowed a broken/nonfunctional <initializer_list> header to ship in Visual C++ 2010, confusing users who noticed its presence. It was removed in Visual C++ 2012 to avoid confusion. Now that the compiler supports initializer lists, <initializer_list> is back, it actually works, and we’ve implemented all of the std::initializer_list constructors and other functions mandated throughout the Standard Library.
* Scoped enums were implemented in the Visual C++ 2012 compiler, but due to a long story involving a compiler bug with /clr, we weren’t able to use them in the STL for that release. In Visual C++ 2013, we were able to get rid of our fake scoped enums (simulated by wrapping traditional unscoped enums in namespaces, which is observably imperfect).
* Over the years, we’ve simulated variadic templates with two different systems of “faux variadic” preprocessor macros – the first system involved repeatedly including subheaders, while the second system (more elegant, as far as crawling horrors go) eliminated the subheaders and replaced them with big backslash-continued macros that were stamped out by other macros. Functions that were supposed to be true variadic, like make_shared<T>(args…), were actually implemented with overloads: make_shared<T>(), make_shared<T>(arg0), make_shared<T>(arg0, arg1), etc. Classes that were supposed to be true variadic, like tuple<Types…>, were actually implemented with default template arguments and partial specializations. This allowed us to bring you make_shared/tuple/etc. years ago, but it had lots of problems. The macros were very difficult to maintain, making it hard to find and fix bugs in the affected code. Spamming out so many overloads and specializations increased compile times, and degraded Intellisense. Finally, there was the infinity problem. We originally stamped out overloads/specializations for 0 to 10 arguments inclusive, but as the amount of variadic machinery increased from TR1 through C++0x’s evolution to C++11’s final form, we lowered infinity from 10 to 5 in Visual C++ 2012 in order to improve compile times for most users (we provided a way to request the old limit of 10 through a macro, _VARIADIC_MAX).
In Visual C++ 2013, we have thoroughly eradicated the faux variadic macros. If you want tuples with 50 types, you can have them now. Furthermore, I am pleased to announce that switching the STL over to real variadic templates has improved compile times and reduced compiler memory consumption. What usually happens during the STL’s development is that the Standard tells us to implement more stuff, so we do, and more stuff takes more time and more memory to compile (which can be mitigated by proper use of precompiled headers). Then the compiler’s testers notice the reduced performance and complain, and I respond with a stony face and a clear conscience, because the Standard told me to. But in this case, faux variadics were so incredibly bloated that removing them made a big observable difference. (Parsing one variadic template isn’t free, but it’s a lot cheaper than parsing 6 or whatever overloads.)
The precise numbers will change as the compiler, the STL, and its dependencies (notably the CRT and Concurrency Runtime) are modified, but when I checked in the final variadic rewrite, I ran some tests and I can share the concrete numbers. My test included all STL headers in x86 release mode. I didn’t actually use anything, but the compiler still has to do a lot of work just to parse everything and also instantiate what the STL uses from itself, so this is a reasonable way to measure the overhead of just dragging in the STL without using it extensively. Comparing Visual C++ 2012 (which defaulted to _VARIADIC_MAX=5) to Visual C++ 2013, preprocessed file size decreased from 3.90 MB to 3.13 MB. Compile time decreased from 2307 ms to 2029 ms. (I measured only the compiler front-end; the back-end took 150-250 ms without optimizations and it isn’t of interest here. Additionally, for complicated reasons that I’ll avoid going into here, I excluded the relatively brief time needed to generate a preprocessed translation unit – but as you can see, counting that time would just increase the differences further.) Finally, I measured compiler memory consumption by generating a precompiled header (a PCH is a compiler memory snapshot), which also decreased from 40.2 MB to 33.3 MB. So, the STL is smaller and faster to compile according to every metric. If you had to increase _VARIADIC_MAX to get your programs to compile with Visual C++ 2012, the improvements will be even more dramatic (for completeness, I measured _VARIADIC_MAX=10 as 6.07 MB, 3325 ms, and 58.5 MB respectively).
(Note: After I performed these measurements at the end of March 2013, the compiler team has significantly reworked some of their internal data structures. As a result, compiler memory consumption in Preview and especially RTM may significantly differ from my original measurements. However, that doesn’t affect my conclusion: considered in isolation, using real variadic templates improves every metric.)
* The Standard requires a few things in the STL to be alias templates. For example, ratio_add<ratio<7, 6>, ratio<11, 10>> is required to be an alias for ratio<34, 15>. Previously, we simulated alias templates with structs containing “type” typedefs (sometimes named “other”). This workaround will be completely eliminated in 2013 RTM.
* The Standard declares over 100 deleted functions throughout the STL, usually to make classes noncopyable. Previously, we simulated this with the traditional C++98/03 workaround: private unimplemented declarations. In 2013 RTM, we will eliminate this workaround, marking free functions as =delete and making member functions public with =delete as mandated by the Standard (except type_info, for horrible reasons which this margin is too narrow to contain).
You may have noticed above that there are some features in the compiler list that aren’t in the library list. The reason is simple: not all Core Language features have an observable impact on the Standard Library. For example, raw string literals don’t require corresponding Standard Library implementation changes (although they make <regex> more convenient to use). Similarly, uniform initialization (excluding initializer lists) and NSDMIs don’t affect the STL. You can say pair<int, int> p{11, 22}, but that doesn’t involve any new or changed machinery on our side. As for defaulted functions, they usually appear in the STL for moves, which (as I explained earlier) is not yet supported by the compiler. There are a few defaulted functions other than moves, we just haven’t updated them yet because implementing them “normally” has few adverse consequences. Finally, delegating constructors and default template arguments for function templates don’t observably affect the Standard Library’s interface, although we’re using them internally. (Interestingly, their internal use is actually necessary – delegating constructors are the only way to implement pair’s piecewise constructor, and default template arguments for function templates are the only way to constrain tuple’s variadic constructor.)
You may also have noticed that I mentioned C++14 in the title. That’s because we’ve implemented the following Standard Library features that have been voted into the C++14 Working Paper:
* The “transparent operator functors” less<>, greater<>, plus<>, multiplies<>, etc.
* make_unique<T>(args…) and make_unique<T[]>(n)
* cbegin()/cend(), rbegin()/rend(), and crbegin()/crend() non-member functions
* The <type_traits> alias templates make_unsigned_t, decay_t, etc. (implemented in RTM but not Preview)
(Please note that there is an FAQ at the bottom of this post, and “Q1: Why are you implementing C++14 Standard Library features when you haven’t finished the C++11 Core Language yet?” is answered there.)
For more information, you can read my proposals N3421 “Making Operator Functors greater<>” (Standardese voted in), N3588 “make_unique” (background), N3656 “make_unique (Revision 1)” (Standardese voted in), and my proposed resolution for Library Issue 2128 “Absence of global functions cbegin/cend” which was accepted along with the rest of N3673 “C++ Library Working Group Ready Issues Bristol 2013”. (A micro-feature guaranteeing that hash<Enum> works was also voted in, but it was implemented back in Visual C++ 2012.)
Walter E. Brown (who is unaffiliated with Microsoft) proposed the <type_traits> alias templates in N3546 “TransformationTraits Redux”, which is unquestionably the best feature voted into C++14. Anyone who dares to question this assertion is hereby sentenced to five years of hard labor writing typename decay<T>::type instead of decay_t<T>.
Additionally, James McNellis (who now maintains our CRT) used the transparent operator functors to reduce <algorithm>’s size by 25%. For example, sort(first, last) and sort(first, last, comp) previously had nearly-duplicate implementations, differing only in how they compared elements. Now, sort(first, last) just calls sort(first, last, less<>()). This cleanup was later extended to algorithms implemented in various internal headers, plus the ones lurking in <numeric> and the member algorithms in <list> and <forward_list>. (A small part of the compile time improvement can probably be attributed to this.) James also contributed the fine-grained container requirements overhaul and several bugfixes mentioned below – thanks James!
Fixes
That’s it for the features. Now, for the fixes. This time around, I kept track of all of the STL fixes we made between Visual C++ 2012 and 2013. This includes both the bugs that were reported through Microsoft Connect, and the bugs that were reported internally (whether filed by myself when I notice something wrong, by our testers, or by other Microsoft teams). My list should be exhaustive, with a few exceptions: I’m not counting bugs filed against our tests when they didn’t involve broken library code, I’ve tried to omit the (happily, few) bugs that were introduced and then fixed after 2012 shipped, and I’m not counting bugs that were reported against the libraries but were actually resolved in the compiler (e.g. a few type traits bugs were actually bugs in the compiler hooks we depend on). Also, only formally filed bugs are listed here. Sometimes we notice and fix bugs that were never filed in our database (e.g. I noticed alignment_of behaving incorrectly for references and fixed that, but nobody had ever reported it). I’m including our internal bug numbers (e.g. DevDiv#1729) in addition to Connect bug numbers when available so if anyone asks me for further details, I can easily look up the bug.
There were a few major overhauls:
* <atomic>, whose entire purpose in life is blazing speed, contained unnecessarily slow implementations for many functions. Wenlei He from our compiler back-end (code generation) team contributed a major rewrite of <atomic>’s implementation. Our performance on all architectures (x86/x64/ARM) should now be optimal, or very close to it. (DevDiv#517261/Connect#770885)
* <type_traits> was significantly reworked in conjunction with compiler changes. (Some type traits, like is_array, can be implemented with ordinary C++ code. Other type traits, like is_constructible, must rely on “compiler hooks” for accurate answers.) This fixed virtually all known type traits bugs, including is_pod, is_assignable, and other type traits behaving incorrectly for void (DevDiv#387795/Connect#733088, DevDiv#424157), is_scalar behaving incorrectly for nullptr_t (DevDiv#417110/Connect#740872), result_of not working with movable-only arguments (DevDiv#459824), the is_constructible family of type traits behaving incorrectly with references (DevDiv#517460), many type traits being incorrectly implemented in terms of other type traits (DevDiv#520660; for example, is_move_assignable<T> is defined as is_assignable<T&, T&&>, but our implementation wasn’t doing exactly that), aligned_union returning insufficiently large types for storage (DevDiv#645232/Connect#781713), and alignment_of emitting spurious warnings for classes with inaccessible destructors (DevDiv#688225/Connect#786574 – fixed in RTM but not Preview).
* In C++98/03, life was easy. STL containers required their elements to be copy-constructible and copy-assignable, unconditionally (C++03 23.1 [lib.container.requirements]/3). In C++11, as movable-only types like unique_ptr and additional member functions like emplace_back() were introduced, the container requirements were rewritten to be fine-grained. For example, if you have a list<T> and you populate it with emplace_back(a, b, c), then T needs to be destructible (obviously) and constructible from (A, B, C) but it doesn’t need to support anything else. This is basically awesome for users, but it requires lots of attention to detail from implementers. James fixed many bugs while overhauling this, but the ones that were formally reported were vector<DefaultConstructible>(10) not compiling (DevDiv#437519), vector<T>(first, last)’s requirements being too strict (DevDiv#437541), container move constructors requiring too much from their elements (DevDiv#566619/Connect#775715), and map/unordered_map<K, V> op[] requiring V to be more than default-constructible (DevDiv#577613/Connect#776500).
* 2013 RTM will contain a <ratio> overhaul, implementing alias templates (DevDiv#693707/Connect#786967) and thoroughly fixing numerous bugs (DevDiv#704582/Connect#788745 was just the tip of the iceberg).
Then there were individual bugfixes:
* std::find()’s implementation has an optimization to call memchr() when possible, but find() was simultaneously calling memchr() too aggressively leading to incorrect results (DevDiv#316853/Connect#703165), and not calling it aggressively enough leading to suboptimal performance (DevDiv#468500/Connect#757385). We rewrote this optimization to be correct and optimal in all cases.
* Operators for comparing shared_ptr/unique_ptr to nullptr_t were missing (DevDiv#328276). Some comparisons compiled anyways due to the construction of temporaries, but the Standard mandates the existence of the operators, and this is observable if you look hard enough. Therefore, we added the operators.
* shared_future was convertible to/from future in ways prohibited by the Standard (DevDiv#361611). We prevented such conversions from compiling.
* In <random>, subtract_with_carry’s streaming operator performed a “full shift” (shifting all of the bits out of an integer in a single operation), which triggers undefined behavior and doesn’t actually work on ARM (DevDiv#373497). We now follow the Standard’s rules so this works on all architectures.
* Taking the addresses of numeric_limits’ static const data members didn’t work with the /Za compiler option (DevDiv#376524). Now it does.
* <ostream> provided unnecessary overloads of std::endl (DevDiv#380718/Connect#730916). While they didn’t appear to be causing any problems, we eliminated them, so only the Standard-mandated overloads are present.
* As required by the Standard, tuple_element<I, array<T, N>> now enforces I < N (DevDiv#410190/Connect#738181).
* <filesystem>’s directory_iterator was returning paths that were too short (DevDiv#411531). (Note that recursive_directory_iterator worked correctly.) We fixed directory_iterator to follow N1975, the Filesystem V2 draft. (Filesystem V3 is on our radar, but it won’t be implemented in 2013 RTM.)
* <filesystem> contained a handwritten implementation called _Strcpy() (DevDiv#422681). Now it uses the CRT’s string copying functionality, and it internally passes around references to arrays for additional bounds safety.
* <ostream>’s op<<() worked with std::hexfloat, but <istream>’s op>>() didn’t (DevDiv#425415/Connect#742775). Now it does.
* In a previous release, iostreams was changed to work correctly with the /vd2 compiler option, but this machinery sometimes emitted compiler warnings. The compiler gained a new pragma, #pragma vtordisp, and iostreams now uses this, avoiding such spurious warnings (DevDiv#430814).
* Due to a typo, _VARIADIC_MAX=7 didn’t compile (DevDiv#433643/Connect#746478). We fixed this typo, then later eradicated the machinery entirely.
* system_error::what()’s return value didn’t follow the Standard (DevDiv#453372/Connect#752770). Now it does.
* codecvt_one_one wouldn’t compile without various using-declarations (DevDiv#453373/Connect#752773). It no longer requires such workarounds.
* Due to a misplaced parenthesis, <chrono>’s duration_cast sometimes didn’t compile (DevDiv#453376/Connect#752794). We fixed the parenthesis.
* Our tests found an extremely obscure deadlock under /clr when holding the locale lock and throwing an exception (e.g. when a std::locale is constructed from a bogus name). We rearranged our code to fix this (DevDiv#453528).
* On ARM, we realized that we were decrementing shared_ptr/weak_ptr’s refcounts in a multithreading-unsafe manner (DevDiv#455917). (Note: x86/x64 were absolutely unaffected.) Although we never observed crashes or other incorrect behavior even after focused testing, we changed the decrements to use sequential consistency, which is definitely correct (although potentially slightly slower than optimal).
* The Standard doesn’t guarantee that tuple_size can be used with things other than tuples (or pairs or arrays). In particular, tuple_size<DerivedFromTuple> isn’t guaranteed to work (DevDiv#457214/Connect#753773). Instead of silently compiling and returning 0, we changed our implementation to static_assert about such cases.
* C++11 says that list::erase() shouldn’t invalidate end iterators (DevDiv#461528). This is a physical consequence of std::list’s representation, so release mode was always correct, but our debug checks complained about invalidation. We fixed them so they now consider end iterators to be preserved.
* Constructing a regex with the flags regex::icase | regex::collate resulted in case-sensitive matching, contrary to the Standard (DevDiv#462743). Now it results in case-insensitive matching.
* Constructing a std::thread resulted in a memory leak reported by _CrtDumpMemoryLeaks() (DevDiv#467504/Connect#757212). This was not a severe, unbounded leak – what happened was that one-time initialization of an internal data structure was allocating memory marked as a “normal block” (observed by the leak tracking machinery), and it also wasn’t being registered for cleanup at CRT shutdown. We fixed this by marking this allocation as a “CRT block” (which is excluded from reporting by default) and also registering it for cleanup.
* Calling wait_for()/wait_until() on futures obtained from packaged_tasks was returning future_status::deferred immediately, which is never supposed to happen (DevDiv#482796/Connect#761829). Now it returns either future_status::timeout or future_status::ready as required by the Standard.
* C++11’s minimized allocator interface doesn’t require allocators to provide a nested rebind struct, but VC didn’t correctly implement this (DevDiv#483844/Connect#762094). We’ve fixed this template machinery to follow the Standardese, N3690 17.6.3.5 [allocator.requirements]/3: “If Allocator is a class template instantiation of the form SomeAllocator<T, Args>, where Args is zero or more type arguments, and Allocator does not supply a rebind member template, the standard allocator_traits template uses SomeAllocator<U, Args> in place of Allocator::rebind<U>::other by default. For allocator types that are not template instantiations of the above form, no default is provided.”
* Another minimized allocator interface bug – in debug mode, std::vector was directly using an allocator instead of going through allocator_traits, which resulted in minimal allocators not working (DevDiv#483851/Connect#762103). We fixed this, and audited all of the containers to rule out similar problems.
* A bug in the Concurrency Runtime powering std::condition_variable resulted in crashes (DevDiv#485243/Connect#762560). ConcRT fixed this.
* vector<bool>, humanity’s eternal nemesis, crashed on x86 with indices over 2 billion (DevDiv#488351/Connect#763795). Note that 2 billion packed bits occupy just 256 MB, so this is entirely possible. We fixed our math so this works.
* pointer_traits didn’t compile with user-defined void pointers (DevDiv#491103/Connect#764717). We reworked our implementation so this compiles (instead of trying to form void& which is forbidden).
* Although it was correct according to the Standard, merge()’s implementation was performing more iterator comparisons than necessary (DevDiv#492840). We changed this (and its related implementations) to be optimal.
* time_put’s return value was correct for char but incorrect for wchar_t, because the implementations had unintentionally diverged (DevDiv#494593/Connect#766065). Now wchar_t works like char.
* C++11 says that istreambuf_iterator::operator*() should return charT by value (DevDiv#494813). Now we do that.
* Constructing a std::locale from a const char * returned from setlocale() could crash, because std::locale’s constructor internally calls setlocale()! (DevDiv#496153/Connect#766648) We fixed this by always storing the constructor’s argument in a std::string, which can’t be invalidated like this.
* <random>’s independent_bits_engine and shuffle_order_engine weren’t calling their _Init() helper functions in their constructors (DevDiv#502244/Connect#768195). That was bad. Now we’re good.
* pow(-1.0, complex<double>(0.5)) took an incorrect shortcut, returning NaN instead of i (DevDiv#503333/Connect#768415). We fixed the shortcut.
* Constructing shared_ptr from nullptr was allocating a reference count control block, which is forbidden by the Standard (DevDiv#520681/Connect#771549). Now such shared_ptrs are truly empty.
* The Standard is very strict about unique_ptr::reset()’s order of operations (DevDiv#523246/Connect#771887). N3690 20.9.1.2.5 [unique.ptr.single.modifiers]/4: “void reset(pointer p = pointer()) noexcept; Effects: assigns p to the stored pointer, and then if the old value of the stored pointer, old_p, was not equal to nullptr, calls get_deleter()(old_p). [ Note: The order of these operations is significant because the call to get_deleter() may destroy *this. —end note ]” We now follow the Standard exactly.
* Interestingly, the Standard permits iostreams to be tied to themselves, but this triggered stack overflows in our implementation (DevDiv#524705/Connect#772293). We now detect self-tying and avoid crashing.
* The Standard requires minmax_element() to find the last biggest element (DevDiv#532622), unlike max_element() which finds the first biggest element. This choice is not arbitrary – it is a fundamental consequence of minmax_element()’s implementation. We fixed minmax_element() to follow the Standard, and carefully audited it for correctness.
* Our implementation declared put_time() as taking tm * (DevDiv#547347/Connect#773846). Now it takes const tm * as required by the Standard.
* In the Nov 2012 CTP, which was released before our Standard Library changes were ready, the compiler team shipped a “fake” <initializer_list> header. This basically worked properly, except that it didn’t have a newline at the end of the file, and that infuriated the /Za compiler option (DevDiv#547397/Connect#773888). Now we have the “real” <initializer_list> from Dinkumware, and we’ve verified that there’s a newline at the end.
* system_clock::to_time_t() attempted to perform rounding, but triggered integer overflow when given enormous inputs (DevDiv#555154/Connect#775105). We now perform truncation, as permitted by the Standard, making us immune to integer overflow.
* The Standard says that forward_iterator_tag shouldn’t derive from output_iterator_tag (DevDiv#557214/Connect#775231), but it did in our implementation. We’ve stopped doing that, and we’ve changed the rest of our code to compensate.
* Due to an obscure compiler bug interacting with operator fake-bool(), unique_ptrs with lambda deleters weren’t always boolean-testable (DevDiv#568465/Connect#775810). Now that unique_ptr has explicit operator bool(), this bug has been completely eradicated.
* Between Visual C++ 2010 and 2012, we introduced a regression where parsing floating-point numbers with iostreams (e.g. “iss >> dbl”) would get the last bit wrong, while the CRT’s strtod() was unaffected. We’ve fixed iostreams to get all the bits right (DevDiv#576315/Connect#776287, also reported as DevDiv#616647/Connect#778982 – fixed in RTM but not Preview).
* <random>’s mt19937 asserted that 0 isn’t a valid seed (DevDiv#577418/Connect#776456). Now it’s considered valid, like the Standard says. We fixed all of the engines to accept 0 seeds and generate correct output.
* <cmath>’s binary overloads weren’t constrained like the unary overloads, resulting in compiler errors in obscure cases (DevDiv#577433/Connect#776471). Although the Standard doesn’t require this to work, we’ve constrained the binary overloads so it works.
* inplace_merge() is one of a few very special algorithms in the Standard – it allocates extra memory in order to do its work, but if it can’t allocate extra memory, it falls back to a slower algorithm instead of failing. Unfortunately, our implementation’s fallback algorithm was incorrect (DevDiv#579795), which went unnoticed for a long time because nobody runs out of memory in the 21st century. We’ve fixed inplace_merge()’s fallback algorithm and audited all fallback algorithms for correctness (including stability). We’ve also added a regression test (capable of delivering a Vulcan nerve pinch to operator new()/etc. on demand) to ensure that this doesn’t happen again.
* Due to an off-by-one error, future_errc’s message() and what() didn’t work (DevDiv#586551). This was introduced when the Standard started saying that future_errc shouldn’t start at 0, and we changed our implementation accordingly – but didn’t notice that the message-translation was still assuming 0-based indexing. We’ve fixed this.
* basic_regex<char>’s implementation permitted high-bit characters, either directly or in character ranges, after an old bugfix – but it rejected high-bit characters specified through regex hex escapes, either directly or in character ranges (DevDiv#604891). Now both are permitted.
* The Standard requires std::functions constructed from null function pointers, null member pointers, and empty std::functions to be empty. Our implementation was constructing non-empty std::functions storing null function pointers/etc., which would crash when invoked. We fixed this to follow the Standard (DevDiv#617384/Connect#779047 – fixed in RTM but not Preview).
* pointer_traits<shared_ptr<Abstract>> didn’t work due to a template metaprogramming subtlety (DevDiv#643180/Connect#781594) – the compiler really doesn’t want to see abstract classes returned by value, even if we’re just doing it for the purposes of decltype. We’ve fixed this machinery so it works for abstract classes.
* In certain cases, our iterator debugging machinery was taking locks followed by no-ops, which is pointlessly slow (DevDiv#650892). This happened in two cases: _ITERATOR_DEBUG_LEVEL=1 which is never the default, and deque. We’ve fixed this so we take locks only in _ITERATOR_DEBUG_LEVEL=2 when we have actual work to protect.
* C++11 says that list::splice() doesn’t invalidate iterators, it just transfers the affected iterators (DevDiv#671816/Connect#785388). This is a physical guarantee, but our debug checks considered such iterators to be invalidated. We’ve updated the checks so they rigorously follow C++11’s rules. (Note: forward_list::splice_after() is still affected; we plan to fix this in the future, but not in 2013 RTM.)
* align()’s implementation in <memory> didn’t follow the Standard (DevDiv#674552). Now it does.
* Lambdas capturing reference_wrappers wouldn’t compile in certain situations (DevDiv#704369/Connect#788701). Now that we’ve fixed conformance problems in reference_wrapper, this code compiles cleanly.
* std::cin no longer overheats the CPU when you hold down spacebar (xkcd#1172).
Breaking Changes
On that note, these features and fixes come with source breaking changes – cases where you’ll have to change your code to conform to C++11, even though it compiled with Visual C++ 2012. Here’s a non-exhaustive list, all of which have been observed in actual code:
* You must #include <algorithm> when calling std::min() or std::max().
* If your code acknowledged our fake scoped enums (traditional unscoped enums wrapped in namespaces), you’ll have to change it. For example, if you were referring to the type std::future_status::future_status, now you’ll have to say std::future_status. Note that most code is unaffected – for example, std::future_status::ready still compiles.
* Similarly, if your code acknowledged our faux alias templates, you’ll have to change it for 2013 RTM. For example, instead of allocator_traits<A>::rebind_alloc<U>::other you’ll have to say allocator_traits<A>::rebind_alloc<U>. Interestingly, although ratio_add<R1, R2>::type will no longer be necessary and you should say ratio_add<R1, R2>, the former will continue to compile. That’s because ratio<N, D> is required to have a “type” typedef for a reduced ratio (which will be the same type if it’s already reduced).
* explicit operator bool() is stricter than operator fake-bool(). explicit operator bool() permits both explicit conversions to bool (e.g. given shared_ptr<X> sp, both static_cast<bool>(sp) and bool b(sp) are valid) and “contextual conversions” to bool (these are the boolean-testable scenarios, e.g. if (sp), !sp, sp && whatever). However, explicit operator bool() forbids implicit conversions to bool, so you can’t say bool b = sp, and you can’t say return sp; given a bool return type.
* Now that we’re using real variadic templates, we aren’t defining _VARIADIC_MAX and its unindicted co-conspirators. We won’t complain if you’re still defining _VARIADIC_MAX, but it’ll have no effect. If you acknowledged our faux-variadic-template macro machinery in any other way, you’ll have to change your code.
* In addition to ordinary keywords, STL headers now strictly forbid macroizing the context-sensitive keywords “override” and “final”.
* reference_wrapper/ref()/cref() now strictly forbid binding to temporary objects.
* <random> now strictly enforces its compiletime preconditions.
* Various STL type traits have the precondition “T shall be a complete type”. This is now enforced more strictly by the compiler, although we do not guarantee that it is enforced in all situations. (STL precondition violations trigger undefined behavior, so the Standard doesn’t guarantee enforcement.)
* The STL does not attempt to support /clr:oldSyntax.
Frequently Asked Questions
Q1: Why are you implementing C++14 Standard Library features when you haven’t finished the C++11 Core Language yet?
A1: That’s a good question with a simple answer. Our compiler team is well aware of the C++11 Core Language features that remain to be implemented. What we’ve implemented here are C++14 Standard Library features. Compiler devs and library devs are not interchangeable – I couldn’t implement major compiler features if my life depended on it (even static_assert would take me months to figure out), and I like to think the reverse is true, although rocket scientists are probably better at pizza delivery than pizza deliverers are at rocket science.
Q2: Fair enough, but you mentioned “C++14 generic lambdas” earlier. Why is your compiler team planning to implement any C++14 Core Language features before finishing all C++11 Core Language features?
A2: As Herb likes to say, “C++14 completes C++11”. The compiler team is pursuing full C++14 conformance, and views all C++11 and C++14 features as a unified bucket of work items. They’re prioritizing these features according to customer demand (including library demand) and implementation cost, so they can deliver the most valuable features as soon as possible. The priority of a feature isn’t affected by when it was voted into the Working Paper. As a result, their post-2013-RTM conformance roadmap places highly valuable C++14 features (e.g. generic lambdas) before less valuable C++11 features (e.g. attributes). Again, please see Herb’s announcement video/post for more information.
Q3: What about the C99 Standard Library, incorporated into C++11 by reference?
A3: Good news – my colleague Pat Brenner worked with Dinkumware to pick up substantial chunks of C99 and integrated them into Visual C++ 2013’s CRT. We’re not done yet, but we’re making progress. Unfortunately, I didn’t have time to deal with the corresponding wrapper headers in the STL (<cmeow> wrapping <meow.h>). Time was extremely limited, and I chose to spend it on getting the variadic template rewrite checked in. I may be able to get the wrapper headers into 2013 RTM, but I cannot promise that yet.
Q4: Will these compiler/library features ship in Visual C++ 2012 Update N, or will we have to wait for Visual C++ 2013?
A4: You’ll have to wait for Visual C++ 2013. I know this isn’t what most people want to hear, so please allow me to explain.
I’m a programmer, and if you’re reading this, I assume you’re a programmer too. So, as programmers, let’s look at the following diff together. This is how <tuple> has changed from Visual C++ 2012 to 2013, as viewed through our internal diff tool “odd”:
Pay special attention to the visual summary on the left. In technical terms, this diff is a horrifying monstrosity. <tuple>’s code is now wonderful, but the changes required to get to this point were basically a complete rewrite. <functional> and the other headers received similar (but lesser) changes.
The VS Update mechanism is primarily for shipping high-priority bugfixes, not for shipping new features, especially massive rewrites with breaking changes (which are tied to equally massive compiler changes).
Major versions like Visual C++ 2013 give us the freedom to change and break lots of stuff. There’s simply no way we can ship this stuff in an Update.
Q5: What about the bugfixes? Can we get those in an Update?
A5: This is an interesting question because the answer depends on my choices (whereas in the previous question, I wouldn’t be allowed to ship such a rewrite in an Update even if I wanted to).
Each team gets to choose which bugfixes they take to “shiproom” for consideration to be included in an Update. There are things shiproom won’t let us get away with (e.g. binary breaking changes are forbidden outside of major versions), but otherwise we’re given latitude to decide things. I personally prioritize bandwidth over latency – that is, I prefer to ship a greater total number of bugfixes in every major version, instead of shipping a lesser total number of bugfixes (over the same period of time) more frequently in multiple Updates.
Going into more detail – backporting fixes takes a nonzero amount of time (especially as branches diverge due to accumulated changes). It’s also riskier – as you may have heard, C++ is a complicated world, and apparently simple changes can have unintended consequences. Even if it’s as small as emitting a spurious warning, we really don’t want Updates to break stuff. Fixing stuff in major versions gives us time to fix the fixes and get everything exactly right.
We do backport STL fixes from time to time (e.g. in Visual C++ 2010 RTM there was a std::string memory leak caused by the Small String Optimization interacting badly with move semantics – that was really bad, so we backported the 3-line fix from 2012 to 2010 SP1), but it’s rare.
Q6: Are you targeting the C++11 International Standard, or a C++14 Working Paper?
A6: We’re usually targeting the current Working Paper (N3690 as of right now), because of the Core/Library Issues that have been resolved since C++11. In the STL, I consider any nonconformance to the current Working Paper to be a bug.
Q7: How do you pronounce “tuple”?
A7: It’s “too-pull”. It does not rhyme with “supple”!
Thanks for reading this extremely long post, and I hope you enjoy using VS 2013’s STL.
Stephan T. Lavavej
Senior Developer – Visual C++ Libraries
0 comments