Open Sourcing MSVC’s STL

Stephan T. Lavavej - MSFT

Today at CppCon 2019, we (the MSVC team) announced that we’re releasing our implementation of the C++ Standard Library (also known as the STL) as open source.

https://github.com/microsoft/STL is our new repository, containing all of our product source code, a new CMake build system, and a README with more information. As it explains, we’re still working on migrating to GitHub. While you can clone and build the code right now, there are several things we need to do (like adding our test suites) before we can begin reviewing pull requests.

Q&A

Q: Why are you doing this?

A: There are several reasons. Working on the STL in GitHub will allow our customers to follow our development as it happens, try out our latest changes, and help improve our pull requests by reviewing them. As C++ Standardization accelerates, with more large features being voted in every year, we believe that accepting major features as open source contributions will be important. (For example, C++20’s chrono and format libraries are potential candidates.) We also want to contribute back to the C++ community by making it possible to take our implementations of major features. (For example, C++17’s charconv.)

Q: What license are you using, and does this directly affect Visual Studio’s customers?

A: MSVC’s STL is distributed under the Apache License v2.0 with LLVM Exceptions, which was recently created by the Clang/LLVM/libc++ project. We chose this permissive open source license because having the same license as libc++ will make it easier to share code between our libraries. To be clear, MSVC’s STL isn’t merging with libc++; they’re still distinct libraries that support different platforms with different data structure representations. However, if libc++’s maintainers are interested in taking feature implementations from MSVC’s STL (e.g. floating-point charconv), or in collaborating on the development of new features in both libraries simultaneously, we’ll be able to help without having to worry about licensing.

As a customer of MSVC’s STL, you might be wondering whether this new license creates new obligations for you. Microsoft’s position is that the text of the Apache License v2.0 with LLVM Exceptions (specifically, the wording of the LLVM Exceptions) clearly states that when you compile your own source code using MSVC’s STL to produce object code or similar output (e.g. static libraries, dynamic libraries, and executables), you aren’t required to provide attribution when shipping your compiled products to your end users. This is another reason we’ve chosen this license: to avoid disrupting our customers’ businesses.

(Additional note, in the interest of completeness: MSVC’s STL contains source code derived from Boost.Math and Ryu, used under the terms of the Boost Software License v1.0. This is another permissive open source license that also contains an explicit exception for object code. Our derived source code is distributed under the Apache License v2.0 with LLVM Exceptions.)

Q: Are you going to open source anything else in the MSVC toolset?

A: We have no such plans. We chose the STL because it’s different from other MSVC libraries and the compiler. Specifically, the STL is fast-evolving and designed by the C++ Standardization Committee, unlike other MSVC libraries. (Being designed by Committee is an advantage for open sourcing! It means that we don’t need to spend any time and energy on feature design review. Implementation strategy and tactics are far more constrained, and therefore easier to review.) The STL is also relatively easy to contribute to, and somewhat loosely coupled, unlike the compiler (where, as a general rule, everything interacts with everything else).

(One exception: there are support libraries for the STL that we may open source in the future, but we have nothing to announce at this time.)

Q: Is there some catch?

A: A fairly small one: we’re going to spend some time overhauling our build system, test infrastructure, and issue tracking, which will delay some work on C++20 library features. (We just finished implementing all C++17 library features, so this shouldn’t be too problematic.) This will allow us to work on the STL more efficiently and ultimately reach C++20 completeness faster.

Acknowledgements

The MSVC team would like to thank Dinkumware and P.J. Plauger for making this possible.

Stephan T. Lavavej - MSFT

Follow    

15 comments

  • Avatar
    FrankHB

    A bit curious: why ‘m’ is lower case and all ‘S’, ‘T’ and ‘L’ are upper case in the repository URL? Is this a MS convention or a GitHub one?
    And it seems that naming the implementation of the standard library “STL” is a bit confusing (with “Stephan T. Lavavej” in discussions in LWG issues, for example). AFAIK MS does not have a fake implementation used like Android’s platform libstdc++ (so the implementations in NDK are deliberately renamed as “STL” to make them look different).

    (It is a bit annoying that all quotation marks are replaced automatically after clicking “submit”…)

    • Stephan T. Lavavej - MSFT
      Stephan T. Lavavej - MSFT

      Regarding casing, we had control over the repo’s casing but not over the organization’s casing. MS projects vary in capitalization; as STL is an acronym, uppercase made sense. That said, it works (or appears to work) case-insensitively.

      We have been referring to the STL this way on VCBlog for many years. There is no real chance for confusion with my initials, as the library doesn’t comment on LWG issues. In addition to “Effective STL” by Scott Meyers, I like to point out that everyone knows what the Electronic Arts EASTL is trying to achieve.

      I was unaware of the Android thing, but I believe that’s additional evidence that thinking of the C++ Standard Library as “the STL” is common.

      • Avatar
        FrankHB

        As an acronym of the library, STL is initially short for “Standard Template Library”, but not the (C++) STandard Library. It’s true that “STL” is also often used as a synonym of the latter, but not everyone know the exact extent it can refer to. Being a library, “STL” is too contextually dependent. There is no single authentic C++ library named “Standard Template Library” now, so it routinely raises more interpretation overhead to make the intent clear: is it the interface, or some specific implementations? OTOH, the C++ standard library is a term about the interface requirements normatively defined by ISO/IEC 14882, which makes it only depends on the version of the standard in this sense.

        This is particularly a problem for your implementation because you don’t specify an implementation-specific official library name like “libstdc++” or “libc++”, so you effectively overloads the already fuzzy phrase “STL” once more. Too often I have to use “Microsoft’s STL” to prevent some potential ambiguities when talking about things like conformance. (Probably over-pedantic, I’m carefully making it technically correct here, although this cannot be entirely accurate because I can’t assert you will not make another “STL” branded with Microsoft. “MSVC’s” STL is somewhat good enough for the status quo, but I’m also not sure whether the library implementation can be supported by other compilers in future. It is like saying “GCC’s” libstdc++ is good enough as per their bugzilla categories, but “GNU’s” libstdc++ may be more meaningful for the users who may be using clang++ + libstdc++.) This verbosity should better have been eliminated.

        Despite the case above, STL and the C++ standard library are certainly not identical in daily use. For instance, the C++ standard library has headers like <cstdio>, which is not interested by “Effective STL” or EASTL (and it still must be a part of your implementation). Even without the C standard library wrappers, there are still some longstanding confusions like this. It probably would not be a problem to people do serious works with the standard library implementations, though.

        Moreover, there exist implementations having different treatment on the terminology. The <bits/stl_xxxxxx.h> internal headers of libstdc++ refer to the implementation comes from traditional STL, which does not include things like std::string and std::forward_list. I prefer this approach because it has less confusions overall.

        As of the comments of the LWG issues… I see the source of your implementation does refer to some LWG issues. So does libc++. Besides, libstdc++ has _GLIBCXX_RESOLVE_LIB_DEFECTS. All lead the readers to LWG issue documents. It would not be a big problem then, but forced overloading resolution on the phrase “STL” after switching to the LWG documents from C++ source still sounds strange.

    • Stephan T. Lavavej - MSFT
      Stephan T. Lavavej - MSFT

      If the “How to Consume” instructions in the readme are insufficient, can you file a GitHub issue explaining what further instructions you’re looking for? We have a “documentation” label for such requests (and we can also accept PRs for them easily, since they don’t depend on test/CI).

  • Avatar
    selvakumar jawahar

    Awesome News!! Great Decision!!! kudos to entire team. It will be very interesting to see cross-pollination of ideas between gcc and msvc

  • Avatar
    Mike Diack

    As others have said, the STL is NOT the C++ standard library…
    Posting this kind of statement only causes confusion.

    • Avatar
      Me Gusta

      It has to. Is there any kind of alternative to accessing the low level Windows platform functionality other than through windows.h?
      This is like looking at the C++ standard library source for Linux and saying “I see, POSIX headers even infects the STL source code…”

      • Stephan T. Lavavej - MSFT
        Stephan T. Lavavej - MSFT

        If you look carefully, you’ll notice that we never include Windows.h from STL headers, directly or indirectly. Only our separately compiled source files (and their internal headers) include Windows.h. This is primarily for throughput and macro cleanliness.

  • Avatar
    Jonathan Wilson

    Am I correct in assuming this is the complete source code for msvcprt140.dll and the debug and static versions of that? (obviously excluding the things linked into msvcprt140.dll from the parts of the base CRT that live in msvcrt.lib and get linked into every binary)

    Also the “support libraries for the STL”, what exactly are those and where do they live?

    • Stephan T. Lavavej - MSFT
      Stephan T. Lavavej - MSFT

      Yep, these are the complete sources for msvcp140.dll and its various flavors. Most of our code is header-only, so the separately compiled DLL/LIB is fairly small (mostly iostreams, multithreading, filesystem, special math). It will become even smaller when we can break ABI.

      The readme’s Block Diagram attempts to explain what the support libraries are: VCRuntime, VCStartup, ConcRT (not mentioned in the diagram), and the UCRT. The first three are shipped within Visual Studio; the UCRT is part of the Windows SDK.

      • Avatar
        Jonathan Wilson

        Good to know there is at least the possibility that vcruntime and vcstartup may at some point in the future go open (I can see some benefits in having those open and also a bunch of negatives)

Leave a comment