August 15th, 2019

Vcpkg: 2019.07 Update

The 2019.07 update of vcpkg, a tool that helps you manage C and C++ libraries on Windows, Linux, and macOS, is now available. This update is a summary of the new functionality and improvements made to vcpkg over the past month. Last month was the first time we created a vcpkg release (Vcpkg: 2019.06 Update).

In this post, we will cover caching in Azure Pipelines with vcpkg in addition to many new port and triplet updates, improvements for port contributors, and new documentation. For a full list of this release’s improvements, check out our changelog on GitHub.

Caching in Azure Pipelines with vcpkg

The public preview of caching in Azure Pipelines is now available, and you can use it with vcpkg!

You can use pipeline caching to improve build time by allowing previously built and cached vcpkg artifacts (including libraries) to be reused in subsequent runs. This allows you to reduce and avoid the cost to rebuild the same libraries for each build run the same libraries. Caching may be especially useful with vcpkg when you are installing and building the same dependencies (libraries) over and over during your build. That process can often be time-consuming if it involves building large libraries.

For example, if you have a C++ application that uses SQLite databases, you’ll likely want to use SQLite3 among other libraries. Each time you run a build on your server, you install vcpkg and the sqlite3 library. Without using pipeline caching, this may take some time:

Now, with Azure Pipelines Caching, we can have a much faster and better experience. One of our community contributors and Microsoft employee, Luca Cappa, created a pipeline task to help you use the CacheBeta pipeline task. We’ll show you how leveraging his scripts reduced the ‘run vcpkg’ build step from 2m 26s to just 14s!

CppBuildTasks Azure DevOps Extension with vcpkg

The CppBuildTasks project pipeline task does the following:

So, in the case of SQLite3, vcpkg will be updated and the sqlite3 library will be installed. It is cached in the pipeline such that in subsequent runs you do not need to install and build the sqlite3 library again.

To get started with CppBuildTasks with Azure Pipelines in your project you can follow the simple CppBuildTasks developer documentation.

Looking at the example, there are a few things to note:

variables:
    # Exact vcpkg's version to fetch.    
    vcpkgGitRef: 5a3b46e9e2d1aa753917246c2801e50aaabbbccc

    # Cache the vcpkg's build artifacts.  
  - task: CacheBeta@0    
    displayName: Cache vcpkg    
    inputs:      
      # As 'key' use the content of the response file, vcpkg's commit id and build agent name.      
      # The key must be one liner, each segment separated by pipe, non-path segments enclosed by      
      # double quotes.      
      key: $(Build.SourcesDirectory)/vcpkg_x64-linux.txt | "$(vcpkgGitRef)" | "$(Agent.Name)"      
      path: '$(Build.BinariesDirectory)/vcpkg'
  • vcpkgGitRef is a specific commit ID for the version of vcpkg you would like to install.
  • task: CacheBeta@0 enables pipeline caching in Azure Pipelines.
  • key: $(Build.SourcesDirectory)/vcpkg_x64-linux.txt | "$(vcpkgGitRef)" | "$(Agent.Name)" uses the source directory for the libraries and response file (which contains a list of packages), the commit ID, and the build agent name to generate a hash to use in the build pipeline.

Pipeline Caching with CppBuildTasks Results

Now, taking a look at installing vcpkg and sqlite3 using the CppBuildTasks script, we can see a remarkable difference in the build time on our Ubuntu server:

Enabling caching reduced the “Run vcpkg” build step from 2m 26s to just 14s (Caching + Run vcpkg).

You can view more examples in the Samples section of the CppBuildTasks GitHub repo.

Ports

We added 37 new ports in the month of July. Some notable additions include: 7zip, basisu, librdkafka, mimalloc, mongoose, and zookeeper. You can view a full list of new ports in our 2019.07 changelog. For a full list of libraries, search for a library name in the GitHub repo ports folder or use the vcpkg search command.

In addition to new ports, we updated 160 existing ports.

Triplets

Vcpkg provides many triplets (target environments) by default. This past month, we continued increasing the number of ports available on Linux – from 823 to 866.

Here is a current list of ports per triplet:

Triplet Ports Available
x64-osx 788
x64-linux 866
x64-windows 1039
x86-windows 1009
x64-windows-static 928
arm64-windows 678
x64-uwp 546
arm-uwp 522

Don’t see a triplet you’d like? You can easily add your own triplets. Details on adding triplets can be found in our documentation.

Improvements for Port Contributors

We also made improvements to the vcpkg infrastructure including a new vcpkg variable and a mechanism to modify and set vcpkg triplet variables on a per port basis. These features are the first steps towards enabling better tool dependencies in vcpkg. Stay tuned!

Passthrough Triplet Variable

Before we added the VCPKG_ENV_PASSTHROUGH triplet variable, the only environment variables on Windows available to the portfile were those found on an allow list hard-coded into vcpkg. This new triplet variable allows us to augment that list with variables defined in the triplet or the environment overrides file.

Environment Overrides

Port authors can add an environment-overrides.cmake file to a port, override, or set, vcpkg triplet variables on a per port basis. For example, this allows ports to specify environment variables that are not allow listed in the vcpkg source to be available to the portfile.

Documentation

We also updated our documentation to reflect these new changes. Check out the following docs for more information on some of the updates outlined in this post in addition to a couple other areas:

Thank you

Thank you to everyone who contributed to vcpkg! We now have 659 total contributors. This release, we’d like to thank the following 11 contributors who made code changes in June:

BillyONeal myd7349
cenit Neumann-A
coryan SuperWig
eao197 tarcila
JackBoosY TartanLlama
jwillemsen

 

Tell Us What You Think

Install vcpkg, give it a try, and let us know what you think. If you run into any issues, or have any suggestions, please report them on the Issues section of our GitHub repository.

We can be reached via the comments below or via email (vcpkg@microsoft.com). You can also find our team – and me – on Twitter @VisualC and @tara_msft.

Author

4 comments

Discussion is closed. Login to edit/delete existing comments.

  • Suresh Kannan

    Very nice to have the caching! It works well, but when I build llvm 8 I get the following build error and have no idea on how to get the vcpkg's build logs. I am new to azure devops. The llvm vcpkg config build correctly on my machine.

    D:\a\1\s\vcpkg\buildtrees\llvm\install-x64-windows-dbg-out.log

    >>>> here is some more of the error.
    Error: Building package llvm:x64-windows failed with: BUILD_FAILED

    Please ensure you're using the latest portfiles with...

    Read more
  • Niels Dekker

    Very happy to hear that Azure Pipelines supports caching vcpkg, thank you! We're trying to use vcpkg at our CI, as installed "out of the box" at vmImage 'vs2017-win2016' (in "C:\vcpkg\"), but kept up-to-date by "git pull" and "bootstrap-vcpkg.bat". The install of third-party-library we depend upon, ITK ("vcpkg install itk") takes almost an hour. Can you please show us how to cache this install? It's not clear to me which vcpkgGitRef to use, we just...

    Read more
    • Luca Luca

        @niels dekker i’ve few ideas on how to implement the cached build, come over to the project discussion
      to describe the status quo the elastix’s build.

  • Vadim Peretokin

    I’ve just checked all of the C++ dependencies we use in Mudlet and all of them were supported, unlike in Conan and Buckaroo! This is good to see. We build for Windows (with mingw), Linux and macOS so a cross-platform package manager is something we’ve been looking forward to. Going to investigate vcpkg further 🙂