Vcpkg: 2019.07 Update
The 2019.07 update of vcpkg, a tool that helps you manage C and C++ libraries on Windows, Linux, and macOS, is now available. This update is a summary of the new functionality and improvements made to vcpkg over the past month. Last month was the first time we created a vcpkg release (Vcpkg: 2019.06 Update).
In this post, we will cover caching in Azure Pipelines with vcpkg in addition to many new port and triplet updates, improvements for port contributors, and new documentation. For a full list of this release’s improvements, check out our changelog on GitHub.
Caching in Azure Pipelines with vcpkg
The public preview of caching in Azure Pipelines is now available, and you can use it with vcpkg!
You can use pipeline caching to improve build time by allowing previously built and cached vcpkg artifacts (including libraries) to be reused in subsequent runs. This allows you to reduce and avoid the cost to rebuild the same libraries for each build run the same libraries. Caching may be especially useful with vcpkg when you are installing and building the same dependencies (libraries) over and over during your build. That process can often be time-consuming if it involves building large libraries.
For example, if you have a C++ application that uses SQLite databases, you’ll likely want to use SQLite3 among other libraries. Each time you run a build on your server, you install vcpkg and the sqlite3 library. Without using pipeline caching, this may take some time:
Now, with Azure Pipelines Caching, we can have a much faster and better experience. One of our community contributors and Microsoft employee, Luca Cappa, created a pipeline task to help you use the CacheBeta pipeline task. We’ll show you how leveraging his scripts reduced the ‘run vcpkg’ build step from 2m 26s to just 14s!
CppBuildTasks Azure DevOps Extension with vcpkg
The CppBuildTasks project pipeline task does the following:
So, in the case of SQLite3, vcpkg will be updated and the sqlite3 library will be installed. It is cached in the pipeline such that in subsequent runs you do not need to install and build the sqlite3 library again.
To get started with CppBuildTasks with Azure Pipelines in your project you can follow the simple CppBuildTasks developer documentation.
Looking at the example, there are a few things to note:
variables: # Exact vcpkg's version to fetch. vcpkgGitRef: 5a3b46e9e2d1aa753917246c2801e50aaabbbccc # Cache the vcpkg's build artifacts. - task: CacheBeta@0 displayName: Cache vcpkg inputs: # As 'key' use the content of the response file, vcpkg's commit id and build agent name. # The key must be one liner, each segment separated by pipe, non-path segments enclosed by # double quotes. key: $(Build.SourcesDirectory)/vcpkg_x64-linux.txt | "$(vcpkgGitRef)" | "$(Agent.Name)" path: '$(Build.BinariesDirectory)/vcpkg'
vcpkgGitRefis a specific commit ID for the version of vcpkg you would like to install.
task: CacheBeta@0enables pipeline caching in Azure Pipelines.
key: $(Build.SourcesDirectory)/vcpkg_x64-linux.txt | "$(vcpkgGitRef)" | "$(Agent.Name)"uses the source directory for the libraries and response file (which contains a list of packages), the commit ID, and the build agent name to generate a hash to use in the build pipeline.
Pipeline Caching with CppBuildTasks Results
Now, taking a look at installing vcpkg and sqlite3 using the CppBuildTasks script, we can see a remarkable difference in the build time on our Ubuntu server:
Enabling caching reduced the “Run vcpkg” build step from 2m 26s to just 14s (Caching + Run vcpkg).
You can view more examples in the Samples section of the CppBuildTasks GitHub repo.
We added 37 new ports in the month of July. Some notable additions include: 7zip, basisu, librdkafka, mimalloc, mongoose, and zookeeper. You can view a full list of new ports in our 2019.07 changelog. For a full list of libraries, search for a library name in the GitHub repo ports folder or use the vcpkg search command.
In addition to new ports, we updated 160 existing ports.
Vcpkg provides many triplets (target environments) by default. This past month, we continued increasing the number of ports available on Linux – from 823 to 866.
Here is a current list of ports per triplet:
Don’t see a triplet you’d like? You can easily add your own triplets. Details on adding triplets can be found in our documentation.
Improvements for Port Contributors
We also made improvements to the vcpkg infrastructure including a new vcpkg variable and a mechanism to modify and set vcpkg triplet variables on a per port basis. These features are the first steps towards enabling better tool dependencies in vcpkg. Stay tuned!
Passthrough Triplet Variable
Before we added the VCPKG_ENV_PASSTHROUGH triplet variable, the only environment variables on Windows available to the portfile were those found on an allow list hard-coded into vcpkg. This new triplet variable allows us to augment that list with variables defined in the triplet or the environment overrides file.
Port authors can add an environment-overrides.cmake file to a port, override, or set, vcpkg triplet variables on a per port basis. For example, this allows ports to specify environment variables that are not allow listed in the vcpkg source to be available to the portfile.
We also updated our documentation to reflect these new changes. Check out the following docs for more information on some of the updates outlined in this post in addition to a couple other areas:
- Maintainer Guidelines and Policies
- Overlay triplets example: build dynamic libraries on Linux – More details on overlay triplets in our 06 Update blog post
Thank you to everyone who contributed to vcpkg! We now have 659 total contributors. This release, we’d like to thank the following 11 contributors who made code changes in June:
Tell Us What You Think
Install vcpkg, give it a try, and let us know what you think. If you run into any issues, or have any suggestions, please report them on the Issues section of our GitHub repository.
We can be reached via the comments below or via email (firstname.lastname@example.org). You can also find our team – and me – on Twitter @VisualC and @tara_msft.
I’ve just checked all of the C++ dependencies we use in Mudlet and all of them were supported, unlike in Conan and Buckaroo! This is good to see. We build for Windows (with mingw), Linux and macOS so a cross-platform package manager is something we’ve been looking forward to. Going to investigate vcpkg further 🙂
Very happy to hear that Azure Pipelines supports caching vcpkg, thank you! We’re trying to use vcpkg at our CI, as installed “out of the box” at vmImage ‘vs2017-win2016’ (in “C:\vcpkg\”), but kept up-to-date by “git pull” and “bootstrap-vcpkg.bat”. The install of third-party-library we depend upon, ITK (“vcpkg install itk”) takes almost an hour. Can you please show us how to cache this install? It’s not clear to me which vcpkgGitRef to use, we just like to use the latest revision! https://github.com/SuperElastix/elastix/blob/6ecc07ca93198d0a20126bcada968f77c83bb6b5/Testing/CI/Azure/ci.yml
@niels dekker i’ve few ideas on how to implement the cached build, come over to the project discussion
to describe the status quo the elastix’s build.
Very nice to have the caching! It works well, but when I build llvm 8 I get the following build error and have no idea on how to get the vcpkg’s build logs. I am new to azure devops. The llvm vcpkg config build correctly on my machine.
>>>> here is some more of the error.
Error: Building package llvm:x64-windows failed with: BUILD_FAILED
Please ensure you’re using the latest portfiles with `.\vcpkg update`, then
submit an issue at https://github.com/Microsoft/vcpkg/issues including:
Vcpkg version: 2019.09.12-nohash
Additionally, attach any relevant sections from the log files above.
##[debug]Exit code 1 received from tool ‘D:\a\1\s\vcpkg\vcpkg.exe’
##[debug]STDIO streams have closed for tool ‘D:\a\1\s\vcpkg\vcpkg.exe’
##[error]Last command execution failed with error code ‘1’.
##[debug]Processed: ##vso[task.issue type=error;]Last command execution failed with error code ‘1’.
##[debug]Error: Error: Last command execution failed with error code ‘1’.
##[error]Error: Last command execution failed with error code ‘1’.
##[debug]Processed: ##vso[task.issue type=error;]Error: Last command execution failed with error code ‘1’.
##[debug]task result: Failed
##[error]vcpkg failed with error: ‘Error: Last command execution failed with error code ‘1’.’.
##[debug]Processed: ##vso[task.issue type=error;]vcpkg failed with error: ‘Error: Last command execution failed with error code ‘1’.’.
##[debug]Processed: ##vso[task.complete result=Failed;]vcpkg failed with error: ‘Error: Last command execution failed with error code ‘1’.’.