{"id":204,"date":"2021-10-13T11:16:24","date_gmt":"2021-10-13T18:16:24","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/engineering-at-microsoft\/?p=204"},"modified":"2021-10-15T13:08:14","modified_gmt":"2021-10-15T20:08:14","slug":"generating-software-bills-of-materials-sboms-with-spdx-at-microsoft","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/engineering-at-microsoft\/generating-software-bills-of-materials-sboms-with-spdx-at-microsoft\/","title":{"rendered":"Generating Software Bills of Materials (SBOMs) with SPDX at Microsoft"},"content":{"rendered":"<p>The <a href=\"https:\/\/www.whitehouse.gov\/briefing-room\/presidential-actions\/2021\/05\/12\/executive-order-on-improving-the-nations-cybersecurity\/\">U.S. Presidential Executive Order on Improving the Nation\u2019s Cybersecurity<\/a>, released on May 12, 2021, came in response to the <a href=\"https:\/\/aka.ms\/sunspot\">SolarWinds supply chain attack<\/a>, and calls for sweeping improvements to modernize Federal Government cybersecurity and enhance software supply chain security. One of the items that they are requiring is a Software Bill of Materials (SBOM).<\/p>\n<p>SBOMs aren\u2019t new to Microsoft. In fact, we have been generating our own proprietary build manifests for years. Since September 2019, Microsoft has also led and co-chaired the <a href=\"https:\/\/www.it-cisq.org\/software-bill-of-materials\/index.htm\">Consortium for Information &amp; Software Quality (CISQ) Tool-to-Tool (3T) SBOM<\/a> cross-industry working group to define a new standard SBOM schema. What <em>is<\/em> new is that Microsoft has chosen, along with the others in 3T, to merge the 3T effort with the Linux Foundation&#8217;s work and use <a href=\"https:\/\/spdx.dev\/\">Software Package Data Exchange (SPDX)<\/a> for all SBOMs we generate, and we have embarked on the mission to do this for <em>all<\/em> software we produce. This means we\u2019ve had to convert our existing manifest generation tools to output JSON files in the <a href=\"https:\/\/www.iso.org\/standard\/81870.html\">ISO\/IEC 5962:2021 standard SPDX 2.2.1 format<\/a>, and we need to roll out this capability across our core engineering systems.<\/p>\n<h3>Why have an SBOM?<\/h3>\n<p>An SBOM is useful to producers and consumers of software, as it provides software transparency, software integrity, and software identity benefits. Here is a bit about each:<\/p>\n<ul>\n<li><strong>Software transparency<\/strong>: SBOMs provide a list of ingredients used in the creation of a piece of software, such as open source software, components, and potentially even build tools. This enables producers and consumers to better inventory and evaluate license and vulnerability risk.<\/li>\n<li><strong>Software integrity<\/strong>: While code signing is still the industry standard for trusting software and its integrity, SBOMs contain package and file checksums to enable consumers to validate the hashes, which can be useful in scenarios when signatures aren\u2019t present.<\/li>\n<li><strong>Software identity<\/strong>: When vulnerabilities (CVEs) are created, they are assigned to a <a href=\"https:\/\/nvd.nist.gov\/products\/cpe\">Common Platform Enumeration (CPE)<\/a> identifier, which can have issues attributing a CPE to a specific piece of software. Software IDs within SBOMs provide a much more accurate way to identify software.<\/li>\n<\/ul>\n<h3>Designing executive order-compliant SBOMs<\/h3>\n<p>The report outlined what fields must be included in our SBOMs, so we mapped the NTIA minimum fields to SPDX 2.2.1:<\/p>\n<table width=\"92%\">\n<tbody>\n<tr>\n<td width=\"18%\"><strong>NTIA field<\/strong><\/td>\n<td width=\"61%\"><strong>NTIA description<\/strong><\/td>\n<td width=\"19%\"><strong>SPDX 2.2.1 field<\/strong><\/td>\n<\/tr>\n<tr>\n<td width=\"18%\">Supplier Name<\/td>\n<td width=\"61%\">The name of an entity that creates, defines, and identifies components<\/td>\n<td width=\"19%\">Package Supplier<\/td>\n<\/tr>\n<tr>\n<td width=\"18%\">Component Name<\/td>\n<td width=\"61%\">Designation assigned to a unit of software defined by the original supplier<\/td>\n<td width=\"19%\">Package Name<\/td>\n<\/tr>\n<tr>\n<td width=\"18%\">Version of the Component<\/td>\n<td width=\"61%\">Identifier used by the supplier to specify a change in software from a previously identified version<\/td>\n<td width=\"19%\">Package Version<\/td>\n<\/tr>\n<tr>\n<td width=\"18%\">Other Unique Identifiers<\/td>\n<td width=\"61%\">Other identifiers that are used to identify a component, or serve as a look-up key for relevant databases<\/td>\n<td width=\"19%\">Package SPDX Identifier<\/td>\n<\/tr>\n<tr>\n<td width=\"18%\">Dependency Relationship<\/td>\n<td width=\"61%\">Characterizing the relationship that an upstream component X is included in software Y<\/td>\n<td width=\"19%\">Relationship<\/td>\n<\/tr>\n<tr>\n<td width=\"18%\">Author of SBOM Data<\/td>\n<td width=\"61%\">The name of the entity that creates the SBOM data for this component<\/td>\n<td width=\"19%\">Creator<\/td>\n<\/tr>\n<tr>\n<td width=\"18%\">Timestamp<\/td>\n<td width=\"61%\">Record of the date and time of the SBOM data assembly<\/td>\n<td width=\"19%\">Created<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>&nbsp;<\/p>\n<p>This helped define the first phase of our implementation of the SPDX spec. We knew we had to include all mandatory fields from the SPDX 2.2 specification <em>plus<\/em> include specific optional fields to establish a baseline for our first implementation. While <em>supplier name, package version, package checksum, and relationship<\/em> fields are optional in SPDX, we are making them mandatory for Microsoft products.<\/p>\n<h3>Generating SBOMs at scale across Microsoft<\/h3>\n<p>Microsoft cares deeply about developer productivity and wants to minimize impact to build times, especially considering we have an average of ~500,000 builds occurring on any given day. Taking this into account, here\u2019s how we\u2019re planning to roll this out to the thousands of Microsoft products we build:<\/p>\n<ol>\n<li>Design tooling to automate SBOM generation at build time.<\/li>\n<li>Produce SBOMs for all official builds.<\/li>\n<li>Pilot this capability with a small customer base to incorporate feedback.<\/li>\n<li>Leverage existing CI\/CD capabilities to intelligently inject our SBOM generation tool into build pipelines, aspiring to have SBOM generation \u201con by default.\u201d<\/li>\n<li>Expand this capability out to our various engineering systems in a phased rollout.<\/li>\n<li>Provide a cross-platform executable binary for non-standard build environments.<\/li>\n<\/ol>\n<h3>Capabilities of our SBOM generator<\/h3>\n<p>Our SPDX SBOM generator tool is cross-plat, supporting Windows, Linux, and Mac environments (and will be open sourced soon). It also provides open source software (OSS) detection for inclusion in the SBOM across NPM, NuGet, PyPI, CocoaPods, Maven, Golang, Rust Crates, RubyGems, containers (and their Linux packages), Gradle, Ivy, GitHub public repositories, and more. It generates two checksums for each package and file \u2013 SHA256 (strong, collision resistant hash) and SHA1 (required per SPDX specification). Our tool also automates digitally signing each SBOM to protect its integrity and then creates a new folder at the root of the build drop called _manifest; this is where the SPDX JSON file is stored. An example is shown below:<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/engineering-at-microsoft\/wp-content\/uploads\/sites\/72\/2021\/10\/addiglio-fig2.png\"><img decoding=\"async\" class=\"wp-image-210 aligncenter\" src=\"https:\/\/devblogs.microsoft.com\/engineering-at-microsoft\/wp-content\/uploads\/sites\/72\/2021\/10\/addiglio-fig2-300x202.png\" alt=\"Image addiglio fig2\" width=\"601\" height=\"405\" srcset=\"https:\/\/devblogs.microsoft.com\/engineering-at-microsoft\/wp-content\/uploads\/sites\/72\/2021\/10\/addiglio-fig2-300x202.png 300w, https:\/\/devblogs.microsoft.com\/engineering-at-microsoft\/wp-content\/uploads\/sites\/72\/2021\/10\/addiglio-fig2.png 624w\" sizes=\"(max-width: 601px) 100vw, 601px\" \/><\/a><\/p>\n<h3>Adding build provenance information to the SBOM<\/h3>\n<p>SBOMs primarily provide transparency about the contents of the build output. At Microsoft, we wanted to go a step further and provide provenance information about the build system where the SBOM was generated and make the SBOM itself tamper-evident. To achieve this, we integrated a signing service with our SBOM generation tool, which performs the following workflow:<\/p>\n<ol>\n<li>At the start of a build, the build service creates a session token that includes claims describing the build (e.g., source code commit ID, build ID, the repository URL) which uniquely identify a build run.<\/li>\n<li>The build service sends this token to the build agent\/runner to use during the build.<\/li>\n<li>The build runs as normal, creating outputs.<\/li>\n<li>An SBOM is created that describes the outputs.<\/li>\n<li>The build agent calls into the signing service, providing both the session token and a hash of the SBOM.<\/li>\n<li>The build service creates a catalog file with a signature that attests that the hash of the SBOM came from the build described by the claims in the sessions token.<\/li>\n<\/ol>\n<h3>Validating our SBOMs at release<\/h3>\n<p>One key scenario that we\u2019ve added is the ability to validate the hashes of all files listed in the SBOM against the hashes of the build drop itself and validate that the digital signature on the SBOM is the trusted signature from Microsoft. If our SBOM validation tool detects a hash mismatch or incorrect signature, our SBOM validation tool will block the deployment. This ensures that nothing was tampered with between build and release. Going forward, we would also like to add checks that, for example, the signature shows that the build came from the expected build definition.<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/engineering-at-microsoft\/wp-content\/uploads\/sites\/72\/2021\/10\/adiglio-figure-1.png\"><img decoding=\"async\" class=\"wp-image-206 aligncenter\" src=\"https:\/\/devblogs.microsoft.com\/engineering-at-microsoft\/wp-content\/uploads\/sites\/72\/2021\/10\/adiglio-figure-1-300x158.png\" alt=\"Image adiglio figure 1\" width=\"600\" height=\"316\" srcset=\"https:\/\/devblogs.microsoft.com\/engineering-at-microsoft\/wp-content\/uploads\/sites\/72\/2021\/10\/adiglio-figure-1-300x158.png 300w, https:\/\/devblogs.microsoft.com\/engineering-at-microsoft\/wp-content\/uploads\/sites\/72\/2021\/10\/adiglio-figure-1-768x405.png 768w, https:\/\/devblogs.microsoft.com\/engineering-at-microsoft\/wp-content\/uploads\/sites\/72\/2021\/10\/adiglio-figure-1.png 821w\" sizes=\"(max-width: 600px) 100vw, 600px\" \/><\/a><\/p>\n<p>We anticipate more exciting announcements to follow on this topic in the future, so stay tuned!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>In this post, Adrian Diglio walks us through how Microsoft is planning to generate SBOMs not just to meet the U.S. Presidential Executive Order on Improving the Nation&#8217;s Cybersecurity, but for all software that Microsoft produces.<\/p>\n","protected":false},"author":72741,"featured_media":170,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[4,13,17,15,12,2,14,16],"class_list":["post-204","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-engineering-at-microsoft","tag-1es","tag-devsecops","tag-executive-order","tag-sbom","tag-secure-supply-chain","tag-security","tag-software-bill-of-materials","tag-spdx"],"acf":[],"blog_post_summary":"<p>In this post, Adrian Diglio walks us through how Microsoft is planning to generate SBOMs not just to meet the U.S. Presidential Executive Order on Improving the Nation&#8217;s Cybersecurity, but for all software that Microsoft produces.<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/engineering-at-microsoft\/wp-json\/wp\/v2\/posts\/204","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/engineering-at-microsoft\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/engineering-at-microsoft\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/engineering-at-microsoft\/wp-json\/wp\/v2\/users\/72741"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/engineering-at-microsoft\/wp-json\/wp\/v2\/comments?post=204"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/engineering-at-microsoft\/wp-json\/wp\/v2\/posts\/204\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/engineering-at-microsoft\/wp-json\/wp\/v2\/media\/170"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/engineering-at-microsoft\/wp-json\/wp\/v2\/media?parent=204"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/engineering-at-microsoft\/wp-json\/wp\/v2\/categories?post=204"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/engineering-at-microsoft\/wp-json\/wp\/v2\/tags?post=204"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}