{"id":36636,"date":"2019-05-14T06:00:59","date_gmt":"2019-05-14T13:00:59","guid":{"rendered":"http:\/\/devblogs.microsoft.com\/premier-developer\/?p=36636"},"modified":"2019-05-06T07:51:10","modified_gmt":"2019-05-06T14:51:10","slug":"azure-devops-pipelines-leveraging-owasp-zap-in-the-release-pipeline","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/premier-developer\/azure-devops-pipelines-leveraging-owasp-zap-in-the-release-pipeline\/","title":{"rendered":"Azure DevOps Pipelines: Leveraging OWASP ZAP in the Release Pipeline"},"content":{"rendered":"<p>In this blog App Dev Manager <strong>Francis Lacroix<\/strong> shows how to integrate OWASP ZAP within a Release pipeline, leveraging Azure Container Instances, and publish these results to Azure DevOps Test Runs.<\/p>\n<hr \/>\n<p>As part of an organization\u2019s automated Release pipeline, it is important to include security scans and report on the results of these scans. One tool used in the industry is the <a href=\"https:\/\/www.owasp.org\/index.php\/OWASP_Zed_Attack_Proxy_Project\">OWASP Zed Attack Proxy (ZAP)<\/a>. In this blog, we will integrate OWASP ZAP within a Release pipeline, leveraging Azure Container Instances, and publish these results to Azure DevOps Test Runs.<\/p>\n<h2>Assumptions<\/h2>\n<p>As this work is based on a PoC for a Premier Developer customer, this solution presented operates within certain assumptions.<\/p>\n<ol>\n<li><strong>Leverage ACI to host OWASP ZAP on demand<\/strong>. The customer did not want to maintain an IaaS based installed of OWASP ZAP, nor did they have an AKS cluster to deploy the OWASP ZAP container into. They wanted an on-demand deployment to minimize management overhead of the security scanning tool.<\/li>\n<li><strong>Import the scan results into Azure DevOps Test Runs<\/strong>. Since the customer already leverages Azure DevOps for automated test runs, they wanted the results of the OWASP ZAP scan in the same tool to present a single view of all test results.<\/li>\n<li><strong>Run on a Microsoft Hosted Windows agent<\/strong>. The customer did not want to manage their own self-hosted agent(s), and requested this be done on a Windows based (VS 2017 at the time) agent.<\/li>\n<\/ol>\n<h2>Issues and Limitations<\/h2>\n<p>Based on the above assumptions, there were a few issues and limitations to overcome:<\/p>\n<ol>\n<li>Due to how ACI handles NATing, OWASP ZAP wasn\u2019t able to bind to the container\u2019s public address (know OWASP issue: see <a href=\"https:\/\/github.com\/zaproxy\/zap-core-help\/wiki\/HelpUiDialogsOptionsLocalproxy\">Behind NAT<\/a>). As such, we were not able to leverage the API, which made us unable to use the task available in the Marketplace. An IaaS base solution could simply use the Talk in the Marketplace.<\/li>\n<li>OWASP ZAP\u2019s report format is not natively supported by the PublishTestResults task. As such, we needed to convert it to a compatible format. A few options are available, we chose to use an XSL Template to convert it to a Nunit3 formatted results file.<\/li>\n<li>The work presented here is part of a Release Pipeline based on the customer needs. However, if it is to be reused in multiple pipelines, it would make more sense to set it up as a <a href=\"https:\/\/docs.microsoft.com\/en-us\/azure\/devops\/pipelines\/library\/task-groups?view=azure-devops\">Task Group<\/a>.<\/li>\n<\/ol>\n<h2>Building the Release Pipeline<\/h2>\n<p>The Release Pipeline itself is fairly simple. In our example, we will have one Artifact, which is an Azure Git artifact containing only the XSLTemplate used to transform the results file for publishing.<\/p>\n<p><img decoding=\"async\" width=\"663\" height=\"429\" class=\"wp-image-36637\" src=\"http:\/\/devblogs.microsoft.com\/premier-developer\/wp-content\/uploads\/sites\/31\/2019\/05\/word-image-15.png\" srcset=\"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-content\/uploads\/sites\/31\/2019\/05\/word-image-15.png 663w, https:\/\/devblogs.microsoft.com\/premier-developer\/wp-content\/uploads\/sites\/31\/2019\/05\/word-image-15-300x194.png 300w\" sizes=\"(max-width: 663px) 100vw, 663px\" \/><\/p>\n<p>We have several Variables set on the Pipeline:<\/p>\n<ul>\n<li>ACI_RESOURCE_GROUP: The name of the Group the ACI instance will be deployed to.<\/li>\n<li>ACI_LOCATION: The geographical location to deploy to<\/li>\n<li>ACI_INSTANCE_NAME: The name of the actual ACI instance deployed in Azure<\/li>\n<li>ACI_STORAGE_ACCOUNT_NAME: The name of the Storage Account to hold the file share used to download the scan results report (more details below).<\/li>\n<li>ACI_SHARE_NAME: The name of the share where the report will be stored<\/li>\n<li>TARGET_SCAN_ADDRESS: The URL for OWASP ZAP to scan<\/li>\n<\/ul>\n<p><img decoding=\"async\" width=\"657\" height=\"277\" class=\"wp-image-36638\" src=\"http:\/\/devblogs.microsoft.com\/premier-developer\/wp-content\/uploads\/sites\/31\/2019\/05\/word-image-16.png\" srcset=\"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-content\/uploads\/sites\/31\/2019\/05\/word-image-16.png 657w, https:\/\/devblogs.microsoft.com\/premier-developer\/wp-content\/uploads\/sites\/31\/2019\/05\/word-image-16-300x126.png 300w\" sizes=\"(max-width: 657px) 100vw, 657px\" \/><\/p>\n<h2>Defining the Release Pipeline<\/h2>\n<p>Once the application portion of the Release pipeline has been configured, the security scan portion can be defined. In our example, this consists of 8 tasks, primarily using the Azure CLI task to create and use the ACI instance (and supporting structures).<\/p>\n<p>Otherwise specified, all the Azure CLI tasks are Inline tasks, using the default configuration options.<\/p>\n<p><img decoding=\"async\" width=\"636\" height=\"461\" class=\"wp-image-36639\" src=\"http:\/\/devblogs.microsoft.com\/premier-developer\/wp-content\/uploads\/sites\/31\/2019\/05\/word-image-17.png\" srcset=\"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-content\/uploads\/sites\/31\/2019\/05\/word-image-17.png 636w, https:\/\/devblogs.microsoft.com\/premier-developer\/wp-content\/uploads\/sites\/31\/2019\/05\/word-image-17-300x217.png 300w\" sizes=\"(max-width: 636px) 100vw, 636px\" \/><\/p>\n<h2>Create Resource Group (if not created)<\/h2>\n<p>This task simply creates (if it doesn\u2019t already exist) the Resource Group which all of the other services will be created in. It leverages the variables defined above and has a simple inline script.<\/p>\n<pre class=\"lang:default decode:true \">rem Create the resource group \r\naz group create -l %ACI_LOCATION% -n %ACI_RESOURCE_GROUP%<\/pre>\n<p>&nbsp;<\/p>\n<h2>Create Storage Account (if not created)<\/h2>\n<p>Similar to the previous task, this task simply create the Storage Account and File Share to be used with the OWAP ZAP Container Instance. This File Share will be mounted in the container instance and used to save the test results file generated by the security scan. The file will then be downloaded to be transformed and published to Azure DevOps Test Runs, as well as kept in archive for audit purposes.<\/p>\n<pre class=\"lang:default decode:true \">rem Create the storage account with the parameters\r\ncall az storage account create -g %ACI_RESOURCE_GROUP% -n %ACI_STORAGE_ACCOUNT_NAME% -l %ACI_LOCATION% --sku Standard_LRS\r\n\r\nrem Create the file share\r\ncall az storage share create -n %ACI_SHARE_NAME% --account-name %ACI_STORAGE_ACCOUNT_NAME%\r\n<\/pre>\n<p>&nbsp;<\/p>\n<h2>Create OWASP Container<\/h2>\n<p>This task does two things:<\/p>\n<ol>\n<li>Gets the storage key used to mount the File Share to the container<\/li>\n<li>Creates the actual OWASP ZAP container instance, based on the \u201czap2docker-stable\u201d image in the Docker repository. Note: The container is created with a public IP address, but it will not be used in this example. This is for reference purposes only.<\/li>\n<\/ol>\n<p>When creating the container instance, note the \u201c&#8211;azure-file-volume-account-name\u201d, \u201c&#8211;azure-file-volume-account-key\u201d, \u201c&#8211;azure-file-volume-share-name\u201d and \u201c&#8211;azure-file-volume-mount-path\u201d. These are used to mount the File Share specified in the variables (and created in the previous task) as \u201c\/zap\/wrk\u201d in the container instance. This is the location the scan reports a written to in the image.<\/p>\n<pre class=\"lang:default decode:true \">rem Get the storage key\r\ncall az storage account keys list -g %ACI_RESOURCE_GROUP% --account-name %ACI_STORAGE_ACCOUNT_NAME% --query \"[0].value\" --output tsv &gt; temp.txt\r\nset \/p STORAGE_KEY=&lt;temp.txt\r\n\r\nrem Create the container\r\ncall az container create -g %ACI_RESOURCE_GROUP% -n %ACI_INSTANCE_NAME% --image owasp\/zap2docker-stable --ip-address public --ports 8080 --azure-file-volume-account-name %ACI_STORAGE_ACCOUNT_NAME% --azure-file-volume-account-key %STORAGE_KEY% --azure-file-volume-share-name %ACI_SHARE_NAME% --azure-file-volume-mount-path \/zap\/wrk\/ --command-line \"zap.sh -daemon -host 0.0.0.0 -port 8080 -config api.key=abcd -config api.addrs.addr.name=.* -config api.addrs.addr.regex=true\"\r\n<\/pre>\n<p>&nbsp;<\/p>\n<h2>Call the Baseline Scan<\/h2>\n<p>Once the container is created, the baseline scan will be called. OWASP ZAP offers a <a href=\"https:\/\/github.com\/zaproxy\/zaproxy\/wiki\/ZAP-Baseline-Scan\">Baseline Scan<\/a> as part of their Docker image. The <a href=\"https:\/\/github.com\/Grunny\/zap-cli\">ZAP CLI<\/a> would also be an option if the Baseline is not sufficient.<\/p>\n<p>The -x parameter will generate the XML report in the location mapped to the File Share above. We use the default config settings, but custom configurations could be provided through the file share.<\/p>\n<pre class=\"lang:default decode:true \">rem Execute the baseline scan\r\nset \"ZAP_COMMAND=\"\/zap\/zap-baseline.py -t %TARGET_SCAN_ADDRESS% -x OWASP-ZAP-Report.xml\"\"\r\naz container exec -g %ACI_RESOURCE_GROUP% -n %ACI_INSTANCE_NAME% --exec-command %ZAP_COMMAND%\r\n<\/pre>\n<p>&nbsp;<\/p>\n<h2>Download the file<\/h2>\n<p>This task will download the \u201cOWASP-ZAP-Report.xml\u201d report to the local agent for conversion and publishing.<\/p>\n<pre class=\"lang:default decode:true \">rem Get the storage key\r\ncall az storage account keys list -g %ACI_RESOURCE_GROUP% --account-name %ACI_STORAGE_ACCOUNT_NAME% --query \"[0].value\" --output tsv &gt; temp.txt\r\nset \/p STORAGE_KEY=&lt;temp.txt\r\nrem Download the file\r\ncall az storage file download --account-name %ACI_STORAGE_ACCOUNT_NAME% --account-key %STORAGE_KEY% -s %ACI_SHARE_NAME% -p OWASP-ZAP-Report.xml --dest %SYSTEM_DEFAULTWORKINGDIRECTORY%\\OWASP-ZAP-Report.xml\r\n<\/pre>\n<p>&nbsp;<\/p>\n<h2>Convert Report Format<\/h2>\n<p>Instead of an Azure CLI task, this will be a PowerShell task to convert the OWASP ZAP report from its native format to a format (in this case, I chose NUnit3 since it was the closest) which can be uploaded to Azure DevOps Test Runs.<\/p>\n<p>The script itself is fairly simple, but it relies on the template which can be found here: <a href=\"https:\/\/dev.azure.com\/francislacroix\/_git\/CodeShare?path=%2FOWASPBlog%2FOWASPToNUnit3.xslt\">https:\/\/dev.azure.com\/francislacroix\/_git\/CodeShare?path=%2FOWASPBlog%2FOWASPToNUnit3.xslt<\/a><\/p>\n<p>The script itself is straightforward, set as Inline while leaving the rest of the parameters to their default value:<\/p>\n<pre class=\"lang:default decode:true \">$XslPath = \"$($Env:SYSTEM_DEFAULTWORKINGDIRECTORY)\\_XSLTemplateFile\\OWASPToNUnit3.xslt\"\r\n$XmlInputPath = \"$($Env:SYSTEM_DEFAULTWORKINGDIRECTORY)\\OWASP-ZAP-Report.xml\"\r\n$XmlOutputPath = \"$($Env:SYSTEM_DEFAULTWORKINGDIRECTORY)\\Converted-OWASP-ZAP-Report.xml\"\r\n$XslTransform = New-Object System.Xml.Xsl.XslCompiledTransform\r\n$XslTransform.Load($XslPath)\r\n$XslTransform.Transform($XmlInputPath, $XmlOutputPath)\r\n<\/pre>\n<p>&nbsp;<\/p>\n<h2>Publish Test Results<\/h2>\n<p>Now that the results have been converted, we can publish them using the built-in \u201cPublish Test Results\u201d task. After adding it, set the following parameters:<\/p>\n<ul>\n<li>Test Result Format: NUnit<\/li>\n<li>Test Result files: Converted-OWASP-ZAP-Report.xml<\/li>\n<li>Search Folder: $(System.DefaultWorkingDirectory)<\/li>\n<li>Test run title: OWASP Tests<\/li>\n<\/ul>\n<p>Adjust the \u201cTest Result files\u201d if there are additional tests in the Release, or if the name was edited in the conversion script.<\/p>\n<h2>Destroy OWASP Container<\/h2>\n<p>Once all the scans are completed, the Container Instance can be destroyed. This is again an inline script using default settings.<\/p>\n<pre class=\"lang:default decode:true \">az container delete -g %ACI_RESOURCE_GROUP% -n %ACI_INSTANCE_NAME% --yes<\/pre>\n<p>&nbsp;<\/p>\n<h2>Closing Comments<\/h2>\n<p>This is a fairly simple use case for integrating OWASP ZAP in a Release pipeline, but the same concepts can be used for many other tools. I hope this helps you improve your automation and security of your software.<\/p>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>As part of an organization\u2019s automated Release pipeline, it is important to include security scans and report on the results of these scans. One tool used in the industry is the OWASP Zed Attack Proxy (ZAP). <\/p>\n","protected":false},"author":582,"featured_media":37840,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[25,22,1],"tags":[153,2571,3],"class_list":["post-36636","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-azure","category-devops","category-permierdev","tag-azure-container-service","tag-azure-devops","tag-team"],"acf":[],"blog_post_summary":"<p>As part of an organization\u2019s automated Release pipeline, it is important to include security scans and report on the results of these scans. One tool used in the industry is the OWASP Zed Attack Proxy (ZAP). <\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/posts\/36636","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/users\/582"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/comments?post=36636"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/posts\/36636\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/media\/37840"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/media?parent=36636"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/categories?post=36636"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/tags?post=36636"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}