Azure DevOps Pipelines: Leveraging OWASP ZAP in the Release Pipeline

Developer Support

In this blog App Dev Manager Francis Lacroix shows how to integrate OWASP ZAP within a Release pipeline, leveraging Azure Container Instances, and publish these results to Azure DevOps Test Runs.

As part of an organization’s automated Release pipeline, it is important to include security scans and report on the results of these scans. One tool used in the industry is the OWASP Zed Attack Proxy (ZAP). In this blog, we will integrate OWASP ZAP within a Release pipeline, leveraging Azure Container Instances, and publish these results to Azure DevOps Test Runs.


As this work is based on a PoC for a Premier Developer customer, this solution presented operates within certain assumptions.

  1. Leverage ACI to host OWASP ZAP on demand. The customer did not want to maintain an IaaS based installed of OWASP ZAP, nor did they have an AKS cluster to deploy the OWASP ZAP container into. They wanted an on-demand deployment to minimize management overhead of the security scanning tool.
  2. Import the scan results into Azure DevOps Test Runs. Since the customer already leverages Azure DevOps for automated test runs, they wanted the results of the OWASP ZAP scan in the same tool to present a single view of all test results.
  3. Run on a Microsoft Hosted Windows agent. The customer did not want to manage their own self-hosted agent(s), and requested this be done on a Windows based (VS 2017 at the time) agent.

Issues and Limitations

Based on the above assumptions, there were a few issues and limitations to overcome:

  1. Due to how ACI handles NATing, OWASP ZAP wasn’t able to bind to the container’s public address (know OWASP issue: see Behind NAT). As such, we were not able to leverage the API, which made us unable to use the task available in the Marketplace. An IaaS base solution could simply use the Talk in the Marketplace.
  2. OWASP ZAP’s report format is not natively supported by the PublishTestResults task. As such, we needed to convert it to a compatible format. A few options are available, we chose to use an XSL Template to convert it to a Nunit3 formatted results file.
  3. The work presented here is part of a Release Pipeline based on the customer needs. However, if it is to be reused in multiple pipelines, it would make more sense to set it up as a Task Group.

Building the Release Pipeline

The Release Pipeline itself is fairly simple. In our example, we will have one Artifact, which is an Azure Git artifact containing only the XSLTemplate used to transform the results file for publishing.

We have several Variables set on the Pipeline:

  • ACI_RESOURCE_GROUP: The name of the Group the ACI instance will be deployed to.
  • ACI_LOCATION: The geographical location to deploy to
  • ACI_INSTANCE_NAME: The name of the actual ACI instance deployed in Azure
  • ACI_STORAGE_ACCOUNT_NAME: The name of the Storage Account to hold the file share used to download the scan results report (more details below).
  • ACI_SHARE_NAME: The name of the share where the report will be stored

Defining the Release Pipeline

Once the application portion of the Release pipeline has been configured, the security scan portion can be defined. In our example, this consists of 8 tasks, primarily using the Azure CLI task to create and use the ACI instance (and supporting structures).

Otherwise specified, all the Azure CLI tasks are Inline tasks, using the default configuration options.

Create Resource Group (if not created)

This task simply creates (if it doesn’t already exist) the Resource Group which all of the other services will be created in. It leverages the variables defined above and has a simple inline script.

rem Create the resource group 
az group create -l %ACI_LOCATION% -n %ACI_RESOURCE_GROUP%


Create Storage Account (if not created)

Similar to the previous task, this task simply create the Storage Account and File Share to be used with the OWAP ZAP Container Instance. This File Share will be mounted in the container instance and used to save the test results file generated by the security scan. The file will then be downloaded to be transformed and published to Azure DevOps Test Runs, as well as kept in archive for audit purposes.

rem Create the storage account with the parameters
call az storage account create -g %ACI_RESOURCE_GROUP% -n %ACI_STORAGE_ACCOUNT_NAME% -l %ACI_LOCATION% --sku Standard_LRS

rem Create the file share
call az storage share create -n %ACI_SHARE_NAME% --account-name %ACI_STORAGE_ACCOUNT_NAME%


Create OWASP Container

This task does two things:

  1. Gets the storage key used to mount the File Share to the container
  2. Creates the actual OWASP ZAP container instance, based on the “zap2docker-stable” image in the Docker repository. Note: The container is created with a public IP address, but it will not be used in this example. This is for reference purposes only.

When creating the container instance, note the “–azure-file-volume-account-name”, “–azure-file-volume-account-key”, “–azure-file-volume-share-name” and “–azure-file-volume-mount-path”. These are used to mount the File Share specified in the variables (and created in the previous task) as “/zap/wrk” in the container instance. This is the location the scan reports a written to in the image.

rem Get the storage key
call az storage account keys list -g %ACI_RESOURCE_GROUP% --account-name %ACI_STORAGE_ACCOUNT_NAME% --query "[0].value" --output tsv > temp.txt
set /p STORAGE_KEY=<temp.txt

rem Create the container
call az container create -g %ACI_RESOURCE_GROUP% -n %ACI_INSTANCE_NAME% --image owasp/zap2docker-stable --ip-address public --ports 8080 --azure-file-volume-account-name %ACI_STORAGE_ACCOUNT_NAME% --azure-file-volume-account-key %STORAGE_KEY% --azure-file-volume-share-name %ACI_SHARE_NAME% --azure-file-volume-mount-path /zap/wrk/ --command-line " -daemon -host -port 8080 -config api.key=abcd -config* -config api.addrs.addr.regex=true"


Call the Baseline Scan

Once the container is created, the baseline scan will be called. OWASP ZAP offers a Baseline Scan as part of their Docker image. The ZAP CLI would also be an option if the Baseline is not sufficient.

The -x parameter will generate the XML report in the location mapped to the File Share above. We use the default config settings, but custom configurations could be provided through the file share.

rem Execute the baseline scan
set "ZAP_COMMAND="/zap/ -t %TARGET_SCAN_ADDRESS% -x OWASP-ZAP-Report.xml""
az container exec -g %ACI_RESOURCE_GROUP% -n %ACI_INSTANCE_NAME% --exec-command %ZAP_COMMAND%


Download the file

This task will download the “OWASP-ZAP-Report.xml” report to the local agent for conversion and publishing.

rem Get the storage key
call az storage account keys list -g %ACI_RESOURCE_GROUP% --account-name %ACI_STORAGE_ACCOUNT_NAME% --query "[0].value" --output tsv > temp.txt
set /p STORAGE_KEY=<temp.txt
rem Download the file
call az storage file download --account-name %ACI_STORAGE_ACCOUNT_NAME% --account-key %STORAGE_KEY% -s %ACI_SHARE_NAME% -p OWASP-ZAP-Report.xml --dest %SYSTEM_DEFAULTWORKINGDIRECTORY%\OWASP-ZAP-Report.xml


Convert Report Format

Instead of an Azure CLI task, this will be a PowerShell task to convert the OWASP ZAP report from its native format to a format (in this case, I chose NUnit3 since it was the closest) which can be uploaded to Azure DevOps Test Runs.

The script itself is fairly simple, but it relies on the template which can be found here:

The script itself is straightforward, set as Inline while leaving the rest of the parameters to their default value:

$XmlOutputPath = "$($Env:SYSTEM_DEFAULTWORKINGDIRECTORY)\Converted-OWASP-ZAP-Report.xml"
$XslTransform = New-Object System.Xml.Xsl.XslCompiledTransform
$XslTransform.Transform($XmlInputPath, $XmlOutputPath)


Publish Test Results

Now that the results have been converted, we can publish them using the built-in “Publish Test Results” task. After adding it, set the following parameters:

  • Test Result Format: NUnit
  • Test Result files: Converted-OWASP-ZAP-Report.xml
  • Search Folder: $(System.DefaultWorkingDirectory)
  • Test run title: OWASP Tests

Adjust the “Test Result files” if there are additional tests in the Release, or if the name was edited in the conversion script.

Destroy OWASP Container

Once all the scans are completed, the Container Instance can be destroyed. This is again an inline script using default settings.

az container delete -g %ACI_RESOURCE_GROUP% -n %ACI_INSTANCE_NAME% --yes


Closing Comments

This is a fairly simple use case for integrating OWASP ZAP in a Release pipeline, but the same concepts can be used for many other tools. I hope this helps you improve your automation and security of your software.