Azure DevOps Pipelines: Leveraging OWASP ZAP in the Release Pipeline

Premier Developer


In this blog App Dev Manager Francis Lacroix shows how to integrate OWASP ZAP within a Release pipeline, leveraging Azure Container Instances, and publish these results to Azure DevOps Test Runs.

As part of an organization’s automated Release pipeline, it is important to include security scans and report on the results of these scans. One tool used in the industry is the OWASP Zed Attack Proxy (ZAP). In this blog, we will integrate OWASP ZAP within a Release pipeline, leveraging Azure Container Instances, and publish these results to Azure DevOps Test Runs.


As this work is based on a PoC for a Premier Developer customer, this solution presented operates within certain assumptions.

  1. Leverage ACI to host OWASP ZAP on demand. The customer did not want to maintain an IaaS based installed of OWASP ZAP, nor did they have an AKS cluster to deploy the OWASP ZAP container into. They wanted an on-demand deployment to minimize management overhead of the security scanning tool.
  2. Import the scan results into Azure DevOps Test Runs. Since the customer already leverages Azure DevOps for automated test runs, they wanted the results of the OWASP ZAP scan in the same tool to present a single view of all test results.
  3. Run on a Microsoft Hosted Windows agent. The customer did not want to manage their own self-hosted agent(s), and requested this be done on a Windows based (VS 2017 at the time) agent.

Issues and Limitations

Based on the above assumptions, there were a few issues and limitations to overcome:

  1. Due to how ACI handles NATing, OWASP ZAP wasn’t able to bind to the container’s public address (know OWASP issue: see Behind NAT). As such, we were not able to leverage the API, which made us unable to use the task available in the Marketplace. An IaaS base solution could simply use the Talk in the Marketplace.
  2. OWASP ZAP’s report format is not natively supported by the PublishTestResults task. As such, we needed to convert it to a compatible format. A few options are available, we chose to use an XSL Template to convert it to a Nunit3 formatted results file.
  3. The work presented here is part of a Release Pipeline based on the customer needs. However, if it is to be reused in multiple pipelines, it would make more sense to set it up as a Task Group.

Building the Release Pipeline

The Release Pipeline itself is fairly simple. In our example, we will have one Artifact, which is an Azure Git artifact containing only the XSLTemplate used to transform the results file for publishing.

We have several Variables set on the Pipeline:

  • ACI_RESOURCE_GROUP: The name of the Group the ACI instance will be deployed to.
  • ACI_LOCATION: The geographical location to deploy to
  • ACI_INSTANCE_NAME: The name of the actual ACI instance deployed in Azure
  • ACI_STORAGE_ACCOUNT_NAME: The name of the Storage Account to hold the file share used to download the scan results report (more details below).
  • ACI_SHARE_NAME: The name of the share where the report will be stored

Defining the Release Pipeline

Once the application portion of the Release pipeline has been configured, the security scan portion can be defined. In our example, this consists of 8 tasks, primarily using the Azure CLI task to create and use the ACI instance (and supporting structures).

Otherwise specified, all the Azure CLI tasks are Inline tasks, using the default configuration options.

Create Resource Group (if not created)

This task simply creates (if it doesn’t already exist) the Resource Group which all of the other services will be created in. It leverages the variables defined above and has a simple inline script.


Create Storage Account (if not created)

Similar to the previous task, this task simply create the Storage Account and File Share to be used with the OWAP ZAP Container Instance. This File Share will be mounted in the container instance and used to save the test results file generated by the security scan. The file will then be downloaded to be transformed and published to Azure DevOps Test Runs, as well as kept in archive for audit purposes.


Create OWASP Container

This task does two things:

  1. Gets the storage key used to mount the File Share to the container
  2. Creates the actual OWASP ZAP container instance, based on the “zap2docker-stable” image in the Docker repository. Note: The container is created with a public IP address, but it will not be used in this example. This is for reference purposes only.

When creating the container instance, note the “–azure-file-volume-account-name”, “–azure-file-volume-account-key”, “–azure-file-volume-share-name” and “–azure-file-volume-mount-path”. These are used to mount the File Share specified in the variables (and created in the previous task) as “/zap/wrk” in the container instance. This is the location the scan reports a written to in the image.


Call the Baseline Scan

Once the container is created, the baseline scan will be called. OWASP ZAP offers a Baseline Scan as part of their Docker image. The ZAP CLI would also be an option if the Baseline is not sufficient.

The -x parameter will generate the XML report in the location mapped to the File Share above. We use the default config settings, but custom configurations could be provided through the file share.


Download the file

This task will download the “OWASP-ZAP-Report.xml” report to the local agent for conversion and publishing.


Convert Report Format

Instead of an Azure CLI task, this will be a PowerShell task to convert the OWASP ZAP report from its native format to a format (in this case, I chose NUnit3 since it was the closest) which can be uploaded to Azure DevOps Test Runs.

The script itself is fairly simple, but it relies on the template which can be found here:

The script itself is straightforward, set as Inline while leaving the rest of the parameters to their default value:


Publish Test Results

Now that the results have been converted, we can publish them using the built-in “Publish Test Results” task. After adding it, set the following parameters:

  • Test Result Format: NUnit
  • Test Result files: Converted-OWASP-ZAP-Report.xml
  • Search Folder: $(System.DefaultWorkingDirectory)
  • Test run title: OWASP Tests

Adjust the “Test Result files” if there are additional tests in the Release, or if the name was edited in the conversion script.

Destroy OWASP Container

Once all the scans are completed, the Container Instance can be destroyed. This is again an inline script using default settings.


Closing Comments

This is a fairly simple use case for integrating OWASP ZAP in a Release pipeline, but the same concepts can be used for many other tools. I hope this helps you improve your automation and security of your software.


Premier Developer
Premier Developer

Premier Support for Developers

Follow Premier   

Gareth Morris
Gareth Morris 2019-05-27 12:55:37
Really nice blog post but I thought would cause the az container exec step to fail, as its using arguments when calling ? I did attempt to create the above and it’s failing at that step due to this issue it appears. I’m using the latest version of azure cli. 
Jan-Rintje Apeldoorn 2019-06-19 04:13:38
I can confirm the problem as stated by the comment of Gereth Morris (Link). The main think that I'm trying to understand is exactly why adding parameters to the az container exec causes it to fail. I would expect that the executable line given would only execute in it's entirety, not in parts. But that doesn't seem to be the case.
Donovin DeCoste (ddecoste) 2019-07-04 12:26:57
Same result here as Gareth and Jan-Rintje. Running the exec command generates a error that the file is not found, it is not handling the arguments"  Curious how it may have worked for your demo above, can you explain how this would work given the response below. Any workaround to this ?
Francis Lacroix 2019-07-05 08:12:25
Good morning. You are all quite correct, this should never have worked, as per And yet it did, while it was in PoC with the customer. I can only assume that there were some changes in the APIs that temporarily allowed it to work. There is a workaround, and it ended up being simpler in a way. Rather than calling the baseline scan from "az container exec", we moved the call to "az container create". The call to the "az container create" should now look like (note the added lines): rem Create the containerset "ZAP_COMMAND="/zap/ -t %TARGET_SCAN_ADDRESS% -x OWASP-ZAP-Report.xml"" call az container create -g %ACI_RESOURCE_GROUP% -n %ACI_INSTANCE_NAME% --image owasp/zap2docker-stable --azure-file-volume-account-name %ACI_STORAGE_ACCOUNT_NAME% --azure-file-volume-account-key %STORAGE_KEY% --azure-file-volume-share-name %ACI_SHARE_NAME% --azure-file-volume-mount-path /zap/wrk/ --command-line %ZAP_COMMAND% sleep 30 Three quick notes: 1) This works for this use case since the scan is only executed once, and the ACI is detroyed after the scan. If you have a use case where you want to run multiple scans, this may not be the approach for you. You can work around this by restarting the container, which will re-excute the command-line, but it's not the cleanest solutio. 2) Note the "sleep 30" call. The reason is simple: The "az container create" call returns once the ACI is created, not once the commandline has been executed, so we need to delay the call to download the result file. You may need to adjust if you're doing a different scan than the baseline. 3) If you have a lingering ACI from a preview failed run, I would recommend deleting it first. I had coworkers using this solution who applied the workaround without deleting the previous ACI, and they ran into odd behaviour. This was resolved with a restart of the ACI, but re-creating it will also prevent it. Once I return to the office, I'll submit an update to the article with these changes.
Reynard Asis
Reynard Asis 2019-07-15 19:42:52
Nice blog. I have one question regarding converting the output file for publishing test result.  How did you add OWASPToNUnit3.xslt to system default directory?