How do you know your tests actually exercise your code? Perhaps you’re using a code coverage tool like coverage.py or the tools built into Visual Studio. Codecov helps you track code coverage: how much of your code is covered, and are you getting more coverage over time? By integrating Codecov into your continuous integration (CI) pipeline, you’ll get great reports that help you improve your coverage. You can read more about Codecov on their features page.
Using Codecov
It’s really easy to get started. Codecov offers a results uploader you can run without having to install any additional tools. The script figures out what language your project is written in, where the coverage results are, and how to get them to the service. Typically when you integrate outside services into your CI pipeline, you also need to manage one or more secrets. These secrets – think passwords and certificates – are required to securely talk to the outside service.
Codecov has a clever feature in their Bash uploader: tokenless upload for public pipelines. By doing a little extra validation on their side, Codecov saves users the trouble of managing secrets. This only works for public projects on specific CI providers, and I’m happy to announce that now, Azure Pipelines is one of them. (If you have private pipelines on Azure Pipelines or anyplace else, you can still use Codecov and will need to manage a token.)
Connecting with Azure Pipelines
To give it a try for yourself, clone one of Codecov’s example repos. I like Python, so that’s where I started. It’s a toy Python project with two methods and one test. You’ll have to change two things about that example repo:
- Replace .travis.yml with an Azure Pipelines YAML definition.
- Instead of using the Python
codecov
uploader, rely on the Bash uploader.
Here’s my example Azure Pipelines file:
trigger:
- master
pool:
vmImage: 'ubuntu-latest'
strategy:
matrix:
Python35:
python.version: '3.5'
Python36:
python.version: '3.6'
Python37:
python.version: '3.7'
steps:
- task: UsePythonVersion@0
inputs:
versionSpec: '$(python.version)'
displayName: 'Use Python $(python.version)'
- script: |
python -m pip install --upgrade pip
pip install coverage
displayName: 'Install coverage'
- script: |
coverage run tests.py
displayName: 'Run tests'
- script: |
bash <(curl -s https://codecov.io/bash)
displayName: 'Upload to codecov.io'
Showing it off
After I install the Codecov GitHub app and run this pipeline, Codecov handles the rest for me. Here’s an example of the graphs it generates:
I get ongoing coverage reports without having to add or manage secrets in my pipeline. Thanks to the folks over at Codecov for adding Azure Pipelines to their tokenless upload infrastructure.
0 comments