Introduction

While Azure managed identities are the preferred authentication method to avoid storing secrets, organizations still need effective strategies for managing existing secrets in Azure KeyVault.

For the purpose of managing secrets in KeyVault, there is a need to:

• Detail in plain language what the purpose of each secret is so that stakeholders can understand it.
• Describe the process used to generate the secret so that we can update the secret as necessary.
• Define the rotation strategy for each secret and indicate whether rotation will require downtime.

Implementing tagging on KeyVault secrets addresses these requirements effectively.

• Users with Azure Key Vault access can view secret tags in the Azure Portal to understand each secret’s purpose. We can enforce this as a standard via Azure Policy applied on the Key Vault resource.
• Bash or PowerShell scripts can retrieve secret tags to generate dynamic metadata views and automate secrets rotation.
• Additionally, we can auto-generate documentation like Markdown files for secrets inventory in security plans.

If you have an existing Azure KeyVault and you are tagging secrets for the first time, one outcome of carrying out a tagging exercise is that we can identify secrets that are no longer in use. This is an important step because removing unused secrets means we are minimizing the risk of access to a specific resource. We will discuss how we approach this in Removing unused Secrets.

Notes

• This tagging strategy applies only to Azure Key Vault secrets, since other secrets management tools may lack both tagging capabilities and command-line interface support for automation.
• This approach using Bash or PowerShell adds maintenance overhead, but enables consistent enforcement and automation.

Tagging Strategy

Here are the proposed keys for implementing a tagging strategy that you can apply on your Azure KeyVault secrets.

Source types

The source type should inform the process used to generate the secret. Here are some examples for source types. As shown, the following contain links to documentation on Microsoft Learn which shows how a user can regenerate the secret.

• storage-blob-sas: Azure Blob Storage Shared Access Signature, see: Grant limited access to data with shared access signatures (SAS) – Azure Storage | Microsoft Learn
• storage-connection-string: Azure Storage Key, see: Manage account access keys – Azure Storage | Microsoft Learn
• acr-username: Azure Container Registry username Azure Container Registry Authentication Options Explained – Azure Container Registry | Microsoft Learn
• acr-password: Azure Container Registry password Azure Container Registry Authentication Options Explained – Azure Container Registry | Microsoft Learn

Automation scripts

The following is an example of a PowerShell script that a user can invoke to generate a view of all Azure KeyVault secrets as a reference. It is required for Azure CLI to be installed and the user is logged into an Azure Subscription.

<#
.SYNOPSIS
    Generates a view of Azure KeyVault secrets.

.DESCRIPTION
    You can generate a table view of all secrets specified in a resource group or a filtered view of matching secrets by providing a name with more details.

.PARAMETER ResourceGroups
    Specifies the resource groups to be processed. This parameter is mandatory and should contain a comma-separated list of resource group names.

.PARAMETER Name
    Specifies the name of a specific secret to process. If this parameter is provided, the script will focus on this particular secret.

.PARAMETER GenerateMarkDown
    If this switch is provided, the script will generate a Markdown output of the secrets.

.EXAMPLE
    .\script.ps1 -ResourceGroups "ResourceGroup1,ResourceGroup2" -GenerateMarkDown
    This example processes all secrets in the specified resource groups and generates a Markdown report.

.NOTES
    Be sure to login via Azure CLI.
#>
param(
    [Parameter(Mandatory = $true)][string]$ResourceGroups,  
    [Parameter(Mandatory = $false)][string]$Name,
    [switch]$GenerateMarkDown)

function  CreateSecret {
    param (
        $VaultName,
        $secret,
        [switch]$shouldTruncate,
        [switch]$fullDetail
    )
    $purpose = "NOT SET"
    $sourceType = "NOT SET"
    $shouldRotate = "NOT SET"
    $updateDowntime = "NOT SET"
    $usage = "NOT SET"
    $rotationPolicy = "NOT SET"

    Write-Host "processing secret:" $secret.name " vault:" $VaultName

    if ($secret.tags."purpose") {

        $purpose = $secret.tags."purpose"

        if (-not $GenerateMarkDown -and $shouldTruncate -and $purpose.Length -gt 12) {
            $purpose = $purpose.Substring(0, 12) + "..."
        }
    }
    if ($secret.tags."source-type") {
        $sourceType = $secret.tags."source-type"
    }
    $versions = az keyvault secret list-versions --name $secret.name --vault-name $VaultName | ConvertFrom-Json
    $secretCreated = [datetime]::Parse($versions[0].attributes.created)
    $activeDays = ([datetime]::Today - $secretCreated).Days

    if ($secret.tags."rotation-policy") {
        $rotationPolicy = $secret.tags."rotation-policy"
        # test if $secret.tags."rotation-policy" is an int or string
        if ($secret.tags."rotation-policy" -is [int]) {            
            if ($activeDays -gt [int]$secret.tags."rotation-policy") {
                $shouldRotate = "Yes"
            }
            else {
                $shouldRotate = "No"
            }

        }
        else {
            $shouldRotate = "Unknown"
        }
    }                 
    if ($secret.tags."update-downtime") {
        if ($secret.tags."update-downtime" -eq "true") {
            $updateDowntime = "Yes"
        }  
        if ($secret.tags."update-downtime" -eq "false") {
            $updateDowntime = "No"
        }
    }   
    if ($secret.tags."source-id") {
        $usage = $secret.tags."source-id"
        if (-not $GenerateMarkDown -and $shouldTruncate -and $usage.Length -gt 12) {
            $usage = $usage.Substring(0, 12) + "..."
        }
        $usage = "Src: $usage" 
    } 

    if ($secret.tags."destination-id") {
        $destination = $secret.tags."destination-id"
        if (-not $GenerateMarkDown -and $shouldTruncate -and $destination.Length -gt 12) {
            $destination = $destination.Substring(0, 12) + "..."
        }

        if ($usage -eq "NOT SET") {
            $usage = "Dest: $destination" 
        }
        else {
            $usage = "$usage /Dest: $destination" 
        }
    } 

    $expires = "Indefinite"
    if ($secret.attributes.expires) {
        $expires = $secret.attributes.expires
    }  

    $item = [PSCustomObject]@{
        Name           = $secret.name
        ShouldRotate   = "$shouldRotate ($activeDays days)"
        SourceType     = $sourceType
        DownTime       = $updateDowntime
        Expires        = $expires     
        Purpose        = $purpose
        Usage          = $usage
        RotationPolicy = $rotationPolicy                                       
    }

    if ($fullDetail) {
        $item | Add-Member -MemberType NoteProperty -Name "Created" -Value $secret.attributes.created
        $item | Add-Member -MemberType NoteProperty -Name "Enabled" -Value $secret.attributes.enabled
    }
    $item | Add-Member -MemberType NoteProperty -Name "KeyVaultName" -Value $VaultName
    return $item
}

$items = @()
$ResourceGroupNames = $ResourceGroups -split ","
foreach ($ResourceGroup in $ResourceGroupNames) {
    $keyVaults = az keyvault list --resource-group $ResourceGroup -o json | ConvertFrom-Json
    foreach ($keyVault in $keyVaults) {
        $secrets = az keyvault secret list --vault-name $keyVault.name -o json | ConvertFrom-Json
        foreach ($secret in $secrets) {

            if ($secret.name -eq $Name) {
                $item = CreateSecret -VaultName $keyVault.name -secret $secret -fullDetail
                $item | Format-List
                break
            }
            $item = CreateSecret -VaultName $keyVault.name -secret $secret -shouldTruncate            
            $items += $item
        }
    }
}

if ($GenerateMarkDown) {
    $markdown = "|Name|What is its purpose?|Where does it live?|How was it generated?|What's the rotation strategy?|Does it cause downtime?|How does the secret get distributed to consumers?|What's the secret's lifespan?|`n"
    $markdown += "|---|--------------------|-------------------|---------------------|-----------------------------|-----------------------|-------------------------------------------------|----------------------------|`n"
    # Name 
    # What is its purpose?
    # Where does it live?
    # How was it generated?
    # What's the rotation strategy? 
    # Does it cause downtime? 
    # How does the secret get distributed to consumers?
    # What’s the secret’s lifespan?
    foreach ($item in $items) {
        $markdown += "|$($item.Name)|$($item.Purpose)|$($item.KeyVaultName)|$($item.SourceType)|$($item.RotationPolicy)|$($item.DownTime)||$($item.Expires)|`n"
    }
    $markdown
    return
}

if (!$Name ) {
    $items | Format-Table -AutoSize
}

The following is an example of a PowerShell script that a user can use to tag secrets.

<#
.SYNOPSIS
    Tags Azure KeyVault secrets with specified metadata.

.DESCRIPTION
    This script tags Azure KeyVault secrets with metadata such as purpose, rotation policy, update downtime, source type, and more. 
    It processes secrets in the specified resource groups and updates the tags accordingly.

.PARAMETER ResourceGroups
    Specifies the resource groups to be processed. This parameter is mandatory and should contain a comma-separated list of resource group names.

.PARAMETER Name
    Specifies the name of a specific secret to process. This parameter is mandatory.

.PARAMETER Purpose
    Specifies the purpose of the secret. This parameter is mandatory.

.PARAMETER RotationPolicy
    Specifies the rotation policy of the secret. This parameter is mandatory.

.PARAMETER UpdateDowntime
    Specifies whether updating the secret causes downtime. This parameter is mandatory.

.PARAMETER SourceType
    Specifies the source type of the secret. This parameter is mandatory.

.PARAMETER SourceOther
    Specifies additional source information if the source type is 'other'. This parameter is optional.

.PARAMETER SourceId
    Specifies the name of the resource or the full resource Id where secret is for/used. This parameter is optional.

.PARAMETER DestinationId
    Specifies the name of the resource or the full resource Id where secret is for/used. This parameter is optional.

.EXAMPLE
    .\script.ps1 -ResourceGroups "ResourceGroup1,ResourceGroup2" -Name "SecretName" -Purpose "ExamplePurpose" -RotationPolicy "30" -UpdateDowntime "false" -SourceType "storage-blob-sas"
    This example tags the specified secret in the specified resource groups with the provided metadata.

.NOTES
    Be sure to login via Azure CLI.
#>
param(
    [Parameter(Mandatory = $true)][string]$ResourceGroups,
    [Parameter(Mandatory = $true)][string]$Name,
    [Parameter(Mandatory = $true)][string]$Purpose,
    [Parameter(Mandatory = $true)][string]$RotationPolicy,
    [Parameter(Mandatory = $true)][string]$UpdateDowntime,
    [Parameter(Mandatory = $true)][string]$SourceType,
    [Parameter(Mandatory = $false)][string]$SourceOther,
    [Parameter(Mandatory = $false)][string]$SourceId,    
    [Parameter(Mandatory = $false)][string]$DestinationId)

$ResourceGroupNames = $ResourceGroups -split ","
foreach ($ResourceGroup in $ResourceGroupNames) {
    $keyVaults = az keyvault list --resource-group $ResourceGroup -o json | ConvertFrom-Json
    foreach ($keyVault in $keyVaults) {
        $secrets = az keyvault secret list --vault-name $keyVault.name -o json | ConvertFrom-Json
        foreach ($secret in $secrets) {
            if ($Name -eq $secret.name) {
                $change = @()
                $change += "purpose=$Purpose"
                $change += "rotation-policy=$RotationPolicy"
                $change += "update-downtime=$UpdateDowntime"
                $change += "source-type=$SourceType"

                if ($SourceOther) {
                    $change += "source-other=$SourceOther"
                }
                else {
                    if ($secret.tags."source-other") {
                        $value = $secret.tags."source-other"
                        $change += "source-other=$value"
                    }
                }

                if ($SourceId) {
                    $change += "source-id=$SourceId"
                }
                else {
                    if ($secret.tags."source-id") {
                        $value = $secret.tags."source-id"
                        $change += "source-id=$value"
                    }
                }

                if ($DestinationId) {
                    $change += "destination-id=$DestinationId"
                }
                else {
                    if ($secret.tags."destination-id") {
                        $value = $secret.tags."destination-id"
                        $change += "destination-id=$value"
                    }
                }

                Write-Host "Updating $Name"
                # update secret with new tags
                $log = az keyvault secret set-attributes --vault-name $keyVault.name --name $Name --tags $change
                if ($LASTEXITCODE -ne 0) {
                    throw "Failed to update secret $Name - $log" 
                }
                else {
                    Write-Host "Updated tags for secret $Name"
                }
                return
            }
        }
    }
}

Removing unused Secrets

During the tagging process, you may identify secrets that are not well known to you or others. It can be helpful to know if those secrets are still being accessed and by whom. This can be done by enabling auditing on Azure KeyVault under Diagnostic setting, choosing the audit category group and selecting a destination of your choice to persist the logs.

Once enabled, you can take a few weeks to review the logs. If you are using Log Analytics Workspace, the following Kusto Query Language (KQL) script can help show the name of the secrets being accessed, the user or services accessing the secrets, whether it was successful and when the secrets are being accessed. Be sure to populate the subId for Subscription Id, rgName for resource group name and vaultName for Azure KeyVault name.

let subId = "";
let rgName = "";
let vaultName = "";

let resourceId = strcat("/SUBSCRIPTIONS/", subId, "/RESOURCEGROUPS/", rgName, "/PROVIDERS/MICROSOFT.KEYVAULT/VAULTS/", vaultName);

AzureDiagnostics | where 
ResourceId == resourceId
and Category == 'AuditEvent'
and OperationName  == 'SecretGet' 
| project 
extract("/secrets/([^/]+)/", 1, id_s), trustedService_s, identity_claim_oid_g, identity_claim_appid_g, clientInfo_s, CallerIPAddress, ResultType, TimeGenerated

These information can help you determine who to reach out to. More importantly, it also identifies secrets that are no longer accessed for a period of time which can mean you can remove those secrets. It is crucial to enable the soft delete feature to ensure that secrets can be recovered if needed. One thing to note is that there is a period of time that a secret is soft deleted for. After that period, it is no longer possible to recover the secret.

Summary

Managing secrets requires us to implement practices such as secrets rotation, monitor usage etc. With a tagging strategy in place along with automation, we can ensure we are following best practices. I hope this blog post helps you in how we can approach this.

Note: the picture that illustrates this article has been generated by AI on Bing Image Creator.