Service Connection guidance for AKS customers using Kubernetes tasks

Eric van Wijk

Kubernetes tasks & Service Connections

Azure DevOps supports Kubernetes deployments with a number of included tasks:

• AzureFunctionOnKubernetes
• HelmDeploy
• Kubernetes
• KubernetesManifest

These tasks can be configured to target a Kubernetes cluster in a number of ways, using the connectionType property:

Kubernetes Service Connection limitations when accessing AKS

You can create a Kubernetes Service Connection with any of the below options:

When selecting the ‘Azure Subscription’ option, Kubernetes needs to be accessible to Azure DevOps at Service Connection configuration time. There may be various reasons Service Connection cannot be created, e.g. you created a private cluster or the cluster has local accounts disabled. In these cases, Azure DevOps is unable to connect to your cluster at Service Connection configuration time and you will observe the dialog to be stuck at ‘Loading namespaces’:

Since Kubernetes 1.24, long-lived tokens are no longer created by default. Kubernetes recommends not to use long-lived tokens. As a result, tasks using a Kubernetes Service Connection created using the Azure Subscription option do not have access to the permanent token required to authenticate and can’t access your Kubernetes cluster. This also results in the ‘Loading namespaces’ dialog to hang.

Use the Azure Resource Manager Service Connection to access AKS

We have added support for the Azure Resource Manager Service Connection type to tasks that did not support it yet:

• AzureFunctionOnKubernetes@1
• KubernetesManifest@1

With these updated tasks, all included Kubernetes tasks can use an Azure Resource Manager Service Connection. For AKS customers, the Azure Resource Manager Service Connection type provides the best method to connect to a private cluster, or a cluster that has local accounts disabled. This method is not dependent on cluster connectivity at the time you create a Service Connection. Access to AKS is deferred to pipeline runtime, which has the following advantages:

• Access to a (private) AKS cluster can be performed from a Self-hosted or Scale set agent with line of sight to the cluster.
• A token is created for every task that uses Azure Resource Manager Service Connection. This ensures you are connecting to Kubernetes with a short-lived token, which is the Kubernetes recommendation.
• AKS can be accessed even when local accounts are disabled.

The following example demonstrates the use of the Azure Resource Manager Service Connection:

- task: AzureFunctionOnKubernetes@1
  displayName: 'Deploy an Azure Function on Kubernetes'
  inputs:
    connectionType: 'Azure Resource Manager'
    azureSubscriptionConnection: '$(subscriptionConnection)'
    azureResourceGroup: $(resourceGroupName)
    kubernetesCluster: $(aksName)
    ...

Common questions & answers

• Q: I receive the following error message: Could not find any secret associated with the service account. What is happening?
  A: You are using the Kubernetes Service Connection with Azure Subscription option. We are updating this method to create long-lived tokens. This is expected to be available mid-May. However, it is recommended to start using the Azure Service Connection type and don’t use long-lived tokens as per Kubernetes guidance.
• Q: I’m using AKS and don’t want to change anything, can I continue to use tasks with the Kubernetes Service Connection as before?
  A: We are updating this method to create long-lived tokens. This is expected to be available mid-May. However, please be aware that this approach is against Kubernetes guidance.
• Q: I’m using the Kubernetes tasks and Kubernetes Service Connection but not AKS. Should I be concerned?
  A: You tasks will continue to work as before.
• Q: Will the Kubernetes Service Connection type be removed?
  A: Our Kubernetes tasks work with any Kubernetes cluster, regardless where they are running. The Kubernetes Service Connection will continue to exist.
• Q: I’m an AKS customer and everything is running fine, should I act?
  A: There is no need to change anything. If you are using the Kubernetes Service Connection and selected Azure Subscription during creation, you should be aware of the Kubernetes guidance on using long-lived tokens
• Q: I’m creating a Kubernetes Environment, and have no option to use Service Connections
  A: In case you can’t access AKS during environment creation time, you can use an empty environment and set the connectionType property of the above mentioned tasks to an Azure Resource Manager Service Connection.
• Q: I have AKS configured with Azure Active Directory RBAC, and my pipeline doesn’t work. Will these updates resolve that?
  A: Accessing Kubernetes when AAD RBAC is enabled is unrelated to token creation and Service Connection usage.
  Update: The KuberentesManifest@1, HelmDeploy@0, Kubernetes@1 and AzureFunctionOnKubernetes@1 tasks have been updated to support kubelogin. To ensure kubelogin is available, a new task KubeloginInstaller@0 has been added. See Release Notes.

Eric van Wijk Product Manager, Azure DevOps

Follow