Kubernetes tasks & Service Connections
Azure DevOps supports Kubernetes deployments with a number of included tasks:
These tasks can be configured to target a Kubernetes cluster in a number of ways, using the connectionType
property:
Connection Type | Description |
---|---|
Azure Resource Manager | Let’s you select an AKS instance. Does not access Kubernetes cluster at Service Connection configuration time. |
Kubernetes Service Connection | Allows you to provide a KubeConfig file, specify a Service Account, or import an AKS instance with the ‘Azure Subscription’ option. The latter requires Kubernetes cluster access at Service Connection configuration time. |
None | Use a pre-created Kubernetes configuration stored locally. |
Kubernetes Service Connection limitations when accessing AKS
You can create a Kubernetes Service Connection with any of the below options:
When selecting the ‘Azure Subscription’ option, Kubernetes needs to be accessible to Azure DevOps at Service Connection configuration time. There may be various reasons Service Connection cannot be created, e.g. you created a private cluster or the cluster has local accounts disabled. In these cases, Azure DevOps is unable to connect to your cluster at Service Connection configuration time and you will observe the dialog to be stuck at ‘Loading namespaces’:
Since Kubernetes 1.24, long-lived tokens are no longer created by default. Kubernetes recommends not to use long-lived tokens. As a result, tasks using a Kubernetes Service Connection created using the Azure Subscription option do not have access to the permanent token required to authenticate and can’t access your Kubernetes cluster. This also results in the ‘Loading namespaces’ dialog to hang.
Use the Azure Resource Manager Service Connection to access AKS
We have added support for the Azure Resource Manager Service Connection type to tasks that did not support it yet:
- AzureFunctionOnKubernetes@1
- KubernetesManifest@1
With these updated tasks, all included Kubernetes tasks can use an Azure Resource Manager Service Connection. For AKS customers, the Azure Resource Manager Service Connection type provides the best method to connect to a private cluster, or a cluster that has local accounts disabled. This method is not dependent on cluster connectivity at the time you create a Service Connection. Access to AKS is deferred to pipeline runtime, which has the following advantages:
- Access to a (private) AKS cluster can be performed from a Self-hosted or Scale set agent with line of sight to the cluster.
- A token is created for every task that uses Azure Resource Manager Service Connection. This ensures you are connecting to Kubernetes with a short-lived token, which is the Kubernetes recommendation.
- AKS can be accessed even when local accounts are disabled.
The following example demonstrates the use of the Azure Resource Manager Service Connection:
- task: AzureFunctionOnKubernetes@1
displayName: 'Deploy an Azure Function on Kubernetes'
inputs:
connectionType: 'Azure Resource Manager'
azureSubscriptionConnection: '$(subscriptionConnection)'
azureResourceGroup: $(resourceGroupName)
kubernetesCluster: $(aksName)
...
Common questions & answers
- Q: I receive the following error message: Could not find any secret associated with the service account. What is happening?
A: You are using the Kubernetes Service Connection with Azure Subscription option. We are updating this method to create long-lived tokens. This is expected to be available mid-May. However, it is recommended to start using the Azure Service Connection type and don’t use long-lived tokens as per Kubernetes guidance. - Q: I’m using AKS and don’t want to change anything, can I continue to use tasks with the Kubernetes Service Connection as before?
A: We are updating this method to create long-lived tokens. This is expected to be available mid-May. However, please be aware that this approach is against Kubernetes guidance. - Q: I’m using the Kubernetes tasks and Kubernetes Service Connection but not AKS. Should I be concerned?
A: You tasks will continue to work as before. - Q: Will the Kubernetes Service Connection type be removed?
A: Our Kubernetes tasks work with any Kubernetes cluster, regardless where they are running. The Kubernetes Service Connection will continue to exist. - Q: I’m an AKS customer and everything is running fine, should I act?
A: There is no need to change anything. If you are using the Kubernetes Service Connection and selected Azure Subscription during creation, you should be aware of the Kubernetes guidance on using long-lived tokens - Q: I’m creating a Kubernetes Environment, and have no option to use Service Connections
A: In case you can’t access AKS during environment creation time, you can use an empty environment and set theconnectionType
property of the above mentioned tasks to an Azure Resource Manager Service Connection. - Q: I have AKS configured with Azure Active Directory RBAC, and my pipeline doesn’t work. Will these updates resolve that?
A: Accessing Kubernetes when AAD RBAC is enabled is unrelated to token creation and Service Connection usage.
Update: The KuberentesManifest@1, HelmDeploy@0, Kubernetes@1 and AzureFunctionOnKubernetes@1 tasks have been updated to support kubelogin. To ensure kubelogin is available, a new task KubeloginInstaller@0 has been added. See Release Notes.
@Eric thank you for the post as we are having these issues right now. I read the section "Use the Azure Resource Manager Service Connection to access AKS" which states at the end that "AKS can be accessed even when local accounts are disabled." but I think this conflicts with the Q&A where you say "A: Accessing Kubernetes when AAD RBAC is enabled is unrelated to token creation. To prevent an interactive prompt, we will...
Hi @Chandler. Full AAD RBAC requires kubelogin, which is currently being implemented. I will publish a new blog post once we are ready, as using this will require pipeline changes.
Local accounts can be disabled independently from AAD RBAC configuration, though many users will this in combination.
I also found a few Github issues that are confirming that it did not previously work to connect to a AKS cluster with local accounts disabled, but I’m not sure if this has since been resolved as this post implies to me:
https://github.com/Azure/kubelogin/issues/20
https://github.com/microsoft/azure-pipelines-tasks/issues/17486
https://github.com/microsoft/azure-pipelines-tasks/issues/15802
https://github.com/microsoft/azure-pipelines-tasks/issues/10022