{"id":15283,"date":"2024-02-23T08:54:14","date_gmt":"2024-02-23T16:54:14","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/ise\/?p=15283"},"modified":"2024-07-18T12:00:51","modified_gmt":"2024-07-18T19:00:51","slug":"consuming-azure-openai-resources-in-aks-with-workload-identities","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/ise\/consuming-azure-openai-resources-in-aks-with-workload-identities\/","title":{"rendered":"Consuming Azure OpenAI resources in AKS with Workload Identities"},"content":{"rendered":"<h2>An Essential Guide for Modern Developers<\/h2>\n<p>The integration of cloud technologies and artificial intelligence has seen significant advancements, notably with Azure OpenAI. As the use of OpenAI in various industries grows, secure and efficient access to these resources is paramount. Microsoft&#8217;s introduction of Workload Identities addresses this need, offering a modern approach to authentication and shifting away from traditional methods.<\/p>\n<p>Key to this development is the need for secure and reliable authentication mechanisms, especially with the increasing reliance on OpenAI technologies. Workload Identities, particularly Azure AD workload identity, provide this security and flexibility. They integrate smoothly with the Azure Identity client library and the Microsoft Authentication Library (MSAL), facilitating easy access to Azure cloud resources.<\/p>\n<p>One major benefit of Workload Identities is their role in simplifying secrets management. This process, often complex and costly, becomes more manageable and cost-effective with Workload Identities, eliminating much of the administrative burden associated with traditional secret management methods.<\/p>\n<p>Additionally, Workload Identities are versatile and compatible with various environments. They support Kubernetes clusters in diverse settings, including various clouds and on-premises, allowing for consistent authentication processes regardless of the hosting environment.<\/p>\n<p>This article will explore the benefits and practical applications of using Azure OpenAI resources in AKS with Workload Identities. It aims to provide valuable insights for both experienced developers and newcomers in the field of cloud and AI integration.<\/p>\n<ul>\n<li><a href=\"#consuming-azure-openai-resources-in-aks-with-workload-identities-an-essential-guide-for-modern-developers\">Consuming Azure OpenAI Resources in AKS with Workload Identities: An Essential Guide for Modern Developers<\/a>\n<ul>\n<li><a href=\"#project-context\">Project context<\/a><\/li>\n<li><a href=\"#pre-requisites\">Pre-requisites<\/a><\/li>\n<li><a href=\"#running-the-bot-locally\">Running the bot locally<\/a><\/li>\n<li><a href=\"#lets-create-the-aks-and-deploy-the-platform\">Lets create the AKS and deploy the platform<\/a><\/li>\n<li><a href=\"#create-the-aks-and-configure-it\">Create the AKS and configure it<\/a><\/li>\n<li><a href=\"#deploy-the-service-into-aks\">Deploy the service into AKS<\/a><\/li>\n<li><a href=\"#access-it\">Access it<\/a><\/li>\n<li><a href=\"#so-how-is-it-getting-the-token\">So how is it getting the token?<\/a><\/li>\n<li><a href=\"#1-the-two-fold-path-to-authentication\">1. The Two-Fold Path to Authentication<\/a><\/li>\n<li><a href=\"#2-setting-up-your-local-development\">2. Setting Up Your Local Development<\/a><\/li>\n<li><a href=\"#3-integrating-aad-with-python-via-langchain\">3. Integrating AAD with Python via LangChain<\/a><\/li>\n<li><a href=\"#in-conclusion\">In Conclusion<\/a><\/li>\n<li><a href=\"#aks-deployment-microsoft-entra-workload-id-with-aks\">AKS deployment. Microsoft Entra Workload ID with AKS<\/a><\/li>\n<li><a href=\"#how-it-works\">How it works<\/a><\/li>\n<li><a href=\"#service-account-annotations\">Service Account Annotations<\/a><\/li>\n<li><a href=\"#pod-labels\">Pod Labels<\/a><\/li>\n<li><a href=\"#lets-see-the-components-in-azure\">Lets see the components in Azure<\/a><\/li>\n<li><a href=\"#useful-docs\">Useful docs<\/a><\/li>\n<li><a href=\"#summary\">Summary<\/a><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<h2>Project context<\/h2>\n<p>Our project was to address a customer challenge that appears to be quite common in large companies: while a wealth of information was available to the Team Members, it was often hard to find and there were occasional concerns regarding its relevance and currency. This led to Team Members either reaching out to Support Teams for help or going directly to subject matter experts, creating inefficiencies in their operations. Collaborating closely, we developed a chatbot system, powered by Azure OpenAI and LangChain, designed to give quick answers to queries in a Q&amp;A format. The deployment had to follow the customer\u2019s production requirements, which included the use of AKS and avoiding the use of secrets. If credentials were used, there had to be a plan to rotate them frequently. The platform was required to integrate with Azure OpenAI, meaning that the chatbot needed credentials to consume the resources. The use of Azure Workload Identities in AKS allowed us to run the platform, consume Azure OpenAI resources and comply with the customer&#8217;s requirements in a secure way.<\/p>\n<h2>Pre-requisites<\/h2>\n<ul>\n<li>An Azure subscription. <a href=\"https:\/\/azure.microsoft.com\/free\/ai-services\">Create one for free<\/a><\/li>\n<li>Access granted to Azure OpenAI in the desired Azure subscription. (Currently, you must submit an application to access Azure OpenAI Service. To apply for access, complete <a href=\"https:\/\/aka.ms\/oai\/access\">this form<\/a>. If you need assistance, open an issue on this repository to contact Microsoft.)<\/li>\n<li><a href=\"https:\/\/code.visualstudio.com\/\">VScode<\/a> with <a href=\"https:\/\/marketplace.visualstudio.com\/items?itemName=ms-vscode-remote.remote-containers\">Devcontainer extension<\/a><\/li>\n<li>Clone repository <a href=\"https:\/\/github.com\/jsburckhardt\/gradio-aks\">https:\/\/github.com\/jsburckhardt\/gradio-aks<\/a><\/li>\n<li>An Azure OpenAI instance with gpt-35 or gpt4 deployments. You can follow the instructions <a href=\"https:\/\/learn.microsoft.com\/en-us\/azure\/ai-services\/openai\/how-to\/create-resource?pivots=web-portal\">here<\/a> to create the resources. As an alternative, a script to deploy resources can be found in the source code. <a href=\"https:\/\/github.com\/jsburckhardt\/gradio-aks\/blob\/main\/infra\/pre-requisites.sh\"><code>pre-requisites.sh<\/code><\/a><\/li>\n<li><a href=\"https:\/\/www.docker.com\/get-started\/\">Docker<\/a> (if running locally in Devcontainer)<\/li>\n<\/ul>\n<h2>Running the bot locally<\/h2>\n<ol>\n<li>\n<p>Open in <a href=\"https:\/\/code.visualstudio.com\/docs\/devcontainers\/containers\">devcontainer<\/a> or <a href=\"https:\/\/docs.github.com\/codespaces\">GitHub Codespaces<\/a><\/p>\n<\/li>\n<li>\n<p>Create your <code>.env<\/code> file. For my example it looks like this:<\/p>\n<pre><code class=\"language-.env\">OPENAI_API_TYPE=azure_ad\r\nOPENAI_API_BASE=&lt;openai url&gt;\r\nOPENAI_API_VERSION=2023-08-01-preview\r\nTEMPERATURE=0.9\r\nDEPLOYMENT_NAME=gpt-35-turbo<\/code><\/pre>\n<\/li>\n<li>\n<p>Instead of using the OpenAI token, we will be using the user&#8217;s roles. Run <code>az login<\/code><\/p>\n<\/li>\n<li>\n<p>Now, by default users don&#8217;t have the <code>Cognitive Services User<\/code> role. Here we will be adding the user role to the resource group hosting the Azure OpenAI<\/p>\n<pre><code class=\"language-bash\">export RG=&lt;rg with ai resource&gt;\r\nexport user=$(az ad signed-in-user show --query \"userPrincipalName\" -o tsv)\r\nexport resourceId=$(az group show -g $RG --query \"id\" -o tsv)\r\naz role assignment create --role \"Cognitive Services User\" --assignee $user --scope $resourceId<\/code><\/pre>\n<p>Wait couple of minutes for the role to be propagated<\/p>\n<\/li>\n<li>\n<p>Time to run the application. Run <code>make gradio<\/code>, visit <a href=\"http:\/\/localhost:8087\">http:\/\/localhost:8087<\/a> and ask a question &#8211; it has memory.\n<img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/ise\/wp-content\/uploads\/sites\/55\/2024\/02\/aks-workload-identity-and-azure-openai-make-gradio.png\" alt=\"make-command\" \/>\n<img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/ise\/wp-content\/uploads\/sites\/55\/2024\/02\/aks-workload-identity-and-azure-openai-bootstrap.png\" alt=\"bootstrap\" \/>\n<img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/ise\/wp-content\/uploads\/sites\/55\/2024\/02\/aks-workload-identity-and-azure-openai-chat.png\" alt=\"Chat\" \/><\/p>\n<\/li>\n<\/ol>\n<p>In summary, you can see the application is running without requiring to pull the OpenAI keys.<\/p>\n<h2>Lets create the AKS and deploy the platform<\/h2>\n<h3>Create the AKS and configure it<\/h3>\n<ol>\n<li>\n<p>Create the AKS with <code>oidc-issuer<\/code>, <code>enable workload-identity<\/code> and <code>enable managed-identity<\/code>. More details about the requirements and limitations can be found <a href=\"https:\/\/learn.microsoft.com\/en-us\/azure\/aks\/workload-identity-overview\">here<\/a>. For the example we will be using <code>default<\/code> namespace.<\/p>\n<pre><code class=\"language-bash\"># Create AKS cluster\r\naz aks create -g \"${RESOURCE_GROUP}\" -n \"${CLUSTER_NAME}\" --node-count 1 --enable-oidc-issuer --enable-workload-identity --enable-managed-identity<\/code><\/pre>\n<\/li>\n<li>\n<p>Now lets configure the identity. Get OIDC URL, create the identity and assign the <code>Cognitive Services User<\/code> role.<\/p>\n<pre><code class=\"language-bash\"># Get OIDC issuer URL\r\nAKS_OIDC_ISSUER=\"$(az aks show -n \"${CLUSTER_NAME}\" -g \"${RESOURCE_GROUP}\" --query \"oidcIssuerProfile.issuerUrl\" -otsv)\"\r\n\r\n# Create user-assigned identity\r\nUSER_ASSIGNED_IDENTITY_NAME=&lt;user name&gt;\r\naz identity create --name \"${USER_ASSIGNED_IDENTITY_NAME}\" --resource-group \"${RESOURCE_GROUP}\" --location \"${LOCATION}\"\r\n\r\n# Assign role to user-assigned identity\r\nRESOURCE_ID=\"$(az group show -g \"${RESOURCE_GROUP}\" --query \"id\" -o tsv)\"\r\naz role assignment create --role \"Cognitive Services User\" --assignee \"$(az identity show --resource-group \"${RESOURCE_GROUP}\" --name \"${USER_ASSIGNED_IDENTITY_NAME}\" --query 'clientId' -otsv)\" --scope \"${RESOURCE_ID}\"<\/code><\/pre>\n<\/li>\n<li>\n<p>Connect to the cluster since we require to deploy the service account that is required for the federation<\/p>\n<pre><code class=\"language-bash\">az aks get-credentials -g \"${RESOURCE_GROUP}\" -n \"${CLUSTER_NAME}\" -a<\/code><\/pre>\n<\/li>\n<li>\n<p>Create the service account in AKS in <code>default<\/code> namespace e.g. <code>SERVICE_ACCOUNT_NAMESPACE=default<\/code><\/p>\n<pre><code class=\"language-bash\"># Create Kubernetes service account\r\ncat &lt;&lt;EOF | kubectl apply -f -\r\napiVersion: v1\r\nkind: ServiceAccount\r\nmetadata:\r\n    annotations:\r\n        azure.workload.identity\/client-id: $(az identity show --resource-group \"${RESOURCE_GROUP}\" --name \"${USER_ASSIGNED_IDENTITY_NAME}\" --query 'clientId' -otsv)\r\n    name: \"${SERVICE_ACCOUNT_NAME}\"\r\n    namespace: \"${SERVICE_ACCOUNT_NAMESPACE}\"\r\nEOF<\/code><\/pre>\n<\/li>\n<li>\n<p>Federate the account and the managed identity.<\/p>\n<pre><code class=\"language-bash\">FEDERATED_IDENTITY_CREDENTIAL_NAME=\"gradio-app-federated-credential\"\r\n\r\naz identity federated-credential create --name \"${FEDERATED_IDENTITY_CREDENTIAL_NAME}\" --identity-name \"${USER_ASSIGNED_IDENTITY_NAME}\" --resource-group \"${RESOURCE_GROUP}\" --issuer \"${AKS_OIDC_ISSUER}\" --subject \"system:serviceaccount:${SERVICE_ACCOUNT_NAMESPACE}:${SERVICE_ACCOUNT_NAME}\"<\/code><\/pre>\n<\/li>\n<\/ol>\n<h3>Deploy the service into AKS<\/h3>\n<p>Now we have everything we need to deploy the application. It is required for the pod to use the service account and have the label <code>azure.workload.identity\/use: \"true\"<\/code>. For deploying into the cluster, we will be using the container created in the source code repository. In other words <code>ghcr.io\/jsburckhardt\/gradio-aks\/chatbot:latest<\/code>. So, lets get into the deployment:<\/p>\n<ol>\n<li>\n<p>Update the values under the <code>release<\/code> folder. There are two manifests that were used to deploy in parallel two chats with different engines.<\/p>\n<\/li>\n<li>\n<p>If you use the code in the repository, all you have to update is the environment section with your resources. Update both files if you want to deploy two resources in parallel: <code>release\/manifest-gpt4.yaml<\/code> and <code>release\/manifest.yaml<\/code><\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/ise\/wp-content\/uploads\/sites\/55\/2024\/02\/aks-workload-identity-and-azure-openai-manifest.png\" alt=\"Manifest\" \/><\/p>\n<\/li>\n<li>\n<p>Apply the manifest<\/p>\n<pre><code class=\"language-bash\">kubectl apply -f release<\/code><\/pre>\n<\/li>\n<\/ol>\n<h3>Access it<\/h3>\n<p>Once the resources are deployed, you can use <code>k9s<\/code> inside the devcontainer to check the pods running. You should see something similar to this:<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/ise\/wp-content\/uploads\/sites\/55\/2024\/02\/aks-workload-identity-and-azure-openai-pods.png\" alt=\"Pods_running\" \/><\/p>\n<p>To get the endpoint, inside <code>k9s<\/code> list the <code>services<\/code>:<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/ise\/wp-content\/uploads\/sites\/55\/2024\/02\/aks-workload-identity-and-azure-openai-svcs.png\" alt=\"SVCs_runnings\" \/><\/p>\n<p>Now you can use the external IP to access the resource.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/ise\/wp-content\/uploads\/sites\/55\/2024\/02\/aks-workload-identity-and-azure-openai-running.png\" alt=\"Running in AKS\" \/><\/p>\n<h2>So how is it getting the token?<\/h2>\n<p>To fully harness the capabilities of Azure OpenAI, authentication is the key. Understanding your options and choosing the right method can be the difference between a seamless experience and a tangled web of complexities. So, let&#8217;s break down the process.<\/p>\n<h3>1. The Two-Fold Path to Authentication<\/h3>\n<ul>\n<li>\n<p><strong>API Key<\/strong>: This is the quickest and easiest option. If you&#8217;re looking to dive into Azure OpenAI without much ado, your API key is readily available in the Azure portal, specifically within your Azure OpenAI resource section.<\/p>\n<\/li>\n<li>\n<p><strong>Azure Active Directory (AAD)<\/strong>: For those with intricate security protocols, the AAD offers a more robust solution. Detailed guidelines on integrating AAD with Azure OpenAI are available <a href=\"https:\/\/learn.microsoft.com\/en-us\/azure\/ai-services\/openai\/how-to\/managed-identity\">here<\/a>.<\/p>\n<\/li>\n<\/ul>\n<h3>2. Setting Up Your Local Development<\/h3>\n<p>If you&#8217;re developing on your local machine, ensure:<\/p>\n<ul>\n<li>\n<p><strong>Azure CLI is Installed<\/strong>: If not, <a href=\"https:\/\/learn.microsoft.com\/en-us\/cli\/azure\/install-azure-cli\">download and install the Azure CLI<\/a>. Once done, initiate it with the <code>az login<\/code> command.<\/p>\n<\/li>\n<li>\n<p><strong>Azure Role Assignment<\/strong>: Grant the &#8220;Cognitive Services OpenAI User&#8221; role specific to your Azure OpenAI resource. This pivotal step allows you to procure a token from AAD for Azure OpenAI usage. This role can be assigned to a diverse range of entities \u2013 from a user or group to a service principal or even a managed identity. Delve deeper into Azure OpenAI RBAC roles <a href=\"https:\/\/learn.microsoft.com\/en-us\/azure\/ai-services\/openai\/how-to\/role-based-access-control\">here<\/a>.<\/p>\n<\/li>\n<\/ul>\n<h2>3. Integrating AAD with Python via LangChain<\/h2>\n<ul>\n<li>\n<p><strong>Install Required Package<\/strong>: Kick off by installing the <code>azure-identity<\/code> package. In our case we are using python <a href=\"https:\/\/learn.microsoft.com\/en-us\/python\/api\/overview\/azure\/identity-readme?view=azure-python\">package<\/a><\/p>\n<\/li>\n<li>\n<p><strong>Set Up the Environment<\/strong>: Update the <code>OPENAI_API_TYPE<\/code> environment variable to <code>azure_ad<\/code>. We set up the variables in <code>.env<\/code> for local development. In the AKS we are using the manifest <code>env<\/code> property to set the values.<\/p>\n<\/li>\n<li>\n<p><strong>Fetch the Token<\/strong>: Utilize the <code>DefaultAzureCredential<\/code> class to obtain your token from AAD. The simple method is invoking the <code>get_token<\/code> function. Here&#8217;s a snippet:<\/p>\n<\/li>\n<\/ul>\n<pre><code class=\"language-python\">token = DefaultAzureCredential().get_token()<\/code><\/pre>\n<p>If you want to get more control over the options to generate the token, you can use the class <code>ChainedTokenCredential<\/code> and provide the list and order of credentials you want to use in your flow. For example, if you want to use ManagedIdentity and fallback to AzureCLI credentials. We could create our credentials instance as:<\/p>\n<pre><code class=\"language-python\">from azure.identity import ChainedTokenCredential, ManagedIdentityCredential, AzureCliCredential\r\n\r\ncredential = ChainedTokenCredential(\r\n    ManagedIdentityCredential(),\r\n    AzureCliCredential()\r\n)<\/code><\/pre>\n<p><strong>DefaultAzureCredential<\/strong> stands as a versatile credential, tailored to handle a myriad of Azure SDK authentication scenarios. The essence of its flexibility stems from its adaptability; the identity it employs is contingent upon the environment. In a bid to fetch an access token, it sequentially requests these identities, halting once a token is obtained:<\/p>\n<ul>\n<li>Environment-based service principal (<a href=\"https:\/\/learn.microsoft.com\/en-us\/python\/api\/azure-identity\/azure.identity.environmentcredential?view=azure-python\">EnvironmentCredential details<\/a>).<\/li>\n<li>Azure workload identity (through the <strong>WorkloadIdentityCredential<\/strong> when set by its webhook).<\/li>\n<li>Azure managed identity (<a href=\"https:\/\/learn.microsoft.com\/en-us\/python\/api\/azure-identity\/azure.identity.managedidentitycredential?view=azure-python\">ManagedIdentityCredential details<\/a>).<\/li>\n<li>For Windows users: A Microsoft application sign-in. <a href=\"https:\/\/learn.microsoft.com\/en-us\/python\/api\/azure-identity\/azure.identity.sharedtokencachecredential?view=azure-python\"><strong>SharedTokenCacheCredential<\/strong> dives deeper<\/a>.<\/li>\n<li>The Azure CLI logged-in identity.<\/li>\n<li>Azure PowerShell logged-in identity.<\/li>\n<li>Azure Developer CLI logged-in identity.<\/li>\n<\/ul>\n<p>While this default procedure serves most, it&#8217;s malleable, courtesy of configurable keyword arguments.<\/p>\n<p>Instantiating the <strong>DefaultAzureCredential<\/strong> involves several keyword parameters, granting developers fine-grain control. These parameters range from defining the Azure AD endpoint authority to toggling specific credentials on or off.<\/p>\n<p>For instance, if you wish to exclude Azure PowerShell or the Azure Developer CLI, just set <code>exclude_powershell_credential<\/code> or <code>exclude_developer_cli_credential<\/code> to True respectively. Each exclusion parameter bears a clear name, making configurations intuitive.<\/p>\n<p>Two primary methods encapsulate the essence of this class:<\/p>\n<ol>\n<li><strong>close()<\/strong>: To terminate the transport session of each chained credential.<\/li>\n<li><strong>get_token(scopes, claims, tenant_id)<\/strong>: Integral to access token acquisition, this method is invoked automatically by Azure SDK clients. It requisitions an access token for the defined scopes, and can incorporate additional claims or specify a tenant.<\/li>\n<\/ol>\n<p>To fetch an access token in Python:<\/p>\n<pre><code class=\"language-python\">token = credential.get_token('desired-scope')<\/code><\/pre>\n<p>In case of authentication failures, an exception (ClientAuthenticationError) is raised, outlining each authentication attempt and the corresponding error.<\/p>\n<h3>In Conclusion<\/h3>\n<p>The DefaultAzureCredential class embodies the marriage of simplicity with flexibility, rendering Azure SDK authentication a breeze for developers. By offering a range of identities and configurable behaviors, it ensures that diverse scenarios and specific needs are catered to effortlessly.<\/p>\n<h2>AKS deployment. Microsoft Entra Workload ID with AKS<\/h2>\n<p>When working with Azure Kubernetes Services (AKS), you might need to access resources like Azure Key Vault or Microsoft Graph. That&#8217;s where <strong>Microsoft Entra Workload ID<\/strong> comes in. It taps into Kubernetes&#8217; built-in features to work with external identity providers.<\/p>\n<p>It uses something called the <strong>Service Account Token Volume Projection<\/strong>. This means pods can take on a Kubernetes identity through service accounts. This setup, combined with OIDC federation, lets Kubernetes apps securely tap into Azure resources using the Microsoft Entra ID.<\/p>\n<p>For developers, the good news is that this all plays nice with Azure Identity client libraries and the MSAL suite, especially if you&#8217;re into application registration.<\/p>\n<h3>How it works<\/h3>\n<p>A workload identity flow, is a sequence of steps for authenticating a workload (such as an application or service) running on Azure Kubernetes Service (AKS) and authorizing it to access Azure resources. Here&#8217;s a step-by-step summary of the flow:<\/p>\n<ol>\n<li>\n<p><strong>Kubelet<\/strong>: Starts the flow by projecting a service account token to the workload. This token is available at a configurable file path.<\/p>\n<\/li>\n<li>\n<p><strong>AKS Workload<\/strong>: The workload retrieves the projected service account token and sends it along with a request for an Azure Active Directory (AD) access token.<\/p>\n<\/li>\n<li>\n<p><strong>Azure Active Directory<\/strong>: Receives the service account token, checks the trust on the application (i.e., verifies that the token is from a trusted source), and validates the token.<\/p>\n<\/li>\n<li>\n<p><strong>Azure AD<\/strong>: Once validation is successful, Azure AD issues an access token back to the AKS workload.<\/p>\n<\/li>\n<li>\n<p><strong>AKS Workload<\/strong>: Uses the received Azure AD access token to access Azure resources.<\/p>\n<\/li>\n<\/ol>\n<p>This flow ensures that only authenticated and authorized workloads can interact with Azure resources, enhancing security by using Azure Active Directory as the central authority for trust and token issuance.<\/p>\n<p>Additional details can be found in the following MS documentation <a href=\"https:\/\/learn.microsoft.com\/en-us\/azure\/aks\/workload-identity-overview?tabs=dotnet#how-it-works\">Use Microsoft Entra Workload ID with Azure Kubernetes Service (AKS)<\/a><\/p>\n<p>In kubernetes world. For those familiar with Microsoft Entra <a href=\"https:\/\/learn.microsoft.com\/en-us\/azure\/aks\/use-azure-ad-pod-identity\">pod-managed identity<\/a>, consider a service account as its Kubernetes counterpart &#8211; the Azure Identity. But, unlike the Azure Identity, a service account is baked right into the core Kubernetes API, not added as a Custom Resource Definition (CRD). Below, we&#8217;ll delve into the various labels and annotations available, guiding you on how to swap the service account token for a Microsoft Entra access token.<\/p>\n<h3>Service Account Annotations<\/h3>\n<p>There are 3 main annotations to consider <code>azure.workload.identity\/client-id<\/code>, <code>azure.workload.identity\/tenant-id<\/code> and <code>azure.workload.identity\/service-account-token-expiration<\/code>. Have a look in this section for more details <a href=\"https:\/\/learn.microsoft.com\/en-us\/azure\/aks\/workload-identity-overview?tabs=dotnet#service-account-annotations\">Service Account Annotations<\/a><\/p>\n<h3>Pod Labels<\/h3>\n<p>Regarding the pod labels, there are 4 we should consider <code>azure.workload.identity\/service-account-token-expiration<\/code>, <code>azure.workload.identity\/skip-containers<\/code>, <code>azure.workload.identity\/inject-proxy-sidecar<\/code> and <code>azure.workload.identity\/proxy-sidecar-port<\/code>. For specifics, have a look into this section <a href=\"https:\/\/learn.microsoft.com\/en-us\/azure\/aks\/workload-identity-overview?tabs=dotnet#pod-labels\">Pod Labels<\/a>.<\/p>\n<h3>Lets see the components in Azure<\/h3>\n<ol>\n<li>\n<p>Managed identity<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/ise\/wp-content\/uploads\/sites\/55\/2024\/02\/aks-workdload-identity-and-azure-openai-managed_identity.png\" alt=\"managed_identity\" \/><\/p>\n<\/li>\n<li>\n<p>Role assigned to the managed identity<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/ise\/wp-content\/uploads\/sites\/55\/2024\/02\/aks-workdload-identity-and-azure-openai-role_assigned.png\" alt=\"role_assigned\" \/><\/p>\n<\/li>\n<li>\n<p>Service account with the managed identity client-id<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/ise\/wp-content\/uploads\/sites\/55\/2024\/02\/aks-workdload-identity-and-azure-openai-service_account.png\" alt=\"service_account\" \/><\/p>\n<\/li>\n<li>\n<p>Federation<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/ise\/wp-content\/uploads\/sites\/55\/2024\/02\/aks-workdload-identity-and-azure-openai-federation_details.png\" alt=\"federation_details\" \/><\/p>\n<\/li>\n<li>\n<p>Apply<\/p>\n<pre><code class=\"language-bash\">kubectl apply -f release<\/code><\/pre>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/ise\/wp-content\/uploads\/sites\/55\/2024\/02\/aks-workdload-identity-and-azure-openai-apply_manifest.png\" alt=\"apply_manifest\" \/><\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/ise\/wp-content\/uploads\/sites\/55\/2024\/02\/aks-workdload-identity-and-azure-openai-pods_running.png\" alt=\"pods_running\" \/><\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/ise\/wp-content\/uploads\/sites\/55\/2024\/02\/aks-workdload-identity-and-azure-openai-pod_description.png\" alt=\"pod_description\" \/><\/p>\n<\/li>\n<\/ol>\n<h2>Useful docs<\/h2>\n<ul>\n<li><a href=\"https:\/\/blog.identitydigest.com\/azuread-federate-mi\/\">Azure Managed Identities with Workload Identity Federation | Identity in the cloud (identitydigest.com)<\/a><\/li>\n<li><a href=\"https:\/\/learn.microsoft.com\/en-us\/azure\/aks\/learn\/tutorial-kubernetes-workload-identity\">Tutorial &#8211; Use a workload identity with an application on Azure Kubernetes Service (AKS) &#8211; Azure Kubernetes Service | Microsoft Learn<\/a><\/li>\n<li><a href=\"https:\/\/learn.microsoft.com\/en-us\/azure\/ai-services\/openai\/how-to\/managed-identity\">How to configure Azure OpenAI Service with managed identities &#8211; Azure OpenAI | Microsoft Learn<\/a><\/li>\n<li><a href=\"https:\/\/blog.identitydigest.com\/azuread-podid-workload-compare\/\">Comparing Azure pod vs workload identity<\/a><\/li>\n<li><a href=\"https:\/\/blog.identitydigest.com\/azuread-federate-k8s\/\">Azure AD workload identity federation with Kubernetes<\/a><\/li>\n<li><a href=\"https:\/\/blog.identitydigest.com\/azuread-federate-mi\/\">Azure Managed Identities with Workload Identity Federation<\/a><\/li>\n<\/ul>\n<h2>Summary<\/h2>\n<p>This blog post examines the integration of Azure OpenAI with Azure Kubernetes Services (AKS) through Workload Identities. It outlines the security and efficiency benefits of using Workload Identities for application authentication, eliminating the need for secret management. The adaptability of Workload Identities across different hosting environments is also highlighted.<\/p>\n<p>The post details the setup process for Azure AD authentication and the utilization of Microsoft Entra Workload ID for Azure resource access within AKS. It provides a practical guide on running applications locally, creating an AKS, deploying services to AKS, and accessing these services.<\/p>\n<p>Furthermore, the blog discusses key Azure components such as Managed Identity and Federation, essential in this integration process. This guide is designed to assist developers in effectively using Azure OpenAI resources in AKS, leveraging the capabilities of Workload Identities.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>This post will showcase the use of Azure OpenAI from AKS without the need for sharing secrets.<\/p>\n","protected":false},"author":114974,"featured_media":15288,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1,3451],"tags":[3447,3495,3496,3497],"class_list":["post-15283","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-cse","category-ise","tag-azcli","tag-azure-openai","tag-openai","tag-workload-identities"],"acf":[],"blog_post_summary":"<p>This post will showcase the use of Azure OpenAI from AKS without the need for sharing secrets.<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/posts\/15283","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/users\/114974"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/comments?post=15283"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/posts\/15283\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/media\/15288"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/media?parent=15283"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/categories?post=15283"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/tags?post=15283"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}