From Redis pressure to a healthier service

The engineering journey behind an Azure Managed Redis migration for Copilot Studio agent flows

Microsoft Copilot Studio helps organizations build agents and automate business processes. Agent flows let those agents carry out repeatable work across connected systems. Our shared service is now the primary backend for agent flows in Copilot Studio, while continuing to support Power Automate workflows. It handles flow creation, management and sharing. Regional Redis deployments, not a single global cache, sit on the critical path for many of those operations.

The service was feeling the strain: high Redis server load, avoidable reads and refresh writes, and recurring client-connection failures that surfaced as customer errors and repeated on-call investigations. More capacity alone would not fix inefficient access or unreliable recovery. Over the following months, we reduced unnecessary work, hardened the clients and moved from Azure Cache for Redis (Classic) to Azure Managed Redis (AMR). The aim was a healthier customer-critical dependency, not just a cheaper one.

The workload behind the Redis migration

Within a deployment scope, up to seven service applications share a cache for connector metadata, environment information, access results, entitlements and expiring coordination state. Customer requests and background refresh jobs both drive traffic. The table summarizes successful Redis client operations over seven complete UTC days after migration (6-12 September 2026). Each geography has primary and secondary regional service deployments, each with its own Redis cache; the table combines their traffic.

Figure 1. Shared regional Redis deployments; this pattern is repeated across regions.

We kept the cache shared, deliberately

Several services read the same connector metadata while workers refreshed it centrally. Giving each service its own Redis would duplicate data and refresh work, and require an invalidation or distribution design. Expiring coordination state also needed common visibility. We kept that shared-data contract rather than combining a cache migration with a data-ownership redesign.

However, sharing couples failures and resource contention. Some deployment groups already had separately configured caches; we retained regional sharing where services used the same data and coordination state. For example, teams may separate caches when data, security or availability needs differ.

The first wins came before the AMR migration

We first asked which Redis calls should never happen. In a 24-hour West Europe baseline, one environment-mapping cache generated about 290 million Redis writes, roughly 16% of our service’s Redis operations in the region, with little corresponding read traffic. We first suppressed those writes while preserving the first-level cache (L1) in each service instance’s memory, then removed that distributed-cache path entirely.

Before that final removal, we also fixed its read path: it started a Redis GET before checking L1, then waited for and discarded the result on a local hit. Moving the read behind the miss check let the L1 hit rate of at least 95% translate into avoided Redis calls. We extended local caching to selected flow-access and billing-context reads, with one-minute lifetimes and explicit invalidation, and cached confirmed ‘not found’ results so deleted workflows did not keep triggering backend work.

These application-side changes reduced the command-processing work reaching Redis before the Azure Managed Redis migration. Our July update reported server load (command-processing busy time) falling from about 75% to 15% in European examples, and from 75-95% to about 40% in US examples.

Figure 2a. Original European server-load example from the July 7 update.

Figure 2b. Original US example from the same update. The captured windows differ.

Modernizing the client hot path

Client correctness was the next concern. Redis clients match replies to pending requests in order, so a failed write must not leave their bookkeeping out of step. StackExchange.Redis 2.13.17 closed the connection after a write failure to keep request and reply queues in sync. It also included other connection-stability fixes.

StackExchange.Redis 3.0 then modernized the I/O hot path. It brought buffer and I/O control into the library and replaced a DOM-based reply parser with a reader-based implementation. The target was less response-processing overhead, especially for large or complex replies, with tighter control over I/O boundaries and threading. We used published StackExchange.Redis benchmarks to guide the upgrade, not as evidence of a performance gain in our production service.

Recovery had to be controlled, too

We also kept connections long-lived, serialized and paced reconnect attempts, and bounded connection-handshake waits. This reduced recovery contention without letting every failing request open a new connection. Recovery also had to support token renewal and handle identity-provider delays.

Planning the Azure Managed Redis migration

Reducing redundant Redis operations and improving client stability gave us more headroom. An Azure Managed Redis migration offered a better cost and scaling profile for the remaining workload. We chose AMR for its multi-threaded Redis Enterprise architecture, clustering and capacity options, rather than simply sizing up the Classic tier. Both services support Microsoft Entra authentication; the migration also completed our identity-only configuration, removing the static-key fallback in migrated scopes.

We planned capacity, authentication and compatibility before starting with limited groups. We temporarily wrote selected shared metadata to both caches while reads stayed on Classic, warming those entries before cutover. By contrast, other lookups refilled from authoritative services. This was not full replication and could not be applied blindly to expiring coordination state. We watched downstream load as the new cache filled.

Each group moved through health gates covering errors, cache misses, tail latency and downstream load. Routing decisions were cached per process, so planned restarts made destination changes predictable. Missing connection configuration needed to fail visibly rather than read from the wrong cache. Sustained regressions meant holding expansion or returning to Classic before retiring it.

Redis migration phases and safeguards

The rollout followed five phases:

The Europe rollout exposed a hidden connection limit

During the Europe rollout, customers began seeing failures and timeouts when creating, saving and enabling flows. An inherited maintenance listener in our authentication configuration was issuing Redis’s SUBSCRIBE command, even though the service did not use publish/subscribe messaging. As a result, these subscriptions disrupted AMR’s normal sharing of backend connections. At regional production scale, the resulting connection demand exhausted the available backend connections.

We restored service by switching the affected deployments back to Classic and restarting application instances in stages to recreate their Redis connections. The permanent fix disabled the unused subscription behavior on AMR while preserving identity authentication, token renewal and ordinary cache operations. The lesson was to validate the behavior introduced by authentication adapters and connection defaults at production scale, not just whether basic reads and writes succeeded.

Cutover was not the end

Retirement required changes to runtime and deployment. We removed temporary duplicate writes, disabled creation of Classic resources and removed consumers of their outputs. Otherwise, a future deployment could still try to read settings or credentials from a deleted cache.

We reconciled exact resource identities, investigated remaining connections and required sustained evidence that neither running services nor future deployments still needed Classic. An idle cache was not enough if another component could reconnect or recreate it.

Finally, retirement completed this stage of the Azure Managed Redis migration and ended the simple rollback path. Recovery would then require provisioning and rebuilding the cache. Classic Redis continued to serve workloads outside this migration’s scope, including those in sovereign cloud regions.

Making the cost reduction real

Retiring the old capacity ended the overlap in costs. The remaining Classic usage includes sovereign cloud regions, which were excluded from this migration. See the AMR planning FAQ for current cloud availability. The comparison includes the full AMR replacement cost.

Relative Redis costs, with the pre-migration baseline set to 100%:

Standard and Basic tiers are excluded. Percentages are rounded and relative to this deployment’s baseline, not customer pricing.

The closure snapshot showed about 72% lower ongoing Redis costs, including the AMR replacement, against our pre-migration baseline.

What stayed with us

The lessons from this Azure Managed Redis migration go beyond the smaller bill. We reduced redundant Redis traffic, improved client behavior and recovery, and made migration and retirement deliberate. Those gains came from understanding the full request path, not simply changing a package version or cache endpoint.

Together, these changes created more operating headroom for Copilot Studio agent flows and Power Automate workflows, at a lower ongoing infrastructure cost.