Chat Copilot Performance improvements for chat and messaging loading

Nilesh Acharya

August 28th, 20230 0

We are introducing enhancements to greatly improve performance, reduce cost and data throughput for customers of Chat Copilot. In the current design, partitions are defined by an ID. Overtime as the CosmosDB grows with increased storage we have seen scenarios with extended processing time when customers try to load the list of chats for a current user or load the list of messages for a selected user.

As a result, we will implement new changes to update partitioning be userID and chatID instead of an ID for chat sessions and chat messages. We have seen improved experiences as a result of the new partitioning method.

For new deployments of Chat Copilot, no changes are required to yield this performance improvement. For existing customers our recommendation is to remove the existing Cosmos DB containers and redeploy to realize the performance update related to the partition schema. To preserve existing chats, containers can be migrated as described here.