Operate with visibility.<\/strong>\u00a0Migration jobs ran aggressively, but source and target utilization were monitored continuously and throttled when needed.<\/li>\n<\/ul>\nThe result was a migration that looked less like a one-click copy operation and more like a carefully sequenced production event.<\/p>\n
1. Move only the data that matters<\/h2>\n
The source estate had thousands of collections across multiple workloads. As with many large systems, not every collection carried equal value.<\/p>\n
Some collections were active. Some were legacy leftovers. Some large collections contained only a subset of documents that still mattered to the workload.<\/p>\n
The team used assessment data to identify what should move and what should not. Junk collections were skipped entirely. Large collections were filtered to include only relevant documents.<\/p>\n
That reduced the amount of data to move and helped the team stay focused on the production cutover goal.<\/p>\n
The lesson is simple:\u00a0the fastest data to migrate is the data you do not migrate<\/strong>.<\/p>\nFor a 47 TB estate, scope control is not just housekeeping. It is one of the most important performance optimizations.<\/p>\n
2. Defer HA and cross-region replication during bulk ingestion<\/h2>\n
High availability and cross-region replication are important for production resilience. But during testing, enabling them during ingestion created write-ahead log pressure.<\/p>\n
As the load ran, WAL logs accumulated. Disk pressure increased. The system eventually hit the kind of failure mode every migration team wants to avoid: storage filling during the migration.<\/p>\n
The team changed the sequence:<\/p>\n
\n- Load the bulk data first.<\/li>\n
- Keep HA and cross-region replicas disabled during the heaviest ingestion period.<\/li>\n
- Turn them back on after the bulk load completed.<\/li>\n<\/ul>\n
This did not remove resilience from the final design. It simply recognized that the migration state and steady state have different priorities.<\/p>\n
During ingestion, optimize for movement. After ingestion, optimize for resilience.<\/p>\n
3. Build indexes after the load<\/h2>\n
Indexes are critical for query performance, but they are not free.<\/p>\n
In testing, building non-unique indexes while rapid ingestion was running caused CPU and disk utilization to spike. The system was accepting a large volume of writes while also doing expensive index work for future reads.<\/p>\n
The team moved non-unique index builds to\u00a0post-ingestion<\/strong>, ran them in\u00a0blocking mode<\/strong>, and paused CDC online sync while the builds completed.<\/p>\nThat created two clean phases:<\/p>\n
\n- Move the data quickly.<\/li>\n
- Prepare the data for steady-state query performance.<\/li>\n<\/ol>\n
This made the migration more predictable and avoided unnecessary resource contention during the most performance-sensitive part of the run.<\/p>\n
4. Do not copy the source topology blindly<\/h2>\n
The source environment was large: 27 nodes arranged as 9 shards with 3 nodes each. It would have been natural to mirror that shape on the target.<\/p>\n
But the team looked at the workload instead of copying the topology.<\/p>\n
No individual collection was too large for one target node. The source was sharded on\u00a0_id<\/code>, while Azure DocumentDB queries for this workload performed better when collections were kept un-sharded.<\/p>\nSo the team kept collections un-sharded on the target and distributed them across a smaller number of nodes by storage volume. Linked collections were kept together on the same node to avoid unnecessary cross-node behavior.<\/p>\n
The final target was a 3-node, 16 TB cluster.<\/p>\n
The lesson: the goal of a migration is not to recreate the source. It is to design the target for the workload you actually want to run.<\/p>\n
5. Split the migration into four balanced jobs<\/h2>\n
The team used UAT to validate the tooling, train operators, find issues, and refine the production runbook before production data was at risk.<\/p>\n
The production migration was split into\u00a0four balanced jobs<\/strong>:<\/p>\n\n- Two jobs carried the largest and highest-throughput collections.<\/li>\n
- Two jobs carried the remaining collections, distributed to balance offline and online load.<\/li>\n<\/ul>\n
This gave the team enough parallelism to move quickly without turning the run into operational chaos.<\/p>\n
The jobs ran at full speed, but not blindly. Source and target utilization were monitored manually. When pressure rose, the team throttled. When the system had room, the jobs continued.<\/p>\n
What made the cutover work?<\/h2>\n
No single trick made the migration successful. The win came from a series of disciplined choices:<\/p>\n
\n\n\n| Choice<\/th>\n | Why it mattered<\/th>\n<\/tr>\n<\/thead>\n |
\n\n| Assess first<\/td>\n | Decisions were based on real collection, storage, and workload data.<\/td>\n<\/tr>\n |
\n| Prove in UAT<\/td>\n | Tooling issues surfaced before production.<\/td>\n<\/tr>\n |
\n| Defer HA and replicas<\/td>\n | Reduced WAL and disk pressure during ingestion.<\/td>\n<\/tr>\n |
\n| Build indexes after load<\/td>\n | Avoided CPU and disk contention while data was moving.<\/td>\n<\/tr>\n |
\n| Keep collections un-sharded<\/td>\n | Matched the target design to the workload, not the source topology.<\/td>\n<\/tr>\n |
\n| Split into four jobs<\/td>\n | Balanced throughput and operational control.<\/td>\n<\/tr>\n |
\n| Monitor continuously<\/td>\n | Allowed the team to run fast and throttle only when needed.<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n The final outcome:\u00a047 TB moved, 7,000+ collections handled, a 27-node source consolidated onto a 3-node Azure DocumentDB target, and production cutover completed in under 48 hours<\/strong>.<\/p>\nTry the pattern on your own migration plan<\/h2>\nIf you are planning a large MongoDB-compatible migration to Azure DocumentDB, start with these questions:<\/p>\n \n- Which collections are still actively needed?<\/li>\n
- Which large collections can be filtered?<\/li>\n
- Which production features should be enabled after ingestion instead of before it?<\/li>\n
- Which indexes can safely move to a post-load phase?<\/li>\n
- Does the target need to mirror the source topology, or should it reflect the actual workload?<\/li>\n
- How can the migration be split into balanced, observable jobs?<\/li>\n<\/ul>\n
Which migration option should you choose for Azure DocumentDB?<\/h2>\nAzure DocumentDB supports multiple migration approaches for different source environments and operational requirements. Review the\u00a0Azure DocumentDB migration options<\/a>\u00a0to compare the available tools and choose the approach that best fits your workload and cutover plan.<\/p>\nLarge migrations become much less risky when the runbook is driven by evidence and the work is sequenced intentionally.<\/p>\n The best migrations are not heroic. They are boring in exactly the right ways.<\/p>\n","protected":false},"excerpt":{"rendered":" Large database migrations are where good planning either pays off or falls apart. In a MongoDB-compatible estate with thousands of collections, the hard part is not only moving data. It is deciding what to move, what to defer, how to sequence the work, and how to keep the system stable while ingestion is running at […]<\/p>\n","protected":false},"author":96034,"featured_media":157,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1,3],"tags":[],"class_list":["post-153","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-azure-document-db","category-mongodb-migrations"],"acf":[],"blog_post_summary":" Large database migrations are where good planning either pays off or falls apart. In a MongoDB-compatible estate with thousands of collections, the hard part is not only moving data. It is deciding what to move, what to defer, how to sequence the work, and how to keep the system stable while ingestion is running at […]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/documentdb\/wp-json\/wp\/v2\/posts\/153","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/documentdb\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/documentdb\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/documentdb\/wp-json\/wp\/v2\/users\/96034"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/documentdb\/wp-json\/wp\/v2\/comments?post=153"}],"version-history":[{"count":3,"href":"https:\/\/devblogs.microsoft.com\/documentdb\/wp-json\/wp\/v2\/posts\/153\/revisions"}],"predecessor-version":[{"id":162,"href":"https:\/\/devblogs.microsoft.com\/documentdb\/wp-json\/wp\/v2\/posts\/153\/revisions\/162"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/documentdb\/wp-json\/wp\/v2\/media\/157"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/documentdb\/wp-json\/wp\/v2\/media?parent=153"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/documentdb\/wp-json\/wp\/v2\/categories?post=153"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/documentdb\/wp-json\/wp\/v2\/tags?post=153"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}} |
![\"Migration](\"https:\/\/devblogs.microsoft.com\/documentdb\/wp-content\/uploads\/sites\/93\/2026\/07\/documentdb-migration-sequencing-dashboard.webp\")
<\/a>