{"id":1248,"date":"2022-04-27T10:00:28","date_gmt":"2022-04-27T17:00:28","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/azure-sql\/?p=1248"},"modified":"2022-04-27T08:57:17","modified_gmt":"2022-04-27T15:57:17","slug":"announcing-general-availability-of-change-data-capture-cdc-on-azure-sql-database","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/azure-sql\/announcing-general-availability-of-change-data-capture-cdc-on-azure-sql-database\/","title":{"rendered":"Announcing General Availability of Change Data Capture (CDC) on Azure SQL Database"},"content":{"rendered":"<p><span class=\"NormalTextRun SCXW236723075 BCX8\">We are excited to announce the\u00a0<\/span><span class=\"NormalTextRun SCXW236723075 BCX8\">general availability (GA)<\/span><span class=\"NormalTextRun SCXW236723075 BCX8\">\u00a0of change\u00a0<\/span><span class=\"NormalTextRun SCXW236723075 BCX8\">d<\/span><span class=\"NormalTextRun SCXW236723075 BCX8\">ata\u00a0<\/span><span class=\"NormalTextRun SCXW236723075 BCX8\">c<\/span><span class=\"NormalTextRun SCXW236723075 BCX8\">apture\u00a0<\/span><span class=\"NormalTextRun SCXW236723075 BCX8\">(CDC)\u00a0<\/span><span class=\"NormalTextRun SCXW236723075 BCX8\">in Azure SQL Database.<\/span><\/p>\n<h5><strong><span data-contrast=\"auto\">What is Change data capture (CDC)?<\/span><\/strong><\/h5>\n<p><span data-contrast=\"auto\">CDC\u00a0<\/span><span data-contrast=\"none\">provides historical change information for a user table by capturing both the fact that Data Manipulation Language (DML) changes (insert \/ update \/ delete) were made and the changed data. Changes are captured by using a capture process that reads changes from the transaction log and places them in corresponding change tables. These change tables provide a historical view of the changes made over time to source tables. CDC functions enable the change data to be consumed easily and systematically.<\/span><\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/azure-sql\/wp-content\/uploads\/sites\/56\/2022\/04\/cdc-diagram.png\"><img decoding=\"async\" class=\"size-medium wp-image-1249 aligncenter\" src=\"https:\/\/devblogs.microsoft.com\/azure-sql\/wp-content\/uploads\/sites\/56\/2022\/04\/cdc-diagram-300x266.png\" alt=\"Image cdc diagram\" width=\"300\" height=\"266\" srcset=\"https:\/\/devblogs.microsoft.com\/azure-sql\/wp-content\/uploads\/sites\/56\/2022\/04\/cdc-diagram-300x266.png 300w, https:\/\/devblogs.microsoft.com\/azure-sql\/wp-content\/uploads\/sites\/56\/2022\/04\/cdc-diagram.png 375w\" sizes=\"(max-width: 300px) 100vw, 300px\" \/><\/a><\/p>\n<p>&nbsp;<\/p>\n<h5><strong><span data-contrast=\"none\">Why use Change data capture (CDC)?\u00a0<\/span><\/strong><\/h5>\n<p><span data-contrast=\"auto\">CDC is a widely used feature by a wide range of customers\u202ffor a variety of purposes as mentioned below, but not limited to:\u00a0<\/span><\/p>\n<ul>\n<li data-leveltext=\"\uf0b7\" data-font=\"Symbol\" data-listid=\"1\" data-aria-posinset=\"1\" data-aria-level=\"1\"><span data-contrast=\"auto\">Tracking\u202fdata changes for auditing<\/span><span data-ccp-props=\"{&quot;134233117&quot;:true,&quot;134233118&quot;:true,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559740&quot;:240}\">\u00a0<\/span><\/li>\n<li data-leveltext=\"\uf0b7\" data-font=\"Symbol\" data-listid=\"1\" data-aria-posinset=\"2\" data-aria-level=\"1\"><span data-contrast=\"auto\">Recording data changes on source database and integrating with other services (e.g. Azure Data Factory) to stream these changes to other destinations<\/span><span data-ccp-props=\"{&quot;134233117&quot;:true,&quot;134233118&quot;:true,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559740&quot;:240}\">\u00a0<\/span><\/li>\n<li data-leveltext=\"\uf0b7\" data-font=\"Symbol\" data-listid=\"1\" data-aria-posinset=\"3\" data-aria-level=\"1\"><span data-contrast=\"auto\">Performing analytics on change data\u202f\u202f<\/span><span data-ccp-props=\"{&quot;134233117&quot;:true,&quot;134233118&quot;:true,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559740&quot;:240}\">\u00a0<\/span><\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<h5><strong><span data-contrast=\"auto\">Change data capture (CDC) on Azure SQL database\u00a0<\/span><\/strong><span data-ccp-props=\"{&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559738&quot;:240}\">\u00a0<\/span><\/h5>\n<p><span data-contrast=\"auto\">CDC is now generally available on Azure SQL databases, enabling customers to track insert \/ update \/ delete data changes on their Azure SQL Database tables. On Azure SQL database, CDC offers a similar functionality to SQL Server and Azure SQL Managed Instance, providing a\u00a0<\/span><strong><span data-contrast=\"auto\">scheduler<\/span><\/strong><span data-contrast=\"auto\">\u00a0which automatically runs change capture and cleanup processes on the change tables. These capture and cleanup processes used to be run as SQL Server Agent jobs on SQL Server on premises and on Azure SQL Managed Instance, but now they run automatically through the scheduler in Azure SQL databases. Customers can still run scans and cleanup manually on demand.<\/span><\/p>\n<p><span data-contrast=\"auto\">CDC on Azure SQL database\u00a0has been one of the most requested features through Azure feedback mechanisms. Multiple customers have successfully used CDC during public preview, such as\u00a0<\/span><a href=\"https:\/\/www.rubrik.com\/\" target=\"_blank\" rel=\"noopener nofollow noreferrer\"><span data-contrast=\"none\">Rubrik<\/span><\/a><span data-contrast=\"auto\">, a major cloud data management company. After using CDC, they highlighted: \u201c<em>Flawless implementation. An extremely useful addition to Azure SQL Databases that enabled us to track and capture changes from them meticulously.<\/em>\u201d\u00a0<\/span><\/p>\n<p>&nbsp;<\/p>\n<h5><strong><span data-contrast=\"auto\">Enabling change data capture (CDC) on an Azure SQL database\u00a0<\/span><\/strong><span data-ccp-props=\"{&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559738&quot;:240}\">\u00a0<\/span><\/h5>\n<p><span data-contrast=\"auto\">Customers will be able to use CDC on Azure SQL databases higher than the S3 (Standard) tier or vCore equivalent. More specifically, CDC cannot be enabled on subcore databases.<\/span><\/p>\n<p><span data-contrast=\"auto\">Enabling CDC on an Azure SQL database is similar to enabling CDC on SQL Server or Azure SQL Managed Instance. Learn more here:\u00a0<\/span><a href=\"https:\/\/docs.microsoft.com\/en-us\/sql\/relational-databases\/track-changes\/enable-and-disable-change-data-capture-sql-server?view=sql-server-ver15\" target=\"_blank\" rel=\"noopener noreferrer\"><span data-contrast=\"none\">Enable CDC<\/span><\/a><span data-ccp-props=\"{&quot;335551550&quot;:6,&quot;335551620&quot;:6}\">\u00a0<\/span><\/p>\n<p>&nbsp;<\/p>\n<h5><strong><span data-contrast=\"auto\">\nSending CDC data changes to other destinations<\/span><\/strong><\/h5>\n<p><span data-contrast=\"auto\">Multiple Microsoft technologies such as Azure Data Factory can be used to move CDC change data to other destinations (e.g. other databases, data warehouses). Other third-party<\/span><span data-contrast=\"auto\">\u00a0services also offer streaming capabilities for changed data from CDC. For instance,\u00a0<\/span><a href=\"https:\/\/www.striim.com\/\" target=\"_blank\" rel=\"noopener nofollow noreferrer\"><span data-contrast=\"none\">Striim<\/span><\/a><span data-contrast=\"auto\">\u00a0and\u00a0<\/span><a href=\"https:\/\/www.qlik.com\/\" target=\"_blank\" rel=\"noopener nofollow noreferrer\"><span data-contrast=\"none\">Qlik<\/span><\/a>\u00a0<span data-contrast=\"auto\">offer integration, processing, delivery, analysis, or visualization capabilities for CDC changes.<\/span><\/p>\n<p>&nbsp;<\/p>\n<p><i><span data-contrast=\"auto\">\u201cReal-time information is vital to the health of the enterprises,\u201d\u00a0<\/span><\/i><span data-contrast=\"auto\">says Codin Pora, VP of Technology and partnership at Striim.<\/span><i><span data-contrast=\"auto\">\u00a0\u201cStriim is excited to support the new change data capture (CDC) capabilities of Azure SQL Database and\u202fhelp companies drive their digital transformation\u202fby bringing together data, people, and processes. Striim, through its Azure SQL Database CDC pipelines, provides real-time data for analytics and intelligence workloads, operational reporting, ML\/AI implementations and many other use cases, creating\u202fvalue as well as competitive\u202fadvantage in a digital-first world.\u202fStriim builds continuous streaming data pipelines with minimal overhead on the source Azure SQL Database systems, while moving database operations (inserts, updates, and deletes) in real time with security, reliability, and transactional integrity.\u201d<\/span><\/i><span data-contrast=\"auto\">\u00a0<\/span><\/p>\n<p><i><span data-contrast=\"auto\">\u201cJoint customers are excited about the potential of leveraging Qlik Data Integration alongside CDC in Azure SQL DB and CDC for SQL MI to securely access more of their valuable data for analytics in the cloud<\/span><\/i><span data-contrast=\"auto\">,\u201d said Kathy Hickey, Vice President, Product Management at Qlik.<\/span><i><span data-contrast=\"auto\">\u00a0\u201cWe are happy to announce that in addition to support for Azure SQL MI as a source, the newly available MS-CDC capabilities will also allow us to support Azure SQL DB sources via our Early Access Program. We look forward to partnering with Microsoft on helping customers leverage these capabilities to confidently create new insights from their Azure managed data sources.\u201d<\/span><\/i><span data-contrast=\"auto\">\u00a0<\/span><\/p>\n<h5><\/h5>\n<p>&nbsp;<\/p>\n<h5><strong><span data-contrast=\"auto\">More Information<\/span><\/strong><\/h5>\n<p>To know more about change data capture (CDC), check out the <a href=\"https:\/\/docs.microsoft.com\/en-us\/sql\/relational-databases\/track-changes\/about-change-data-capture-sql-server\">documentation<\/a>.<\/p>\n<p>Learn module:\u00a0<a href=\"https:\/\/docs.microsoft.com\/en-us\/learn\/modules\/azure-sql-replication\/\" target=\"_blank\" rel=\"noopener noreferrer\">Data replication on Azure SQL Databases &#8211; Learn | Microsoft Docs<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>We are excited to announce the\u00a0general availability (GA)\u00a0of change\u00a0data\u00a0capture\u00a0(CDC)\u00a0in Azure SQL Database. What is Change data capture (CDC)? CDC\u00a0provides historical change information for a user table by capturing both the fact that Data Manipulation Language (DML) changes (insert \/ update \/ delete) were made and the changed data. Changes are captured by using a capture [&hellip;]<\/p>\n","protected":false},"author":29335,"featured_media":81,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[510,465,469,509,508],"class_list":["post-1248","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-azure-sql","tag-azure-sql-database","tag-azuresql","tag-azuresqldb","tag-cdc","tag-change-data-capture"],"acf":[],"blog_post_summary":"<p>We are excited to announce the\u00a0general availability (GA)\u00a0of change\u00a0data\u00a0capture\u00a0(CDC)\u00a0in Azure SQL Database. What is Change data capture (CDC)? CDC\u00a0provides historical change information for a user table by capturing both the fact that Data Manipulation Language (DML) changes (insert \/ update \/ delete) were made and the changed data. Changes are captured by using a capture [&hellip;]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/azure-sql\/wp-json\/wp\/v2\/posts\/1248","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/azure-sql\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/azure-sql\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/azure-sql\/wp-json\/wp\/v2\/users\/29335"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/azure-sql\/wp-json\/wp\/v2\/comments?post=1248"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/azure-sql\/wp-json\/wp\/v2\/posts\/1248\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/azure-sql\/wp-json\/wp\/v2\/media\/81"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/azure-sql\/wp-json\/wp\/v2\/media?parent=1248"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/azure-sql\/wp-json\/wp\/v2\/categories?post=1248"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/azure-sql\/wp-json\/wp\/v2\/tags?post=1248"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}