{"id":1209,"date":"2020-05-19T08:30:21","date_gmt":"2020-05-19T15:30:21","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/cosmosdb\/?p=1209"},"modified":"2020-05-19T00:18:16","modified_gmt":"2020-05-19T07:18:16","slug":"announce-azure-synapse-link-no-etl-analytics","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/cosmosdb\/announce-azure-synapse-link-no-etl-analytics\/","title":{"rendered":"Unlock near real-time, no-ETL analytics at scale with Azure Synapse Link for Azure Cosmos DB"},"content":{"rendered":"<h6><span style=\"font-size: 12pt;\"><em>Co-authored by Sri Chintala and Ramnandan Krishnamurthy, Azure Cosmos DB Team<\/em><\/span><\/h6>\n<p>In today\u2019s environment, adopting a data-driven culture has become critical for enterprises across industries. At the core of this culture is the need to deliver analytics over operational data in near real-time to optimize business processes and unlock new opportunities. Over the years, customers have chosen Azure Cosmos DB as the fully managed, blazingly fast NoSQL database for modern application development at any scale. With the growing volume of operational data and the opportunity to unlock powerful insights from this data, also come the challenges of handling and analyzing this data at scale.<\/p>\n<p>Traditionally, to ensure there is no performance impact on operational workloads, enterprises have maintained separate online transactional processing (OLTP) system and online analytical processing (OLAP) system. While the OLTP system is optimized for mission-critical transactional workloads, the OLAP system is optimized independently for analytical workloads over larger volumes of data. The friction and time to get data from OLTP systems to OLAP systems can cause a significant delay of hours, days and even weeks, which results in sub-optimal decisions made on stale data.<\/p>\n<h3>Announcing Azure Synapse Link for Azure Cosmos DB for cloud-native HTAP<\/h3>\n<p>We\u2019re excited to announce public preview of <a href=\"https:\/\/aka.ms\/cosmosdb-synapselink\">Azure Synapse Link for Azure Cosmos DB<\/a>, a cloud-native hybrid transactional and analytical processing (HTAP) capability that enables near-real time analytics over operational data in Azure Cosmos DB. Azure Synapse Link finally breaks down the barrier that has long existed between the OLTP and OLAP systems. Azure Synapse Link \u201clinks\u201d your Azure Cosmos DB to Synapse Analytics in Azure, providing the ability to get immediate insights on your business.<\/p>\n<p>With just a \u2018single click\u2019 you can now analyze large volumes of operational data in Azure Cosmos DB <strong>in near real-time <\/strong>with <strong>no ETL pipelines<\/strong> and <strong>no performance impact on transactional workloads<\/strong>.<\/p>\n<p><img decoding=\"async\" class=\"wp-image-1211\" src=\"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2020\/05\/a-circuit-board-description-automatically-generat-2.png\" alt=\"A circuit board Description automatically generated\" \/><\/p>\n<p>This capability unlocks new business scenarios to raise alerts based on live trends, build near real-time dashboards, and business experiences based on user behavior. Business analysts, data engineers and data scientists can now use Azure Synapse Analytics to run near real-time BI, AI and other big data analytical workloads over operational data in Azure Cosmos DB at scale.<\/p>\n<h3>Sneak-peek into the magic of Azure Synapse Link<\/h3>\n<h2><img decoding=\"async\" width=\"1241\" height=\"498\" class=\"wp-image-1212\" style=\"font-size: 1rem;\" src=\"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2020\/05\/word-image-2.png\" srcset=\"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2020\/05\/word-image-2.png 1241w, https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2020\/05\/word-image-2-300x120.png 300w, https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2020\/05\/word-image-2-1024x411.png 1024w, https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2020\/05\/word-image-2-768x308.png 768w\" sizes=\"(max-width: 1241px) 100vw, 1241px\" \/><\/h2>\n<p>Azure Synapse Link is comprised of two main components:<\/p>\n<ul>\n<li><strong>Azure Cosmos DB analytical store<\/strong>: A fully-managed column-oriented \u2018analytical store\u2019 within containers in addition to the existing row-oriented \u2018transactional store\u2019. The analytical store is fully isolated from the transactional store such that queries over the analytical store have no impact on your transactional workloads. Updates to the operational data are automatically synced from transactional store to analytical store in near real-time within minutes. Learn more about <a href=\"https:\/\/aka.ms\/analytical-store-overview\">Azure Cosmos DB analytical store<\/a>.<\/li>\n<li><strong>Azure Synapse Analytics run-time support:<\/strong> Native integration of the Azure Cosmos DB analytical store with the various analytics runtimes supported by Azure Synapse Analytics, which can query the analytical store directly. No further data transformations are required to analyze data from the analytical store. As of today, Azure Synapse Analytics supports Apache Spark and Synapse SQL serverless with Azure Cosmos DB analytical store.<\/li>\n<\/ul>\n<p>Below you can see some examples of the power of Azure Synapse Link:<\/p>\n<ul>\n<li>Create Synapse Spark tables over Azure Cosmos DB containers with a simplified programming model. Then leverage the power of Spark\u2019s distributed processing to perform joins and other complex aggregations between Spark tables. This allows you to transform and enrich the operational data in Azure Cosmos DB directly with Synapse Spark.<\/li>\n<\/ul>\n<p><img decoding=\"async\" width=\"1430\" height=\"553\" class=\"wp-image-1213\" src=\"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2020\/05\/a-screenshot-of-a-social-media-post-description-a-4.png\" alt=\"A screenshot of a social media post Description automatically generated\" srcset=\"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2020\/05\/a-screenshot-of-a-social-media-post-description-a-4.png 1430w, https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2020\/05\/a-screenshot-of-a-social-media-post-description-a-4-300x116.png 300w, https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2020\/05\/a-screenshot-of-a-social-media-post-description-a-4-1024x396.png 1024w, https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2020\/05\/a-screenshot-of-a-social-media-post-description-a-4-768x297.png 768w\" sizes=\"(max-width: 1430px) 100vw, 1430px\" \/><\/p>\n<ul>\n<li>Build ML models over operational data in Azure Cosmos DB directly with Azure Synapse Analytics\u2019 native integration with Azure Machine Learning and Apache Spark ML. Then deploy these models for real-time scoring in applications.<\/li>\n<\/ul>\n<p><img decoding=\"async\" width=\"1431\" height=\"794\" class=\"wp-image-1214\" src=\"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2020\/05\/a-screenshot-of-a-social-media-post-description-a-5.png\" alt=\"A screenshot of a social media post Description automatically generated\" srcset=\"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2020\/05\/a-screenshot-of-a-social-media-post-description-a-5.png 1431w, https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2020\/05\/a-screenshot-of-a-social-media-post-description-a-5-300x166.png 300w, https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2020\/05\/a-screenshot-of-a-social-media-post-description-a-5-1024x568.png 1024w, https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2020\/05\/a-screenshot-of-a-social-media-post-description-a-5-768x426.png 768w\" sizes=\"(max-width: 1431px) 100vw, 1431px\" \/><\/p>\n<ul>\n<li>Query operational data from Azure Cosmos DB and build SQL views leveraging the native integration with Synapse SQL serverless and its full expressiveness of T-SQL language. Model and publish auto-refreshing BI dashboards over Azure Cosmos DB directly with familiar BI tools such as Power BI.<\/li>\n<\/ul>\n<p><img decoding=\"async\" width=\"1320\" height=\"634\" class=\"wp-image-1215\" src=\"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2020\/05\/a-screenshot-of-a-cell-phone-description-automati-2.png\" alt=\"A screenshot of a cell phone Description automatically generated\" srcset=\"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2020\/05\/a-screenshot-of-a-cell-phone-description-automati-2.png 1320w, https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2020\/05\/a-screenshot-of-a-cell-phone-description-automati-2-300x144.png 300w, https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2020\/05\/a-screenshot-of-a-cell-phone-description-automati-2-1024x492.png 1024w, https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2020\/05\/a-screenshot-of-a-cell-phone-description-automati-2-768x369.png 768w\" sizes=\"(max-width: 1320px) 100vw, 1320px\" \/><\/p>\n<p>Here are a few common use cases across industry verticals which benefit from Azure Synapse Link:<\/p>\n<ul>\n<li>Supply Chain Analytics, Forecasting &amp; Reporting<\/li>\n<li>Real-time personalization<\/li>\n<li>Predictive Maintenance, Anomaly Detection in IoT scenarios<\/li>\n<\/ul>\n<p>Read more about <a href=\"https:\/\/aka.ms\/cosmosdb-analytical-usecases\">Azure Synapse Link use cases<\/a>.<\/p>\n<p>With Azure Synapse Link for Azure Cosmos DB, enterprises can now focus on unlocking precious insights from their operational data in near real-time at scale. By exploiting near real-time data to power predictive analytics, enterprises can identify emerging marketing opportunities and make business decisions as quickly as the data arrives. This will truly differentiate the leaders from the followers in the ongoing digital transformation journey.<\/p>\n<h3><strong>Next Steps<\/strong><\/h3>\n<p>To learn more, check out our documentation on <a href=\"https:\/\/aka.ms\/cosmosdb-synapselink\">Azure Synapse Link for Azure Cosmos DB<\/a>. To get started, check out our guide on <a href=\"https:\/\/aka.ms\/synapse-link-how-to\">how-to configure Azure Synapse Link <\/a> and visit our <a href=\"https:\/\/aka.ms\/synapse-link-faq\">FAQ page<\/a> for answers to commonly asked questions. You can also check out samples to get started on <a href=\"https:\/\/aka.ms\/cosmosdb-synapselink-samples\">Github<\/a>.<\/p>\n<p>For any feedback or suggestions to improve our product offering, please reach out to us directly at <a href=\"mailto:cosmosdbsynapselink@microsoft.com\">cosmosdbsynapselink@microsoft.com<\/a>.<\/p>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Learn about how you can use Azure Synapse Link for Azure Cosmos DB to analyze large volumes of operational data directly in near real-time with no ETL pipelines and no performance impact on transactional workloads<\/p>\n","protected":false},"author":26969,"featured_media":1211,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1216,12,13],"tags":[1242,1227,1234,1229],"class_list":["post-1209","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-analytics","category-announcements","category-news","tag-analytics","tag-azure-synapse-analytics","tag-bi","tag-htap"],"acf":[],"blog_post_summary":"<p>Learn about how you can use Azure Synapse Link for Azure Cosmos DB to analyze large volumes of operational data directly in near real-time with no ETL pipelines and no performance impact on transactional workloads<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/posts\/1209","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/users\/26969"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/comments?post=1209"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/posts\/1209\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/media\/1211"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/media?parent=1209"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/categories?post=1209"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/tags?post=1209"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}