{"id":41393,"date":"2023-07-19T00:29:59","date_gmt":"2023-07-19T07:29:59","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/premier-developer\/?p=41393"},"modified":"2023-07-17T06:45:14","modified_gmt":"2023-07-17T13:45:14","slug":"part-1-unlock-the-power-of-azure-data-factory-a-guide-to-boosting-your-data-ingestion-process","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/premier-developer\/part-1-unlock-the-power-of-azure-data-factory-a-guide-to-boosting-your-data-ingestion-process\/","title":{"rendered":"Part 1 – Unlock the Power of Azure Data Factory: A Guide to Boosting Your Data Ingestion Process"},"content":{"rendered":"

John Folberth<\/a> and Joe Fitzgerald<\/a> share sample guidance for developing and deploying an Azure Data Factory into multiple environments.<\/p>\n

\n

Introduction<\/h3>\n

In the fast-paced world of cloud architecture, securely collecting, ingesting, and preparing data for health care industry solutions has become an essential requirement. And that’s where\u00a0Azure Data Factory<\/a>\u00a0(ADF) comes in. As an Extract, Transform, and Load (ETL) cloud service, ADF empowers you to scale-out serverless data integration and data transformation with ease.<\/p>\n

Imagine being able to effortlessly create data-driven workflows that orchestrate data movement and transform massive amounts of data in a single stroke. With ADF’s code-free UI, intuitive authoring, and comprehensive monitoring and management capabilities, you can turn this vision into reality.<\/p>\n

The health care industry presents numerous opportunities where ADF can play a pivotal role. From\u00a0donor-patient cross-matching<\/a>\u00a0to\u00a0health data consortiums<\/a>,\u00a0risk prediction for surgeries<\/a>, and\u00a0population health management<\/a>, ADF can be a game-changer in delivering efficient and effective solutions.<\/p>\n

However, transitioning from an architecture diagram to a fully functional data factory in a real-world scenario is no small feat. Many organizations start with an ADF in a development environment and eventually need to promote it to staging and production environments. So inevitably, there\u2019s that moment when you impress your team and manager because you have the Azure Data Factory working just the way you want it to be working in the development environment. Then your manager says, \u201cThis is Great! Let\u2019s get it into Staging and Production as soon as we can!\u201d And then you realize, \u201cUh Oh!\u201d, this isn\u2019t going to be easy! This process can be daunting, but fear not \u2013 we’re here to guide you every step of the way.<\/p>\n

In this multi-part blog series,\u00a0@Joe_Fitzgerald<\/a>\u00a0and\u00a0@j_folberth<\/a>\u00a0will provide you with invaluable sample guidance for developing and deploying an Azure Data Factory to multiple environments. By the end of this series, you’ll be equipped with the skills and knowledge to proudly add “DevOps Engineer” to your professional title.<\/p>\n

This blog series provides a sample guide that will cover the following topics:<\/p>\n

Part 1<\/a><\/p>\n

    \n
  1. Architecture and Scenario<\/li>\n
  2. Creating resources in Azure<\/li>\n
  3. Create Azure Storage Containers<\/li>\n
  4. Create Azure Key Vaults<\/li>\n
  5. Create Azure Data Factory: With Key Vault Access<\/li>\n<\/ol>\n

    Part 2<\/p>\n

      \n
    1. Configure Azure Data Factory Source Control<\/li>\n
    2. Construct Azure Data Factory Data Pipeline<\/li>\n
    3. Publishing Concept for Azure Data Factory<\/li>\n
    4. Configure Deployed Azure Resources.<\/li>\n<\/ol>\n

      Part 3<\/p>\n

        \n
      1. The YAML Pipeline Structure<\/li>\n
      2. The Publish Process<\/li>\n
      3. ARM Template Parameterization<\/li>\n
      4. ADF ARM Template Deployment<\/li>\n<\/ol>\n

        Part 4<\/p>\n

          \n
        1. How to use Azure DevOps Pipeline Templates<\/li>\n<\/ol>\n

          Part 5<\/p>\n

            \n
          1. How to deploy Azure Data Factory Linked Templates<\/li>\n<\/ol>\n

            Check out the series in the Healthcare and Life Sciences Tech Community here<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"

            John Folberth and Joe Fitzgerald share sample guidance for developing and deploying an Azure Data Factory into multiple environments. Introduction In the fast-paced world of cloud architecture, securely collecting, ingesting, and preparing data for health care industry solutions has become an essential requirement. And that’s where\u00a0Azure Data Factory\u00a0(ADF) comes in. As an Extract, Transform, and […]<\/p>\n","protected":false},"author":582,"featured_media":41394,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[10643],"tags":[154,2571],"class_list":["post-41393","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-azure-data-factory","tag-azure-data-factory","tag-azure-devops"],"acf":[],"blog_post_summary":"

            John Folberth and Joe Fitzgerald share sample guidance for developing and deploying an Azure Data Factory into multiple environments. Introduction In the fast-paced world of cloud architecture, securely collecting, ingesting, and preparing data for health care industry solutions has become an essential requirement. And that’s where\u00a0Azure Data Factory\u00a0(ADF) comes in. As an Extract, Transform, and […]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/posts\/41393","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/users\/582"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/comments?post=41393"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/posts\/41393\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/media\/41394"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/media?parent=41393"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/categories?post=41393"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/tags?post=41393"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}