{"id":16168,"date":"2023-11-07T09:45:24","date_gmt":"2023-11-07T17:45:24","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/?p=16168"},"modified":"2023-11-07T09:44:13","modified_gmt":"2023-11-07T17:44:13","slug":"lets-build-a-custom-microsoft-graph-connector","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/lets-build-a-custom-microsoft-graph-connector\/","title":{"rendered":"Let\u2019s build a custom Microsoft Graph connector"},"content":{"rendered":"<p>Welcome back to our journey into the world of Microsoft Graph connectors! If you&#8217;re eager to integrate external data sources into Microsoft Graph to appear across Microsoft 365 experiences, you&#8217;re in the right place. This post provides a step-by-step guide to creating custom connections, opening the door to new data insights and collaboration capabilities. Let&#8217;s dive in!<\/p>\n<h2>Getting started<\/h2>\n<p>The best way to understand how Microsoft Graph connectors work is to look at a simple example. For this article, we\u2019ll use a <a href=\"https:\/\/adoption.microsoft.com\/en-us\/sample-solution-gallery\/sample\/pnp-graph-connector-dotnet-csharp-markdown\/\">.NET-based sample<\/a> that shows how you can ingest local markdown files extracting metadata from their frontmatter (e.g., title, description, tags, date, etc.).<\/p>\n<p><img decoding=\"async\" class=\"aligncenter wp-image-16171\" src=\"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-content\/uploads\/sites\/73\/2023\/11\/word-image-16168-3.png\" alt=\"Screenshot of cocal markdown files imported to Microsoft 365 using a custom Microsoft Graph connector displayed in Microsoft Search results\" width=\"1099\" height=\"566\" srcset=\"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-content\/uploads\/sites\/73\/2023\/11\/word-image-16168-3.png 1099w, https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-content\/uploads\/sites\/73\/2023\/11\/word-image-16168-3-300x155.png 300w, https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-content\/uploads\/sites\/73\/2023\/11\/word-image-16168-3-1024x527.png 1024w, https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-content\/uploads\/sites\/73\/2023\/11\/word-image-16168-3-768x396.png 768w\" sizes=\"(max-width: 1099px) 100vw, 1099px\" \/><\/p>\n<p><strong>PRO TIP: <\/strong>We have samples available in other programming languages available too. See Graph connector samples in the <a href=\"https:\/\/adoption.microsoft.com\/en-us\/sample-solution-gallery\/?keyword=&amp;sort-by=updateDateTime-true&amp;page=1&amp;product=Microsoft+Graph+connectors\">Sample Solution Gallery<\/a> for the latest Graph connector samples.<\/p>\n<p>The sample is a .NET console application project that consists of the following key files:<\/p>\n<ul>\n<li><strong>setup.ps1<\/strong> &#8211; a PowerShell script that creates an Entra app registration, grants it the necessary API permissions with admin consent and configures a client secret<\/li>\n<li><strong>ConnectionConfiguration.cs<\/strong> &#8211; defines the external connection and its schema<\/li>\n<li><strong>ConnectionService.cs<\/strong> &#8211; implements the portion of the connector that manages the external connection<\/li>\n<li><strong>ContentService.cs<\/strong> &#8211; implements the portion of the connector that manages external content<\/li>\n<\/ul>\n<h2>Connection configuration<\/h2>\n<p>Because the Microsoft Graph .NET SDK is strongly typed, you have access to the types, models, and properties as you configure the connection using Microsoft Graph entities from the SDK.<\/p>\n<p><script src=\"https:\/\/gist.github.com\/waldekmastykarz\/207c47910e8c0a59e044a7ef8f455754.js\"><\/script><\/p>\n<h2>Connection service<\/h2>\n<p>The connection service is responsible for managing the external connection where you&#8217;ll ingest content from outside sources. In this sample, it creates the external connection and configures its schema using the information from the ConnectionConfiguration class.<\/p>\n<p><script src=\"https:\/\/gist.github.com\/waldekmastykarz\/d6eda0d7ae228d6fefd9c1ce5e7eb2b2.js\"><\/script><\/p>\n<p><strong>PRO TIP:<\/strong> Creating schema is a long-running operation, taking upwards of a few minutes. To wait for its completion, the sample uses a <a href=\"https:\/\/github.com\/pnp\/graph-connectors-samples\/blob\/a49eab384f5b55e1fc9fdeb4ec53debdd001aa32\/samples\/dotnet-csharp-markdown\/CompleteJobWithDelayHandler.cs\">custom middleware<\/a>.<\/p>\n<h2>Content service<\/h2>\n<p>The content service is responsible for processing and enriching the external data items that are ingested into Microsoft Graph by Graph connectors so that it matches the schema defined by the external connection. It performs tasks such as indexing, extraction, normalization, and enrichment of the data items.<\/p>\n<p><script src=\"https:\/\/gist.github.com\/waldekmastykarz\/5b5dc09759f97ee3611d2d2bb4a1db16.js\"><\/script><\/p>\n<h2>Extract, Transform, Load&#8230; sound familiar?<\/h2>\n<p>If you\u2019ve worked with data integration in the past, the <strong>Extract Transform Load<\/strong> (ETL) term will sound familiar \u2013 that\u2019s exactly what a Graph connector is doing! It\u2019s extracting data from a source store, changing its shape and loading it to a destination store.<\/p>\n<p>The contents of the Extract and Transform methods will be different for each data source.<\/p>\n<p>The <strong>Extract method<\/strong> connects to the external data source and loads its data. If you\u2019re importing content that\u2019s secured and shouldn\u2019t be visible to everyone in the organization, be sure to retrieve the permissions configured on the items.<\/p>\n<p><strong>PRO TIP: <\/strong>Read more about sending access control lists (ACLs) on items <a href=\"https:\/\/learn.microsoft.com\/en-us\/graph\/connecting-external-content-manage-items#access-control-list\">here;<\/a> and how to create external groups if ACLs from the source system are different from Microsoft Entra IDs <a href=\"https:\/\/learn.microsoft.com\/en-us\/graph\/connecting-external-content-external-groups\">here<\/a>.<\/p>\n<p>Also, if you work with a large data source, you should consider only retrieving content that has changed since the last ingestion. The result of the Extract method is a collection of entities that represent content and metadata (and optionally permissions and activities) from the external system.<\/p>\n<p>The <strong>Transform method<\/strong> is responsible for transforming the data to match the schema defined on the external connection. Here, you\u2019ll also ensure that each item has a unique ID, dates are properly formatted, and URLs are absolute so that users can navigate to the original content items in the external system. The Transform method returns a collection of ExternalItem objects that represent the items from the external system that should be ingested to Microsoft 365.<\/p>\n<p><strong>PRO TIP:<\/strong> While you could combine the Extract and Transform method in one, we suggest keeping them separate. It helps separate the concerns of these methods and maintain a better overview of the ingestion process.<\/p>\n<p>Finally, the <strong>Load method<\/strong> is responsible for ingesting external items to Microsoft 365 using Microsoft Graph APIs. If your Graph connector supports incremental ingestion, this is where you should record the date and time of the last successful ingestion.<\/p>\n<h2>It\u2019s time to build<\/h2>\n<p>Now that we know how a Graph connector is built, let\u2019s run it to see it working in practice.<\/p>\n<p><strong>PRO TIP:<\/strong> Before continuing, be sure to install prerequisites listed in the sample\u2019s <a href=\"https:\/\/github.com\/pnp\/graph-connectors-samples\/tree\/a49eab384f5b55e1fc9fdeb4ec53debdd001aa32\/samples\/dotnet-csharp-markdown\">readme file<\/a>.<\/p>\n<h2>Create Microsoft Entra app registration<\/h2>\n<p>To begin, let\u2019s create the Microsoft Entra app registration. Start PowerShell and run <strong>.\\setup.ps1<\/strong>. The script will use the Microsoft Graph PowerShell SDK to create a new Entra app registration. It will <a id=\"post-16168-_Int_TtkKgwiK\"><\/a>grant it API permissions which the Graph connector requires to create the external connection and ingest content. It will also grant admin consent for these permissions. Finally, it will create a client secret and store information about the Entra app registration in .NET user secrets so that the Graph connector\u2019s code can access it.<\/p>\n<h2>Build the Graph connector<\/h2>\n<p>Next, build the Graph connector\u2019s code. In the command line, run <strong>dotnet build<\/strong> or use its equivalent in your IDE. Change the working directory to <strong>bin\\Debug\\net7.0<\/strong> where the compiled binaries are stored. Then, run <strong>.\\connector create-connection<\/strong>. This will use the connection service to create the external connection and configure its schema.<\/p>\n<p><strong>PRO TIP:<\/strong> Provisioning schema takes several minutes, and we recommend that you check its status every minute to avoid unnecessary use of Microsoft Graph APIs.<\/p>\n<p>When the connection is created, you can proceed with importing the content. In the terminal <a id=\"post-16168-_Int_Um0ZcOu3\"><\/a>run <strong>.\\connector load-content<\/strong>. This will start the content service and have it extract, transform and load the content.<\/p>\n<h2>Include content in all search results<\/h2>\n<p>To see the imported content in search results, the easiest way is to include it in the All results vertical. In the browser, go to <strong>admin.microsoft.com<\/strong>. From the side navigation, choose <strong>Settings &gt; Search &amp; intelligence<\/strong>. Activate the <strong>Data sources<\/strong> tab.<\/p>\n<p><img decoding=\"async\" width=\"1099\" height=\"445\" class=\"wp-image-16172\" src=\"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-content\/uploads\/sites\/73\/2023\/11\/word-image-16168-4.png\" srcset=\"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-content\/uploads\/sites\/73\/2023\/11\/word-image-16168-4.png 1099w, https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-content\/uploads\/sites\/73\/2023\/11\/word-image-16168-4-300x121.png 300w, https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-content\/uploads\/sites\/73\/2023\/11\/word-image-16168-4-1024x415.png 1024w, https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-content\/uploads\/sites\/73\/2023\/11\/word-image-16168-4-768x311.png 768w\" sizes=\"(max-width: 1099px) 100vw, 1099px\" \/><\/p>\n<p>In the list of data sources, locate the newly created <strong>Waldek Mastykarz (blog); .NET<\/strong> source and click the <strong>Include Connector Results<\/strong> link to include content from this data source in all results.<\/p>\n<p><img decoding=\"async\" width=\"1101\" height=\"103\" class=\"wp-image-16173\" src=\"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-content\/uploads\/sites\/73\/2023\/11\/word-image-16168-5.png\" srcset=\"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-content\/uploads\/sites\/73\/2023\/11\/word-image-16168-5.png 1101w, https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-content\/uploads\/sites\/73\/2023\/11\/word-image-16168-5-300x28.png 300w, https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-content\/uploads\/sites\/73\/2023\/11\/word-image-16168-5-1024x96.png 1024w, https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-content\/uploads\/sites\/73\/2023\/11\/word-image-16168-5-768x72.png 768w\" sizes=\"(max-width: 1101px) 100vw, 1101px\" \/><\/p>\n<p>Next, go to microsoft365.com and sign in with your work account. In the navbar, search for <strong>how to test api<\/strong>. You should see a result like the following:<\/p>\n<p><img decoding=\"async\" width=\"1100\" height=\"458\" class=\"wp-image-16174\" src=\"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-content\/uploads\/sites\/73\/2023\/11\/word-image-16168-6.png\" srcset=\"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-content\/uploads\/sites\/73\/2023\/11\/word-image-16168-6.png 1100w, https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-content\/uploads\/sites\/73\/2023\/11\/word-image-16168-6-300x125.png 300w, https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-content\/uploads\/sites\/73\/2023\/11\/word-image-16168-6-1024x426.png 1024w, https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-content\/uploads\/sites\/73\/2023\/11\/word-image-16168-6-768x320.png 768w\" sizes=\"(max-width: 1100px) 100vw, 1100px\" \/><\/p>\n<p>Congratulations! You\u2019ve just used Microsoft Search to find external content ingested using a custom Microsoft Graph connector.<\/p>\n<h2>Next steps<\/h2>\n<p>You\u2019ve just built and used a simple custom Microsoft Graph connector built on .NET that ingests local markdown files to Microsoft 365. Microsoft Graph connectors offer a lot of flexibility for controlling what content is imported to Microsoft Graph and who can access it.<\/p>\n<p><a href=\"https:\/\/adoption.microsoft.com\/en-us\/sample-solution-gallery\/?keyword=&amp;sort-by=&amp;page=1&amp;product=Microsoft+Graph+connectors\">Try other Graph connector samples<\/a> built by Microsoft and the Microsoft 365 community, to see what\u2019s possible, and check out our documentation to <a href=\"https:\/\/learn.microsoft.com\/graph\/connecting-external-content-connectors-overview\">learn more about Microsoft Graph connectors<\/a>.<\/p>\n<p>More resources:<\/p>\n<ul>\n<li>Visit our Microsoft 365 Dev Center<\/li>\n<li>Follow us on <a href=\"https:\/\/twitter.com\/Microsoft365Dev\">Microsoft 365 Developer (@Microsoft365Dev) \/ X<\/a> for the latest news and announcements<\/li>\n<li>Check out demos and videos on <a href=\"https:\/\/www.youtube.com\/@Microsoft365Developer\">Microsoft 365 Developer &#8211; YouTube<\/a><\/li>\n<\/ul>\n<p>Happy coding!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Integrate external data sources into Microsoft Graph to appear across Microsoft 365 experiences with custom Graph connectors.<\/p>\n","protected":false},"author":74222,"featured_media":16206,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[3],"tags":[157],"class_list":["post-16168","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-microsoft-graph","tag-microsoft-graph-connectors"],"acf":[],"blog_post_summary":"<p>Integrate external data sources into Microsoft Graph to appear across Microsoft 365 experiences with custom Graph connectors.<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-json\/wp\/v2\/posts\/16168","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-json\/wp\/v2\/users\/74222"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-json\/wp\/v2\/comments?post=16168"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-json\/wp\/v2\/posts\/16168\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-json\/wp\/v2\/media\/16206"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-json\/wp\/v2\/media?parent=16168"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-json\/wp\/v2\/categories?post=16168"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-json\/wp\/v2\/tags?post=16168"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}