{"id":9195,"date":"2021-12-16T08:00:19","date_gmt":"2021-12-16T16:00:19","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/?p=9195"},"modified":"2021-12-16T08:59:17","modified_gmt":"2021-12-16T16:59:17","slug":"microsoft-graph-data-connect-demystifying-user-scopes","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/microsoft-graph-data-connect-demystifying-user-scopes\/","title":{"rendered":"Microsoft Graph Data Connect: Demystifying User Scopes"},"content":{"rendered":"

Microsoft Graph Data Connect allows you to extract data in bulk from your Microsoft 365 environments using Azure Data Factory pipelines. When creating a pipeline to extract Microsoft 365 data using Microsoft Graph Data Connect, you need to define what I refer to as a \u201cData Contract\u201d. That contract represents the scope, properties and filters that you will apply to the data that you\u2019re retrieving. Users will always need to define what Output columns they wish to extract (e.g. toRecipients and cc Recipients for email messages, ReplyTo and Body for Teams Chat messages, etc.) and for most data sets they will also need to specify a date range for the objects to be extracted. Some of the data sets that are available in Microsoft Graph Data Connect also allow users to specify a user scope for the extraction (e.g. messages data set, Teams chat data set, contacts data set, etc.)<\/p>\n

User Scope<\/h3>\n

The user scope option lets you either specify that you wish to extract objects for All users in the Office 365 tenant<\/em> or to Select groups from the Office 365 tenant<\/em>. Selecting the latter option will allow you to select one or more Azure Active Directory Security or Microsoft 365 groups. In talking to customers and partners, we\u2019ve come to realize that there is some confusion around how this user scope option works, and what effect it will have on the resulting data extracted. This article covers different scenarios to set the record straight and help you understand how to properly leverage the user scope in your Microsoft Graph Data Connect solutions.<\/p>\n

Examples<\/h4>\n

To better illustrate the functioning of the Microsoft Graph Data Connect user scope, we will be using a few examples we designed using one of our test Microsoft 365 tenant. In our scenarios, we have created a new Azure Active Directory security group named Alex and Allan<\/strong> which contains two members from our organization: Alex Wilber and Allan Deyoung. Both members have been assigned an Office 365 license and have been exchanging emails with their peers.<\/p>\n

We have built an Azure Data Factory pipeline which extracts information using the Microsoft Graph Data Connect BasicDataSet_v0.Message_v1 <\/strong>data set which represents all emails in users\u2019 mailboxes. We filtered the Output columns <\/em>to only keep the id, from, toRecipients, ccRecipients and bccRecipients fields. In the Date <\/em>filter section, we only selected the past 24 hours as the timespan to retrieve emails based on their sent date. Using the User scope<\/em> option, we selected the newly created security group named Alex and<\/strong> Allan<\/strong> as the only group of users to extract information from.<\/p>\n

\"Overview<\/a><\/p>\n

Figure 1: Overview of the Copy Data activity’s source<\/span><\/em><\/p>\n

Now that we have our pipeline defined, let us take a look at various instances of emails revolving around Alex and Allan. The following table summarizes various test emails that were sent across members of our test organization and whether or not these are going to be captured by our pipeline.<\/p>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
From<\/u><\/strong><\/span><\/td>\nTo<\/u><\/strong><\/span><\/td>\nCC<\/u><\/strong><\/span><\/td>\nBCC<\/u><\/strong><\/span><\/td>\nIncluded by MGDC Messages?<\/u><\/strong><\/span><\/td>\n<\/tr>\n
Nestor Wilke<\/span><\/td>\nAlex Wilber<\/span><\/td>\n<\/td>\n<\/td>\n\u2714\ufe0f<\/span><\/td>\n<\/tr>\n
Nestor Wilke<\/span><\/td>\nAllan Deyoung<\/span><\/td>\n<\/td>\n<\/td>\n\u2714\ufe0f<\/span><\/td>\n<\/tr>\n
Nestor Wilke<\/span><\/td>\nAlex Wilber<\/span><\/td>\nJ.Smith@external.com<\/span><\/td>\n<\/td>\n\u2714\ufe0f<\/span><\/td>\n<\/tr>\n
Nestor Wilke<\/span><\/td>\nMiriam Graham<\/span><\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/tr>\n
Nestor Wilke<\/span><\/td>\nMiriam Graham<\/span><\/td>\nAlex Wilber<\/span><\/td>\n<\/td>\n\u2714\ufe0f<\/span><\/td>\n<\/tr>\n
Nestor Wilke<\/span><\/td>\nMiriam Graham<\/span><\/td>\n<\/td>\nAllan Deyoung<\/span><\/td>\n\u2714\ufe0f<\/span><\/td>\n<\/tr>\n
Nestor Wilke<\/span><\/td>\nMiriam Graham<\/span><\/p>\n

Allan Deyoung<\/span><\/td>\n

<\/td>\n<\/td>\n\u2714\ufe0f<\/span><\/td>\n<\/tr>\n
Allan Deyoung<\/span><\/td>\nMegan Graham<\/span><\/td>\n<\/td>\n<\/td>\n\u2714\ufe0f<\/span><\/td>\n<\/tr>\n
Allan Deyoung<\/span><\/td>\nMegan Graham<\/span><\/td>\nPatti Fernandez<\/span><\/td>\n<\/td>\n\u2714\ufe0f<\/span><\/td>\n<\/tr>\n
Allan Deyoung<\/span><\/td>\nMegan Graham<\/span><\/td>\n<\/td>\nPatti Fernandez<\/span><\/td>\n\u2714\ufe0f<\/span><\/td>\n<\/tr>\n
Allan Deyoung<\/span><\/td>\nAlex Wilber<\/span><\/td>\n<\/td>\n<\/td>\n\u2714\ufe0f<\/span><\/td>\n<\/tr>\n
Allan Deyoung<\/span><\/td>\nJ.Smith@external.com<\/span><\/td>\n<\/td>\n<\/td>\n\u2714\ufe0f<\/span><\/td>\n<\/tr>\n
Alex Wilber<\/span><\/td>\nMegan Graham<\/span><\/td>\n<\/td>\n<\/td>\n\u2714\ufe0f<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Table 1 \u2013 Scenarios for the messages data set<\/em><\/p>\n

From the table above, we can see that we are essentially returning every object where Alex Wilber or Allan Deyoung are at least in the From, To, CC or BCC field even if they include other actors. Only objects that only include actors other than Alex and Allan are being filtered out. Let us now consider the same examples, but this time for the Sent Items data set.<\/p>\n

The Sent Items data set only contains emails that were sent from the selected users\u2019 mailboxes. Using this data set over the Messages data set can sometime be a great way to filter noise from external emails and reduce duplication. We can see from Table 1-2 that we are getting different results than with the Messages data set:<\/p>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
From<\/u><\/strong><\/span><\/td>\nTo<\/u><\/strong><\/span><\/td>\nCC<\/u><\/strong><\/span><\/td>\nBCC<\/u><\/strong><\/span><\/td>\nIncluded by MGDC SentItems?<\/u><\/strong><\/span><\/td>\n<\/tr>\n
Nestor Wilke<\/span><\/td>\nAlex Wilber<\/span><\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/tr>\n
Nestor Wilke<\/span><\/td>\nAllan Deyoung<\/span><\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/tr>\n
Nestor Wilke<\/span><\/td>\nAlex Wilber<\/span><\/td>\nJ.Smith@external.com<\/span><\/td>\n<\/td>\n<\/td>\n<\/tr>\n
Nestor Wilke<\/span><\/td>\nMiriam Graham<\/span><\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/tr>\n
Nestor Wilke<\/span><\/td>\nMiriam Graham<\/span><\/td>\nAlex Wilber<\/span><\/td>\n<\/td>\n<\/td>\n<\/tr>\n
Nestor Wilke<\/span><\/td>\nMiriam Graham<\/span><\/td>\n<\/td>\nAllan Deyoung<\/span><\/td>\n<\/td>\n<\/tr>\n
Nestor Wilke<\/span><\/td>\nMiriam Graham<\/span><\/p>\n

Allan Deyoung<\/span><\/td>\n

<\/td>\n<\/td>\n<\/td>\n<\/tr>\n
Allan Deyoung<\/span><\/td>\nMegan Graham<\/span><\/td>\n<\/td>\n<\/td>\n\u2714\ufe0f<\/span><\/td>\n<\/tr>\n
Allan Deyoung<\/span><\/td>\nMegan Graham<\/span><\/td>\nPatti Fernandez<\/span><\/td>\n<\/td>\n\u2714\ufe0f<\/span><\/td>\n<\/tr>\n
Allan Deyoung<\/span><\/td>\nMegan Graham<\/span><\/td>\n<\/td>\nPatti Fernandez<\/span><\/td>\n\u2714\ufe0f<\/span><\/td>\n<\/tr>\n
Allan Deyoung<\/span><\/td>\nAlex Wilber<\/span><\/td>\n<\/td>\n<\/td>\n\u2714\ufe0f<\/span><\/td>\n<\/tr>\n
Allan Deyoung<\/span><\/td>\nJ.Smith@external.com<\/span><\/td>\n<\/td>\n<\/td>\n\u2714\ufe0f<\/span><\/td>\n<\/tr>\n
Alex Wilber<\/span><\/td>\nMegan Graham<\/span><\/td>\n<\/td>\n<\/td>\n\u2714\ufe0f<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Table 2 \u2013 Scenarios for the sent items data set<\/em><\/p>\n

The third data set is the newly released Inbox data set. This data set represents all emails that are currently sitting in a user\u2019s Inbox folder. It is very important to note that if a user uses folders inside of their mailbox for email classification, the emails in these folders will not be captured by the Inbox data set. The following two screenshots represent Alex Wilber\u2019s mailbox. Figure 2 shows the emails that Alex has stored directly within its Inbox folder, whereas Figure 3 depicts emails are moved under a folder named Project Arkana<\/em>.<\/p>\n

\"Alex<\/a><\/p>\n

Figure 2 – Alex Wilber’s Inbox folder<\/span><\/em><\/p>\n

\"Alex<\/a><\/p>\n

Figure 3 – Alex Wilber’s Project Arkana folder content<\/span><\/em><\/p>\n

Let us now configure our Azure Data Factory pipeline and extract emails for today using the Inbox data set. Just like the two previous examples, I am setting a sentDateTime filter to extract only emails for today, and I am using the User scope<\/strong> filter to only extract items for Alex and Allan.<\/p>\n

\"Inbox<\/a><\/p>\n

Figure 4 – Inbox pipeline configuration<\/span><\/em><\/p>\n

 <\/p>\n\n\n\n\n\n\n\n\n
Folder<\/u><\/strong><\/span><\/td>\nFrom<\/u><\/strong><\/span><\/td>\nSubject<\/u><\/strong><\/span><\/td>\nIncluded by MGDC Inbox?<\/u><\/strong><\/span><\/td>\n<\/tr>\n
Inbox<\/span><\/td>\nNestor Wilke<\/span><\/td>\nEmail from Nestor 1<\/span><\/td>\n\u2714\ufe0f<\/span><\/td>\n<\/tr>\n
Inbox<\/span><\/td>\nAllan Deyoung<\/span><\/td>\nEmail from Allan 2<\/span><\/td>\n\u2714\ufe0f<\/span><\/td>\n<\/tr>\n
Inbox<\/span><\/td>\nAllan Deyoung<\/span><\/td>\nEmail from Allan 3<\/span><\/td>\n\u2714\ufe0f<\/span><\/td>\n<\/tr>\n
Project Arkana<\/span><\/td>\nAllan Deyoung<\/span><\/td>\nEmail from Allan 1<\/span><\/td>\n<\/td>\n<\/tr>\n
Project Arkana<\/span><\/td>\nNestor Wilke<\/span><\/td>\nEmail from Nestor 2<\/span><\/td>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Table 3 \u2013 Scenarios for the inbox data set<\/em><\/p>\n

Get Started Today<\/h3>\n

Hopefully this article helped shed some light on how the User scope<\/strong> filter works inside of Microsoft Graph Data Connect and that it helped you better understand the differences between the Messages, Sent Items and Inbox data set. If you want to learn more about Microsoft Graph Data Connect you can refer to our official documentation and tutorials at Overview of Microsoft Graph Data Connect – Microsoft Graph | Microsoft Docs<\/a><\/p>\n

Happy Coding!<\/p>\n","protected":false},"excerpt":{"rendered":"

Microsoft Graph Data Connect allows you to extract data in bulk from your Microsoft 365 environments using Azure Data Factory pipelines. This article covers different scenarios to help you understand how to properly leverage the user scope in your Microsoft Graph Data Connect solutions.<\/p>\n","protected":false},"author":72603,"featured_media":9248,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[3],"tags":[31],"class_list":["post-9195","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-microsoft-graph","tag-microsoft-graph-data-connect"],"acf":[],"blog_post_summary":"

Microsoft Graph Data Connect allows you to extract data in bulk from your Microsoft 365 environments using Azure Data Factory pipelines. This article covers different scenarios to help you understand how to properly leverage the user scope in your Microsoft Graph Data Connect solutions.<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-json\/wp\/v2\/posts\/9195","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-json\/wp\/v2\/users\/72603"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-json\/wp\/v2\/comments?post=9195"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-json\/wp\/v2\/posts\/9195\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-json\/wp\/v2\/media\/9248"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-json\/wp\/v2\/media?parent=9195"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-json\/wp\/v2\/categories?post=9195"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-json\/wp\/v2\/tags?post=9195"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}