{"id":510,"date":"2025-02-21T02:41:36","date_gmt":"2025-02-21T02:41:36","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/all-things-azure\/?p=510"},"modified":"2025-02-21T02:41:36","modified_gmt":"2025-02-21T02:41:36","slug":"visualize-roi-of-your-github-copilot-usage-how-it-works","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/all-things-azure\/visualize-roi-of-your-github-copilot-usage-how-it-works\/","title":{"rendered":"Visualize ROI of your GitHub Copilot Usage, How it works!"},"content":{"rendered":"<div class=\"markdown-heading\" dir=\"auto\">\n<p dir=\"auto\" tabindex=\"-1\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/all-things-azure\/wp-content\/uploads\/sites\/83\/2024\/10\/HC0400_MS_AzureDeveloperBlogSeries-Banner_103124_DC_V2-02-2.png\" alt=\"Image HC0400 MS AzureDeveloperBlogSeries Banner 103124 DC V2 02 2\" \/><\/p>\n<\/div>\n<p dir=\"auto\" tabindex=\"-1\"><div class=\"alert alert-success\"><p class=\"alert-divider\"><i class=\"fabric-icon fabric-icon--Lightbulb\"><\/i><strong>Tips<\/strong><\/p>In today&#8217;s enterprise software development landscape, organizations are increasingly investing in AI-powered development tools like GitHub Copilot to boost developer productivity. However, the standard 28-day usage metrics often fall short of providing the comprehensive insights needed to truly understand ROI and adoption patterns across large organizations. The Copilot Usage Advanced Dashboard fills this critical gap by offering a powerful, enterprise-grade solution that captures and analyzes long-term usage data across multiple teams and organizations. Built with familiar tools like Elasticsearch and Grafana, this dashboard enables engineering leaders to track adoption trends, identify optimization opportunities, and quantify their Copilot investment&#8217;s impact through detailed breakdowns by language, editor, and team. Beyond just tracking lines of code, the dashboard provides insights into seat utilization, activation rates, and usage patterns that help organizations optimize their Copilot licenses and drive stronger developer adoption. For engineering teams evaluating or scaling their Copilot implementation, this tool provides the data-driven insights needed to make informed decisions about resource allocation and demonstrate clear business value to stakeholders.<\/div><\/p>\n<hr \/>\n<h2 dir=\"auto\" tabindex=\"-1\">Architecture: How it works!<\/h2>\n<p>The Copilot Usage Advanced Dashboard offers a comprehensive suite of features designed to revolutionize how organizations track and analyze their Copilot usage. At its core, the solution leverages Elasticsearch for persistent data storage, breaking free from the traditional 28-day limitation and enabling analysis across extended time periods, from months to years. The system intelligently structures data with Organization and Team fields, facilitating granular filtering and cross-organizational insights. Data integrity is maintained through unique hash key generation for each data point, ensuring accurate and up-to-date information through automatic updates. The Grafana-based visualization layer offers complete flexibility, allowing teams to customize dashboards according to their specific needs and stakeholder requirements. Proactive usage monitoring is enabled through Grafana&#8217;s built-in alerting system, which can notify administrators about potential issues such as prolonged user inactivity. Furthermore, the platform&#8217;s open architecture supports seamless integration with third-party systems, whether for exporting data to external visualization platforms or incorporating additional data sources for comprehensive analytics, making it a truly versatile solution for enterprise-scale Copilot management.<a href=\"https:\/\/github.com\/satomic\/copilot-usage-advanced-dashboard\">\u00a0Copilot Usage Advanced Dashboard <\/a>\u00a0almost fully utilizes data from Copilot APIs, The APIs used are listed below representing Copilot usage in multi organizations and teams from different dimensions:<\/p>\n<ul dir=\"auto\">\n<li><a title=\"List teams of an onganization\" href=\"https:\/\/docs.github.com\/en\/enterprise-cloud@latest\/rest\/teams\/teams?apiVersion=2022-11-28#list-teams\">List teams of an organization<\/a><\/li>\n<li><a title=\"Get a summary of Copilot usage for a team\" href=\"https:\/\/docs.github.com\/en\/enterprise-cloud@latest\/rest\/copilot\/copilot-metrics?apiVersion=2022-11-28#get-copilot-metrics-for-a-team\">Get a summary of Copilot metrics for a team<\/a><\/li>\n<li><a title=\"Get Copilot seat information and settings for an organization\" href=\"https:\/\/docs.github.com\/en\/enterprise-cloud@latest\/rest\/copilot\/copilot-user-management?apiVersion=2022-11-28#get-copilot-seat-information-and-settings-for-an-organization\">Get Copilot seat information and settings for an organization<\/a><\/li>\n<li><a title=\"List all Copilot seat assignments for an organization\" href=\"https:\/\/docs.github.com\/en\/enterprise-cloud@latest\/rest\/copilot\/copilot-user-management?apiVersion=2022-11-28#list-all-copilot-seat-assignments-for-an-organization\">List all Copilot seat assignments for an organization<\/a><\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p><figure id=\"attachment_822\" aria-labelledby=\"figcaption_attachment_822\" class=\"wp-caption alignnone\" ><a href=\"https:\/\/devblogs.microsoft.com\/all-things-azure\/wp-content\/uploads\/sites\/83\/2025\/02\/MS_AzureBlog_Batch-2_Blog-2_Diagram_021425.png\"><img decoding=\"async\" class=\"wp-image-822 size-large\" src=\"https:\/\/devblogs.microsoft.com\/all-things-azure\/wp-content\/uploads\/sites\/83\/2025\/02\/MS_AzureBlog_Batch-2_Blog-2_Diagram_021425-1024x837.png\" alt=\"Architecture: How it works!\" width=\"1024\" height=\"837\" srcset=\"https:\/\/devblogs.microsoft.com\/all-things-azure\/wp-content\/uploads\/sites\/83\/2025\/02\/MS_AzureBlog_Batch-2_Blog-2_Diagram_021425-1024x837.png 1024w, https:\/\/devblogs.microsoft.com\/all-things-azure\/wp-content\/uploads\/sites\/83\/2025\/02\/MS_AzureBlog_Batch-2_Blog-2_Diagram_021425-300x245.png 300w, https:\/\/devblogs.microsoft.com\/all-things-azure\/wp-content\/uploads\/sites\/83\/2025\/02\/MS_AzureBlog_Batch-2_Blog-2_Diagram_021425-768x628.png 768w, https:\/\/devblogs.microsoft.com\/all-things-azure\/wp-content\/uploads\/sites\/83\/2025\/02\/MS_AzureBlog_Batch-2_Blog-2_Diagram_021425-1536x1255.png 1536w, https:\/\/devblogs.microsoft.com\/all-things-azure\/wp-content\/uploads\/sites\/83\/2025\/02\/MS_AzureBlog_Batch-2_Blog-2_Diagram_021425.png 1920w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/a><figcaption id=\"figcaption_attachment_822\" class=\"wp-caption-text\">Architecture: How it works!<\/figcaption><\/figure><\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<h3 class=\"whitespace-pre-wrap break-words\"><strong>Data Collection<\/strong><\/h3>\n<p class=\"whitespace-pre-wrap break-words\">The system utilizes four key Copilot APIs to gather comprehensive usage data:<\/p>\n<ul>\n<li style=\"list-style-type: none;\">\n<ul class=\"-mt-1 [li&gt;&amp;]:mt-2 list-disc space-y-2 pl-8\">\n<li class=\"whitespace-normal break-words\">Organization team listing<\/li>\n<li class=\"whitespace-normal break-words\">Team usage summaries<\/li>\n<li class=\"whitespace-normal break-words\">Organizational seat information and settings<\/li>\n<li class=\"whitespace-normal break-words\">Seat assignment tracking<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<h3 class=\"whitespace-pre-wrap break-words\"><strong>Key Features<\/strong><\/h3>\n<h4>1. Long-term Data Persistence<\/h4>\n<ul>\n<li style=\"list-style-type: none;\">\n<ul class=\"-mt-1 [li&gt;&amp;]:mt-2 list-disc space-y-2 pl-8\">\n<li class=\"whitespace-normal break-words\">Historical data stored in Elasticsearch beyond the 28-day limit<\/li>\n<li class=\"whitespace-normal break-words\">Flexible time period selection for trend analysis<\/li>\n<li class=\"whitespace-normal break-words\">Granular organizational and team-level data filtering<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<h4>2. Advanced Analytics<\/h4>\n<ul>\n<li style=\"list-style-type: none;\">\n<ul class=\"-mt-1 [li&gt;&amp;]:mt-2 list-disc space-y-2 pl-8\">\n<li class=\"whitespace-normal break-words\">Language and editor usage breakdown<\/li>\n<li class=\"whitespace-normal break-words\">Team performance comparisons<\/li>\n<li class=\"whitespace-normal break-words\">Seat utilization metrics<\/li>\n<li class=\"whitespace-normal break-words\">Activation rate tracking<\/li>\n<li class=\"whitespace-normal break-words\">Custom alert configuration for inactive users<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<h4>3. Integration Capabilities<\/h4>\n<ul>\n<li style=\"list-style-type: none;\">\n<ul class=\"-mt-1 [li&gt;&amp;]:mt-2 list-disc space-y-2 pl-8\">\n<li class=\"whitespace-normal break-words\">Seamless integration with existing Grafana deployments<\/li>\n<li class=\"whitespace-normal break-words\">Export capabilities for third-party visualization tools<\/li>\n<li class=\"whitespace-normal break-words\">Flexible data source additions for comprehensive DevOps metrics<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<hr \/>\n<p>&nbsp;<\/p>\n<p><div class=\"alert alert-success\"><p class=\"alert-divider\"><i class=\"fabric-icon fabric-icon--Lightbulb\"><\/i><strong>Why Built This<\/strong><\/p>This project aims to offer a more comprehensive and lasting view of GitHub Copilot usage beyond the default 28-day limit. Recognizing that many customers prefer proven tools like Grafana for monitoring and analysis, I set out to create a solution that integrates seamlessly into existing workflows without forcing users to learn new technologies. By storing and updating data over time, supporting multiple organizations and teams, and providing flexible options for external data integration, this project helps users gain deeper insights into both ROI of Copilot usage or and related DevOps metrics\u2014such as DORA\u2014within a unified dashboard if they want. <\/div><\/p>\n<div class=\"markdown-heading\" dir=\"auto\">\n<h2 class=\"heading-element\" dir=\"auto\" tabindex=\"-1\">Online Demo Environment<\/h2>\n<\/div>\n<blockquote>\n<p dir=\"auto\">Two dashboards are designed, they both can exist at the same time in Grafana.<\/p>\n<\/blockquote>\n<div class=\"markdown-heading\" dir=\"auto\">\n<h3 class=\"heading-element\" dir=\"auto\" tabindex=\"-1\">Copilot Usage Advanced Dashboard<\/h3>\n<\/div>\n<blockquote>\n<p dir=\"auto\">Copilot Metrics Viewer compatible dashboard<\/p>\n<\/blockquote>\n<blockquote>\n<p dir=\"auto\">If you are familiar with the\u00a0<a href=\"https:\/\/github.com\/github-copilot-resources\/copilot-metrics-viewer\">copilot-metrics-viewer<\/a>\u00a0project, then please try this dashboard and use it in subsequent deployments.<\/p>\n<\/blockquote>\n<ul>\n<li style=\"list-style-type: none;\">\n<ul>\n<li dir=\"auto\">link: <a href=\"https:\/\/softrin.com\/d\/be7hpbvvhst8gc\/copilot-usage-advanced-dashboard\">https:\/\/softrin.com\/d\/be7hpbvvhst8gc\/copilot-usage-advanced-dashboard<\/a><\/li>\n<li dir=\"auto\">username\uff1a<code>demouser<\/code><\/li>\n<li dir=\"auto\">password\uff1a<code>demouser<\/code><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p dir=\"auto\"><a href=\"https:\/\/github.com\/satomic\/copilot-usage-advanced-dashboard\/blob\/main\/image\/image_KGwt1NLyRT.png\" target=\"_blank\" rel=\"noopener noreferrer\"><img decoding=\"async\" class=\"alignnone\" src=\"https:\/\/github.com\/satomic\/copilot-usage-advanced-dashboard\/raw\/main\/image\/image_KGwt1NLyRT.png\" alt=\"Copilot Usage Advanced Dashboard preview\" width=\"1342\" height=\"1221\" \/><\/a><\/p>\n<div class=\"markdown-heading\" dir=\"auto\">\n<h3 dir=\"auto\" tabindex=\"-1\"><\/h3>\n<h3 class=\"heading-element\" dir=\"auto\" tabindex=\"-1\">Copilot Usage Advanced Dashboard Original<\/h3>\n<\/div>\n<blockquote>\n<p dir=\"auto\">New designed dashboard<\/p>\n<\/blockquote>\n<ul>\n<li style=\"list-style-type: none;\">\n<ul>\n<li dir=\"auto\">Link: <a href=\"https:\/\/softrin.com\/d\/a98455d6-b401-4a53-80ad-7af9f97be6f4\/copilot-usage-advanced-dashboard-original\">https:\/\/softrin.com\/d\/a98455d6-b401-4a53-80ad-7af9f97be6f4\/copilot-usage-advanced-dashboard-original<\/a><\/li>\n<li dir=\"auto\">username\uff1a<code>demouser<\/code><\/li>\n<li dir=\"auto\">password\uff1a<code>demouser<\/code><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p dir=\"auto\"><a href=\"https:\/\/github.com\/satomic\/copilot-usage-advanced-dashboard\/blob\/main\/image\/cpuad_full_FkIGG_4fzg.png\" target=\"_blank\" rel=\"noopener noreferrer\"><img decoding=\"async\" class=\"alignnone\" src=\"https:\/\/github.com\/satomic\/copilot-usage-advanced-dashboard\/raw\/main\/image\/cpuad_full_FkIGG_4fzg.png\" alt=\"Copilot Usage Advanced Dashboard Original preview\" width=\"1261\" height=\"1270\" \/><\/a><\/p>\n<p>&nbsp;<\/p>\n<div class=\"markdown-heading\" dir=\"auto\">\n<h2 class=\"heading-element\" dir=\"auto\" tabindex=\"-1\"><span style=\"font-size: 24pt;\">Variables<\/span><\/h2>\n<\/div>\n<p dir=\"auto\">Supports four filtering varibales, namely<\/p>\n<ul dir=\"auto\">\n<li>Organization<\/li>\n<li>Team<\/li>\n<li>Language<\/li>\n<li>Editor<\/li>\n<\/ul>\n<p dir=\"auto\">The choice of variables is dynamically associated with the data display<\/p>\n<p dir=\"auto\"><a href=\"https:\/\/github.com\/satomic\/copilot-usage-advanced-dashboard\/blob\/main\/image\/image_5IjYRz7UJC.png\" target=\"_blank\" rel=\"noopener noreferrer\"><img decoding=\"async\" class=\"alignnone\" src=\"https:\/\/github.com\/satomic\/copilot-usage-advanced-dashboard\/raw\/main\/image\/image_5IjYRz7UJC.png\" alt=\"Variables filter\" width=\"696\" height=\"461\" \/><\/a><\/p>\n<div class=\"markdown-heading\" dir=\"auto\">\n<h1 dir=\"auto\" tabindex=\"-1\"><\/h1>\n<h2 class=\"heading-element\" dir=\"auto\" tabindex=\"-1\"><span style=\"font-size: 24pt;\">Features<\/span><\/h2>\n<\/div>\n<div class=\"markdown-heading\" dir=\"auto\">\n<h2 class=\"heading-element\" dir=\"auto\" tabindex=\"-1\"><span style=\"font-size: 18pt;\">Copilot Usage Advanced Dashboard<\/span><\/h2>\n<\/div>\n<div class=\"markdown-heading\" dir=\"auto\">\n<h3 class=\"heading-element\" dir=\"auto\" tabindex=\"-1\">1. Organization<\/h3>\n<blockquote class=\"code-line\" dir=\"auto\" data-line=\"122\">\n<p class=\"code-line\" dir=\"auto\" data-line=\"122\">First, based on\u00a0<a href=\"https:\/\/docs.github.com\/en\/enterprise-cloud@latest\/rest\/teams\/teams?apiVersion=2022-11-28#list-teams\" data-href=\"https:\/\/docs.github.com\/en\/enterprise-cloud@latest\/rest\/teams\/teams?apiVersion=2022-11-28#list-teams\">List teams of an onganization<\/a>, get all the teams under the Organization, and then based on\u00a0<a href=\"https:\/\/docs.github.com\/en\/enterprise-cloud@latest\/rest\/copilot\/copilot-usage?apiVersion=2022-11-28#get-a-summary-of-copilot-usage-for-a-team\" data-href=\"https:\/\/docs.github.com\/en\/enterprise-cloud@latest\/rest\/copilot\/copilot-usage?apiVersion=2022-11-28#get-a-summary-of-copilot-usage-for-a-team\">Get a summary of Copilot usage for a team<\/a>, sum and calculate the data of all teams under the Organization to get complete Organization-level data.<\/p>\n<\/blockquote>\n<ul class=\"code-line\" dir=\"auto\" data-line=\"124\">\n<li class=\"code-line\" dir=\"auto\" data-line=\"124\">Acceptance Rate Average =\u00a0<code>sum(total_acceptances_count) \/ sum(total_suggestions_count)<\/code><\/li>\n<li class=\"code-line\" dir=\"auto\" data-line=\"125\">Cumulative Number of Acceptence (Count) =\u00a0<code>sum(total_acceptances_count)<\/code><\/li>\n<li class=\"code-line\" dir=\"auto\" data-line=\"126\">Cumulative Number of Suggestions (Count) =\u00a0<code>sum(total_suggestions_count)<\/code><\/li>\n<li class=\"code-line\" dir=\"auto\" data-line=\"127\">Cumulative Number of Lines of Code Accepted =\u00a0<code>sum(total_lines_accepted)<\/code><\/li>\n<li class=\"code-line\" dir=\"auto\" data-line=\"128\">Acceptance Rate (%) =\u00a0<code>total_acceptances_count \/ total_suggestions_count<\/code><\/li>\n<li class=\"code-line\" dir=\"auto\" data-line=\"129\">Total Active Users =\u00a0<code>total_active_users<\/code><\/li>\n<li class=\"code-line\" dir=\"auto\" data-line=\"130\">Total Suggestions &amp; Acceptances Count =\u00a0<code>total_suggestions_count<\/code>\u00a0&amp;\u00a0<code>total_acceptances_count<\/code><\/li>\n<li class=\"code-line\" dir=\"auto\" data-line=\"131\">Total Lines Suggested &amp; Accepted =\u00a0<code>total_lines_suggested\u00a0<\/code>&amp;\u00a0<code>total_lines_accepted<\/code><\/li>\n<\/ul>\n<\/div>\n<p dir=\"auto\"><a href=\"https:\/\/github.com\/satomic\/copilot-usage-advanced-dashboard\/blob\/main\/image\/image_WVNHVnb2OZ.png\" target=\"_blank\" rel=\"noopener noreferrer\"><img decoding=\"async\" class=\"alignnone\" src=\"https:\/\/github.com\/satomic\/copilot-usage-advanced-dashboard\/raw\/main\/image\/image_WVNHVnb2OZ.png\" alt=\"Organization\" width=\"2221\" height=\"1225\" \/><\/a><\/p>\n<div class=\"markdown-heading\" dir=\"auto\">\n<h3 class=\"heading-element\" dir=\"auto\" tabindex=\"-1\">2. Teams<\/h3>\n<blockquote class=\"code-line\" dir=\"auto\" data-line=\"136\">\n<p class=\"code-line\" dir=\"auto\" data-line=\"136\">Based on the breakdown data in\u00a0<a href=\"https:\/\/docs.github.com\/en\/enterprise-cloud@latest\/rest\/copilot\/copilot-metrics?apiVersion=2022-11-28#get-copilot-metrics-for-a-team\" data-href=\"https:\/\/docs.github.com\/en\/enterprise-cloud@latest\/rest\/copilot\/copilot-usage?apiVersion=2022-11-28#get-a-summary-of-copilot-usage-for-a-team\">Get a summary of Copilot metrics for a team<\/a>, the data is aggregated by Teams to obtain data comparisons of different Teams.<\/p>\n<\/blockquote>\n<ul class=\"code-line\" dir=\"auto\" data-line=\"138\">\n<li class=\"code-line\" dir=\"auto\" data-line=\"138\">Number of Teams =\u00a0<code>unique_count(team_slug)<\/code><\/li>\n<li class=\"code-line\" dir=\"auto\" data-line=\"139\">Top Teams by Accepted Prompts =\u00a0<code>sum(acceptances_count).groupby(team_slug)<\/code><\/li>\n<li class=\"code-line\" dir=\"auto\" data-line=\"140\">Top Teams by Acceptance Rate =\u00a0<code>sum(acceptances_count).groupby(team_slug) \/ sum(suggestions_count).groupby(team_slug)<\/code><\/li>\n<li class=\"code-line\" dir=\"auto\" data-line=\"141\">Team Breakdown =\u00a0<code>sum(*).groupby(team_slug)<\/code><\/li>\n<\/ul>\n<\/div>\n<p dir=\"auto\"><a href=\"https:\/\/github.com\/satomic\/copilot-usage-advanced-dashboard\/blob\/main\/image\/image_TGcs3tD7Cs.png\" target=\"_blank\" rel=\"noopener noreferrer\"><img decoding=\"async\" class=\"alignnone\" src=\"https:\/\/github.com\/satomic\/copilot-usage-advanced-dashboard\/raw\/main\/image\/image_TGcs3tD7Cs.png\" alt=\"Teams\" width=\"2221\" height=\"1116\" \/><\/a><\/p>\n<div class=\"markdown-heading\" dir=\"auto\">\n<h3 class=\"heading-element\" dir=\"auto\" tabindex=\"-1\">3. Languages<\/h3>\n<blockquote class=\"code-line\" dir=\"auto\" data-line=\"147\">\n<p class=\"code-line\" dir=\"auto\" data-line=\"147\">Based on the breakdown data in\u00a0<a href=\"https:\/\/docs.github.com\/en\/enterprise-cloud@latest\/rest\/copilot\/copilot-usage?apiVersion=2022-11-28#get-a-summary-of-copilot-usage-for-a-team\" data-href=\"https:\/\/docs.github.com\/en\/enterprise-cloud@latest\/rest\/copilot\/copilot-usage?apiVersion=2022-11-28#get-a-summary-of-copilot-usage-for-a-team\">Get a summary of Copilot usage for a team<\/a>, the data is aggregated according to Languages \u200b\u200bto obtain data comparisons for different Languages.<\/p>\n<\/blockquote>\n<ul class=\"code-line\" dir=\"auto\" data-line=\"149\">\n<li class=\"code-line\" dir=\"auto\" data-line=\"149\">Number of Languages=\u00a0<code>unique_count(language)<\/code><\/li>\n<li class=\"code-line\" dir=\"auto\" data-line=\"150\">Top Languages by Accepted Prompts =\u00a0<code>sum(acceptances_count).groupby(language)<\/code><\/li>\n<li class=\"code-line\" dir=\"auto\" data-line=\"151\">Top Languages by Acceptance Rate =\u00a0<code>sum(acceptances_count).groupby(language) \/ sum(suggestions_count).groupby(language)<\/code><\/li>\n<li class=\"code-line\" dir=\"auto\" data-line=\"152\">Languages Breakdown =\u00a0<code>sum(*).groupby(language)<\/code><\/li>\n<\/ul>\n<\/div>\n<p dir=\"auto\"><a href=\"https:\/\/github.com\/satomic\/copilot-usage-advanced-dashboard\/blob\/main\/image\/image_YHXpu1wRf2.png\" target=\"_blank\" rel=\"noopener noreferrer\"><img decoding=\"async\" class=\"alignnone\" src=\"https:\/\/github.com\/satomic\/copilot-usage-advanced-dashboard\/raw\/main\/image\/image_YHXpu1wRf2.png\" alt=\"Languages\" width=\"2220\" height=\"1264\" \/><\/a><\/p>\n<div class=\"markdown-heading\" dir=\"auto\">\n<h3 class=\"heading-element\" dir=\"auto\" tabindex=\"-1\">4. Editors<\/h3>\n<blockquote class=\"code-line\" dir=\"auto\" data-line=\"158\">\n<p class=\"code-line\" dir=\"auto\" data-line=\"158\">Based on the breakdown data in\u00a0<a href=\"https:\/\/docs.github.com\/en\/enterprise-cloud@latest\/rest\/copilot\/copilot-usage?apiVersion=2022-11-28#get-a-summary-of-copilot-usage-for-a-team\" data-href=\"https:\/\/docs.github.com\/en\/enterprise-cloud@latest\/rest\/copilot\/copilot-usage?apiVersion=2022-11-28#get-a-summary-of-copilot-usage-for-a-team\">Get a summary of Copilot usage for a team<\/a>, the data is aggregated by Editors to obtain data comparisons for different Editors.<\/p>\n<\/blockquote>\n<ul class=\"code-line\" dir=\"auto\" data-line=\"160\">\n<li class=\"code-line\" dir=\"auto\" data-line=\"160\">Number of Editors =\u00a0<code>unique_count(editor)<\/code><\/li>\n<li class=\"code-line\" dir=\"auto\" data-line=\"161\">Top Editors by Accepted Prompts =\u00a0<code>sum(acceptances_count).groupby(editor)<\/code><\/li>\n<li class=\"code-line\" dir=\"auto\" data-line=\"162\">Top Editors by Acceptance Rate =\u00a0<code>sum(acceptances_count).groupby(editor) \/ sum(suggestions_count).groupby(editor)<\/code><\/li>\n<li class=\"code-line\" dir=\"auto\" data-line=\"163\">Editors Breakdown =\u00a0<code>sum(*).groupby(editor)<\/code><\/li>\n<\/ul>\n<\/div>\n<p dir=\"auto\"><a href=\"https:\/\/github.com\/satomic\/copilot-usage-advanced-dashboard\/blob\/main\/image\/image_9P1zJxBMaO.png\" target=\"_blank\" rel=\"noopener noreferrer\"><img decoding=\"async\" class=\"alignnone\" src=\"https:\/\/github.com\/satomic\/copilot-usage-advanced-dashboard\/raw\/main\/image\/image_9P1zJxBMaO.png\" alt=\"Editors\" width=\"2220\" height=\"1063\" \/><\/a><\/p>\n<div class=\"markdown-heading\" dir=\"auto\">\n<h3 class=\"heading-element\" dir=\"auto\" tabindex=\"-1\">5. Copilot Chat<\/h3>\n<blockquote class=\"code-line\" dir=\"auto\" data-line=\"169\">\n<p class=\"code-line\" dir=\"auto\" data-line=\"169\">Based on the data from\u00a0<a href=\"https:\/\/docs.github.com\/en\/enterprise-cloud@latest\/rest\/copilot\/copilot-usage?apiVersion=2022-11-28#get-a-summary-of-copilot-usage-for-a-team\" data-href=\"https:\/\/docs.github.com\/en\/enterprise-cloud@latest\/rest\/copilot\/copilot-usage?apiVersion=2022-11-28#get-a-summary-of-copilot-usage-for-a-team\">Get a summary of Copilot usage for a team<\/a>, we can get the usage of Copilot Chat.<\/p>\n<\/blockquote>\n<ul class=\"code-line\" dir=\"auto\" data-line=\"171\">\n<li class=\"code-line\" dir=\"auto\" data-line=\"171\">Acceptance Rate Average =\u00a0<code>sum(total_chat_acceptances) \/ sum(total_chat_turns)<\/code><\/li>\n<li class=\"code-line\" dir=\"auto\" data-line=\"172\">Cumulative Number of Acceptances =\u00a0<code>sum(total_chat_acceptances)<\/code><\/li>\n<li class=\"code-line\" dir=\"auto\" data-line=\"173\">Cumulative Number of Turns =\u00a0<code>sum(total_chat_turns)<\/code><\/li>\n<li class=\"code-line\" dir=\"auto\" data-line=\"174\">Total Acceptances | Total Turns Count =\u00a0<code>total_chat_acceptances<\/code>\u00a0|\u00a0<code>total_chat_turns<\/code><\/li>\n<li class=\"code-line\" dir=\"auto\" data-line=\"175\">Total Active Copilot Chat Users =\u00a0<code>total_active_chat_users<\/code><\/li>\n<\/ul>\n<\/div>\n<p dir=\"auto\"><a href=\"https:\/\/github.com\/satomic\/copilot-usage-advanced-dashboard\/blob\/main\/image\/image_MzSdsbHdmw.png\" target=\"_blank\" rel=\"noopener noreferrer\"><img decoding=\"async\" class=\"alignnone\" src=\"https:\/\/github.com\/satomic\/copilot-usage-advanced-dashboard\/raw\/main\/image\/image_MzSdsbHdmw.png\" alt=\"Copilot Chat\" width=\"2223\" height=\"1264\" \/><\/a><\/p>\n<div class=\"markdown-heading\" dir=\"auto\">\n<h3 class=\"heading-element\" dir=\"auto\" tabindex=\"-1\">6. Seat Analysis<\/h3>\n<blockquote class=\"code-line\" dir=\"auto\" data-line=\"181\">\n<p class=\"code-line\" dir=\"auto\" data-line=\"181\">Based on the data analysis of\u00a0<a href=\"https:\/\/docs.github.com\/en\/enterprise-cloud@latest\/rest\/copilot\/copilot-user-management?apiVersion=2022-11-28#get-copilot-seat-information-and-settings-for-an-organization\" data-href=\"https:\/\/docs.github.com\/en\/enterprise-cloud@latest\/rest\/copilot\/copilot-user-management?apiVersion=2022-11-28#get-copilot-seat-information-and-settings-for-an-organization\">Get Copilot seat information and settings for an organization<\/a>\u00a0and\u00a0<a href=\"https:\/\/docs.github.com\/en\/enterprise-cloud@latest\/rest\/copilot\/copilot-user-management?apiVersion=2022-11-28#list-all-copilot-seat-assignments-for-an-organization\" data-href=\"https:\/\/docs.github.com\/en\/enterprise-cloud@latest\/rest\/copilot\/copilot-user-management?apiVersion=2022-11-28#list-all-copilot-seat-assignments-for-an-organization\">List all Copilot seat assignments for an organization<\/a>, the seat allocation and usage are presented in a unified manner.<\/p>\n<\/blockquote>\n<ul class=\"code-line\" dir=\"auto\" data-line=\"183\">\n<li class=\"code-line\" dir=\"auto\" data-line=\"183\">Copilot Plan Type =\u00a0<code>count(seats).groupby(plan_type)<\/code><\/li>\n<li class=\"code-line\" dir=\"auto\" data-line=\"184\">Total =\u00a0<code>seat_breakdown.total<\/code><\/li>\n<li class=\"code-line\" dir=\"auto\" data-line=\"185\">Active in this Cycle =\u00a0<code>seat_breakdown.active_this_cycle<\/code><\/li>\n<li class=\"code-line\" dir=\"auto\" data-line=\"186\">Assigned But Never Used =\u00a0<code>last_activity_at.isnan()<\/code><\/li>\n<li class=\"code-line\" dir=\"auto\" data-line=\"187\">Inactive in this Cycle =\u00a0<code>seat_breakdown.inactive_this_cycle<\/code><\/li>\n<li class=\"code-line\" dir=\"auto\" data-line=\"188\">Ranking of Inactive Users ( \u2265 2 days ) =\u00a0<code>today - last_activity_at<\/code><\/li>\n<li class=\"code-line\" dir=\"auto\" data-line=\"189\">All assigned seats =\u00a0<code>*<\/code><\/li>\n<\/ul>\n<\/div>\n<p dir=\"auto\"><a href=\"https:\/\/github.com\/satomic\/copilot-usage-advanced-dashboard\/blob\/main\/image\/image_vNpkYpc-xW.png\" target=\"_blank\" rel=\"noopener noreferrer\"><img decoding=\"async\" class=\"alignnone\" src=\"https:\/\/github.com\/satomic\/copilot-usage-advanced-dashboard\/raw\/main\/image\/image_vNpkYpc-xW.png\" alt=\"Seat Analysis\" width=\"2223\" height=\"1264\" \/><\/a><\/p>\n<div class=\"markdown-heading\" dir=\"auto\">\n<h3 class=\"heading-element\" dir=\"auto\" tabindex=\"-1\">7. Breakdown Heatmap<\/h3>\n<blockquote class=\"code-line\" dir=\"auto\" data-line=\"195\">\n<p class=\"code-line\" dir=\"auto\" data-line=\"195\">Based on the breakdown data in\u00a0<a href=\"https:\/\/docs.github.com\/en\/enterprise-cloud@latest\/rest\/copilot\/copilot-usage?apiVersion=2022-11-28#get-a-summary-of-copilot-usage-for-a-team\" data-href=\"https:\/\/docs.github.com\/en\/enterprise-cloud@latest\/rest\/copilot\/copilot-usage?apiVersion=2022-11-28#get-a-summary-of-copilot-usage-for-a-team\">Get a summary of Copilot usage for a team<\/a>, we analyze the data from two dimensions: Languages \u200b\u200band Editors. We can clearly see what combination of Languages \u200b\u200band Editors can achieve the best Copilot usage effect.<\/p>\n<\/blockquote>\n<ul class=\"code-line\" dir=\"auto\" data-line=\"197\">\n<li class=\"code-line\" dir=\"auto\" data-line=\"197\">Active Users Count (Group by Language) =\u00a0<code>active_users.groupby(language)<\/code><\/li>\n<li class=\"code-line\" dir=\"auto\" data-line=\"198\">Accept Rate by Count (%) =\u00a0<code>sum(acceptances_count).groupby(language) \/ sum(suggestions_count).groupby(language)<\/code><\/li>\n<li class=\"code-line\" dir=\"auto\" data-line=\"199\">Accept Rate by Lines (%) =\u00a0<code>sum(lines_accepted).groupby(language) \/ sum(lines_suggested).groupby(language)<\/code><\/li>\n<li class=\"code-line\" dir=\"auto\" data-line=\"200\">Active Users Count (Group by Editor) =\u00a0<code>active_users.groupby(editor)<\/code><\/li>\n<li class=\"code-line\" dir=\"auto\" data-line=\"201\">Accept Rate by Count (%) =\u00a0<code>sum(acceptances_count).groupby(editor) \/ sum(suggestions_count).groupby(editor)<\/code><\/li>\n<li class=\"code-line\" dir=\"auto\" data-line=\"202\">Accept Rate by Lines (%) =\u00a0<code>sum(lines_accepted).groupby(editor) \/ sum(lines_suggested).groupby(editor)<\/code><\/li>\n<\/ul>\n<\/div>\n<p dir=\"auto\"><a href=\"https:\/\/github.com\/satomic\/copilot-usage-advanced-dashboard\/blob\/main\/image\/image_i7-wXGj-UA.png\" target=\"_blank\" rel=\"noopener noreferrer\"><img decoding=\"async\" class=\"alignnone\" src=\"https:\/\/github.com\/satomic\/copilot-usage-advanced-dashboard\/raw\/main\/image\/image_i7-wXGj-UA.png\" alt=\"Breakdown Heatmap\" width=\"2221\" height=\"1264\" \/><\/a><\/p>\n<div class=\"markdown-heading\" dir=\"auto\">\n<h2 dir=\"auto\" tabindex=\"-1\"><\/h2>\n<h2 class=\"heading-element\" dir=\"auto\" tabindex=\"-1\"><span style=\"font-size: 18pt;\">Copilot Usage Advanced Dashboard Original<\/span><\/h2>\n<\/div>\n<div class=\"markdown-heading\" dir=\"auto\">\n<h3 class=\"heading-element\" dir=\"auto\" tabindex=\"-1\">1. Copilot Seat Info &amp; Top Languages<\/h3>\n<\/div>\n<ul dir=\"auto\">\n<li>You can view the distribution of seats, Enterprise or Business? and overall activation trends. And for users who don&#8217;t use Copilot, they are ranked based on the length of inactivity and list users who have never activated.<\/li>\n<li>Ranking Language and Teams based on usage<\/li>\n<\/ul>\n<p dir=\"auto\"><a href=\"https:\/\/github.com\/satomic\/copilot-usage-advanced-dashboard\/blob\/main\/image\/image_raciReXvQY.png\" target=\"_blank\" rel=\"noopener noreferrer\"><img decoding=\"async\" class=\"alignnone\" src=\"https:\/\/github.com\/satomic\/copilot-usage-advanced-dashboard\/raw\/main\/image\/image_raciReXvQY.png\" alt=\"Copilot Seat Info &amp; Top Languages\" width=\"2560\" height=\"1271\" \/><\/a><\/p>\n<div class=\"markdown-heading\" dir=\"auto\">\n<h3 dir=\"auto\" tabindex=\"-1\"><\/h3>\n<h3 class=\"heading-element\" dir=\"auto\" tabindex=\"-1\">2. Copilot Usage Total Insight<\/h3>\n<\/div>\n<p dir=\"auto\">You can analyze the total number of recommendations and adoption rate trends based on Count Lines and Chats<\/p>\n<p dir=\"auto\"><a href=\"https:\/\/github.com\/satomic\/copilot-usage-advanced-dashboard\/blob\/main\/image\/image_6lcv61qm2_.png\" target=\"_blank\" rel=\"noopener noreferrer\"><img decoding=\"async\" class=\"alignnone\" src=\"https:\/\/github.com\/satomic\/copilot-usage-advanced-dashboard\/raw\/main\/image\/image_6lcv61qm2_.png\" alt=\"Copilot Usage Total Insight\" width=\"2560\" height=\"1271\" \/><\/a><\/p>\n<div class=\"markdown-heading\" dir=\"auto\">\n<h3 dir=\"auto\" tabindex=\"-1\"><\/h3>\n<h3 class=\"heading-element\" dir=\"auto\" tabindex=\"-1\">3. Copilot Usage Breakdown Insight<\/h3>\n<\/div>\n<p dir=\"auto\">You can analyze the effect of Copilot in different languages \u200b\u200band different editor combinations.<\/p>\n<p dir=\"auto\"><a href=\"https:\/\/github.com\/satomic\/copilot-usage-advanced-dashboard\/blob\/main\/image\/image_RJ6lvMkZlP.png\" target=\"_blank\" rel=\"noopener noreferrer\"><img decoding=\"async\" class=\"alignnone\" src=\"https:\/\/github.com\/satomic\/copilot-usage-advanced-dashboard\/raw\/main\/image\/image_RJ6lvMkZlP.png\" alt=\"Copilot Usage Breakdown Insight\" width=\"2560\" height=\"1271\" \/><\/a><\/p>\n<div class=\"markdown-heading\" dir=\"auto\"><\/div>\n<h2 dir=\"auto\">How these metrics translate into ROI insights<\/h2>\n<p class=\"whitespace-pre-wrap break-words\">Let me provide two concrete example scenarios that illustrate how these metrics can translate into ROI insights:<\/p>\n<h3 class=\"whitespace-pre-wrap break-words\">Scenario 1: Enterprise Software Company<\/h3>\n<ul class=\"[&amp;:not(:last-child)_ul]:pb-1 [&amp;:not(:last-child)_ol]:pb-1 list-disc space-y-1.5 pl-7\">\n<li class=\"whitespace-normal break-words\">Initial situation: 200 developers at $100,000 annual salary each<\/li>\n<li class=\"whitespace-normal break-words\">Observed metrics:\n<ul class=\"[&amp;:not(:last-child)_ul]:pb-1 [&amp;:not(:last-child)_ol]:pb-1 list-disc space-y-1.5 pl-7\">\n<li class=\"whitespace-normal break-words\">20% code acceptance rate<\/li>\n<li class=\"whitespace-normal break-words\">More conservative time savings estimate: 0.4 hours per day<\/li>\n<li class=\"whitespace-normal break-words\">Assuming 78% active usage remains constant<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<p class=\"whitespace-pre-wrap break-words\">ROI calculation:<\/p>\n<ul class=\"[&amp;:not(:last-child)_ul]:pb-1 [&amp;:not(:last-child)_ol]:pb-1 list-disc space-y-1.5 pl-7\">\n<li class=\"whitespace-normal break-words\">Time savings: 0.4 hours \u00d7 200 developers = 80 hours daily<\/li>\n<li class=\"whitespace-normal break-words\">Developer hourly cost: $100,000 \u00f7 2080 working hours = ~$48\/hour<\/li>\n<li class=\"whitespace-normal break-words\">Daily productivity gain: 80 hours \u00d7 $48 = $3,840<\/li>\n<li class=\"whitespace-normal break-words\">Annual productivity gain: $998,400 (260 working days)<\/li>\n<li class=\"whitespace-normal break-words\">Copilot cost: $19 \u00d7 200 developers \u00d7 12 months = $45,600 annually<\/li>\n<li class=\"whitespace-normal break-words\"><strong>Net ROI: 2,089%<\/strong> ($998,400 gain &#8211; $45,600 cost = $952,800 benefit)<\/li>\n<\/ul>\n<h3><\/h3>\n<h3 class=\"whitespace-pre-wrap break-words\">Scenario 2: Early-Stage Startup<\/h3>\n<ul class=\"[&amp;:not(:last-child)_ul]:pb-1 [&amp;:not(:last-child)_ol]:pb-1 list-disc space-y-1.5 pl-7\">\n<li class=\"whitespace-normal break-words\">Initial situation: 12-person development team<\/li>\n<li class=\"whitespace-normal break-words\">Observed metrics:\n<ul class=\"[&amp;:not(:last-child)_ul]:pb-1 [&amp;:not(:last-child)_ol]:pb-1 list-disc space-y-1.5 pl-7\">\n<li class=\"whitespace-normal break-words\">20% code acceptance rate<\/li>\n<li class=\"whitespace-normal break-words\">90% active user rate<\/li>\n<li class=\"whitespace-normal break-words\">Adjusted time savings: 15% faster completion<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<p class=\"whitespace-pre-wrap break-words\">ROI calculation:<\/p>\n<ul class=\"[&amp;:not(:last-child)_ul]:pb-1 [&amp;:not(:last-child)_ol]:pb-1 list-disc space-y-1.5 pl-7\">\n<li class=\"whitespace-normal break-words\">Original timeline: 6 months to launch<\/li>\n<li class=\"whitespace-normal break-words\">With Copilot: 5.1 months to launch (0.9 months saved)<\/li>\n<li class=\"whitespace-normal break-words\">Monthly burn rate: $100,000<\/li>\n<li class=\"whitespace-normal break-words\">Cost savings from faster launch: $90,000<\/li>\n<li class=\"whitespace-normal break-words\">Copilot cost: $19 \u00d7 12 developers \u00d7 12 months = $2,736 annually<\/li>\n<li class=\"whitespace-normal break-words\"><strong>Net ROI: 3,190%<\/strong> ($90,000 savings &#8211; $2,736 cost = $87,264 benefit)<\/li>\n<\/ul>\n<h2 dir=\"auto\"><span data-olk-copy-source=\"MessageBody\">Summary <\/span><\/h2>\n<p dir=\"auto\">Monitoring and visualizing GitHub Copilot metrics through ES + Grafana provides engineering leaders with real-time insights into their AI tooling investments. By tracking key metrics like code acceptance rates, active user engagement, and adoption status across teams, organizations can accurately measure the tangible impact on developer productivity and calculate precise ROI figures. Grafana&#8217;s powerful visualization capabilities transform complex usage data into clear, actionable dashboards that help justify AI investments, identify adoption patterns, and optimize tool utilization. This transparent, data-driven approach not only validates the business value of Copilot but also helps teams fine-tune their AI assistance strategy based on concrete metrics rather than anecdotal evidence. For engineering organizations looking to quantify the benefits of AI-powered development, this monitoring solution offers an essential bridge between raw metrics and meaningful business outcomes.<\/p>\n<p dir=\"auto\">\n","protected":false},"excerpt":{"rendered":"<p>Architecture: How it works! The Copilot Usage Advanced Dashboard offers a comprehensive suite of features designed to revolutionize how organizations track and analyze their Copilot usage. At its core, the solution leverages Elasticsearch for persistent data storage, breaking free from the traditional 28-day limitation and enabling analysis across extended time periods, from months to years. [&hellip;]<\/p>\n","protected":false},"author":174024,"featured_media":517,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1,20,19],"tags":[28,48,21,47,49],"class_list":["post-510","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-azure","category-developer-productivity","category-github-copilot","tag-copilot","tag-dashboard","tag-github","tag-metrics","tag-visualization"],"acf":[],"blog_post_summary":"<p>Architecture: How it works! The Copilot Usage Advanced Dashboard offers a comprehensive suite of features designed to revolutionize how organizations track and analyze their Copilot usage. At its core, the solution leverages Elasticsearch for persistent data storage, breaking free from the traditional 28-day limitation and enabling analysis across extended time periods, from months to years. [&hellip;]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/all-things-azure\/wp-json\/wp\/v2\/posts\/510","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/all-things-azure\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/all-things-azure\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/all-things-azure\/wp-json\/wp\/v2\/users\/174024"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/all-things-azure\/wp-json\/wp\/v2\/comments?post=510"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/all-things-azure\/wp-json\/wp\/v2\/posts\/510\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/all-things-azure\/wp-json\/wp\/v2\/media\/517"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/all-things-azure\/wp-json\/wp\/v2\/media?parent=510"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/all-things-azure\/wp-json\/wp\/v2\/categories?post=510"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/all-things-azure\/wp-json\/wp\/v2\/tags?post=510"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}