We can use Metrics Advisor to:<\/p>\n
- \n
- Analyze multi-dimensional data from multiple data sources<\/li>\n
- Identify and correlate anomalies<\/li>\n
- Configure and fine-tune the anomaly detection model used on your data<\/li>\n
- Diagnose anomalies and help with root cause analysis<\/li>\n<\/ul>\n
Basic Concepts<\/h2>\n
To get started, you must onboard your data from one of the supported data source types and configure the data feed settings. See the “How-to: Onboard your metric data to Metrics Advisor”<\/a> article for detailed instructions on how to configure your data. As data is ingested, you can fine-tune the anomaly detection to fit your scenario and subscribe to alerts with notification hooks that will let you know of any anomalies detected. Moving forward, the service will provide diagnostic insights including root cause analysis and recommended actions while additionally using your feedback to perform automatic model customization and earlier notice of future anomalies.<\/p>\n
Metric, Dimensions, and Timeseries<\/h3>\n
The service can connect to various data sources. Consumers specify the metrics<\/em> in the data-source that they want service to monitor and the dimensions<\/em> of the metric. Service produces timeseries<\/em> data for each metric across dimensions<\/em>.<\/p>\n
![\"\"](\"https:\/\/devblogs.microsoft.com\/azure-sdk\/wp-content\/uploads\/sites\/58\/2020\/10\/20201013-metics-dimensions-time-series.png\")
<\/p>\nAnomaly<\/h3>\n
An anomaly<\/em> is an abnormal data point in the timeseries<\/em>.<\/p>\n
![\"\"](\"https:\/\/devblogs.microsoft.com\/azure-sdk\/wp-content\/uploads\/sites\/58\/2020\/10\/20201013-anomaly.png\")
<\/p>\nAnomaly Detection<\/h3>\n
Consumers can specify the anomaly detection configurations<\/em> for the metric<\/em> and timeseries<\/em>, which will be used to detect the abnormal data point (i.e. anomalies).<\/p>\n
![\"\"](\"https:\/\/devblogs.microsoft.com\/azure-sdk\/wp-content\/uploads\/sites\/58\/2020\/10\/20201013-anomaly-detection.png\")
<\/p>\nAlert<\/h3>\n
The alert configuration<\/em> can be used to combine multiple anomaly detection configurations<\/em> and produce alerts.<\/p>\n
![\"\"](\"https:\/\/devblogs.microsoft.com\/azure-sdk\/wp-content\/uploads\/sites\/58\/2020\/10\/20201013-alert.png\")
<\/p>\nIncident<\/h3>\n
When anomalies appear for a long period of time or show some pattern, then service treats it as an incident. An alerting configuration<\/em> can have notification hooks attached to it and an alert will be sent to the hook. Currently the supported hooks are – email and webhook.<\/p>\n
![\"\"](\"https:\/\/devblogs.microsoft.com\/azure-sdk\/wp-content\/uploads\/sites\/58\/2020\/10\/20201013-incident.png\")
<\/p>\nGet the package<\/h2>\n
Today we want to demonstrate how to use the Python Metrics Advisor client library<\/a> to configure the Metrics Advisor service, monitor anomalies, and handle incidents.<\/p>\n
Run the following command to install the library:<\/p>\n
pip install azure-ai-metricsadvisor --pre\n<\/code><\/pre>\nWe also support other languages including .NET<\/a>, JavaScript\/TypeScript<\/a>, and Java<\/a>.<\/p>\n
Configure Metrics Advisor Service<\/h2>\n
To configure Metrics Advisor service, we need an authorized
MetricsAdvisorAdministrationClient<\/code> which needs two keys: – The subscription key to your Metrics Advisor resource. You can find this in the Keys and Endpoint section of your resource in the Azure portal. – The API key for your Metrics Advisor instance. You can find this in the workspace for Metrics Advisor, in API keys on the left navigation menu.<\/p>\nThen you can use the keys to create a client:<\/p>\n
from azure.ai.metricsadvisor import (\n MetricsAdvisorKeyCredential,\n MetricsAdvisorAdministrationClient,\n)\nclient = MetricsAdvisorAdministrationClient (<service_endpoint>,\n MetricsAdvisorKeyCredential(<subscription_key>, <api_key>))\n<\/code><\/pre>\nNow let’s get our data onboarded. Metrics Advisor supports connecting different types of data sources. In this demo, we will use the data from SQL Server.<\/p>\n
import datetime\nfrom azure.ai.metricsadvisor.models import (\n SQLServerDataFeed,\n DataFeedSchema,\n Metric,\n Dimension,\n DataFeedOptions,\n DataFeedRollupSettings,\n DataFeedMissingDataPointFillSettings\n )\n\ndata_feed = client.create_data_feed(\n name=\"My data feed\",\n source=SQLServerDataFeed(\n connection_string=<sql_server_connection_string>,\n query=<query>,\n ),\n granularity=\"Daily\",\n schema=DataFeedSchema(\n metrics=[\n Metric(name=\"cost\", display_name=\"Cost\"),\n Metric(name=\"revenue\", display_name=\"Revenue\")\n ],\n dimensions=[\n Dimension(name=\"category\", display_name=\"Category\"),\n Dimension(name=\"city\", display_name=\"City\")\n ],\n timestamp_column=\"Timestamp\"\n ),\n ingestion_settings=datetime.datetime(2019, 10, 1),\n options=DataFeedOptions(\n data_feed_description=\"cost\/revenue data feed\",\n rollup_settings=DataFeedRollupSettings(\n rollup_type=\"AutoRollup\",\n rollup_method=\"Sum\",\n rollup_identification_value=\"__SUM__\"\n ),\n missing_data_point_fill_settings=DataFeedMissingDataPointFillSettings(\n fill_type=\"SmartFilling\"\n ),\n access_mode=\"Private\"\n )\n)\n<\/code><\/pre>\nHere we set rollup_type to “AutoRollup” and rollup_method to “Sum” so the service will automatically use “Sum” method to calculate the aggregations for the ingested data.<\/p>\n
After the data is ingested, a default detection configuration is automatically applied to each metric. Now, we want to configure an
AnomalyAlertConfiguration<\/code> to tell the service in which conditions an alert needs to be triggered.<\/p>\nfrom azure.ai.metricsadvisor.models import (\n MetricAlertConfiguration,\n MetricAnomalyAlertScope,\n TopNGroupScope,\n MetricAnomalyAlertConditions,\n SeverityCondition,\n MetricBoundaryCondition,\n MetricAnomalyAlertSnoozeCondition\n)\n\nalert_config = client.create_anomaly_alert_configuration(\n name=\"my alert config\",\n description=\"alert config description\",\n cross_metrics_operator=\"AND\",\n metric_alert_configurations=[\n MetricAlertConfiguration(\n detection_configuration_id=<anomaly_detection_configuration_id>,\n alert_scope=MetricAnomalyAlertScope(\n scope_type=\"WholeSeries\"\n ),\n alert_conditions=MetricAnomalyAlertConditions(\n severity_condition=SeverityCondition(\n min_alert_severity=\"Low\",\n max_alert_severity=\"High\"\n )\n )\n ),\n MetricAlertConfiguration(\n detection_configuration_id=<anomaly_detection_configuration_id>,\n alert_scope=MetricAnomalyAlertScope(\n scope_type=\"TopN\",\n top_n_group_in_scope=TopNGroupScope(\n top=10,\n period=5,\n min_top_count=5\n )\n ),\n alert_conditions=MetricAnomalyAlertConditions(\n metric_boundary_condition=MetricBoundaryCondition(\n direction=\"Up\",\n upper=50\n )\n ),\n alert_snooze_condition=MetricAnomalyAlertSnoozeCondition(\n auto_snooze=2,\n snooze_scope=\"Metric\",\n only_for_successive=True\n )\n ),\n ],\n hook_ids=[<hook_id>]\n)\n<\/code><\/pre>\nQuery anomalies and alerts<\/h2>\n
After the configurations are set, Metrics Advisor service will keep running and monitoring anomalies for us. If we assign hooks when creating
AnomalyAlertConfiguration<\/code>, callbacks will be invoked when alerts pop up. Of course, we can useMetricsAdvisorClient<\/code> to query anomalies and alerts as well.<\/p>\nimport datetime\nfrom azure.ai.metricsadvisor import (\n MetricsAdvisorKeyCredential,\n MetricsAdvisorClient,\n)\nclient = MetricsAdvisorClient(<service_endpoint>,\n MetricsAdvisorKeyCredential(<subscription_key>, <api_key>))\n\nresults = client.list_alerts_for_alert_configuration(\n alert_configuration_id=<alert_config_id>,\n start_time=datetime.datetime(2020, 1, 1),\n end_time=datetime.datetime(2020, 9, 9),\n time_mode=\"AnomalyTime\",\n)\nprint(\"Alerts\")\nfor result in results:\n print(\" Alert id: {}\".format(result.id))\n print(\" Created on: {}\".format(result.created_on))\n\nresults = client.list_anomalies_for_alert(\n alert_configuration_id=<alert_config_id>,\n alert_id=<alert_id>,\n)\nprint(\"Anomalies\")\nfor result in results:\n print(\" Detection configuration id: {}\".format(result.detection_configuration_id))\n print(\" Severity: {}\".format(result.severity))\n print(\" Status: {}\".format(result.status))\n<\/code><\/pre>\nOutput:<\/p>\n
Alerts\n Created on: 2020-09-12 01:15:16.406000+00:00\n Alert id: 17465dcc000\n Created on: 2020-09-12 01:14:17.184000+00:00\n Alert id: 17460b66400\n\nAnomalies\n Detection configuration id: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx\n Severity: Low\n Status: Active\n<\/code><\/pre>\nYou are all set! Azure Metrics Advisor service will monitor your timeseries and trigger alerts when something is wrong. Enjoy!<\/p>\n
Further Documentation<\/h2>\n
- \n
- Metrics Advisor documentation<\/a><\/li>\n
- Python Reference Documentation<\/a>, Readme<\/a><\/li>\n
- .NET Reference Documentation<\/a>, Readme<\/a><\/li>\n
- JavaScript\/TypeScript Reference Documentation<\/a>, Readme<\/a><\/li>\n
- Java Reference Documentation<\/a>, Readme<\/a><\/li>\n<\/ul>\n
<\/p>\n
Azure SDK Releases<\/a><\/div><\/p>\nAzure SDK Blog Contributions<\/h2>\n
Thank you for reading this Azure SDK blog post! We hope that you learned something new and welcome you to share this post. We are open to Azure SDK blog contributions. Please contact us at azsdkblog@microsoft.com<\/a> with your topic and we’ll get you setup as a guest blogger.<\/p>\n
Azure SDK Links<\/h2>\n
- \n
- Azure SDK Website: aka.ms\/azsdk<\/a><\/li>\n
- Azure SDK Intro (3 minute video): aka.ms\/azsdk\/intro<\/a><\/li>\n
- Azure SDK Intro Deck (PowerPoint deck): aka.ms\/azsdk\/intro\/deck<\/a><\/li>\n
- Azure SDK Releases: aka.ms\/azsdk\/releases<\/a><\/li>\n
- Azure SDK Blog: aka.ms\/azsdk\/blog<\/a><\/li>\n
- Azure SDK Twitter: twitter.com\/AzureSDK<\/a><\/li>\n
- Azure SDK Design Guidelines: aka.ms\/azsdk\/guide<\/a><\/li>\n
- Azure SDKs & Tools: azure.microsoft.com\/downloads<\/a><\/li>\n
- Azure SDK Central Repository: github.com\/azure\/azure-sdk<\/a><\/li>\n
- Azure SDK for .NET: github.com\/azure\/azure-sdk-for-net<\/a><\/li>\n
- Azure SDK for Java: github.com\/azure\/azure-sdk-for-java<\/a><\/li>\n
- Azure SDK for Python: github.com\/azure\/azure-sdk-for-python<\/a><\/li>\n
- Azure SDK for JavaScript\/TypeScript: github.com\/azure\/azure-sdk-for-js<\/a><\/li>\n
- Azure SDK for Android: github.com\/Azure\/azure-sdk-for-android<\/a><\/li>\n
- Azure SDK for iOS: github.com\/Azure\/azure-sdk-for-ios<\/a><\/li>\n
- Azure SDK for Go: github.com\/Azure\/azure-sdk-for-go<\/a><\/li>\n
- Azure SDK for C: github.com\/Azure\/azure-sdk-for-c<\/a><\/li>\n
- Azure SDK for C++: github.com\/Azure\/azure-sdk-for-cpp<\/a><\/li>\n<\/ul>\n
<\/p>\n
<\/p>\n","protected":false},"excerpt":{"rendered":"
The new Metrics Advisor service allows you to easily detect anomalies in your data. This blog will show you how to set the service up and use it in your application.<\/p>\n","protected":false},"author":41895,"featured_media":714,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[734,750,767,766,162],"class_list":["post-700","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-azure-sdk","tag-azure","tag-azure-sdk","tag-cognitive","tag-metricsadvisor","tag-python"],"acf":[],"blog_post_summary":"
The new Metrics Advisor service allows you to easily detect anomalies in your data. This blog will show you how to set the service up and use it in your application.<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/azure-sdk\/wp-json\/wp\/v2\/posts\/700","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/azure-sdk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/azure-sdk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/azure-sdk\/wp-json\/wp\/v2\/users\/41895"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/azure-sdk\/wp-json\/wp\/v2\/comments?post=700"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/azure-sdk\/wp-json\/wp\/v2\/posts\/700\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/azure-sdk\/wp-json\/wp\/v2\/media\/714"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/azure-sdk\/wp-json\/wp\/v2\/media?parent=700"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/azure-sdk\/wp-json\/wp\/v2\/categories?post=700"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/azure-sdk\/wp-json\/wp\/v2\/tags?post=700"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
- Azure SDK Intro (3 minute video): aka.ms\/azsdk\/intro<\/a><\/li>\n
- Python Reference Documentation<\/a>, Readme<\/a><\/li>\n
- Metrics Advisor documentation<\/a><\/li>\n