{"id":2144,"date":"2016-01-07T02:40:27","date_gmt":"2016-01-07T10:40:27","guid":{"rendered":"https:\/\/www.microsoft.com\/reallifecode\/index.php\/2016\/01\/07\/categorizing-driver-risk-with-machine-learning\/"},"modified":"2020-03-18T14:04:35","modified_gmt":"2020-03-18T21:04:35","slug":"categorizing-driver-risk-with-machine-learning","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/ise\/categorizing-driver-risk-with-machine-learning\/","title":{"rendered":"Categorizing Driver Risk with Machine Learning"},"content":{"rendered":"<h1 id=\"categorizing-driver-risk-with-machine-learning\">Categorizing Driver Risk with Machine Learning<\/h2>\n<p>Usage-based insurance (UBI), also known as pay as you drive, is a type of automobile insurance whereby the costs of motor insurance are dependent upon the type of vehicle used, measured against time, distance, behavior and place.<\/p>\n<p>One important requirement of UBI is the ability to quickly quantify or classify the behavior of drivers based on the telemetry data gathered during their recent trips. Insurance companies are motivated to get this trip classification quickly so they can quickly adjust the user\u2019s premiums.<\/p>\n<p>This case study describes a project with Toyota where we have constructed a machine learning web service that takes summary telemetry data as the input and assigns a risk rating to the trip.<\/p>\n<p><strong>Some familiarity with Machine Learning is useful to thoroughly understand this case study.<\/strong><\/p>\n<p>Getting good labeled data to build a trained machine learning model is always a challenge. To build this illustrative machine learning model we have reduced the risk rating from a scale of 0 to 100 to a simpler assignment into one of four risk categories (1 to 4), representing excellent, good, fair, and high-risk drivers. This allows us to use a supervised learning model which is a good fit for this problem because it can produce a readily trainable performant model for multi-class data. The supervised model is also possible because the insurance actuaries are the people who review and classify the trips with different risk ratings. This labelling knowledge is proprietary per company and closely guarded by the actuaries.<\/p>\n<p>For an insurance company, our Risk Rating prototype could be integrated with existing data to provide a more complex set of risk classes. Insurance companies already have rich data to enumerate risk classes that are valid, so they could easily define \u201cexcellent\u201d or \u201crisky\u201d according to this existing data.<\/p>\n<p>We don\u2019t have such kind of history data, so we have to manually label the data based on our own understanding of safe driving. We believe the classification result will be more reliable if this model is built by insurance companies.<\/p>\n<p>Here are some of our findings during manual labelling. These observations are not definitive just called out to provide context for others that might manually label driving data.<\/p>\n<ol>\n<li>We defined 4 different categories for drivers (High, Fair, Good, Low)<\/li>\n<li>The main factors that determine the risk rating are Acceleration, Lane Changes, Turns and Braking.<\/li>\n<li>Acceleration and Brake values have a positive correlation.<\/li>\n<li>We do have touch or voice data to test our hypotheses around interaction\/distraction contributing to risk.<\/li>\n<\/ol>\n<p>The main risk factors, acceleration, lane changes, turns and braking are detected while preprocessing the telemetry data. We have built an Apache Storm Topology that processes the telemetry in real-time and tallies up the number of events (both high-risk and normal) and passes that information to the ML Model.<\/p>\n<p>Acceleration and braking are simply computed from the accelerometer and GPS location data, Lane Changes are a more complex filter that detects turns within a target range (in both directions, e.g. left then right, or right then left) during a specific time window. Turns are a simpler filter that similarly detects heading changes over a time window.<\/p>\n<p>Below you can see the entire machine learning workflow, and run it locally or in your Azure Subscription. To run this in your azure subscription you\u2019ll need to create an <a href=\"https:\/\/azure.microsoft.com\/en-us\/documentation\/services\/machine-learning\/\">Azure Machine Learning<\/a> Workspace (note the workspace id), and retrieve the authorization token for that workspace. These tasks are done via the <a href=\"https:\/\/manage.windowsazure.com\/\">Azure Portal<\/a>.<\/p>\n<h2 id=\"setup-and-initialization\">Setup and Initialization<\/h2>\n<div class=\"language-python highlighter-rouge\">\n<pre class=\"highlight\"><code><span class=\"c\">#<\/span>\r\n<span class=\"c\"># Here we're setting variables so we don't have to keep entering these <\/span>\r\n<span class=\"c\"># strings. We can also now hide these values once, when we publish\/share <\/span>\r\n<span class=\"c\"># this notebook.<\/span>\r\n<span class=\"c\">#<\/span>\r\n<span class=\"kn\">import<\/span> <span class=\"nn\">uuid<\/span>\r\n\r\n<span class=\"n\">workspace_id<\/span><span class=\"o\">=<\/span><span class=\"s\">'ff11c7e836424e0893675620a8341749'<\/span>\r\n<span class=\"n\">authorization_token<\/span><span class=\"o\">=<\/span><span class=\"s\">'fdf2c48c67d345459905ff25de7a8677'<\/span>\r\n\r\n<span class=\"c\"># This is set here once so that any re-deployments of the web service below<\/span>\r\n<span class=\"c\"># *replace* the existing web service, instead of creating a new one for <\/span>\r\n<span class=\"c\"># each deployment.<\/span>\r\n\r\n<span class=\"n\">service_id<\/span> <span class=\"o\">=<\/span> <span class=\"nb\">str<\/span><span class=\"p\">(<\/span><span class=\"n\">uuid<\/span><span class=\"o\">.<\/span><span class=\"n\">uuid4<\/span><span class=\"p\">())<\/span>\r\n<\/code><\/pre>\n<\/div>\n<div class=\"language-python highlighter-rouge\">\n<pre class=\"highlight\"><code><span class=\"c\">#<\/span>\r\n<span class=\"c\"># This initializes the Azure Machine Learning Python Client, found:<\/span>\r\n<span class=\"c\"># https:\/\/github.com\/Azure\/Azure-MachineLearning-ClientLibrary-Python<\/span>\r\n<span class=\"c\">#<\/span>\r\n<span class=\"kn\">import<\/span> <span class=\"nn\">pandas<\/span>\r\n<span class=\"kn\">from<\/span> <span class=\"nn\">azureml<\/span> <span class=\"kn\">import<\/span> <span class=\"n\">Workspace<\/span>\r\n\r\n<span class=\"n\">ws<\/span> <span class=\"o\">=<\/span> <span class=\"n\">Workspace<\/span><span class=\"p\">(<\/span>\r\n    <span class=\"n\">workspace_id<\/span><span class=\"o\">=<\/span><span class=\"n\">workspace_id<\/span><span class=\"p\">,<\/span>\r\n    <span class=\"n\">authorization_token<\/span><span class=\"o\">=<\/span><span class=\"n\">authorization_token<\/span><span class=\"p\">,<\/span>\r\n    <span class=\"n\">endpoint<\/span><span class=\"o\">=<\/span><span class=\"s\">'https:\/\/studioapi.azureml.net'<\/span>\r\n<span class=\"p\">)<\/span>\r\n\r\n<span class=\"c\"># In my workspace I have the data as a data set, I retrieve it for local <\/span>\r\n<span class=\"c\"># processing.<\/span>\r\n\r\n<span class=\"c\"># This is how you read the data from a local file<\/span>\r\n<span class=\"n\">frame<\/span> <span class=\"o\">=<\/span> <span class=\"n\">pandas<\/span><span class=\"o\">.<\/span><span class=\"n\">read_csv<\/span><span class=\"p\">(<\/span><span class=\"s\">\"DriverRisk.csv\"<\/span><span class=\"p\">)<\/span>\r\n\r\n<span class=\"c\"># This is how you read a dataset from the Azure Machine Learning Workspace and Convert it to a Pandas DataFrame.<\/span>\r\n<span class=\"c\">#ds = ws.datasets['DriverRisk.csv']<\/span>\r\n<span class=\"c\">#frame = ds.to_dataframe()<\/span>\r\n<\/code><\/pre>\n<\/div>\n<h2 id=\"preparing-the-data\">Preparing the Data<\/h2>\n<div class=\"language-python highlighter-rouge\">\n<pre class=\"highlight\"><code><span class=\"c\"># Inspect the first few rows of data.<\/span>\r\n<span class=\"n\">frame<\/span><span class=\"o\">.<\/span><span class=\"n\">head<\/span><span class=\"p\">()<\/span>\r\n<\/code><\/pre>\n<\/div>\n<div>\n<table class=\"dataframe\" border=\"1\">\n<thead>\n<tr style=\"text-align: right;\">\n<th><\/th>\n<th>TER<\/th>\n<th>TERS<\/th>\n<th>SER<\/th>\n<th>SERS<\/th>\n<th>EHBR<\/th>\n<th>EHBRS<\/th>\n<th>SHBR<\/th>\n<th>SHBRS<\/th>\n<th>ACC<\/th>\n<th>ACCS<\/th>\n<th>&#8230;<\/th>\n<th>TURN<\/th>\n<th>TURNS<\/th>\n<th>SHAKE<\/th>\n<th>SHAKES<\/th>\n<th>LANEC<\/th>\n<th>LANECS<\/th>\n<th>USER<\/th>\n<th>TIME<\/th>\n<th>SESSION<\/th>\n<th>CATEGORY<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<th>0<\/th>\n<td>14.88<\/td>\n<td>0.067572<\/td>\n<td>0.00<\/td>\n<td>0.00000<\/td>\n<td>0<\/td>\n<td>0<\/td>\n<td>72.29<\/td>\n<td>-0.257627<\/td>\n<td>17.17<\/td>\n<td>0.242583<\/td>\n<td>&#8230;<\/td>\n<td>0.45<\/td>\n<td>0.45<\/td>\n<td>1.07<\/td>\n<td>0.022617<\/td>\n<td>4.29<\/td>\n<td>0.417315<\/td>\n<td>aferman<\/td>\n<td>8\/23\/2013 2:37<\/td>\n<td>FD91E5CE-1CA5-7BD5-E557-23FFB9317899_8.txt<\/td>\n<td>1<\/td>\n<\/tr>\n<tr>\n<th>1<\/th>\n<td>3.50<\/td>\n<td>0.015894<\/td>\n<td>0.00<\/td>\n<td>0.00000<\/td>\n<td>0<\/td>\n<td>0<\/td>\n<td>22.15<\/td>\n<td>-0.078938<\/td>\n<td>11.49<\/td>\n<td>0.162334<\/td>\n<td>&#8230;<\/td>\n<td>0.44<\/td>\n<td>0.44<\/td>\n<td>5.07<\/td>\n<td>0.107166<\/td>\n<td>2.03<\/td>\n<td>0.197471<\/td>\n<td>zhnan<\/td>\n<td>4\/23\/2013 14:29<\/td>\n<td>9DFCB353-CCCD-6285-F50A-5937B2D6460E_0.txt<\/td>\n<td>3<\/td>\n<\/tr>\n<tr>\n<th>2<\/th>\n<td>0.00<\/td>\n<td>0.000000<\/td>\n<td>0.00<\/td>\n<td>0.00000<\/td>\n<td>0<\/td>\n<td>0<\/td>\n<td>1.62<\/td>\n<td>-0.005773<\/td>\n<td>2.50<\/td>\n<td>0.035321<\/td>\n<td>&#8230;<\/td>\n<td>0.91<\/td>\n<td>0.91<\/td>\n<td>0.00<\/td>\n<td>0.000000<\/td>\n<td>2.50<\/td>\n<td>0.243191<\/td>\n<td>zhnan<\/td>\n<td>5\/5\/2013 13:09<\/td>\n<td>D967750B-529B-7CFF-4C53-08B0FC1A4F63_6.txt<\/td>\n<td>3<\/td>\n<\/tr>\n<tr>\n<th>3<\/th>\n<td>8.25<\/td>\n<td>0.037464<\/td>\n<td>8.25<\/td>\n<td>-0.02956<\/td>\n<td>0<\/td>\n<td>0<\/td>\n<td>29.70<\/td>\n<td>-0.105845<\/td>\n<td>5.19<\/td>\n<td>0.073326<\/td>\n<td>&#8230;<\/td>\n<td>0.33<\/td>\n<td>0.33<\/td>\n<td>6.92<\/td>\n<td>0.146269<\/td>\n<td>3.46<\/td>\n<td>0.336576<\/td>\n<td>zhnan<\/td>\n<td>3\/16\/2013 0:24<\/td>\n<td>D838407B-31DE-44EB-DBC1-A0B6D9068C2F_0.txt<\/td>\n<td>3<\/td>\n<\/tr>\n<tr>\n<th>4<\/th>\n<td>0.00<\/td>\n<td>0.000000<\/td>\n<td>0.00<\/td>\n<td>0.00000<\/td>\n<td>0<\/td>\n<td>0<\/td>\n<td>0.00<\/td>\n<td>0.000000<\/td>\n<td>13.05<\/td>\n<td>0.184374<\/td>\n<td>&#8230;<\/td>\n<td>0.22<\/td>\n<td>0.22<\/td>\n<td>0.87<\/td>\n<td>0.018389<\/td>\n<td>3.48<\/td>\n<td>0.338521<\/td>\n<td>zhnan<\/td>\n<td>2\/27\/2013 13:00<\/td>\n<td>C5765F7B-4A45-5748-C426-F13E8683EF3A_0.txt<\/td>\n<td>3<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>5 rows \u00d7 22 columns<\/p>\n<\/div>\n<p>There are many features in the original dataset, however there aren\u2019t that many that are determinants of the risk category. After doing separate experiments to identify the strongest determinant features we identify ACCS (Acceleration), LANECS (Lane Changes), TURNS (Turns) and BRKS (Braking) as the key features to include.<\/p>\n<p>Here we are selecting a set of columns from the entire list. This subset has been chosen because the correlation of the input features against risk category is above a previously chosen threshold. The entire list of raw features is:<\/p>\n<div class=\"language-python highlighter-rouge\">\n<pre class=\"highlight\"><code><span class=\"c\"># Enumerate the existing column headings.<\/span>\r\n<span class=\"n\">frame<\/span><span class=\"o\">.<\/span><span class=\"n\">columns<\/span>\r\n<\/code><\/pre>\n<\/div>\n<div class=\"highlighter-rouge\">\n<pre class=\"highlight\"><code>Index([u'TER', u'TERS', u'SER', u'SERS', u'EHBR', u'EHBRS', u'SHBR', u'SHBRS',\r\n       u'ACC', u'ACCS', u'BRK', u'BRKS', u'TURN', u'TURNS', u'SHAKE',\r\n       u'SHAKES', u'LANEC', u'LANECS', u'USER', u'TIME', u'SESSION',\r\n       u'CATEGORY'],\r\n      dtype='object')\r\n<\/code><\/pre>\n<\/div>\n<p>From those we select: ACCS,LANECS,TURNS,BRKS\nNote: the the difference between TER and TERS is the TERS column (like all the columns with the extra S) is normalized data.<\/p>\n<div class=\"language-python highlighter-rouge\">\n<pre class=\"highlight\"><code><span class=\"c\"># Pick just the major determinants for the model<\/span>\r\n<span class=\"c\"># TODO: Show the component analysis that motivates this selection.<\/span>\r\n<span class=\"n\">feature_cols<\/span> <span class=\"o\">=<\/span> <span class=\"p\">[<\/span><span class=\"s\">'ACCS'<\/span><span class=\"p\">,<\/span> <span class=\"s\">'LANECS'<\/span><span class=\"p\">,<\/span> <span class=\"s\">'TURNS'<\/span><span class=\"p\">,<\/span> <span class=\"s\">'BRKS'<\/span><span class=\"p\">]<\/span>\r\n<\/code><\/pre>\n<\/div>\n<div class=\"language-python highlighter-rouge\">\n<pre class=\"highlight\"><code><span class=\"c\"># Process the basic data into feature vector data and labeled category<\/span>\r\n<span class=\"c\"># data. We use numpy and sklearn tools to massage the data.<\/span>\r\n\r\n<span class=\"kn\">import<\/span> <span class=\"nn\">numpy<\/span> <span class=\"kn\">as<\/span> <span class=\"nn\">np<\/span>\r\n<span class=\"kn\">from<\/span> <span class=\"nn\">sklearn.preprocessing<\/span> <span class=\"kn\">import<\/span> <span class=\"n\">label_binarize<\/span>\r\n\r\n<span class=\"c\"># Pull out features we care about and category labels<\/span>\r\n<span class=\"n\">X<\/span> <span class=\"o\">=<\/span> <span class=\"n\">frame<\/span><span class=\"p\">[<\/span><span class=\"n\">feature_cols<\/span><span class=\"p\">]<\/span>\r\n<span class=\"n\">y_raw<\/span> <span class=\"o\">=<\/span> <span class=\"n\">frame<\/span><span class=\"o\">.<\/span><span class=\"n\">CATEGORY<\/span>\r\n\r\n<span class=\"c\"># This creates four classes from the raw data<\/span>\r\n<span class=\"n\">y<\/span> <span class=\"o\">=<\/span> <span class=\"n\">label_binarize<\/span><span class=\"p\">(<\/span><span class=\"n\">y_raw<\/span><span class=\"p\">,<\/span> <span class=\"n\">classes<\/span><span class=\"o\">=<\/span><span class=\"n\">np<\/span><span class=\"o\">.<\/span><span class=\"n\">unique<\/span><span class=\"p\">(<\/span><span class=\"n\">y_raw<\/span><span class=\"p\">))<\/span>\r\n<span class=\"n\">n_classes<\/span> <span class=\"o\">=<\/span> <span class=\"n\">y<\/span><span class=\"o\">.<\/span><span class=\"n\">shape<\/span><span class=\"p\">[<\/span><span class=\"mi\">1<\/span><span class=\"p\">]<\/span>\r\n<\/code><\/pre>\n<\/div>\n<div class=\"language-python highlighter-rouge\">\n<pre class=\"highlight\"><code><span class=\"c\"># We split the (limited amount of) data we have into training and test data.<\/span>\r\n\r\n<span class=\"kn\">import<\/span> <span class=\"nn\">numpy<\/span> <span class=\"kn\">as<\/span> <span class=\"nn\">np<\/span>\r\n<span class=\"kn\">from<\/span> <span class=\"nn\">sklearn.cross_validation<\/span> <span class=\"kn\">import<\/span> <span class=\"n\">train_test_split<\/span>\r\n\r\n<span class=\"c\"># Split data into train and test<\/span>\r\n<span class=\"n\">random_state<\/span> <span class=\"o\">=<\/span> <span class=\"n\">np<\/span><span class=\"o\">.<\/span><span class=\"n\">random<\/span><span class=\"o\">.<\/span><span class=\"n\">RandomState<\/span><span class=\"p\">(<\/span><span class=\"mi\">0<\/span><span class=\"p\">)<\/span>\r\n<span class=\"n\">X_train<\/span><span class=\"p\">,<\/span> <span class=\"n\">X_test<\/span><span class=\"p\">,<\/span> <span class=\"n\">y_train<\/span><span class=\"p\">,<\/span> <span class=\"n\">y_test<\/span> <span class=\"o\">=<\/span> <span class=\"n\">train_test_split<\/span><span class=\"p\">(<\/span><span class=\"n\">X<\/span><span class=\"p\">,<\/span> <span class=\"n\">y<\/span><span class=\"p\">,<\/span> \r\n                                                    <span class=\"n\">test_size<\/span><span class=\"o\">=.<\/span><span class=\"mi\">6<\/span><span class=\"p\">,<\/span> \r\n                                                    <span class=\"n\">random_state<\/span><span class=\"o\">=<\/span><span class=\"n\">random_state<\/span><span class=\"p\">)<\/span>\r\n<\/code><\/pre>\n<\/div>\n<h2 id=\"building-and-training-the-model\">Building and Training the Model<\/h2>\n<div class=\"language-python highlighter-rouge\">\n<pre class=\"highlight\"><code><span class=\"c\"># Here we actually instantiate and train the multi-classs classifier.<\/span>\r\n\r\n<span class=\"kn\">from<\/span> <span class=\"nn\">sklearn<\/span> <span class=\"kn\">import<\/span> <span class=\"n\">svm<\/span>\r\n<span class=\"kn\">from<\/span> <span class=\"nn\">sklearn.multiclass<\/span> <span class=\"kn\">import<\/span> <span class=\"n\">OneVsRestClassifier<\/span>\r\n\r\n<span class=\"c\"># Build the one vs all classifier (and test it)<\/span>\r\n<span class=\"n\">classifier<\/span> <span class=\"o\">=<\/span> <span class=\"n\">OneVsRestClassifier<\/span><span class=\"p\">(<\/span><span class=\"n\">svm<\/span><span class=\"o\">.<\/span><span class=\"n\">SVC<\/span><span class=\"p\">(<\/span><span class=\"n\">kernel<\/span><span class=\"o\">=<\/span><span class=\"s\">'linear'<\/span><span class=\"p\">,<\/span> <span class=\"n\">probability<\/span><span class=\"o\">=<\/span><span class=\"bp\">True<\/span><span class=\"p\">,<\/span> \r\n                                         <span class=\"n\">random_state<\/span><span class=\"o\">=<\/span><span class=\"n\">random_state<\/span><span class=\"p\">))<\/span>\r\n<\/code><\/pre>\n<\/div>\n<h2 id=\"quick-performance-inspection\">Quick Performance Inspection<\/h2>\n<div class=\"language-python highlighter-rouge\">\n<pre class=\"highlight\"><code><span class=\"c\"># Here we generate labels for the test data to compare against the actual<\/span>\r\n<span class=\"c\"># labels.<\/span>\r\n<span class=\"n\">y_score<\/span> <span class=\"o\">=<\/span> <span class=\"n\">classifier<\/span><span class=\"o\">.<\/span><span class=\"n\">fit<\/span><span class=\"p\">(<\/span><span class=\"n\">X_train<\/span><span class=\"p\">,<\/span> <span class=\"n\">y_train<\/span><span class=\"p\">)<\/span><span class=\"o\">.<\/span><span class=\"n\">decision_function<\/span><span class=\"p\">(<\/span><span class=\"n\">X_test<\/span><span class=\"p\">)<\/span>\r\n<\/code><\/pre>\n<\/div>\n<div class=\"language-python highlighter-rouge\">\n<pre class=\"highlight\"><code><span class=\"c\"># We compute some basic statistics about the classifiers performance<\/span>\r\n<span class=\"c\"># using the test data.<\/span>\r\n\r\n<span class=\"kn\">import<\/span> <span class=\"nn\">numpy<\/span> <span class=\"kn\">as<\/span> <span class=\"nn\">np<\/span>\r\n<span class=\"n\">obs<\/span> <span class=\"o\">=<\/span> <span class=\"n\">y_score<\/span>\r\n<span class=\"n\">pred<\/span> <span class=\"o\">=<\/span> <span class=\"n\">y_test<\/span>\r\n\r\n<span class=\"n\">mae<\/span> <span class=\"o\">=<\/span> <span class=\"n\">np<\/span><span class=\"o\">.<\/span><span class=\"n\">mean<\/span><span class=\"p\">(<\/span><span class=\"nb\">abs<\/span><span class=\"p\">(<\/span><span class=\"n\">pred<\/span><span class=\"o\">-<\/span><span class=\"n\">obs<\/span><span class=\"p\">))<\/span>\r\n<span class=\"n\">rmse<\/span> <span class=\"o\">=<\/span> <span class=\"n\">np<\/span><span class=\"o\">.<\/span><span class=\"n\">sqrt<\/span><span class=\"p\">(<\/span><span class=\"n\">np<\/span><span class=\"o\">.<\/span><span class=\"n\">mean<\/span><span class=\"p\">((<\/span><span class=\"n\">pred<\/span><span class=\"o\">-<\/span><span class=\"n\">obs<\/span><span class=\"p\">)<\/span><span class=\"o\">**<\/span><span class=\"mi\">2<\/span><span class=\"p\">))<\/span>\r\n<span class=\"n\">rae<\/span> <span class=\"o\">=<\/span> <span class=\"n\">np<\/span><span class=\"o\">.<\/span><span class=\"n\">mean<\/span><span class=\"p\">(<\/span><span class=\"nb\">abs<\/span><span class=\"p\">(<\/span><span class=\"n\">pred<\/span><span class=\"o\">-<\/span><span class=\"n\">obs<\/span><span class=\"p\">))<\/span><span class=\"o\">\/<\/span><span class=\"n\">np<\/span><span class=\"o\">.<\/span><span class=\"n\">mean<\/span><span class=\"p\">(<\/span><span class=\"nb\">abs<\/span><span class=\"p\">(<\/span><span class=\"n\">obs<\/span><span class=\"o\">-<\/span><span class=\"n\">np<\/span><span class=\"o\">.<\/span><span class=\"n\">mean<\/span><span class=\"p\">(<\/span><span class=\"n\">obs<\/span><span class=\"p\">)))<\/span>\r\n<span class=\"n\">rse<\/span> <span class=\"o\">=<\/span> <span class=\"n\">np<\/span><span class=\"o\">.<\/span><span class=\"n\">mean<\/span><span class=\"p\">((<\/span><span class=\"n\">pred<\/span><span class=\"o\">-<\/span><span class=\"n\">obs<\/span><span class=\"p\">)<\/span><span class=\"o\">**<\/span><span class=\"mi\">2<\/span><span class=\"p\">)<\/span><span class=\"o\">\/<\/span><span class=\"n\">np<\/span><span class=\"o\">.<\/span><span class=\"n\">mean<\/span><span class=\"p\">((<\/span><span class=\"n\">obs<\/span><span class=\"o\">-<\/span><span class=\"n\">np<\/span><span class=\"o\">.<\/span><span class=\"n\">mean<\/span><span class=\"p\">(<\/span><span class=\"n\">obs<\/span><span class=\"p\">))<\/span><span class=\"o\">**<\/span><span class=\"mi\">2<\/span><span class=\"p\">)<\/span>\r\n\r\n<span class=\"k\">print<\/span><span class=\"p\">(<\/span><span class=\"s\">\"Mean Absolute Error: \"<\/span> <span class=\"o\">+<\/span> <span class=\"nb\">str<\/span><span class=\"p\">(<\/span><span class=\"nb\">round<\/span><span class=\"p\">(<\/span><span class=\"n\">mae<\/span><span class=\"p\">,<\/span><span class=\"mi\">6<\/span><span class=\"p\">)))<\/span>\r\n<span class=\"k\">print<\/span><span class=\"p\">(<\/span><span class=\"s\">\"Root Mean Squared Error: \"<\/span> <span class=\"o\">+<\/span> <span class=\"nb\">str<\/span><span class=\"p\">(<\/span><span class=\"nb\">round<\/span><span class=\"p\">(<\/span><span class=\"n\">rmse<\/span><span class=\"p\">,<\/span><span class=\"mi\">6<\/span><span class=\"p\">)))<\/span>\r\n<span class=\"k\">print<\/span><span class=\"p\">(<\/span><span class=\"s\">\"Relative Absolute Error: \"<\/span> <span class=\"o\">+<\/span> <span class=\"nb\">str<\/span><span class=\"p\">(<\/span><span class=\"nb\">round<\/span><span class=\"p\">(<\/span><span class=\"n\">rae<\/span><span class=\"p\">,<\/span><span class=\"mi\">6<\/span><span class=\"p\">)))<\/span>\r\n<span class=\"k\">print<\/span><span class=\"p\">(<\/span><span class=\"s\">\"Relative Squared Error: \"<\/span> <span class=\"o\">+<\/span> <span class=\"nb\">str<\/span><span class=\"p\">(<\/span><span class=\"nb\">round<\/span><span class=\"p\">(<\/span><span class=\"n\">rse<\/span><span class=\"p\">,<\/span><span class=\"mi\">6<\/span><span class=\"p\">)))<\/span>\r\n<\/code><\/pre>\n<\/div>\n<div class=\"highlighter-rouge\">\n<pre class=\"highlight\"><code>Mean Absolute Error: 1.424722\r\nRoot Mean Squared Error: 1.594763\r\nRelative Absolute Error: 2.878017\r\nRelative Squared Error: 3.947494\r\n<\/code><\/pre>\n<\/div>\n<h2 id=\"a-deeper-look-at-precision-and-recall\">A Deeper Look at Precision and Recall<\/h2>\n<div class=\"language-python highlighter-rouge\">\n<pre class=\"highlight\"><code><span class=\"c\"># This shows how to generate the precision and recall curves.<\/span>\r\n<span class=\"c\"># We generate these curves for each category.<\/span>\r\n\r\n<span class=\"kn\">from<\/span> <span class=\"nn\">sklearn.metrics<\/span> <span class=\"kn\">import<\/span> <span class=\"n\">precision_recall_curve<\/span>\r\n<span class=\"kn\">from<\/span> <span class=\"nn\">sklearn.metrics<\/span> <span class=\"kn\">import<\/span> <span class=\"n\">average_precision_score<\/span>\r\n\r\n<span class=\"c\"># Compute Precision-Recall and plot curve<\/span>\r\n<span class=\"n\">precision<\/span> <span class=\"o\">=<\/span> <span class=\"nb\">dict<\/span><span class=\"p\">()<\/span>\r\n<span class=\"n\">recall<\/span> <span class=\"o\">=<\/span> <span class=\"nb\">dict<\/span><span class=\"p\">()<\/span>\r\n<span class=\"n\">average_precision<\/span> <span class=\"o\">=<\/span> <span class=\"nb\">dict<\/span><span class=\"p\">()<\/span>\r\n<span class=\"k\">for<\/span> <span class=\"n\">i<\/span> <span class=\"ow\">in<\/span> <span class=\"nb\">range<\/span><span class=\"p\">(<\/span><span class=\"n\">n_classes<\/span><span class=\"p\">):<\/span>\r\n    <span class=\"n\">precision<\/span><span class=\"p\">[<\/span><span class=\"n\">i<\/span><span class=\"p\">],<\/span> <span class=\"n\">recall<\/span><span class=\"p\">[<\/span><span class=\"n\">i<\/span><span class=\"p\">],<\/span> <span class=\"n\">_<\/span> <span class=\"o\">=<\/span> <span class=\"n\">precision_recall_curve<\/span><span class=\"p\">(<\/span><span class=\"n\">y_test<\/span><span class=\"p\">[:,<\/span> <span class=\"n\">i<\/span><span class=\"p\">],<\/span>\r\n                                                        <span class=\"n\">y_score<\/span><span class=\"p\">[:,<\/span> <span class=\"n\">i<\/span><span class=\"p\">])<\/span>\r\n    <span class=\"n\">average_precision<\/span><span class=\"p\">[<\/span><span class=\"n\">i<\/span><span class=\"p\">]<\/span> <span class=\"o\">=<\/span> <span class=\"n\">average_precision_score<\/span><span class=\"p\">(<\/span><span class=\"n\">y_test<\/span><span class=\"p\">[:,<\/span> <span class=\"n\">i<\/span><span class=\"p\">],<\/span> <span class=\"n\">y_score<\/span><span class=\"p\">[:,<\/span> <span class=\"n\">i<\/span><span class=\"p\">])<\/span>\r\n\r\n<span class=\"c\"># Compute micro-average ROC curve and ROC area<\/span>\r\n<span class=\"n\">precision<\/span><span class=\"p\">[<\/span><span class=\"s\">\"micro\"<\/span><span class=\"p\">],<\/span> <span class=\"n\">recall<\/span><span class=\"p\">[<\/span><span class=\"s\">\"micro\"<\/span><span class=\"p\">],<\/span> <span class=\"n\">_<\/span> <span class=\"o\">=<\/span> <span class=\"n\">precision_recall_curve<\/span><span class=\"p\">(<\/span><span class=\"n\">y_test<\/span><span class=\"o\">.<\/span><span class=\"n\">ravel<\/span><span class=\"p\">(),<\/span>\r\n    <span class=\"n\">y_score<\/span><span class=\"o\">.<\/span><span class=\"n\">ravel<\/span><span class=\"p\">())<\/span>\r\n<span class=\"n\">average_precision<\/span><span class=\"p\">[<\/span><span class=\"s\">\"micro\"<\/span><span class=\"p\">]<\/span> <span class=\"o\">=<\/span> <span class=\"n\">average_precision_score<\/span><span class=\"p\">(<\/span><span class=\"n\">y_test<\/span><span class=\"p\">,<\/span> <span class=\"n\">y_score<\/span><span class=\"p\">,<\/span>\r\n                                                     <span class=\"n\">average<\/span><span class=\"o\">=<\/span><span class=\"s\">\"micro\"<\/span><span class=\"p\">)<\/span>\r\n<span class=\"c\"># Later in the code ...<\/span>\r\n\r\n<span class=\"c\"># Plot the precision and recall curves.<\/span>\r\n<span class=\"o\">%<\/span><span class=\"n\">matplotlib<\/span> <span class=\"n\">inline<\/span>\r\n\r\n<span class=\"kn\">import<\/span> <span class=\"nn\">matplotlib<\/span>\r\n<span class=\"kn\">import<\/span> <span class=\"nn\">numpy<\/span> <span class=\"kn\">as<\/span> <span class=\"nn\">np<\/span>\r\n<span class=\"kn\">import<\/span> <span class=\"nn\">matplotlib.pyplot<\/span> <span class=\"kn\">as<\/span> <span class=\"nn\">plt<\/span>\r\n\r\n<span class=\"c\"># Plot Precision-Recall curve<\/span>\r\n<span class=\"n\">plt<\/span><span class=\"o\">.<\/span><span class=\"n\">clf<\/span><span class=\"p\">()<\/span>\r\n<span class=\"n\">plt<\/span><span class=\"o\">.<\/span><span class=\"n\">plot<\/span><span class=\"p\">(<\/span><span class=\"n\">recall<\/span><span class=\"p\">[<\/span><span class=\"mi\">0<\/span><span class=\"p\">],<\/span> <span class=\"n\">precision<\/span><span class=\"p\">[<\/span><span class=\"mi\">0<\/span><span class=\"p\">],<\/span> <span class=\"n\">label<\/span><span class=\"o\">=<\/span><span class=\"s\">'Precision-Recall curve'<\/span><span class=\"p\">)<\/span>\r\n<span class=\"n\">plt<\/span><span class=\"o\">.<\/span><span class=\"n\">xlabel<\/span><span class=\"p\">(<\/span><span class=\"s\">'Recall'<\/span><span class=\"p\">)<\/span>\r\n<span class=\"n\">plt<\/span><span class=\"o\">.<\/span><span class=\"n\">ylabel<\/span><span class=\"p\">(<\/span><span class=\"s\">'Precision'<\/span><span class=\"p\">)<\/span>\r\n<span class=\"n\">plt<\/span><span class=\"o\">.<\/span><span class=\"n\">ylim<\/span><span class=\"p\">([<\/span><span class=\"mf\">0.0<\/span><span class=\"p\">,<\/span> <span class=\"mf\">1.05<\/span><span class=\"p\">])<\/span>\r\n<span class=\"n\">plt<\/span><span class=\"o\">.<\/span><span class=\"n\">xlim<\/span><span class=\"p\">([<\/span><span class=\"mf\">0.0<\/span><span class=\"p\">,<\/span> <span class=\"mf\">1.0<\/span><span class=\"p\">])<\/span>\r\n<span class=\"n\">plt<\/span><span class=\"o\">.<\/span><span class=\"n\">title<\/span><span class=\"p\">(<\/span><span class=\"s\">'Precision-Recall example: AUC={0:0.2f}'<\/span><span class=\"o\">.<\/span><span class=\"n\">format<\/span><span class=\"p\">(<\/span><span class=\"n\">average_precision<\/span><span class=\"p\">[<\/span><span class=\"mi\">0<\/span><span class=\"p\">]))<\/span>\r\n<span class=\"n\">plt<\/span><span class=\"o\">.<\/span><span class=\"n\">legend<\/span><span class=\"p\">(<\/span><span class=\"n\">loc<\/span><span class=\"o\">=<\/span><span class=\"s\">\"lower left\"<\/span><span class=\"p\">)<\/span>\r\n<span class=\"n\">plt<\/span><span class=\"o\">.<\/span><span class=\"n\">show<\/span><span class=\"p\">()<\/span>\r\n\r\n<span class=\"c\"># Plot Precision-Recall curve for each class<\/span>\r\n<span class=\"n\">plt<\/span><span class=\"o\">.<\/span><span class=\"n\">clf<\/span><span class=\"p\">()<\/span>\r\n<span class=\"n\">plt<\/span><span class=\"o\">.<\/span><span class=\"n\">plot<\/span><span class=\"p\">(<\/span><span class=\"n\">recall<\/span><span class=\"p\">[<\/span><span class=\"s\">\"micro\"<\/span><span class=\"p\">],<\/span> <span class=\"n\">precision<\/span><span class=\"p\">[<\/span><span class=\"s\">\"micro\"<\/span><span class=\"p\">],<\/span>\r\n         <span class=\"n\">label<\/span><span class=\"o\">=<\/span><span class=\"s\">'micro-average Precision-recall curve (area = {0:0.2f})'<\/span>\r\n               <span class=\"s\">''<\/span><span class=\"o\">.<\/span><span class=\"n\">format<\/span><span class=\"p\">(<\/span><span class=\"n\">average_precision<\/span><span class=\"p\">[<\/span><span class=\"s\">\"micro\"<\/span><span class=\"p\">]))<\/span>\r\n<span class=\"k\">for<\/span> <span class=\"n\">i<\/span> <span class=\"ow\">in<\/span> <span class=\"nb\">range<\/span><span class=\"p\">(<\/span><span class=\"n\">n_classes<\/span><span class=\"p\">):<\/span>\r\n    <span class=\"n\">plt<\/span><span class=\"o\">.<\/span><span class=\"n\">plot<\/span><span class=\"p\">(<\/span><span class=\"n\">recall<\/span><span class=\"p\">[<\/span><span class=\"n\">i<\/span><span class=\"p\">],<\/span> <span class=\"n\">precision<\/span><span class=\"p\">[<\/span><span class=\"n\">i<\/span><span class=\"p\">],<\/span>\r\n             <span class=\"n\">label<\/span><span class=\"o\">=<\/span><span class=\"s\">'Precision-recall curve of class {0} (area = {1:0.2f})'<\/span>\r\n                   <span class=\"s\">''<\/span><span class=\"o\">.<\/span><span class=\"n\">format<\/span><span class=\"p\">(<\/span><span class=\"n\">i<\/span><span class=\"p\">,<\/span> <span class=\"n\">average_precision<\/span><span class=\"p\">[<\/span><span class=\"n\">i<\/span><span class=\"p\">]))<\/span>\r\n\r\n<span class=\"n\">plt<\/span><span class=\"o\">.<\/span><span class=\"n\">xlim<\/span><span class=\"p\">([<\/span><span class=\"mf\">0.0<\/span><span class=\"p\">,<\/span> <span class=\"mf\">1.0<\/span><span class=\"p\">])<\/span>\r\n<span class=\"n\">plt<\/span><span class=\"o\">.<\/span><span class=\"n\">ylim<\/span><span class=\"p\">([<\/span><span class=\"mf\">0.0<\/span><span class=\"p\">,<\/span> <span class=\"mf\">1.05<\/span><span class=\"p\">])<\/span>\r\n<span class=\"n\">plt<\/span><span class=\"o\">.<\/span><span class=\"n\">xlabel<\/span><span class=\"p\">(<\/span><span class=\"s\">'Recall'<\/span><span class=\"p\">)<\/span>\r\n<span class=\"n\">plt<\/span><span class=\"o\">.<\/span><span class=\"n\">ylabel<\/span><span class=\"p\">(<\/span><span class=\"s\">'Precision'<\/span><span class=\"p\">)<\/span>\r\n<span class=\"n\">plt<\/span><span class=\"o\">.<\/span><span class=\"n\">title<\/span><span class=\"p\">(<\/span><span class=\"s\">'Extension of Precision-Recall curve to multi-class'<\/span><span class=\"p\">)<\/span>\r\n<span class=\"n\">plt<\/span><span class=\"o\">.<\/span><span class=\"n\">legend<\/span><span class=\"p\">(<\/span><span class=\"n\">loc<\/span><span class=\"o\">=<\/span><span class=\"s\">\"lower right\"<\/span><span class=\"p\">)<\/span>\r\n<span class=\"n\">plt<\/span><span class=\"o\">.<\/span><span class=\"n\">show<\/span><span class=\"p\">()<\/span>\r\n<\/code><\/pre>\n<\/div>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/cse\/wp-content\/uploads\/sites\/55\/2020\/03\/2016-01-07-Categorizing-Driver-Risk-with-Machine-Learning-output_19_0.png\" alt=\"png\" \/><\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/cse\/wp-content\/uploads\/sites\/55\/2020\/03\/2016-01-07-Categorizing-Driver-Risk-with-Machine-Learning-output_19_1.png\" alt=\"png\" \/><\/p>\n<h2 id=\"deploy-the-model-as-an-azure-machine-learning-web-service\">Deploy the Model as an Azure Machine Learning Web Service<\/h2>\n<div class=\"language-python highlighter-rouge\">\n<pre class=\"highlight\"><code><span class=\"c\"># This is how to deploy the model to Azure Machine Learning as a web<\/span>\r\n<span class=\"c\"># service. By doing this, others can invoke the model with their telemetry<\/span>\r\n<span class=\"c\"># data to get a risk rating for their data.<\/span>\r\n<span class=\"c\">#<\/span>\r\n<span class=\"c\"># This webservice is exactly the same as an Azure Machine Learning web<\/span>\r\n<span class=\"c\"># service deployed from the AzureML Studio -- all the same examples and<\/span>\r\n<span class=\"c\"># and sample code are available.<\/span>\r\n\r\n<span class=\"kn\">import<\/span> <span class=\"nn\">uuid<\/span>\r\n<span class=\"kn\">from<\/span> <span class=\"nn\">azureml<\/span> <span class=\"kn\">import<\/span> <span class=\"n\">services<\/span>\r\n\r\n<span class=\"nd\">@services.publish<\/span><span class=\"p\">(<\/span><span class=\"n\">workspace_id<\/span><span class=\"p\">,<\/span> <span class=\"n\">authorization_token<\/span><span class=\"p\">)<\/span>\r\n<span class=\"c\"># This is where we set the service_id, created above. Not including this<\/span>\r\n<span class=\"c\"># line would result in a new service being created for each invocation.<\/span>\r\n<span class=\"nd\">@services.service_id<\/span><span class=\"p\">(<\/span><span class=\"n\">service_id<\/span><span class=\"p\">)<\/span>\r\n<span class=\"nd\">@services.types<\/span><span class=\"p\">(<\/span><span class=\"n\">accs<\/span><span class=\"o\">=<\/span><span class=\"nb\">float<\/span><span class=\"p\">,<\/span> <span class=\"n\">lanecs<\/span><span class=\"o\">=<\/span><span class=\"nb\">float<\/span><span class=\"p\">,<\/span> <span class=\"n\">turns<\/span><span class=\"o\">=<\/span><span class=\"nb\">float<\/span><span class=\"p\">,<\/span> <span class=\"n\">brks<\/span><span class=\"o\">=<\/span><span class=\"nb\">float<\/span><span class=\"p\">)<\/span>\r\n<span class=\"nd\">@services.returns<\/span><span class=\"p\">(<\/span><span class=\"nb\">float<\/span><span class=\"p\">)<\/span>\r\n<span class=\"k\">def<\/span> <span class=\"nf\">demoservice<\/span><span class=\"p\">(<\/span><span class=\"n\">accs<\/span><span class=\"p\">,<\/span> <span class=\"n\">lanecs<\/span><span class=\"p\">,<\/span> <span class=\"n\">turns<\/span><span class=\"p\">,<\/span> <span class=\"n\">brks<\/span><span class=\"p\">):<\/span>\r\n    <span class=\"c\"># predict the risk rating category<\/span>\r\n    <span class=\"k\">return<\/span> <span class=\"n\">classifier<\/span><span class=\"o\">.<\/span><span class=\"n\">predict<\/span><span class=\"p\">([<\/span><span class=\"n\">accs<\/span><span class=\"p\">,<\/span> <span class=\"n\">lanecs<\/span><span class=\"p\">,<\/span> <span class=\"n\">turns<\/span><span class=\"p\">,<\/span> <span class=\"n\">brks<\/span><span class=\"p\">])<\/span>\r\n<\/code><\/pre>\n<\/div>\n<h2 id=\"use-azure-machine-learning-web-service-version\">Use Azure Machine Learning Web Service Version<\/h2>\n<div class=\"language-python highlighter-rouge\">\n<pre class=\"highlight\"><code><span class=\"c\"># Here's a simple test showing how to use the deployed web service.<\/span>\r\n<span class=\"n\">test_row<\/span> <span class=\"o\">=<\/span> <span class=\"mi\">0<\/span>\r\n<span class=\"n\">obs<\/span> <span class=\"o\">=<\/span> <span class=\"n\">y_raw<\/span><span class=\"o\">.<\/span><span class=\"n\">iloc<\/span><span class=\"p\">[<\/span><span class=\"n\">test_row<\/span><span class=\"p\">]<\/span>\r\n<span class=\"n\">feature_vec<\/span> <span class=\"o\">=<\/span> <span class=\"n\">X<\/span><span class=\"o\">.<\/span><span class=\"n\">iloc<\/span><span class=\"p\">[<\/span><span class=\"n\">test_row<\/span><span class=\"p\">]<\/span>\r\n<span class=\"n\">result<\/span> <span class=\"o\">=<\/span> <span class=\"n\">demoservice<\/span><span class=\"p\">(<\/span><span class=\"n\">feature_vec<\/span><span class=\"o\">.<\/span><span class=\"n\">ACCS<\/span><span class=\"p\">,<\/span> <span class=\"n\">feature_vec<\/span><span class=\"o\">.<\/span><span class=\"n\">LANECS<\/span><span class=\"p\">,<\/span> \r\n                     <span class=\"n\">feature_vec<\/span><span class=\"o\">.<\/span><span class=\"n\">TURNS<\/span><span class=\"p\">,<\/span> <span class=\"n\">feature_vec<\/span><span class=\"o\">.<\/span><span class=\"n\">BRKS<\/span><span class=\"p\">)<\/span>\r\n<span class=\"n\">result<\/span>\r\n<\/code><\/pre>\n<\/div>\n<h2 id=\"opportunities-for-reuse\">Opportunities for Reuse<\/h2>\n<p>Since this notebook lives in GitHub, there are at least two reusable components:<\/p>\n<ol>\n<li>The Machine Learning Model, Data, and Services can be instantiated by anyone in their subscription and they can create a risk rating service. This makes it easier for us to build a machine learning solution with a partner and give it to them to continue to use after our collaboration has ended. This addresses one particular challenge the Azure Machine Learning Studio doesn\u2019t quite address (yet).<\/li>\n<li>The Jupyter notebook can be modified and reused as a means to explore machine learning algorithms locally (on a local laptop), then when ready can be \u201cproductized\u201d into Azure.<\/li>\n<\/ol>\n","protected":false},"excerpt":{"rendered":"<p>This case study describes a project with Toyota where we have constructed a machine learning web service that takes summary telemetry data as the input and assigns a risk rating to the trip.<\/p>\n","protected":false},"author":21345,"featured_media":12517,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[19],"tags":[160,372],"class_list":["post-2144","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-machine-learning","tag-driving","tag-usage-based-insurance-ubi"],"acf":[],"blog_post_summary":"<p>This case study describes a project with Toyota where we have constructed a machine learning web service that takes summary telemetry data as the input and assigns a risk rating to the trip.<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/posts\/2144","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/users\/21345"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/comments?post=2144"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/posts\/2144\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/media\/12517"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/media?parent=2144"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/categories?post=2144"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/tags?post=2144"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}