{"id":10746,"date":"2017-06-26T10:53:26","date_gmt":"2017-06-26T17:53:26","guid":{"rendered":"https:\/\/blogs.msdn.microsoft.com\/dotnet\/?p=10746"},"modified":"2021-09-30T10:09:27","modified_gmt":"2021-09-30T17:09:27","slug":"dot-net-recommendation-system-for-net-applications-using-azure-machine-learning","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/dotnet\/dot-net-recommendation-system-for-net-applications-using-azure-machine-learning\/","title":{"rendered":".NET AI\/ML themed blogs: Building recommendation engine for .NET applications using Azure Machine Learning"},"content":{"rendered":"<p><strong><em>\u23f1<\/em><\/strong><em><span style=\"color: #339966\"> Updated on <strong>June 28, 2017<\/strong> <\/span><\/em><\/p>\n<div role=\"main\">\n<div class=\"gist-content-wrapper\" id=\"gist-pjax-container\">\n<div class=\"container new-discussion-timeline experiment-repo-nav\">\n<div class=\"repository-content gist-content\">\n<div class=\"js-gist-file-update-container js-task-list-container file-box\">\n<div class=\"file\" id=\"file-fs-ds-ml-first-md\">\n<div class=\"readme blob instapaper_body\" id=\"readme\">\n<article class=\"markdown-body entry-content\"><i>&#128588; Welcome to the first of \u00a0.NET&#8217;s new AI and Machine Learning themed blog entries! We have set up this space as a place to share and discuss the work we will be doing with AI and Machine Learning. Through a series of blog posts, we would like to show you different ways on how .NET developers can leverage Machine learning and AI to build smarter, intelligent applications.Over the next few months, we hope to use this space to start discussions, engage with our community around these concepts. We will be monitoring this space closely, and encourage you to contribute through ideas, commentary and more! &#128588;<\/i>\n<br>In this very first post, I would like to cover how you can use Machine Learning as a service using different cloud API offerings today.\u00a0We will look at <a href=\"https:\/\/studio.azureml.net\/\">Azure Machine Learning<\/a> Studio to start with.To go over these in detail, we will go ahead and build a movie recommendation service using the <a href=\"https:\/\/grouplens.org\/datasets\/movielens\/\">movielens dataset<\/a>. \u00a0Recommendation engines \u00a0are used everywhere today predicting search queries, music, books, movies and a lot more! \u00a0<a href=\"https:\/\/www.tastehit.com\/blog\/personalized-recommendations-at-netflix\/\">Online sources claim that more than 75% of movies or shows watched on Netflix<\/a> are powered through the Netflix movie recommendation engine. Let us see if we can better this number with the recommendation engine we can build!<\/p>\n<p>The post has the following sections:<\/p>\n<ul>\n<li><a href=\"#Background\">Background<\/a><\/li>\n<li><a href=\"#Prerequisites\">Prerequisites for this tutorial<\/a><\/li>\n<li><a href=\"#Approach\">Approach for building this recommendation engine<\/a><\/li>\n<li><a href=\"#recommendation\">Creating the Movie Recommendation model in Azure Machine Learning\nStudio<\/a><\/li>\n<li><a href=\"#asp.net\">Creating our Movie Recommendation Web application with ASP.NET Core<\/a><\/li>\n<li><a href=\"#wrapup\">Wrap Up<\/a><\/li>\n<\/ul>\n<p><span>Don\u2019t have time? Don\u2019t worry! The complete solution is also <\/span><a href=\"https:\/\/github.com\/dotnet-architecture\/RecommendationEngine\">available on GitHub<\/a><span> and the corresponding <\/span><a href=\"https:\/\/gallery.cortanaintelligence.com\/Experiment\/MovieLens-Movie-Recommender\">Azure Machine Learning studio experiment<\/a><span> can be found here.<\/span><\/p>\n<h4 id=\"Background\">Background<\/h4>\n<p>Artificial Intelligence (AI) is exploding in popularity today. Like many things, it&#8217;s not new either. In fact, some rather exciting things about AI <a href=\"https:\/\/www.youtube.com\/watch?v=aygSMgK3BEM\">have been promised since the 1960s<\/a>! Although not all bright-eyed promises about AI have come to pass, the subset of AI known as Machine Learning (ML) has been used in various computing systems for quite some time. Now more then ever, the availability of compute power coupled together with powerful algorithms and availability of massive data is leading to a unique opportunity to build smarter applications providing differentiated experiences.<\/p>\n<p>If you&#8217;re not familiar with machine learning, there is a bit to learn up front. Although it&#8217;s not necessary to get an advanced degree in this subject matter to be successful, there will be some learning required on your part if you&#8217;re starting from nothing.Machine Learning (and Data Science) is fundamentally a scientific process where you explore data, manipulate it to your benefit, try different approaches to solving a problem, iterate on each approach, and evaluate your work. Although programming is heavily involved, it&#8217;s more of a means to an end.Through a series of blog posts, we would like to show you different ways on how .NET developers can leverage Machine learning and AI to build engaging customer experiences.To aid with this, we would like to go over the different scenarios on how .NET developers can leverage Machine learning and AI to build engaging customer experiences.In the process, we would love to learn from your feedback as we explore this space together.<\/p>\n<h4 id=\"Prerequisites\">Prerequisites for this tutorial<\/h4>\n<p>Please make sure you have the following to make this experience work for Azure:<\/p>\n<ul>\n<li>An active Azure account. If you don&#8217;t have one, you can sign up for a <a href=\"https:\/\/azure.microsoft.com\/pricing\/free-trial\/\">Free Azure Trial<\/a>.<\/li>\n<li>Go through the getting started experience with <a href=\"https:\/\/azure.microsoft.com\/en-us\/trial\/get-started-machine-learning\/\">Azure Machine Learning Studio<\/a>.<\/li>\n<li><span>While not explicitly required but <a href=\"https:\/\/www.visualstudio.com\/vs\/net-development\/\">.NET development<\/a> works best with <\/span><span><a href=\"https:\/\/www.visualstudio.com\/vs\/net-development\/\">Visual Studio<\/a>\u00a0<\/span>you will require .NET components installed.<\/li>\n<li><a href=\"http:\/\/files.grouplens.org\/datasets\/movielens\/ml-20m.zip\">MovieLens 20M dataset<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/dotnet-architecture\/RecommendationEngine\">ASP.NET Core web app prototype using AzureML movie recommender (preview)<\/a><\/li>\n<li>[Optional] <a href=\"https:\/\/gallery.cortanaintelligence.com\/Experiment\/MovieLens-Movie-Recommender\">MovieLens 1M Movie Recommendation Model<\/a><\/li>\n<\/ul>\n<h4 id=\"Approach\"><span>Approach <\/span><\/h4>\n<p>Azure Machine Learning Studio comes pre-packages with a variety of sample data. For this tutorial we will use the MovieLens dataset which comes with movie ratings, titles, genres and more. \u00a0In terms of an approach for building our movie recommendation engine we have the following options.<\/p>\n<dl>\n<dt>Population Averages<\/dt>\n<dd>The simplest recommendation engine one can build would be using population averages. So essentially take all reviews for a given movie and create an average rating. If the average rating is higher than a certain threshold say 4\/5, we can recommend this movie to our user. \u00a0This simple solution might work well but it does not account for user preferences. I might be a huge action movie fan and love the &#8216;Mission Impossible&#8217; series, but my friend might be more so a Sci-fi fan and loves &#8216;Star Trek&#8217;. Both these movies might have high ratings based on population averages but neither \u00a0me, nor my friend would be impressed with a recommendation engine providing these suggestions.<\/dd>\n<dt><strong>Content-based<\/strong><\/dt>\n<dd>Content-based filtering methods are based on a description of the item and a profile of the user\u2019s preference.\u00a0 In a content-based recommender system, keywords are used to describe the items and a user profile is built to indicate the type of item this user likes. In other words, these algorithms try to recommend items that are similar to those that a user liked in the past.<\/dd>\n<dt><strong>Collaborative filtering<\/strong><\/dt>\n<dd><em>&#8216;Collaborative filtering&#8217;<\/em> operates under the underlying assumption that if a person A has the same opinion as a person B on an issue, A is more likely to have B&#8217;s opinion on a different issue than that of a randomly chosen person.<\/dd>\n<dd>These two text book recommendation approaches <em>&#8216;content-based approach&#8217;<\/em> and a <em>&#8216;collaborative filtering&#8217;<\/em>\u00a0solve the problem of lack of preferences, presented by population averages. <a href=\"https:\/\/msdn.microsoft.com\/en-us\/library\/azure\/dn905987.aspx\">Azure\u2019s MatchBox Recommender<\/a> combines the best of both worlds to provide the most accurate recommendations.<\/dd>\n<dt><strong>Graphical Models<\/strong><\/dt>\n<dd>While a hybrid approach using Content based and colloborative filtering works well, it has two underlying assumptions. The first, that user preferences are static and do not change over time, second the relationship between user and item preferences are simple. In reality, this is not true. Graphical models help out providing a solution to the two assumptions above e.g. the Markovian structure, can capture the temporal aspects of user preferences in recommendation system but in the interest to keep this blog simple, we won&#8217;t discuss this approach in much detail here.<\/dd>\n<\/dl>\n<p>For this model we will build a hybrid approach based content-based and collaborative filtering using Azure Machine Learning studio.<\/p>\n<p>Once the model is completed, we will evaluate it by trying to predict ratings for movies that the model has not seen and compare it against the actual rating for these movies. Once we are happy with its performance, we will then publish this model as a web service and call it using REST through ASP.NET Core MVC application. You can <a href=\"https:\/\/github.com\/dotnet-architecture\/RecommendationEngine\">find the completed sample here<\/a>. All you need to do is to change the API key and URI in the \u2018appsettings.json\u2019 file to match your azure account and model settings.<\/p>\n<h4 id=\"getstart\">Let\u2019s get started<\/h4>\n<p>If you are short on time, you can just go ahead <a href=\"https:\/\/gallery.cortanaintelligence.com\/Experiment\/MovieLens-Movie-Recommender\">open up the already completed movie recommender based upon the MovieLens 1M dataset here<\/a>\u00a0and you can directly jump to \u2018Setup a webservice section\u2019<strong>.<\/strong> If you would like to understand a wee-bit more on how this model was built you can follow along the steps below.<\/p>\n<h4 id=\"recommendation\">Creating the Movie Recommendation model in Azure Machine Learning Studio<\/h4>\n<p>An analytic framework that one goes through while incorporating machine learning involves pre-preprocessing, feature extraction and selection, applying the machine learning model and some kind of scoring to understand how well the model worked. So, let\u2019s get started!<\/p>\n<h5><span>Step 1: \u2018Choosing the dataset\u2019<\/span><\/h5>\n<p>We will use the MovieLens20M dataset which you can <a href=\"http:\/\/files.grouplens.org\/datasets\/movielens\/ml-20m.zip\">download from here<\/a>. \u00a0The dataset for most part is\nalready pre-processed and comes with movies.csv which has the movieId, along with the title of the movie and genres. The ratings.csv contains the userId, movieId, rating and time-stamp.<\/p>\n<p>Login into Azure Machine Learning studio and using the DataSets tab upload both movies.csv and ratings.csv as new datasets.<\/p>\n<p><span><a href=\"http:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/datasets1.png\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/datasets1.png\" alt=\"\" width=\"619\" height=\"144\" class=\"alignnone wp-image-12175\" srcset=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/datasets1.png 2347w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/datasets1-300x70.png 300w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/datasets1-768x179.png 768w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/datasets1-1024x239.png 1024w\" sizes=\"(max-width: 619px) 100vw, 619px\" \/><\/a><\/span><\/p>\n<p><a href=\"http:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/datasets.png\"><\/a>After you have uploaded these two datasets, you can now start using them.<\/p>\n<h5><span>Step 2: \u2018Preparing the data to build the model\u2019<\/span><\/h5>\n<p>Now that the datasets have been uploaded, move to the experiments tab and create a new experiment and choose \u2018Blank Experiment\u2019. Go ahead and name your experiment, say \u2018MovieLens Movie Recommender\u2019.<\/p>\n<h5><span>Step 2a: Choosing the datasets<\/span><\/h5>\n<p>All models essentially start with the dataset, so let\u2019s go ahead and drop the Ratings (MovieLens) dataset to our experiment.<\/p>\n<p><a href=\"http:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/movielens-recommender.png\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/movielens-recommender.png\" alt=\"\" width=\"359\" height=\"136\" class=\"alignnone wp-image-12185\" srcset=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/movielens-recommender.png 1113w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/movielens-recommender-300x113.png 300w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/movielens-recommender-768x291.png 768w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/movielens-recommender-1024x387.png 1024w\" sizes=\"(max-width: 359px) 100vw, 359px\" \/><\/a><\/p>\n<p>Once added you can go ahead and visualize the dataset.<\/p>\n<p><a href=\"http:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/visualize1.png\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/visualize1.png\" alt=\"\" width=\"382\" height=\"252\" class=\"alignnone wp-image-12187\" srcset=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/visualize1.png 1285w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/visualize1-300x198.png 300w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/visualize1-768x506.png 768w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/visualize1-1024x675.png 1024w\" sizes=\"(max-width: 382px) 100vw, 382px\" \/><\/a><\/p>\n<p>This is a neat trick as it allows you to view the dataset and the transformations during the course of \u00a0your model. As you can see this dataset comes in with the 20M ratings and 4 columns: userId, movieId, rating and time-stamp as shown below:<\/p>\n<p><a href=\"http:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/visualize-ratings.png\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/visualize-ratings.png\" alt=\"\" width=\"374\" height=\"267\" class=\"alignnone wp-image-12195\" srcset=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/visualize-ratings.png 1050w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/visualize-ratings-300x214.png 300w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/visualize-ratings-768x549.png 768w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/visualize-ratings-1024x731.png 1024w\" sizes=\"(max-width: 374px) 100vw, 374px\" \/><\/a><\/p>\n<h5><span>Step 2b Feature Engineering<\/span><\/h5>\n<p>In order, to get the best result for our recommendation engine, we next need to do some feature engineering, which means to pick only columns in this dataset which we believe will impact the prediction. \u2018UserId\u2019, \u2018MovieId\u2019, \u2018Rating\u2019 all seem important but the &#8216;Timestamp&#8217; on when this rating came in doesn\u2019t seem all that important.<\/p>\n<p>So let\u2019s go ahead and remove the &#8216;Timestamp&#8217; column by using the \u2018select columns in dataset\u2019. You can type \u2018select column\u2019 in the experiment pane to find this widget.<\/p>\n<p><a href=\"http:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/featureeng.png\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/featureeng.png\" alt=\"\" width=\"169\" height=\"120\" class=\"alignnone wp-image-10785\" srcset=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/featureeng.png 460w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/featureeng-300x212.png 300w\" sizes=\"(max-width: 169px) 100vw, 169px\" \/><\/a><\/p>\n<p>Go ahead and connect the dataset with the select columns in the dataset module. You will notice a red warning icon, which is there as we need to choose the columns to select.<\/p>\n<p><a href=\"http:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/select.png\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/select.png\" alt=\"\" width=\"575\" height=\"158\" class=\"alignnone wp-image-12205\" srcset=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/select.png 1857w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/select-300x82.png 300w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/select-768x211.png 768w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/select-1024x281.png 1024w\" sizes=\"(max-width: 575px) 100vw, 575px\" \/><\/a><\/p>\n<p>Select \u2018userId\u2019, \u2018movieId\u2019 and \u2018rating\u2019 columns respectively.<\/p>\n<p>In addition to the user ratings, with the MovieLens dataset we also have the movie genre available as an item feature, making use of this item feature will improve this model and also allow us to avoid the cold start problem which I will describe in the next section. So let&#8217;s make use of it as well.<\/p>\n<p><a href=\"http:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/itemfeatures-1.png\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/itemfeatures.1.png\" alt=\"\" width=\"582\" height=\"218\" class=\"alignnone wp-image-12215\" \/><\/a><\/p>\n<h5><span>Step 2c Splitting the dataset<\/span><\/h5>\n<p>When training a machine learning model, we always need to do two things per dataset, first shuffle the data and next split the data into training and test datasets. A 50\/50 split between training data and test data is very common and doing this allows us to avoid the problem of overfitting or underfitting.<\/p>\n<p>To split your dataset, you can use the split data module available under Data Transformation category. Be sure to use the &#8216;Recommender Split&#8217; in splitting mode for this model.<\/p>\n<p><a href=\"http:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/recommender2.png\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/recommender2.png\" alt=\"\" width=\"537\" height=\"210\" class=\"alignnone wp-image-12216\" \/><\/a><\/p>\n<h5><span>Step 3: Using the Matchbox Recommender <\/span><\/h5>\n<p>Now that our data has been prepared we can go ahead and apply a machine learning model. Azure Machine Learning studio comes packaged with many common machine learning models. <a href=\"https:\/\/msdn.microsoft.com\/library\/azure\/fa4aa69d-2f1c-4ba4-ad5f-90ea3a515b4c\">Matchbox recommender<\/a> is one such model.<\/p>\n<p>There are two primary approaches to <span>recommendation <\/span>systems. The first is a content-based approach, which makes use of features for both users and items. The is a collaborative filtering approach. The underlying assumption of the collaborative filtering approach is that if a person A has the same opinion as a person B on an issue, A is more likely to have B&#8217;s opinion on a different issue than that of a randomly chosen person<\/p>\n<p><span>The Matchbox recommender combines collaborative filtering with a content-based approach. It is therefore considered a <\/span><strong>hybrid recommender<\/strong><span>. When a user is relatively new to the system, predictions are improved by making use of the feature information about the user, thus addressing the well-known &#8220;cold-start&#8221; problem. We don&#8217;t have any user features available in the MovieLens dataset so instead we will use item features (e.g. genre) and provide recommendations on related items, this will help resolve the &#8220;cold-start&#8221; problem.<\/span><\/p>\n<p>Remember,\u00a0once you have collected a sufficient number of ratings from a particular user, it is possible to make fully personalized predictions for them based on their specific ratings rather than on their features alone. Hence, there is a smooth transition from content-based recommendations to recommendations based on collaborative filtering. Even if user or item features are not available, Matchbox will still work in its collaborative filtering mode.<\/p>\n<p>The first input to the Train MatchBox Recommender requires a triplet of user-item-rating, \u00a0so let&#8217;s start with that.<\/p>\n<p><a href=\"http:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/trainmatchbox.png\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/trainmatchbox.png\" alt=\"\" width=\"1363\" height=\"132\" class=\"alignnone size-full wp-image-12085\" srcset=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/trainmatchbox.png 1363w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/trainmatchbox-300x29.png 300w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/trainmatchbox-768x74.png 768w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/trainmatchbox-1024x99.png 1024w\" sizes=\"(max-width: 1363px) 100vw, 1363px\" \/><\/a><\/p>\n<p>If you hover over the other two inputs the matchbox recommender takes in, you can see it can be easily extended to use more user features etc. but we will do this later!<\/p>\n<p>Go ahead and add the Matchbox Recommender (Train) module to our experiment and take the \u2018training\u2019 data input from the split data module and connect the two.\u00a0 The properties window for this module will allow you to choose properties specific to this module like the number of traits to run, number of iterations and the number of recommendations to make.<\/p>\n<p><a href=\"http:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/matchbox.png\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/matchbox.png\" alt=\"\" width=\"532\" height=\"169\" class=\"alignnone wp-image-10815\" srcset=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/matchbox.png 1602w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/matchbox-300x95.png 300w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/matchbox-768x244.png 768w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/matchbox-1024x325.png 1024w\" sizes=\"(max-width: 532px) 100vw, 532px\" \/><\/a><\/p>\n<p>Next, connect the third input for the MatchBox Recommender with the item features dataset (movieId, Genre) as well. This will allow our MatchBox Recommender to work as a hybrid recommender.<\/p>\n<p><a href=\"http:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/matchbox1.png\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/matchbox1.png\" alt=\"\" width=\"412\" height=\"311\" class=\"alignnone wp-image-12226\" srcset=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/matchbox1.png 1113w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/matchbox1-300x226.png 300w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/matchbox1-768x580.png 768w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/matchbox1-1024x773.png 1024w\" sizes=\"(max-width: 412px) 100vw, 412px\" \/><\/a><\/p>\n<p>Once training is complete you can also go ahead and save your trained model to be used again. This will save you time w.r.t. retraining the same model again and again. You can do so by using the following option shown below:<\/p>\n<p><a href=\"http:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/trainedmodel-1.png\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/trainedmodel.1.png\" alt=\"\" width=\"443\" height=\"272\" class=\"alignnone wp-image-12585\" \/><\/a><\/p>\n<h5><span>Step 4: \u2018Scoring our Matchbox Recommender \u2019<\/span><\/h5>\n<p>Once training is complete the next step is to go ahead and score the recommender. Just like the train Matchbox recommender drag and drop the score matchbox recommender and connect the output of train matchbox recommender and the test data from the \u2018Split Data\u2019 module to it.\u00a0 This will allow us to score the trained model against the test data.<\/p>\n<p>The recommender kind prediction property will allow us to choose between performing a rating, item and related recommender kinds.<\/p>\n<p><a href=\"http:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/scoretype.png\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/scoretype.png\" alt=\"\" width=\"197\" height=\"130\" class=\"alignnone wp-image-12095\" srcset=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/scoretype.png 314w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/scoretype-300x198.png 300w\" sizes=\"(max-width: 197px) 100vw, 197px\" \/><\/a>\nWe will use the rating prediction to evaluate our algorithm and the related items prediction to then recommend a set of movies.<\/p>\n<p><a href=\"http:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/score.png\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/score.png\" alt=\"\" width=\"513\" height=\"303\" class=\"alignnone wp-image-12236\" srcset=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/score.png 1254w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/score-300x177.png 300w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/score-768x453.png 768w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/score-1024x604.png 1024w\" sizes=\"(max-width: 513px) 100vw, 513px\" \/><\/a><\/p>\n<p>At this point, we are almost done, go ahead and run your model! Once you are run is complete and successful you should see green tick marks against your modules.<\/p>\n<h5><span>Step 5: Evaluating your model<\/span><\/h5>\n<p>To figure out, how our model is performing, we will need to go ahead and evaluate our model. Go ahead and drag and drop the &#8216;Evaluate Recommender&#8217; module.<\/p>\n<p>Evaluate Recommender compares the ground truth ratings of the test dataset with the predicted ratings of the scored dataset, computes the mean absolute error (MAE) and the root mean squared error (RMSE), and returns both metrics in the output dataset. <span>RMSE<\/span><span>\u00a0is a frequently used measure of the differences between values (sample and population values) predicted by a model or an estimator and the values actually observed.<\/span><\/p>\n<p><a href=\"http:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/evaluate.png\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/evaluate.png\" alt=\"\" width=\"472\" height=\"312\" class=\"alignnone wp-image-12575\" srcset=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/evaluate.png 1296w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/evaluate-300x198.png 300w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/evaluate-768x507.png 768w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/evaluate-1024x676.png 1024w\" sizes=\"(max-width: 472px) 100vw, 472px\" \/><\/a><\/p>\n<p>As a comparison point the RMSE for Netflix using\u00a0<span>BellKor&#8217;s Pragmatic Chaos was measured to be\u00a00.8553. Go ahead and re-run your model, once complete visualize your RMSE metric and do find out whether we beat the Netflix recommender!, not too bad ;)!<\/span><\/p>\n<p><a href=\"http:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/metric.png\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/metric.png\" alt=\"\" width=\"317\" height=\"140\" class=\"alignnone wp-image-12595\" srcset=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/metric.png 1062w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/metric-300x132.png 300w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/metric-768x338.png 768w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/metric-1024x451.png 1024w\" sizes=\"(max-width: 317px) 100vw, 317px\" \/><\/a><\/p>\n<p>&nbsp;<\/p>\n<h5><span>Step 6: Set up and deploy our web service <\/span><\/h5>\n<p>For our ASP.NET core application to be able to consume this model in production we need to go ahead and set this up as a service. You can set this up as a service using the \u2018Set Up Web Service\u2019 option and picking the Predictive Web Service setup option.<\/p>\n<p>Once the \u2018Predictive Web Service is setup\u2019, you should see the webservice input and output modules appear. Modify them as follows:<\/p>\n<p><a href=\"http:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/webservice.png\"><\/a>This out-of-the-box\u00a0configuration would work for us but it would require providing an additional input \u2018timestamp\u2019 to our prediction service which will eventually get filtered away. We can optimize this flow a little by directly going to the \u2018Score Matchbox Recommender input\u2019 shown above.<\/p>\n<p>Another thing to make sure before we go ahead and publish this webservice is that the \u2018Score MatchBox Recommender\u2019 is set to s<strong>core all recommended item selection <\/strong>and maximum number of items is set to &#8216;6&#8217;, for our companion app to work best.\u00a0\u00a0This is a must for it work in a production\/web-service environment.<\/p>\n<p><a href=\"http:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/productionscoring.png\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/productionscoring.png\" alt=\"\" width=\"169\" height=\"216\" class=\"alignnone wp-image-12605\" srcset=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/productionscoring.png 505w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/productionscoring-235x300.png 235w\" sizes=\"(max-width: 169px) 100vw, 169px\" \/><\/a><\/p>\n<p>To speed up the setup phase you can also add another split as shown for scoring. This will save you time without affecting your results as your model has already been trained at this point, but this is optional.<\/p>\n<p>One last thing to make sure of is that the webservice output is only accepting the six movie recommendations, this will require an additional &#8216;Select Columns in Dataset&#8217;. When you are done your model would look like as follows:<\/p>\n<p><a href=\"http:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/webservice2.png\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/webservice2.png\" alt=\"\" width=\"405\" height=\"310\" class=\"alignnone wp-image-13126\" srcset=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/webservice2.png 909w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/webservice2-300x229.png 300w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/webservice2-768x587.png 768w\" sizes=\"(max-width: 405px) 100vw, 405px\" \/><\/a><\/p>\n<h5><span>Step 7: Testing our webservice <\/span><\/h5>\n<p>After deploying your webservice, you should see the following dashboard. The dashboard should provide you the API Key, which we will use to connect to our webservice. In addition to this, we can also test out the default end-point using the test capability provided. So, let\u2019s go ahead and do that.<\/p>\n<p><a href=\"http:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/movielens.png\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/movielens.png\" alt=\"\" width=\"751\" height=\"243\" class=\"alignnone wp-image-10835\" srcset=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/movielens.png 1592w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/movielens-300x97.png 300w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/movielens-768x248.png 768w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/movielens-1024x331.png 1024w\" sizes=\"(max-width: 751px) 100vw, 751px\" \/><\/a><\/p>\n<p>For testing this, I am going to pick a movie from the MovieLens database that I have seen, provide a rating for it and see how well this model works. I picked the <a href=\"https:\/\/en.wikipedia.org\/wiki\/Seven_(1995_film)\">Movie \u2018Se7en\u2019<\/a> (id:47), and since I really liked the movie I am going to pick a 5\/5 rating for this movie and submit my input to get a predictive result for a choice of six recommended related movies that I would like.<\/p>\n<p>You should get response back as a JSON object:<\/p>\n<p><a href=\"http:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/predictive.png\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/predictive-3.png\" alt=\"\" width=\"1276\" height=\"73\" class=\"alignnone size-full wp-image-13095\" \/><\/a><\/p>\n<p>Mapping the MovieID\u2019s returned to Movie Titles, this seems to work fairly well.<\/p>\n<p><a href=\"http:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/IDTitle.png\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/IDTitle.png\" alt=\"\" width=\"373\" height=\"128\" class=\"alignnone wp-image-13105\" srcset=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/IDTitle.png 824w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/IDTitle-300x103.png 300w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/IDTitle-768x264.png 768w\" sizes=\"(max-width: 373px) 100vw, 373px\" \/><\/a><\/p>\n<h5><span>Step 8: Integrating the movie recommendation service to my app<\/span><\/h5>\n<p>Azure Machine Learning studio makes the web-service integration easy by also providing you the .NET code you need. Click the \u2018Request\/Response\u2019 link in the Default End-point section.<\/p>\n<p><a href=\"http:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/endpoint.png\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/endpoint.png\" alt=\"\" width=\"573\" height=\"148\" class=\"alignnone wp-image-10856\" srcset=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/endpoint.png 893w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/endpoint-300x78.png 300w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/endpoint-768x199.png 768w\" sizes=\"(max-width: 573px) 100vw, 573px\" \/><\/a><\/p>\n<p>This will take you to the request\/response API documentation page for this Movie Recommender. Go ahead and traverse to the sample code section. It will provide you the code you need, in this case the C# code to connect to this service as a part of the InvokeRequestResponseService method.<\/p>\n<p>All you need to do to customize it, is to replace the apiKey string with the apiKey for your webservice and you are good to go!. You can find the apiKey on the original dashboard for this webservice.<\/p>\n<p>The sample provided with this post specifies the \u00a0apiKey and the URI in the appsettings.json file.<\/p>\n<h4 id=\"asp.net\">Creating our ASP.NET Core movie recommendation website<\/h4>\n<p>With our movie recommendation model in Azure ready to go, we can go next and start creating our ASP.NET Core website.<\/p>\n<h5><span>Step 9: Creating our movie-recommendation website with ASP.NET Core<\/span><\/h5>\n<p>We are on home stretch now, g<a href=\"https:\/\/github.com\/dotnet-architecture\/RecommendationEngine\">o ahead and clone this sample to start with<\/a>. Alternatively you can also create a new ASP.NET Core MVC app using the File-&gt;New experience in Visual Studio using the code sample provided as a reference.<\/p>\n<p>The first thing we need to do in order to make the code sample work is to <a href=\"https:\/\/github.com\/dotnet-architecture\/RecommendationEngine\/blob\/master\/movierecommender\/appsettings.json\">add the &#8216;apikey&#8217; and &#8216;uri&#8217; to the appsettings.json file<\/a>. You will <span>obtain these after building your model in Azure Machine Learning Studio.<\/span><\/p>\n<p>This very simple app has the following components as you would expect:<\/p>\n<ul>\n<li>Model (Movie) which contains a list of Movie ID\u2019s and Movie Titles. On model instantiation, the model is populated with a list of IMDB movie titles.<\/li>\n<li>Controller (MoviesController) which handles user interaction, work with the model, logic to query our Azure movie recommendation service and selection of which views to recommend<\/li>\n<li>Views (\u2018Choose\u2019 and \u2018Recommend\u2019), the former view will allow a user to choose movies liked in the past and the second view \u2018Recommend\u2019 will display the list of recommended movies based upon that data point.<\/li>\n<\/ul>\n<p>You can build and start the app up, the app will startup up in the &#8216;Choose&#8217; view as shown below. Choosing a movie you have liked in the past say &#8216;Air Force One (1997)&#8217; will result in a JSON request getting created and then <a href=\"https:\/\/github.com\/dotnet-architecture\/RecommendationEngine\/blob\/master\/movierecommender\/Controllers\/MoviesController.cs\">querying a prediction from the web-service using this code snippet<\/a>\u00a0(<em><span>InvokeRequestResponseService<\/span><\/em>).<\/p>\n<p>The resulting JSON object containing the recommended \u2018movie id\u2019s are then mapped to \u2018movie-name\u2019s which are represented as a part of the Recommended view.<\/p>\n<p><a href=\"http:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/views2.png\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/views2.png\" alt=\"\" width=\"2518\" height=\"709\" class=\"alignnone size-full wp-image-13136\" srcset=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/views2.png 2518w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/views2-300x84.png 300w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/views2-768x216.png 768w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2017\/06\/views2-1024x288.png 1024w\" sizes=\"(max-width: 2518px) 100vw, 2518px\" \/><\/a><\/p>\n<h4 id=\"wrapup\">Wrap Up<\/h4>\n<p>This tutorial shows one of the many examples where, Machine Learning can benefit your .NET application. Using Azure Machine Learning studio however is only one of many other ways how .NET developers can integrate ML and AI into their applications. In the\u00a0coming tutorial we will look at how you can use .NET with other ML\/AI technologies.<\/p>\n<p>Thank you for reading to the end, I look forward to hearing your thoughts and feedback on this post.<\/p>\n<\/article>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>\u23f1 Updated on June 28, 2017 &#128588; Welcome to the first of \u00a0.NET&#8217;s new AI and Machine Learning themed blog entries! We have set up this space as a place to share and discuss the work we will be doing with AI and Machine Learning. Through a series of blog posts, we would like to [&hellip;]<\/p>\n","protected":false},"author":11288,"featured_media":58792,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[685],"tags":[9,17,32,40,93],"class_list":["post-10746","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-dotnet","tag-net-core","tag-net-ml-ai","tag-asp-net-core","tag-azure-machine-learning-studio","tag-machine-learning"],"acf":[],"blog_post_summary":"<p>\u23f1 Updated on June 28, 2017 &#128588; Welcome to the first of \u00a0.NET&#8217;s new AI and Machine Learning themed blog entries! We have set up this space as a place to share and discuss the work we will be doing with AI and Machine Learning. Through a series of blog posts, we would like to [&hellip;]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts\/10746","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/users\/11288"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/comments?post=10746"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts\/10746\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/media\/58792"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/media?parent=10746"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/categories?post=10746"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/tags?post=10746"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}