{"id":20680,"date":"2018-11-08T12:57:16","date_gmt":"2018-11-08T20:57:16","guid":{"rendered":"https:\/\/blogs.msdn.microsoft.com\/dotnet\/?p=20125"},"modified":"2024-01-12T11:31:59","modified_gmt":"2024-01-12T19:31:59","slug":"announcing-ml-net-0-7-machine-learning-net","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/dotnet\/announcing-ml-net-0-7-machine-learning-net\/","title":{"rendered":"Announcing ML.NET 0.7 (Machine Learning .NET)"},"content":{"rendered":"<p><a href=\"https:\/\/user-images.githubusercontent.com\/1712635\/48220219-96602f00-e343-11e8-918e-5ca6c0dbc3b3.png\" target=\"_blank\" rel=\"noopener\"><img decoding=\"async\" style=\"max-width: 100%;\" src=\"https:\/\/user-images.githubusercontent.com\/1712635\/48220219-96602f00-e343-11e8-918e-5ca6c0dbc3b3.png\" alt=\"ML.NET icon\" \/><\/a><\/p>\n<p>We&#8217;re excited to announce today the release of ML.NET 0.7 &#8211; the latest release of the cross-platform and open source machine learning framework for .NET developers (<a href=\"https:\/\/devblogs.microsoft.com\/dotnet\/introducing-ml-net-cross-platform-proven-and-open-source-machine-learning-framework\/\" rel=\"nofollow\">ML.NET 0.1 was released at \/\/Build 2018<\/a>). This release focuses on enabling better support for recommendation based ML tasks, enabling anomaly detection, enhancing the customizability of the machine learning pipelines, enabling using ML.NET in x86 apps, and more.<\/p>\n<p>This blog post provides details about the following topics in the ML.NET 0.7 release:<\/p>\n<ul>\n<li><a href=\"#enhanced-support-for-recommendation-tasks-with-matrix-factorization\" rel=\"nofollow\">Enhanced support for recommendation tasks with Matrix Factorization<\/a><\/li>\n<li><a href=\"#enabled-anomaly-detection-scenarios-and-detecting-unusual-events\" rel=\"nofollow\">Enabled anomaly detection scenarios &#8211; detecting unusual events rel<\/a><\/li>\n<li><a href=\"#improved-customizability-of-mlnet-pipelines\" rel=\"nofollow\">Improved customizability of ML pipelines<\/a><\/li>\n<li><a href=\"#x86-support-in-addition-to-x64\" rel=\"nofollow\">x86 support<\/a><\/li>\n<li><a href=\"#nimbusml---experimental-python-bindings-for-mlnet\" rel=\"nofollow\">NimbusML &#8211; experimental Python bindings for ML.NET<\/a><\/li>\n<li><a href=\"#get-started\" rel=\"nofollow\">Get started with ML.NET v07<\/a><\/li>\n<\/ul>\n<h2><a id=\"enhanced-support-for-recommendation-tasks-with-matrix-factorization\" class=\"anchor\" href=\"#enhanced-support-for-recommendation-tasks-with-matrix-factorization\"><\/a>Enhanced support for recommendation tasks with Matrix Factorization<\/h2>\n<p><a target=\"_blank\" rel=\"noopener\"><img decoding=\"async\" style=\"max-width: 100%;\" src=\"https:\/\/user-images.githubusercontent.com\/1712635\/48220475-374eea00-e344-11e8-9fcb-ca4c134aa5bd.png\" alt=\"Recommendation icon\" \/><\/a><\/p>\n<p>Recommender systems enable producing a list of recommendations for products in a catalog, songs, movies, and more. We have improved support for creating recommender systems in ML.NET by adding Matrix factorization (MF), a common approach to recommendations when you have data on how users rated items in your catalog. For example, you might know how users rated some movies and want to recommend which other movies they are likely to watch next.<\/p>\n<p>We added MF to ML.NET because it is often significantly faster than Field-Aware Factorization Machines (which we added in <a href=\"https:\/\/blogs.msdn.microsoft.com\/dotnet\/2018\/07\/09\/announcing-ml-net-0-3\/#ffm-section\" rel=\"nofollow\">ML.NET 0.3<\/a>) and it can support ratings which are continuous number ratings (e.g. 1-5 stars) instead of boolean values (&#8220;liked&#8221; or &#8220;didn&#8217;t like&#8221;). Even though we just added MF, you might still want to use FFM if you want to take advantage of other information beyond the rating a user assigns to an item (e.g. movie genre, movie release date, user profile). A more in-depth discussion of the differences can be found <a href=\"https:\/\/www.csie.ntu.edu.tw\/~cjlin\/talks\/recsys.pdf\" rel=\"nofollow\">here<\/a>.<\/p>\n<p>A sample app using Matrix Factorization based on user&#8217;s ratings can be found <a href=\"https:\/\/github.com\/dotnet\/machinelearning-samples\/tree\/master\/samples\/csharp\/getting-started\/MatrixFactorization_MovieRecommendation\">here (Movie Recommendation Engine)<\/a>. In this sample app you can see how to use ML.NET to build a movie recommendation engine where you have data such as UserId, ProductId and Ratings available to you for what users bought and rated.<\/p>\n<p>ML.NET&#8217;s MF uses <a href=\"https:\/\/github.com\/cjlin1\/libmf\">LIBMF<\/a>.<\/p>\n<h2><a id=\"enabled-anomaly-detection-scenarios-and-detecting-unusual-events\" class=\"anchor\" href=\"#enabled-anomaly-detection-scenarios-and-detecting-unusual-events\"><\/a>Enabled anomaly detection scenarios &#8211; detecting unusual events<\/h2>\n<p><a href=\"https:\/\/user-images.githubusercontent.com\/1712635\/48219726-1b4a4900-e342-11e8-9175-21a232fcbaab.png\" target=\"_blank\" rel=\"noopener\">\n<img decoding=\"async\" style=\"max-width: 100%;\" src=\"https:\/\/user-images.githubusercontent.com\/1712635\/48219726-1b4a4900-e342-11e8-9175-21a232fcbaab.png\" alt=\"Anomaly detection icon\" height=\"10\" \/><\/a><\/p>\n<p><a href=\"https:\/\/en.wikipedia.org\/wiki\/Anomaly_detection\" rel=\"nofollow\">Anomaly detection<\/a> enables identifying unusual values or events. It is used in scenarios such as fraud detection (identifying suspicious credit card transactions) and server monitoring (identifying unusual activity).<\/p>\n<p>ML.NET 0.7 enables detecting two types of anomalous behavior:<\/p>\n<ul>\n<li>Spike detection: spikes are attributed to sudden yet temporary bursts in values of the input data. These could be outliers due to outages, cyber-attacks, viral web content, etc.<\/li>\n<li>Change point detection: change points mark the beginning of more persistent deviations in the behavior of the data. For example, if product sales are relatively consistent and become more popular (monthly sales double), there is a change point when the trend changes.<\/li>\n<\/ul>\n<p>These anomalies can be detected on two types of data using different ML.NET components:<\/p>\n<ul>\n<li style=\"list-style-type: none;\">\n<ul>\n<li><code>IidSpikeDetector<\/code> and <code>IidChangePointDetector<\/code> are used on data assumed to be from one stationary distribution (each data point is independent of previous data, such as the number of retweets of each tweet).<\/li>\n<li><code>SsaSpikeDetector<\/code> and <code>SsaChangePointDetector<\/code> are used on data that has a season\/trend components (perhaps ordered by time, such as product sales)<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>Sample code using anomaly detection with ML.NET can be found <a href=\"https:\/\/github.com\/dotnet\/machinelearning\/blob\/7fb76b026d0035d6da4d0b46bd3f2a6e3c0ce3f1\/test\/Microsoft.ML.TimeSeries.Tests\/TimeSeriesDirectApi.cs\">here<\/a>.<\/p>\n<h2><a id=\"improved-customizability-of-mlnet-pipelines\" class=\"anchor\" href=\"#improved-customizability-of-mlnet-pipelines\"><\/a>Improved customizability of ML.NET pipelines<\/h2>\n<p><a href=\"https:\/\/user-images.githubusercontent.com\/1712635\/48220606-aa586080-e344-11e8-86a9-ced9e596b91a.png\" target=\"_blank\" rel=\"noopener\"><img decoding=\"async\" style=\"max-width: 100%;\" src=\"https:\/\/user-images.githubusercontent.com\/1712635\/48220606-aa586080-e344-11e8-86a9-ced9e596b91a.png\" alt=\"Pipeline icon\" \/><\/a><\/p>\n<p>ML.NET offers a variety of data transformations (e.g. processing text, images, categorical features, etc.). However, some use cases require application-specific transformations, such as calculating <a href=\"https:\/\/en.wikipedia.org\/wiki\/Cosine_similarity\" rel=\"nofollow\">cosine similarity<\/a> between two text columns. We have now added support for custom transforms so you can easily include custom business logic.<\/p>\n<p>The <code>CustomMappingEstimator<\/code> allows you to write your own methods to process data and bring them into the ML.NET pipeline. Here is what it would look like in the pipeline:<\/p>\n<div class=\"highlight highlight-source-cs\">\n<pre class=\"lang:default decode:true\">var estimator = mlContext.Transforms.CustomMapping&lt;MyInput, MyOutput&gt;(MyLambda.MyAction, \"MyLambda\")\r\n    .Append(...)\r\n    .Append(...)<\/pre>\n<\/div>\n<p>Below is the definition of what this custom mapping will do. In this example, we convert the text label (&#8220;spam&#8221; or &#8220;ham&#8221;) to a boolean label (true or false).<\/p>\n<div class=\"highlight highlight-source-cs\">\n<pre class=\"lang:default decode:true  \">public class MyInput\r\n{\r\n    public string Label { get; set; }\r\n}\r\n\r\npublic class MyOutput\r\n{\r\n    public bool Label { get; set; }\r\n}\r\n\r\npublic class MyLambda\r\n{\r\n    [Export(\"MyLambda\")]\r\n    public ITransformer MyTransformer =&gt; ML.Transforms.CustomMappingTransformer&lt;MyInput, MyOutput&gt;(MyAction, \"MyLambda\");\r\n\r\n    [Import]\r\n    public MLContext ML { get; set; }\r\n\r\n    public static void MyAction(MyInput input, MyOutput output)\r\n    {\r\n        output.Label= input.Label == \"spam\" ? true : false;\r\n    }\r\n}<\/pre>\n<\/div>\n<p>A more complete example of the <code>CustomMappingEstimator<\/code> can be found <a href=\"https:\/\/github.com\/dotnet\/machinelearning\/blob\/d68388a1c9994a5b429b194b64b2b0782834cb78\/test\/Microsoft.ML.Tests\/Transformers\/CustomMappingTests.cs#L55\">here<\/a>.<\/p>\n<h2><a id=\"x86-support-in-addition-to-x64\" class=\"anchor\" href=\"#x86-support-in-addition-to-x64\"><\/a>x86 support in addition to x64<\/h2>\n<p><a href=\"https:\/\/user-images.githubusercontent.com\/1712635\/48220746-fefbdb80-e344-11e8-8234-cb394f9d971b.png\" target=\"_blank\" rel=\"noopener\"><img decoding=\"async\" style=\"max-width: 100%;\" src=\"https:\/\/user-images.githubusercontent.com\/1712635\/48220746-fefbdb80-e344-11e8-8234-cb394f9d971b.png\" alt=\"Pipeline icon\" \/><\/a><\/p>\n<p>With this release of ML.NET you can now train and use machine learning models on x86 \/ 32-bit architecture devices (Windows only, for now). Previously, ML.NET was limited to x64 devices (Windows, Linux and Mac).\nNote that some components that are based on external dependencies (e.g. TensorFlow) are not available in x86-Windows.<\/p>\n<h2><a id=\"nimbusml---experimental-python-bindings-for-mlnet\" class=\"anchor\" href=\"#nimbusml---experimental-python-bindings-for-mlnet\"><\/a><a href=\"https:\/\/github.com\/microsoft\/nimbusml\">NimbusML<\/a> &#8211; experimental Python bindings for ML.NET<\/h2>\n<p><a href=\"https:\/\/user-images.githubusercontent.com\/1712635\/48221253-6ebe9600-e346-11e8-83b7-4fea089385bc.png\" target=\"_blank\" rel=\"noopener\"><img decoding=\"async\" style=\"max-width: 100%;\" src=\"https:\/\/user-images.githubusercontent.com\/1712635\/48221253-6ebe9600-e346-11e8-83b7-4fea089385bc.png\" alt=\"Python logo\" \/><\/a><\/p>\n<p><a href=\"https:\/\/github.com\/microsoft\/nimbusml\">NimbusML<\/a> provides experimental Python bindings for ML.NET. We have seen feedback from the external community and internal teams regarding the use of multiple programming languages. We wanted to enable as many people as possible to benefit from ML.NET and help teams to work together more easily. ML.NET not only enables data scientists to train and use machine learning models in Python (with components that can also be used in <a href=\"http:\/\/scikit-learn.org\/stable\/\" rel=\"nofollow\">scikit-learn<\/a> pipelines), but it also enables saving models which can be easily used in .NET applications through ML.NET (see <a href=\"https:\/\/docs.microsoft.com\/en-us\/nimbusml\/loadsavemodels\" rel=\"nofollow\">here<\/a> for more details).<\/p>\n<h2><a id=\"in-case-you-missed-it-provide-your-feedback-on-the-new-api\" class=\"anchor\" href=\"#in-case-you-missed-it-provide-your-feedback-on-the-new-api\"><\/a>In case you missed it: provide your feedback on the new API<\/h2>\n<p><a href=\"https:\/\/blogs.msdn.microsoft.com\/dotnet\/2018\/10\/08\/announcing-ml-net-0-6-machine-learning-net\/\" rel=\"nofollow\">ML.NET 0.6<\/a> introduced a new set of APIs for ML.NET that provide enhanced flexibility. These APIs in 0.7 and upcoming versions are still evolving and we would love to get your feedback so you can help shape the long-term API for ML.NET.<\/p>\n<p>Want to get involved? Start by providing feedback through issues at the <a href=\"https:\/\/github.com\/dotnet\/machinelearning\/issues\">ML.NET GitHub repo<\/a>!<\/p>\n<h2><a id=\"additional-resources\" class=\"anchor\" href=\"#additional-resources\"><\/a>Additional resources<\/h2>\n<ul>\n<li>The most important <strong>ML.NET concepts<\/strong> for understanding the new API are introduced <a href=\"https:\/\/docs.microsoft.com\/en-us\/dotnet\/machine-learning\/basic-concepts-model-training-in-mldotnet\">here<\/a>.<\/li>\n<li>A <strong>cookbook (How to guides)<\/strong> that shows how to use these APIs for a variety of existing and new scenarios can be found <a href=\"https:\/\/docs.microsoft.com\/en-us\/dotnet\/machine-learning\/how-to-guides\/\">here<\/a>.<\/li>\n<li>A <strong>ML.NET API Reference<\/strong> with all the documented APIs can be found <a href=\"https:\/\/docs.microsoft.com\/en-us\/dotnet\/api\/?view=ml-dotnet\">here<\/a>.<\/li>\n<\/ul>\n<h2><a id=\"get-started\" class=\"anchor\" href=\"#get-started\"><\/a>Get started!<\/h2>\n<p><a href=\"https:\/\/user-images.githubusercontent.com\/1712635\/48221401-d70d7780-e346-11e8-8588-6e5cd3a38341.png\" target=\"_blank\" rel=\"noopener\"><img decoding=\"async\" style=\"max-width: 100%;\" src=\"https:\/\/user-images.githubusercontent.com\/1712635\/48221401-d70d7780-e346-11e8-8588-6e5cd3a38341.png\" alt=\"Get started icon\" \/><\/a><\/p>\n<p>If you haven\u2019t already, get started with <a href=\"https:\/\/www.microsoft.com\/net\/learn\/apps\/machine-learning-and-ai\/ml-dotnet\/get-started\" rel=\"nofollow\">ML.NET here<\/a> Next, explore some other great resources:<\/p>\n<ul>\n<li>Tutorials and resources at the <a href=\"https:\/\/docs.microsoft.com\/en-us\/dotnet\/machine-learning\/\" rel=\"nofollow\">Microsoft Docs ML.NET Guide<\/a><\/li>\n<li>Code samples at the <a href=\"https:\/\/github.com\/dotnet\/machinelearning-samples\">machinelearning-samples GitHub repo<\/a><\/li>\n<\/ul>\n<p>We look forward to your feedback and welcome you to file issues with any suggestions or enhancements in the <a href=\"https:\/\/github.com\/dotnet\/machinelearning\">ML.NET GitHub repo<\/a>.<\/p>\n<p><em>This blog was authored by Gal Oshri and Cesar de la Torre<\/em><\/p>\n<p>Thanks,<\/p>\n<p>The ML.NET Team<\/p>\n","protected":false},"excerpt":{"rendered":"<p>We&#8217;re excited to announce today the release of ML.NET 0.7 &#8211; the latest release of the cross-platform and open source machine learning framework for .NET developers (ML.NET 0.1 was released at \/\/Build 2018). This release focuses on enabling better support for recommendation based ML tasks, enabling anomaly detection, enhancing the customizability of the machine learning [&hellip;]<\/p>\n","protected":false},"author":362,"featured_media":21740,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[685,328,688,691],"tags":[9,93,96],"class_list":["post-20680","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-dotnet","category-aiml","category-machine-learning","category-ml-dotnet","tag-net-core","tag-machine-learning","tag-ml-net"],"acf":[],"blog_post_summary":"<p>We&#8217;re excited to announce today the release of ML.NET 0.7 &#8211; the latest release of the cross-platform and open source machine learning framework for .NET developers (ML.NET 0.1 was released at \/\/Build 2018). This release focuses on enabling better support for recommendation based ML tasks, enabling anomaly detection, enhancing the customizability of the machine learning [&hellip;]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts\/20680","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/users\/362"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/comments?post=20680"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts\/20680\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/media\/21740"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/media?parent=20680"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/categories?post=20680"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/tags?post=20680"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}