{"id":34823,"date":"2021-10-04T16:16:33","date_gmt":"2021-10-04T23:16:33","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/dotnet\/?p=34823"},"modified":"2021-10-14T07:41:26","modified_gmt":"2021-10-14T14:41:26","slug":"ml-net-and-model-builder-october-updates","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/dotnet\/ml-net-and-model-builder-october-updates\/","title":{"rendered":"ML.NET Updates &#038; Announcing Notebooks in Visual Studio"},"content":{"rendered":"<p><a href=\"https:\/\/dot.net\/ml\" target=\"_blank\" rel=\"noopener\">ML.NET<\/a> is an open-source, cross-platform machine learning framework for .NET developers that enables integration of custom machine learning into .NET apps.<\/p>\n<p>In this post, we&#8217;ll cover the following items:<\/p>\n<ol>\n<li><a href=\"#model-builder-updates\">Model Builder updates<\/a><\/li>\n<li><a href=\"#progress-on-addressing-ml-net-pain-points\">Progress on addressing ML.NET pain points<\/a><\/li>\n<li><a href=\"#get-started-and-resources\">Get started and resources<\/a><\/li>\n<\/ol>\n<h2>Model Builder updates<\/h2>\n<h3>Notebook Editor in Visual Studio<\/h3>\n<p>Interactive Notebooks are used extensively in data science and machine learning. They are great for data exploration and preparation, experimentation, model explainability, and even education.<\/p>\n<p>Last year, .NET Interactive Notebooks were <a href=\"https:\/\/devblogs.microsoft.com\/dotnet\/net-interactive-preview-3-vs-code-insiders-and-polyglot-notebooks\/\" target=\"_blank\" rel=\"noopener\">announced<\/a>, and you can currently use .NET Interactive Notebooks in VS Code as an <a href=\"https:\/\/marketplace.visualstudio.com\/items?itemName=ms-dotnettools.dotnet-interactive-vscode\" target=\"_blank\" rel=\"noopener\">extension<\/a>.<\/p>\n<p>After talking to customers, the team decided to experiment with Interactive Notebooks in Visual Studio which has resulted in the new <a href=\"https:\/\/marketplace.visualstudio.com\/items?itemName=MLNET.notebook\" target=\"_blank\" rel=\"noopener\">Notebook Editor<\/a> extension!<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2021\/09\/mb-notebook-graph.png\" alt=\"Screenshot of Notebook with a graph in Visual Studio\" \/><\/p>\n<h4>Getting started with Notebook Editor<\/h4>\n<p>Notebook Editor is only available in Visual Studio 2022 starting with Preview 4 and is currently offered as an experimental (preview) extension.<\/p>\n<p>To try it out, you should first:<\/p>\n<ol>\n<li>Install Visual Studio 2022 Preview 4 (or newer).<\/li>\n<li>Install the <a href=\"https:\/\/marketplace.visualstudio.com\/items?itemName=MLNET.notebook\" target=\"_blank\" rel=\"noopener\">Notebook Editor extension<\/a> from the Visual Studio Marketplace.<\/li>\n<\/ol>\n<p>Then, there are two entry points to get started with Notebook Editor in Visual Studio.<\/p>\n<p>The first entry point is from ML.NET Model Builder, where you can get a generated Notebook with content based on your own data and model.<\/p>\n<p>To get a Notebook from Model Builder:<\/p>\n<ol>\n<li>Install the latest version of <a href=\"https:\/\/aka.ms\/mb-2022\" target=\"_blank\" rel=\"noopener\">Model Builder<\/a> for VS 2022.<\/li>\n<li>Train a model with Model Builder and go to the <strong>Consume<\/strong> step.<\/li>\n<li>Under <strong>Project templates<\/strong>, add the <strong>Notebook<\/strong> to your solution.\n<a href=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2021\/10\/mb-consume-notebook-project-template.png\"><img decoding=\"async\" class=\"alignleft size-full wp-image-35090\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2021\/10\/mb-consume-notebook-project-template.png\" alt=\"Screenshot of Consume step in Model Builder with Notebook Project Template\" width=\"2255\" height=\"1302\" srcset=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2021\/10\/mb-consume-notebook-project-template.png 2255w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2021\/10\/mb-consume-notebook-project-template-300x173.png 300w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2021\/10\/mb-consume-notebook-project-template-1024x591.png 1024w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2021\/10\/mb-consume-notebook-project-template-768x443.png 768w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2021\/10\/mb-consume-notebook-project-template-1536x887.png 1536w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2021\/10\/mb-consume-notebook-project-template-2048x1182.png 2048w\" sizes=\"(max-width: 2255px) 100vw, 2255px\" \/><\/a><\/li>\n<li>Double click on the <strong>.ipynb<\/strong> file that is now in the <strong>Solution Explorer<\/strong> to open the Notebook in <strong>Notebook Editor<\/strong>.\n<img decoding=\"async\" class=\"alignleft\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2021\/09\/ipynb-solution-explorer.png\" alt=\"Screenshot of the Notebook file in Solution Explorer, ML.NET and Model Builder October Updates\" width=\"426\" height=\"532\" \/><\/li>\n<\/ol>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>The generated Notebook from Model Builder contains:<\/p>\n<ul>\n<li>The training pipeline for the model chosen by Model Builder so that you can see how your model was trained and easily re-train<\/li>\n<li>Plots and graphs for data exploration and model explainability techniques so that you can more easily understand and explain your data and model<\/li>\n<\/ul>\n<p><img decoding=\"async\" class=\"alignnone\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2021\/09\/mb-notebook.png\" alt=\"Screenshot of the Notebook generated by Model Builder, ML.NET and Model Builder October Updates\" width=\"1478\" height=\"922\" \/><\/p>\n<p>The second entry point for Notebooks is to simply add a new Notebook from the <strong>Add New Item<\/strong> dialog.<\/p>\n<ol>\n<li>Right click on your project in the <strong>Solution Explorer<\/strong>.<\/li>\n<li>Select <strong>Add &gt; New Item\u2026<\/strong><\/li>\n<li>In the <strong>Add New Item<\/strong> dialog, select <strong>Notebook<\/strong> and add it to your project.\n<img decoding=\"async\" class=\"alignnone\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2021\/09\/add-new-notebook.png\" alt=\"Screenshot of Notebook in Add New Item dialog, ML.NET and Model Builder October Updates\" width=\"1179\" height=\"817\" \/><\/li>\n<\/ol>\n<p>This creates a blank Notebook with no content. You can try adding some C# code and running a cell in the Notebook like this:<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2021\/09\/new-notebook.png\" alt=\"Screenshot of System.Console.WriteLine(&quot;Hello World&quot;) in a Notebook cell\" \/><\/p>\n<p>This is the first version of Notebooks in Visual Studio! If you have any feedback, issues, or questions about Notebook Editor or Notebooks in Visual Studio, please file an issue in our <a href=\"https:\/\/github.com\/dotnet\/machinelearning-modelbuilder\/issues\/new\/choose\" target=\"_blank\" rel=\"noopener\">GitHub repo<\/a>.<\/p>\n<h3>Consumption code improvements<\/h3>\n<p>After you train a model in Model Builder, the Consumption file is generated and added to your project. This Consumption file contains a Predict() method which you can use to make predictions with your model in your end-user application.<\/p>\n<p>This method abstracts away several steps that are needed to consume an ML.NET model:<\/p>\n<ol>\n<li>Initializing an MLContext<\/li>\n<li>Loading the model<\/li>\n<li>Creating a PredictionEngine<\/li>\n<li>Using the PredictionEngine and the model to make the prediction on the input data<\/li>\n<\/ol>\n<p>In the previously generated model consumption code, these steps all happened inside the Predict() method, meaning that these all happened every time the Predict() method was called. This resulted in decreased performance on each prediction.<\/p>\n<p>So, we updated the code to make it a lot more efficient where all of these steps only happen once when using the Predict() method.<\/p>\n<p>The new code is demonstrated below:<\/p>\n<pre><code class=\"language-csharp\">public static ModelOutput Predict(ModelInput input)\r\n{\r\n    var predEngine = PredictEngine.Value;\r\n    return predEngine.Predict(input);\r\n}\r\n\r\nprivate static PredictionEngine&lt;ModelInput, ModelOutput&gt; CreatePredictEngine()\r\n{\r\n    var mlContext = new MLContext();\r\n    ITransformer mlModel = mlContext.Model.Load(MLNetModelPath, out var _);\r\n    return mlContext.Model.CreatePredictionEngine&lt;ModelInput, ModelOutput&gt;(mlModel);\r\n}<\/code><\/pre>\n<p>Read more about all of this month&#8217;s updates in the <a href=\"https:\/\/github.com\/dotnet\/machinelearning-modelbuilder\/blob\/main\/docs\/release-notes\/16.7.3.2143002.md\" target=\"_blank\" rel=\"noopener\">Release Notes<\/a>.<\/p>\n<h2>Progress on addressing ML.NET pain points<\/h2>\n<p>As mentioned in the last ML.NET <a href=\"https:\/\/devblogs.microsoft.com\/dotnet\/ml-net-june-updates-model-builder\/#ml-net-survey-results\" target=\"_blank\" rel=\"noopener\">blog post<\/a>, the following items were found as top pain points or blockers in this year&#8217;s ML.NET customer development:<\/p>\n<ol>\n<li>Small ML.NET Community<\/li>\n<li>Afraid Microsoft will abandon the framework<\/li>\n<li>Lack of \/ quality docs and samples<\/li>\n<li>Lack of deep learning support<\/li>\n<li>Specific ML scenario or algorithm not supported by ML.NET<\/li>\n<\/ol>\n<p>Below we have outlined the steps we&#8217;ve taken so far and progress we&#8217;ve made in each area.<\/p>\n<h3>Small ML.NET Community<\/h3>\n<p>The team continues to host the Machine Learning .NET Community Standup every other week to talk about what we&#8217;re working on and to educate and engage with the community. We&#8217;ve also added a new story to the ML.NET Customer Showcase and are working on adding more.<\/p>\n<p>We are also encouraging contributions to ML.NET. The <a href=\"https:\/\/github.com\/dotnet\/machinelearning\/labels\/good%20first%20issue\" target=\"_blank\" rel=\"noopener\">first good issue<\/a> and <a href=\"https:\/\/github.com\/dotnet\/machinelearning\/issues?q=is%3Aopen+is%3Aissue+label%3Aup-for-grabs\" target=\"_blank\" rel=\"noopener\">up-for-grab<\/a> issues on GitHub are a great place to start!<\/p>\n<p>Additionally, following the previous .NET monthly themes of F#, Razor, and IoT, October will be focused on machine learning! The team is currently planning out lots of machine learning and ML.NET content and is looking forward to working with the community on this.<\/p>\n<h3>Afraid Microsoft will abandon the framework<\/h3>\n<p>ML.NET is .NET, and to make it feel more a part of .NET, we&#8217;ve decided to align with the .NET release schedule. This means that we will ship our next version of ML.NET (v1.7.0) with .NET 6.0 in November 2021 and will ship subsequent major releases (ML.NET 2.0, 3.0, etc.) with major releases of .NET. We will ship production-ready preview version releases in between so that we can continue adding new features to the framework throughout the year.<\/p>\n<p>We are also taking steps to organize the <a href=\"https:\/\/github.com\/dotnet\/machinelearning\" target=\"_blank\" rel=\"noopener\">dotnet\/machinelearning<\/a> repo and keep it up to date. We are currently revising our triage processes so that we can address your issues and feedback faster. Issues will be linked to version releases in the <a href=\"https:\/\/github.com\/dotnet\/machinelearning\/projects\">Projects<\/a> section of the repo so you can see what we&#8217;re actively working on and when we plan to release.<\/p>\n<p>Check out the <a href=\"https:\/\/github.com\/dotnet\/machinelearning\/blob\/main\/ROADMAP.md\" target=\"_blank\" rel=\"noopener\">roadmap<\/a> to see what we have planned for ML.NET this year.<\/p>\n<h3>Lack of \/ quality docs and samples<\/h3>\n<p>We have invested more resources into content development to make sure our Docs stay up to date and that we add documentation for new features faster as well as add more relevant samples.<\/p>\n<p><a href=\"https:\/\/github.com\/luisquintanilla\" target=\"_blank\" rel=\"noopener\">@luisquintanilla<\/a> (Microsoft content developer) and <a href=\"https:\/\/github.com\/jwood803\" target=\"_blank\" rel=\"noopener\">@jwood803<\/a> (ML.NET community member and newly contracted docs\/samples developer) have both been working hard to ensure that we increase the quality of ML.NET documentation. They have set several goals, including reducing the average days to close Docs issues and publishing documentation for new features no more than two weeks after a new feature is released.<\/p>\n<p>In the past two months, 19 articles have been updated, and a new <a href=\"https:\/\/docs.microsoft.com\/dotnet\/machine-learning\/how-to-guides\/label-images-for-object-detection-using-vott\" target=\"_blank\" rel=\"noopener\">article<\/a> on how to label images for object detection has been added. This month, the team is working on adding two new tutorials for image classification and recommendation in Model Builder to Docs as well as updating the samples to the newest version of ML.NET.<\/p>\n<p>You can file issues and make suggestions for ML.NET documentation in the <a href=\"https:\/\/github.com\/dotnet\/docs\" target=\"_blank\" rel=\"noopener\">dotnet\/docs<\/a> repo and for ML.NET samples in the <a href=\"https:\/\/github.com\/dotnet\/machinelearning-samples\" target=\"_blank\" rel=\"noopener\">dotnet\/machinelearning-samples<\/a> repo.<\/p>\n<h3>Lack of deep learning support<\/h3>\n<p>This past year we&#8217;ve been working on our plan for deep learning in .NET, and now we are ready to execute that plan to expand ML.NET&#8217;s deep learning support.<\/p>\n<p>As part of this plan, we will:<\/p>\n<ul>\n<li>Make it easier to consume ONNX models in ML.NET using the ONNX Runtime (RT)<\/li>\n<li>Fully support and productionize <a href=\"https:\/\/github.com\/xamarin\/TorchSharp\" target=\"_blank\" rel=\"noopener\">TorchSharp<\/a> for building neural networks in .NET<\/li>\n<li>Build a bridge between TorchSharp and ML.NET<\/li>\n<\/ul>\n<p>Read more about the deep learning plan and leave your feedback in this <a href=\"https:\/\/github.com\/dotnet\/machinelearning\/issues\/5918\" target=\"_blank\" rel=\"noopener\">tracking issue<\/a>.<\/p>\n<h3>Specific ML scenario or algorithm not supported by ML.NET<\/h3>\n<p>We have added Named Entity Recognition to the <a href=\"https:\/\/github.com\/dotnet\/machinelearning\/blob\/main\/ROADMAP.md\" target=\"_blank\" rel=\"noopener\">roadmap<\/a> which has been a highly requested scenario since ML.NET was released.<\/p>\n<p>The deep learning plan will also enable a variety of other scenarios so that you can train custom models for object detection, NLP tasks, and more in .NET.<\/p>\n<p>If there is still a scenario or algorithm missing that is not covered in the roadmap, please let us know by <a href=\"https:\/\/github.com\/dotnet\/machinelearning\/issues\/new?assignees=&amp;labels=&amp;template=suggest-a-feature.md&amp;title=\" target=\"_blank\" rel=\"noopener\">filing an issue<\/a>.<\/p>\n<h2>Get started and resources<\/h2>\n<p>Learn more about ML.NET and Model Builder in <a href=\"https:\/\/aka.ms\/mlnet-docs\" target=\"_blank\" rel=\"noopener\">Microsoft Docs<\/a>.<\/p>\n<p>If you run into any issues, feature requests, or feedback, please file an issue in the <a href=\"https:\/\/github.com\/dotnet\/machinelearning\" target=\"_blank\" rel=\"noopener\">ML.NET API repo<\/a> or the <a href=\"https:\/\/github.com\/dotnet\/machinelearning-modelbuilder\" target=\"_blank\" rel=\"noopener\">ML.NET Tooling (Model Builder &amp; ML.NET CLI) repo<\/a> on GitHub.<\/p>\n<p>Join the <a href=\"https:\/\/aka.ms\/virtual-mlnet-community-discord\" target=\"_blank\" rel=\"noopener\">ML.NET Community Discord<\/a>.<\/p>\n<p>Tune in to the <a href=\"https:\/\/dotnet.microsoft.com\/live\/community-standup\" target=\"_blank\" rel=\"noopener\">Machine Learning .NET Community Standup<\/a> every other Wednesday at 10am Pacific Time.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Check out what&#8217;s new in  ML.NET and Model Builder. <\/p>\n","protected":false},"author":721,"featured_media":34824,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[685,688,691],"tags":[7189,7188],"class_list":["post-34823","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-dotnet","category-machine-learning","category-ml-dotnet","tag-machinelearning","tag-mlnet"],"acf":[],"blog_post_summary":"<p>Check out what&#8217;s new in  ML.NET and Model Builder. <\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts\/34823","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/users\/721"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/comments?post=34823"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts\/34823\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/media\/34824"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/media?parent=34823"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/categories?post=34823"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/tags?post=34823"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}