{"id":45765,"date":"2023-05-18T10:05:00","date_gmt":"2023-05-18T17:05:00","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/dotnet\/?p=45765"},"modified":"2024-12-13T14:15:49","modified_gmt":"2024-12-13T22:15:49","slug":"object-detection-ml-dotnet-model-builder","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/dotnet\/object-detection-ml-dotnet-model-builder\/","title":{"rendered":"Object Detection in ML.NET Model Builder"},"content":{"rendered":"<p><a href=\"https:\/\/dot.net\/ml\">ML.NET<\/a> is an open-source, cross-platform machine learning framework for .NET developers that enables integration of custom machine learning models into .NET apps.<\/p>\n<p>A new version of Model Builder is now available which includes support for object detection using your local CPU or GPU. <\/p>\n<p>Download or update to the latest version of <a href=\"https:\/\/marketplace.visualstudio.com\/items?itemName=MLNET.ModelBuilder2022\">Model Builder<\/a> and try it out!<\/p>\n<h2>What is object detection?<\/h2>\n<p>Object detection is a computer vision problem. While closely related to image classification, object detection performs image classification at a more granular scale. Object detection both locates and categorizes entities within images. Use object detection when images contain multiple objects of different types.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2023\/05\/dog-object-detection-2.png\" alt=\"Picture of pug with bounding box\" \/><\/p>\n<p>Some use cases for object detection include:<\/p>\n<ul>\n<li>Workplace safety<\/li>\n<li>Object counting<\/li>\n<li>Activity recognition<\/li>\n<li>Robotics<\/li>\n<li>Self-driving cars<\/li>\n<\/ul>\n<h2>Object detection in Model Builder<\/h2>\n<p>We&#8217;re excited to announce you can now train object detection models in Model Builder using your local CPU or GPU.<\/p>\n<p>The local object detection scenario in Model Builder is powered by the Object Detection API in ML.NET.<\/p>\n<p>Similar to other deep learning APIs in ML.NET like <a href=\"https:\/\/devblogs.microsoft.com\/dotnet\/introducing-the-ml-dotnet-text-classification-api-preview\/\">Text Classification<\/a> and <a href=\"https:\/\/devblogs.microsoft.com\/dotnet\/sentence-similarity-mlnet-model-builder\/\">Sentence Similarity<\/a>, the Object Detection API is a high-level abstraction where you only need to provide your data and a few parameters to help guide the model training process. <\/p>\n<p>Under the covers, the Object Detection API leverages some of the latest techniques from Microsoft Research and is backed by a Transformer-based neural network architecture built with TorchSharp. For more details on the underlying model, see the <a href=\"https:\/\/arxiv.org\/pdf\/2111.14725.pdf\">Searching the Space of Vision Transformer<\/a> paper. <\/p>\n<p>We&#8217;ve made the process of getting started with the Object Detection API even easier by integrating it into tools like Model Builder.<\/p>\n<p>If you don&#8217;t have a GPU or need more powerful compute than the one available on your device, you also have the option of using Azure.<\/p>\n<h2>Get started with object detection locally<\/h2>\n<p>Download or update to the latest version of <a href=\"https:\/\/marketplace.visualstudio.com\/items?itemName=MLNET.ModelBuilder2022\">Model Builder<\/a><\/p>\n<ol>\n<li>\n<p>Start Model Builder by right-clicking a .NET project and choose <strong>Add &gt; Machine Learning Model<\/strong><\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2023\/05\/add-ml-model.png\" alt=\"Model Builder add ML model\" \/><\/p>\n<\/li>\n<li>\n<p>Choose the <strong>Object Detection<\/strong> scenario.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2023\/05\/object-dection-scenario.png\" alt=\"Model Builder object detection scenario with red rectangle\" \/><\/p>\n<\/li>\n<li>\n<p>Choose one of the local environments or Azure. We strongly recommend using a GPU if you have one. For more details on using GPUs in Model Builder, see the <a href=\"https:\/\/learn.microsoft.com\/dotnet\/machine-learning\/how-to-guides\/install-gpu-model-builder\">Model Builder GPU guidance<\/a>.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2023\/05\/object-detection-environment.png\" alt=\"Model Builder object detection environment screen\" \/><\/p>\n<\/li>\n<li>\n<p>Add your data. Model Builder supports both the VoTT and COCO formats.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2023\/05\/object-detection-data.png\" alt=\"Model Builder object detection data screen\" \/><\/p>\n<\/li>\n<li>\n<p>Train your model. This may take a few minutes depending on how much data you have and the hardware you&#8217;re using to train your model.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2023\/05\/object-detection-training.png\" alt=\"Model Builder object detection train screen\" \/><\/p>\n<\/li>\n<li>\n<p>Try out a few images to see whether the model is working as expected.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2023\/05\/object-detection-eval.png\" alt=\"Model Builder object detection train screen\" \/><\/p>\n<\/li>\n<\/ol>\n<p>At this point, you can move to the next steps and use your model inside of your .NET applications.<\/p>\n<p>For more details, check out the <a href=\"https:\/\/learn.microsoft.com\/dotnet\/machine-learning\/tutorials\/object-detection-model-builder\">Model Builder object detection tutorial<\/a><\/p>\n<h2>Acknowledgements<\/h2>\n<p>We&#8217;d like to thank our Microsoft Research and TorchSharp partners for helping us deliver these new scenarios and capabilities in ML.NET.<\/p>\n<p>Thanks to Ambrosio Blanco, Yaoyao Chang, Niklas Gustaffson, Han Hu, Yangyu Huang, Hang Li, Qingtao Li, Zhe Liu, Houwen Peng, Chi Wang, Quifeng Yin, Yilei Yang, and many others.<\/p>\n<h2>Additional resources<\/h2>\n<p>Learn more about ML.NET, Model Builder, and the ML.NET CLI in <a href=\"https:\/\/learn.microsoft.com\/dotnet\/machine-learning\/\">Microsoft Docs<\/a>.<\/p>\n<p>If you run into any issues, feature requests, or feedback, please file an issue in the <a href=\"https:\/\/github.com\/dotnet\/machinelearning\/issues\">ML.NET<\/a> and <a href=\"https:\/\/github.com\/dotnet\/machinelearning-modelbuilder\/issues\">Model Builder<\/a> repos.<\/p>\n<p>Join the <a href=\"https:\/\/aka.ms\/virtual-mlnet-community-discord\">ML.NET Community Discord<\/a> or #machine-learning channel on the <a href=\"https:\/\/aka.ms\/dotnet-discord\">.NET Development Discord<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Object detection using local environments is now available in ML.NET Model Builder to help you solve computer vision problems using machine learning.<\/p>\n","protected":false},"author":26108,"featured_media":45766,"comment_status":"open","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[685,328,688,691],"tags":[7189,7188,568],"class_list":["post-45765","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-dotnet","category-aiml","category-machine-learning","category-ml-dotnet","tag-machinelearning","tag-mlnet","tag-ai"],"acf":[],"blog_post_summary":"<p>Object detection using local environments is now available in ML.NET Model Builder to help you solve computer vision problems using machine learning.<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts\/45765","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/users\/26108"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/comments?post=45765"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts\/45765\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/media\/45766"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/media?parent=45765"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/categories?post=45765"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/tags?post=45765"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}