One of the most difficult tasks of building machine learning applications is deploying them to production. The ML.NET team is exploring ways to simplify the process and would like to hear your feedback.<\/p>\n
When it comes to deploying machine learning models as web services, the bare minimum you need is a single endpoint to handle making predictions. One way to do that in .NET is using a technique known as “route-to-code.” In this post, I’ll show how “route-to-code” can help you quickly build highly scalable machine learning web services in about 60 lines of code!<\/p>\n
For more information on “route-to-code”<\/a>, see the ASP.NET documentation.<\/p>\n For web applications, it’s recommended to use the If you’re using the CLI, you can use the following command to install the NuGet package.<\/p>\n A standard ASP.NET convention is to configure your application’s services and request pipelines in a class called This code defines and builds the application’s web host. It also registers a Machine learning models use patterns learned from the training process to generate predictions using new data as input. The machine learning model used in this sample analyzes sentiment from input text and categorizes it as positive or negative.<\/p>\n Define both of these classes in your application.<\/p>\n The To process incoming requests, you’ll want to create a handler. The handler is a method that leverages the Now that you have a handler, configure your application to route requests to your handler. Add the following code inside the This code maps HTTP POST requests to the Your final That’s all you need to serve your machine learning model as an HTTP API!<\/p>\n To test your API, run the application and make an HTTP POST request with a JSON body containing the You should receive an The In this post, we showed how “route-to-code” can help you quickly write a highly scalable machine learning ASP.NET web service. Try deploying your own machine learning models using the minimal “route-to-code” method and give us feedback<\/a> on how to make it better.<\/p>\n You can find a complete version of this application in the ML.NET HTTP API GitHub repository<\/a>. In that repository you’ll also find a sample<\/a> that uses the API Endpoints NuGet package to enable model binding and OpenAPI\/Swagger.<\/p>\n","protected":false},"excerpt":{"rendered":" Learn how to serve machine learning models from HTTP APIs using minimal configuration<\/p>\n","protected":false},"author":26108,"featured_media":32051,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[685,691],"tags":[4,93,96],"class_list":["post-32010","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-dotnet","category-ml-dotnet","tag-net","tag-machine-learning","tag-ml-net"],"acf":[],"blog_post_summary":" Learn how to serve machine learning models from HTTP APIs using minimal configuration<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts\/32010","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/users\/26108"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/comments?post=32010"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts\/32010\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/media\/32051"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/media?parent=32010"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/categories?post=32010"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/tags?post=32010"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}Install the Microsoft.Extensions.ML NuGet package<\/h2>\n
PredictionEnginePool<\/code> service. This is a scalable service which provides an ObjectPool<\/code> containing PredictionEngine<\/code> objects that use an ML.NET model to make predictions on new data. The PredictionEnginePool<\/code> service is part of the Microsoft.Extensions.ML<\/code> NuGet package<\/a>.<\/p>\ndotnet add package Microsoft.Extensions.ML\r\n<\/code><\/pre>\nConfigure the web application<\/h3>\n
Startup<\/code>. Because we’re working with a single service and endpoint, we’ll instead configure our application inside the Program<\/code> class. Add the following code to your Main<\/code> method.<\/p>\nWebHost.CreateDefaultBuilder()\r\n .ConfigureServices(services => {\r\n \/\/ Register PredictionEnginePool service \r\n services.AddPredictionEnginePool<Input,Output>()\r\n .FromUri(\"https:\/\/github.com\/dotnet\/samples\/raw\/master\/machine-learning\/models\/sentimentanalysis\/sentiment_model.zip\");\r\n })\r\n .Configure(app => {\r\n app.UseHttpsRedirection();\r\n })\r\n .Build()\r\n .Run();\r\n<\/code><\/pre>\nPredictionEnginePool<\/code> for a model hosted on GitHub. Once registered you can use this service anywhere in your application using dependency injection.<\/p>\nDefine model input and output schemas<\/h3>\n
public class Input\r\n{\r\n public string SentimentText;\r\n\r\n [ColumnName(\"Label\")]\r\n public bool Sentiment;\r\n}\r\n\r\npublic class Output\r\n{\r\n [ColumnName(\"PredictedLabel\")]\r\n public bool Prediction { get; set; }\r\n\r\n public float Probability { get; set; }\r\n\r\n public float Score { get; set; }\r\n}\r\n<\/code><\/pre>\nInput<\/code> and Output<\/code> classes in this case define the schema of the model’s input and output respectively. Given an input value in the SentimentText<\/code> property, the model outputs a boolean value for the Prediction<\/code>, where zero is negative and one is positive sentiment.<\/p>\nCreate a handler to make predictions<\/h3>\n
HttpContext<\/code> to access registered services (in this case PredictionEnginePool<\/code>), read the request, and write out a response.<\/p>\nstatic async Task PredictHandler(HttpContext http)\r\n{\r\n \/\/ Get PredictionEnginePool service\r\n var predEngine = http.RequestServices.GetRequiredService<PredictionEnginePool<Input,Output>>();\r\n\r\n \/\/ Deserialize HTTP request JSON body\r\n var input = await JsonSerializer.DeserializeAsync<Input>(http.Request.Body);\r\n\r\n \/\/ Predict using PredictionEnginePool service\r\n var prediction = predEngine.Predict(input);\r\n\r\n \/\/ Return prediction as JSON response\r\n await http.Response.WriteAsJsonAsync(prediction);\r\n}\r\n<\/code><\/pre>\nConfigure routes<\/h3>\n
Configure<\/code> method.<\/p>\napp.UseRouting();\r\napp.UseEndpoints(endpoints => {\r\n \/\/ Define prediction endpoint\r\n endpoints.MapPost(\"\/predict\", PredictHandler);\r\n});\r\n<\/code><\/pre>\npredict<\/code> endpoint and uses the PredictHandler<\/code> previously created to process those requests.<\/p>\nMain<\/code> method should look like the following:<\/p>\nWebHost.CreateDefaultBuilder()\r\n .ConfigureServices(services => {\r\n \/\/ Register PredictionEnginePool service \r\n services.AddPredictionEnginePool<Input,Output>()\r\n .FromUri(\"https:\/\/github.com\/dotnet\/samples\/raw\/master\/machine-learning\/models\/sentimentanalysis\/sentiment_model.zip\");\r\n })\r\n .Configure(app => {\r\n app.UseHttpsRedirection();\r\n app.UseRouting();\r\n app.UseEndpoints(endpoints => {\r\n \/\/ Define prediction endpoint\r\n endpoints.MapPost(\"\/predict\", PredictHandler);\r\n });\r\n })\r\n .Build()\r\n .Run(); \r\n<\/code><\/pre>\nTest your API<\/h2>\n
SentimentText<\/code> property.<\/p>\n{\r\n \"SentimentText\": \"This is a very bad steak\"\r\n}\r\n<\/code><\/pre>\nOutput<\/code> response similar to the following:<\/p>\n{\r\n \"prediction\": false,\r\n \"probability\": 0.5,\r\n \"score\": 0\r\n}\r\n<\/code><\/pre>\nfalse<\/code> value in prediction<\/code> indicates that the SentimentText<\/code> provided in the request is negative.<\/p>\nConclusion<\/h2>\n