{"id":38941,"date":"2022-02-28T11:31:36","date_gmt":"2022-02-28T18:31:36","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/dotnet\/?p=38941"},"modified":"2022-02-28T11:31:36","modified_gmt":"2022-02-28T18:31:36","slug":"training-a-ml-dotnet-model-with-azure-ml","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/dotnet\/training-a-ml-dotnet-model-with-azure-ml\/","title":{"rendered":"Training a ML.NET Model with Azure ML"},"content":{"rendered":"

Model Builder makes it easy to get started with Machine Learning and create your first model. As you gather more data over time, you may want to continuously refine or retrain your model. Using a combination of CLI and Azure tooling, you can train a new ML.NET model and integrate the training into a pipeline. This blog post shows an example of a training pipeline that can be easily rerun using Azure. <\/p>\n

We\u2019re going to use Azure Machine Learning Datasets<\/a> to track data and an Azure ML Pipeline to train a new model. This retraining pipeline can then be triggered by Azure DevOps.\nIn this post, we will cover: <\/p>\n

    \n
  1. Creating an Azure Machine Learning Dataset<\/li>\n
  2. Training a ML.NET model via the Azure Machine Learning CLI (v2)<\/li>\n
  3. Creating a pipeline in Azure DevOps for re-training<\/li>\n<\/ol>\n

    Prerequisites<\/h2>\n
      \n
    1. Azure Machine Learning Workspace<\/a><\/li>\n
    2. Compute Cluster<\/a> in the workspace<\/li>\n<\/ol>\n

      Creating an Azure Machine Learning Dataset<\/h2>\n
        \n
      1. \n

        Open the workspace in the Microsoft Azure Machine Learning Studio<\/a>. <\/p>\n<\/li>\n

      2. \n

        We need to create a file dataset. Navigate to Datasets<\/strong>. Click + Create Dataset<\/strong>. <\/p>\n<\/li>\n

      3. \n

        Choose the datasource. We will upload a copy of this song popularity dataset<\/a> available from Kaggle. It\u2019s a fairly large dataset that I don\u2019t want to maintain locally. <\/p>\n<\/li>\n

      4. \n

        Give the dataset a unique name and make sure to choose “File\u201d as the Dataset type.\n\"Screenshot<\/p>\n<\/li>\n

      5. \n

        Upload from a local file to the default workspaceblobstore<\/em>. Take note of the file name.\n\"Screenshot<\/p>\n<\/li>\n

      6. \n

        When the data upload finishes, create the dataset. <\/p>\n<\/li>\n

      7. \n

        Click on the completed dataset to view it. Confirm the preview available in the Explore<\/strong> tab looks correct. <\/p>\n<\/li>\n

      8. \n

        Make note of the dataset name, file name, and if you uploaded multiple versions, the version number. We will use these values in the next step. <\/p>\n<\/li>\n<\/ol>\n

        Training a ML.NET model via Azure Machine Learning<\/h2>\n

        Now that we have a dataset uploaded to Azure ML we can create an Azure ML training pipeline, and use Azure CLI v2 to run it. The pipeline below will create a Docker container with a ML.NET CLI instance that will conduct the training.<\/p>\n

          \n
        1. \n

          Create the Dockerfile and save it in a new folder for this experiment. If not familiar with Dockerfiles, these file types don’t have an extension. The file should be called “Dockerfile” with no extension, and contain the following: <\/p>\n

          FROM mcr.microsoft.com\/dotnet\/sdk:6.0\r\nRUN dotnet tool install -g microsoft.mlnet-linux-x64\r\nENV PATH=\"$PATH:\/root\/.dotnet\/tools\"<\/code><\/pre>\n<\/li>\n
        2. \n
          We will need to figure out our ML.NET CLI command to train our model. If needed, see installation instructions for the ML.NET CLI<\/a>.\n\"ML.NET<\/p>\n
          We\u2019re doing regression and will specify a dataset and label column. Text classification and recommendation are also supported for tabular files. Check the command information or ML.NET CLI docs<\/a> for more details on other training scenarios. <\/p>\n
          Make sure to include the option --verbosity q<\/code>, as some of the CLI features can cause problems in the Linux environment. <\/p>\n
          mlnet regression --dataset <YOUR_DATA_FILE_NAME> --label-col <YOUR_LABEL> --output outputs --log-file-path outputs\/logs --verbosity q<\/code><\/p>\n<\/li>\n
        3. \n
          Create the AzureTrain.yml file in the same folder as the Dockerfile. This is what will be passed to the Azure CLI.  By using input data in the pipeline, Azure ML will download the file dataset to our compute. The training file can then be referenced directly. We just need to specify the path in the command to the ML.NET CLI.  Do the following:<\/p>\n