{"id":34821,"date":"2018-01-04T10:08:59","date_gmt":"2018-01-04T18:08:59","guid":{"rendered":"https:\/\/blog.xamarin.com\/?p=34821"},"modified":"2019-04-01T18:31:09","modified_gmt":"2019-04-02T01:31:09","slug":"coreml-programming-xamarin-mac-f","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/xamarin\/coreml-programming-xamarin-mac-f\/","title":{"rendered":"CoreML Programming with Xamarin.Mac and F#"},"content":{"rendered":"

Although Xamarin technologies are most commonly used to develop mobile applications, Xamarin.Mac makes it easy to use your preferred .NET language to develop desktop Mac apps. There are many C# examples in Xamarin\u2019s mac-samples<\/a> directory on Github, but it’s also easy to use F# to develop desktop Mac apps as well.<\/p>\n

Currently, there is no support for Storyboards when working with F# and Xamarin.Mac, so this leaves you with two options: develop your UI in code or manually add the links between the Storyboard XML and your F# code. Both are straightforward, but in this example we’ll build the UI in code.<\/p>\n

The following code shows a simple custom NSView<\/code> that the app uses to set the values of the input parameters:<\/p>\n

\r\n[] \r\ntype InputView (identifier : string, items : int list) as this =   \r\n    inherit NSView ()  \r\n  \r\n    let updated = new Event()  \r\n    let spinner = new NSPopUpButton()  \r\n  \r\n    do  \r\n      let label = new NSText()  \r\n      label.Value <- identifier  \r\n      label.Editable <- false  \r\n \r\n      this.AddSubview label  \r\n      this.AddSubview spinner  \r\n  \r\n      spinner.Target <- this  \r\n      spinner.Action  List.map (fun i -> i.ToString())  \r\n      |> List.toArray  \r\n      |> spinner.AddItems  \r\n  \r\n      label.Frame <- new CGRect(10., 10., 100., 50.)  \r\n      spinner.Frame <- new CGRect(10., 70., 100., 50.)  \r\n  \r\n    [CLIEvent] \r\n    member this.Updated = updated.Publish  \r\n  \r\n    []  \r\n    member this.SpinnerValueSelected =   \r\n       spinner.TitleOfSelectedItem  \r\n       |> float  \r\n       |> updated.Trigger  \r\n<\/code><\/pre>\n
The class is typical of those that bridge the worlds of F# and the Mac. The AllowNullLiteralAttribute<\/code> allows null references to the type, while CLIEventAttribute<\/code> and OutletAttribute<\/code> are used to map a native message to a .NET event. In this case, the assignments to spinner.Target<\/code> and spinner.Action<\/code> specify that the system should call the method associated with the Selector<\/code> \u201cSpinnerValueSelected\u201d when the spinner\u2019s value changes. That association is created with the OutletAttribute<\/code> applied to this.SpinnerValueSelected<\/code>. The code retrieves the text value, converts it to a floating point value, and triggers the Updated<\/code> event.<\/p>\n
Programming for the Mac will be very familiar to those who are accustomed to programming for iOS: you have an entry point that creates an NSApplicationDelegate<\/code> which has one or more NSViewController<\/code> objects that, in turn, have some number of NSView<\/code> objects. However, it isn’t quite<\/em> as simple as swapping NS<\/code> for UI<\/code> as the prefix for your objects! MacOS has a longer history than iOS and there are more legacy approaches and quirks than there are in iOS. For instance, NSView<\/code> objects are not necessarily backed by CoreAnimation<\/code> layers, and instead of NSViewController<\/code> objects, you may work with NSWindowController<\/code> objects.<\/p>\n
Using CoreML with F#<\/h2>\n
The app has three InputView<\/code> objects that specify the number of solar panels, greenhouses, and acreage of a property for sale on Mars. The app uses a (very simple) CoreML model file to estimate the price of such properties. CoreML is a framework introduced in MacOS High Sierra and iOS 11 for inferencing (making calculations) with machine learning models trained using other tools. At this point, you can neither train nor access the internals of a CoreML model.<\/p>\n
To use a machine learning model with CoreML, the first step is to convert the model from the native training library to a CoreML .mlmodel file. This is done using Apple\u2019s CoreML Tools<\/a> or an open-source extension. The second step is to \u201ccompile\u201d the .mlmodel using xcrun coremlcompiler compile model.mlmodel outputfolder<\/code>. For a detailed walkthrough of training and converting an ML model to CoreML and Tensorflow Android Inference, see my MSDN Magazine article<\/a>.<\/p>\n
A compiled CoreML model results in a folder, e.g., MarsHabitatPricer.mlmodel<\/code> that contains binary definitions of the model structure and weights. You may<\/em> load that model from the Web (and that would be fine for a small model such as this), but you generally load it as a resource with code, such as:<\/p>\n
let LoadCoreMLModel modelName =   \r\n      let assetPath = NSBundle.MainBundle.GetUrlForResource(modelName, \"mlmodelc\")  \r\n      let (model, err) = MLModel.Create(assetPath)  \r\n      match err  null with   \r\n      | true -> raise  model  \r\n<\/code><\/pre>\n
You\u2019d want to return an Option<\/code> or a Maybe<\/code> in a robust app, but, for learning purposes, we\u2019ll just throw an Exception<\/code> if there\u2019s a problem.<\/p>\n
CoreML apparently has its own internal datatypes for representing values, so instead of directly passing the input values, you need to wrap them in an IMLFeatureProvider<\/code> instance that provides a mapping from strings to values. In this case, we have three features: the number of solar panels, the number of greenhouses, and the acreage. Those values are stored in mutable variables and, when any one of them is modified, we trigger an InputsChanged<\/code> event:<\/p>\n
type MarsHabitatPricerInput ()  =   \r\n    inherit NSObject()  \r\n  \r\n    let mutable solarPanels = 1.   \r\n    let mutable greenhouses = 1.  \r\n    let mutable plotsize = 1000.  \r\n  \r\n    let inputsChanged = new Event()  \r\n  \r\n    []  \r\n    member this.InputsChanged = inputsChanged.Publish  \r\n  \r\n    member this.SolarPanelCount f  = solarPanels <- f ; inputsChanged.Trigger(this)   \r\n    member this.GreenhousesCount f = greenhouses <- f; inputsChanged.Trigger(this)   \r\n    member this.Plotsize f = plotsize  List.map (fun s -> new NSString())  \r\n             |> Array.ofList  \r\n             |> fun ss -> new NSSet(ss)  \r\n  \r\n        member this.GetFeatureValue featureName =   \r\n            match featureName with   \r\n            | \"solarPanels\" -> MLFeatureValue.Create(solarPanels)  \r\n            | \"greenhouses\" -> MLFeatureValue.Create(greenhouses)  \r\n            | \"size\" -> MLFeatureValue.Create(plotsize)  \r\n            | _ -> raise <| new ArgumentOutOfRangeException(\"featureName\")  \r\n  \r\n<\/code><\/pre>\n
As you probably figured out, we subscribe an instance of this MarsHabitatPricerInput<\/code> type to each of the InputView<\/code> objects’ Updated<\/code> events, so changing the input values leads to triggering the MarsHabitatPricerInput.InputsChanged<\/code> event. That, in turn, leads us to:<\/p>\n
this.PriceInput.InputsChanged.Add (fun featureProvider ->   \r\n            let (prediction, err) = this.Model.GetPrediction(featureProvider)  \r\n            match err  null with   \r\n            | true -> raise  this.UpdatePrediction(prediction.GetFeatureValue(\"price\").DoubleValue)  \r\n        )  \r\n<\/code><\/pre>\n
We pass the MarsHabitatPricerInput<\/code> object to the MLModel<\/code> object\u2019s GetPrediction<\/code> method. Again, we raise an exception on an error and otherwise retrieve the model\u2019s \u201cprice\u201d output, which we know to be a Double<\/code>. The details of the model\u2019s input and output strings and datatypes come from the training-and-conversion process.<\/p>\n
The final application, available on Github<\/a>, looks like this:<\/p>\n
\"\"<\/a><\/p>\n
More Machine Learning Examples from Xamarin<\/h2>\n
Xamarin allows developers to deploy machine learning apps across desktops and mobile devices. You can learn more about developing programs based on CoreML by reading our Intro to CoreML<\/a> article. When you’re ready to go deeper into developing apps that use both CoreML and TensorFlow Android Inference, see my article Deliver On-Device Machine Learning Solutions<\/a>.<\/p>\n
Discuss this post in the forums!<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"
Although Xamarin technologies are most commonly used to develop mobile applications, Xamarin.Mac makes it easy to use your preferred .NET language to develop desktop Mac apps. There are many C# examples in Xamarin\u2019s mac-samples directory on Github, but it’s also easy to use F# to develop desktop Mac apps as well.<\/p>\n","protected":false},"author":556,"featured_media":42348,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[2,362],"tags":[713,3617,3616,4],"class_list":["post-34821","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-developers","category-macos","tag-f","tag-machine-learning","tag-macos","tag-xamarin-platform"],"acf":[],"blog_post_summary":"
Although Xamarin technologies are most commonly used to develop mobile applications, Xamarin.Mac makes it easy to use your preferred .NET language to develop desktop Mac apps. There are many C# examples in Xamarin\u2019s mac-samples directory on Github, but it’s also easy to use F# to develop desktop Mac apps as well.<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/xamarin\/wp-json\/wp\/v2\/posts\/34821","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/xamarin\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/xamarin\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/xamarin\/wp-json\/wp\/v2\/users\/556"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/xamarin\/wp-json\/wp\/v2\/comments?post=34821"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/xamarin\/wp-json\/wp\/v2\/posts\/34821\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/xamarin\/wp-json\/wp\/v2\/media\/42348"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/xamarin\/wp-json\/wp\/v2\/media?parent=34821"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/xamarin\/wp-json\/wp\/v2\/categories?post=34821"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/xamarin\/wp-json\/wp\/v2\/tags?post=34821"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}