CoreML Programming with Xamarin.Mac and F#
Although Xamarin technologies are most commonly used to develop mobile applications, Xamarin.Mac makes it easy to use your preferred .NET language to develop desktop Mac apps. There are many C# examples in Xamarin’s mac-samples directory on Github, but it’s also easy to use F# to develop desktop Mac apps as well.
Currently, there is no support for Storyboards when working with F# and Xamarin.Mac, so this leaves you with two options: develop your UI in code or manually add the links between the Storyboard XML and your F# code. Both are straightforward, but in this example we’ll build the UI in code.
The following code shows a simple custom
NSView that the app uses to set the values of the input parameters:
type InputView (identifier : string, items : int list) as this =
inherit NSView ()
let updated = new Event()
let spinner = new NSPopUpButton()
let label = new NSText()
label.Value <- identifier
label.Editable <- false
spinner.Target <- this
spinner.Action List.map (fun i -> i.ToString())
label.Frame <- new CGRect(10., 10., 100., 50.)
spinner.Frame <- new CGRect(10., 70., 100., 50.)
member this.Updated = updated.Publish
member this.SpinnerValueSelected =
The class is typical of those that bridge the worlds of F# and the Mac. The
AllowNullLiteralAttribute allows null references to the type, while
OutletAttribute are used to map a native message to a .NET event. In this case, the assignments to
spinner.Action specify that the system should call the method associated with the
Selector “SpinnerValueSelected” when the spinner’s value changes. That association is created with the
OutletAttribute applied to
this.SpinnerValueSelected. The code retrieves the text value, converts it to a floating point value, and triggers the
Programming for the Mac will be very familiar to those who are accustomed to programming for iOS: you have an entry point that creates an
NSApplicationDelegate which has one or more
NSViewController objects that, in turn, have some number of
NSView objects. However, it isn’t quite as simple as swapping
UI as the prefix for your objects! MacOS has a longer history than iOS and there are more legacy approaches and quirks than there are in iOS. For instance,
NSView objects are not necessarily backed by
CoreAnimation layers, and instead of
NSViewController objects, you may work with
Using CoreML with F#
The app has three
InputView objects that specify the number of solar panels, greenhouses, and acreage of a property for sale on Mars. The app uses a (very simple) CoreML model file to estimate the price of such properties. CoreML is a framework introduced in MacOS High Sierra and iOS 11 for inferencing (making calculations) with machine learning models trained using other tools. At this point, you can neither train nor access the internals of a CoreML model.
To use a machine learning model with CoreML, the first step is to convert the model from the native training library to a CoreML .mlmodel file. This is done using Apple’s CoreML Tools or an open-source extension. The second step is to “compile” the .mlmodel using
xcrun coremlcompiler compile model.mlmodel outputfolder. For a detailed walkthrough of training and converting an ML model to CoreML and Tensorflow Android Inference, see my MSDN Magazine article.
A compiled CoreML model results in a folder, e.g.,
MarsHabitatPricer.mlmodel that contains binary definitions of the model structure and weights. You may load that model from the Web (and that would be fine for a small model such as this), but you generally load it as a resource with code, such as:
let LoadCoreMLModel modelName =
let assetPath = NSBundle.MainBundle.GetUrlForResource(modelName, "mlmodelc")
let (model, err) = MLModel.Create(assetPath)
match err null with
| true -> raise model
You’d want to return an
Option or a
Maybe in a robust app, but, for learning purposes, we’ll just throw an
Exception if there’s a problem.
CoreML apparently has its own internal datatypes for representing values, so instead of directly passing the input values, you need to wrap them in an
IMLFeatureProvider instance that provides a mapping from strings to values. In this case, we have three features: the number of solar panels, the number of greenhouses, and the acreage. Those values are stored in mutable variables and, when any one of them is modified, we trigger an
type MarsHabitatPricerInput () =
let mutable solarPanels = 1.
let mutable greenhouses = 1.
let mutable plotsize = 1000.
let inputsChanged = new Event()
member this.InputsChanged = inputsChanged.Publish
member this.SolarPanelCount f = solarPanels <- f ; inputsChanged.Trigger(this)
member this.GreenhousesCount f = greenhouses <- f; inputsChanged.Trigger(this)
member this.Plotsize f = plotsize List.map (fun s -> new NSString())
|> fun ss -> new NSSet(ss)
member this.GetFeatureValue featureName =
match featureName with
| "solarPanels" -> MLFeatureValue.Create(solarPanels)
| "greenhouses" -> MLFeatureValue.Create(greenhouses)
| "size" -> MLFeatureValue.Create(plotsize)
| _ -> raise <| new ArgumentOutOfRangeException("featureName")
As you probably figured out, we subscribe an instance of this
MarsHabitatPricerInput type to each of the
Updated events, so changing the input values leads to triggering the
MarsHabitatPricerInput.InputsChanged event. That, in turn, leads us to:
this.PriceInput.InputsChanged.Add (fun featureProvider ->
let (prediction, err) = this.Model.GetPrediction(featureProvider)
match err null with
| true -> raise this.UpdatePrediction(prediction.GetFeatureValue("price").DoubleValue)
We pass the
MarsHabitatPricerInput object to the
GetPrediction method. Again, we raise an exception on an error and otherwise retrieve the model’s “price” output, which we know to be a
Double. The details of the model’s input and output strings and datatypes come from the training-and-conversion process.
The final application, available on Github, looks like this:
More Machine Learning Examples from Xamarin
Xamarin allows developers to deploy machine learning apps across desktops and mobile devices. You can learn more about developing programs based on CoreML by reading our Intro to CoreML article. When you’re ready to go deeper into developing apps that use both CoreML and TensorFlow Android Inference, see my article Deliver On-Device Machine Learning Solutions.