{"id":44036,"date":"2019-06-24T11:02:55","date_gmt":"2019-06-24T18:02:55","guid":{"rendered":"http:\/\/devblogs.microsoft.com\/xamarin\/?p=44036"},"modified":"2019-08-30T08:28:53","modified_gmt":"2019-08-30T15:28:53","slug":"dex-counting-xamarin-android-improved","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/xamarin\/dex-counting-xamarin-android-improved\/","title":{"rendered":"Faster Xamarin.Android Builds &#038; Smaller Dex Files"},"content":{"rendered":"<p><span data-offset-key=\"8q78s-0-0\">One of our current focus areas in Xamarin.Android is build performance. <\/span><span class=\"hardreadability\"><span data-offset-key=\"8q78s-1-0\">The &#8220;inner dev loop&#8221; <\/span><\/span><span class=\"adverb\"><span data-offset-key=\"8q78s-2-0\">directly<\/span><\/span><span class=\"hardreadability\"><span data-offset-key=\"8q78s-3-0\"> impacts developer productivity&#8211;the time it takes to make a small code change and see the result on a device or emulator<\/span><\/span><span data-offset-key=\"8q78s-4-0\">. Reevaluating parts of Xamarin.Android&#8217;s codebase has been how we&#8217;ve been able to make progress.<\/span><\/p>\n<h3>Dex Limits<\/h3>\n<p>We have also noticed that Xamarin.Android developers hit the multidex limit often. If an application is mostly .NET code, the amount of Java code should be\u00a0<em>under\u00a0<\/em>the multidex limit. We had a suspicion there was more to the problem.<\/p>\n<h3>A Story About Aapt<\/h3>\n<p>Xamarin.Android runs several command-line tools to create a native Android application. One such tool is <a href=\"https:\/\/elinux.org\/Android_aapt\">aapt<\/a> (Android Asset Packaging Tool), which is used to process Android resource files and create Android <a href=\"https:\/\/en.wikipedia.org\/wiki\/Android_application_package\">APK files<\/a>. <code>aapt<\/code> also generates the identifiers that Android developers are familiar with, such as the <code>Resource<\/code>\u00a0class in C# or the <code>R<\/code>\u00a0class in Java.<\/p>\n<p><code>aapt<\/code> (or\u00a0<a href=\"https:\/\/developer.android.com\/studio\/command-line\/aapt2\">aapt2<\/a>) runs a few times during a typical Xamarin.Android build:<\/p>\n<ul>\n<li style=\"list-style-type: none;\">\n<ul>\n<li style=\"list-style-type: none;\">\n<ol>\n<li>Generate an <code>R.java<\/code>\u00a0file for each Java library.<\/li>\n<li>Create a <code>Resource.designer.cs<\/code>\u00a0file, so we can access the resources from C#.<\/li>\n<li>Produce an APK file that can be deployed to an emulator or device.<\/li>\n<\/ol>\n<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>As you might imagine, <code>aapt<\/code> takes a bit of time to run, so it became an area of investigation.<\/p>\n<h4>Hello World<\/h4>\n<p>We began looking at how Xamarin.Android invokes <code>aapt<\/code> in the case of Java libraries. If we look at the example of a &#8220;Hello World&#8221; Xamarin.Forms app, the <code>aapt<\/code> command-line looks something like this:<\/p>\n<pre class=\"lang:sh decode:true \">aapt package \r\n    -f -m \r\n    --non-constant-id \r\n    --auto-add-overlay \r\n    --max-res-version 28 \r\n    -M obj\\Debug\\lp\\1\\jl\\manifest\\AndroidManifest.xml \r\n    -J obj\\Debug\\android\\src \r\n    -S obj\\Debug\\res \r\n    -S obj\\Debug\\lp\\1\\jl\\res \r\n    -S obj\\Debug\\lp\\2\\jl\\res \r\n    -S obj\\Debug\\lp\\3\\jl\\res \r\n    -S obj\\Debug\\lp\\4\\jl\\res \r\n    -S obj\\Debug\\lp\\5\\jl\\res \r\n    -S obj\\Debug\\lp\\6\\jl\\res \r\n    -S obj\\Debug\\lp\\7\\jl\\res \r\n    -S obj\\Debug\\lp\\8\\jl\\res \r\n    -I \"C:\\Program Files (x86)\\Android\\android-sdk\\platforms\\android-28\\android.jar\"<\/pre>\n<p><em>NOTE: we have numbers &amp; small directory names to workaround the <a href=\"https:\/\/docs.microsoft.com\/en-us\/windows\/desktop\/fileio\/naming-a-file#maximum-path-length-limitation\">MAX_PATH limit<\/a> on Windows.<\/em><\/p>\n<h4>Android Support Libraries<\/h4>\n<p>This command produces an <code>R.java<\/code>\u00a0file for the first Java library used by the app, placing it in <code>obj\\Debug\\android\\src<\/code>\u00a0in a <a href=\"http:\/\/www.ntu.edu.sg\/home\/ehchua\/programming\/java\/j9c_packageclasspath.html\">Java directory structure<\/a>. Xamarin.Android runs eight such commands in parallel: one per library with a different <code>-M<\/code>\u00a0switch. The Java libraries in this particular project are the <a href=\"https:\/\/developer.android.com\/topic\/libraries\/support-library\">Android support libraries<\/a>.<\/p>\n<p>We end up with eight files on disk, for example:<\/p>\n<ul>\n<li style=\"list-style-type: none;\">\n<ul>\n<li style=\"list-style-type: none;\">\n<ul>\n<li><code>obj\\Debug\\android\\src\\android\\support\\library1\\R.java<\/code><\/li>\n<li><code>obj\\Debug\\android\\src\\android\\support\\library2\\R.java<\/code><\/li>\n<li><code>obj\\Debug\\android\\src\\android\\support\\library3\\R.java<\/code><\/li>\n<li><code>obj\\Debug\\android\\src\\android\\support\\library4\\R.java<\/code><\/li>\n<li><code>obj\\Debug\\android\\src\\android\\support\\library5\\R.java<\/code><\/li>\n<li><code>obj\\Debug\\android\\src\\android\\support\\library6\\R.java<\/code><\/li>\n<li><code>obj\\Debug\\android\\src\\android\\support\\library7\\R.java<\/code><\/li>\n<li><code>obj\\Debug\\android\\src\\android\\support\\library8\\R.java<\/code><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>The theory (or question) was if we could avoid running <code>aapt<\/code>\u00a0so many times to improve build times? But something else didn&#8217;t seem quite right&#8230; When comparing the <code>R.java<\/code>\u00a0files from a Xamarin project to a similar Java project in Android Studio, the <code>R.java<\/code>\u00a0files had quite a few more fields!<\/p>\n<p>Specifically, we compared the <code>android\\support\\compat\\R$drawable<\/code>\u00a0class:<\/p>\n<ul>\n<li style=\"list-style-type: none;\">\n<ul>\n<li style=\"list-style-type: none;\">\n<ul>\n<li>Android Studio\/Java: 11 fields<\/li>\n<li>Xamarin.Forms: 300+ fields<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>Comparing the APK as a whole, the Xamarin app had thousands more fields than the Java app!<\/p>\n<p>Thinking about the <code>aapt<\/code>\u00a0command, the command produced an <code>R.java<\/code>\u00a0file as if each library referenced <em>every other library<\/em>. We generated multiple\u00a0<code>R.java<\/code>\u00a0files containing every field in the application.<\/p>\n<p>The Java libraries, of course, do not need all of the fields,\u00a0<em>some\u00a0<\/em>of the Java libraries depend on one another. How can we figure out which ones?<\/p>\n<h3>Looking at Android Studio<\/h3>\n<p>Many parts of Android are open source, so we looked at Google&#8217;s implementation. Android Studio uses a file named\u00a0<code>R.txt<\/code>, an easy-to-parse, text-based, version of <code>R.java<\/code>\u00a0to declare identifiers needed by each Java library. Xamarin.Android already had access to this file in most cases, but our MSBuild targets were not using <code>R.txt<\/code>\u00a0yet.<\/p>\n<p>At a high level, Android Studio does the following:<\/p>\n<ul>\n<li style=\"list-style-type: none;\">\n<ul>\n<li style=\"list-style-type: none;\">\n<ul>\n<li><code>aapt<\/code>\u00a0is invoked to generate an <code>R.txt<\/code>\u00a0file for the entire Android app.<\/li>\n<li>An <code>R.txt<\/code>\u00a0file that is shipped alongside each library makes it possible to map the values back to the app&#8217;s <code>R.txt<\/code>\u00a0file. The library&#8217;s <code>R.txt<\/code>\u00a0only lists Android resources that it uses.<\/li>\n<li>Smaller <code>R.java<\/code>\u00a0files are generated using the integer values from the app&#8217;s <code>R.txt<\/code>.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>The process was not <i>too\u00a0<\/i>complex, so we could implement this in C# for Xamarin.Android. We already had C# code that could parse <code>R.txt<\/code>\u00a0files, so the only thing\u00a0<em>new\u00a0<\/em>would be something that could generate <code>R.java<\/code>\u00a0files.<\/p>\n<h3>The Results<\/h3>\n<p>We saw around 12,000 less fields just from this change:<\/p>\n<p><img decoding=\"async\" class=\"wp-image-44092 aligncenter\" src=\"http:\/\/devblogs.microsoft.com\/xamarin\/wp-content\/uploads\/sites\/44\/2019\/05\/2019-05-03-16_39_49-PowerPoint-Slide-Show-Presentation1-300x196.png\" alt=\"\" width=\"673\" height=\"440\" srcset=\"https:\/\/devblogs.microsoft.com\/xamarin\/wp-content\/uploads\/sites\/44\/2019\/05\/2019-05-03-16_39_49-PowerPoint-Slide-Show-Presentation1-300x196.png 300w, https:\/\/devblogs.microsoft.com\/xamarin\/wp-content\/uploads\/sites\/44\/2019\/05\/2019-05-03-16_39_49-PowerPoint-Slide-Show-Presentation1-768x502.png 768w, https:\/\/devblogs.microsoft.com\/xamarin\/wp-content\/uploads\/sites\/44\/2019\/05\/2019-05-03-16_39_49-PowerPoint-Slide-Show-Presentation1-1024x669.png 1024w, https:\/\/devblogs.microsoft.com\/xamarin\/wp-content\/uploads\/sites\/44\/2019\/05\/2019-05-03-16_39_49-PowerPoint-Slide-Show-Presentation1.png 1706w\" sizes=\"(max-width: 673px) 100vw, 673px\" \/><\/p>\n<p>The APK file was also around 120KB smaller.<\/p>\n<p>We also saw an impact to build times. Xamarin.Android builds no longer ran <code>aapt<\/code>\u00a0N times, and we were able to implement the\u00a0<code>R.txt<\/code> parser and\u00a0<code>R.java<\/code>\u00a0writer in an efficient manner in C#. Using MSBuild&#8217;s <code>\/clp:performancesummary<\/code>\u00a0option, we could see the duration for these MSBuild targets:<\/p>\n<pre class=\"toolbar:2 lang:default decode:true\">Before:\r\n3173 ms  _GenerateJavaDesignerForComponent          1 calls\r\n\r\nAfter:\r\n  20 ms  GenerateLibraryResources                   1 calls<\/pre>\n<p>Saving three seconds off of &#8220;Hello World&#8221; is great! We think these improvements could have an even bigger impact on larger Xamarin.Android projects.<\/p>\n<h3>Conclusion<\/h3>\n<p>If your Xamarin.Android app currently requires multidex, due to an error such as:<\/p>\n<pre class=\"toolbar:2 lang:default decode:true\">trouble writing output: \r\n    Too many field references to fit in one dex file: 70468; max is 65536.<\/pre>\n<p>In future versions of Xamarin.Android, an app hitting the &#8220;too many field references&#8221; error might not require multidex anymore.<\/p>\n<p>This enhancement are available in the latest <em><a href=\"https:\/\/devblogs.microsoft.com\/visualstudio\/visual-studio-2019-version-16-2-preview-2\/\" target=\"_blank\" rel=\"noopener noreferrer\">Visual Studio 2019 version 16.2 Preview<\/a><\/em> and <em><a href=\"https:\/\/devblogs.microsoft.com\/visualstudio\/visual-studio-2019-for-mac-version-8-1-is-now-available-and-a-preview-for-8-2\/\" target=\"_blank\" rel=\"noopener noreferrer\">Visual Studio for Mac 2019 version 8.2 Preview<\/a><\/em>.<\/p>\n<p>Further reading:<\/p>\n<ul>\n<li style=\"list-style-type: none;\">\n<ul>\n<li>See the <a href=\"https:\/\/github.com\/xamarin\/xamarin-android\/pull\/2896\">pull request on Github<\/a>, for further detail and source code.<\/li>\n<li>See\u00a0<a href=\"https:\/\/devblogs.microsoft.com\/xamarin\/shrinking-android-app-size\/\">Shrinking Your Android App Size<\/a> for more tips &amp; tricks for smaller APKs.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>One of our current focus areas in Xamarin.Android is build performance and reducing the multidex count. Developer productivity is directly impacted by the &#8220;inner dev loop&#8221;: the time it takes to make a small code change and see the result on a device or emulator. Performance is a feature!<\/p>\n","protected":false},"author":1345,"featured_media":44092,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[313,2,291,367],"tags":[5,648,784,4,16],"class_list":["post-44036","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-android","category-developers","category-xamarin-platform","category-xamarin-forms","tag-android","tag-performance","tag-support-libraries","tag-xamarin-platform","tag-xamarin-forms"],"acf":[],"blog_post_summary":"<p>One of our current focus areas in Xamarin.Android is build performance and reducing the multidex count. Developer productivity is directly impacted by the &#8220;inner dev loop&#8221;: the time it takes to make a small code change and see the result on a device or emulator. Performance is a feature!<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/xamarin\/wp-json\/wp\/v2\/posts\/44036","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/xamarin\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/xamarin\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/xamarin\/wp-json\/wp\/v2\/users\/1345"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/xamarin\/wp-json\/wp\/v2\/comments?post=44036"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/xamarin\/wp-json\/wp\/v2\/posts\/44036\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/xamarin\/wp-json\/wp\/v2\/media\/44092"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/xamarin\/wp-json\/wp\/v2\/media?parent=44036"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/xamarin\/wp-json\/wp\/v2\/categories?post=44036"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/xamarin\/wp-json\/wp\/v2\/tags?post=44036"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}