Share
Embed
To ∞ (~65K) and beyond!
It's 2016 even for us Android developers. And among many of the problems that we carry over from the past years, the loathed, dreadful, why-in-the-name-of-Andy-Rubin limit of 65536 methods is the most notable of them.
This talk focuses on analyzing this "condition" from a pragmatic and down-to-earth perspective for developers. You will get to understand the problem and its possible solutions, each one of them presented with pros and cons. Bonus round: useful tips and some sarcasm.
Sebastiano Gottardo
April 08, 2016
Transcript
62 slides
-
To ∞ (~65K) and beyond! Sebastiano Gottardo, Android Engineer @
Musixmatch +SebastianoGottardo - @rotxed 1 -
Overview 2 • Context • 65K methods limit • Possible
approaches • MultiDex • Secondary DEX • Shrinking (and reasoning) • Bottom line -
Context • OSS, community, libraries, frameworks The Android open-source community
is enormous, and brings to the table thousands of libraries and frameworks. • ✓ Faster development, contributions and improvements These tools allows you to build better apps quicker. In addition, contributions by other users lead to improvements. • ✗ Using too many can lead to heavy APK, long loading times, bugs As we will learn, there are tradeoffs. 4 -
Dalvik Executable (DEX) • Container of the compiled source code
.java files are compiled into .class files, then packed into a .dex file 6 -
Where is the limit • Not runtime, not DEX, but
Dalvik’s instruction set! 7 https://goo.gl/ajlxwX -
Crossing the limit • Unable to build an APK •
Panic • Console output 8 Older build system Newer build system -
Context • No(t likely) Lose retro-compatibility! • ART (Android Runtime)
The new runtime used starting with Lollipop provides a way around. • What to do then? There are different solutions/approaches. 10 -
Approaches • MultiDex Google’s own solution. • Secondary DEX Get
your hands dirty. • Shrinking / Reasoning ProGuard is your friend, and so is your brain. 11 -
MultiDex • Google’s way around this limitation aka what’s in
the docs • Still not the “recommended” approach More on that later on. • Different working mechanisms depending on platform Dalvik ≠ ART. 13 -
MultiDex with ART • App’s source code is compiled and
packed into multiple DEX files Ultimately packed in the APK • At install time, the OS compiles the code of the DEX files into a single OAT file (ELF) AOT compilation instead of JIT compilation 15 -
MultiDex with Dalvik • App’s source code is compiled and
packed into a single DEX file Ultimately packed in the APK • Dalvik is unaware of DEX files other than the main one classes.dex • Multiple DEX files must be loaded at runtime Can be troublesome (a.k.a. sucks) 16 -
Setup 1/3 1. Gradle-wise setup 17 android { compileSdkVersion 23
buildToolsVersion "23.0.2" defaultConfig { ... minSdkVersion 15 targetSdkVersion 23 ... // Enabling multidex support. multiDexEnabled true } ... } -
Setup 2/3 2. Include the MultiDex support library 18
dependencies { compile 'com.android.support:multidex:1.0.0' } -
Setup 3/3 3. Enable MultiDex in your code 19 <manifest
xmlns:android="http://schemas.android.com/apk/res/android" package="com.example.android.multidex.myapplication"> <application ... android:name="android.support.multidex.MultiDexApplication"> ... </application> </manifest> a public class YourAwesomeApplication extends MultiDexApplication { ... } b @Override protected void attachBaseContext(Context base) { super.attachBaseContext(base); MultiDex.install(this); } c -
Advantages • Well integrated with the build chain No need
to pull off complicated tricks. • Seamless support across platform versions Being a support library. One code, one love! • Extremely easy to enable Literally takes 3 minutes. 20 -
Disadvantages • Limited in choosing what goes where Esp. true
for libraries that sport native components. • MultiDex on Dalvik is unreliable For 14 < API < 21 crashes may occur. • Secondary DEX files are loaded at runtime Startup time is increased! (unless you defer installation) • Build times increase Variants may help (dev — min 21, speeds things up) 21 -
Secondary DEX • Historically, the first workaround A blog post
by Fred Chung, a Googler at the time (2007) • Relied on customizing the build chain Some Ant magic • Used an interface-based approach 23 -
How does it work • Put one or more libraries
into a separate DEX file Limited to .jar files, .aar not supported • At runtime, load the secondary DEX on a custom ClassLoader Different from the default ClassLoader! • Invoke the libraries’ methods Interface, Reflection 24 -
Quick example w/ Reflection 25 twitter = new TwitterFactory().getInstance(); twitter.setOAuthConsumer(TWITTER_APP_KEY,
TWITTER_APP_SECRET); twitter.setOAuthAccessToken(new AccessToken(token, tokenSecret)); -
Quick example w/ Reflection 26 ... twitter = twitterManager.initializeTwitter(TWITTER_APP_KEY, TWITTER_APP_SECRET,
token, tokenSecret); ... public Object initializeTwitter(String TWITTER_APP_KEY, String TWITTER_APP_SECRET, String token, String tokenSecret) { try { Object twitterFactoryInstance = getClassCached(twitterFactoryFQN).newInstance(); Method getInstanceMethod = getMethodCached(twitterFactoryFQN, "getInstance", null); Object twitterInstance = getInstanceMethod.invoke(twitterFactoryInstance, null); Method setOAuthConsumerMethod = getMethodCached(twitterFQN, "setOAuthConsumer", String.class, String.class); setOAuthConsumerMethod.invoke(twitterInstance, TWITTER_APP_KEY, TWITTER_APP_SECRET); Class atClass = getClassCached(accessTokenFQN); Object atInstance = atClass.getDeclaredConstructor(new Class[] {String.class, String.class}).newInstance(token, tokenSecret); Method setOAuthAccessTokenMethod = getMethodCached(twitterFQN, "setOAuthAccessToken", atClass); setOAuthAccessTokenMethod.invoke(twitterInstance, atInstance); return twitterInstance; } catch (Exception e) { e.printStackTrace(); } return null; } -
Advantages • Choose exactly what goes where Take that, MultiDex
• Choose exactly when to load secondary DEX So not to impact cold start times 27 -
Disadvantages • Quite hard to setup Copy DEX to the
appropriate dir, create ClassLoader instance • Quite hard to work with Esp. if you rely on Reflection • Versioning! How to handle updates? 28 -
Improvements • Same approach as Google’s MultiDex Patches the Class
Loader so to look for other DEX files • Specify the dependencies as ‘provided’ So to be resolved at compile/build time, stripped in the final APK • Best of two worlds (kind of) Removes the need for ifaces, Reflection, still a bit cumbersome 29 -
— Google MultiDex online documentation Before configuring your app to
enable use of 65K or more method references, you should take steps to reduce the total number of references called by your app code, including methods defined by your app code or included libraries. 31 -
ProGuard in pills • Shrink, optimize, obfuscate • Works both
for your code and for third-party libraries • Has an impact on APK size, methods number, performance and security 32 -
ProGuard ProTips • Get familiar with writing appropriate rules The
stricter, the better 33 -keep class !android.support.v7.internal.view.menu.**,android.support.** {*;} • What’s wrong with the following keep rule? -keep class com.google.** { *; } -keepnames class com.google.** { *; } -keepclassmembernames class * { @com.google.android.gms.common.annotation.KeepName *; } • Adopts an exclusion-based approach -
ProGuard ProTips 34 -keep class !android.support.v7.internal.view.menu.**,android.support.** {*;} -keep class com.google.**
{ *; } -keepnames class com.google.** { *; } -keepclassmembernames class * { @com.google.android.gms.common.annotation.KeepName *; } • Adopts an exclusion-based approach • Get familiar with writing appropriate rules The stricter, the better • What’s wrong with the following keep rule? -
ProGuard: default rules 35 android { ... buildTypes { release
{ minifyEnabled true proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro' } } } • ProGuard already includes common rules for common cases ~/sdk/tools/proguard/lib/proguard-android.txt -keepclasseswithmembernames class * { native <methods>; } -
DexGuard: beefier, for beefier companies 36 • ProGuard’s bigger (and
pricier) brother • Advanced features: res/strings/.so encryption, stronger obfuscation, tamper detection, its own multidex! and more • Pluggable encryption! I mean, seriously, that exclamation mark has a reason • Default rules are much more extensive -
ProGuard and third-party libs 37 • Many provide the ProGuard
statement (a.k.a RTFM) e.g., Square, Facebook, … • Pay attention to ‘keep-all’ rules! especially for libraries with many methods -
… wait, how do I count libraries methods? 39 …
how do I count methods at all?! -
inloop’s “APK method count” • Simple website with drag-and-drop interface
• Drop the APK and see the (locally) computed count No APKs uploaded :+1: • http://inloop.github.io/apk-method-count/ 42 ✓ Easy, immediate, private ✗ Build every time -
ClassyShark • Useful Java tool to inspect APKs Written by
Boris Farber (Googler) • Inspect number of classes, members, methods, dependencies, … Acts as a simple decompiler, too 43 -
ClassyShark • Useful Java tool to inspect APKs Written by
Boris Farber (Googler) • Inspect number of classes, members, methods, dependencies, … Acts as a simple decompiler, too 46 ✓ Easy, plenty of info, multi-purpose ✗ Build every time, more oriented to agnostic debugging -
dexcount-gradle-plugin • Gradle plugin, developed by KeepSafe • See the
current methods count every time you build Just open the Gradle console • Outputs per-package information in a ${variant}.txt file 47 -
dexcount-gradle-plugin • Gradle plugin, developed by KeepSafe • See the
current methods count every time you build Just open the Gradle console • Outputs per-package information in a ${variant}.txt file 50 ✓ Keep the count controlled at a glance ✗ Cumbersome to see single deps count -
MethodsCount • Developed by Sebastiano Gottardo, Nicola Miotto, Dario Marcato
• Allows to see methods and dependencies for libraries beforehand • Compare different libraries in terms of “lightweight-ness” 51 -
MethodsCount 52 • More than 15000 libraries and dependencies What’s
missing is calculated upon the first request • For each entry 1. dex/jar size 2. direct count + dependencies count 3. multiple versions (monitor changes over time) -
MethodsCount - AS plugin 56 • Parses `build.gradle` and directly
shows the count • Convenient when you drop a dependency right away -
MethodsCount - AS plugin 57 • Parses `build.gradle` and directly
shows the count • Convenient when you drop a dependency right away -
MethodsCount 58 ✓ See the count before having to hit
the limit ✓ Compare different libraries Worth having 4000 methods instead of 900? ✓ Analyze dependencies No, Guava is not a good excuse to have 15000+ methods ✓ Integrated in AS, thanks to the plugin ✗ The count is before ProGuard → Upper bound! -
Bottom line • The limit is here, at least for
some time Learn to deal with it • The recommended way is “Do NOT cross the limit!” And use ProGuard. Your nerves will thank you later! • If you do need to cross, there are solutions (with drawbacks) MultiDex, secondary DEX 60 -
Routine 1. PM shows interest in a new library :eyes_roll:
2. With dex-gradle-plugin, check the current count 3. Check MethodsCount to see how many methods, size and deps 4. Evaluate whether fits a secondary DEX 5. Consider alternatives (again, MethodsCount) 61