Ok Multiplatform! (Droidcon NYC 2018)

Video: https://www.youtube.com/watch?v=Q8B4eDirgk0

Okio is a small library that powers a lot of Square’s open source software, such as OkHttp, Moshi and Wire. Okio makes I/O easy by solving the most common problems in a simple and efficient way.

At Square, we’re investing in Kotlin. We love the language and the tooling, and we love how Kotlin makes us more productive. We’re excited about being able to run Kotlin on multiple platforms, and we’d love to be able to harness the power of Okio on Web and iOS - that’s why we’ve embarked on a journey to migrate Okio to multiplatform Kotlin!

In this talk we’ll share our experiences and namely:

- What worked for us and what didn’t
- Our strategy for moving fast without breaking code
- Maintaining compatibility: Java source vs Kotlin source vs bytecode
- Issues we’ve encountered along the way and ways to work around them
- Performance considerations
- How this impacts OkHttp, Retrofit, Moshi & Wire

This talk should be of interest to anyone who works with multiplatform Kotlin or wants to learn more about it.

Transcript

1. Ok Multiplatform! @jessewilson @egorand
2. What’s Okio?
3. • Complements java.io and java.nio • Better API • Better performance • Started as a part of OkHttp • Basis for many Square OSS libraries What’s Okio?
4. Okio OkHttp Moshi Wire Retrofit
5. Why Kotlin?
9. Kotlinizing Okio
10. Attempt #1: Koio
11. • Build a library like Okio but for Kotlin • Multiplatform by design • JDK6, JDK7, JDK8 and JS support FAILED Attempt #1: Koio
12. What went wrong?
13. • Too ambitious! • Missing primitives (UTF-8 encode/decode, synchronization, etc.) • Early days of multiplatform Kotlin
14. Attempt #2: okio-ktx
16. • New okio-kotlin module • Extension functions for frequently used factory methods NEVER SHIPPED Attempt #2: okio-ktx
17. What went wrong?
18. • No path to multiplatform • Redundant APIs for Kotlin callers • We didn’t get to use Kotlin features in Okio!
19. Attempt #3: Okio 2
21. Goals
22. Multiplatform • Okio 2 will start as a JVM-only library • Okio 2 will be written in 100% Kotlin • Okio 2 will gradually get support for JS and Native
23. JVM options for I/O java.io straightforward blocking API good performance max ~2000 threads doing I/O java.nio clumsy non-blocking API (callbacks!) good performance no thread count ceiling
24. JVM options for I/O java.io straightforward blocking API good performance max ~2000 threads doing I/O java.nio clumsy non-blocking API (callbacks!) good performance no thread count ceiling kotlin.coroutines straightforward blocking API good performance no thread count ceiling
25. Idiomatic Kotlin API • We want good experience for both Kotlin and Java users • e.g. extension functions that can be used as static methods on Java
26. Okio 1.x val source = Okio.buffer(Okio.source(File("README.md"))) Okio 2.x val source = File("README.md").source().buffer()
27. Okio 1.x public byte getByte(int pos) val bytes = ByteString.decodeHex(”cafebabe”) val byte = bytes.getByte(0) Okio 2.x @JvmName(“getByte") operator fun get(index: Int): Byte val bytes = "cafebabe".decodeHex() val byte = bytes[0]
28. public int getSize() val size = str.getSize() Okio 1.x val bytes = ByteString.decodeHex(”cafebabe”) val size: Int val size = str.size Okio 2.x val bytes = "cafebabe".decodeHex()
29. Costs
30. Costs • Everyone who’s using Okio (or OkHttp, or Retrofit) now has a dependency on Kotlin standard library… • …which is 939 KiB… • …but it shrinks to 7 KiB with R8 or ProGuard
31. The Journey to Okio 2
32. Step #1: Gradle
33. Project Structure
34. dependencies { classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:${versions.kotlin}" classpath "org.jetbrains.kotlin:kotlin-native-gradle-plugin:${versions.kotlinNative}" } Project level
35. apply plugin: 'org.jetbrains.kotlin.platform.common' apply plugin: 'org.jetbrains.kotlin.platform.jvm' apply plugin: 'org.jetbrains.kotlin.platform.js' apply plugin: 'org.jetbrains.kotlin.platform.native'
36. 'kotlin': [ 'gradlePlugin': "org.jetbrains.kotlin:kotlin-gradle-plugin:${versions.kotlin}", 'stdLib': [ 'common': "org.jetbrains.kotlin:kotlin-stdlib-common", 'jdk8': "org.jetbrains.kotlin:kotlin-stdlib-jdk8", 'jdk7': "org.jetbrains.kotlin:kotlin-stdlib-jdk7", 'jdk6': "org.jetbrains.kotlin:kotlin-stdlib", 'js': "org.jetbrains.kotlin:kotlin-stdlib-js", ], 'test': [ 'common': "org.jetbrains.kotlin:kotlin-test-common", 'annotations': "org.jetbrains.kotlin:kotlin-test-annotations-common", 'jdk': "org.jetbrains.kotlin:kotlin-test-junit", 'js': "org.jetbrains.kotlin:kotlin-test-js", ], 'native': [ 'gradlePlugin': "org.jetbrains.kotlin:kotlin-native-gradle-plugin:${versions.kotlinNative}", ] ], Dependencies
37. Publishing
38. Milestone #1 • We’ve got a fully working Kotlin multiplatform project! • Albeit with zero Kotlin files and JVM-only support
39. Step #2: ⌥⇧⌘K
40. • Convert Java files to Kotlin one-by-one • Rely on existing Java unit tests! Step #2: ⌥⇧⌘K
41. Milestone #2 • 100% Kotlin! • Still JVM-only
42. Step #3: Backfill
43. okio/jvm/src class PureKotlin { ... } Kotlin-only files okio/src class PureKotlin { ... }
44. Java → Kotlin override fun available() = Math.min(size, Integer.MAX_VALUE).toInt() override fun available() = minOf(size, Int.MAX_VALUE).toInt()
45. Platform-specific helpers okio/jvm/src System.arraycopy(src, srcPos, dest, destPos, length) okio/src expect fun arraycopy(src, srcPos, dest, destPos, length) okio/jvm/src actual fun arraycopy(src, srcPos, dest, destPos, length) { System.arraycopy(src, srcPos, dest, destPos, length) }
46. Java only /** Writes the contents of this byte string to `out`. */ @Throws(IOException::class) open fun write(out: OutputStream) { out.write(data) }
47. Step #4: JS/Native
48. • Create actual classes • Implement platform-specific helpers Step #4: JS/Native
49. Milestone #3 • 100% Kotlin! • JS/Native proof of concept
50. Compatibility
51. Binary NoSuchMethodError app.jar MyActivity.java library-v1.jar app.jar library-v2.jar javac java
52. Source
53. // Okio.java public Sink sink(File file) { return new OutputStreamSink(new FileOutputStream(file)); } Okio 1.x public static void main(String[] args) { Sink sink = Okio.sink(new File("README.md")); } fun main(args: Array<String>) { val sink = Okio.sink(File("README.md")) } Kotlin Java
54. Okio 2.x public static void main(String[] args) { Sink sink = Okio.sink(new File("README.md")); } fun main(args: Array<String>) { val sink = File(“README.md”).sink() } Kotlin Java // Okio.kt fun File.sink(): Sink = FileOutputStream(this).sink()
55. Okio 2.x public static void main(String[] args) { Sink sink = Okio.sink(new File("README.md")); } fun main(args: Array<String>) { val sink = File(“README.md”).sink() } Kotlin Java // Okio.kt @file:JvmName(“Okio") fun File.sink(): Sink = FileOutputStream(this).sink()
56. Migrating
60. Okio 2 • Binary-compatible with Okio 1.x • Source-compatible with Okio 1.x for Java users • Source-incompatible with Okio 1.x for Kotlin users • But better! Kotlin-friendly API
61. JApicmp
62. // @JvmOverloads open fun substring(beginIndex: Int = 0, endIndex: Int = size): ByteString ./gradlew :okio:jvm:japicmp > Task :okio:jvm:japicmp FAILED FAILURE: Build failed with an exception. * What went wrong: Execution failed for task ':okio:jvm:japicmp'. > A failure occurred while executing Comparing [jvm-2.0.0-SNAPSHOT.jar] with [okio-1.14.1.jar] > Detected binary changes between jvm-2.0.0-SNAPSHOT.jar and okio-1.14.1.jar. See failure report at file:// /okio/okio/jvm/build/reports/japi.txt
63. Quirks to watch out for • Internal classes are still public in bytecode • Classes and methods are final by default
64. Testing
65. • Keep your Java tests! • They’re calling your API from a Java user’s perspective • Add Kotlin tests on top • They’re testing Kotlin API • Try to promote your Kotlin tests to common Testing
66. ByteString.kt
67. expect class ByteString // Trusted internal constructor doesn't clone data. internal constructor(data: ByteArray) : Comparable<ByteString> { internal val data: ByteArray internal var hashCode: Int internal var utf8: String? /** Constructs a new `String` by decoding the bytes as `UTF-8`. */ fun utf8(): String /** * Returns this byte string encoded as [Base64](http://www.ietf.org/rfc/rfc2045.txt). In violation * of the RFC, the returned string does not wrap lines at 76 columns. */ fun base64(): String /** Returns this byte string encoded as [URL-safe Base64](http://www.ietf.org/rfc/rfc4648.txt). */ fun base64Url(): String /** Returns this byte string encoded in hexadecimal. */ fun hex(): String ... } okio/src/main/okio/ByteString.kt
68. actual open class ByteString internal actual constructor( internal actual val data: ByteArray ) : Serializable, Comparable<ByteString> { /** Writes the contents of this byte string to `out`. */ @Throws(IOException::class) open fun write(out: OutputStream) { out.write(data) } ... } okio/jvm/src/main/okio/ByteString.kt
69. Roadblocks
70. “Allow expect declarations with a default implementation” https://youtrack.jetbrains.com/issue/KT-20427
71. “Annotate relevant standard library annotations with @OptionalExpectation” https://youtrack.jetbrains.com/issue/KT-24478 (@OptionalExpectation has been introduced in 1.2.60)
72. “Class delegation should retain checked exceptions in JVM bytecode” https://youtrack.jetbrains.com/issue/KT-23935
73. “expect function with default parameters can't be annotated with JvmOverloads” https://youtrack.jetbrains.com/issue/KT-24357 FIXED
74. “Multiplatform: docs don't contain KDoc attached to expect members” https://github.com/Kotlin/dokka/issues/307
75. Benchmarks
76. 1.x 2.x
78. 1.x 2.x
79. 1.x 2.x
80. Results
81. OKIO2 OKIO2 OKIO2 OKIO2 Released August 27, 2018
82. Placeholder for the blog post screenshot
83. Future Plans
84. • Continue backfilling interfaces into common • Coroutines! • Kotlin Native! Future Plans
85. • OkHttp - someday • Retrofit - someday • Moshi - someday soon • Wire - someday really soon
86. Recommendations
87. • Start with the JVM target first • Rely on unit tests • Use JApicmp • Backfill • Report Kotlin multiplatform issues! Recommendations
88. Thanks! @jessewilson @egorand