Compose UI for... a light switch? (KotlinConf 2024)

It runs on Android, iOS, desktop, and the web, but in this talk we'll cover how to get Compose UI running on a smart light switch built on embedded Linux.

Follow along as we journey through the process of discovering the device backdoor, figuring out how to run custom software, standing up Kotlin and Compose UI for embedded Linux, and finally building our own UI for the switch. We'll cover the intended use case of this effort, as well as other potential uses of this technology. Finally, the actual switch will be in the building for a live demo.

Video: coming soon

Transcript

1. a light switch? Compose UI for… Jake Wharton.com
2. Compose UI for… a light switch? Jake Wharton.com
3. Part I The nerd snipe
12. courtesy homekitnews.com
13. courtesy homekitnews.com
14. courtesy homekitnews.com
15. courtesy homekitnews.com courtesy homekitnews.com
16. courtesy homekitnews.com courtesy homekitnews.com
17. courtesy homekitnews.com courtesy homekitnews.com
18. courtesy homekitnews.com courtesy homekitnews.com
19. courtesy homekitnews.com courtesy homekitnews.com
20. $ adb connect 192.168.2.53 connected to 192.168.2.53:5555
21. $ adb connect 192.168.2.53 connected to 192.168.2.53:5555 $ adb shell [root@rk3326_64:/]#
22. Demo
23. Part II Research phase
25. $ tar -xzf zulu17* && rm zulu17*.tar.gz
26. $ tar -xzf zulu17* && rm zulu17*.tar.gz $ adb push zulu17* /userdata zulu17.50.19-ca-jre17.0.11-linux_aarch64/: 264 files pushed, 0 skipped. 0.2 MB/s (143556329 bytes in 601.001s)
27. $ tar -xzf zulu17* && rm zulu17*.tar.gz $ adb push zulu17* /userdata zulu17.50.19-ca-jre17.0.11-linux_aarch64/: 264 files pushed, 0 skipped. 0.2 MB/s (143556329 bytes in 601.001s) $ echo 'class Hello { > public static void main(String... args) { > System.out.println("Hello, light switch!"); > } > }' > Hello.java
28. $ tar -xzf zulu17* && rm zulu17*.tar.gz $ adb push zulu17* /userdata zulu17.50.19-ca-jre17.0.11-linux_aarch64/: 264 files pushed, 0 skipped. 0.2 MB/s (143556329 bytes in 601.001s) $ echo 'class Hello { > public static void main(String... args) { > System.out.println("Hello, light switch!"); > } > }' > Hello.java $ javac Hello.java
29. $ tar -xzf zulu17* && rm zulu17*.tar.gz $ adb push zulu17* /userdata zulu17.50.19-ca-jre17.0.11-linux_aarch64/: 264 files pushed, 0 skipped. 0.2 MB/s (143556329 bytes in 601.001s) $ echo 'class Hello { > public static void main(String... args) { > System.out.println("Hello, light switch!"); > } > }' > Hello.java $ javac Hello.java $ adb push Hello.class /userdata
30. pushed, 0 skipped. 0.2 MB/s (143556329 bytes in 601.001s) $ echo 'class Hello { > public static void main(String... args) { > System.out.println("Hello, light switch!"); > } > }' > Hello.java $ javac Hello.java $ adb push Hello.class /userdata Hello.class: 1 files pushed, 0 skipped. 0.2 MB/s (1329 bytes in 1.001s) $ adb shell
31. cd /userdata $ echo 'class Hello { > public static void main(String... args) { > System.out.println("Hello, light switch!"); > } > }' > Hello.java $ javac Hello.java $ adb push Hello.class /userdata Hello.class: 1 files pushed, 0 skipped. 0.2 MB/s (1329 bytes in 1.001s) $ adb shell [root@rk3326_64:/]#
32. zulu*/bin/java -cp . Hello > public static void main(String... args) { > System.out.println("Hello, light switch!"); > } > }' > Hello.java $ javac Hello.java $ adb push Hello.class /userdata Hello.class: 1 files pushed, 0 skipped. 0.2 MB/s (1329 bytes in 1.001s) $ adb shell [root@rk3326_64:/]# cd /userdata [root@rk3326_64:/userdata]#
33. > }' > Hello.java $ javac Hello.java $ adb push Hello.class /userdata Hello.class: 1 files pushed, 0 skipped. 0.2 MB/s (1329 bytes in 1.001s) $ adb shell [root@rk3326_64:/]# cd /userdata [root@rk3326_64:/userdata]# zulu*/bin/java -cp . Hello Hello, light switch! [root@rk3326_64:/userdata]#
34. flutter_weston_func(){ export LD_LIBRARY_PATH=${FLUTTER_PATH}/bundle/lib:`echo $LD_LIBRARY_PATH` #step1: start weston if [ -f $ORBHAL_FILE ]; then cpu_str=$(cat $ORBHAL_FILE | grep platform_sdk_str | awk -F '=' '{print $ if [ "$cpu_str"x = "RK3326S"x ]; then if [ -f /usr/sbin/slt_gpu_light ]; then /usr/sbin/slt_gpu_light >> /tmp/gpu.log 2>&1 fi fi fi #step2: start gui if [ $? -eq 0 ];then pidof mixpad_gui >/dev/null if [ $? -ne 0 ];then echo "gui_start]Now,start flutter_gui, rotation:$ORB_ROTATION " >>/tmp/m
36. flutter_weston_func(){ export LD_LIBRARY_PATH=${FLUTTER_PATH}/bundle/lib:`echo $LD_LIBRARY_PATH` #step1: start weston if [ -f $ORBHAL_FILE ]; then cpu_str=$(cat $ORBHAL_FILE | grep platform_sdk_str | awk -F '=' '{print $ if [ "$cpu_str"x = "RK3326S"x ]; then if [ -f /usr/sbin/slt_gpu_light ]; then /usr/sbin/slt_gpu_light >> /tmp/gpu.log 2>&1 fi fi fi #step2: start gui if [ $? -eq 0 ];then pidof mixpad_gui >/dev/null if [ $? -ne 0 ];then echo "gui_start]Now,start flutter_gui, rotation:$ORB_ROTATION " >>/tmp/m
37. fi #step2: start gui if [ $? -eq 0 ];then pidof mixpad_gui >/dev/null if [ $? -ne 0 ];then echo "gui_start]Now,start flutter_gui, rotation:$ORB_ROTATION " >>/tmp/m echo "output:all:rotate${ORB_ROTATION}" > $WESTON_CONFIG cd $FLUTTER_PATH ./mixpad_gui -c $GUI_DATA_PATH -r $ORB_ROTATION -b bundle >> /tmp/mixpad gui_pid=$! echo $gui_pid > $GUI_PID_FILE else echo "gui_start]flutter_gui alread start,do nothing. " >>/tmp/mixpad_gui fi else echo "gui_start],Error:start flutter_gui failed" >>/tmp/mixpad_gui.log fi }
38. fi #step2: start gui if [ $? -eq 0 ];then pidof mixpad_gui >/dev/null if [ $? -ne 0 ];then echo "gui_start]Now,start flutter_gui, rotation:$ORB_ROTATION " >>/tmp/m echo "output:all:rotate${ORB_ROTATION}" > $WESTON_CONFIG cd $FLUTTER_PATH ./mixpad_gui -c $GUI_DATA_PATH -r $ORB_ROTATION -b bundle >> /tmp/mixpad gui_pid=$! echo $gui_pid > $GUI_PID_FILE else echo "gui_start]flutter_gui alread start,do nothing. " >>/tmp/mixpad_gui fi else echo "gui_start],Error:start flutter_gui failed" >>/tmp/mixpad_gui.log fi }
43. application { Window( onCloseRequest = ::exitApplication, undecorated = true, state = rememberWindowState(placement = Fullscreen), ) { Text("Hello, Desktop!") } }
51. Part III Pushing pixels
54. fun main() { println("Hello!") }
55. $ ./gradlew -q assemble
56. adb push build/.../kntest.kexe /userdata $ ./gradlew -q assemble $
57. $ ./gradlew -q assemble $ adb push build/.../kntest.kexe /userdata build/bin/linuxArm64/debugExecutable/kntest.exe...file pushed, 0 skipped. 1.0 MB/s (1699880 bytes in 1.668s) $
58. adb shell $ ./gradlew -q assemble $ adb push build/.../kntest.kexe /userdata build/bin/linuxArm64/debugExecutable/kntest.exe...file pushed, 0 skipped. 1.0 MB/s (1699880 bytes in 1.668s) $
59. $ ./gradlew -q assemble $ adb push build/.../kntest.kexe /userdata build/bin/linuxArm64/debugExecutable/kntest.exe...file pushed, 0 skipped. 1.0 MB/s (1699880 bytes in 1.668s) $ adb shell [root@rk3326_64:/]#
60. /userdata/kntest.kexe $ ./gradlew -q assemble $ adb push build/.../kntest.kexe /userdata build/bin/linuxArm64/debugExecutable/kntest.exe...file pushed, 0 skipped. 1.0 MB/s (1699880 bytes in 1.668s) $ adb shell [root@rk3326_64:/]#
61. $ ./gradlew -q assemble $ adb push build/.../kntest.kexe /userdata build/bin/linuxArm64/debugExecutable/kntest.exe...file pushed, 0 skipped. 1.0 MB/s (1699880 bytes in 1.668s) $ adb shell [root@rk3326_64:/]# /userdata/kntest.kexe Hello! [root@rk3326_64:/]#
62. compilation linking .kexe .kt
63. compilation linking .kexe .kt cinterop .h .so
64. compilation linking .kexe .kt cinterop .h .so
65. $ ls /usr/lib/x86_64-linux-gnu/*drm*
66. $ ls /usr/lib/x86_64-linux-gnu/*drm* libdrm.so.2 -> libdrm.so.2.4.0 libdrm.so.2.4.0 libdrm_amdgpu.so.1 -> libdrm_amdgpu.so.1.0.0 libdrm_amdgpu.so.1.0.0 libdrm_intel.so.1 -> libdrm_intel.so.1.0.0 libdrm_intel.so.1.0.0 libdrm_nouveau.so.2 -> libdrm_nouveau.so.2.0.0 libdrm_nouveau.so.2.0.0 libdrm_radeon.so.1 -> libdrm_radeon.so.1.0.1 libdrm_radeon.so.1.0.1 $
67. $ ls /usr/lib/x86_64-linux-gnu/*drm* libdrm.so.2 -> libdrm.so.2.4.0 libdrm.so.2.4.0 libdrm_amdgpu.so.1 -> libdrm_amdgpu.so.1.0.0 libdrm_amdgpu.so.1.0.0 libdrm_intel.so.1 -> libdrm_intel.so.1.0.0 libdrm_intel.so.1.0.0 libdrm_nouveau.so.2 -> libdrm_nouveau.so.2.0.0 libdrm_nouveau.so.2.0.0 libdrm_radeon.so.1 -> libdrm_radeon.so.1.0.1 libdrm_radeon.so.1.0.1 $ ls /usr/include/drm
68. $ ls /usr/lib/x86_64-linux-gnu/*drm* libdrm.so.2 -> libdrm.so.2.4.0 libdrm.so.2.4.0 libdrm_amdgpu.so.1 -> libdrm_amdgpu.so.1.0.0 libdrm_amdgpu.so.1.0.0 libdrm_intel.so.1 -> libdrm_intel.so.1.0.0 libdrm_intel.so.1.0.0 libdrm_nouveau.so.2 -> libdrm_nouveau.so.2.0.0 libdrm_nouveau.so.2.0.0 libdrm_radeon.so.1 -> libdrm_radeon.so.1.0.1 libdrm_radeon.so.1.0.1 $ ls /usr/include/drm amdgpu_drm.h armada_drm.h drm_fourcc.h
69. omap_drm.h panfrost_drm.h pvr_drm.h qaic_accel.h qxl_drm.h radeon_drm.h tegra_drm.h v3d_drm.h vc4_drm.h vgem_drm.h virtgpu_drm.h vmwgfx_drm.h xe_drm.h $
70. adb shell omap_drm.h panfrost_drm.h pvr_drm.h qaic_accel.h qxl_drm.h radeon_drm.h tegra_drm.h v3d_drm.h vc4_drm.h vgem_drm.h virtgpu_drm.h vmwgfx_drm.h xe_drm.h $
71. panfrost_drm.h pvr_drm.h qaic_accel.h qxl_drm.h radeon_drm.h tegra_drm.h v3d_drm.h vc4_drm.h vgem_drm.h virtgpu_drm.h vmwgfx_drm.h xe_drm.h $ adb shell [root@rk3326_64:/]#
72. ls /usr/lib/*drm* panfrost_drm.h pvr_drm.h qaic_accel.h qxl_drm.h radeon_drm.h tegra_drm.h v3d_drm.h vc4_drm.h vgem_drm.h virtgpu_drm.h vmwgfx_drm.h xe_drm.h $ adb shell [root@rk3326_64:/]#
73. vgem_drm.h virtgpu_drm.h vmwgfx_drm.h xe_drm.h $ adb shell [root@rk3326_64:/]# ls /usr/lib/*drm* libdrm.so -> libdrm.so.2.4.0 libdrm.so.2 -> libdrm.so.2.4.0 libdrm.so.2.4.0 libdrm_rockchip.so -> libdrm_rockchip.so.1.0.0 libdrm_rockchip.so.1 -> libdrm_rockchip.so.1.0.0 libdrm_rockchip.so.1.0.0 [root@rk3326_64:/]#
74. ls /usr/include vgem_drm.h virtgpu_drm.h vmwgfx_drm.h xe_drm.h $ adb shell [root@rk3326_64:/]# ls /usr/lib/*drm* libdrm.so -> libdrm.so.2.4.0 libdrm.so.2 -> libdrm.so.2.4.0 libdrm.so.2.4.0 libdrm_rockchip.so -> libdrm_rockchip.so.1.0.0 libdrm_rockchip.so.1 -> libdrm_rockchip.so.1.0.0 libdrm_rockchip.so.1.0.0 [root@rk3326_64:/]#
75. xe_drm.h $ adb shell [root@rk3326_64:/]# ls /usr/lib/*drm* libdrm.so -> libdrm.so.2.4.0 libdrm.so.2 -> libdrm.so.2.4.0 libdrm.so.2.4.0 libdrm_rockchip.so -> libdrm_rockchip.so.1.0.0 libdrm_rockchip.so.1 -> libdrm_rockchip.so.1.0.0 libdrm_rockchip.so.1.0.0 [root@rk3326_64:/]# ls /usr/include /bin/sh: /usr/include: not found [root@rk3326_64:/]#
78. [root@rk3326_64:/]# strings /usr/lib/libdrm.so
79. [root@rk3326_64:/]# strings /usr/lib/libdrm.so T$_" J)\] wUS/ &wB?( |3K4H' o&{aY 6=^h3z +NV< l!l0ev __gmon_start__ _ITM_deregisterTMCloneTable _ITM_registerTMCloneTable __cxa_finalize strlen gnu_dev_major
80. .eh_frame .init_array .fini_array .data.rel.ro .dynamic .got .got.plt .data .bss [root@rk3326_64:/]#
81. 1 [root@rk3326_64:/]# strings /usr/lib/libdrm.so | grep -c drmModeGetResources .eh_frame .init_array .fini_array .data.rel.ro .dynamic .got .got.plt .data .bss [root@rk3326_64:/]#
82. 0 [root@rk3326_64:/]# .dynamic .got .got.plt .data .bss [root@rk3326_64:/]# strings /usr/lib/libdrm.so | grep -c drmModeGetResources 1 [root@rk3326_64:/]#
83. 0 [root@rk3326_64:/]# .dynamic .got .got.plt .data .bss [root@rk3326_64:/]# strings /usr/lib/libdrm.so | grep -c drmModeGetResources 1 [root@rk3326_64:/]# strings /usr/lib/libdrm.so | grep -c drmIsMaster
84. .bss [root@rk3326_64:/]# strings /usr/lib/libdrm.so | grep -c drmModeGetResources 1 [root@rk3326_64:/]# strings /usr/lib/libdrm.so | grep -c drmIsMaster 0 [root@rk3326_64:/]# $
85. exit .bss [root@rk3326_64:/]# strings /usr/lib/libdrm.so | grep -c drmModeGetResources 1 [root@rk3326_64:/]# strings /usr/lib/libdrm.so | grep -c drmIsMaster 0 [root@rk3326_64:/]# $
86. [root@rk3326_64:/]# strings /usr/lib/libdrm.so | grep -c drmModeGetResources 1 [root@rk3326_64:/]# strings /usr/lib/libdrm.so | grep -c drmIsMaster 0 [root@rk3326_64:/]# exit $
87. [root@rk3326_64:/]# strings /usr/lib/libdrm.so | grep -c drmModeGetResources 1 [root@rk3326_64:/]# strings /usr/lib/libdrm.so | grep -c drmIsMaster 0 [root@rk3326_64:/]# exit $ adb pull /usr/lib/libdrm.so
88. [root@rk3326_64:/]# strings /usr/lib/libdrm.so | grep -c drmModeGetResources 1 [root@rk3326_64:/]# strings /usr/lib/libdrm.so | grep -c drmIsMaster 0 [root@rk3326_64:/]# exit $ adb pull /usr/lib/libdrm.so /usr/lib/libdrm.so: 1 file pulled, 0 skipped. 0.6 MB/s (63976 bytes in 0.109s)
89. compilation linking .kexe .kt cinterop .h .so
90. kotlin { linuxArm64 { binaries.executable { entryPoint = "main" } } }
91. kotlin { linuxArm64 { compilations.main.cinterops { create("lightswitch") { packageName("lightswitch") includeDirs( "device/include/", "device/include/libdrm", ) header("device/include/xf86drm.h") header("device/include/xf86drmMode.h") } } binaries.executable { entryPoint = "main" }
92. header("device/include/xf86drm.h") header("device/include/xf86drmMode.h") } } 
 binaries.executable { entryPoint = "main" // Work around KT-48082. linkerOpts( "-L${file("device/lib").absolutePath}", "-ldrm", // Do not check symbols within our shared libraries. "--allow-shlib-undefined", ) } } }
93. fun main() { println("Hello!") }
94. private const val devicePath = "/dev/dri/card0" fun main() { println("Hello!") val deviceFd = open(devicePath, O_RDWR or O_CLOEXEC) }
95. private const val devicePath = "/dev/dri/card0" fun main() { println("Hello!") val deviceFd = open(devicePath, O_RDWR or O_CLOEXEC) check(deviceFd != -1) { "Couldn't open $devicePath" } }
96. private const val devicePath = "/dev/dri/card0" fun main() { println("Hello!") val deviceFd = open(devicePath, O_RDWR or O_CLOEXEC) check(deviceFd != -1) { "Couldn't open $devicePath" } println("Got DRM device $deviceFd") }
97. private const val devicePath = "/dev/dri/card0" fun main() { println("Hello!") val deviceFd = open(devicePath, O_RDWR or O_CLOEXEC) check(deviceFd != -1) { "Couldn't open $devicePath" } println("Got DRM device $deviceFd") close(deviceFd) }
98. private const val devicePath = "/dev/dri/card0" fun main() { println("Hello!") val deviceFd = open(devicePath, O_RDWR or O_CLOEXEC) check(deviceFd != -1) { "Couldn't open $devicePath" } println("Got DRM device $deviceFd") try { val resourcesPtr = drmModeGetResources(deviceFd) ?: throw IllegalStateException("DRM resources") println("Got DRM resources") drmModeFreeResources(resourcesPtr) } finally { close(deviceFd) }
99. private const val devicePath = "/dev/dri/card0" fun main() { println("Hello!") val deviceFd = open(devicePath, O_RDWR or O_CLOEXEC) check(deviceFd != -1) { "Couldn't open $devicePath" } println("Got DRM device $deviceFd") try { val resourcesPtr = drmModeGetResources(deviceFd) ?: throw IllegalStateException("DRM resources") println("Got DRM resources") try { val connectorPtr = findConnector(deviceFd, resourcesPtr) ?: throw IllegalStateException("DRM connector") try { connectorPtr.pointed.modes!![0].let { mode -> println("Resolution ${mode.vdisplay}x${mode.hdisplay}") } val encoderPtr = findEncoder(deviceFd, resourcesPtr, connectorPtr) ?: throw IllegalStateException("DRM encoder") println("Got DRM encoder") try { val crtcId = encoderPtr.pointed.crtc_id // The Flutter embedded code does another loop here if the CRTC ID is missing. I'm not // entirely convinced that's needed, so guard the condition for now. check(crtcId != 0U) { "Encoder has no CTRC ID!" } val crtcPtr = drmModeGetCrtc(deviceFd, crtcId) ?: throw IllegalStateException("DRM CRTC") println("Got CRTC for ID $crtcId") println("All good!") } finally { drmModeFreeEncoder(encoderPtr) } } finally { drmModeFreeConnector(connectorPtr) } } finally { drmModeFreeResources(resourcesPtr) } } finally { close(deviceFd) } }
100. $ ./run_debug.sh
101. Hello! Got DRM device 3 Got DRM resources Resolution 320x170 Got DRM encoder Got CRTC for ID 60 All good! $ $ ./run_debug.sh build/bin/linuxArm64/debugExecutable/composeui- lightswitch...file pushed, 0 skipped. 1.0 MB/s (1699880 bytes in 1.668s)
102. create("lightswitch") { packageName("lightswitch") includeDirs( "device/include/", "device/include/libdrm", ) header("device/include/xf86drm.h") header("device/include/xf86drmMode.h") } } 
 binaries.executable { entryPoint = "main" // Work around KT-48082. linkerOpts( "-L${file("device/lib").absolutePath}", "-ldrm", // Do not check symbols within our shared libraries.
103. create("lightswitch") { packageName("lightswitch") includeDirs( "device/include/", "device/include/libdrm", ) header("device/include/GLES2/gl2.h") header("device/include/GLES2/gl2ext.h") header("device/include/EGL/egl.h") header("device/include/EGL/eglext.h") header("device/include/gbm.h") header("device/include/xf86drm.h") header("device/include/xf86drmMode.h") } } 
 binaries.executable { entryPoint = "main"
104. } } 
 binaries.executable { entryPoint = "main" // Work around KT-48082. linkerOpts( "-L${file("device/lib").absolutePath}", "-ldrm", // Do not check symbols within our shared libraries. "--allow-shlib-undefined", ) } } }
105. } } 
 binaries.executable { entryPoint = "main" // Work around KT-48082. linkerOpts( "-L${file("device/lib").absolutePath}", "-ldrm", "-lmali", // Do not check symbols within our shared libraries. "--allow-shlib-undefined", ) } } }
106. // Lots of DRM, GBM, EGL setup code while (true) { println("Draw!!!") glClear(GL_COLOR_BUFFER_BIT.toUInt()) glClearColor(0.2f, 0.3f, 0.5f, 1.0f) eglSwapBuffers(gl.display, gl.surface) val nextBo = gbm_surface_lock_front_buffer(gbm.surfacePtr) val nextFb = drm_fb_get_from_bo(drm, nextBo) drmModePageFlip(…) // Wait for page flip code… }
107. $ ./run_debug.sh
108. $ ./run_debug.sh Opened DRM device 3 Got DRM resources Got DRM connector Got DRM encoder Resolution: 320x170 Freeing DRM encoder Freeing DRM connector Freeing DRM resources Got CRTC for ID 60 Got GBM device Created GBM surface Got EGL display arm_release_ver of this libmali is 'g2p0-01eac0', rk_so_ver is '7'. Initialized EGL display
109. GL_KHR_robustness GL_KHR_robust_buffer_access_behavior GL_EXT_draw_elements_base_vertex GL_OES_draw_elements_base_vertex GL_EXT_buffer_storage GL_ARM_texture_unnormalized_coordinates Page flip! Draw!!! Page flip! Draw!!! Page flip! Draw!!! Page flip! Draw!!! Page flip! Draw!!! Page flip! Draw!!! Page flip! Draw!!!
110. GL_KHR_robustness GL_KHR_robust_buffer_access_behavior GL_EXT_draw_elements_base_vertex GL_OES_draw_elements_base_vertex GL_EXT_buffer_storage GL_ARM_texture_unnormalized_coordinates Page flip! Draw!!! Page flip! Draw!!! Page flip! Draw!!! Page flip! Draw!!! Page flip! Draw!!! Page flip! Draw!!! Page flip! Draw!!!
111. Draw!!! Page flip! Draw!!! Page flip! Draw!!! Page flip! Draw!!! Page flip! Draw!!! Destroying EGL surface Destroying EGL context Terminating EGL display Destroyed GBM surface Destroying GBM device Freeing CRTC Closing DRM device
112. Part IV Skiko
135. $ cat /usr/bin/skiko-clang
136. $ cat /usr/bin/skiko-clang exec docker exec skiko-arm clang++ $@ $ docker run --privileged --rm \ tonistiigi/binfmt --install amd64
137. $ cat /usr/bin/skiko-clang exec docker exec skiko-arm clang++ $@ $ docker run --privileged --rm \ tonistiigi/binfmt --install amd64 $ docker run -i -d \ --name skiko-arm \ -v /home/jake/skiko:/home/jake/skiko \ --platform linux/arm64 \ ubuntu:20.04
138. $ cat /usr/bin/skiko-clang exec docker exec skiko-arm clang++ $@ $ docker run --privileged --rm \ tonistiigi/binfmt --install amd64 $ docker run -i -d \ --name skiko-arm \ -v /home/jake/skiko:/home/jake/skiko \ --platform linux/arm64 \ ubuntu:20.04 $ docker exec -it skiko-arm bash
140. // Lots of DRM, GBM, EGL setup code val context = DirectContext.makeEGL()
141. // Lots of DRM, GBM, EGL setup code val context = DirectContext.makeEGL() // Inside render loop val renderTarget = BackendRenderTarget.makeGL(…) val surface = Surface.makeFromBackendRenderTarget(…) val canvas = surface.canvas
142. // Lots of DRM, GBM, EGL setup code val context = DirectContext.makeEGL() // Inside render loop val renderTarget = BackendRenderTarget.makeGL(…) val surface = Surface.makeFromBackendRenderTarget(…) val canvas = surface.canvas canvas.clear(Color.RED)
143. val context = DirectContext.makeEGL() // Inside render loop val renderTarget = BackendRenderTarget.makeGL(…) val surface = Surface.makeFromBackendRenderTarget(…) val canvas = surface.canvas canvas.clear(Color.RED) val bluePaint = Paint().apply { color = Color.BLUE } canvas.drawCircle(50f, 50f, 100f, bluePaint)
144. val context = DirectContext.makeEGL() // Inside render loop val renderTarget = BackendRenderTarget.makeGL(…) val surface = Surface.makeFromBackendRenderTarget(…) val canvas = surface.canvas canvas.clear(Color.RED) val bluePaint = Paint().apply { color = Color.BLUE } canvas.drawCircle(50f, 50f, 100f, bluePaint) val greenPaint = Paint().apply { color = Color.GREEN } canvas.drawLine(10f, 10f, 110f, 110f, greenPaint)
145. val context = DirectContext.makeEGL() // Inside render loop val renderTarget = BackendRenderTarget.makeGL(…) val surface = Surface.makeFromBackendRenderTarget(…) val canvas = surface.canvas canvas.clear(Color.RED) val bluePaint = Paint().apply { color = Color.BLUE } canvas.drawCircle(50f, 50f, 100f, bluePaint) val greenPaint = Paint().apply { color = Color.GREEN } canvas.drawLine(10f, 10f, 110f, 110f, greenPaint) surface.flushAndSubmit()
148. Part V Input handling
149. [root@rk3326_64:/]# evtest
150. [root@rk3326_64:/]# evtest No device specified, trying to scan all of /dev/input/event* Available devices: /dev/input/event0: rk8xx_pwrkey /dev/input/event1: fts_ts /dev/input/event2: adc-keys /dev/input/event3: gpio_keys /dev/input/event4: gpio_keys2 /dev/input/event5: lightsensor-level Select the device event number [0-5]:
151. 1 [root@rk3326_64:/]# evtest No device specified, trying to scan all of /dev/input/event* Available devices: /dev/input/event0: rk8xx_pwrkey /dev/input/event1: fts_ts /dev/input/event2: adc-keys /dev/input/event3: gpio_keys /dev/input/event4: gpio_keys2 /dev/input/event5: lightsensor-level Select the device event number [0-5]:
152. /dev/input/event4: gpio_keys2 /dev/input/event5: lightsensor-level Select the device event number [0-5]: 1 Input driver version is 1.0.1 Input device ID: bus 0x18 vendor 0x0 product 0x0 version 0x0 Input device name: "fts_ts" Supported events: Event type 0 (EV_SYN) Event type 1 (EV_KEY) Event code 330 (BTN_TOUCH) Event type 3 (EV_ABS) Event code 48 (ABS_MT_TOUCH_MAJOR) Value 0 Min 0 Max 255 Event code 53 (ABS_MT_POSITION_X) Value 0 Min 0
153. Event type 3 (EV_ABS) Event code 48 (ABS_MT_TOUCH_MAJOR) Value 0 Min 0 Max 255 Event code 53 (ABS_MT_POSITION_X) Value 0 Min 0 Max 170 Event code 54 (ABS_MT_POSITION_Y) Value 0 Min 0 Max 320 Event code 57 (ABS_MT_TRACKING_ID) Value 0 Min 0 Max 15 Properties:
154. Properties: Property type 1 (INPUT_PROP_DIRECT) Testing ... (interrupt to exit)
155. Properties: Property type 1 (INPUT_PROP_DIRECT) Testing ... (interrupt to exit) Event: type 3 (EV_ABS), code 57 (ABS_MT_TRACKING_ID), value 0 Event: type 3 (EV_ABS), code 48 (ABS_MT_TOUCH_MAJOR), value 9 Event: type 3 (EV_ABS), code 53 (ABS_MT_POSITION_X), value 80 Event: type 3 (EV_ABS), code 54 (ABS_MT_POSITION_Y), value 171 Event: ++++++++++++++ SYN_MT_REPORT ++++++++++++ Event: type 1 (EV_KEY), code 330 (BTN_TOUCH), value 1 Event: -------------- SYN_REPORT ------------ Event: type 3 (EV_ABS), code 57 (ABS_MT_TRACKING_ID), value 0 Event: type 3 (EV_ABS), code 48 (ABS_MT_TOUCH_MAJOR), value 9 Event: type 3 (EV_ABS), code 53 (ABS_MT_POSITION_X), value 80 Event: type 3 (EV_ABS), code 54 (ABS_MT_POSITION_Y), value 171 Event: ++++++++++++++ SYN_MT_REPORT ++++++++++++ Event: -------------- SYN_REPORT ------------ Event: type 3 (EV_ABS), code 57 (ABS_MT_TRACKING_ID), value 0 Event: type 3 (EV_ABS), code 48 (ABS_MT_TOUCH_MAJOR), value 9
156. Event: type 1 (EV_KEY), code 330 (BTN_TOUCH), value 1 Event: -------------- SYN_REPORT ------------ Event: type 3 (EV_ABS), code 57 (ABS_MT_TRACKING_ID), value 0 Event: type 3 (EV_ABS), code 48 (ABS_MT_TOUCH_MAJOR), value 9 Event: type 3 (EV_ABS), code 53 (ABS_MT_POSITION_X), value 80 Event: type 3 (EV_ABS), code 54 (ABS_MT_POSITION_Y), value 171 Event: ++++++++++++++ SYN_MT_REPORT ++++++++++++ Event: -------------- SYN_REPORT ------------ Event: type 3 (EV_ABS), code 57 (ABS_MT_TRACKING_ID), value 0 Event: type 3 (EV_ABS), code 48 (ABS_MT_TOUCH_MAJOR), value 9 Event: type 3 (EV_ABS), code 53 (ABS_MT_POSITION_X), value 80 Event: type 3 (EV_ABS), code 54 (ABS_MT_POSITION_Y), value 171 Event: ++++++++++++++ SYN_MT_REPORT ++++++++++++ Event: -------------- SYN_REPORT ------------ Event: type 1 (EV_KEY), code 330 (BTN_TOUCH), value 0 Event: ++++++++++++++ SYN_MT_REPORT ++++++++++++ Event: -------------- SYN_REPORT ------------
157. internal class TouchInput { private var isButtonDown = false private var nextButtonUp = false private var nextX = 0 private var nextY = 0 fun process(event: input_event): TouchEvent? { return null } }
158. private var nextY = 0 fun process(event: input_event): TouchEvent? { when (event.type.convert<Int>()) { EV_ABS -> { when (event.code.convert<Int>()) { ABS_MT_POSITION_X -> { nextX = event.value } ABS_MT_POSITION_Y -> { nextY = event.value } } } EV_KEY -> { when (event.code.convert<Int>()) { BTN_TOUCH -> { when (event.value) {
159. nextY = event.value } } } EV_KEY -> { when (event.code.convert<Int>()) { BTN_TOUCH -> { when (event.value) { 0 -> { nextButtonUp = true } } } } } EV_SYN -> { when (event.code.convert<Int>()) { 0 -> {
160. } } } } EV_SYN -> { when (event.code.convert<Int>()) { 0 -> { val eventType = when { !isButtonDown -> PointerEventType.Press nextButtonUp -> PointerEventType.Release else -> PointerEventType.Move } val position = Offset( nextX.toFloat(), nextY.toFloat()) val timeMillis = event.time.tv_sec * 1000 + event.time.tv_usec / 1000 isButtonDown = !nextButtonUp
161. val timeMillis = event.time.tv_sec * 1000 + event.time.tv_usec / 1000 isButtonDown = !nextButtonUp nextButtonUp = false return TouchEvent( eventType = eventType, position = position, timeMillis = timeMillis, ) } } } } return null }
162. Part VI Compose UI
172. // Lots of DRM, GBM, EGL, Skiko setup code val scene = MultiLayerComposeScene( size = IntSize( width = width, height = height, ), coroutineContext = ComposeUiMainDispatcher, )
174. // Inside render loop val renderTarget = BackendRenderTarget.makeGL(…) val surface = Surface.makeFromBackendRenderTarget(…) val canvas = surface.canvas canvas.clear(Color.RED) val bluePaint = Paint().apply { color = Color.BLUE } canvas.drawCircle(50f, 50f, 100f, bluePaint) val greenPaint = Paint().apply { color = Color.GREEN } canvas.drawLine(10f, 10f, 110f, 110f, greenPaint) surface.flushAndSubmit()
175. canvas.clear(Color.RED) val bluePaint = Paint().apply { color = Color.BLUE } canvas.drawCircle(50f, 50f, 100f, bluePaint) val greenPaint = Paint().apply { color = Color.GREEN } canvas.drawLine(10f, 10f, 110f, 110f, greenPaint) // Inside render loop val renderTarget = BackendRenderTarget.makeGL(…) val surface = Surface.makeFromBackendRenderTarget(…) val canvas = surface.canvas scene.render(canvas.asComposeCanvas(), nanoTime()) surface.flushAndSubmit()
176. val canvas = surface.canvas scene.render(canvas.asComposeCanvas(), nanoTime()) surface.flushAndSubmit() // In the event handler touch.process(event)?.let { touchEvent -> scene.sendPointerEvent( eventType = touchEvent.eventType, position = touchEvent.position, timeMillis = touchEvent.timeMillis, type = PointerType.Touch, ) }
177. // Lots of DRM, GBM, EGL, Skiko setup code val scene = MultiLayerComposeScene( size = IntSize( width = width, height = height, ), coroutineContext = ComposeUiMainDispatcher, )
178. // Lots of DRM, GBM, EGL, Skiko setup code val scene = MultiLayerComposeScene( size = IntSize( width = width, height = height, ), coroutineContext = ComposeUiMainDispatcher, ) scene.setContent { Button(onClick = {}) {} }
182. Part VII It's a switch
183. controller mixpad-gui rs485
184. controller mixpad-gui rs485 ember-host
185. controller mixpad-gui rs485 ember-host UDS /oem/ember-host/data/dm_socket_api.uds
186. val uds = UnixSocketAddress( "/oem/ember-host/data/dm_socket_api.uds", ) val socket = aSocket(SelectorManager(Dispatchers.Default)) .tcp() .connect(uds)
187. val uds = UnixSocketAddress( "/oem/ember-host/data/dm_socket_api.uds", ) val socket = aSocket(SelectorManager(Dispatchers.Default)) .tcp() .connect(uds) val writer = socket.openWriteChannel() writer.writeStringUtf8(value) writer.close(null)
188. val uds = UnixSocketAddress( "/oem/ember-host/data/dm_socket_api.uds", ) val socket = aSocket(SelectorManager(Dispatchers.Default)) .tcp() .connect(uds) val writer = socket.openWriteChannel() writer.writeStringUtf8(value) writer.close(null) val result = socket.openReadChannel() .readRemaining() .readUTF8Line() .orEmpty()
189. Caused by: kotlin.IllegalArgumentException: Failed to open iconv for charset ISO-8859-1 with error code 22 at 0 kfun:kotlin.Throwable#<init>(kotlin.String?){} + 91 at 1 kfun:kotlin.Exception#<init>(kotlin.String?){} + 83 at 2 kfun:kotlin.RuntimeException#<init>(kotlin.String?){} + 83 at 3 kfun:kotlin.IllegalArgumentException#<init>(kotlin.String?){} + 8 at 4 kfun:io.ktor.utils.io.charsets#checkErrors(kotlinx.cinterop.CPoin at 5 kfun:io.ktor.utils.io.charsets.CharsetIconv.<init>#internal + 515 at 6 kfun:io.ktor.utils.io.charsets.Charsets#<init>(){} + 275 at 7 kfun:io.ktor.utils.io.charsets.Charsets.$init_global#internal + 1 at 8 CallInitGlobalPossiblyLock + 487 at 9 kfun:io.ktor.utils.io.charsets.Charsets#<get-$instance>#static(){
198. Demo
199. github.com/JakeWharton/skia-pack main branch github.com/JakeWharton/skiko jw.egl.2024-04-16 branch github.com/JakeWharton/compose-multiplatform-core jw.linux-arm.2024-04-26 branch github.com/JakeWharton/ktor jw.charset-lazy.2024-05-14 branch
200. github.com/JakeWharton/composeui-lightswitch
201. The End
202. Compose UI for… a light switch? The End Jake Wharton.com
203. a light switch? Compose UI for… Jake Wharton.com