$30 off During Our Annual Pro Sale. View Details »

How to use Macrobenchmark

How to use Macrobenchmark

Google I/O extended Android Korea 2024 에서 벤치마크 사용 설명서란 주제로 발표한 내용입니다. https://festa.io/events/5509

Veronikapj

July 20, 2024
Tweet

More Decks by Veronikapj

Other Decks in Programming

Transcript

  1. ஏ੿ਸ ా೧ ୡӝ ࢿמਸ ࢸ੿೤פ׮. ޙઁܳ ੌਵఃח ਗੋਸ ഛੋ೤פ׮. ஏ੿ೞӝ

    ਗੋ ౵ঈ ࢿמ ޙઁܳ ೧Ѿೞח ߑߨ ഛੋೠ ղਊী ٮۄ ௏٘ܳ ࣻ੿೤פ׮. ௏٘ ࣻ੿ ࢿמਸ ஏ੿ೞҊ ୡӝ ࢿמҗ ࠺Ү೤פ׮. ୡӝ ࢿמҗ ࠺Ү 🙆 Intro
  2. Intro ௏٘ী ऀয੓ח Unstable Parameters ܳ ݽف ୶੸ೞҊ ࣻ੿ೞ૑ ݃ࣁਃ.

    🙅 хਵ۽ ନ૑ ݈ӝ! ೧׼ ஹನ੷࠶੄ ܻஹನ૑࣌ਸ ਬߊೞח গפݫ੉࣌ਸ ࢏ઁೞ૑݃ࣁਃ. ૒хী ٮۄ ੍ӝ য۰਍ ୭੸ചܳ प೯ೞ૑ ݃ࣁਃ.
  3. ஏ੿ਸ ా೧ ୡӝ ࢿמਸ ࢸ੿೤פ׮. ޙઁܳ ੌਵఃח ਗੋਸ ഛੋ೤פ׮. ஏ੿ೞӝ

    ਗੋ ౵ঈ ࢿמ ޙઁܳ ೧Ѿೞח ߑߨ Intro ഛੋೠ ղਊী ٮۄ ௏٘ܳ ࣻ੿೤פ׮. ௏٘ ࣻ੿ ࢿמਸ ஏ੿ೞҊ ୡӝ ࢿמҗ ࠺Ү೤פ׮. ୡӝ ࢿמҗ ࠺Ү
  4. ࢿמ ޙઁܳ ખ ؊ औѱ ଺ӝ ਤೠ ஏ੿ী ؀ೠ ੉ঠӝ

    ൓݃ߨ੉ ইתפ׮. 🧙 # Performance
  5. Intro জ ղࠗ ز੘ ঌ ೙ਃ হ੉ UiAutomatorܳ ࢎ ਊ೧ࢲ

    ࢿמਸ ஏ੿೤פ׮. Jetpack Macrobenchmark ਬ੷о ࠁח Ѫҗ э਷ ࢿמਸ ഛੋ೤פ׮. Performance ബҗ ஏ੿ਸ ਤೠ ୶ୌೞח ߑߨ ࢿמ ஏ੿ पઁ ࢎਊ ജ҃੄ ࢿמ CIীࢲ प೯ ଵҊ : పझ౟ী ؀ೠ ӝࠄ੸ੋ ੉೧
  6. ޖ঺ਸ ஏ੿ ೡ ࣻ ੓աਃ? Intro জ੉ द੘ೡ ٸ৬ ੹୓

    ۽٘ ؼ ٸ ঴݃ա Ѧܻח૑ Startup Timing 01 Frame Timing 02 Frameਸ ࢤࢿೞחؘ ঴݃ա Ѧܻח૑ Trace Section 03 ׮নೠ ஏ੿ਸ ೧ࠅ ࣻ ੓ח ׮ਊب API Memory Usage 04 জ੉ ঴݃ա ݆਷ ݫݽܻܳ ࢎਊೞҊ ੓ח૑ Power 05 জ੉ ঴݃ա ݆਷ ߓఠܻܳ ࣗݽೞҊ ੓ח૑ + https://developer.android.com/topic/performance/benchmarking/macrobenchmark-metrics
  7. ஏ੿ೞӝ Macrobenchmark ݽٕ Macrobenchmarkח জ ௏٘৬ ܻ࠙ غয জ ஏ੿

    పझ౟ प೯ਸ ׸׼ೞח com.android.test ݽٕ੉ ೙ਃ೤פ׮.
  8. ஏ੿ೞӝ Layout ҳઑ • Column : HeavyItem • PublishedText •

    PublishDate.format • Layout : ImageDisplay • ImagePlaceHolder • ItemTag
  9. @RunWith(AndroidJUnit4::class) class HeavyScreenBenchmark { @get:Rule val benchmarkRule = MacrobenchmarkRule() @Test

    fun accelerateHeavyScreenCompilation() = benchmark(CompilationMode.Partial()) fun benchmark(compilationMode: CompilationMode) { rule.measureRepeated( packageName = "com.compose.performance", metrics = listOf( FrameTimingMetric(), TraceSectionMetric("HeavyItem", TraceSectionMetric.Mode.Sum) ), compilationMode = compilationMode, startupMode = StartupMode.WARM, iterations = 10, setupBlock = { setupBlock() }, measureBlock = { measureBlock() } ) } }
  10. @RunWith(AndroidJUnit4::class) class HeavyScreenBenchmark { @get:Rule val benchmarkRule = MacrobenchmarkRule() @Test

    fun accelerateHeavyScreenCompilation() fun benchmark(compilationMode: CompilationMode) { } } class MacrobenchmarkRule : TestRule JUnit ӏ஗ਸ ࢎਊೞৈ গ೒ܻா੉࣌੄ द੘, झ௼܀݂ ژח গפݫ੉࣌җ э਷ ؀ӏݽ ੘সਸ ߮஖݃ఊೞח ߑߨਸ ઁҕ೤פ׮. https://developer.android.com/reference/kotlin/androidx/benchmark/macro/junit4/MacrobenchmarkRule
  11. @RunWith(AndroidJUnit4::class) class HeavyScreenBenchmark { @get:Rule val benchmarkRule = MacrobenchmarkRule() @Test

    fun accelerateHeavyScreenCompilation() = benchmark(CompilationMode.Partial()) fun benchmark(compilationMode: CompilationMode) { rule.measureRepeated( packageName = "com.compose.performance", metrics = listOf( FrameTimingMetric(), TraceSectionMetric("HeavyItem", TraceSectionMetric.Mode.Sum) ), compilationMode = compilationMode, startupMode = StartupMode.WARM, iterations = 10, setupBlock = { setupBlock() }, measureBlock = { measureBlock() } ) } }
  12. @RunWith(AndroidJUnit4::class) class HeavyScreenBenchmark { @Test fun accelerateHeavyScreenCompilation() = benchmark(CompilationMode.Partial()) }

    https://developer.android.com/reference/androidx/benchmark/macro/CompilationMode CompilationMode.Partial জ द੘ ࢿמ ஏ੿ೞҊ ੉೧ೞחؘ ࢎਊ (Default) CompilationMode.Full জ प೯ ઺ runtime ࢿמ ࠺Ү ߂ ѐࢶ JIT ୭੸ച۽ ੋೠ ࢿמ ߸ز ਃੋ ઁѢ CompilationMode.None ࢎ੹ ஹ౵ੌ হ਺. Baseline Pro fi le җ Benchmark ղীࢲ ࠺Ү CompilationMode
  13. @RunWith(AndroidJUnit4::class) class HeavyScreenBenchmark { @get:Rule val benchmarkRule = MacrobenchmarkRule() @Test

    fun accelerateHeavyScreenCompilation() = benchmark(CompilationMode.Partial()) fun benchmark(compilationMode: CompilationMode) { rule.measureRepeated( packageName = "com.compose.performance", metrics = listOf( FrameTimingMetric(), TraceSectionMetric("HeavyItem", TraceSectionMetric.Mode.Sum) ), compilationMode = compilationMode, startupMode = StartupMode.WARM, iterations = 10, setupBlock = { setupBlock() }, measureBlock = { measureBlock() } ) } }
  14. @RunWith(AndroidJUnit4::class) class HeavyScreenBenchmark { fun benchmark(compilationMode: CompilationMode) { rule.measureRepeated( .

    . . startupMode = StartupMode.WARM, } Sta rt upMode https://developer.android.com/topic/performance/benchmarking/macrobenchmark-metrics StartupMode.COLD Sta rt up Benchmarkী ࢎਊ setupBlock৬ mesureBlock ࢎ੉ী জ ೐۽ࣁझ ઙܐ StartupMode.WARM Frame timing Benchmarkী ࢎਊ setupBlock റী ೐۽ࣁझо ઙܐغ૑ ঋ਺ ೐۽ࣁझܳ ઙܐೞ૑ ঋҊ ݽٚ प೯ ઺ੋ Activityܳ ײ਺ Activity द੘ ߑधҗ పझ౟ द੘ द੄ ೐۽ࣁझ ࢚క ߸҃ StartupMode.HOT п Activity ߹ நय ݫழפ્ਸ ஏ੿ೞחؘ ࢎਊ ೐۽ࣁझ, ੉੹ী प೯ ઺੉؍ Activityب ੤द੘ غ૑ ঋ਺ Null Macrobenchmark о ইޖѪب ೞ૑ ঋח ݽ٘ killProcess() ۽ ૒੽ ஶ౟܀
  15. @RunWith(AndroidJUnit4::class) class HeavyScreenBenchmark { @get:Rule val benchmarkRule = MacrobenchmarkRule() @Test

    fun accelerateHeavyScreenCompilation() = benchmark(CompilationMode.Partial()) fun benchmark(compilationMode: CompilationMode) { rule.measureRepeated( packageName = "com.compose.performance", metrics = listOf( FrameTimingMetric(), TraceSectionMetric("HeavyItem", TraceSectionMetric.Mode.Sum) ), compilationMode = compilationMode, startupMode = StartupMode.WARM, iterations = 10, setupBlock = { setupBlock() }, measureBlock = { measureBlock() } ) } } SetupBlock ਗೞח ஏ੿ ചݶ ੹ Frame ө૑੄ ز੘ (۽Ӓੋ ١)
  16. @RunWith(AndroidJUnit4::class) class HeavyScreenBenchmark { @get:Rule val benchmarkRule = MacrobenchmarkRule() @Test

    fun accelerateHeavyScreenCompilation() = benchmark(CompilationMode.Partial()) fun benchmark(compilationMode: CompilationMode) { rule.measureRepeated( packageName = "com.compose.performance", metrics = listOf( FrameTimingMetric(), TraceSectionMetric("HeavyItem", TraceSectionMetric.Mode.Sum) ), compilationMode = compilationMode, startupMode = StartupMode.WARM, iterations = 10, setupBlock = { setupBlock() }, measureBlock = { measureBlock() } ) } } MeasureBlock ஏ੿ਸ ਗೞח ز੘
  17. MeasureBlock ஏ੿ਸ ਗೞח ز੘ override fun MacrobenchmarkScope.measureBlock() { pressHome() startTaskActivity("accelerate_heavy")

    device.wait( Until.hasObject(By.res("list_of_items")), 5_000) val feed = device.findObject(By.res("list_of_items")) feed.setGestureMargin(device.displayWidth / 5) repeat(2) { feed.drag( Point( feed.visibleCenter.x, feed.visibleBounds.top ) ) Thread.sleep(500) } }
  18. MeasureBlock TestTagܳ ࠢৈࢲ ਗೞח resource ଺ইоӝ @Composable fun ScreenContent(items: List<HeavyItem>)

    { LazyVerticalGrid( modifier = Modifier .fillMaxSize() .testTag(“list_of_items"), . . . ) { items(items) { item -> HeavyItem(item) } } } device.wait(Until.hasObject(By.res("list_of_items")), 5_000)
  19. MeasureBlock TestTagܳ ࠢৈࢲ ਗೞח resource ଺ইоӝ @Composable fun ScreenContent(items: List<HeavyItem>)

    { LazyVerticalGrid( modifier = Modifier .fillMaxSize() .testTag(“list_of_items"), . . . ) { items(items) { item -> HeavyItem(item) } } } device.wait(Until.hasObject(By.res("list_of_items")), 5_000)
  20. MeasureBlock ஏ੿ਸ ਗೞח ز੘ - झ௼܀ override fun MacrobenchmarkScope.measureBlock() {

    pressHome() startTaskActivity("accelerate_heavy") device.wait( Until.hasObject(By.res("list_of_items")), 5_000) val feed = device.findObject(By.res("list_of_items")) feed.setGestureMargin(device.displayWidth / 5) repeat(2) { feed.drag( Point( feed.visibleCenter.x, feed.visibleBounds.top ) ) Thread.sleep(500) } }
  21. @RunWith(AndroidJUnit4::class) class HeavyScreenBenchmark { @get:Rule val benchmarkRule = MacrobenchmarkRule() @Test

    fun accelerateHeavyScreenCompilation() = benchmark(CompilationMode.Partial()) fun benchmark(compilationMode: CompilationMode) { rule.measureRepeated( packageName = "com.compose.performance", metrics = listOf( FrameTimingMetric(), TraceSectionMetric("HeavyItem", TraceSectionMetric.Mode.Sum) ), compilationMode = compilationMode, startupMode = StartupMode.WARM, iterations = 10, setupBlock = { setupBlock() }, measureBlock = { measureBlock() } ) } } ஏ੿ ೦ݾ https://developer.android.com/topic/performance/benchmarking/macrobenchmark-metrics
  22. @RunWith(AndroidJUnit4::class) class HeavyScreenBenchmark { fun benchmark(compilationMode: CompilationMode) { rule.measureRepeated( .

    . . metrics = listOf( FrameTimingMetric() . . .), } frameDurationCpuMs P50 11.9, P90 24.0, P95 30.3, P99 90.6 frameOverrunMs P50 3.5, P90 21.2, P95 25.9, P99 252.0 FrameTimingMetric 👉 CPUীࢲ UI झۨ٘৬ RenterThreadীࢲ ೐ۨ੐੉ ࢤࢿغחؘ Ѧܽ दр https://developer.android.com/topic/performance/benchmarking/macrobenchmark-metrics
  23. @RunWith(AndroidJUnit4::class) class HeavyScreenBenchmark { fun benchmark(compilationMode: CompilationMode) { rule.measureRepeated( .

    . . metrics = listOf( FrameTimingMetric() . . .), } frameDurationCpuMs P50 11.9, P90 24.0, P95 30.3, P99 90.6 frameOverrunMs P50 3.5, P90 21.2, P95 25.9, P99 252.0 FrameTimingMetric 👉 CPUীࢲ UI झۨ٘৬ RenterThreadীࢲ ೐ۨ੐੉ ࢤࢿغחؘ Ѧܽ दр 👉 ઱য૓ ೐ۨ੐੉ ӝೠਸ ֈӟ दр +(নࣻ)ੌ ࣻ۾ ߡߢѢܿ, - (਺ࣻ)ח ঴݃ա ࡅܲ૑ Android 12(API 31) https://developer.android.com/topic/performance/benchmarking/macrobenchmark-metrics
  24. ஏ੿ೞӝ TraceSection Metric • Column : HeavyItem • PublishedText •

    PublishDate.format • Layout : ImageDisplay • ImagePlaceHolder • ItemTag Layout ҳઑ ߹۽ ࢸ੿
  25. listOf( FrameTimingMetric(), TraceSectionMetric("HeavyItem", TraceSectionMetric.Mode.Sum), TraceSectionMetric("PublishedText", TraceSectionMetric.Mode.Sum), TraceSectionMetric("PublishDate.format", TraceSectionMetric.Mode.Sum), TraceSectionMetric("ImageDisplay", TraceSectionMetric.Mode.Sum),

    TraceSectionMetric("ImagePlaceholder", TraceSectionMetric.Mode.Sum), TraceSectionMetric("ItemTag", TraceSectionMetric.Mode.Sum) ) @RunWith(AndroidJUnit4::class) class HeavyScreenBenchmark { fun benchmark(compilationMode: CompilationMode) { rule.measureRepeated( . . . metrics = listOf(FrameTimingMetric(). . .), } TraceSectionMetric Trace Section ੉ ઁҕػ sectionName җ ੌ஖ೞח പࣻ৬ ࣗਃػ दр ஭୛ ୭ࣗदр, ઺ঔ ч, ୭؀ दр (ms) h tt ps://developer.android.com/topic/pe rf ormance/benchmarking/macrobenchmark-metrics?hl=ko#trace-section
  26. TraceSectionMetric.Mode TraceSectionMetric("HeavyItem", TraceSectionMetric.Mode.Sum), https://developer.android.com/reference/androidx/benchmark/macro/TraceSectionMetric.Mode Mode.First SectionName ੌ஖ೞח ୐ߣ૩ ࣂ࣌੄ instance

    ࣗਃदр ஭୛ Mode.Sum SectionName ੌ஖ೞח ݽٚ ࣂ࣌ instance ࣗਃ दр ೤҅ Mode.Max SectionName ੌ஖ೞח ੹୓ ࣂ࣌ ઺ о੢ ௾ ч ӝ۾ Mode.Min SectionName ੌ஖ೞח ੹୓ ࣂ࣌ ઺ о੢ ੘਷ ч ӝ۾ Mode.Average SectionName ੌ஖ೞח ੹୓ ࣂ࣌੄ ಣӐ Mode.Count SectionName ੌ஖ೞח ࣂ࣌ ѐࣻ 1.3.0 beta+ 1.3.0 beta+ 1.3.0 beta+ 1.3.0 beta+
  27. ஏ੿ೞӝ _count : SectionName җ ੌ஖ೞח іࣻ _ms : Section

    ஏ੿ػ ࣗਃ दр TraceSectionMetric ஏ੿ Ѿҗ
  28. ஏ੿ೞӝ HeavyItem PublishedText PublishDate.format ImageDisplay ImagePlaceHolder ItemTag Min Median Max

    HeavyItem 46.4 50.2 53.0 PublishedText 6.9 7.7 8.6 PublishDate.format 0.7 0.8 0.9 ImageDisplay 27.1 29.1 30.4 ImagePlaceHolder 17.4 18.6 19.3 ItemTag 2.7 3.6 5.4 TraceSectionMetric ஏ੿ Ѿҗ
  29. Trace ౵ੌ ਤ஖ Measure > build > outputs > connected_android_text_additional_output

    > benchmarkRelease > connected > ‘connected device’ ਗੋ౵ঈ
  30. উ٘۽੉٘ झౚ٣য় प೯ ೙ਃ হ਺ ٣߄੉झ੄ ੹୓ Threadܳ ࠅ ࣻ

    ੓਺ Trace ઺, प೯غח SQL quries ਗੋ౵ঈ Web-based tool Gives you whole picture https://youtu.be/Z96wfbID_Yc?si=pOe8w7hT7IEbsnb-&t=655 Investigate deeper H tt ps://ui.pe rf e tt o.dev/
  31. ਗੋ౵ঈ System-tracing call cha r ੹୓ दр ইې ഐ୹ػ ೣࣻ

    ࣗਃ दр੄ ୨೤ Trace Code Untrace Code
  32. ਗੋ౵ঈ https://developer.android.com/develop/ui/compose/tooling/tracing ಴दػ ৔৉݅ ୶੸ য়ߡ೻٘о ੸Ҋ জ ࢿמী ௾

    ৔ೱਸ ޷ ஖૑ ঋ਺ জ੄ ݽٚ ೣࣻ ഐ୹ਸ ୶੸ জ ࢿמী ௾ ৔ೱਸ ޷஖૑݅ ߊࢤೞ ח ࢚ട, ഐ୹غח ೣࣻ, ഐ୹ ࠼بܳ ੹୓੸ਵ۽ ౵ঈ Composition function ನೣغয ੓૑ ঋ਺ Composition function ನೣ Composition Tracing +
  33. // :app module implementation( “androidx.compose.runtime:runtime-tracing:1.0.0-beta01" ) https://developer.android.com/develop/ui/compose/tooling/tracing // :mesure module

    implementation( “androidx.tracing:tracing-perfetto:1.0.0") implementation( "androidx.tracing:tracing-perfetto-binary:1.0.0") // gradle configuration androidx.benchmark.fullTracing.enable=true
  34. ImagePlaceHolder ѐࢶ 1. ੉޷૑ܳ ࠺زӝ۽ ۽٘ 2. ؊ ࡈܻ ۽٘غب۾

    ੘਷ ੉޷૑ ࢎਊ 3. ೙ਃೠ ௼ӝী ٮۄ ઑ੿غח Vector Drawable ࢎਊ ௏٘ࣻ੿
  35. 1600 x 1600 px @Composable fun imagePlaceholder() = trace("ImagePlaceholder") {

    painterResource(R.drawable.placeholder) } ௏٘ࣻ੿
  36. ѐࢶ Ѿҗ Before A ft er Max ӝળ 19.3 ীࢲ

    4.4 ۽ 77.2% хࣗ ୡӝ ࢿמҗ ࠺Ү
  37. ஏ੿ਸ ా೧ ୡӝ ࢿמਸ ࢸ੿೤פ׮. ޙઁܳ ੌਵఃח ਗੋਸ ഛੋ೤פ׮. ஏ੿ೞӝ

    ਗੋ ౵ঈ ࢿמ ޙઁܳ ೧Ѿೞח ߑߨ ഛੋೠ ղਊী ٮۄ ௏٘ܳ ࣻ੿೤פ׮. ௏٘ ࣻ੿ ࢿמਸ ஏ੿ೞҊ ୡӝ ࢿמҗ ࠺Ү೤פ׮. ୡӝ ࢿמҗ ࠺Ү 🙆 Intro
  38. ஏ੿ਸ ా೧ ୡӝ ࢿמਸ ࢸ੿೤פ׮. ޙઁܳ ੌਵఃח ਗੋਸ ഛੋ೤פ׮. ஏ੿ೞӝ

    ਗੋ ౵ঈ ഛੋೠ ղਊী ٮۄ ௏٘ܳ ࣻ੿೤פ׮. ௏٘ ࣻ੿ ࢿמਸ ஏ੿ೞҊ ୡӝ ࢿמҗ ࠺Ү೤פ׮. ୡӝ ࢿמҗ ࠺Ү ੿ܻ Macrobenchmark ࢸ੿ೞӝ ஏ੿ чٜ੄ ੄޷ ঌইࠁӝ Ѿҗ ч ࠺Үೞӝ Profiler ژח Perfetto chart ੉೧ೞӝ ޖѢ਍ ੘স оߺѱ ѐࢶೞӝ ࣻ੿ റ ബҗо ੓ח૑ ࠺Үೞӝ