Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Android Graphics Performance

Android Graphics Performance

Tips, tricks, tools, and techniques used by engineers from the Android UI Graphics team for getting the best performance and smoothest UI from Android applications.

Video available at:
https://developers.google.com/events/io/sessions/325418001

Romain Guy

May 16, 2013
Tweet

More Decks by Romain Guy

Other Decks in Programming

Transcript

  1. & Graphics Performance Welcome/Android Graphics & Performance Chet Haase &

    Romain Guy, Android Framework engineers (Graphics & animations)
  2. & Chet Haase Romain Guy Welcome/Android Graphics & Performance Chet

    Haase & Romain Guy, Android Framework engineers (Graphics & animations)
  3. Reordering & merging A future update of Android introduces a

    major change in the hardware accelerated 2D rendering pipeline: re-ordering and merging of drawing commands
  4. Save Cancel Maximize compatibility Include metadata This allows for more

    efficient rendering without changing anything in your application. Here is an example of a simple Android UI
  5. Until now Android would always render drawing commands in order.

    Android now first re-orders the commands to find optimal batches for the GPU. This avoids changing GPU state. Then Android merges commands together to reduce the number of draw calls to the GPU. In this example we start with 8 in-order calls and end up with 3 out-of- order draw calls.
  6. Include metadata Maximize compatibility Cancel Save Order of drawing commands

    Until now Android would always render drawing commands in order. Android now first re-orders the commands to find optimal batches for the GPU. This avoids changing GPU state. Then Android merges commands together to reduce the number of draw calls to the GPU. In this example we start with 8 in-order calls and end up with 3 out-of- order draw calls.
  7. Include metadata Maximize compatibility Cancel Save 1. Re-ordering Until now

    Android would always render drawing commands in order. Android now first re-orders the commands to find optimal batches for the GPU. This avoids changing GPU state. Then Android merges commands together to reduce the number of draw calls to the GPU. In this example we start with 8 in-order calls and end up with 3 out-of- order draw calls.
  8. Include metadata Maximize compatibility Cancel Save 2. Merging Until now

    Android would always render drawing commands in order. Android now first re-orders the commands to find optimal batches for the GPU. This avoids changing GPU state. Then Android merges commands together to reduce the number of draw calls to the GPU. In this example we start with 8 in-order calls and end up with 3 out-of- order draw calls.
  9. Multi-threading In a future update of Android, the hardware accelerated

    2D rendering pipeline makes better use of multi-core systems by using multiple threads to perform certain tasks.
  10. The blue bar show the UI thread performing normal rendering.

    However the renderer now uses Renderscript to generate drop shadows and thus use all the available core (4 cores in this example.) Paths are also now generated on background threads, as shown in yellow.
  11. Drawing The blue bar show the UI thread performing normal

    rendering. However the renderer now uses Renderscript to generate drop shadows and thus use all the available core (4 cores in this example.) Paths are also now generated on background threads, as shown in yellow.
  12. Drawing Shadows Shadows Shadows Shadows The blue bar show the

    UI thread performing normal rendering. However the renderer now uses Renderscript to generate drop shadows and thus use all the available core (4 cores in this example.) Paths are also now generated on background threads, as shown in yellow.
  13. Drawing Paths Paths Shadows Shadows Shadows Shadows The blue bar

    show the UI thread performing normal rendering. However the renderer now uses Renderscript to generate drop shadows and thus use all the available core (4 cores in this example.) Paths are also now generated on background threads, as shown in yellow.
  14. Non-rectangular clipping A future update of Android adds support for

    hardware accelerated non-rectangular clipping This includes clipping with paths (circles, curves, rounded rects, etc.) and transformed rects (3D rotations for instance)
  15. @Override protected void onDraw(Canvas canvas) { // Clip with a

    shape Path clip = getPath(); canvas.clipPath(clip); // Draw the content for (int i = 0; i < mLines,length; i++) { TextLine line = mLines[i]; canvas.drawText(line.text, line.x, line.y, mPaint); } } Non-rectangular clips can also be achieved by specifying the Region.Op parameter of the various clip*() methods (you can add, xor, subtract, etc.). Non-rect clipping can be triggered by rotations.
  16. Overdraw In developer options you can now turn on a

    “Show GPU overdraw” feature Apps will be highlighted with various colors. Each color represents the amount of overdraw for each pixel. 1x means the pixel was drawn twice, 2x means the pixel was drawn 3 times, etc.
  17. Overdraw In developer options you can now turn on a

    “Show GPU overdraw” feature Apps will be highlighted with various colors. Each color represents the amount of overdraw for each pixel. 1x means the pixel was drawn twice, 2x means the pixel was drawn 3 times, etc.
  18. Overdraw In developer options you can now turn on a

    “Show GPU overdraw” feature Apps will be highlighted with various colors. Each color represents the amount of overdraw for each pixel. 1x means the pixel was drawn twice, 2x means the pixel was drawn 3 times, etc.
  19. Overdraw Blue Green Red Deep Red 1x 2x 3x 4x

    In developer options you can now turn on a “Show GPU overdraw” feature Apps will be highlighted with various colors. Each color represents the amount of overdraw for each pixel. 1x means the pixel was drawn twice, 2x means the pixel was drawn 3 times, etc.
  20. 0 3 6 9 12 15 Time in ms Frames

    Update display lists Process display lists Swap buffers In Android 4.1 we introduced a new rendering profiling tool in developer settings This tool was useful to create performance graphs such as this one It was however difficult to use since it required command line tools and a spreadsheet
  21. In a future update of Android profiling is a lot

    easier. You can now see the profiling graph directly on screen. The old method is also available. The green line indicates the 16ms threshold (60fps.)
  22. In a future update of Android profiling is a lot

    easier. You can now see the profiling graph directly on screen. The old method is also available. The green line indicates the 16ms threshold (60fps.)
  23. performTraversals draw getDL drawDisplayList systrace flush drawing commands systrace was

    introduced in Android 4.1 and we’ve added several improvements in 4.2 and future updates of Android.
  24. android:sdk $ cd platform-tools/ android:platform-tools $ 㾑 ./systrace.py gfx view

    freq sched In a future update of Android systrace is also easier to run, no need to change developer options All you need is to run the script from the command line In this example we’re tracing graphics (gfx), UI toolkit (view), CPU frequencies (freq) and the kernel scheduler (sched)
  25. And since systrace is easier to use, we added more

    information. If you enable OpenGL tracing in developer options you will be able to see each individual OpenGL call in systrace (use the gfx tag when running systrace.) You can also enable glGetError() checks after each GL call.
  26. import android.os.Trace; @Override public View getView(int pos, View view, ViewGroup

    parent) { Trace.beginSection("getView"); if (view == null) { view = createView(); } // Trace time spent binding data Trace.beginSection("bind"); bindView(pos, view); Trace.endSection(); Trace.endSection(); return view; } We’re introducing a new API, android.os.Trace, to let you add your own information to systrace. Systrace events are lightweight and add almost no overhead. Using it is easy: simply wrap the section of code to trace with beginSection/endSection. You can nest sections.
  27. android:sdk $ cd platform-tools/ android:platform-tools $ 㾑 ./systrace.py -a com.example.myapp

    You won’t see your events by default, you must specify the package name of your app using the -a option when invoking systrace. You can specify several packages and combine it with the built-in tags such as gfx and view.
  28. Trilinear filtering Trilinear filtering was introduced in Android 4.2 and

    can be used to improve the visual quality of your bitmaps if you scale them down to less than 50% of their original size
  29. Here is an example with trilinear filtering turned off (left)

    and on (right) You can see the trilinear filtering does a much better job at smoothing the image (The images have been scaled up many times)
  30. Off On Here is an example with trilinear filtering turned

    off (left) and on (right) You can see the trilinear filtering does a much better job at smoothing the image (The images have been scaled up many times)
  31. private void loadData() { // Load bitmap Bitmap b =

    getBitmap(); // Enable trilinear filtering b.setHasMipMap(true); } Here is how you can enable trilinear filtering on a Bitmap You can also enable trilinear filtering on a BitmapDrawable if you prefer
  32. Canvas layers Let’s talk a bit about Canvas layers (as

    opposed to View.setLayerType.) There are two types of layers and using them can create interesting performance issues
  33. @Override protected void onDraw(Canvas canvas) { // Create a clipped

    layer canvas.save(); canvas.saveLayer(x, y, width, height, Canvas.CLIP_TO_LAYER_SAVE_FLAG); // Draw stuff canvas.drawBitmap(bugDroid, 0.0f, 0.0f, null); canvas.restore(); } The first type of layers is called “clipped layer”. Such layers are created by passing the flag Canvas.CLIP_TO_LAYER_SAVE_FLAG when calling Canvas.saveLayer().
  34. saveLayer() Here is how clipped layers work. After calling saveLayer(),

    only the drawing commands that intersect with that layer will be rendered. They are also rendered only in that layer. In practice this means the renderer must create an offscreen render target (bitmap in software, FBO+texture in hardware.)
  35. Here is how clipped layers work. After calling saveLayer(), only

    the drawing commands that intersect with that layer will be rendered. They are also rendered only in that layer. In practice this means the renderer must create an offscreen render target (bitmap in software, FBO+texture in hardware.)
  36. Here is how clipped layers work. After calling saveLayer(), only

    the drawing commands that intersect with that layer will be rendered. They are also rendered only in that layer. In practice this means the renderer must create an offscreen render target (bitmap in software, FBO+texture in hardware.)
  37. @Override protected void onDraw(Canvas canvas) { // Create an unclipped

    layer canvas.save(); canvas.saveLayer(x, y, width, height, 0); // Draw stuff canvas.drawBitmap(bugDroid, 0.0f, 0.0f, null); canvas.restore(); } The second type of layers is called “unclipped layer”. Such layers are created by NOT passing the flag Canvas.CLIP_TO_LAYER_SAVE_FLAG when calling Canvas.saveLayer().
  38. saveLayer() With unclipped layers, a drawing command will be drawn

    in any layer it intersects, including the original canvas. In this example you can see the bitmap is drawn in both the layer we’ve created and the original drawing surface. This is how fading edges are implemented on Android. Unclipped layers are expensive: each command is executed N times (N=number of active layers.)
  39. With unclipped layers, a drawing command will be drawn in

    any layer it intersects, including the original canvas. In this example you can see the bitmap is drawn in both the layer we’ve created and the original drawing surface. This is how fading edges are implemented on Android. Unclipped layers are expensive: each command is executed N times (N=number of active layers.)
  40. With unclipped layers, a drawing command will be drawn in

    any layer it intersects, including the original canvas. In this example you can see the bitmap is drawn in both the layer we’ve created and the original drawing surface. This is how fading edges are implemented on Android. Unclipped layers are expensive: each command is executed N times (N=number of active layers.)
  41. Using alpha with care Android offers many easy ways to

    apply alpha on a view. Using alpha can however have a measurable impact the performance of your application.
  42. Each line shows a different way to set alpha on

    a View. Some will trigger an animation, some won’t. What matters is they all trigger the same side effect: the creation of an expensive offscreen buffer.
  43. view.setAlpha(0.5f); View.ALPHA.set(view, 0.5f); ObjectAnimation.ofFloat(view, "alpha", 0.5f) view.animate().alpha(0.5f); view.setAnimation(new AlphaAnimation(1.0f, 0.5f));

    Each line shows a different way to set alpha on a View. Some will trigger an animation, some won’t. What matters is they all trigger the same side effect: the creation of an expensive offscreen buffer.
  44. view.setAlpha(0.5f); View.ALPHA.set(view, 0.5f); ObjectAnimation.ofFloat(view, "alpha", 0.5f) view.animate().alpha(0.5f); view.setAnimation(new AlphaAnimation(1.0f, 0.5f));

    Canvas.saveLayerAlpha(l, t, r, b, 127, Canvas.CLIP_TO_LAYER_SAVE_FLAG); == Each line shows a different way to set alpha on a View. Some will trigger an animation, some won’t. What matters is they all trigger the same side effect: the creation of an expensive offscreen buffer.
  45. Why is it important to use a separate buffer to

    apply the alpha? Here is an example. Let’s imagine a View containing 3 photos (bitmaps.) If we animate the opacity to 50% without using a separate buffer the result would look like this one.
  46. Why is it important to use a separate buffer to

    apply the alpha? Here is an example. Let’s imagine a View containing 3 photos (bitmaps.) If we animate the opacity to 50% without using a separate buffer the result would look like this one.
  47. Instead of applying the alpha separately, we can use a

    separate buffer in which we draw the 3 photos and then we apply alpha to that buffer. The result can be seen above and looks a lot better.
  48. Instead of applying the alpha separately, we can use a

    separate buffer in which we draw the 3 photos and then we apply alpha to that buffer. The result can be seen above and looks a lot better.
  49. // Not this textView.setAlpha(alpha); // But this int newTextColor =

    (int) (0xFF * alpha) << 24 | baseTextColor & 0xFFFFFF; textView.setTextColor(newTextColor); If you are setting alpha on a TextView and you don’t have a background you can simply bake the alpha in the text color instead. This also applies to background colors, drawing primitives, etc.
  50. // Not this imageView.setAlpha(alpha); // But this imageView.setImageAlpha((int) (alpha *

    255)); Similarly with ImageView, you can apply the alpha directly to the image instead of the View itself.
  51. // Not this customView.setAlpha(alpha); // But this int alpha =

    (int) (255 * slider.getProgress() / 100.0f); paint.setAlpha(alpha); canvas.draw*(..., paint); If you are writing a custom view and the drawing commands don’t overlap, you can simply set the alpha on the paint.
  52. // Or use a layer view.setLayerType(View.LAYER_TYPE_HARDWARE, null); // Transient layer

    view.animate().alpha(0).withLayer(); By far the easiest thing to do is to use a hardware layer. The extra buffer copy will happen only once (or every time the view changes) instead of on every frame.
  53. // API level 16+ @Override public boolean hasOverlappingRendering() { //

    Don't lie to us! return false; } If you have a custom view and you know your content does not overlap, override this method and return false. This will trigger an automatic optimization in the hardware rendering pipeline that will bypass the use of a separate buffer.
  54. 640 dp 400 dp Canvas Canvas is the API used

    to draw on Android You should however be very careful with its dimensions
  55. @Override protected void onDraw(Canvas canvas) { // Get the dimensions

    of the Canvas int w = canvas.getWidth(); int h = canvas.getHeight(); canvas.drawRect(0, 0, w, h, mPaint); } Here is code I have seen in many applications The result of these calls might surprise you
  56. 1280 px 800 px View 300 px 600 px Canvas

    is the API used to draw on Android You should however be very careful with its dimensions What will the previous code sample yield as a result?
  57. In hardware, the API returns the dimensions of the View

    In software, the API returns the dimensions of the window... or the Bitmap if you draw into a bitmap
  58. With hardware rendering 600x300 px (size of the View) In

    hardware, the API returns the dimensions of the View In software, the API returns the dimensions of the window... or the Bitmap if you draw into a bitmap
  59. With hardware rendering With software rendering 600x300 px 1280x800 px

    (size of the View) (size of the window) In hardware, the API returns the dimensions of the View In software, the API returns the dimensions of the window... or the Bitmap if you draw into a bitmap
  60. @Override protected void onDraw(Canvas canvas) { // Keep the jellybeans

    canvas.clipRect(l, t, r, b); // Rotate the jar canvas.rotate(-30.0f, pX, pY); // Draw the jar canvas.drawBitmap(mJellyBeans, x, y, null); } You should be very careful when applying rotations with clipping Here is a simple example of code that clips first, then rotates before drawing a bitmap
  61. The previous code will produce something like this. Note that

    the clip rect remains screen aligned. This is trivial to handle for the OpenGL pipeline, a simple scissoring operation.
  62. 1. Clip The previous code will produce something like this.

    Note that the clip rect remains screen aligned. This is trivial to handle for the OpenGL pipeline, a simple scissoring operation.
  63. 2. Rotate The previous code will produce something like this.

    Note that the clip rect remains screen aligned. This is trivial to handle for the OpenGL pipeline, a simple scissoring operation.
  64. 3. Draw The previous code will produce something like this.

    Note that the clip rect remains screen aligned. This is trivial to handle for the OpenGL pipeline, a simple scissoring operation.
  65. The previous code will produce something like this. Note that

    the clip rect remains screen aligned. This is trivial to handle for the OpenGL pipeline, a simple scissoring operation.
  66. @Override protected void onDraw(Canvas canvas) { // Rotate the jar

    canvas.rotate(-30.0f, pX, pY); // Keep the jellybeans canvas.clipRect(l, t, r, b); // Draw the jar canvas.drawBitmap(mJellyBeans, x, y, null); } What if we change the order of operations and rotate then clip?
  67. This is a much more expensive operation. As you can

    see the rectangle is not screen aligned anymore. We need to take a more expensive code path that relies on the stencil buffer.
  68. 1. Rotate This is a much more expensive operation. As

    you can see the rectangle is not screen aligned anymore. We need to take a more expensive code path that relies on the stencil buffer.
  69. 2. Clip This is a much more expensive operation. As

    you can see the rectangle is not screen aligned anymore. We need to take a more expensive code path that relies on the stencil buffer.
  70. 3. Draw This is a much more expensive operation. As

    you can see the rectangle is not screen aligned anymore. We need to take a more expensive code path that relies on the stencil buffer.
  71. This is a much more expensive operation. As you can

    see the rectangle is not screen aligned anymore. We need to take a more expensive code path that relies on the stencil buffer.
  72. Stencil buffer The stencil buffer is a mask in the

    GPU. To use the stencil buffer we must clear it and draw into it. This means that transformed clipping operations cause extra drawing commands to be executed This also means we’re using more fillrate
  73. Stencil buffer The stencil buffer is a mask in the

    GPU. To use the stencil buffer we must clear it and draw into it. This means that transformed clipping operations cause extra drawing commands to be executed This also means we’re using more fillrate
  74. 640 px 400 px View Imagine a view with the

    following dimensions Let’s now invalidate that View partially
  75. Invalidate 640 px 400 px (170,125) (470,275) Imagine a view

    with the following dimensions Let’s now invalidate that View partially
  76. @Override protected void onDraw(Canvas canvas) { // Query the current

    clip Rect clip = canvas.getClipBounds(); // ??? Log.d("I/O", "clip = " + clip); } What would you expect the getClipBounds() API to return? (Note: use the variant that takes a REct to avoid allocations)
  77. In hardware, the API returns the dimensions of the View

    In software, the API returns the dimensions of the dirty rect Discussion about multiple invalidates (dirty unions)
  78. With hardware rendering 0, 0, 640, 400 (bounds of the

    View) In hardware, the API returns the dimensions of the View In software, the API returns the dimensions of the dirty rect Discussion about multiple invalidates (dirty unions)
  79. With hardware rendering With software rendering 0, 0, 640, 400

    170,125, 470, 275 (bounds of the View) (bounds of the dirty rect) In hardware, the API returns the dimensions of the View In software, the API returns the dimensions of the dirty rect Discussion about multiple invalidates (dirty unions)
  80. Reordering barriers Reordering and merging can optimize your applications but

    specific Canvas calls can prevent reordering. We refer to these calls as “barriers”. Reordering can only happen before or after but cannot work across barriers.
  81. Non-rectangular clips The first type of barrier is caused by

    non-rectangular clips. Be careful as non-rect clips can be introduced by rotations. For instance, rotating a View without setting a layer on that view first will cause non-rect clipping to occur.
  82. saveLayer() The second type of barrier is created by calling

    saveLayer() (or its variant saveLayerAlpha().) Be careful as saveLayerAlpha() can be triggered on your behalf by calling View.setAlpha() as discussed earlier.
  83. More info Parleys.com For Butter or Worse Google I/O 2012

    Various Android GUI & performance talks Accelerated Android Rendering Google I/O 2011
  84. More info Romain’s Tips & Tricks www.curious-creature.org Chet’s Tips &

    Tricks goo.gl/y9JZr Android Performance Case Study graphics-geek.blogspot.com