Adobe AIR 3.5 Overview (for BlackBerry 10)

Do you want to achieve the best possible performance for your Adobe® AIR® application? Are you a game developer? The Adobe AIR runtime has been updated to version 3.5 in BlackBerry® 10.2! This deck was presented at BlackBerry Jam Asia 2013 in Hong Kong, covering how to achieve 60FPS on games and applications with hardware accelerated Stage3D.

1. Adobe AIR 3.5 Overview JAM852 Demian Borba @demianborba September 26-27, 2013
2. About me Ÿ  Graduated in Industrial Design (CEFET-PR) Ÿ  Graduated in Computer Science (UFAL) Ÿ  Crazy about Design, UX, Business and Technology Ÿ  Professor at University of California San Diego (UCSD) Ÿ  Working with Interactive Media for over 13 years Ÿ  Founder of Action Creations, interactive agency in California Ÿ  Developer Evangelist, 90% #upintheair Ÿ  Launch the Startup Jam program Ÿ  Launched the Tech Centers in Universities 2
3. X Ÿ  X 3
4. Tech Centers 4
5. Agenda Ÿ  Why Adobe AIR? Ÿ  Defining Performance Ÿ  Getting to know the Stage3D API Ÿ  Stage3D libraries, case study: Starling Framework Ÿ  Setting up your environment (Flash Builder or FDT) Ÿ  Creating/Testing your first AIR 3.5 mobile project for BlackBerry 10 Ÿ  Demo: Adobe Gaming SDK Ÿ  Bonus: the BlackBerry 10 Experience App Framework for AIR Ÿ  Bonus: creating BAR files from SWFs (GAP for AIR) Ÿ  What we are trying to solve Ÿ  Questions & Answers 5
6. Why Adobe AIR? 6 Source: Vision Mobile Developer Economics 2013 78% of developers interested in revenue, use two or more platforms concurrently
7. Source: Vision Mobile Developer Economics 2013 Main platforms for developers 7
8. What is it? 8
9. Defining Performance 9 Computer performance is characterized by the amount of useful work accomplished by a computer system compared to the time and resources used. Source: Wikipedia (http://en.wikipedia.org/wiki/Computer_performance)
10. Defining Performance 10 PERFORMANCE UX = Source: Michael Wolf, Director of Technology at Cynergy
11. Understand the guidelines 11 developer.blackberry.com/design/bb10
12. developer.blackberry.com/design/bb10 12
13. Built for BlackBerry http://developer.blackberry.com/builtforblackberry 13
14. Built for BlackBerry : Games 14 http://downloads.blackberry.com/upr/developers/downloads/BuiltForBlackBerry-GameDeveloperChecklist.pdf
15. BlackBerry 10.2 Ÿ  Adobe AIR 3.5 „  Stage3D „  StageVideo „  Computing performance improvements 15
16. Getting to know the Stage3D API Available in BlackBerry 10.2 16
17. The past Ÿ  3D engines available for Flash: „  PaperVision 3D „  Away3D etc Ÿ  There was demand for real-time 3D rendering in Flash Ÿ  Scenario: „  No 3D hardware acceleration „  Rendering using software mode (relied on the CPU) „  Software mode is slow (can’t render detailed 3D scenes) 17
18. After Flash Player 11 And now with BlackBerry 10.2: Ÿ  New opportunities for developers (games and apps) Ÿ  Developers can leverage 3D hardware acceleration Ÿ  No need to rely on the CPU for 3D rendering anymore Ÿ  3D rendering handled by a secondary processor, the GPU Ÿ  GPU (Graphics Processing Unit) „  Part of the video hardware of a computer „  Totally dedicated to rendering objects in 3D 18
19. With Stage3D you can Ÿ  Take full advantage of the user’s GPU directly from Flash Ÿ  Have lightning-fast rendering required for Complex 3D Models Ÿ  Have realistic 3D effects (not possible before) Ÿ  Have a responsive game play Ÿ  Reach more users (Flash is available on almost all computers) Ÿ  Abstract the need of special hardware or runtime libraries Ÿ  You can reach iOS, Android and BlackBerry 10 devices Ÿ  Create immersive 3D experiences (engagement) Ÿ  Can build extremely fluid 2D experiences 19
20. 3D Hardware Acceleration Ÿ  The GPU is included in all modern computers Ÿ  The software (your app) just define the 3D scene Ÿ  It passes the 3D scene data to the GPU Ÿ  The GPU processes the data and renders the scene 20 CPU (responsible to define the scene) GPU (responsible to process and render the scene) Passes the definition to the GPU
21. What makes a 3D scene? Ÿ  A group of 3D geometries (meshes) Ÿ  Each geometry is a set of triangles Ÿ  Each triangle is a set of vertices Ÿ  Eventually some rendering information, such as texture or colors Ÿ  3D scenes can be rendered: „  Using SOFTWARE MODE (older, on the CPU) „  Using HARDWARE MODE (newer, on the GPU) 21
22. Software mode rendering Ÿ  Example with Flash: PaperVision3D Ÿ  The 3D engine: „  receives the 3D information „  receives the stream of vertices „  calculates the screen positions of the triangles „  prompts the Flash Player to natively render each triangle, one by one „  the process repeats in a series of “fill” draw operations Ÿ  Software mode rendering is: „  Slow (Flash Player renders scenes with a max of 4000 triangles) „  Not accurate (render by triangle, not per pixel) „  Triangles would render sometimes out of place 22
23. Hardware mode rendering Ÿ  Examples with Stage3D: Starling (2D), Flare3D Ÿ  The software simply define the geometries (works efficiently) Ÿ  Then passes it to the GPU to be rendered Ÿ  Then the GPU (specialized in 3D), renders the scene = final image Ÿ  The hardware mode rendering is: „  Much faster than software mode (can render more than 1M triangles) 250X „  Handled by the CPU, specialized in 3D rendering „  Receives vertices, renders triangles, that is it! 23
24. 3D rendering pipeline Ÿ  Hardware-based 3D rendering is facilitated by a 3D rendering pipeline Ÿ  Pipeline = Rendering Process Ÿ  Older = Fixed Function Graphics Pipeline (Rigid) Ÿ  Newer = Programmable Graphics Pipeline (Flexible) 24
25. Fixed Function Pipeline Ÿ  Simply receive geometry data as input Ÿ  Process the data through the pipeline blocks Ÿ  Produce the final output rendered image 25
26. Fixed Function Pipeline Ÿ  To work with a fixed function pipeline, you feed the GPU with: „  geometry description (vertices and triangles) „  textures to apply to the geometries „  the position and orientation of the geometries in the 3D scene „  the position and orientation of the eye point (3D camera) „  lights (how many, what color, and what position) „  additional parameters needed to specify how the rendering will take place Ÿ  The projected and clipped data is then passed down the pipeline to the rasterizer stage, which performs the texture mapping. It also eventually applies fogging, alpha blending effects, and performs the depth buffering test necessary for rendering triangle pixels with the proper depth sorted order. 26
27. Fixed Function Pipeline Ÿ  It has worked well for many years, but it is rigid Ÿ  Bad for lightning (has to use standard shading models) Ÿ  Isn’t flexible enough for developers to do creative work Ÿ  Renderings look very similar to each other Ÿ  Then the programmable graphics pipeline was introduced 27
28. Programmable Pipeline 28
29. Programmable Pipeline Ÿ  2 new programmable blocks (Vertex Shader & Fragment Shader) Ÿ  You get to write code snippets (called Shaders) Ÿ  How vertices are transformed and modified = Vertex Shader Ÿ  How triangle pixels are rendered = Fragment Shader Ÿ  You can apply all kinds of lighting techniques Ÿ  Possible to add shadows Ÿ  Possible to add hardware accelerated bones systems Ÿ  Possible to add many cool effects 29
30. Stage3D Ÿ  Stage3D is completely based on the programmable function pipeline Ÿ  With power also comes responsibility: „  You are required to write your own Shaders „  You cannot simply just set a few parameters and render a project as before using the fixed function pipeline Ÿ  Coding can get very complex and time consuming Ÿ  Recommendation? „  Use libraries for Flash such as Away3D, Flare3D, Starling etc 30
31. Advantages of Stage3D Ÿ  When programming for Stage3D, Flash Player abstracts the hardware specific tasks necessary to make it work (ex: Context3D) Ÿ  Different from DirectX or OpenGL, where your application must target and be tuned/tested for several different pieces of hardware Ÿ  Stage3D runs on AIR 3.2 and higher (desktop and mobile) Ÿ  You don't really target specific hardware in order to take advantage of the specific power available in each platform Ÿ  Stage3D acts as a layer between your code and the actual hardware Ÿ  You can combine 3D hardware accelerated code with regular 2D Flash content 31
32. Restrictions of Stage3D Ÿ  Write one piece of code to run anywhere limits hardware specific features (abstraction has to take place) Ÿ  Example: „  Modern GPU hardware supports Shader Model 4.0 „  Stage3D supports Shader Model 2.0 Ÿ  You can face limitations not present on newer hardware Ÿ  As a result, some of the advanced effects using Shaders that you see in today’s AAA games might not be feasible with Stage3D. 32
33. How Stage3D works 33
34. Stage3D Ÿ  Rendered in the rectangular viewport of each specific Stage3D Ÿ  The regular 2D Flash content appears overlaid on top of it Ÿ  You get the rendering of your 3D scene, using hardware acceleration Ÿ  You get the 2D stuff (such as a game UI) on top Ÿ  You have a multitude of Stage3Ds available Ÿ  Each Stage3D has its own rectangular viewport Ÿ  The various Stage3D and StageVideo layers can partially (or even totally) overlap. 34
35. Accessing Stage3D with AS3 Ÿ  To access the Stage3D API using ActionScript code, you’ll declare one of the Stage3D stages Ÿ  The primary class of the Stage3D API is not Stage3D itself. It is a class called Context3D (it contains the surface on which you will render the 3D objects, and includes all the methods and properties needed to perform the rendering) 35 var stage3D:Stage3D = stage.stage3Ds[0];
36. Context3D Ÿ  Context3D is the main entity you'll work with when using Stage3D. Ÿ  The first thing you'll do with a Stage3D object is request a Context3D, like this: stage.stage3Ds[0].addEventListener( Event.CONTEXT3D_CREATE, initStage3D ); stage.stage3Ds[0].requestContext3D(); ... protected function initStage3D(e:Event):void { context3D = stage.stage3Ds[0].context3D; } 36
37. Defining geometry (Vertex) Ÿ  In Stage3D, the 3D scene that you render is comprised of a set of geometries (3D meshes). Each geometry is defined by a set of triangles. And each triangle is defined as a set of vertices. Ÿ  All vertices that describe a geometry are packed together in a structure called a Vertex Buffer Ÿ  It can then be uploaded in one batch to the GPU memory. Sample: 37
38. And it goes on 38 wraps the Vertex Buffer data into the Stage3D API creating an additional Index Buffer (square) use the IndexBuffer3D to upload your index buffer to the GPU
39. Samples 39 http://www.romatica.com/dev/samples/alternativa3d/mario_collada/index.html
40. Raw Stage3D 40 It’s very complex and time consuming!
41. Stage3D libraries Case Study: Starling Framework 41
42. Suggestion Ÿ  Use frameworks to help you with Stage3D, such as: 42
43. Raw Stage3D vs Framework 43 Ÿ  Task: Creating a simple texture in Stage3D
44. Stage3D implementation 44 // create the vertices var vertices:Vector.<Number> = Vector.<Number>([ -0.5,-0.5,0, 0, 0, // x, y, z, u, v -0.5, 0.5, 0, 0, 1, 0.5, 0.5, 0, 1, 1, 0.5, -0.5, 0, 1, 0]); // create the buffer to upload the vertices var vertexbuffer:VertexBuffer3D = context3D.createVertexBuffer(4, 5); // upload the vertices vertexbuffer.uploadFromVector(vertices, 0, 4); // create the buffer to upload the indices var indexbuffer:IndexBuffer3D = context3D.createIndexBuffer(6); // upload the indices indexbuffer.uploadFromVector (Vector.<uint>([0, 1, 2, 2, 3, 0]), 0, 6); // create the bitmap texture var bitmap:Bitmap = new TextureBitmap(); // create the texture bitmap to upload the bitmap var texture:Texture = context3D.createTexture(bitmap.bitmapData.width, bitmap.bitmapData.height, Context3DTextureFormat.BGRA, false); // upload the bitmap texture.uploadFromBitmapData(bitmap.bitmapData); // create the mini assembler var vertexShaderAssembler :AGALMiniAssembler = new AGALMiniAssembler(); // assemble the vertex shader vertexShaderAssembler.assemble( Context3DProgramType.VERTEX, "m44 op, va0, vc0\n" + // pos to clipspace "mov v0, va1" // copy uv );     var fragmentShaderAssembler :AGALMiniAssembler = new AGALMiniAssembler(); // assemble the fragment shader fragmentShaderAssembler.assemble( Context3DProgramType.FRAGMENT, "tex ft1, v0, fs0 <2d,linear, nomip>;\n" + "mov oc, ft1" ); // create the shader program var program:Program3D = context3D.createProgram(); // upload the vertex and fragment shaders program.upload( vertexShaderAssembler.agalcode, fragmentShaderAssembler.agalcode); // clear the buffer context3D.clear ( 1, 1, 1, 1 ); // set the vertex buffer context3D.setVertexBufferAt(0, vertexbuffer, 0, Context3DVertexBufferFormat.FLOAT_3); context3D.setVertexBufferAt(1, vertexbuffer, 3, Context3DVertexBufferFormat.FLOAT_2); // set the texture context3D.setTextureAt( 0, texture ); // set the shaders program context3D.setProgram( program ); // create a 3D matrix var m:Matrix3D = new Matrix3D(); // apply rotation to the matrix to rotate vertices along the Z axis m.appendRotation(getTimer()/50, Vector3D.Z_AXIS); // set the program constants (matrix here) context3D.setProgramConstantsFromMatrix(Context3DProgramType.VERTEX, 0, m, true); // draw the triangles context3D.drawTriangles(indexbuffer); // present the pixels to the screen context3D.present();  
45. Starling implementation // create a Texture object out of an embedded bitmap var _texture:Texture = Texture.fromBitmap ( new EmbeddedBitmap() ); // create an Image object our of the Texture var _image:Image = new Image(_texture); // set the properties _image.pivotX = 50; _image.pivotY = 50; _image.x = 300; _image.y = 150; _image.rotation = Math.PI/4; // display it addChild(_image);   45
46. Working with Starling Ÿ  The Starling framework implements common APIs found in Flash: „  Sprite, Image, Text, Button, MovieClip „  familiar event model „  support for particles effects „  open source (so anyone can add to it) 46 http://gamua.com/starling
47. Sample : Hungry Hero 47 FlashBuilder Hungry Hero
48. Setting up your environment Flash Builder or FDT 48
49. 1. Visit Developer Site http://developer.blackberry.com/air 49
50. 2. Download the SDK http://developer.blackberry.com/air/download 50
51. 3. Using Flash Builder or FDT? Download and Install the PLUGIN http://developer.blackberry.com/air/documentation/install_bb10sdk_air_update_site.html 51
52. 4. Download AIR SDK 3.5 Runtime 3.5 (Windows) http://download.macromedia.com/air/win/download/3.5/AdobeAIRInstaller.exe Runtime 3.5 (Mac) http://download.macromedia.com/air/mac/download/3.5/AdobeAIR.dmg SDK 3.5 (Windows) http://download.macromedia.com/air/win/download/3.5/AIRSDK_Compiler.zip SDK 3.5 (Mac) http://download.macromedia.com/air/mac/download/3.5/AIRSDK_Compiler.tbz2 Merge AIR SDK: http://fdt.powerflasher.com/docs/AIR_SDK_Merging 52
53. Creating/Testing your first AIR 3.5 mobile project for BlackBerry 10 53
54. Demo http://developer.blackberry.com/air/documentation/create_first_app.html package { import flash.display.Sprite; import flash.display.StageAlign; import flash.display.StageScaleMode; import qnx.fuse.ui.text.Label; [SWF(height="1280", width="768", frameRate="60", backgroundColor="#FFFFFF")] public class AIRHelloWorld extends Sprite { public function AIRHelloWorld() { // support autoOrients stage.align = StageAlign.TOP_LEFT; stage.scaleMode = StageScaleMode.NO_SCALE; //Initialize the UI initializeUI(); } private function initializeUI():void { // create a label var myLabel:Label = new Label(); // Define a label width myLabel.width = 200; // set the label to appear at the center of the // screen myLabel.x = (stage.stageWidth - myLabel.width) /2; myLabel.y = 54 FDT Hello World
55. Demo Adobe Gaming SDK 55
56. Demo: Adobe Gaming SDK 56 http://gaming.adobe.com/technologies/gamingsdk
57. Bonus: The BlackBerry 10 Experience App Framework for Adobe AIR 57
58. Download Samples https://github.com/blackberry/Samples-for-AIR/tree/master/appframework 58 List with context menu Tabbed Navigation Sample Navigation Drill Down Simple To Do app FDT
59. Bonus: Creating BAR files from SWFs (GAP for AIR) 59
60. The SWF + Required Files 60
61. Descriptor File 61
62. Screen Sizes 62
63. Vendor Portal 63
64. Demo 64 GAP for AIR
65. Download GAP for AIR 65 GAP for AIR http://dborba.com/download/gapforairmac.zip http://dborba.com/download/gapforairwin.zip How to use GAP for AIR: http://dborba.com
67. A P P ?
68. Your BlackBerry 10 app can 68 • Bluetooth 4.0 • Wifi • NFC • USB • HDMI • DLNA • Triggers SENSE UNDERSTAND ADAPT • BBM • Files • Screen • Video • Game GPS Accelerometer Altimeter Gyroscope Camera Screens Keyboard Cascades UI Framework My Location Geofence Places Map Automotive Hub Contacts Calendar Notifications UI Outputs based on Sensor Inputs and User Preferences CONNECT SHARE
69. D O I T
70. Suggestions Ÿ  Find a real world problem Ÿ  Never bring TECHNOLOGY before the PROBLEM Ÿ  Analyze the current solutions Ÿ  Validate your idea Ÿ  Build your MVP (Minimum Viable Product) Ÿ  Sense, understand and adapt 70
71. You can make it better 71
72. 72 W H A T A R E Y O U W A I T I N G F O R ?
73. Thank you. JAM852 : Adobe AIR 3.5 Overview Demian Borba @demianborba September 26-27, 2013