$30 off During Our Annual Pro Sale. View Details »

Threedee Tales From Urban Bohemia

Threedee Tales From Urban Bohemia

An introduction to 3D graphics on the web using WebGL.

Vicent Martí

July 07, 2013
Tweet

More Decks by Vicent Martí

Other Decks in Programming

Transcript

  1. View Slide

  2. 3D is hard

    View Slide

  3. We’re building a
    videogame

    View Slide

  4. We’re building a
    videogame

    View Slide

  5. View Slide

  6. View Slide

  7. View Slide

  8. (You are being a smartass)

    View Slide

  9. View Slide

  10. WHAT?

    View Slide

  11. WebGL is
    A JavaScript API
    to render graphics
    accessing directly the
    GPU

    View Slide

  12. WebGL is not
    An API for 3D
    graphics

    View Slide

  13. WebGL is not
    An OpenGL
    Wrapper

    View Slide

  14. angle
    angle
    OpenGL ES 2.0
    to DirectX 9
    translation

    View Slide

  15. angle
    angle
    OpenGL ES 2.0
    to DirectX 9
    translation

    View Slide

  16. WHERE?

    View Slide

  17. View Slide

  18. DIV

    View Slide

  19. CANVAS
    DIV

    View Slide

  20. WebGL
    context
    CANVAS “webgl”
    .getContext(“webgl”)
    WebGL
    context
    the

    View Slide

  21. WebGL
    context
    WebGL
    context(s)
    a b a.getContext(“webg
    b.getContext(“webg

    View Slide

  22. WHEN?

    View Slide

  23. DRAW
    DRAW
    DRAW DRAW
    DRAW
    DRAW
    DRAW DRAW
    Not when you
    want to

    View Slide

  24. When the browser tells
    you to.

    View Slide

  25. When the browser tells
    you to.
    requestAnimationFrame

    View Slide

  26. BRO WHAT ABOUT
    DOUBLE BUFFERING

    View Slide

  27. View Slide

  28. Tearing

    View Slide

  29. Buffer A Buffer B

    View Slide

  30. Buffer A Buffer B

    View Slide

  31. Buffer A Buffer B

    View Slide

  32. JavaScript doesn’t
    execute while the DOM
    is being rendered!

    View Slide

  33. JavaScript doesn’t
    execute while the DOM
    is being rendered!

    View Slide

  34. HOW?

    View Slide

  35. Introducing...
    Jimmy
    the Vertex

    View Slide

  36. I’m not looking
    forward to this...

    View Slide

  37. (x, y, z)

    View Slide

  38. +x
    +y
    +z
    RIGHT HAND REPRESENT

    View Slide

  39. View Slide

  40. View Slide

  41. View Slide

  42. Local Space

    View Slide

  43. World Space

    View Slide

  44. Camera Space

    View Slide

  45. Camera Space

    View Slide

  46. Screen Space

    View Slide

  47. xx xy xz
    Yx Yy Yz
    Zx Zy Zz
    { } World
    Space
    Local
    Space
    Model Matrix

    View Slide

  48. xx xy xz
    Yx Yy Yz
    Zx Zy Zz
    { } Camera
    Space
    World
    Space
    View Matrix

    View Slide

  49. xx xy xz
    Yx Yy Yz
    Zx Zy Zz
    { } Screen
    Space
    Camera
    Space
    Projection Matrix

    View Slide

  50. We could do the matrix
    operations in JavaScript

    View Slide

  51. We could do the matrix
    operations in JavaScript
    BUT WE
    DON’T

    View Slide

  52. SHADER
    is a program that
    runs on the GPU.
    SHADER
    a

    View Slide

  53. (THAT
    WAS
    EASY)

    View Slide

  54. N
    N
    It begins with an...

    View Slide

  55. OT
    JAVASCRIPT
    N
    N
    It begins with an...

    View Slide

  56. GLSL
    GLSL
    OpenGL
    Shader Language

    View Slide

  57. precision highp float;
    varying vec2 t;
    varying vec2 pos;
    uniform float time;
    uniform vec2 mouse;
    uniform int mouseLeft;
    uniform sampler2D tex0;
    void main() {
    float v1 = (sin(t.s+time) + 1.0) / 2.0;
    float v2 = (cos(t.t+time) + 1.0) / 2.0;
    float d = distance(mouse, pos);
    vec2 tt = vec2(t.s+sin(time/10.0), t.t+cos(time/10.0));
    vec4 c1 = texture2D(tex0, tt) * 1.1;
    float avg = (c1.r+c1.g+c1.b)/3.0;
    float r = c1.r+v1*pow(avg,4.0) - pow(d,pow(avg,2.0) +float(mouseLeft)*avg);
    float g = c1.g+v2*pow(avg,4.0) - pow(d,pow(avg,2.0) +float(mouseLeft)*avg);
    float b = c1.g - pow(d,pow(avg,2.0) +float(mouseLeft)*avg);
    gl_FragColor = vec4(r, g, b, 1.0);
    }

    View Slide

  58. precision highp float;
    varying vec2 t;
    varying vec2 pos;
    uniform float time;
    uniform vec2 mouse;
    uniform int mouseLeft;
    uniform sampler2D tex0;
    void main() {
    float v1 = (sin(t.s+time) + 1.0) / 2.0;
    float v2 = (cos(t.t+time) + 1.0) / 2.0;
    float d = distance(mouse, pos);
    vec2 tt = vec2(t.s+sin(time/10.0), t.t+cos(time/10.0));
    vec4 c1 = texture2D(tex0, tt) * 1.1;
    float avg = (c1.r+c1.g+c1.b)/3.0;
    float r = c1.r+v1*pow(avg,4.0) - pow(d,pow(avg,2.0) +float(mouseLeft)*avg);
    float g = c1.g+v2*pow(avg,4.0) - pow(d,pow(avg,2.0) +float(mouseLeft)*avg);
    float b = c1.g - pow(d,pow(avg,2.0) +float(mouseLeft)*avg);
    gl_FragColor = vec4(r, g, b, 1.0);
    }
    WELL THIS
    KINDA
    LOOKS LIKE
    JAVASCRIPT
    WELL THIS
    KINDA
    LOOKS LIKE
    JAVASCRIPT

    View Slide

  59. precision highp float;
    varying vec2 t;
    varying vec2 pos;
    uniform float time;
    uniform vec2 mouse;
    uniform int mouseLeft;
    uniform sampler2D tex0;
    void main() {
    float v1 = (sin(t.s+time) + 1.0) / 2.0;
    float v2 = (cos(t.t+time) + 1.0) / 2.0;
    float d = distance(mouse, pos);
    vec2 tt = vec2(t.s+sin(time/10.0), t.t+cos(time/10.0));
    vec4 c1 = texture2D(tex0, tt) * 1.1;
    float avg = (c1.r+c1.g+c1.b)/3.0;
    float r = c1.r+v1*pow(avg,4.0) - pow(d,pow(avg,2.0) +float(mouseLeft)*avg);
    float g = c1.g+v2*pow(avg,4.0) - pow(d,pow(avg,2.0) +float(mouseLeft)*avg);
    float b = c1.g - pow(d,pow(avg,2.0) +float(mouseLeft)*avg);
    gl_FragColor = vec4(r, g, b, 1.0);
    }

    View Slide

  60. precision highp float;
    varying vec2 t;
    varying vec2 pos;
    uniform float time;
    uniform vec2 mouse;
    uniform int mouseLeft;
    uniform sampler2D tex0;
    void main() {
    float v1 = (sin(t.s+time) + 1.0) / 2.0;
    float v2 = (cos(t.t+time) + 1.0) / 2.0;
    float d = distance(mouse, pos);
    vec2 tt = vec2(t.s+sin(time/10.0), t.t+cos(time/10.0));
    vec4 c1 = texture2D(tex0, tt) * 1.1;
    float avg = (c1.r+c1.g+c1.b)/3.0;
    float r = c1.r+v1*pow(avg,4.0) - pow(d,pow(avg,2.0) +float(mouseLeft)*avg);
    float g = c1.g+v2*pow(avg,4.0) - pow(d,pow(avg,2.0) +float(mouseLeft)*avg);
    float b = c1.g - pow(d,pow(avg,2.0) +float(mouseLeft)*avg);
    gl_FragColor = vec4(r, g, b, 1.0);
    }

    View Slide

  61. precision highp float;
    varying vec2 t;
    varying vec2 pos;
    uniform float time;
    uniform vec2 mouse;
    uniform int mouseLeft;
    uniform sampler2D tex0;
    void main() {
    float v1 = (sin(t.s+time) + 1.0) / 2.0;
    float v2 = (cos(t.t+time) + 1.0) / 2.0;
    float d = distance(mouse, pos);
    vec2 tt = vec2(t.s+sin(time/10.0), t.t+cos(time/10.0));
    vec4 c1 = texture2D(tex0, tt) * 1.1;
    float avg = (c1.r+c1.g+c1.b)/3.0;
    float r = c1.r+v1*pow(avg,4.0) - pow(d,pow(avg,2.0) +float(mouseLeft)*avg);
    float g = c1.g+v2*pow(avg,4.0) - pow(d,pow(avg,2.0) +float(mouseLeft)*avg);
    float b = c1.g - pow(d,pow(avg,2.0) +float(mouseLeft)*avg);
    gl_FragColor = vec4(r, g, b, 1.0);
    }
    <br/>

    View Slide

  62. document.getElementById
    document.getElementById

    View Slide

  63. GLSL
    GLSL
    GLSL
    compile
    compile
    compile link

    View Slide

  64. xx xy xz
    Yx Yy Yz
    Zx Zy Zz
    { }
    xx xy xz
    Yx Yy Yz
    Zx Zy Zz
    { }
    Perspective Matrix
    Model View Matrix

    View Slide

  65. attribute vec3 aVertexPosition;
    uniform mat4 uMVMatrix;
    uniform mat4 uPMatrix;
    void main(void) {
    gl_Position = uPMatrix * uMVMatrix *
    vec4(aVertexPosition, 1.0);
    }

    View Slide

  66. attribute vec3 aVertexPosition;
    uniform mat4 uMVMatrix;
    uniform mat4 uPMatrix;
    void main(void) {
    gl_Position = uPMatrix * uMVMatrix *
    vec4(aVertexPosition, 1.0);
    }
    Output

    View Slide

  67. attribute vec3 aVertexPosition;
    uniform mat4 uMVMatrix;
    uniform mat4 uPMatrix;
    void main(void) {
    gl_Position = uPMatrix * uMVMatrix *
    vec4(aVertexPosition, 1.0);
    }
    Output
    Uniforms

    View Slide

  68. attribute vec3 aVertexPosition;
    uniform mat4 uMVMatrix;
    uniform mat4 uPMatrix;
    void main(void) {
    gl_Position = uPMatrix * uMVMatrix *
    vec4(aVertexPosition, 1.0);
    }
    Output
    Uniforms
    Attribute

    View Slide

  69. View Slide

  70. View Slide

  71. xx xy xz
    Yx Yy Yz
    Zx Zy Zz
    { }
    xx xy xz
    Yx Yy Yz
    Zx Zy Zz
    { } Passing
    Uniforms
    var ptr =
    gl.getUniformLocation(shaderProgram, "uPMatrix")
    gl.uniformMatrix4fv(
    ptr,
    false,
    new Float32Array(vertexes));

    View Slide

  72. View Slide

  73. -1.0, -1.0, 1.0,
    1.0, -1.0, 1.0,
    1.0, 1.0, 1.0,
    -1.0, 1.0, 1.0,
    -1.0, -1.0, -1.0,
    -1.0, 1.0, -1.0,
    1.0, 1.0, -1.0,
    1.0, -1.0, -1.0,
    -1.0, 1.0, -1.0,
    -1.0, 1.0, 1.0,
    1.0, 1.0, 1.0,
    1.0, 1.0, -1.0,
    -1.0, -1.0, -1.0,
    1.0, -1.0, -1.0,
    Here come the vertices

    View Slide

  74. -1.0, -1.0, 1.0,
    1.0, -1.0, 1.0,
    1.0, 1.0, 1.0,
    -1.0, 1.0, 1.0,
    -1.0, -1.0, -1.0,
    -1.0, 1.0, -1.0,
    1.0, 1.0, -1.0,
    1.0, -1.0, -1.0,
    -1.0, 1.0, -1.0,
    -1.0, 1.0, 1.0,
    1.0, 1.0, 1.0,
    1.0, 1.0, -1.0,
    -1.0, -1.0, -1.0,
    1.0, -1.0, -1.0,
    Here come the vertices

    View Slide

  75. -1.0, -1.0, 1.0,
    1.0, -1.0, 1.0,
    1.0, 1.0, 1.0,
    -1.0, 1.0, 1.0,
    -1.0, -1.0, -1.0,
    -1.0, 1.0, -1.0,
    1.0, 1.0, -1.0,
    1.0, -1.0, -1.0,
    -1.0, 1.0, -1.0,
    -1.0, 1.0, 1.0,
    1.0, 1.0, 1.0,
    1.0, 1.0, -1.0,
    -1.0, -1.0, -1.0,
    1.0, -1.0, -1.0,
    Here come the vertices

    View Slide

  76. -1.0, -1.0, 1.0,
    1.0, -1.0, 1.0,
    1.0, 1.0, 1.0,
    -1.0, 1.0, 1.0,
    -1.0, -1.0, -1.0,
    -1.0, 1.0, -1.0,
    1.0, 1.0, -1.0,
    1.0, -1.0, -1.0,
    -1.0, 1.0, -1.0,
    -1.0, 1.0, 1.0,
    1.0, 1.0, 1.0,
    1.0, 1.0, -1.0,
    -1.0, -1.0, -1.0,
    1.0, -1.0, -1.0,
    Here come the vertices

    View Slide

  77. -1.0, -1.0, 1.0,
    1.0, -1.0, 1.0,
    1.0, 1.0, 1.0,
    -1.0, 1.0, 1.0,
    -1.0, -1.0, -1.0,
    -1.0, 1.0, -1.0,
    1.0, 1.0, -1.0,
    1.0, -1.0, -1.0,
    -1.0, 1.0, -1.0,
    -1.0, 1.0, 1.0,
    1.0, 1.0, 1.0,
    1.0, 1.0, -1.0,
    -1.0, -1.0, -1.0,
    1.0, -1.0, -1.0,
    Here come the vertices

    View Slide

  78. cubeVertices = gl.createBuffer();
    gl.bindBuffer(gl.ARRAY_BUFFER, cubeVertices);
    gl.bufferData(gl.ARRAY_BUFFER,
    new Float32Array(vertices), gl.STATIC_DRAW);
    gl.bindBuffer(gl.ARRAY_BUFFER, cubeVertices);
    gl.vertexAttribPointer(vertexPositionAttribute, 3,
    gl.FLOAT, false, 0, 0);
    0,1,0
    1,0,0
    0,1,1
    1,0,1
    1,1,1
    Passing Vertices

    View Slide

  79. cubeVertices = gl.createBuffer();
    gl.bindBuffer(gl.ARRAY_BUFFER, cubeVertices);
    gl.bufferData(gl.ARRAY_BUFFER,
    new Float32Array(vertices), gl.STATIC_DRAW);
    gl.bindBuffer(gl.ARRAY_BUFFER, cubeVertices);
    gl.vertexAttribPointer(vertexPositionAttribute, 3,
    gl.FLOAT, false, 0, 0);
    0,1,0
    1,0,0
    0,1,1
    1,0,1
    1,1,1
    Passing Vertices

    View Slide

  80. xx xy xz
    Yx Yy Yz
    Zx Zy Zz
    { }
    xx xy xz
    Yx Yy Yz
    Zx Zy Zz
    { }
    Uniforms
    gl_Position
    0,1,0
    1,0,0
    0,1,1
    1,0,1
    1,1,1
    [aVertexPosition]
    drawArrays*

    View Slide

  81. xx xy xz
    Yx Yy Yz
    Zx Zy Zz
    { }
    xx xy xz
    Yx Yy Yz
    Zx Zy Zz
    { }
    Uniforms
    gl_Position
    0,1,0
    1,0,0
    0,1,1
    1,0,1
    1,1,1
    [aVertexPosition]
    drawArrays*

    View Slide

  82. xx xy xz
    Yx Yy Yz
    Zx Zy Zz
    { }
    xx xy xz
    Yx Yy Yz
    Zx Zy Zz
    { }
    Uniforms
    gl_Position
    0,1,0
    1,0,0
    0,1,1
    1,0,1
    1,1,1
    [aVertexPosition]
    drawArrays*

    View Slide

  83. xx xy xz
    Yx Yy Yz
    Zx Zy Zz
    { }
    xx xy xz
    Yx Yy Yz
    Zx Zy Zz
    { }
    Uniforms
    gl_Position
    0,1,0
    1,0,0
    0,1,1
    1,0,1
    1,1,1
    [aVertexPosition]
    drawArrays*

    View Slide

  84. xx xy xz
    Yx Yy Yz
    Zx Zy Zz
    { }
    xx xy xz
    Yx Yy Yz
    Zx Zy Zz
    { }
    Uniforms
    gl_Position
    0,1,0
    1,0,0
    0,1,1
    1,0,1
    1,1,1
    [aVertexPosition]
    drawArrays*

    View Slide

  85. Rasterization

    View Slide

  86. View Slide

  87. View Slide

  88. Fragments

    View Slide

  89. Fragment
    Shader
    varying lowp vec4 vColor;
    void main(void) {
    gl_FragColor = vColor;
    }

    View Slide

  90. Fragment
    Shader
    varying lowp vec4 vColor;
    void main(void) {
    gl_FragColor = vColor;
    }
    Output

    View Slide

  91. Fragment
    Shader
    varying lowp vec4 vColor;
    void main(void) {
    gl_FragColor = vColor;
    }
    Output
    Variant

    View Slide

  92. Variants
    Vertex
    Shader
    Fragment
    Shader

    View Slide

  93. View Slide

  94. View Slide

  95. Color

    View Slide

  96. Color
    Texture

    View Slide

  97. Color
    Texture
    Lighting

    View Slide

  98. Color
    Texture
    Lighting
    Specularity

    View Slide

  99. Color
    Texture
    Lighting
    Specularity
    Normal
    Mapping

    View Slide

  100. Color
    Texture
    Lighting
    Specularity
    Normal
    Mapping
    ...

    View Slide

  101. Clear Canvas
    Load
    Perspective
    M
    atrixes
    Load
    O
    bject M
    atrixes
    Render O
    bject

    View Slide

  102. View Slide

  103. Three.js
    https://github.com/mrdoob/three.js
    A.K.A. use

    View Slide

  104. 3D is hard

    View Slide

  105. 3D is hard
    hard to understand

    View Slide

  106. 3D is hard
    hard to understand
    hard to learn

    View Slide

  107. Hard to get right

    View Slide

  108. But we need to try

    View Slide

  109. View Slide

  110. twitter.com/vmg
    github.com/vmg

    View Slide

  111. With demos from:
    http://wagerfield.github.io/flat-surface-shader/
    http://lab.aerotwist.com/webgl/surface/
    http://www.mrdoob.com/lab/javascript/beachballs/
    http://plumegraph.org/

    View Slide