Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Let's Build a Cross-platform 3D Engine with Kot...

David
June 05, 2020

Let's Build a Cross-platform 3D Engine with Kotlin/Multiplatform

When creating a game, you want your game to be available on all platforms to target as many players as possible. Developing a game engine is a challenge: you have to deal with mathematical concepts, animations, player interactions... Creating an engine that runs on different platforms is even more challenging. Is Kotlin and Kotlin/Multiplatform could be the right tool to build this kind of engine? Let's check this out together.

Message for you, so you can already be aware what you can expect from this session:
Using existing Java game engines, you can create a game that runs on the JVM, in a browser or on Android and iOS. But it requires some complex tooling, when tools are available for the platform you're targeting.

This session is about the creation of a game engine that runs the same way in a browser, in a JVM, on iOS or Android.

(Conference For Kotliners - 05/06/2020 - Online)

David

June 05, 2020
Tweet

More Decks by David

Other Decks in Programming

Transcript

  1. You, at the end of 
 this conference Game 


    development Kotlin Multiplatform
  2. Hi all. My studio, Sickhead, did the port of Slay

    the Spire to Switch. We struggled for a bit trying to convert the Java to C++ code directly using existing third party tools. The resulting C++ code pretty much didn't work at all and took tons of manual changes to make work. After several months of failure taking that path we changed gears. We instead converted Java to C# which required much less manual fixes to make work and solved all the GC memory management issues. This means we had a C# version of LibGDX in there as well as all the game code. We then just added a new LibGDC backend with new OpenGL bindings and new sound, input implementations. At that point we had a PC port of Spire using C#. We then used our own mature C# IL to C++ cross compiler and runtime we developed to port XNA and MonoGame titles to generate binaries which run on Switch, PS4, XB1, and other platforms. While Java -> C# -> C++ was a crazy idea when we started... it actually works beautifully and makes perfect sense now. HTTPS://WWW.BADLOGICGAMES.COM/FORUM/VIEWTOPIC.PHP?F=11&T=29625
  3. Hi all. My studio, Sickhead, did the port of Slay

    the Spire to Switch. We struggled for a bit trying to convert the Java to C++ code directly using existing third party tools. The resulting C++ code pretty much didn't work at all and took tons of manual changes to make work. After several months of failure taking that path we changed gears. We instead converted Java to C# which required much less manual fixes to make work and solved all the GC memory management issues. This means we had a C# version of LibGDX in there as well as all the game code. We then just added a new LibGDC backend with new OpenGL bindings and new sound, input implementations. At that point we had a PC port of Spire using C#. We then used our own mature C# IL to C++ cross compiler and runtime we developed to port XNA and MonoGame titles to generate binaries which run on Switch, PS4, XB1, and other platforms. While Java -> C# -> C++ was a crazy idea when we started... it actually works beautifully and makes perfect sense now. HTTPS://WWW.BADLOGICGAMES.COM/FORUM/VIEWTOPIC.PHP?F=11&T=29625 MY STUDIO, SICKHEAD, 
 DID THE PORT OF SLAY THE SPIRE TO SWITCH.
 WHILE JAVA -> C# -> C++ WAS A CRAZY IDEA WHEN WE STARTED... IT ACTUALLY WORKS BEAUTIFULLY AND MAKES PERFECT SENSE NOW.
  4. external open class ScrollbarComponentSystem(app: pc.Application) : ComponentSystem { override fun

    on(name: String, callback: HandleEvent, scope: Any): EventHandler override fun off(name: String, callback: HandleEvent, scope: Any): EventHandler override fun fire(name: Any, arg1: Any, arg2: Any, arg3: Any, arg4: Any): EventHandler override fun once(name: String, callback: HandleEvent, scope: Any): EventHandler override fun hasEvent(name: String): Boolean }
  5. external interface `T$8` { var volume: Number? get() = definedExternally

    set(value) = definedExternally var pitch: Number? get() = definedExternally set(value) = definedExternally var loop: Boolean? get() = definedExternally set(value) = definedExternally )
  6. void main() {
 vec4 sum = vec4(0);
 
 float step

    = iResolution.y;
 // y
 for(float i = -area ; i <= area ; i += 1.) {
 // x
 for(float j = -area ; j <= area ; j += 1.) {
 float x = v_texCoords.x + i / iResolution.x;
 float y = v_texCoords.y + j / iResolution.y;
 sum += texture2D(u_texture, vec2(x, y)) * 0.005;
 }
 }
 gl_FragColor = texture2D(u_texture, v_texCoords) + sum;
 } OpenGL Shading Language (GLSL)
  7. MATRIX [ 1 | 0 | 0 | 0 ]


    [ 0 | 1 | 0 | 0 ]
 [ 0 | 0 | 1 | 0 ]
 [ 0 | 0 | 0 | 1 ]
  8. Scale MATRIX [ 1 | 0 | 0 | 0

    ]
 [ 0 | 1 | 0 | 0 ]
 [ 0 | 0 | 1 | 0 ]
 [ 0 | 0 | 0 | 1 ] X Z Y Translation Rotation
  9. data class Mat4(…) { // ... operator fun times(m: Mat4):

    Mat4 { val t = transpose(this) return Mat4( Float4(dot(t.x, m.x), dot(t.y, m.x), dot(t.z, m.x), dot(t.w, m.x)), Float4(dot(t.x, m.y), dot(t.y, m.y), dot(t.z, m.y), dot(t.w, m.y)), Float4(dot(t.x, m.z), dot(t.y, m.z), dot(t.z, m.z), dot(t.w, m.z)), Float4(dot(t.x, m.w), dot(t.y, m.w), dot(t.z, m.w), dot(t.w, m.w)) ) } }
  10. MATRIX [ 1 | 0 | 0 | 0 ]


    [ 0 | 1 | 0 | 0 ]
 [ 0 | 0 | 1 | 0 ]
 [ 0 | 0 | 0 | 1 ] How to access it?
  11. val mat: Mat4 = … val x = mat[0][4] val

    y = mat[1][4] val z = mat[2][4] 
 

  12. data class Mat4(…) { // ... inline val scale: Float3

    get() = … inline val translation: Float3 get() = … val rotation: Float3 get() { … } } 
 

  13. data class Float3(var x: Float = 0.0f, var y: Float

    = 0.0f, var z: Float = 0.0f) { inline var r: Float get() = x set(value) { x = value } inline var g: Float get() = y set(value) { y = value } inline var b: Float get() = z set(value) { z = value } }
  14. data class Mat4(…) { // ... inline val scale: Float3

    get() = … inline val translation: Float3 get() = … val rotation: Float3 get() { … } }
  15. data class Float3( var x: Float = 0.0f, var y:

    Float = 0.0f, var z: Float = 0.0f) val translation: Float3 = mat4.translation val (x, y, z) = translation
  16. plugins { kotlin("multiplatform") version "1.3.60" } kotlin { js {

    } jvm { } sourceSets { val commonMain by getting { dependencies { implementation(kotlin("stdlib-common")) } } js().compilations["main"].defaultSourceSet { dependencies { implementation(kotlin("stdlib-js")) } } jvm().compilations["main"].defaultSourceSet { dependencies { implementation(kotlin("stdlib-jdk8")) } } } }
  17. inline fun pow(x: Float, y: Float) = StrictMath.pow( x.toDouble(), y.toDouble()

    ).toFloat() inline fun pow(x: Float, y: Float) = x.pow(y) java.lang.StrictMath
  18. interface GL { fun uniform1i(uniform: Uniform, data: Int) fun uniform2f(uniform:

    Uniform, first: Float, second: Float) fun uniform3f(uniform: Uniform, first: Float, second: Float, third: Float) // … } class WebGL(private val gl: WebGLRenderingContext) : GL { override fun uniform1i(uniform: Uniform, data: Int) { gl.uniform1i(uniform.uniformLocation, data) } override fun uniform2f(uniform: Uniform, first: Float, second: Float) { gl.uniform2f(uniform.uniformLocation, first, second) } override fun uniform3f(uniform: Uniform, first: Float, second: Float, third: Float) { gl.uniform3f(uniform.uniformLocation, first, second, third) } } COMMON
  19. interface GL { fun uniform1i(uniform: Uniform, data: Int) fun uniform2f(uniform:

    Uniform, first: Float, second: Float) fun uniform3f(uniform: Uniform, first: Float, second: Float, third: Float) // … } class WebGL(private val gl: WebGLRenderingContext) : GL { override fun uniform1i(uniform: Uniform, data: Int) { gl.uniform1i(uniform.uniformLocation, data) } override fun uniform2f(uniform: Uniform, first: Float, second: Float) { gl.uniform2f(uniform.uniformLocation, first, second) } override fun uniform3f(uniform: Uniform, first: Float, second: Float, third: Float) { gl.uniform3f(uniform.uniformLocation, first, second, third) } } COMMON KOTLIN/JS
  20. expect class GLContext { internal fun createContext(): GL } actual

    class GLContext { internal actual fun createContext(): GL { return LwjglGL(…) } } COMMON
  21. expect class GLContext { internal fun createContext(): GL } actual

    class GLContext { internal actual fun createContext(): GL { return LwjglGL(…) } } COMMON KOTLIN/JVM
  22. IN OTHER LANGUAGES, THIS CAN OFTEN BE ACCOMPLISHED BY BUILDING

    A SET OF INTERFACES IN THE COMMON CODE AND IMPLEMENTING THESE INTERFACES IN PLATFORM-SPECIFIC MODULES. HOWEVER, THIS APPROACH IS NOT IDEAL IN CASES WHEN YOU HAVE A LIBRARY ON ONE OF THE PLATFORMS THAT IMPLEMENTS THE FUNCTIONALITY YOU NEED, AND YOU'D LIKE TO USE THE API OF THIS LIBRARY DIRECTLY WITHOUT EXTRA WRAPPERS.
  23. IN OTHER LANGUAGES, THIS CAN OFTEN BE ACCOMPLISHED BY BUILDING

    A SET OF INTERFACES IN THE COMMON CODE AND IMPLEMENTING THESE INTERFACES IN PLATFORM-SPECIFIC MODULES. HOWEVER, THIS APPROACH IS NOT IDEAL IN CASES WHEN YOU HAVE A LIBRARY ON ONE OF THE PLATFORMS THAT IMPLEMENTS THE FUNCTIONALITY YOU NEED, AND YOU'D LIKE TO USE THE API OF THIS LIBRARY DIRECTLY WITHOUT EXTRA WRAPPERS.
  24. // Common expect class Uniform // JVM actual class Uniform(val

    address: Int) // JVM override fun uniform2f(uniform: Uniform, first: Float, second: Float) { glUniform2f(uniform.address, first, second) } COMMON KOTLIN/JVM 1. Producing 2. Consuming
  25. // Common expect class Uniform // JVM actual class Uniform(val

    address: Int) // JVM override fun uniform2f(uniform: Uniform, first: Float, second: Float) { glUniform2f(uniform.address, first, second) } COMMON KOTLIN/JVM 1. Producing 2. Consuming
  26. val vertices = gl.createBuffer() gl.bindBuffer(GL.ARRAY_BUFFER, vertices) gl.bufferData(GL.ARRAY_BUFFER, mesh.vertices.convertPositions(), GL.STATIC_DRAW) gl.vertexAttribPointer(

    index = shader.getAttrib("aVertexPosition"), size = 3, type = GL.FLOAT, normalized = false, stride = 0, offset = 0 ) gl.enableVertexAttribArray(shader.getAttrib("aVertexPosition"))
  27. val vertices = gl.createBuffer() gl.bindBuffer(GL.ARRAY_BUFFER, vertices) gl.bufferData(GL.ARRAY_BUFFER, mesh.vertices.convertPositions(), GL.STATIC_DRAW) gl.vertexAttribPointer(

    index = shader.getAttrib("aVertexPosition"), size = 3, type = GL.FLOAT, normalized = false, stride = 0, offset = 0 ) gl.enableVertexAttribArray(shader.getAttrib("aVertexPosition")) X Y Z
  28. val colors = gl.createBuffer() gl.bindBuffer(GL.ARRAY_BUFFER, colors) gl.bufferData(GL.ARRAY_BUFFER, mesh.vertices.convertColors(), GL.STATIC_DRAW) gl.bindBuffer(GL.ARRAY_BUFFER,

    colors) gl.vertexAttribPointer( index = shader.getAttrib("aVertexColor"), size = 4, type = GL.FLOAT, normalized = false, stride = 0, offset = 0 ) gl.enableVertexAttribArray(shader.getAttrib("aVertexColor"))
  29. val colors = gl.createBuffer() gl.bindBuffer(GL.ARRAY_BUFFER, colors) gl.bufferData(GL.ARRAY_BUFFER, mesh.vertices.convertColors(), GL.STATIC_DRAW) gl.bindBuffer(GL.ARRAY_BUFFER,

    colors) gl.vertexAttribPointer( index = shader.getAttrib("aVertexColor"), size = 4, type = GL.FLOAT, normalized = false, stride = 0, offset = 0 ) gl.enableVertexAttribArray(shader.getAttrib("aVertexColor")) Red Green Blue Alpha
  30. val vertices = gl.createBuffer() gl.bindBuffer(GL.ARRAY_BUFFER, vertices) gl.bufferData(GL.ARRAY_BUFFER, mesh.vertices.convertPositions(), GL.STATIC_DRAW) gl.vertexAttribPointer(

    index = shader.getAttrib("aVertexPosition"), size = 3, type = GL.FLOAT, normalized = false, stride = 0, offset = 0 ) gl.enableVertexAttribArray(shader.getAttrib("aVertexPosition")) val colors = gl.createBuffer() gl.bindBuffer(GL.ARRAY_BUFFER, colors) gl.bufferData(GL.ARRAY_BUFFER, mesh.vertices.convertColors(), GL.STATIC_DRAW) gl.bindBuffer(GL.ARRAY_BUFFER, colors) gl.vertexAttribPointer( index = shader.getAttrib("aVertexColor"), size = 4, type = GL.FLOAT, normalized = false, stride = 0, offset = 0 ) gl.enableVertexAttribArray(shader.getAttrib("aVertexColor")) val verticesOrder = gl.createBuffer() gl.bindBuffer(GL.ELEMENT_ARRAY_BUFFER, verticesOrder) gl.bufferData(GL.ELEMENT_ARRAY_BUFFER, mesh.verticesOrder.convertOrder(), GL.STATIC_DRAW) gl.drawElements(mesh.drawType.glType, mesh.verticesOrder.size, GL.UNSIGNED_SHORT, 0) Vertices Colors Order
  31. attribute vec4 aVertexPosition; uniform mat4 uModelViewMatrix; uniform mat4 uProjectionMatrix; void

    main() { gl_Position = uProjectionMatrix * uModelViewMatrix * aVertexPosition; }
  32. attribute vec4 aVertexPosition; uniform mat4 uModelViewMatrix; uniform mat4 uProjectionMatrix; void

    main() { gl_Position = uProjectionMatrix * uModelViewMatrix * aVertexPosition; } Change for every vertex Common for all vertices
  33. class DemoSuzanne : Game { private val model: Drawable =

    fileHandler.get("suzanne.protobuf") fun render(delta: Seconds) { model.draw() } } Usage Synchronous loading
  34. class DemoSuzanne : Game { private val model: Drawable by

    fileHandler.get("suzanne.protobuf") fun render(delta: Seconds) { model.draw() } }
  35. open class Content<R>(val filename: String) { private var isLoaded: Boolean

    = false private var content: R? = null operator fun getValue(thisRef: Any?, property: KProperty<*>): R { if (isLoaded) { return content!! } else { throw EarlyAccessException(filename, property.name) } } }
  36. private fun <T> asyncContent(filename: String, enc: (ByteArray) -> T): Content<T>

    { val url = computeUrl(filename) val jsonFile = XMLHttpRequest() jsonFile.responseType = XMLHttpRequestResponseType.Companion.ARRAYBUFFER jsonFile.open("GET", url, true) val content = Content<T>(filename) jsonFile.onload = { _ -> if (jsonFile.readyState == 4.toShort() && jsonFile.status == 200.toShort()) { val element = enc((jsonFile.response as ArrayBuffer).toByteArray()) content.load(element) } } jsonFile.send() return content }
  37. class DemoSuzanne : Game { private val model: Drawable by

    fileHandler.get("suzanne.protobuf") fun render(delta: Seconds) { model.draw() } } Content of the wrapper
 Not the wrapper itself
  38. actual fun run(gameFactory: () -> Game) { this.game = gameFactory()

    window.requestAnimationFrame(::loading) } private fun loading(now: Double) { if (!fileHandler.isFullyLoaded()) { window.requestAnimationFrame(::loading) } else { game.create() game.resume() window.requestAnimationFrame(::render) } } Get all resources to load Start the game Wait for resources to be loaded
  39. { "asset": { "generator": "Khronos glTF Blender I/O v1.1.46", "version":

    "2.0" }, "nodes": [ { "name": "Light", "rotation": [ 0.16907575726509094, 0.7558803558349609, -0.27217137813568115, 0.570947527885437 ] } ], "buffers": [ { "byteLength": 26400, "uri": "data:application/octet-stream;base64,…" } ] } GLTF
  40. @Serializable class Mesh( @ProtoId(1) val vertices: List<Vertex> = emptyList(), @ProtoId(2)

    val verticesOrder: IntArray = intArrayOf() ) @Serializable class Vertex( @ProtoId(1) val position: Position, @ProtoId(2) val color: Color, @ProtoId(3) val normal: Normal, @ProtoId(4) val influence: Influence )
  41. fun readProtobuf(data: ByteArray): Model { val deserializer = ProtoBuf(context =

    serialModule()) return deserializer.load(serializer(), data) } fun writeProtobuf(model: Model): ByteArray { val serializer = ProtoBuf(context = serialModule()) return serializer.dump(Model.serializer(), model) }
  42. uniform mat4 uJointTransformationMatrix[MAX_JOINTS]; attribute vec3 aVertexPosition; attribute int aJointId; void

    main() { mat4 uJointMatrix = uJointTransformationMatrix[aJointId]; gl_Position = uJointMatrix * vec4(aVertexPosition, 1.0); } All animation moves transformation Get transformation for this vertex Apply transformation