Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Haskell SpriteKit — A Purely Functional API for...

Haskell SpriteKit — A Purely Functional API for a Stateful Animation System & Physics Engine

VIDEOS:
* https://www.youtube.com/watch?v=kd8mlbN0Mws (Functional Conf 2017, best choice right now)
* https://www.youtube.com/watch?v=H_z4NKvxf1U (Curry On 2017, more detail than YOW! at the end, but slides out of sync for first 1.5min)
* https://www.youtube.com/watch?v=GaorHAlUkVs (YOW! Lambda Jam 2017, slightly shorter)

Paper with additional details: http://www.cse.unsw.edu.au/%7Echak/papers/CK17.html

Graphics, animation, and games programming in Haskell faces a dilemma. We can either use existing frameworks with their highly imperative APIs (such as OpenGL, Cocos2D, or SpriteKit) or we waste a lot of energy trying to re-engineer those rather complex systems from scratch. Or, maybe, we can escape the dilemma. Instead of a Haskell program directly manipulating the mutable object-graph of existing high-level frameworks, we provide an API for purely functional transformations of a Haskell data structure, together with an adaptation layer that transcribes those transformations into edits of the mutable object-graph.

I this talk, I will explain how I used this approach to architect a Haskell binding to the animation system and physics engine of Apple’s SpriteKit framework. I will discuss both how the binding is structured internally and how it achieves the translation of Haskell side changes to SpriteKit and vice versa, such that it is sufficiently efficient. Moreover, I will explain how to use the Haskell library to implement animations and games.

This talk is aimed at an intermediate level. It will use Haskell and SpriteKit as concrete technology in all examples, but it requires neither previous experience with Haskell nor with SpriteKit. In a similar vain, the talk is based on Apple’s SpriteKit framework, as there is no reasonable (in terms of effort) way around platform-specific technology if you want something that is simultaneously state-of-the-art, practically useful, and convenient to use. Nevertheless, the concepts explained in the talk transcend the specific technology and are generally applicable. Moreover, the entire code base of the Haskell binding is open source and available from https://github.com/mchakravarty/HaskellSpriteKit

Presented at
* YOW! Lambda Jam, Sydney, May 2017: http://lambdajam.yowconference.com.au/speakers/manuel-chakravarty-4/
* IFIP WG2.8, Edinburgh, June 2017
* Curry On, Barcelona, June 2017: http://www.curry-on.org/2017/sessions/haskell-spritekit-a-case-study-in-turning-a-stateful-into-a-purely-functional-api.html
* Functional Conf, Bangalore, November 2017: https://functionalconf.com/proposal.html?id=3939

Manuel Chakravarty

June 19, 2017
Tweet

More Decks by Manuel Chakravarty

Other Decks in Programming

Transcript

  1. Manuel M T Chakravarty & Gabriele Keller Applicative & UNSW

    Sydney Haskell SpriteKit A Purely Functional API for a Stateful Animation System & Physics Engine mchakravarty TacticalGrace justtesting.org haskellformac.com
  2. The Dilemma SKShapeNode *shipOverlayShape = [[SKShapeNode alloc] init]; shipOverlayShape.path =

    boundingPath; shipOverlayShape.strokeColor = [SKColor clearColor]; shipOverlayShape.fillColor = [SKColor colorWithRed:0.0 green:1.0 blue:0.0 alpha:0.5]; [ship addChild:shipOverlayShape];
  3. The Dilemma SKShapeNode *shipOverlayShape = [[SKShapeNode alloc] init]; shipOverlayShape.path =

    boundingPath; shipOverlayShape.strokeColor = [SKColor clearColor]; shipOverlayShape.fillColor = [SKColor colorWithRed:0.0 green:1.0 blue:0.0 alpha:0.5]; [ship addChild:shipOverlayShape];
  4. The Dilemma SKShapeNode *shipOverlayShape = [[SKShapeNode alloc] init]; shipOverlayShape.path =

    boundingPath; shipOverlayShape.strokeColor = [SKColor clearColor]; shipOverlayShape.fillColor = [SKColor colorWithRed:0.0 green:1.0 blue:0.0 alpha:0.5]; [ship addChild:shipOverlayShape]; data GameState = GameState { gsSize :: (Double, Double) , time :: Millisecond , level :: Int , lifes :: Int , towers :: [Tower] , creeps :: [Creep] , shots :: [Shot] }
  5. The Compromise Purely Functional API data GameState = GameState {

    gsSize :: (Double, Double) , time :: Millisecond , level :: Int , lifes :: Int , towers :: [Tower] , creeps :: [Creep] , shots :: [Shot] }
  6. The Compromise Purely Functional API data GameState = GameState {

    gsSize :: (Double, Double) , time :: Millisecond , level :: Int , lifes :: Int , towers :: [Tower] , creeps :: [Creep] , shots :: [Shot] } functional imperative object-oriented
  7. The Compromise Purely Functional API represent scenes with algebraic data

    types data GameState = GameState { gsSize :: (Double, Double) , time :: Millisecond , level :: Int , lifes :: Int , towers :: [Tower] , creeps :: [Creep] , shots :: [Shot] } functional imperative object-oriented
  8. The Compromise Purely Functional API translate transformations into state changes

    represent scenes with algebraic data types data GameState = GameState { gsSize :: (Double, Double) , time :: Millisecond , level :: Int , lifes :: Int , towers :: [Tower] , creeps :: [Creep] , shots :: [Shot] } functional imperative object-oriented
  9. Purely Functional API data GameState = GameState { gsSize ::

    (Double, Double) , time :: Millisecond , level :: Int , lifes :: Int , towers :: [Tower] , creeps :: [Creep] , shots :: [Shot] } functional imperative object-oriented
  10. Purely Functional API data GameState = GameState { gsSize ::

    (Double, Double) , time :: Millisecond , level :: Int , lifes :: Int , towers :: [Tower] , creeps :: [Creep] , shots :: [Shot] } functional imperative object-oriented Step ❶ — state of the art
  11. Purely Functional API data GameState = GameState { gsSize ::

    (Double, Double) , time :: Millisecond , level :: Int , lifes :: Int , towers :: [Tower] , creeps :: [Creep] , shots :: [Shot] } functional imperative object-oriented Step ❶ — state of the art Step ❷ — purification
  12. Purely Functional API data GameState = GameState { gsSize ::

    (Double, Double) , time :: Millisecond , level :: Int , lifes :: Int , towers :: [Tower] , creeps :: [Creep] , shots :: [Shot] } functional imperative object-oriented Step ❶ — state of the art Step ❷ — purification Step ❸ — efficiency
  13. SpriteKit Physics, fields & particles Animation & lighting Joints &

    constraints Lazy Lambda Simple side scroller
  14. — node without visuals 0 pipes moving nodes pipe pair

    l ground physics score contact l l
  15. — node without visuals — attached actions 0 pipes moving

    nodes pipe pair l ground physics score contact l l
  16. — node without visuals — attached actions — attached physics

    body 0 pipes moving nodes pipe pair l ground physics score contact l l
  17. — node without visuals — attached actions — attached physics

    body 0 pipes moving nodes pipe pair l ground physics score contact l l
  18. 0 pipes moving nodes pipe pair l ground physics score

    contact l l SKScene SKLabelNode SKNode SKNode SKSpriteNode SKNode SKNode SKNode SKSpriteNode SKSpriteNode SKSpriteNode SKSpriteNode
  19. SKLabelNode SKScene SKNode SKSpriteNode SKEffectNode SKShapeNode SKLightNode ộ OOism #1:

    Subclassing class SKNode: … { var frame: CGRect func calculatedFrame() -> CGRect var position: CGPoint var zRotation: CGFloat var xScale: CGFloat ⋮ }
  20. SKLabelNode SKScene SKNode SKSpriteNode SKEffectNode SKShapeNode SKLightNode ộ OOism #1:

    Subclassing this one is odd class SKNode: … { var frame: CGRect func calculatedFrame() -> CGRect var position: CGPoint var zRotation: CGFloat var xScale: CGFloat ⋮ }
  21. SKLabelNode SKScene SKNode SKSpriteNode SKEffectNode SKShapeNode SKLightNode ộ NSResponder LambdaScene

    class NSResponder: NSObject { func mouseDown(with: NSEvent) func mouseUp(with: NSEvent) func keyDown(with: NSEvent) func keyUp(with: NSEvent) ⋮ }
  22. SKLabelNode SKScene SKNode SKSpriteNode SKEffectNode SKShapeNode SKLightNode ộ NSResponder LambdaScene

    class NSResponder: NSObject { func mouseDown(with: NSEvent) func mouseUp(with: NSEvent) func keyDown(with: NSEvent) func keyUp(with: NSEvent) ⋮ } user-defined subclass to override event handlers
  23. OOism #2: Mutable Properties class SKNode: … { var frame:

    CGRect var position: CGPoint var zRotation: CGFloat var xScale: CGFloat ⋮ } Lambda tilts (rotation) depending on vertical velocity mutation
  24. 0 pipes moving nodes pipe pair l ground physics score

    contact l l class SKScene: SKEffectNode { var backgroundColor: NSColor func update(_ currentTime: TimeInterval) ⋮ }
  25. 0 pipes moving nodes pipe pair l ground physics score

    contact l l class SKScene: SKEffectNode { var backgroundColor: NSColor func update(_ currentTime: TimeInterval) ⋮ } called once per frame
  26. OOism #3: In-place Graph Edits Pipe pairs appear and disappear

    at regular intervals class SKNode: … { var children: [SKNode] { get } func addChild(_ node: SKNode) ⋮ } mutation
  27. 0 pipes moving nodes ground physics l l pipe pair

    score contact pipe pair score contact
  28. 0 pipes moving nodes ground physics l l pipe pair

    score contact pipe pair score contact pipe pair score contact
  29. 0 pipes moving nodes ground physics l l pipe pair

    score contact pipe pair score contact
  30. The Three Issues Subclassing Inherited properties & event handlers Mutable

    properties Inplace updates by callbacks Inplace graph edits Add, delete & rearrange nodes
  31. data Node u = Node { nodeName :: Maybe String

    , nodePosition :: Point , nodeZRotation :: GFloat , nodeUserData :: u ⋮ } | Label { nodeName :: Maybe String , nodePosition :: Point , nodeZRotation :: GFloat , nodeUserData :: u ⋮ , labelText :: String ⋮ } | Sprite { … } | Shape { … } ⋮
  32. data Node u = Node { nodeName :: Maybe String

    , nodePosition :: Point , nodeZRotation :: GFloat , nodeUserData :: u ⋮ } | Label { nodeName :: Maybe String , nodePosition :: Point , nodeZRotation :: GFloat , nodeUserData :: u ⋮ , labelText :: String ⋮ } | Sprite { … } | Shape { … } ⋮ shared fields of ”superclass” node included in each variant
  33. data Node u = Node { nodeName :: Maybe String

    , nodePosition :: Point , nodeZRotation :: GFloat , nodeUserData :: u ⋮ } | Label { nodeName :: Maybe String , nodePosition :: Point , nodeZRotation :: GFloat , nodeUserData :: u ⋮ , labelText :: String ⋮ } | Sprite { … } | Shape { … } ⋮ shared fields of ”superclass” node included in each variant each variant (except Node) has specific fields
  34. data Scene sceneData nodeData = Scene { sceneName :: Maybe

    String , sceneChildren :: [Node nodeData] , sceneData :: sceneData , sceneBackgroundColor :: Color ⋮ }
  35. data Scene sceneData nodeData = Scene { sceneName :: Maybe

    String , sceneChildren :: [Node nodeData] , sceneData :: sceneData , sceneBackgroundColor :: Color ⋮ } children of a Scene are all Nodes
  36. data Scene sceneData nodeData = Scene { sceneName :: Maybe

    String , sceneChildren :: [Node nodeData] , sceneData :: sceneData , sceneBackgroundColor :: Color ⋮ } children of a Scene are all Nodes the Scene and the Nodes may contain extra user-defined data
  37. data Node u = Node { … } | Sprite

    { nodeName :: Maybe String , nodeZRotation :: GFloat ⋮ } ⋮ data Scene sceneData nodeData = Scene { sceneName :: Maybe String , sceneChildren :: [Node nodeData] ⋮ }
  38. data Node u = Node { … } | Sprite

    { nodeName :: Maybe String , nodeZRotation :: GFloat ⋮ } ⋮ data Scene sceneData nodeData = Scene { sceneName :: Maybe String , sceneChildren :: [Node nodeData] ⋮ } immutable, but must change to tilt the bird
  39. data Node u = Node { … } | Sprite

    { nodeName :: Maybe String , nodeZRotation :: GFloat ⋮ } ⋮ data Scene sceneData nodeData = Scene { sceneName :: Maybe String , sceneChildren :: [Node nodeData] , sceneUpdate :: Maybe (SceneUpdate sceneData nodeData) ⋮ } immutable, but must change to tilt the bird
  40. data Node u = Node { … } | Sprite

    { nodeName :: Maybe String , nodeZRotation :: GFloat ⋮ } ⋮ data Scene sceneData nodeData = Scene { sceneName :: Maybe String , sceneChildren :: [Node nodeData] , sceneUpdate :: Maybe (SceneUpdate sceneData nodeData) ⋮ } immutable, but must change to tilt the bird callback called once per frame
  41. data Node u = Node { … } | Sprite

    { nodeName :: Maybe String , nodeZRotation :: GFloat ⋮ } ⋮ data Scene sceneData nodeData = Scene { sceneName :: Maybe String , sceneChildren :: [Node nodeData] , sceneUpdate :: Maybe (SceneUpdate sceneData nodeData) ⋮ } immutable, but must change to tilt the bird type SceneUpdate sd nd = Scene sd nd -> TimeInterval -> Scene sd nd callback called once per frame pure scene transformer
  42. data Scene sceneData nodeData = Scene { sceneName :: Maybe

    String , sceneChildren :: [Node nodeData] , sceneUpdate :: Maybe (SceneUpdate sceneData nodeData) ⋮ } type SceneUpdate sd nd = Scene sd nd -> TimeInterval -> Scene sd nd
  43. data Scene sceneData nodeData = Scene { sceneName :: Maybe

    String , sceneChildren :: [Node nodeData] , sceneUpdate :: Maybe (SceneUpdate sceneData nodeData) , sceneHandleEvent :: Maybe (EventHandler sceneData) ⋮ } type SceneUpdate sd nd = Scene sd nd -> TimeInterval -> Scene sd nd type EventHandler sceneData = Event -> sceneData -> Maybe sceneData
  44. data Scene sceneData nodeData = Scene { sceneName :: Maybe

    String , sceneChildren :: [Node nodeData] , sceneUpdate :: Maybe (SceneUpdate sceneData nodeData) , sceneHandleEvent :: Maybe (EventHandler sceneData) ⋮ } type SceneUpdate sd nd = Scene sd nd -> TimeInterval -> Scene sd nd type EventHandler sceneData = Event -> sceneData -> Maybe sceneData transforms game state in response to input events
  45. data Scene sceneData nodeData = Scene { sceneName :: Maybe

    String , sceneChildren :: [Node nodeData] , sceneData :: sceneData ⋮ } data Node u = Node { … , nodeChildren :: [Node u] ⋮ } | Label { … , nodeChildren :: [Node u] ⋮ } ⋮
  46. data Scene sceneData nodeData = Scene { sceneName :: Maybe

    String , sceneChildren :: [Node nodeData] , sceneData :: sceneData ⋮ } data Node u = Node { … , nodeChildren :: [Node u] ⋮ } | Label { … , nodeChildren :: [Node u] ⋮ } ⋮ these fields describe the scene tree
  47. 0 pip movin groun l l pip score 0 pi

    movin groun l l pip score pip score spawnPipePair new subtree ObjC -> Haskell Haskell -> ObjC
  48. 0 pip movin groun l l pip score 0 pi

    movin groun l l pip score pip score spawnPipePair new subtree old subtree ObjC -> Haskell Haskell -> ObjC
  49. Eager Marshalling Is Infeasible Hidden State Nodes have private state

    that would get lost Efficiency Marshalling a whole tree to change a few properties
  50. data Scene sc nd = Scene { … } —

    Haskell record type SceneUpdate sc nd = Scene sc nd -> TimeInterval -> Scene sc nd once per frame 0 moving nodes ground physics l l l
  51. data Scene sc nd = Scene { … } —

    Haskell record type SceneUpdate sc nd = Scene sc nd -> TimeInterval -> Scene sc nd once per frame Scene sc nd lazy marshalling 0 moving nodes ground physics l l l
  52. data Scene sc nd = Scene { … } —

    Haskell record type SceneUpdate sc nd = Scene sc nd -> TimeInterval -> Scene sc nd once per frame Scene sc nd lazy marshalling SceneUpdate sc nd Scene sc nd 0 moving nodes ground physics l l l
  53. data Scene sc nd = Scene { … } —

    Haskell record type SceneUpdate sc nd = Scene sc nd -> TimeInterval -> Scene sc nd once per frame Scene sc nd lazy marshalling SceneUpdate sc nd Scene sc nd compute diff 0 moving nodes ground physics l l l
  54. data Scene sc nd = Scene { … } —

    Haskell record type SceneUpdate sc nd = Scene sc nd -> TimeInterval -> Scene sc nd once per frame Scene sc nd lazy marshalling SceneUpdate sc nd Scene sc nd compute diff apply changes 0 moving nodes ground physics l l l
  55. Lazy Marshalling 0 moving nodes ground physics l l l

    lazy marshalling Scene { sceneName = , sceneChildren = , sceneData = , sceneBackgroundColor = ⋮ }
  56. Lazy Marshalling 0 moving nodes ground physics l l l

    lazy marshalling Scene { sceneName = , sceneChildren = , sceneData = , sceneBackgroundColor = ⋮ } thunks (suspended computations)
  57. Lazy Marshalling 0 moving nodes ground physics l l l

    lazy marshalling Scene { sceneName = , sceneChildren = , sceneData = , sceneBackgroundColor = ⋮ } thunks (suspended computations)
  58. Lazy Marshalling marshalSKScene :: SKScene -> Scene sceneData nodeData marshalSKScene

    skScene = Scene { sceneName = unsafePerformIO $(objc … ) , sceneChildren = unsafePerformIO $ do { nodes <- $(objc … ) ; unsafeInterleaveNSArrayTolistOfNode nodes } ⋮ }
  59. Lazy Marshalling marshalSKScene :: SKScene -> Scene sceneData nodeData marshalSKScene

    skScene = Scene { sceneName = unsafePerformIO $(objc … ) , sceneChildren = unsafePerformIO $ do { nodes <- $(objc … ) ; unsafeInterleaveNSArrayTolistOfNode nodes } ⋮ } produces a thunk (unevaluated expression)
  60. Lazy Marshalling marshalSKScene :: SKScene -> Scene sceneData nodeData marshalSKScene

    skScene = Scene { sceneName = unsafePerformIO $(objc … ) , sceneChildren = unsafePerformIO $ do { nodes <- $(objc … ) ; unsafeInterleaveNSArrayTolistOfNode nodes } ⋮ } produces a thunk (unevaluated expression) inline Objective-C
  61. 0 moving nodes ground physics l l l Scene {

    sceneName = , sceneChildren = , sceneData = , sceneBackgroundColor = ⋮ } Compute Diff & Update
  62. 0 moving nodes ground physics l l l Scene {

    sceneName = , sceneChildren = , sceneData = , sceneBackgroundColor = ⋮ } Compute Diff & Update Scene { sceneName = ”New Name” , sceneChildren = , sceneData = , sceneBackgroundColor = ⋮ } SceneUpdate sc nd
  63. 0 moving nodes ground physics l l l Scene {

    sceneName = , sceneChildren = , sceneData = , sceneBackgroundColor = ⋮ } Compute Diff & Update Scene { sceneName = ”New Name” , sceneChildren = , sceneData = , sceneBackgroundColor = ⋮ } SceneUpdate sc nd compute diff
  64. 0 moving nodes ground physics l l l Scene {

    sceneName = , sceneChildren = , sceneData = , sceneBackgroundColor = ⋮ } Compute Diff & Update Scene { sceneName = ”New Name” , sceneChildren = , sceneData = , sceneBackgroundColor = ⋮ } SceneUpdate sc nd compute diff
  65. Compute Diff & Update marshalScene :: Scene sceneData nodeData —

    original scene -> Scene sceneData nodeData — updated scene -> IO SKScene marshalScene originalScene Scene{..} = do { … ; case reallyUnsafePtrEquality# originalName sceneName of 1# -> return () _ -> $(objc …) ; updateChildren skNode originalChildren sceneChildren ⋮ }
  66. Compute Diff & Update marshalScene :: Scene sceneData nodeData —

    original scene -> Scene sceneData nodeData — updated scene -> IO SKScene marshalScene originalScene Scene{..} = do { … ; case reallyUnsafePtrEquality# originalName sceneName of 1# -> return () _ -> $(objc …) ; updateChildren skNode originalChildren sceneChildren ⋮ }
  67. Object Caching & Identity data Node u = Node {

    nodeName :: Maybe String , nodePosition :: Point ⋮ , nodeForeign :: Maybe SKNode } | Label { nodeName :: Maybe String , nodePosition :: Point ⋮ , nodeForeign :: Maybe SKNode , labelText :: String ⋮ } ⋮ if available, basis for marshalling this node to ObjC land