rMIXr: how we learned to stop worrying and love the graph

In this talk we present a case study in the use of the web audio APIs. Specifically, our use of them for the creation of a rapidly developed
prototype application. The app, called rMIXr (http://rmixr.com), is a simple digital audio workstation (DAW) for fan remix contests. We created rMIXr in 48 hours at the Midem Hack Day in June 2015. We’ll give a brief demo of the app and show multi-channel sync. We'll also show various effects as well as cutting/time-slicing.
During the talk we will go through our solutions to these issues, showing by example a number of practical ways to get things done with the Web Audio APIs. We will walk through our abstractions and object models for building a fully functional DAW in the browser. These abstractions were necessary because we included a number of effects from third party libraries. Some of these effects libraries do not comply with the “source and sink” model that the web audio API predicates. Our abstractions gave us a more flexible graph structure that was critical to making this application work. While not perfect, it was certainly excellent for the rapid prototype scenario of the hackday. We’ll also talk about some potential improvements to our abstractions. We will then show how to use localstorage to communicate across browser windows and how this approach can be harnessed to quickly create flexible UI elements with multiple windows.
Slides as given at the 2016 Web Audio Conference in Atlanta, GA. Full abstract at https://smartech.gatech.edu/handle/1853/54668

Transcript

1. rMIXr: how we learned to stop worrying and love the graph
2. Ben Fields @alsothings Sam Phippen @samphippen
3. this talk represents our experience using the web audio APIs to go from nothing to a reasonably complete prototype in 48 hours about a year ago. actual API usage may vary.
4. ⚠⚠⚠⚠⚠⚠⚠⚠⚠⚠ ACTUAL API USAGE MAY VARY ⚠⚠⚠⚠⚠⚠⚠⚠⚠⚠
5. Midem Hack Day
6. [[[[midem branding]]]]
8. DJ Spotify festivalBag
9. rMIXr: white-labelled DAW in the browser
13. Effects in rMIXr
14. `
16. Source Sink
17. Source Sink .connect
18. Source Sink .connect
19. Source Sink Fx/ Analysis
20. Source Sink Fx/ Analysis .connect
21. Source Sink Fx/ Analysis .connect .connect
26. OMG NO
27. Source Sink Fx/ Analysis .connect .connect
28. Converb Converb .input Source Sink .connect .connect
36. How to do UI for effects?
43. Icon font didn’t load lol
49. Ben will explain how this works
50. The local storage API is totally a message bus
51. Last write wins is totally a good consistency model
53. server side audio rendering is for suckers
54. thus: need to serialise state and associate it with page location
56. every time you change a setting,
57. every time you change a setting, the new state is serialised to the url
58. generalise to *all* UI elements
63. a channel’s state channel strip’s state
65. hash decoding all the way down
66. http://rmixr.com/queen/#{"channels":[{"effects":[],"loopPoints": [],"notmuted":true},{"effects":[{"name":"threebandEQ","params": {"bassgain":"21","midgain":"-24","treblegain":"16"}}, {"name":"chorus","params":{"rate":2.41,"feedback":0.2,"delay": 0.05}}],"loopPoints":[]},{"effects":[],"loopPoints": [],"notmuted":true},{"effects":[{"name":"converb","params": {"dry":0.5,"wet":0.59}}],"loopPoints":[],"notmuted":false}, {"effects":[{"name":"stereoPan","params": {"pan":-0.1}}],"loopPoints":[],"notmuted":true},{"effects": [{"name":"stereoPan","params":{"pan":0.11}}],"loopPoints": [],"notmuted":false}]}
67. the fragment id *is* the application state engine
68. at least tweeting and entry submission is easy
70. thanks twitter.
72. at least tweeting and entry submission is easy
73. at least the back button works
74. {this is where we attempt a demo}
75. so:
76. unifying abstractions
77. local storage can be a messaging queue
78. even single page web apps can be RESTful
79. Let’s have some questions a!/samphippen !/alsothings