Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The Lost Art of MIDI – Bringing <bgsound> Back ...

The Lost Art of MIDI – Bringing <bgsound> Back to the Web

In the days of Geocities and Angelfire, a quirky HTML tag called ⟨bgsound⟩ enabled sound files to play in the background of webpages. Usually, these files were in the MIDI format. What a glorious era that was! Sadly, ⟨bgsound⟩ has been removed from browsers and MIDI is obscure and hard to play back. In this talk, we'll bring MIDI and ⟨bgsound⟩ back from the dead using WebAssembly, Emscripten, Web Audio, and Web Components. When we're finished, you'll be able to give your webpages the 90's treatment in a modern, standards-compliant way!

Links from the talk:

Timidity – Play MIDI files in the browser w/ Web Audio, WebAssembly, and libtimidity
https://github.com/feross/timidity

bg-sound – Web Component to emulate the old-school HTML element
https://github.com/feross/bg-sound

BitMidi – Listen to free MIDI songs, download the best MIDI files, and share the best MIDIs on the web
https://bitmidi.com

Feross's Blog
https://feross.org

Feross's Twitter
https://twitter.com/feross

Feross Aboukhadijeh

November 16, 2018
Tweet

More Decks by Feross Aboukhadijeh

Other Decks in Technology

Transcript

  1. The Lost Art of MIDI Bringing <bgsound> Back to the

    Web Image Credit: hexeosis.tumblr.com
  2. • 1971 – FTP • 1974 – TCP • 1982

    – MIDI • 1991 – HTTP • 1995 – SSH • 2001 – BitTorrent • 2009 – Bitcoin • 2015 – HTTP/2
  3. 90 3C 40 -----------------> time • 90 = Note on

    • 3C = Which key? • 40 = How hard was it pressed?
  4. General MIDI • 128 notes (~10 octaves) • 16 channels

    • 128 programs (instrument sounds)
  5. 1-8 Piano 9-16 Chromatic Percussion 17-24 Organ 25-32 Guitar 33-40

    Bass 41-48 Strings 49-56 Ensemble 57-64 Brass 65-72 Reed 73-80 Pipe 81-88 Synth Lead 89-96 Synth Pad 97-104 Synth Effects 105-112 Ethnic 113-120 Percussive 121-128 Sound Effects
  6. Pianos 1. Acoustic Grand Piano 2. Bright Acoustic Piano 3.

    Electric Grand Piano 4. Honky-tonk Piano 5. Electric Piano 1 6. Electric Piano 2 7. Harpsichord 8. Clavi
  7. static void load_instrument(MidSong *song, ...) { ... if (song->ifp ==

    NULL) { DEBUG_MSG("Instrument `%s' can't be found.\n", name); // Added for JavaScript port song->load_requests[song->load_request_count] = strdup(name); song->load_request_count += 1; ... }
  8. // Added for JavaScript port extern int mid_get_load_request_count(MidSong *song) {

    return song->load_request_count; } extern char *mid_get_load_request(MidSong *song, int index) { return song->load_requests[index]; }
  9. Pointers // Allocate memory const ptr = lib._malloc(byteLength) // Free

    memory lib._free(ptr) // Convert a `char*` to a JavaScript string const str = lib.Pointer_stringify(ptr)
  10. function loadSong (buffer) { const bufferPtr = lib._malloc(buffer.byteLength) lib.HEAPU8.set(buffer, bufferPtr)

    const iStreamPtr = lib._mid_istream_open_mem(bufferPtr, buffer.byteLength) const songPtr = lib._mid_song_load(iStreamPtr) return songPtr }
  11. let songPtr = loadSong(buffer) const instruments = getMissingInstruments(songPtr) if (instruments.length

    > 0) { await Promise.all( instruments.map(instrument => fetchInstrument(instrument)) ) lib._mid_song_free(songPtr) songPtr = loadSong(buffer) } lib._mid_song_start(songPtr)
  12. function getMissingInstruments (songPtr) { const missingCount = lib._mid_get_load_request_count(songPtr) const missingInstruments

    = [] for (let i = 0; i < missingCount; i++) { const instrumentPtr = lib._mid_get_load_request(songPtr, i) const instrument = lib.Pointer_stringify(instrumentPtr) missingInstruments.push(instrument) } return missingInstruments }
  13. async function fetchInstrument (instrument) { const response = await fetch(`${urlPrefix}/${instrument}`)

    const buf = await response.arrayBuffer() lib.FS.writeFile(instrument, buf) }
  14. function onAudioProcess (event) { const sampleCount = readMidiData() const output0

    = event.outputBuffer.getChannelData(0) const output1 = event.outputBuffer.getChannelData(1) for (let i = 0; i < sampleCount; i++) { output0[i] = array[i * 2] / 0x7FFF output1[i] = array[i * 2 + 1] / 0x7FFF } for (let i = sampleCount; i < BUFFER_SIZE; i++) { output0[i] = 0 output1[i] = 0 } }
  15. function readMidiData () { const byteCount = lib._mid_song_read_wave(songPtr, buffer, ...)

    array.set( lib.HEAP16.subarray(bufferPtr / 2, (bufferPtr + byteCount) / 2) ) return byteCount / BYTES_PER_SAMPLE // sampleCount }
  16. const Timidity = require('timidity') const player = new Timidity() player.load('/my-file.mid')

    player.play() player.on('timeupdate', seconds => { console.log(seconds) })
  17. Make it a Web Component Before <bgsound src="sound.mid"> After <bg-sound

    src="sound.mid"></bg-sound> <script src="bg-sound.min.js"></script>
  18. class BgSound extends HTMLElement { connectedCallback () { this.player =

    new Timidity() this.player.load(this.src) this.player.play() } disconnectedCallback () { this.player.destroy() } } window.customElements.define('bg-sound', BgSound)
  19. The Mythical Man-Month The programmer, like the poet, works only

    slightly removed from pure thought-stuff. He builds castles in the air, from air, creating by exertion of the imagination. Few media of creation are so flexible, so easy to polish and rework, so readily capable of realizing grand conceptual structures. — Fred Brooks