Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Go gamedev: XM music

Iskander (Alex) Sharipov
February 27, 2024
64

Go gamedev: XM music

Iskander (Alex) Sharipov

February 27, 2024
Tweet

Transcript

  1. My Go gamedev story • I create games with Ebitengine

    • I make libraries for gamedev in Go • I write talks and articles about gamedev in Go • t.me/go_gamedev (Russian-speaking) creator I’m using Ebitengine for around 2-3 years now
  2. Desktop: Linux, Windows, MacOS Mobile: Android Also works in your

    browser (itch.io) Has Steam integration (achievements, etc.)
  3. Ebitengine audio for music • Supports mp3 and ogg out-of-the

    box • Your own stream reader implementation is possible
  4. Ebitengine audio for music • Supports mp3 and ogg out-of-the

    box • Your own stream reader implementation is possible • Works with 16-bit 2-channel PCM LE streams
  5. Ebitengine audio for music • Supports mp3 and ogg out-of-the

    box • Your own stream reader implementation is possible • Works with 16-bit 2-channel PCM LE streams • Works well on every platform I tested my games on
  6. Stereo 16-bit PCM Little Endian • PCM are given to

    the audio driver as a final step • OGG and MP3 formats allow compact storage • A ~4 min PCM data can have a size of ~50MB This is why most players “decode” OGG/MP3 into PCM on-the-fly, so you can avoid this large memory overhead.
  7. music.ogg stream player audio sys Stream Reads OGG data and

    turns them into the 16-bit PCM LE bytes the player expects to get.
  8. music.ogg stream player audio sys (audio) Player This is your

    audio system API object. It’s a bridge between your stream implementation and the underlying audio system. Players are reusable, they wrap a single stream at a time. You can create tons of Player objects in your game.
  9. music.ogg stream player audio sys Audio system This part is

    usually unseen for a game developer. We can assume that it’s some kind of a low-level library that speaks to the audio systems on different platforms.
  10. Roboden music story I used Drozerix tracks from modarchive for

    my Roboden game. They were in XM format, so I converted them to OGG.
  11. Roboden music story I used Drozerix tracks from modarchive for

    my Roboden game. They were in XM format, so I converted them to OGG. At some point, the game archive became quite big for a web build.
  12. Problems with OGG (and MP3) • Large size (a problem

    for mobiles and web) • Lack of the “sources” (they’re also “lossy”)
  13. Problems with OGG (and MP3) • Large size (a problem

    for mobiles and web) • Lack of the “sources” (they’re also “lossy”) • Harder to do dynamic fancy stuff with the sound
  14. Let’s go one step back The “source” of my music

    (Drizerix tracks) is XM. XM file size: 71 KB Converted OGG file size: 1.8 MB (~1843 KB) It’s about x25 times smaller!
  15. The modular music • Smaller file size • The music

    file itself is a source • Almost the “code is data” approach
  16. Some games that used modular music • Deus Ex (2000,

    IT format) • Unreal Tournament (1998, IT format) • Age of Wonders (1996, IT format) • Star Control 2 (1992, MOD format) • Several first Final Fantasy games (MOD format) …most modular formats can be converted to XM
  17. XM music format Stands for “Extended MOD”. It’s like MOD,

    but better (it’s even more compact thanks to the simple compression scheme).
  18. OGG XM Stores the compressed music track data Stores the

    instructions about how to play the music and samples data
  19. OGG XM Stores the compressed music track data Stores the

    instructions about how to play the music and samples data Can’t be edited by a human Can be easily edited using a Tracker software
  20. OGG XM Stores the compressed music track data Stores the

    instructions about how to play the music and samples data Can’t be edited by a human Can be easily edited using a Tracker software Can’t be transformed on-the-fly during the playback Can be manipulated by a program in many ways
  21. OGG XM Stores the compressed music track data Stores the

    instructions about how to play the music and samples data Can’t be edited by a human Can be easily edited using a Tracker software Can’t be transformed on-the-fly during the playback Can be manipulated by a program in many ways Avg. size is 3-7 MB Avg. size is 50-500 KB
  22. Comparing XM, IT, S3M • All of them are modular

    music formats • XM and IT are less limiting than S3M
  23. Comparing XM, IT, S3M • All of them are modular

    music formats • XM and IT are less limiting than S3M • XM is more popular than the other two nowadays
  24. Comparing XM, IT, S3M • All of them are modular

    music formats • XM and IT are less limiting than S3M • XM is more popular than the other two nowadays • MilkyTracker can convert IT and S3M to XM
  25. music.xm stream player audio sys music.xm Contains the instructions for

    an XM-player. Also stores the necessary samples data inside the XM file.
  26. music.xm stream player audio sys Stream Plays a role of

    an XM-player. It evaluates the XM instructions and produces the output PCM bytes.
  27. XM file layout 0 1 2 3 4 5 6

    7 8 9 Pattern order (just an array of indexes) 0, 1, 0, 0, 2, 3, 4, 4, 4, 5, 6, 7, 1, 8, 9, 2, 0, 1, 1
  28. • Channel number • Instrument ID • Notes (pitch) •

    Weapon type • Weapon owner • Projectile power Using music data as gameplay elements
  29. • Channel number • Instrument ID • Notes (pitch) •

    Weapon type • Weapon owner • Projectile power Using music data as gameplay elements
  30. • Channel number • Instrument ID • Notes (pitch) •

    Weapon type • Weapon owner • Projectile power Using music data as gameplay elements
  31. for _, patternIndex := range t.Module.PatternOrder { p := &t.Module.Patterns[patternIndex]

    for j := range p.Rows { row := &p.Rows[j] for _, noteID := range row.Notes { n := module.Notes[noteID] kind := t.GetInstrumentKind(n.Instrument) if kind != edrum.UndefinedInstrument { // Skip this instrument. It will be played by the player. continue } // Remove this note from the track. } } }
  32. Step 2: extract selected instrument samples Can be done programmatically

    or manually via Tracker software (like MilkyTracker).
  33. Step 3: create a note map For every note “removed”

    from the track, remember its timings and other info like instrument type. Render these note bars to the players when they need to play them.
  34. Step 4: read the MIDI stream For every MIDI “play

    note” event play instrument’s associated sample. gitlab.com/gomidi/midi/ MIDI device gomidi game PC
  35. Summary • The track is played without drums • There

    is an interactive drum notes map • The drum will play original samples
  36. Summary • The track is played without drums • There

    is an interactive drum notes map • The drum will play original samples • Every drum stroke is /real/ and affects the song
  37. What else can we do? • Collect player stats, like

    rhythm consistency • Create tab sheets for an XM track automatically • Play XM tracks at different speed and effects • This is not limited to drums-only, any MIDI-device will do • Record the player and build a combined XM track • Build colored sound wave based on inst&chan index
  38. My XM player library for Go • High performance (zero-alloc

    repeated plays) • Sample interpolation & volume ramping support • Dependency-free • Ebitengine-compatible • Exports XM files and parsers github.com/quasilyte/xm
  39. XM playback There are two main aspects to it: 1.

    Evaluating the effects/notes for a “tick” 2. Rendering the PCM bytes for the given tick (1) is XM-specific, (2) is what any player would do Rendering the PCM dominates the run time: 90-95%
  40. Benchmarks We’ll be using 3 different tracks: 1. Industrial Porn

    (Drozerix) 2. Old Bulls (Aruan); a MOD file converted to XM 3. Crush (Drozerix) OGG player uses the converted XM->OGG file
  41. Benchmarks There are 2 main parts of playing the music:

    • Loading the file (preparing it to be played) • Streaming the PCM bytes (playing the music)
  42. Benchmarks: decoding (ns/op) Benchmark OGG XM XM (lerp) Decode1 6.27

    ms 3.30 ms 3.46 ms Decode2 4.95 ms 1.56 ms 3.58 ms Decode3 5.03 ms 4.45 ms 4.98 ms
  43. Benchmarks: decoding (ns/op) Benchmark OGG XM XM (lerp) Decode1 slowest

    ~90% faster ~80% faster Decode2 slowest ~317% faster ~38% faster Decode3 slowest ~13% faster ~same
  44. Benchmarks: playing (ns/op) Benchmark OGG XM XM (lerp) Play1 4245

    ms 1235 ms Same as previous Play2 4292 ms 540 ms Same as previous Play3 2609 ms 1627 ms Same as previous
  45. Benchmarks: playing (ns/op) Benchmark OGG XM XM (lerp) Play1 slowest

    ~343% faster Same as previous Play2 slowest ~795% faster Same as previous Play3 slowest ~160% faster Same as previous
  46. Benchmarks: conclusion • XM players are not slow • XM

    players can be zero alloc If XM-style music fits your game, use it directly instead of converting it to OGG (or MP3)
  47. Stages separation • Decoding: compile the XM module • Playback:

    generate PCM bytes from the module Compilation happens only once. A module can be played multiple times. This library favors the playback efficiency (zero alloc).
  48. Sample loops A sample can “loop”: • Forward loop •

    Ping-pong loop (bidirectional) This means there are 3 “modes”: no loop, forward, pingpong
  49. Sample loops A sample can “loop”: • Forward loop •

    Ping-pong loop (bidirectional) This means there are 3 “modes”: no loop, forward, pingpong We can unify all of them (for branchless performance)
  50. Ping-pong loop 0 1 2 3 4 Played as 0,

    1, 2, 3, 4, 3, 2, 1, …
  51. Unrolled ping-pong loop 0 1 2 3 4 Loop start

    Loop end 3 2 1 Now we only have “forward” loops
  52. Sample interpolation (lerp, etc.) There are (at least) two ways:

    • A genuine interpolation during a playback • A precomputed subsamples approach My library uses the latter
  53. Precomputed subsamples • Injects subsamples during the track compilation •

    Requires more memory due to the extra samples • Has zero CPU cost during the playback
  54. Precomputed subsamples • Injects subsamples during the track compilation •

    Requires more memory due to the extra samples • Has zero CPU cost during the playback • Can be sample-size dependent (adaptive)
  55. Volume ramping Only a few first bytes of the “tick”

    require ramping. Process “tick” in two loops: with and without ramping.
  56. n := s.module.bytesPerTick const rampBytes = 2 * 2 *

    numRampPoints for i := 0; i < rampBytes; i += 4 { // ... generate PCM with ramping } // 80-90% of bytes don’t need ramping: for i := rampBytes; i < n; i += 4 { // ... generate PCM without ramping (super fast) }
  57. Using other modular music formats These formats can be converted

    to XM easily: • MOD -> XM (I use MilkyTracker for this conversion) • S3M -> XM (MilkyTracker and modplug) • IT -> XM (MilkyTracker) Amiga frequencies can be converted to linear too.
  58. Links • XM file format overview • A tiny XM

    player implementation in C • MilkyTracker sources (implements XM as well) • Modarchive (modular music collection) • My XM library for Go • Ebitengine Discord channel (international)
  59. What I want you to remember from this talk •

    Game development in Go is a thing (try it out!)
  60. What I want you to remember from this talk •

    Game development in Go is a thing (try it out!) • Modular music (esp. XM) is still relevant
  61. What I want you to remember from this talk •

    Game development in Go is a thing (try it out!) • Modular music (esp. XM) is still relevant • You can play the XM music in Ebitengine directly
  62. What I want you to remember from this talk •

    Game development in Go is a thing (try it out!) • Modular music (esp. XM) is still relevant • You can play the XM music in Ebitengine directly • Modular music can sound cool (Deus Ex OST, Drozerix)
  63. What I want you to remember from this talk •

    Game development in Go is a thing (try it out!) • Modular music (esp. XM) is still relevant • You can play the XM music in Ebitengine directly • Modular music can sound cool (Deus Ex OST, Drozerix) • XM players are not slow (see benchmarks)