Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Writing a High Performance Database in Go

Sponsored · SiteGround - Reliable hosting with speed, security, and support you can count on. →

Writing a High Performance Database in Go

My talk from GopherCon 2014.

benbjohnson

April 24, 2014

Transcript

44 slides
  1. Things that need to be really f*cking fast Things that

    need to be pretty fast User Management Schema Management Query Parsing Backup / Recovery Bulk Data Insertion etc...
  2. Things that need to be really f*cking fast Things that

    need to be pretty fast Query Execution User Management Schema Management Query Parsing Backup / Recovery Bulk Data Insertion etc...
  3. Basics of Bolt Pure Go port of LMDB Memory-mapped B+tree

    MVCC, ACID transactions Zero copy reads
  4. Batch Size 1 Bolt Batch Benchmarks Performance 10 100 1000

    Baseline 9x Baseline 45x Baseline 90x Baseline Disclaimer: YMMV
  5. Use a channel to stream changes Transaction Coalescing Group changes

    into single transaction Either all changes commit or rollback
  6. See also: Albert Strasheim’s “Serialization in Go” Talk Encoding Performance

    http://www.slideshare.net/albertstrasheim/serialization-in-go https://github.com/cloudflare/goser
  7. // Create a byte slice with the same size as

    type T. var value = make([]byte, unsafe.Sizeof(T{}) // Map a typed pointer from the byte slice and update it. var t = (*T)unsafe.Pointer(&value[0]) t.ID = 123 t.MyIntValue = 20 // Insert value into database. db.Update(func(tx *bolt.Tx) error { return tx.Bucket(“T”).Put([]byte(“123”), value) }) Map a struct to a []byte
  8. // Start a read transaction. db.View(func(tx *bolt.Tx) error { c

    := tx.Bucket(“T”).Cursor() // Iterate over each value in the bucket. for k, v := c.First(); k != nil; k, v = c.Next() { var t = (*T)unsafe.Pointer(&value[0]) // ... do something with “t” ... } return nil }) Map a []byte to a struct

More Decks by benbjohnson

Other Decks in Technology