Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Dockercon SF 2018: Self-upgrading Servers with ...

Dockercon SF 2018: Self-upgrading Servers with Docker

Discover how we implemented a server that doesn’t require any downtime to be extended with new features.

Babelfish (bblf.sh) is an open source project created at source{d} that provides language parsing for *any* programming language.

Each parser is written in its own language, so we can’t just simply put everything in a binary.

So how does this work?

We are able to update and upgrade the API server by running Moby inside of Moby (we need to go deeper!) and using the Docker hub as our repository for parsers.

In addition to this, each container image for parsers needs to follow some quite strict specifications: and in order to do so we build a set of images via Makefile and multi-stage builds.

All in all creates an efficient system that although is architecturally complex provides a simple developer experience.

This talk covers all the juicy details of the architecture and the lessons we learned while developing it.

Francesc Campoy Flores

June 14, 2018
Tweet

More Decks by Francesc Campoy Flores

Other Decks in Technology

Transcript

  1. '112', '97', '99', '107', '97', '103', '101', '32', '109', '97',

    '105', '110', '10', '10', '105', '109', '112', '111', '114', '116', '32', '40', '10', '9', '34', '102', '109', '116', '34', '10', '41', '10', '10', '102', '117', '110', '99', '32', '109', '97', '105', '110', '40', '41', '32', '123', '10', '9', '102', '109', '116', '46', '80', '114', '105', '110', '116', '108', '110', '40', '34', '72', '101', '108', '108', '111', '44', '32', '112', '108', '97', '121', '103', '114', '111', '117', '110', '100', '34', '41', '10', '125', '10' package main import “fmt” func main() { fmt.Println(“Hello, Chicago”) } What is Source Code
  2. package package IDENT main ; import import STRING "fmt" ;

    func func IDENT main ( ) package main import “fmt” func main() { fmt.Println(“Hello, Chicago”) } What is Source Code { IDENT fmt . IDENT Println ( STRING "Hello, Chicago" ) ; } ;
  3. … a self-hosted server for universal source code parsing, turning

    code files into Universal Abstract Syntax Trees
  4. Parsing Go in Go is easy Parsing Java in Java

    is easy Parsing Go in Java is doable … I guess Parsing Java in Go is interesting … Parsing C++ in Go is why are you like this? Parsing X in Y
  5. • Good UX ◦ Intuitive commands, flags, args ... ◦

    Reasonable defaults • Frictionless installation ◦ As few dependencies as possible ◦ One line to run is good ◦ No lines to run is even better DevRel: reducing time to “wow”!
  6. Docker in Docker! Using Docker-in-Docker for your CI or testing

    environment? Think twice jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/
  7. Docker in Docker! However, contrary to popular belief, Docker-in-Docker is

    not 100% made of sparkles, ponies, and unicorns
  8. Scaling • Pluggable strategy • Default behavior: ◦ 1 container

    per request ◦ Extra time for reuse • Not implemented yet • Upgrade is for now Remove + Installation bblfshctl driver \ install --update \ foo Upgrading
  9. Docker in Docker Docker Socket - Hard to use, previously

    - Much easier now - Lots of features we don’t use - Requires Docker to be used no matter what libcontainer - Fewer features - Fewer dependencies - Runs on Linux without Docker