Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Comeback of Grids

ProdOps
April 14, 2016

Comeback of Grids

The year 2015 put containers in the spotlight, but now that everyone is using Docker it is time to put containers in production. Grid clusters are again on the rise, this time in the form of Google Kubernetes, Docker Swarm, Amazon ECS and Mesosphere just to name a few. This talk will explain the basic principles of how grids solve problems related to managing large fleets of containers, and what new challenges grids introduce to application developers.

Presented at http://www.meetup.com/London-DevOps/events/229579550/ and at https://devcontlv-march2016.events.co.il/

ProdOps

April 14, 2016
Tweet

More Decks by ProdOps

Other Decks in Technology

Transcript

  1. Agenda • Buzzing trends in Soft. Architecture • Containers •

    Schedulers • Containers inside schedulers
  2. What is MicroServices Architecture? Loosely coupled service oriented architecture with

    bounded contexts not updated at the same time minimal knowledge of own environment
  3. monitoring configuration router layout data access queue cache scheduler micro

    services infrastructure services authentication authorization form handling file uploads web scraping indexing searching application services e-mail pdf generation payments workflow engine pagination load balancer
  4. “If you can’t feed a team with two pizzas, it’s

    too large. That limits a task force to five to seven people, depending on their appetites” - Jeff Bezos medium.com/@benorama/ the-evolution-of-software-architecture-bd6ea674c477
  5. Infrastructure kubernetes coreos / fleet geard mesos or or or

    consul helios etcd centurion EC2 Rack space GCE DO Tutum Heroku
  6. Mesos is a Distributed OS kernel • • • •

    • mesosphere.com/blog/2015/04/23/apple-details-j-a-r-v-i-s-the-mesos-framework-that-runs-siri/ www.wired.com/2013/03/google-borg-twitter-mesos
  7. Mesos - abstract resource manager virtual machines docker App A

    App A MESOS host | host | host | host | host | host
  8. github.com/mesosphere/marathon • Highly available schedulers • Constraints for application instances

    • Service discovery - DNS & Load balancing • Health checks • Event subscription endpoint • Graphical UI • JSON / REST APIs for scriptability • Metrics Mesosphere Marathon
  9. • Docker compatible • Grid/cluster management • AWS APIs •

    Custom & default scheduler • Docker Compose support for local dev • Monitoring / Logging • Custom docker repositories + AWS Repo Elastic Container Service
  10. • Self healing • Service discovery & load balancing •

    Secrets & config management • Automated rollouts & rollbacks • Storage orchestration • Automatic binpacking
  11. Docker Swarm • Docker Engine API • Integrated Networking &

    Volumes • High Scalability - 1k hosts 50k containers • High Availability with failovers • Pluggable scheduler • Pluggable host discovery
  12. • Task Groups == K8s Pods == Docker Compose •

    Service discovery - Consul • Health checks - HTTP • Containers - or - ◦ Docker / Rocket / runc … ◦ Single binary apps: Java / Go / … • or! Virtual Machines (like OpenStack Nova) • “Run Everywhere”, “Batch”, “Service”, “Cron” • Constraints • Multi Datacenter + Continental • Easy installation using GO & single binary • Lack of Zookeeper - uses Consul instead • Monitoring • Pluggable service discovery: Consul / ZK / etcd / … • Self healing • Automatic binpacking • Orhestration of tasks on Windows
  13. Thank you! We invite you to join Operations UK Facebook

    group on dvps.me/OpsUK www.devopspro.co.uk