Orchestration in 2014 - WindyCityRails

ORCHESTRATION IN 2014 Dr Nic Williams starkandwayne.com @drnic

STARK & WAYNE | DR NIC | ORCHESTRATION IN 2014
| WINDY CITY RAILS 2014 In the beginning, there was a real-world problem

| WINDY CITY RAILS 2014 Unique business problems don't require unique deployment solutions

| WINDY CITY RAILS 2014 orchestration? • Costs of software systems • What we really need? • What exists today? • What's still missing?

| WINDY CITY RAILS 2014 @drnic • Previously at Engine Yard • Lots and lots of Ruby projects • 2 yrs consulting around Cloud Foundry • Investigates all the things

| WINDY CITY RAILS 2014 Stark & wayne • 8 staff • Cloud Foundry training, installations, managed service • Consultancy for nextgen Platforms • blog.starkandwayne.com

| WINDY CITY RAILS 2014

| WINDY CITY RAILS 2014 COST OF SOFTWARE SYSTEMS

| WINDY CITY RAILS 2014 your business sales $$$$ profit $$$$

| WINDY CITY RAILS 2014 sales $$$$ your business profit $$$$ cost of sales -$

| WINDY CITY RAILS 2014 sales $$$$ your business profit $$$$ cost of sales -$ R&D -$

| WINDY CITY RAILS 2014 sales $$$$ your business profit $$$$ cost of sales -$ R&D -$ operations -$

| WINDY CITY RAILS 2014 cost of iT r&d • Avg salary in SF $150k ($200k total) • Costs explode with bespoke software • Risks of failure • Increases in operational costs

| WINDY CITY RAILS 2014 sales $$$$ your business loss $$ cost of sales -$ R&D -$$$ operations -$$ DIY software can dramatically change P&L

| WINDY CITY RAILS 2014 sales 0 your business losses $$$$$ cost of sales 0 R&D -$$$ operations -$$ Outages dramatically change everything

| WINDY CITY RAILS 2014 You are expensive :)

| WINDY CITY RAILS 2014 value • Amount of value from R&D? • Time until value? • Guarantee of value?

| WINDY CITY RAILS 2014 Buy this today http://bit.ly/phoenixproject

| WINDY CITY RAILS 2014 We are in IT

| WINDY CITY RAILS 2014 Embedded in conference

| WINDY CITY RAILS 2014 Shortest name

| WINDY CITY RAILS 2014 WHAT WE REALLY NEED?

| WINDY CITY RAILS 2014 CIOs: next 10 yrs • Goal: reduce time to value • Goal: unify access to all data • Goal: mobile access

| WINDY CITY RAILS 2014 • Goal: reduce time to value • Goal: unify access to all data • Goal: mobile access • Solution: Orchestration • Solution: API everything

| WINDY CITY RAILS 2014 YOUR ORG

| WINDY CITY RAILS 2014 orchestration goals for engineering http://12factor.net/

| WINDY CITY RAILS 2014 goals for orchestration • Deploy quickly - apps/dependencies • Deploy language/framework of choice • HTTP routing to web processes • Integrate DBs/services/APIs • Streaming logs, including platform components • Simple, fast scaling • Automatic health recovery • . . .

| WINDY CITY RAILS 2014 goals for orchestration • Deploy quickly - apps/dependencies • Deploy language/framework of choice • HTTP routing to web processes • Integrate DBs/services/APIs • Streaming logs, including platform components • Simple, fast scaling • Automatic health recovery • Packaging assurance/version control • Reproduce production for debugging • Auditing who/what did what change • Access controls; Share platform across organization(s) • Proportional billing to internal organizations

| WINDY CITY RAILS 2014 goals for orchestration • Let you focus on engineering • Let you focus on being valuable

| WINDY CITY RAILS 2014 WHAT EXISTS TODAY? Orchestration

| WINDY CITY RAILS 2014 Guide to total value Every solution costs

| WINDY CITY RAILS 2014 Use these until someone says you can't...

| WINDY CITY RAILS 2014 Use these until someone says you can't... • $10.80/mth/512Mb • No free containers • Flexible container sizes • Portable to own infrastructure • $33.60/mth/512Mb • First container free • 512M, 1G, 6G container sizes • No portal/private option

| WINDY CITY RAILS 2014 • Deploy quickly - apps/dependencies • Deploy language/framework of choice • HTTP routing to web processes • Integrate DBs/services/APIs • Streaming logs, including platform components • Simple, fast scaling • Automatic health recovery yes yes Heroku buildpacks yes yes yes yes yes yes yes yes yes yes

| WINDY CITY RAILS 2014 • Packaging assurance/version control • Reproduce production for debugging • Auditing who/what did what change • Access controls; Share platform across organization(s) • Proportional billing to internal organizations • Portable platform - AWS, GCE, OpenStack, etc no yes yes no yes yes yes yes yes no Heroku buildpacks

| WINDY CITY RAILS 2014 We want our own orchestration

| WINDY CITY RAILS 2014 sales $$$$ your business loss $$ cost of sales -$ R&D -$$$ operations -$$ Beware DIY Orchestration

| WINDY CITY RAILS 2014 no unix app for all this • Deploy quickly - apps/dependencies • Deploy language/framework of choice • HTTP routing to web processes • Integrate DBs/services/APIs • Streaming logs, including platform components • Simple, fast scaling • Automatic health recovery • Packaging assurance/version control • Reproduce production for debugging • Auditing who/what did what change • Access controls; Share platform across organization(s) • Proportional billing to internal organizations • Portable platform - AWS, GCE, OpenStack, etc

| WINDY CITY RAILS 2014 Orchestration is a system not an app

| WINDY CITY RAILS 2014 Double check why you can't just use these... before proceeding...

| WINDY CITY RAILS 2014 What about Docker?

| WINDY CITY RAILS 2014 docker is delightful • packaging • containers • process monitoring (v1.2) • port & data binding to host machine • big community of example Dockerfiles • demos really well!

| WINDY CITY RAILS 2014 1. 2. 3. Local Docker

| WINDY CITY RAILS 2014 • Deploy quickly - apps/dependencies • Deploy language/framework of choice • HTTP routing to web processes • Integrate DBs/services/APIs • Streaming logs, including platform components • Simple, fast scaling • Automatic health recovery yes no yes yes yes yes yes cluster of docker containers... on single docker server

| WINDY CITY RAILS 2014 • Deploy quickly - apps/dependencies • Deploy language/framework of choice • HTTP routing to web processes • Integrate DBs/services/APIs • Streaming logs, including platform components • Simple, fast scaling • Automatic health recovery yes no yes no no no yes cluster of docker containers... across cluster of servers that come & go

| WINDY CITY RAILS 2014 • Packaging assurance/version control • Reproduce production for debugging • Auditing who/what did what change • Access controls; Share platform across organization(s) • Proportional billing to internal organizations • Portable platform - AWS, GCE, OpenStack, etc yes no no no yes yes cluster of docker containers... across cluster of servers that come & go

| WINDY CITY RAILS 2014 Docker is a unit to be orchestrated

| WINDY CITY RAILS 2014 Need system-wide orchestration

| WINDY CITY RAILS 2014 Projects orchestrating docker • Cloud Foundry • Mesos • Deis • Flynn • Flocker • Open Shift

| WINDY CITY RAILS 2014 trycf.starkandwayne.com

| WINDY CITY RAILS 2014 demo

| WINDY CITY RAILS 2014 running orchestration in production • Cloud Foundry - BOSH • Mesos - DIY • Deis - DIY • Flynn - DIY • Flocker - DIY • Open Shift - DIY

| WINDY CITY RAILS 2014 • Deploy quickly - apps/dependencies • Deploy language/framework of choice • HTTP routing to web processes • Integrate DBs/services/APIs • Streaming logs, including platform components • Simple, fast scaling • Automatic health recovery • . . . http://docs.cloudfoundry.org/concepts/architecture/

| WINDY CITY RAILS 2014 BOSH also resurrects all servers No single points of failure

| WINDY CITY RAILS 2014 BOSH also scales servers up and out BOSH upgrades across versions v179 v180

| WINDY CITY RAILS 2014 Why BOSH? > Platform operators deserve nice things too

| WINDY CITY RAILS 2014 goals for orchestration • Deploy quickly - apps/dependencies • Deploy language/framework of choice • HTTP routing to web processes • Integrate DBs/services/APIs • Streaming logs, including platform components • Simple, fast scaling • Automatic health recovery • Packaging assurance/version control • Reproduce production for debugging • Auditing who/what did what change • Access controls; Share platform across organization(s) • Proportional billing to internal organizations • Portable platform - AWS, GCE, OpenStack, etc

| WINDY CITY RAILS 2014 suggestions • Avoid DIY • Try CF - trycf.starkandwayne.com • full lifecycle orchestration ‣ Cloud Foundry for running Apps ‣ BOSH for running Cloud Foundry

| WINDY CITY RAILS 2014 WHAT'S STILL MISSING?

| WINDY CITY RAILS 2014 Orchestrating data services is hard

| WINDY CITY RAILS 2014 Buy this today http://bit.ly/phoenixproject

Orchestration in 2014 - WindyCityRails

Orchestration in 2014 - WindyCityRails

More Decks by Dr Nic Williams

Other Decks in Technology

Featured

Transcript