Evolution of Microservices - Adrian Cockroft

Discussion of techniques to solve common problems that people run into when microservices are in production and being updated continuously in large organizations.

Adrian Cockcroft has had a long career working at the leading edge of technology. He’s always been fascinated by what comes next, and he writes and speaks extensively on a range of subjects. At Battery, he advises the firm and its portfolio companies about technology issues and also assists with deal sourcing and due diligence.

Before joining Battery, Adrian helped lead Netflix’s migration to a large scale, highly available public-cloud architecture and the open sourcing of the cloud-native NetflixOSS platform. Prior to that at Netflix he managed a team working on personalization algorithms and service-oriented refactoring.

Adrian was a founding member of eBay Research Labs, developing advanced mobile applications and even building his own homebrew phone, years before iPhone and Android launched. As a distinguished engineer at Sun Microsystems he wrote the best-selling “Sun Performance and Tuning” book and was chief architect for High Performance Technical Computing.

He graduated from The City University, London with a Bsc in Applied Physics and Electronics, and was named one of the top leaders in Cloud Computing in 2011 and 2012 by SearchCloudComputing magazine. He can usually be found on Twitter @adrianco.

Transcript

1. The Evolution of Microservices Adrian Cockcroft @adrianco Technology Fellow - Battery Ventures slideshare.net/adriancockcroft June 2016
2. What does @adrianco do? @adrianco Technology Due Diligence on Deals Presentations at Companies and Conferences Tech and Board Advisor Support for Portfolio Companies Consulting and Training Networking with Interesting People Tinkering with Technologies Vendor Relationships Previously: Netflix, eBay, Sun Microsystems, Cambridge Consultants, City University London - BSc Applied Physics
3. Agenda Why now? Microservice Architectures What’s Missing? Migration and Simulation What’s Next?
4. Key Goals of the CIO? Align IT with the business Develop products faster Try not to get breached
5. Security Blanket Failure Insecure applications hidden behind firewalls make you feel safe until the breach happens… http://peanuts.wikia.com/wiki/Linus'_security_blanket
6. “Web scale” vs. “Enterprise”
7. “Webscale” Freedom and responsibility High trust
8. “Enterprise” Bureaucracy and blame Low trust
9. How can everyone get speed, low cost, and better usability?
10. Mixed methods: Disaggregation into microservices helps!
11. @adrianco Example Monolith: Sign Up Login Home Page Payment Method Personal Data Reports Monolithic “kitchen sink” database Monolithic application Complex mix of queries User Because one part of the monolithic application and database holds sensitive data all of it is subject to the most rigorous policies
12. @adrianco Microservices version: Sign Up Login Home Page Payment Method Personal Data Reports Optimized datastores Microservices separation of concerns Isolated single purpose connections User Because each microservice can conform to the appropriate policy, demands for agility can be separated from requirements for security Segregated team owns secure data sources and infrequent updates Segregated team owns rapid improvement of most common use cases
13. @adrianco In Search of Segmentation Ops Dev Datacenters AD/LDAP Roles VLAN Networks Hypervisor IPtables Docker Links AWS Accounts IAM Roles VPC Security Groups Calico Policy Docker Net/Weave
14. @adrianco Hierarchical Segmentation B C A B C E F D E F Homepage Team Security Group Reports Team Security Group VPC Z - Manage a small number of large network spaces D An AWS oriented example… AWS Account - Manage across multiple accounts containers and links
15. “You build it, you run it.” Werner Vogels 2006
16. Developer responsibilities: Faster, cheaper, safer
17. Speeding Up The Platform Datacenter Snowflakes • Deploy in months • Live for years
18. Speeding Up The Platform Datacenter Snowflakes • Deploy in months • Live for years Virtualized and Cloud • Deploy in minutes • Live for weeks
19. Speeding Up The Platform Datacenter Snowflakes • Deploy in months • Live for years Virtualized and Cloud • Deploy in minutes • Live for weeks Container Deployments • Deploy in seconds • Live for minutes/hours
20. Speeding Up The Platform Datacenter Snowflakes • Deploy in months • Live for years Virtualized and Cloud • Deploy in minutes • Live for weeks Container Deployments • Deploy in seconds • Live for minutes/hours Lambda Deployments • Deploy in milliseconds • Live for seconds
21. Speeding Up The Platform AWS Lambda is leading exploration of serverless architectures in 2016 Datacenter Snowflakes • Deploy in months • Live for years Virtualized and Cloud • Deploy in minutes • Live for weeks Container Deployments • Deploy in seconds • Live for minutes/hours Lambda Deployments • Deploy in milliseconds • Live for seconds
22. What Happened? Rate of change increased Cost and size and risk of change reduced
23. Microservices
24. A Microservice Definition Loosely coupled service oriented architecture with bounded contexts
25. A Microservice Definition Loosely coupled service oriented architecture with bounded contexts If every service has to be updated at the same time it’s not loosely coupled
26. A Microservice Definition Loosely coupled service oriented architecture with bounded contexts If every service has to be updated at the same time it’s not loosely coupled If you have to know too much about surrounding services you don’t have a bounded context. See the Domain Driven Design book by Eric Evans.
27. Microservice Architectures Configuration Tooling Discovery Routing Observability Development: Languages and Container Operational: Orchestration and Deployment Infrastructure Datastores Policy: Architectural and Security Compliance
28. Next Generation Applications Fill in the gaps, rapidly evolving ecosystem choices Archaius LaunchDarkly Habitat Configuration Lambda Docker Spinnaker Tooling Etcd Eureka Consul Discovery Compose Linkerd Weave Routing Zipkin Prometheus Hystrix Observability Development: components interfaces languages e.g. Docker Hub, Artifactory, Datawire Quark, Go, Rust Operational: Mesos, Kubernetes, Swarm, Nomad for private clouds. ECS, Mesos, GKS for public Datastores: Orchestrated, Distributed Ephemeral e.g. Cassandra, or DBaaS e.g. DynamoDB Policy: Security compliance e.g. Docker Content Trust. Architecture compliance e.g. Cloud Foundry
29. What could go wrong?
30. @adrianco Timeouts and Retries Edge Service Good Service Good Service Bad config: Every service defaults to 2 second timeout, two retries
31. @adrianco Timeouts and Retries Edge Service Good Service Good Service Bad config: Every service defaults to 2 second timeout, two retries
32. @adrianco Timeouts and Retries Edge Service Good Service Good Service Bad config: Every service defaults to 2 second timeout, two retries Edge Service not responding Overloaded service not responding Failed Service If anything breaks, everything upstream stops responding
33. @adrianco Timeouts and Retries Edge Service Good Service Good Service Bad config: Every service defaults to 2 second timeout, two retries Edge Service not responding Overloaded service not responding Failed Service If anything breaks, everything upstream stops responding Retries add unproductive work
34. @adrianco Timeouts and Retries Edge Service Good Service Budgeted timeout, one retry Failed Service
35. @adrianco Timeouts and Retries Edge Service Good Service Budgeted timeout, one retry Failed Service 3s 1s 1s Fast fail response after 2s Upstream timeout must always be longer than total downstream timeout * retries delay No unproductive work while fast failing
36. @adrianco Timeouts and Retries Edge Service Good Service Budgeted timeout, failover retry Failed Service For replicated services with multiple instances never retry against a failed instance No extra retries or unproductive work Good Service
37. @adrianco Timeouts and Retries Edge Service Good Service Budgeted timeout, failover retry Failed Service 3s 1s For replicated services with multiple instances never retry against a failed instance No extra retries or unproductive work Good Service Successful response delayed 1s
38. Cloud Native Monitoring and Microservices
39. Interesting architectures have a lot of microservices! Flow visualization is a big challenge. See http://www.slideshare.net/LappleApple/gilt-from-monolith-ruby-app-to-micro-service-scala-service-architecture
40. Simulated Microservices Model and visualize microservices Simulate interesting architectures Generate large scale configurations Eventually stress test real tools Code: github.com/adrianco/spigo Simulate Protocol Interactions in Go Visualize with D3 See for yourself: http://simianviz.surge.sh Follow @simianviz for updates ELB Load Balancer Zuul API Proxy Karyon Business Logic Staash Data Access Layer Priam Cassandra Datastore Three Availability Zones Denominator DNS Endpoint
41. Definition of an architecture { "arch": "lamp", "description":"Simple LAMP stack", "version": "arch-0.0", "victim": "webserver", "services": [ { "name": "rds-mysql", "package": "store", "count": 2, "regions": 1, "dependencies": [] }, { "name": "memcache", "package": "store", "count": 1, "regions": 1, "dependencies": [] }, { "name": "webserver", "package": "monolith", "count": 18, "regions": 1, "dependencies": ["memcache", "rds-mysql"] }, { "name": "webserver-elb", "package": "elb", "count": 0, "regions": 1, "dependencies": ["webserver"] }, { "name": "www", "package": "denominator", "count": 0, "regions": 0, "dependencies": ["webserver-elb"] } ] } Header includes chaos monkey victim New tier name Tier package 0 = non Regional Node count List of tier dependencies See for yourself: http://simianviz.surge.sh/lamp
42. Running Spigo $ ./spigo -a lamp -j -d 2 2016/01/26 23:04:05 Loading architecture from json_arch/lamp_arch.json 2016/01/26 23:04:05 lamp.edda: starting 2016/01/26 23:04:05 Architecture: lamp Simple LAMP stack 2016/01/26 23:04:05 architecture: scaling to 100% 2016/01/26 23:04:05 lamp.us-east-1.zoneB.eureka01....eureka.eureka: starting 2016/01/26 23:04:05 lamp.us-east-1.zoneA.eureka00....eureka.eureka: starting 2016/01/26 23:04:05 lamp.us-east-1.zoneC.eureka02....eureka.eureka: starting 2016/01/26 23:04:05 Starting: {rds-mysql store 1 2 []} 2016/01/26 23:04:05 Starting: {memcache store 1 1 []} 2016/01/26 23:04:05 Starting: {webserver monolith 1 18 [memcache rds-mysql]} 2016/01/26 23:04:05 Starting: {webserver-elb elb 1 0 [webserver]} 2016/01/26 23:04:05 Starting: {www denominator 0 0 [webserver-elb]} 2016/01/26 23:04:05 lamp.*.*.www00....www.denominator activity rate 10ms 2016/01/26 23:04:06 chaosmonkey delete: lamp.us-east-1.zoneC.webserver02....webserver.monolith 2016/01/26 23:04:07 asgard: Shutdown 2016/01/26 23:04:07 lamp.us-east-1.zoneB.eureka01....eureka.eureka: closing 2016/01/26 23:04:07 lamp.us-east-1.zoneA.eureka00....eureka.eureka: closing 2016/01/26 23:04:07 lamp.us-east-1.zoneC.eureka02....eureka.eureka: closing 2016/01/26 23:04:07 spigo: complete 2016/01/26 23:04:07 lamp.edda: closing -a architecture lamp -j graph json/lamp.json -d run for 2 seconds
43. Open Zipkin A common format for trace annotations A Java tool for visualizing traces Standardization effort to fold in other formats Driven by Adrian Cole (currently at Pivotal) Extended to load Spigo generated trace files
44. Trace for one Spigo Flow
45. Migrating to Microservices See for yourself: http://simianviz.surge.sh/migration Endpoint ELB PHP MySQL MySQL Next step Controls node placement distance Select models
46. Migrating to Microservices See for yourself: http://simianviz.surge.sh/migration Step 1 - Add Memcache Step 2 - Add Web Proxy Service
47. Migrating to Microservices See for yourself: http://simianviz.surge.sh/migration Step 3 - Add Data Access Layer Step 4 - Add Microservices Data Access node.js memcache per zone
48. Migrating to Microservices See for yourself: http://simianviz.surge.sh/migration Step 5 - Add Cassandra Step 6 - Remove MySQL 12 node cross zone Cassandra cluster MySQL
49. Migrating to Microservices See for yourself: http://simianviz.surge.sh/migration Step 7 - Add Second Region Step 8 - Connect Cassandra Regions Endpoint with location routed DNS
50. Migrating to Microservices See for yourself: http://simianviz.surge.sh/migration Step 9 - Add Third Region Endpoint with location routed DNS
51. @adrianco Simple Architecture Principles Symmetry Invariants Stable assertions No special cases
52. What’s Next?
53. Serverless
54. Serverless Architectures AWS Lambda getting some early wins Google Cloud Functions, Azure Functions alpha launched IBM OpenWhisk - open sourced Startup activity: iron.io , serverless.com, apex.run toolkit
55. @adrianco Monitorless Architecture API Gateway Kinesis S3 DynamoDB
56. @adrianco Monitorless Architecture API Gateway Kinesis S3 DynamoDB
57. @adrianco Monitorless Architecture API Gateway Kinesis S3 DynamoDB
58. AWS Lambda Reference Arch http://www.allthingsdistributed.com/2016/05/aws-lambda-serverless-reference-architectures.html
59. Serverless Programming Model Event driven functions Role based permissions Whitelisted API based security Good for simple single threaded code
60. Serverless Cost Efficiencies 100% useful work, no agents, overheads 100% utilization, no charge between requests No need to size capacity for peak traffic Anecdotal costs ~1% of conventional system Ideal for low traffic, Corp IT, spiky workloads
61. Serverless Work in Progress Tooling for ease of use Multi-region HA/DR patterns Debugging and testing frameworks Monitoring, end to end tracing
62. DIY Serverless Operating Challenges Startup latency Execution overhead Charging model Capacity planning
63. Learn More…
64. @adrianco “We see the world as increasingly more complex and chaotic because we use inadequate concepts to explain it. When we understand something, we no longer see it as chaotic or complex.” Jamshid Gharajedaghi - 2011 Systems Thinking: Managing Chaos and Complexity: A Platform for Designing Business Architecture
65. Q&A Adrian Cockcroft @adrianco http://slideshare.com/adriancockcroft Technology Fellow - Battery Ventures See www.battery.com for a list of portfolio investments
66. Security Visit http://www.battery.com/our-companies/ for a full list of all portfolio companies in which all Battery Funds have invested. Palo Alto Networks Enterprise IT Operations & Management Big Data Compute Networking Storage