Smart services & smart clients: How microservices change the way you build and deploy code.

At Runscope we've standardized the idea of small independent "smart" services that can be quickly built, deployed, and scaled. This talk examines lessons learned from writing these services as well as patterns found in other platforms. We present a detailed look at the code that allow us to build dozens of services, serving billions of requests, while deploying to our cluster over 20 times a day.

Transcript

1. Smart Services & Smart Clients How microservices change the way you build and deploy code Frank Stratton
 Co-founder/CTO @FrankRStratton Runscope
2. Solve API Problems Fast Monitor, log, measure & share your API usage.
5. 60 services 8 engineers
6. Why microservices?
7. 0 000,000 000,000 000,000 000,000 000,000 May 2013 Sep Jan May Sep Jan 2015 Scaling the infrastructure Scaling the team
8. Microservices are highly independent
9. independent codebases independent deploys independent teams
10. There are a lot of moving parts…
11. If you don't invest in INFRASTRUCTURE don't invest in MICROSERVICES
12. Microservices = SOA + DevOps
13. Dashboard Identity get_teams()
14. Dashboard Identity get_teams() host001.runscope.com host002.runscope.com
15. import requests def get_teams(): url = “http://host002.runscope.com/teams” resp = requests.get(url) return resp.json() Dashboard Identity get_teams()
16. url = “http://host002.runscope.com/teams” url = “http://host003.runscope.com/teams” Dashboard Identity get_teams() X Identity
17. if ENVIRONMENT == “local”: url = “http://localhost:5000/teams” elif ENVIRONMENT == “test”: url = “http://test001.runscope.com/teams” elif ENVIRONMENT == “prod”: url = “http://host003.runscope.com/teams” Dashboard Identity get_teams()
18. How do you find a service?
19. How do you find a service? Smart Client
20. Smart Client import requests def get_teams(): url = “http://host001.runscope.com/teams” resp = requests.get(url) return resp.json()
21. Smart Client import smart_client def get_teams(): url = “http://host001.runscope.com/teams” resp = smart_client.get(url) return resp.json()
22. Smart Client import smart_client def get_teams(): url = “service://identity/teams” resp = smart_client.get(url) return resp.json()
23. Smart Client Service discovery service://identity/... Retry failed GET/PUT/DELETE (idempotent) requests Run HTTP requests asynchronously
24. Smart Client Atlas
25. Smart Client ZooKeeper Atlas DynamoDB /hosts/identity/host001.runscope.com = 5000 /hosts/identity/host001.runscope.com/enabled = True Register service metadata in Zookeeper & DynamoDB
26. Smart Client Atlas Sidecar Runs HAProxy on each host. Reads cluster state from atlas. Watches for changes to services and updates the local configuration. host001 host002 host003
27. Smart Client
28. Smart Client Service A Service A
29. Smart Client Service B Service B
30. Smart Client smart_client.get(“service://identity/teams”) requests.get(“http://localhost:4000/teams”, headers={“Runscope-Service”: “identity”})
31. haproxy.cfg frontend http-in bind *:4000 mode http acl host_api hdr(runscope-service) -i api acl host_billing hdr(runscope-service) -i billing acl host_mission-control hdr(runscope-service) -i mission-control acl host_elasticsearch hdr(runscope-service) -i elasticsearch use_backend api-backend if host_api use_backend billing-backend if host_billing use_backend mission-control-backend if host_mission-control use_backend elasticsearch-backend if host_elasticsearch
32. Smart Client smart_client.get(“service://identity/teams”) requests.get(“http://localhost:4000/teams”, headers={“Runscope-Service”: “identity”})
33. How do you find a service? Smart Client
34. How do you make building services easy?
35. How do you make building services easy? Smart Service
36. Smart Service
37. Smart Service Runscope/healthcheck
38. Smart Service bugsnag runscope-daemon alchemist smart-config
39. Smart Service Configuration Driven defaults.py REDIS_SERVICE = “redis-cluster-1” REDIS_DB = 2 BUGSNAG_API_KEY = “xyz123”
40. Smart Service Configuration Driven config = SmartConfig('example-service', defaults=example_service.defaults) app = SmartService('example-service', config=config)
41. Smart Service Various additions to the flask app object from flask import current_app current_app.realm current_app.redis current_app.database current_app.api
42. Smart Service Common Logging Configuration /var/log/runscope/<service>.access.log /var/log/runscope/<service>.error.log current_app.logger.warn(“Uh oh!”) bugsnag.notify(e)
43. $> generate-service my-awesome-service Creating a service skeleton with this configuration: Service name: my-awesome-service Destination directory: ~/runscope/my-awesome-service Python package name: my_awesome_service HTTP port: 5004 1. Generating files in ~/runscope/my-awesome-service 2. Initializing Git repository in ~/runscope/my-awesome-service 3. Creating virtualenv my-awesome-service 4. Installing Python prerequisites (requirements.txt and setup.py) All done! my-awesome-service source code can be found in ~/runscope/my-awesome-service Smart Service Auto Generate Service Skeletons
44. Smart Service Reduce Cognitive Overhead
45. How do you make building services easy? Smart Service
46. What happens when you have so many services?
47. What happens when you have so many services? A Lot :)
48. Automate everything
49. Language Agnostic
50. Language Agnostic HTTP is our common ‘language’
51. No shared databases
52. Dashboard Traffic Gateway API Postgresql
53. Dashboard Traffic Gateway API Postgresql X
54. Dashboard Traffic Gateway API Identity Postgresql Flask Redis A service owns it’s own datastore(s).
55. Dashboard Traffic Gateway API And exposes an API for that data. Identity GET /teams/<id> PUT /teams/<id> GET /buckets ...
56. Invest in deployment automation.
57. Invest in deployment automation. 1-Click Deploys
59. 0 10 20 30 40 50 Jun 2014 Jul Aug Sep Oct Nov Dec Jan Feb Mar 2015 Deploys per day
60. Test & Monitor your APIs
61. Test Lifecycle Local Dev {{identity_host}} = http://localhost Local Agents
62. Test Lifecycle Test Realm Local Dev {{identity_host}} = http://test001.runscope.com Local Agents Trigger URLs
63. Test Lifecycle Test Realm Production Local Dev {{identity_host}} = http://host002.runscope.com Local Agents Trigger URLs Schedules
64. 0 10 20 30 40 50 Jun 2014 Jul Aug Sep Oct Nov Dec Jan Feb Mar 2015 Deploys per day
65. Smart Services & Smart Clients How microservices change the way you build and deploy code Frank Stratton
 @FrankRStratton Runscope
66. Thanks! Frank Stratton
 @FrankRStratton Runscope
67. Thanks! p.s. - we’re hiring :) Frank Stratton
 @FrankRStratton Runscope https://www.runscope.com/jobs