CoreOS: A Tutorial on Hyperscale Infrastructure

Transcript

1. Brandon Philips @brandonphilips | brandon.philips@coreos.com Slides and Demos: goo.gl/AgPy2E A Tutorial on Hyperscale Infrastructure Free Stickers @ Podium!
2. Expectations Tutorial
3. Quick Level-Set Follow Along
4. Secure the Internet MISSION
5. Separate Apps from OS STRATEGY
6. Make Servers Consistent STRATEGY
7. Make Servers Easy to Upgrade STRATEGY
8. Simplify Application Management STRATEGY
9. On Any Infrastructure STRATEGY
16. 3 Application packaging Clustering Linux at scale
17. #GIFEE Borg/Omega Linux Chubby
18. #GIFEE Borg/Omega Linux Chubby
19. #GIFEE Borg/Omega Linux Chubby
20. Avoid single points of failure Regularly update software Consistent environment Why build #GIFEE?
21. Why build #GIFEE? Regularly update software
22. Application Packaging 1
23. Containers Run apps consistently
24. Abstract away app from the OS OS App
27. Base software managed by CoreOS kernel systemd OpenSSH
28. Protect apps from each other Isolated network namespace Isolated file system namespace Mixed versions of dependencies eg. python 3.4 & python 2.7
29. $ sudo rkt run coreos.com/etcd:v2.0.0 $ sudo rkt run coreos.com/etcd:v2.0.0 \ --cpu=750m --memory=128M $ sudo rkt run --net=host coreos.com/etcd:v2.0.0 rkt run
30. Search container metadata Identify vulnerabilities Explain update actions Clair container security auditing
31. After scanning millions of containers we found that over 80% still had Heartbleed 80% Clair container security auditing
33. In-Progress Universal Container Format Packaged Downloaded Verified
34. Container Basics DEMO
35. Linux at Scale 2
36. CoreOS Linux Run containers any platform
37. Patches to the OS and kernel are hard Retest after updates No automation SECURITY Dependency breakage Uptime risk APPLICATION
39. Auto-updating browsers fixed security We got HTML5 at the same time
40. Atomic operating system updates
41. Atomic operating system updates
42. CoreOS Linux Run containers any platform
43. Toolbox Tutorial DEMO
44. Clustering 3
45. Patches to the OS and kernel are hard Retest after updates No automation SECURITY Dependency breakage Uptime risk APPLICATION
46. Patches to the OS and kernel are hard No automation SECURITY Uptime risk APPLICATION
47. Operations Paradise Easy scale out Painless app upgrades Tolerant of machine failure
48. App Req/sec: 6,000 App Healthy: True
49. App Req/sec: 6,000 App Healthy: True
50. App Req/sec: 7,000 App Healthy: True
51. App Req/sec: 8,000 App Healthy: True
52. App Req/sec: 7,000 App Healthy: True
53. App Req/sec: 6,000 App Healthy: True
54. App Req/sec: 8,000 App Healthy: True
55. App Req/sec: 7,000 App Healthy: True
56. App Req/sec: 8,000 App Healthy: True
57. App Req/sec: 8,000 App Healthy: True
58. App Req/sec: 8,000 App Healthy: True
59. API for easing scaling, painless upgrades, and failure tolerance
60. Simple cluster operations Secure and Simple API Friendly operational tools db
61. Fault-tolerant database to support complex set of needs Leader election Cluster-wide Semaphores Service discovery Dynamic configuration
62. Hard Computer Science Problem ?
63. Hard Computer Science Problem Chubby
64. Distributed reliable key-value store for the most critical data of a distributed system
65. Etcd Basics DEMO
73. No existing “cloud native” solutions Simple HTTP + JSON APIs Dynamic reconfiguration Why build etcd?
74. Simple key/value “Distributed etc” Keys are versioned Changes can be watch
75. Cluster-wide reboot lock - locksmith Service discovery - vulcand, skydns Cluster orchestration - k8s, cloud foundry
76. Industry Adoption 500+ projects on Github
77. Etcd Clustering DEMO
78. Simple cluster operations Secure and Simple API Friendly operational tools
79. Kubernetes Basics DEMO
80. Scheduler gets work to the servers
81. $ scp app host:/opt $ ssh host systemd-run /opt/app
82. $ scp app host:/opt $ ssh host systemd-run /opt/app
83. $ fab deploy:app
84. $ fab deploy:app
85. $ fab deploy:app
86. $ fab deploy:gpu-app
87. $ fab deploy:gpu-app
88. $ fab deploy:gpu-app
89. App Deployment DEMO
90. ReplicaSets drive the cluster to a desired state
91. pod env=prod app=web pod env=prod app=web pod env=prod app=web rs web-prod select(env=prod,app=web) count=1
92. pod env=prod app=web pod env=prod app=web pod env=prod app=web rs web-prod select(env=prod,app=web) count=1
93. pod env=prod app=web rs web-prod select(env=prod,app=web) count=1
94. App Failure DEMO
95. pod env=prod app=web rs web-prod select(env=prod,app=web) count=5
96. pod env=prod app=web pod env=prod app=web pod env=prod app=web pod env=prod app=web pod env=prod app=web rs web-prod select(env=prod,app=web) count=5
97. App Scaling DEMO
98. Services find scheduled apps (pods) and load balance them
99. pod env=dev app=web pod env=test app=web pod env=prod app=web
100. pod env=dev app=web pod env=test app=web pod env=prod app=web service test.example.com select(env=dev,app=web) service beta.example.com select(env=test,app=web) OR select(env=prod,app=web) service example.com select(env=prod,app=web)
101. pod env=test app=web pod env=prod app=web pod env=prod app=web pod env=dev app=web pod env=test app=web pod env=prod app=web service test.example.com select(env=dev,app=web) service beta.example.com select(env=test,app=web) OR select(env=prod,app=web) service example.com select(env=prod,app=web)
102. pod app=foo,version=1 service foo.cluster.local select(app=foo)
103. pod app=foo,version=1 pod app=foo,version=2 service foo.cluster.local select(app=foo)
104. pod app=foo,version=1 pod app=foo,version=2 service foo.cluster.local select(app=foo)
105. Service Discovery DEMO
106. pod env=prod app=web pod env=prod app=web pod env=prod app=web pod env=prod app=web pod env=prod app=web rs web-prod select(env=prod,app=web) count=5
107. pod env=prod app=web pod env=prod app=web pod env=prod app=web pod env=p rod pod env=prod app=web rs web-prod-4950 select(app=web,hash=4950) count=5 pod env=prod app=web pod env=prod app=web pod env=prod app=web pod env=p rod pod env=prod app=web rs web-prod-1328 select(app=web,hash=1328) count=5 deployment web-prod select(app=web) count=5
108. App Upgrades DEMO
109. Upstream rktnetes Auth/OIDC Node self-signed TLS
110. Scaling 15x scheduler performance 30k pods on 1k nodes SIG-scale
111. 3 Application packaging Clustering Linux at scale
112. Sounds good, but... Is anyone successful with all this in prod?
113. Publically traded options exchange
114. Containers on CoreOS are powering ISE's high-throughput, low-latency financial exchange Running in production Bare metal & AWS Billions of transactions a day 150 million req/sec
116. Secure the Internet MISSION
117. Separate Apps from OS STRATEGY
118. Make Servers Consistent STRATEGY
119. Make Servers Easy to Upgrade STRATEGY
120. Simplify Application Management STRATEGY
121. On Any Infrastructure STRATEGY
123. Guides & Tools coreos.com/kubernetes
125. Linux
127. Runs on any platform Consistency across clouds Trivial dev, test, & prod Faster time to market
128. Training San Francisco September 13 & 14 New York City September 27 & 28 San Francisco October 11 & 12 New York City October 25 & 26 Seattle November 10 & 11 https://coreos.com/training
129. Build, Store and Distribute your Containers quay.io
130. Thank you! Brandon Philips @brandonphilips | brandon@coreos.com | coreos.com We’re hiring in all departments! Email: careers@coreos.com Positions: coreos.com/ careers
131. Bonus: Hacking Setup DEMO