Configuration Management 101

Given at FOSDEM 2014

Transcript

1. Configuration Management 101! FOSDEM 2014
2. Sean OMeara! someara@opscode.com! @someara
3. Sean OMeara! someara@opscode.com! @someara
4. Sean OMeara! someara@getchef.com! @someara
5. whoami
6. Part 1
7. The Dawn of Configuration Management
8. • What is configuration management?! • Strategies and techniques for managing configuration and its complexity! • The art of change management
9. Manual Configuration
10. • Intuitive! • How we all start out! • Log into machine, manipulate with fingers! • Make with the clicky clicky! • Long tradition
11. • Somehow feels the “safest"! • First instinct in emergencies! • This is an illusion! • Do not do this
12. • Labor intensive! • Error prone! • Difficult to reproduce! • Obviously unsustainable
13. Scripting
14. • setup.sh! • setup.pl! • setup.py! • setup.rb
15. • doit.sh! • doit.pl! • doit.py! • doit.rb
16. • Ad-hoc in nature! • Loss of history! • Lacks testing methodology! • A step in the right direction
17. File Distribution
18. • NFS! • SMB! • AFS! • SSHFS! • GlusterFS
19. • uucp! • rcp! • ftp! • http! • scp
20. •Distributed systems! •Shares often managed manually or with scripts! •Package repositories! •Pull is better than push! •Scp on a cron *
21. Execution Management
22. • Image management! • Snapshots and cloning! • Containers
23. • SSH on a for loop! • Func! • Commands on message queues! • ISConf
24. • Loss of history! • Image sprawl! • Easy to order change across nodes
25. Convergent Operators! (promises)
27. Tools
28. • CFEngine! • Bcfg2! • Puppet! • Chef! • Salt! • Ansible
30. Part 2
31. Policy http://www.flickr.com/photos/sfllaw/222795669/
32. • /etc/passwd should be mode 0644! • /etc/shadow should be mode 0600
33. • user ‘kermit’ should exist! • user ‘fonzi’ should exist! • group ‘muppets’ should exist! • group ‘muppets’ should contain kermit and fonzi
34. • package ‘ntpd’ should be installed! • ntpd should sync with our AD service! • service ‘ntpd’ should be running
35. • package ‘httpd’ should be installed! • httpd should be expose /mnt/software/java! • service ‘httpd’ should be running
36. • The Java JDK, version 7u45, found on an internally hosted web server, should be installed into /usr/local/jdk-7u45/
37. Polices are declarations about the state of things in a system
38. Polices are applied repeatedly and repair the system when needed
39. Policies often change
40. • package ‘widget-factory’ should be installed at version 1.2.3
41. • package ‘widget-factory’ should be installed at version 1.3.0
42. http://www.flickr.com/photos/jakepjohnson/4937767595 Repeatability
43. Repeatable -> Idempotent -> Convergent
44. • Scripts are not generally repeatable
47. • But they can be!
49. ! Idempotent operations can be applied infinite times and will yield the same result every time
51. Idempotent http://www.flickr.com/photos/ian_munroe/4758240536/
52. http://www.flickr.com/photos/ian_munroe/4758240536/ Idempotent NOT GOOD ENOUGH
53. ! Convergent operations test state and repair if needed
56. ! A control loop keeps the system stable and allows for change when policy is updated
57. Autonomous agent Policy: The box should be closed
58. Convergence
62. Converging with Bash
63. git clone git@github.com:someara/ cbash.git
77. Convergence and Iteration
84. Does order matter?
85. YES
87. Promises http://www.flickr.com/photos/nazzen9009/6809694353/
88. • Agents are autonomous! • A promise is a signal or message perceived by an observer.! • Promises may or may not be kept.! • Agents can observe other agents! • Agents only have local information *! • Inner workings of agents are assumed to be unknown http://markburgess.org/BookOfPromises.pdf
89. • Agents have intentions (possible behaviors)! • Agents can make assessments about other agents http://markburgess.org/BookOfPromises.pdf
90. • Configuration Management tools embody tenants of Promise Theory intentionally or not
91. Domain Specific Languages
92. ! DSLs restrict machine instructions to convergent operations
93. ! DSLs manage ordering
95. type subject intentions
97. type subject intentions
98. signal
100. type subject intention
101. observation
103. type subject intentions
105. type intention subject
106. signal
107. Intermission
109. Part 3
110. Composition
112. Recipes
113. resource one resource two resource three
114. { testable intent
115. recipe[http::server]
116. recipe[http::server]
117. recipes supporting files
118. Types
120. interface implementation
122. intentions parameters
124. new scope intention implementation
125. new scope intention implementation
126. Artifacts
127. metadata
129. metadata
131. http v0.1.0 chef-server api yum v3.0.0
132. Delivery
133. • nodes request their own initial run_list
134. recipe[httpd::server] chef-server api run_list: http v0.1.0
135. recipe[httpd::server] chef-server api run_list: http v0.1.0 recipe[openssh::server] openssh v3.2.1
136. recipe[ntp::client] chef-server api run_list: http v0.1.0 recipe[openssh::server] openssh v3.2.1 recipe[httpd::server] ntp v1.0.0
137. • Push vs Pull! • Networking considerations! • Machines down for maintenance! • Machines that don’t exist yet
138. Dependencies
145. recipe[widgetfactory] chef-server api run_list: http v0.1.0 yum v3.0.0 widgetfactory v1.0.0
146. Integration testing
147. • Test that a set of agents has achieved their combined goal
148. • lsof -i :80! • ps -ef | grep httpd! • curl localhost 2>&1 > /dev/null
149. • Berkshelf! • Vagrant! • Kitchen.ci! • Bats! • Serverspec
150. Environments
151. • Environments constrain cookbook versions! • Environments can set data
154. • Environments can be used to test branches! • Environments can be used to segregate machines! • Environments can be manipulated programatically
157. http v0.1.0 chef-server api http v0.2.0 openssh v1.2.3 postgresql v3.2.1
158. recipe[widgetfactory] run_list: http v0.1.0 yum v3.0.0 widgetfactory v1.0.0 chef_environment: production
159. recipe[widgetfactory] run_list: http v0.2.0 yum v3.0.0 widgetfactory v1.0.0 chef_environment: staging
160. Part 4
161. Clusters http://www.flickr.com/photos/youraccount/5938852370/
162. Typical Cluster
163. loadbalancer application db-slave db-master
164. Production httpd 0.1.0
165. Production Staging httpd 0.1.0 httpd 0.1.0
166. Production Staging UUID httpd 0.1.0 httpd 0.1.0 httpd 0.2.0
167. Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0
168. Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0
169. Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0
170. Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0
171. Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0
172. Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0
173. Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0
174. Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0
175. Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0
176. Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0
177. Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0
178. Production Staging httpd 0.2.0 httpd 0.2.0
179. Production httpd 0.2.0
180. An Ordering Problem
181. loadbalancer application
182. • Take a machine out of the pool! • Drain the connections! • Modify configuration! • Insert it back into the pool
183. loadbalancer application
184. loadbalancer application
185. loadbalancer application
186. loadbalancer application
187. loadbalancer application
188. loadbalancer application
189. loadbalancer application
190. loadbalancer application
191. loadbalancer application
192. loadbalancer application
193. Orchestration
194. • Conductor showing signals to autonomous agents (creative policy manipulation)! • External actor controlling sequencing (execution management)! • Application level sequencing (vector clocks, etc)
195. • Infrastructures are snowflakes! • Solutions are unique to applications by nature! • Configuration Management 201
196. • There is no separation between ‘infrastructure’ and ‘application’! • Distributed systems are hard! • Specialists need to work together
197. Devops
198. • Study Promise Theory! • Study distributed systems! • Develop high quality primitives! • Be excellent to each other
199. Fin