Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Configuration Management 101

Sponsored · Your Podcast. Everywhere. Effortlessly. Share. Educate. Inspire. Entertain. You do you. We'll handle the rest. →
Avatar for someara someara
February 04, 2014
Programming
3
780

Configuration Management 101

Given at FOSDEM 2014

Avatar for someara

someara

February 04, 2014
Tweet

More Decks by someara

See All by someara
Docker Docker Docker Chef
Avatar for someara someara
0
490
Hand Crafted Artisinal Chef Resources
Avatar for someara someara
0
600
Configuration Management Camp 2015
Avatar for someara someara
1
560
Cookbook Reusability
Avatar for someara someara
0
800
TDI with ChefDK 0.0.1
Avatar for someara someara
2
870
Configuration Management 101 @ Scale12x
Avatar for someara someara
2
890
Introduction to Chef - Scale 10x
Avatar for someara someara
2
460
Introduction to Chef - NYLUG Jan 2012
Avatar for someara someara
2
300
Introduction to Chef - LISA11
Avatar for someara someara
10
6.2k

Other Decks in Programming

See All in Programming
new(1.26) ← これすき / kamakura.go #8
Avatar for utagawa kiki utgwkk
0
1.9k
AI時代のソフトウェア開発でも「人が仕様を書く」から始めよう-医療IT現場での実践とこれから
Avatar for kouki.miura koukimiura
0
140
守る「だけ」の優しいEMを抜けて、 事業とチームを両方見る視点を身につけた話
Avatar for Mitsui maroon8021
3
410
RubyとGoでゼロから作る証券システム： 高信頼性が求められるシステムのコードの外側にある設計と運用のリアル
Avatar for free_world21 free_world21
0
230
社内規程RAGの精度を73.3% → 100%に改善した話
Avatar for oharu121 oharu121
13
7.8k
New in Go 1.26 Implementing go fix in product development
Avatar for sunecosuri sunecosuri
0
380
AWS Infrastructure as Code の新機能 2025 総まとめ 〜SA 4人による怒涛のデモ祭り〜
Avatar for Kenji Kono konokenj
10
3.3k
TROCCOで実現するkintone+BigQueryによるオペレーション改善
Avatar for shota sato ssxota
0
150
株式会社 Sun terras カンパニーデック
Avatar for Sun terras.Inc sunterras
0
2k
nilとは何か 〜interfaceの構造とnil!=nilから理解する〜
Avatar for kuro kuro_kurorrr
3
1.8k
「やめとこ」がなくなった — 1月にZennを始めて22本書いた AI共創開発のリアル
Avatar for atani atani14
0
360
米国のサイバーセキュリティタイムラインと見る Goの暗号パッケージの進化
Avatar for tom twinkle tomtwinkle
2
510

Featured

See All Featured
Beyond borders and beyond the search box: How to win the global "messy middle" with AI-driven SEO
Avatar for David Carrasco davidcarrasco
3
67
Building the Perfect Custom Keyboard
Avatar for Naoto Takai takai
2
710
Kristin Tynski - Automating Marketing Tasks With AI
Avatar for Tech SEO Connect techseoconnect
PRO
0
190
Accessibility Awareness
Avatar for Sarah Abderemane sabderemane
0
74
StorybookのUI Testing Handbookを読んだ
Avatar for Shingo Yamazaki zakiyama
31
6.6k
The Psychology of Web Performance [Beyond Tellerrand 2023]
Avatar for Tammy Everts tammyeverts
49
3.3k
Being A Developer After 40
Avatar for Adrian Kosmaczewski akosma
91
590k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
Avatar for Sam Stephenson sstephenson
162
16k
YesSQL, Process and Tooling at Scale
Avatar for Rocio Delgado rocio
174
15k
Leveraging Curiosity to Care for An Aging Population
Avatar for Cassini Nazir cassininazir
1
190
[SF Ruby Conf 2025] Rails X
Avatar for Vladimir Dementyev palkan
2
820
Fireside Chat
Avatar for paigeccino paigeccino
42
3.8k

Transcript

  1. Configuration Management 101! FOSDEM 2014

  2. Sean OMeara! [email protected]! @someara

  3. Sean OMeara! [email protected]! @someara

  4. Sean OMeara! [email protected]! @someara

  5. whoami

  6. Part 1

  7. The Dawn of Configuration Management

  8. • What is configuration management?! • Strategies and techniques for

    managing configuration and its complexity! • The art of change management

  9. Manual Configuration

  10. • Intuitive! • How we all start out! • Log

    into machine, manipulate with fingers! • Make with the clicky clicky! • Long tradition

  11. • Somehow feels the “safest"! • First instinct in emergencies!

    • This is an illusion! • Do not do this

  12. • Labor intensive! • Error prone! • Difficult to reproduce!

    • Obviously unsustainable

  13. Scripting

  14. • setup.sh! • setup.pl! • setup.py! • setup.rb

  15. • doit.sh! • doit.pl! • doit.py! • doit.rb

  16. • Ad-hoc in nature! • Loss of history! • Lacks

    testing methodology! • A step in the right direction

  17. File Distribution

  18. • NFS! • SMB! • AFS! • SSHFS! • GlusterFS

  19. • uucp! • rcp! • ftp! • http! • scp

  20. •Distributed systems! •Shares often managed manually or with scripts! •Package

    repositories! •Pull is better than push! •Scp on a cron *

  21. Execution Management

  22. • Image management! • Snapshots and cloning! • Containers

  23. • SSH on a for loop! • Func! • Commands

    on message queues! • ISConf

  24. • Loss of history! • Image sprawl! • Easy to

    order change across nodes

  25. Convergent Operators! (promises)

  26. None

  27. Tools

  28. • CFEngine! • Bcfg2! • Puppet! • Chef! • Salt!

    • Ansible
  29. None

  30. Part 2

  31. Policy http://www.flickr.com/photos/sfllaw/222795669/

  32. • /etc/passwd should be mode 0644! • /etc/shadow should be

    mode 0600

  33. • user ‘kermit’ should exist! • user ‘fonzi’ should exist!

    • group ‘muppets’ should exist! • group ‘muppets’ should contain kermit and fonzi

  34. • package ‘ntpd’ should be installed! • ntpd should sync

    with our AD service! • service ‘ntpd’ should be running

  35. • package ‘httpd’ should be installed! • httpd should be

    expose /mnt/software/java! • service ‘httpd’ should be running

  36. • The Java JDK, version 7u45, found on an internally

    hosted web server, should be installed into /usr/local/jdk-7u45/

  37. Polices are declarations about the state of things in a

    system

  38. Polices are applied repeatedly and repair the system when needed

  39. Policies often change

  40. • package ‘widget-factory’ should be installed at version 1.2.3

  41. • package ‘widget-factory’ should be installed at version 1.3.0

  42. http://www.flickr.com/photos/jakepjohnson/4937767595 Repeatability

  43. Repeatable -> Idempotent -> Convergent

  44. • Scripts are not generally repeatable

  45. None
  46. None

  47. • But they can be!

  48. None

  49. ! Idempotent operations can be applied infinite times and will

    yield the same result every time
  50. None

  51. Idempotent http://www.flickr.com/photos/ian_munroe/4758240536/

  52. http://www.flickr.com/photos/ian_munroe/4758240536/ Idempotent NOT GOOD ENOUGH

  53. ! Convergent operations test state and repair if needed

  54. None
  55. None

  56. ! A control loop keeps the system stable and allows

    for change when policy is updated

  57. Autonomous agent Policy: The box should be closed

  58. Convergence

  59. None
  60. None
  61. None

  62. Converging with Bash

  63. git clone [email protected]:someara/ cbash.git

  64. None
  65. None
  66. None
  67. None
  68. None
  69. None
  70. None
  71. None
  72. None
  73. None
  74. None
  75. None
  76. None

  77. Convergence and Iteration

  78. None
  79. None
  80. None
  81. None
  82. None
  83. None

  84. Does order matter?

  85. YES

  86. None

  87. Promises http://www.flickr.com/photos/nazzen9009/6809694353/

  88. • Agents are autonomous! • A promise is a signal

    or message perceived by an observer.! • Promises may or may not be kept.! • Agents can observe other agents! • Agents only have local information *! • Inner workings of agents are assumed to be unknown http://markburgess.org/BookOfPromises.pdf

  89. • Agents have intentions (possible behaviors)! • Agents can make

    assessments about other agents http://markburgess.org/BookOfPromises.pdf

  90. • Configuration Management tools embody tenants of Promise Theory intentionally

    or not

  91. Domain Specific Languages

  92. ! DSLs restrict machine instructions to convergent operations

  93. ! DSLs manage ordering

  94. None

  95. type subject intentions

  96. None

  97. type subject intentions

  98. signal

  99. None

  100. type subject intention

  101. observation

  102. None

  103. type subject intentions

  104. None

  105. type intention subject

  106. signal

  107. Intermission

  108. None

  109. Part 3

  110. Composition

  111. None

  112. Recipes

  113. resource one resource two resource three

  114. { testable intent

  115. recipe[http::server]

  116. recipe[http::server]

  117. recipes supporting files

  118. Types

  119. None

  120. interface implementation

  121. None

  122. intentions parameters

  123. None

  124. new scope intention implementation

  125. new scope intention implementation

  126. Artifacts

  127. metadata

  128. None

  129. metadata

  130. None

  131. http v0.1.0 chef-server api yum v3.0.0

  132. Delivery

  133. • nodes request their own initial run_list

  134. recipe[httpd::server] chef-server api run_list: http v0.1.0

  135. recipe[httpd::server] chef-server api run_list: http v0.1.0 recipe[openssh::server] openssh v3.2.1

  136. recipe[ntp::client] chef-server api run_list: http v0.1.0 recipe[openssh::server] openssh v3.2.1 recipe[httpd::server]

    ntp v1.0.0

  137. • Push vs Pull! • Networking considerations! • Machines down

    for maintenance! • Machines that don’t exist yet

  138. Dependencies

  139. None
  140. None
  141. None
  142. None
  143. None
  144. None

  145. recipe[widgetfactory] chef-server api run_list: http v0.1.0 yum v3.0.0 widgetfactory v1.0.0

  146. Integration testing

  147. • Test that a set of agents has achieved their

    combined goal

  148. • lsof -i :80! • ps -ef | grep httpd!

    • curl localhost 2>&1 > /dev/null

  149. • Berkshelf! • Vagrant! • Kitchen.ci! • Bats! • Serverspec

  150. Environments

  151. • Environments constrain cookbook versions! • Environments can set data

  152. None
  153. None

  154. • Environments can be used to test branches! • Environments

    can be used to segregate machines! • Environments can be manipulated programatically
  155. None
  156. None

  157. http v0.1.0 chef-server api http v0.2.0 openssh v1.2.3 postgresql v3.2.1

  158. recipe[widgetfactory] run_list: http v0.1.0 yum v3.0.0 widgetfactory v1.0.0 chef_environment: production

  159. recipe[widgetfactory] run_list: http v0.2.0 yum v3.0.0 widgetfactory v1.0.0 chef_environment: staging

  160. Part 4

  161. Clusters http://www.flickr.com/photos/youraccount/5938852370/

  162. Typical Cluster

  163. loadbalancer application db-slave db-master

  164. Production httpd 0.1.0

  165. Production Staging httpd 0.1.0 httpd 0.1.0

  166. Production Staging UUID httpd 0.1.0 httpd 0.1.0 httpd 0.2.0

  167. Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0

  168. Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0

  169. Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0

  170. Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0

  171. Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0

  172. Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0

  173. Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0

  174. Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0

  175. Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0

  176. Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0

  177. Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0

  178. Production Staging httpd 0.2.0 httpd 0.2.0

  179. Production httpd 0.2.0

  180. An Ordering Problem

  181. loadbalancer application

  182. • Take a machine out of the pool! • Drain

    the connections! • Modify configuration! • Insert it back into the pool

  183. loadbalancer application

  184. loadbalancer application

  185. loadbalancer application

  186. loadbalancer application

  187. loadbalancer application

  188. loadbalancer application

  189. loadbalancer application

  190. loadbalancer application

  191. loadbalancer application

  192. loadbalancer application

  193. Orchestration

  194. • Conductor showing signals to autonomous agents (creative policy manipulation)!

    • External actor controlling sequencing (execution management)! • Application level sequencing (vector clocks, etc)

  195. • Infrastructures are snowflakes! • Solutions are unique to applications

    by nature! • Configuration Management 201

  196. • There is no separation between ‘infrastructure’ and ‘application’! •

    Distributed systems are hard! • Specialists need to work together

  197. Devops

  198. • Study Promise Theory! • Study distributed systems! • Develop

    high quality primitives! • Be excellent to each other

  199. Fin