Upgrade to Pro — share decks privately, control downloads, hide ads and more …
Sign up for free

Configuration Management 101 @ Scale12x

Avatar for someara someara
February 28, 2014
Technology
940
2

Configuration Management 101 @ Scale12x

Avatar for someara

someara

February 28, 2014

More Decks by someara

See All by someara
Docker Docker Docker Chef
Avatar for someara someara
0
550
Hand Crafted Artisinal Chef Resources
Avatar for someara someara
0
630
Configuration Management Camp 2015
Avatar for someara someara
1
600
Cookbook Reusability
Avatar for someara someara
0
830
TDI with ChefDK 0.0.1
Avatar for someara someara
2
900
Configuration Management 101
Avatar for someara someara
3
830
Introduction to Chef - Scale 10x
Avatar for someara someara
2
490
Introduction to Chef - NYLUG Jan 2012
Avatar for someara someara
2
330
Introduction to Chef - LISA11
Avatar for someara someara
10
6.2k

Other Decks in Technology

See All in Technology
スキマバイトという雇用形態は存在しないので、雇用期間1日のアルバイト契約を事業者とワーカーが結ぶ手続きを支えるのがタイミーなんです
Avatar for しが あきとし(akitoshiga) akitoshiga
0
210
React改造してみた
Avatar for Mooncake Sugar tsukina_7mochi
0
120
今さら理解するウェブフォントver.2026
Avatar for takanorip takanorip
3
900
DevelopersIO 2026 知識の地図を描く ─ オントロジーとは何か？
Avatar for Satoru Ishikawa ishikawa_satoru
1
1.7k
Conditional PED-ANOVA: Hyperparameter Importance in Hierarchical & Dynamic Search Spaces
Avatar for Preferred Networks pfn
PRO
0
140
AIがコードを書く時代に、エンジニアは何で自分の価値を証明するのか / In an era where AI writes code how do engineers prove their value
Avatar for CARTA Engineering carta_engineering
3
1.1k
触りたくないデータに、どう触るか AI時代のRBACアクセスガバナンス
Avatar for 株式会社IVRy（社員登壇資料） ivry_presentationmaterials
0
280
Gemini Intelligence 入門 ― AppFunctions と Gemini Nano
Avatar for takuro abe takuroabe
0
290
コンテナ運用におけるネットワーク・コンテナコストの削減方法／How to Reduce Network and Container Costs in Container Operations
Avatar for モブエンジニア masakiokuda
2
300
20261017福岡devIO 「AIは"良いダッシュボード"を作れるか？
Avatar for 武田 jtakeda
0
190
Webの常識が通じないHTMLメールのダークモード
Avatar for teamLab teamlab
PRO
0
130
SAENM ~Jupyterで残すPoCの試行錯誤、未来の自分と仲間を助ける記録！~
Avatar for Manabu TERADA terapyon
0
120

Featured

See All Featured
Data-driven link building: lessons from a $708K investment (BrightonSEO talk)
Avatar for Szymon Słowik szymonslowik
1
1.4k
How to Build an AI Search Optimization Roadmap - Criteria and Steps to Take #SEOIRL
Avatar for Aleyda Solis aleyda
1
2.2k
From π to Pie charts
Avatar for Rasagy Sharma rasagy
1
400
Leading Effective Engineering Teams in the AI Era
Avatar for Addy Osmani addyosmani
9
2.7k
Context Engineering - Making Every Token Count
Avatar for Addy Osmani addyosmani
9
1.2k
Game over? The fight for quality and originality in the time of robots
Avatar for Wayne Barker wayneb77
1
310
Digital Projects Gone Horribly Wrong (And the UX Pros Who Still Save the Day) - Dean Schuster
Avatar for UX Y'all uxyall
1
3.4k
Typedesign – Prime Four
Avatar for Hannes Fritz hannesfritz
42
3.2k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
Avatar for Tammy Everts tammyeverts
56
3.5k
How STYLIGHT went responsive
Avatar for nonsquared nonsquared
100
6.3k
Save Time (by Creating Custom Rails Generators)
Avatar for Garrett Dimon garrettdimon
PRO
33
5.3k
Site-Speed That Sticks
Avatar for Harry Roberts csswizardry
13
1.6k

Transcript

  1. Configuration Management 101! Scale 12x

  2. AKA

  3. Sean Drops the Fucking Science

  4. Sean OMeara! [email protected]! @someara

  5. Sean OMeara! [email protected]! @someara

  6. Sean OMeara! [email protected]! @someara

  7. whoami

  8. Part 1

  9. The Dawn of Configuration Management

  10. • What is configuration management?! • Strategies and techniques for

    managing configuration and its complexity! • The art of change management

  11. Manual Configuration

  12. • Intuitive! • How we all start out! • Log

    into machine, manipulate with fingers! • Make with the clicky clicky! • Long tradition

  13. • Somehow feels the “safest"! • First instinct in emergencies!

    • This is an illusion! • Do not do this

  14. • Labor intensive! • Error prone! • Difficult to reproduce!

    • Obviously unsustainable

  15. Scripting

  16. • setup.sh! • setup.pl! • setup.py! • setup.rb

  17. • doit.sh! • doit.pl! • doit.py! • doit.rb

  18. • Ad-hoc in nature! • Loss of history! • Lacks

    testing methodology! • A step in the right direction

  19. File Distribution

  20. • NFS! • SMB! • AFS! • SSHFS! • GlusterFS

  21. • uucp! • rcp! • ftp! • http! • scp

  22. •Distributed systems! •Shares often managed manually or with scripts! •Package

    repositories! •Pull is better than push! •Scp on a cron *

  23. Execution Management

  24. • Image management! • Snapshots and cloning! • Containers

  25. • SSH on a for loop! • Func! • Commands

    on message queues! • ISConf

  26. • Loss of history! • Image sprawl! • Easy to

    order change across nodes

  27. Convergent Operators! (promises)

  28. None

  29. Mark Burgess

  30. The rest of us

  31. Tools

  32. • CFEngine! • Bcfg2! • Puppet! • Chef! • Salt!

    • Ansible
  33. None

  34. Part 2

  35. Policy http://www.flickr.com/photos/sfllaw/222795669/

  36. • /etc/passwd should be mode 0644! • /etc/shadow should be

    mode 0600

  37. • user ‘kermit’ should exist! • user ‘fonzi’ should exist!

    • group ‘muppets’ should exist! • group ‘muppets’ should contain kermit and fonzi

  38. • package ‘ntpd’ should be installed! • ntpd should sync

    with our AD service! • service ‘ntpd’ should be running

  39. • package ‘httpd’ should be installed! • httpd should be

    expose /mnt/software/java! • service ‘httpd’ should be running

  40. • The Java JDK, version 7u45, found on an internally

    hosted web server, should be installed into /usr/local/jdk-7u45/

  41. Polices are declarations about the state of things in a

    system

  42. Polices are applied repeatedly and repair the system when needed

  43. Policies often change

  44. • package ‘widget-factory’ should be installed at version 1.2.3

  45. • package ‘widget-factory’ should be installed at version 1.3.0

  46. http://www.flickr.com/photos/jakepjohnson/4937767595 Repeatability

  47. Repeatable -> Idempotent -> Convergent

  48. • Scripts are not generally repeatable

  49. None
  50. None

  51. • But they can be!

  52. None

  53. ! Idempotent operations can be applied infinite times and will

    yield the same result every time
  54. None

  55. Idempotent http://www.flickr.com/photos/ian_munroe/4758240536/

  56. http://www.flickr.com/photos/ian_munroe/4758240536/ Idempotent NOT GOOD ENOUGH

  57. ! Convergent operations test state and repair if needed

  58. None
  59. None

  60. ! A control loop keeps the system stable and allows

    for change when policy is updated

  61. Autonomous agent Policy: The box should be closed

  62. Convergence

  63. None
  64. None
  65. None

  66. Converging with Bash

  67. git clone [email protected]:someara/ cbash.git

  68. None
  69. None
  70. None
  71. None
  72. None
  73. None
  74. None
  75. None
  76. None
  77. None
  78. None
  79. None
  80. None

  81. Convergence and Iteration

  82. None
  83. None
  84. None
  85. None
  86. None
  87. None

  88. Does order matter?

  89. YES

  90. None

  91. Promises http://www.flickr.com/photos/nazzen9009/6809694353/

  92. • Agents are autonomous! • A promise is a signal

    or message perceived by an observer.! • Promises may or may not be kept.! • Agents can observe other agents! • Agents only have local information *! • Inner workings of agents are assumed to be unknown http://markburgess.org/BookOfPromises.pdf

  93. • Agents have intentions (possible behaviors)! • Agents can make

    assessments about other agents http://markburgess.org/BookOfPromises.pdf

  94. • Configuration Management tools embody tenants of Promise Theory intentionally

    or not

  95. Domain Specific Languages

  96. ! DSLs restrict machine instructions to convergent operations

  97. ! DSLs manage ordering

  98. None

  99. type subject intentions

  100. None

  101. type subject intentions

  102. signal

  103. None

  104. type subject intention

  105. observation

  106. None

  107. type subject intentions

  108. None

  109. type intention subject

  110. signal

  111. Intermission

  112. None

  113. Part 3

  114. Composition

  115. None

  116. Recipes

  117. resource one resource two resource three

  118. { testable intent

  119. recipe[http::server]

  120. recipe[http::server]

  121. recipes supporting files

  122. Types

  123. None

  124. interface implementation

  125. None

  126. intentions parameters

  127. None

  128. new scope intention implementation

  129. new scope intention implementation

  130. Artifacts

  131. metadata

  132. None

  133. metadata

  134. None

  135. http v0.1.0 chef-server api yum v3.0.0

  136. Delivery

  137. • nodes request their own initial run_list

  138. recipe[httpd::server] chef-server api run_list: http v0.1.0

  139. recipe[httpd::server] chef-server api run_list: http v0.1.0 recipe[openssh::server] openssh v3.2.1

  140. recipe[ntp::client] chef-server api run_list: http v0.1.0 recipe[openssh::server] openssh v3.2.1 recipe[httpd::server]

    ntp v1.0.0

  141. • Push vs Pull! • Networking considerations! • Machines down

    for maintenance! • Machines that don’t exist yet

  142. Dependencies

  143. None
  144. None
  145. None
  146. None
  147. None
  148. None

  149. recipe[widgetfactory] chef-server api run_list: http v0.1.0 yum v3.0.0 widgetfactory v1.0.0

  150. Integration testing

  151. • Test that a set of agents has achieved their

    combined goal

  152. • lsof -i :80! • ps -ef | grep httpd!

    • curl localhost 2>&1 > /dev/null

  153. • Berkshelf! • Vagrant! • Kitchen.ci! • Bats! • Serverspec

  154. Environments

  155. • Environments constrain cookbook versions! • Environments can set data

  156. None
  157. None

  158. • Environments can be used to test branches! • Environments

    can be used to segregate machines! • Environments can be manipulated programatically
  159. None
  160. None

  161. http v0.1.0 chef-server api http v0.2.0 openssh v1.2.3 postgresql v3.2.1

  162. recipe[widgetfactory] run_list: http v0.1.0 yum v3.0.0 widgetfactory v1.0.0 chef_environment: production

  163. recipe[widgetfactory] run_list: http v0.2.0 yum v3.0.0 widgetfactory v1.0.0 chef_environment: staging

  164. Part 4

  165. Clusters http://www.flickr.com/photos/youraccount/5938852370/

  166. Typical Cluster

  167. loadbalancer application db-slave db-master

  168. Production httpd 0.1.0

  169. Production Staging httpd 0.1.0 httpd 0.1.0

  170. Production Staging UUID httpd 0.1.0 httpd 0.1.0 httpd 0.2.0

  171. Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0

  172. Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0

  173. Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0

  174. Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0

  175. Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0

  176. Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0

  177. Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0

  178. Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0

  179. Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0

  180. Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0

  181. Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0

  182. Production Staging httpd 0.2.0 httpd 0.2.0

  183. Production httpd 0.2.0

  184. An Ordering Problem

  185. loadbalancer application

  186. • Take a machine out of the pool! • Drain

    the connections! • Modify configuration! • Insert it back into the pool

  187. loadbalancer application

  188. loadbalancer application

  189. loadbalancer application

  190. loadbalancer application

  191. loadbalancer application

  192. loadbalancer application

  193. loadbalancer application

  194. loadbalancer application

  195. loadbalancer application

  196. loadbalancer application

  197. Orchestration

  198. • Conductor showing signals to autonomous agents (creative policy manipulation)!

    • External actor controlling sequencing (execution management)! • Application level sequencing (vector clocks, etc)

  199. • Infrastructures are snowflakes! • Solutions are unique to applications

    by nature! • Configuration Management 201

  200. • There is no separation between ‘infrastructure’ and ‘application’! •

    Distributed systems are hard! • Specialists need to work together

  201. Devops

  202. • Study Promise Theory! • Study distributed systems! • Develop

    high quality primitives! • Be excellent to each other

  203. Fin