Salt: How To Be Truly Lazy

Talk on SaltStack given at PyCon Australia 2013

Transcript

1. @LexualChocolate Lex Hider github.com/lexual JBA How To Be Truly Lazy PyCon Australia 2013 saltstack.com
3. @LexualChocolate Lex Hider github.com/lexual JBA How To Be Truly Lazy PyCon Australia 2013 saltstack.com
4. Alternate Title
8. Jacob Kaplan-Moss Django Co-creator
15. Agenda: Configuration Management Saltified
16. # salt's hello world >> salt '*' test.ping • Salt overview • Remote exec demo Agenda
17. Agenda
18. Agenda SALTY
19. @LexualChocolate Lex Hider github.com/lexual JBA How To Be Truly Lazy PyCon Australia 2013
20. Not a Devops guy or sys-admin No powerpoint skills Never used chef/puppet
21. Homer Jay Simpson Patron Saint of the Lazy
23. ARCHITECTURE
24. Master Architecture
25. Architecture Minions
27. CONFIGURATION MANAGEMENT SALT STATES Sys-admin becomes just like programming, editing text files in your favourite editor
28. REMOTE EXECUTION
29. • no ssh PARALLEL EXECUTION
30. Targeting
32. • Static info available at startup • Use to target for remote execution • Available in Configuration Management (Salt States) SALT GRAINS
33. DEMO
34. UNDER THE HOOD
35. WRITTEN IN PYTHON APACHE LICENSED
36. VERY ACTIVE COMMUNITY • 8th most unique contributors in 2012 out of all github.com hosted projects. • Bugs often fixed in a few days, if not hours.
37. SALT IS LIGHTWEIGHT A SINGLE MASTER CAN MANAGE 1000s OF SERVERS
38. • MASTER DAEMON • MINION DAEMON • PUB/SUB • PORTS 4505/4506 ON MASTER
39. • Efficient binary serialization format. MessagePack: It's like JSON, but fast and small.
40. EVERYTHING ON THE WIRE IS ENCRYPTED • PUBLIC KEYS TO AUTHENTICATE WITH MASTER • KEYS GENERATED FOR YOU, JUST TELL MASTER TO ACCEPT • AES ENCRYPTION FOR PAYLOAD COMMUNICATION
41. SALT'S PHILOSOPHY SIMPLICITY
44. # Install a minion wget -O - http://bootstrap.saltstack.org | sudo sh # Install a master wget -O - http://bootstrap.saltstack.org | sudo sh -s -- -M INSTALLING SALT (salt bootstrap)
45. CONFIGURING MASTERS + MINIONS • Pretty much works out of the box • Shouldn't need to change master's config • Single change to minion config to know where the master is: # /etc/salt/minion # master: <ip/domain name of master> master: salt.lexual.com
46. SALT EXECUTION MODULES ARE JUST PYTHON FUNCTIONS This is the actual code for: >> salt '*' test.ping # modules/test.py def ping(): return True
47. B.C.M • A Google Doc with steps to follow to create a dev build or to deploy a new production server ;(
49. • Single command to deploy dev or production build, from single salt state tree. • Dev & production builds nearly identical • Single command to spin up new cloud server as a new minion • Single command to spin up new virtual machine as new dev build. A.C.M
50. SALT STATES: YAML + JINJA (CONFIGURATION MANAGEMENT) YAML # A list - a - b - c # A dict first_name: homer last_name: simpson JINJA (BASICALLY DJANGO TEMPLATE) {{ some_variable }} {% if True %} {% endif %} {% for foo in bars %}
51. THIS IS JUST THE DEFAULT! CAN USE: • Python code • Jinja/Mako/Wempy • YAML/JSON • pydsl • Write your own "Renderer" • States are just a data structure!
52. DEFAULTS Just the default Can always easily write your own in python: • renderers (default: yaml + jinja) • execution modules (python functions) • returners (default: send back to master) alternatives: mysql, redis, etc, etc. • state modules (mostly wrappers around exec modules) • Use, the source Luke. github.com/saltstack
53. STATE FILES (SLS) /srv/salt/top.sls /srv/salt/common.sls /srv/salt/nginx.sls /srv/salt/gitrepo/init.sls ...
54. STATE TOP FILE (TARGETING) # /srv/salt/top.sls base: '*': - common 'demo*': - sl 'role:django_website': - match: grain - django ...
55. SIMPLE STATE FILES # /srv/salt/common.sls common_packages: pkg.installed: - pkgs: - vim - tmux # /srv/salt/sl/init.sls sl: pkg.installed
56. HIGH STATE • Tell Salt to look at the top. sls and apply the relevant states to the relevant minions • Idempotence • salt '*' state.highstate
57. Riak example (1/3) Deploy a dozen near-identical servers • Only different configuration on each host was the IP in config file. # /etc/riak/app.config ... {pb_ip, "10.240.2.145" }, ... {http, [ {"127.0.0.1", 8098 }, {"10.240.2.145", 8098 } ]}, ...
58. Riak example (2/3) (File Server) /etc/riak/app.config: file.managed: - source: salt://riak/app.config - mode: 644 - template: jinja - require: - pkg: riak - context: internal_ip:{{ salt['network.ip_addrs']()[0] }}
59. Riak example (3/3) (single source config) • Only different configuration on each host was the IP in config file. # /srv/salt/riak/app.config ... {pb_ip, "{{ internal_ip }}" }, ... {http, [ {"127.0.0.1", 8098 }, {"{{ internal_ip }}", 8098 } ]}, ...
61. PILLAR: GLOBAL VALUES FOR MINIONS • SECURITY: Sensitive Data • TARGETED (top.sls) • DRY #/srv/pillar/django.sls {% if grains['is_dev'] %} user: vagrant {% else %} user: ubuntu {% endif %}
62. PILLAR (cont.) {{ pillar['user'] }}: user.present: - home: /home/{{ pillar['user'] }} - groups: - sudo /home/{{ pillar['user'] }}/.vimrc file.managed: - source: salt://vimrc • Use to set password, and put into config file.
63. SALT STATE DEMO
64. SALT CLOUD pip install apache-libcloud salt-cloud sudo salt-cloud -p djangoproject djangoproj1 # wait 2m14.208s > sudo salt '*' test.ping djangoproj1: True
66. SALT CLOUD DEMO
67. SALTY VAGRANT DEMO
69. CONCLUSION • Salt is awesome • Salt does *much* more than I have shown • The most important thing is you're using a CM tool, which one is much less important. • RTFM: it's fantastic!!
70. linkd.in/12Kgg5K WE'RE HIRING! • Django/Python Developer • Melbourne Work with some cool tech: • Salt • Riak (no-SQL db) • Pandas/Numpy/Scipy • git • AWS
72. BREAK GLASS IN CASE OF SALT DEMO FAILURE
73. > sudo salt '*' pkg.list_upgrades djangoproj1: ------------- ... python: 2.7.3-0ubuntu2.2 python-minimal: 2.7.3-0ubuntu2.2 python-paramiko: 1.7.7.1-2ubuntu1 python2.7: 2.7.3-0ubuntu3.2 python2.7-minimal: 2.7.3-0ubuntu3.2 ...
74. > sudo salt '*' pkg.list_upgrades djangoproj1: ------------- ... python: ---------- new: 2.7.3-0ubuntu2.2 old: 2.7.3-0ubuntu2 python-minimal: ---------- new: 2.7.3-0ubuntu2.2 old: 2.7.3-0ubuntu2 ...
75. > sudo salt '*' status.uptime djangoproj1: 00:51:31 up 11 min, 0 users, load average: 0.06, 0.19, 0.15 > sudo salt 'django*' system.reboot > sleep 2m && sudo salt 'django*' test.ping djangoproj1: True
76. > sudo salt 'dj*' cmd.run ls /etc/salt djangoproj1: minion minion.d minion.dpkg-dist pki > sudo salt 'dj*' cmd.exec_code python2 "print [x**2 for x in xrange(13)]" djangoproj1: [0, 1, 4, 9, 16, 25, 36, 49, 64, 81, 100, 121, 144] > salt 'dj*' cmd.exec_code python2 "import salt; print salt.version.__version__" djangoproj1: 0.15.3
77. sudo salt '*' grains.item lsb_description demo4: lsb_description: Ubuntu 11.10 demo2: lsb_description: Ubuntu 12.04.2 LTS djangoproj1: lsb_description: Ubuntu 12.04.2 LTS demo1: lsb_description: Ubuntu 12.04.2 LTS
78. sudo salt '*' cmd.run "python --version" demo4: Python 2.7.2+ demo2: Python 2.7.3 djangoproj1: Python 2.7.3 demo1: Python 2.7.3
79. sudo salt '*' cmd.exec_code python "import sys; print sys.version" demo4: 2.7.2+ (default, Jul 20 2012, 22:12:53) [GCC 4.6.1] demo2: 2.7.3 (default, Aug 1 2012, 05:14:39) [GCC 4.6.3] djangoproj1: 2.7.3 (default, Apr 10 2013, 06:20:15) [GCC 4.6.3] demo1: 2.7.3 (default, Aug 1 2012, 05:14:39) [GCC 4.6.3]