5 Things You Didn't Know About Chef

This is the slide deck I used to present at Big Ruby 2013.

http://www.bigrubyconf.com/

Transcript

1. 5 Things You Didn't Know About Chef Joshua Timberman joshua@opscode.com jtimberman community.opscode.com Thursday, February 28, 13
2. Chef is an Automation Platform for developers & systems engineers to build & manage Infrastructure as Code. "743 lines of #opschef to build new product infrastructure from scratch. Thank you, @opscode.” -@bdha, August 18, 2011 What is Chef? Thursday, February 28, 13
3. Chef is a Framework • Configuration management • Reasonability • Idempotent, Convergent • Flexibility • Library & Primitives • TIMTOWTDI http://www.flickr.com/photos/elitatt/6980379333/ Thursday, February 28, 13
4. package "haproxy" do action :install end template "/etc/haproxy/haproxy.cfg" do source "haproxy.cfg.erb" owner "root" group "root" mode 0644 notifies :restart, "service[haproxy]" end service "haproxy" do supports :restart => true action [:enable, :start] end Chef Enables Infrastructure as Code • Resources • Recipes • Cookbooks and Roles • Source Code Thursday, February 28, 13
5. Agenda • In-place File Editing for Greater Good • Use Chef's Built-in Version Matching • REPLs are fun, so Chef has one! • Take the Resource Collection for a Stroll • The Anatomy of Loading and Executing a Single Recipe http://www.flickr.com/photos/koalazymonkey/3590953001/ • Bonus, if we have time! Thursday, February 28, 13
6. http://www.flickr.com/photos/dolarz/2333292651/ In-place File Editing (for Greater Good™) Thursday, February 28, 13
7. net.ipv4.conf.default.rp_filter = 1 net.ipv4.conf.default.accept_source_route = 0 kernel.sysrq = 0 kernel.core_uses_pid = 1 net.ipv4.tcp_synack_retries = 2 net.ipv4.conf.all.send_redirects = 0 net.ipv4.conf.default.send_redirects = 0 net.ipv4.conf.all.accept_source_route = 0 net.ipv4.conf.all.accept_redirects = 0 net.ipv4.conf.all.secure_redirects = 0 net.ipv4.conf.all.log_martians = 1 net.ipv4.conf.default.accept_source_route = 0 net.ipv4.conf.default.accept_redirects = 0 net.ipv4.conf.default.secure_redirects = 0 net.ipv4.icmp_echo_ignore_broadcasts = 1 net.ipv4.tcp_syncookies = 1 net.ipv4.conf.all.rp_filter = 1 net.ipv4.conf.default.rp_filter = 1 net.ipv6.conf.default.router_solicitations = 0 net.ipv6.conf.default.accept_ra_rtr_pref = 0 net.ipv6.conf.default.accept_ra_pinfo = 0 net.ipv6.conf.default.accept_ra_defrtr = 0 net.ipv6.conf.default.autoconf = 0 net.ipv6.conf.default.dad_transmits = 0 net.ipv6.conf.default.max_addresses = 1 kernel.exec-shield = 1 kernel.randomize_va_space = 1 I need to just update this one line... http://www.flickr.com/photos/dmswart/6507071701/ < Right here? Thursday, February 28, 13
8. http://www.flickr.com/photos/rhysasplundh/4807749216/ Thursday, February 28, 13
9. Danger: Anti-pattern? • This makes it hard to reason about where changes to files are coming from • Manage whole-file resources (template, cookbook_file, remote_file) • But sometimes you have to edit a line... Thursday, February 28, 13
10. Introducing Chef::Util::FileEdit ruby_block "edit etc hosts" do block do rc = Chef::Util::FileEdit.new("/etc/hosts") rc.search_file_replace_line( /^127\.0\.0\.1 localhost$/, "127.0.0.1 #{new_fqdn} #{new_hostname} localhost" ) rc.write_file end end Thursday, February 28, 13
11. Chef::Util::FileEdit methods... #if matched, replace the whole line with newline. search_file_replace_line(regex, newline) #if matched, replace the match (all occurances) with the replace parameter search_file_replace(regex, replace) #if matched, delete the line search_file_delete_line(regex) #if matched, delete the match (all occurances) from the line search_file_delete(regex) #if matched, insert newline after each matching line insert_line_after_match(regex, newline) #if not matched, insert newline at the end of the file insert_line_if_no_match(regex, newline) #write the new file content write_file Thursday, February 28, 13
12. Sean OMeara's line cookbook • Sean OMeara wrote a "line" cookbook. • Resource/Provider for managing single lines. • http://ckbk.it/line http://www.flickr.com/photos/49889874@N05/6743622955/ Thursday, February 28, 13
13. Use Chef's Built-in Version Matching http://www.flickr.com/photos/tschiae/8395137978/ Thursday, February 28, 13
14. Places to match versions • Platforms • Cookbooks • Software/packages Thursday, February 28, 13
15. Some Caveats • Match versions that are dotted notation up to 3 places • X.Y.Z • 5.6, 1.8.0, etc Thursday, February 28, 13
16. Common Example if node['platform_version'].to_f >= 10.04 if node['platform_version'] == "6.3" Thursday, February 28, 13
17. Chef::VersionConstraint • First class comparison operators • Allows X.Y.Z version strings • Not lossy, unlike (potentially) #to_i or #to_f. http://docs.opscode.com/essentials_cookbook_versions_constraints.html Thursday, February 28, 13
18. How to use Chef::VersionConstraint • `require` the library • Initialize a Chef::VersionConstraint object w/ the constraint • Send the include? method with the version to match Thursday, February 28, 13
19. Constraint Operator = Equal > Greater than < Less than >= Greater than / equal <= Less than / equal ~> Approximately greater than (pessimistic) Thursday, February 28, 13
20. Chef::VersionConstraint require 'chef/version_constraint' Chef::VersionConstraint.new(">= 10.7.0").include?("10.6.0") #=> false Chef::VersionConstraint.new(">= 10.7.0").include?("10.7.3") #=> true Chef::VersionConstraint.new(">= 10.7.0").include?("10.8.2") #=> true Thursday, February 28, 13
21. Version Examples # On CentOS 5.8 Chef::VersionConstraint.new("~> 6.0"). include?(node['platform_version']) #=> false # On CentOS 6.3 Chef::VersionConstraint.new("~> 6.0"). include?(node['platform_version']) #=> true Thursday, February 28, 13
22. Debian 6.0.3 vs. #to_f 6.0 == "6.0.3".to_f # => true 6.0 < "6.0.3".to_f # => false 6.0 > "6.0.3".to_f # => false Thursday, February 28, 13
23. Debian 6.0.3 vs. #to_f 6.0 == "6.0.3".to_f # => true 6.0 < "6.0.3".to_f # => false 6.0 > "6.0.3".to_f # => false Chef::VersionConstraint.new("> 6.0"). include?("6.0.3") # => true Thursday, February 28, 13
24. REPLs are fun, Chef has one! http://www.flickr.com/photos/tanaka_juuyoh/4434232143/ Thursday, February 28, 13
25. chef-shell, the Chef REPL • Chef 11: chef-shell • Chef 10: shef • Built on IRB • Three modes/contexts: • main, attributes, recipe http://www.flickr.com/photos/51064559@N06/5746903658/ Thursday, February 28, 13
26. chef-shell main context • Access node attributes • Interact with a Chef Server • It's IRB... Thursday, February 28, 13
27. chef-shell main % chef-shell loading configuration: none (standalone session) Session type: standalone Loading...done. This is the chef-shell. Chef Version: 11.2.0 http://www.opscode.com/chef http://wiki.opscode.com/display/chef/Home run `help' for help, `exit' or ^D to quit. Ohai2u jtimberman@champagne chef > Thursday, February 28, 13
28. chef-shell main chef > help chef-shell Help ================================================== | Command | Description ================================================== | Lots of commands | and their descriptions chef > halp :-) Thursday, February 28, 13
29. chef-shell main context • Many of the commands available work with a Chef Server's API. • So let's start up chef-shell configured for a Chef Server. Thursday, February 28, 13
30. Interact with a Chef Server % chef-shell -z loading configuration: /etc/chef/client.rb Session type: client ... chef > Chef::Config[:chef_server_url] => "https://api.opscode.com/organizations/my_org" chef > node.run_list => role[mac_os_x], role[workstation] Thursday, February 28, 13
31. chef-shell attributes_mode • Context of a cookbook's attributes file. • Useful to paste content from your cookbook. • Follows the same rules as Chef loading attributes files Thursday, February 28, 13
32. chef-shell attributes_mode chef > attributes_mode chef:attributes > default['shef'] = "Fun" => "Fun" chef:attributes > node['shef'] = "Awesometown" Chef::Exceptions::ImmutableAttributeModification: Node attributes are read-only when you do not specify which precedence level to set. To set an attribute use code like `node.default["key"] = "value"' chef:attributes > node.default['shef'] = "Awesometown" => "Awesometown" Thursday, February 28, 13
33. chef-shell recipe_mode • Context of a Chef Recipe. • All resources, recipe DSL methods are available. • Highly useful for debugging! • Run in client mode for interacting with a Chef Server. Thursday, February 28, 13
34. chef-shell recipe_mode % sudo chef-shell -z chef > recipe_mode chef:recipe > package "git" => <package[git] @name: "git" SNIP> chef:recipe > package "emacs" => <package[emacs] @name: "emacs" SNIP> Thursday, February 28, 13
35. chef-shell recipe_mode chef:recipe > resources ["package[git]", "package[emacs]"] => ["package[git]", "package[emacs]"] chef:recipe > run_chef Thursday, February 28, 13
36. chef-shell recipe_mode chef:recipe > search(:node, "role:workstation") => [node[doppelbock], node[champagne], node[merlot]] chef:recipe > data_bag_item(:users, "jtimberman") => data_bag_item[:users, "jtimberman", {"groups"=>["jtimberman", "wheel", "sysadmin", "dialout", "cdrom", "floppy", "audio", "video", "plugdev", "sambashare", "sbuild"], "comment"=>"Joshua Timberman"} Thursday, February 28, 13
37. Chef Shell Extension library % pry 1.9.3 (main):0 > require 'chef' 1.9.3 (main):0 > require 'chef/shell/ext' 1.9.3 (main):0 > Shell::Extensions.extend_context_object(self) 1.9.3 (main):0 > Chef::Config.from_file("/etc/chef/client.rb") 1.9.3 (main):0 > nodes.all => [node[doppelbock], node[champagne], node[menthe], node[merlot], node[virt1test], node[cask]] Use this in: knife plugins your libraries your application Thursday, February 28, 13
38. Take the Resource Collection for a Stroll http://www.flickr.com/photos/bradhoc/5682038880 Thursday, February 28, 13
39. The Resource Collection • Chef loads recipes looking for resources • Each resource is added to the ResourceCollection • Once done, Chef walks the resource collection, taking the appropriate action Thursday, February 28, 13
40. Consider this recipe: package "apache2" template "/etc/apache2/apache2.conf" do mode 0644 owner "www-data" group "www-data" notifies :restart, "service[apache2]" end service "apache2" do supports :reload => true, :status => true action [:enable, :start] end Thursday, February 28, 13
41. Paste the recipe in chef-shell then use resources() chef:recipe > resources ["package[apache2]", "template[/etc/apache2/ apache2.conf]", "service[apache2]"] • resources returns an array of all the resources • Each resource has a type and a name. Thursday, February 28, 13
42. run_context.resource_collection • The run_context loads and tracks the context of the Chef run. • The resource collection is part of the run context. • Access the resource collection directly with run_context.resource_collection Thursday, February 28, 13
43. Show the resource collection chef:recipe > run_context.resource_collection => #<Chef::ResourceCollection:0x000000015cc690 ... snip ... @resources_by_name={"package[apache2]"=>0, "template[/etc/apache2/apache2.conf]"=>1, "service[apache2]"=>2}, @insert_after_idx=nil> • resource_by_name is a hash of all the resources. • This is what #resources() looks up. • Note the numeric index Thursday, February 28, 13
44. Walk the resource collection chef:recipe > run_context.resource_collection.each do |r| puts "#{r} is a #{r.resource_name}" end package[apache2] is a package template[/etc/apache2/apache2.conf] is a template service[apache2] is a service Thursday, February 28, 13
45. The Anatomy of Loading and Executing a Single Recipe http://www.flickr.com/photos/a_cooper/2552248132/ Thursday, February 28, 13
46. chef-apply • Started from a gist • Now included in Chef 11 • /usr/bin/chef-apply • Thanks, Bryan Berry! https://gist.github.com/danielsdeleo/2920702 Thursday, February 28, 13
47. Load the required libraries require 'chef/client' require 'chef/providers' require 'chef/resources' Thursday, February 28, 13
48. Set up the run_context Chef::Config[:solo] = true client = Chef::Client.new client.node = Chef::Node.build("apply-node") client.run_ohai client.build_node run_context = Chef::RunContext.new( client.node, {}, client.events ) Thursday, February 28, 13
49. Load the recipe recipe = Chef::Recipe.new( "(chef-apply cookbook)", "(chef-apply recipe)", run_context ) recipe.from_file(recipe_path) Thursday, February 28, 13
50. Load the context and converge runner = Chef::Runner.new(run_context) runner.converge Thursday, February 28, 13
51. chef-apply takes STDIN... % while true do nc -l 8787 | sudo chef-apply -s done # Elsewhere... telnet 10.1.1.127 8787 package "git" ^] ^D Never run this anywhere. Thursday, February 28, 13
52. http://www.flickr.com/photos/michaelheiss/3090102907/ There is a lot more to Chef... Thursday, February 28, 13
53. More about Chef • learnchef.com • www.opscode.com/chef/install • docs.opscode.com • lists.opscode.com • community.opscode.com Thursday, February 28, 13
54. chefconf.opscode.com Thursday, February 28, 13
55. Round 6, BONUS ROUND! Thursday, February 28, 13
56. FAQ: How do you test cookbooks? Thursday, February 28, 13
57. FAQ: Testing • Chef 10.14+, “why-run” mode • Vagrant! • Test Kitchen (1.0 coming soon) • ChefSpec • Cucumber Chef Thursday, February 28, 13
58. FAQ: Why is Chef Server so hard to install? Thursday, February 28, 13
59. It isn't! (anymore) http://www.flickr.com/photos/peregrinari/3801964067 Thursday, February 28, 13
60. Chef 11 • Chef 11 Solves this • Single package installation • Two commands! Thursday, February 28, 13
61. FAQ: Installation difficulty • Configuration management systems are complex in their own right. • The Chef Server is a publishing system with an API. • Chef is a service-oriented web application. Thursday, February 28, 13