Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Chef and OpenStack

Chef and OpenStack

Presentation from my talk at OpenStack Denver 5/20/2013

Joshua Timberman

May 20, 2013
Tweet

More Decks by Joshua Timberman

Other Decks in Technology

Transcript

  1. % whoami • System Administrator • I work for Opscode

    • I write Chef cookbooks totally legit mustache! Monday, May 20, 13
  2. Setting Expectations... • Both Chef and OpenStack are large frameworks

    • Opscode does 3 day training classes on Chef • Deploying OpenStack can take weeks, if not months • This talk is a technical overview/preview Monday, May 20, 13
  3. Items of Manipulation (Resources) • Nodes • Networking • Files

    • Directories • Symlinks • Mounts • Routes • Users • Groups • Packages • Services • Filesystems Monday, May 20, 13
  4. App LBs App Servers App DB Cache App DBs Now

    we need a caching layer Monday, May 20, 13
  5. Round Robin DNS App Servers App DB Cache App DBs

    Floating IP? Your's Is a Snowflake Monday, May 20, 13
  6. App LBs App Servers < Shiny! DB slaves Cache DB

    Cache DBs Complexity Increases Quickly Are we monitoring?? Monday, May 20, 13
  7. Why Chef? • Rackspace Cloud • AT&T • Dreamhost •

    HP Cloud • IBM SmartCloud • Cloudscaling • Others... Monday, May 20, 13
  8. The Chef Framework • Reasonability • Flexibility • Library &

    Primitives • TIMTOWTDI • Sane defaults http://www.flickr.com/photos/wonderlane/3609342683/sizes/l/in/photostream/ Monday, May 20, 13
  9. The Chef Tool(s) • ohai • chef-client • chef-shell •

    knife • The Ruby language Omnibus - Full Stack Native Packages Monday, May 20, 13
  10. The Chef Flavors • Chef Solo • Chef Server •

    Opscode Hosted Chef • Opscode Private Chef http://www.flickr.com/photos/seanbuchandpt/6020113519/ Monday, May 20, 13
  11. The Chef API • HTTPS, RESTful API w/ JSON, RSA

    key auth • Infrastructure data store such as node data • Search Service • Derivative Services? http://www.flickr.com/photos/core-materials/4419853626/sizes/o/in/photostream/ Monday, May 20, 13
  12. The Chef Community • Apache License, Version 2.0 • Hundreds

    of Individual and Corporate contributors. • Hundreds of cookbooks available from the community • http://community.opscode.com Monday, May 20, 13
  13. Chef Enables Infrastructure as Code • Resources • Recipes •

    Cookbooks and Roles • Source Code http://www.flickr.com/photos/louisb/4555295187/ Monday, May 20, 13
  14. package "openstack-nova-common" do action :install end directory "/etc/nova" do owner

    "nova" group "nova" mode 00700 end template "/etc/nova/nova.conf" do source "nova.conf.erb" owner "nova" group "nova" mode 00644 Declarative abstraction to system resources Monday, May 20, 13
  15. package "openstack-nova-common" do action :install end template "/etc/nova/nova.conf" do source

    "nova.conf.erb" owner "nova" group "nova" mode 00644 notifies :restart, "service[openstack-nova-api]" end service "openstack-nova-api" do supports :status => true, :restart => true action [:enable, :start] end Monday, May 20, 13
  16. def install_package(name, version) package_name = "#{name}=#{version}" package_name = name if

    @is_virtual_package run_command_with_systems_locale( :command => "apt-get -q -y #{expand_options(@new_resource.options)} install #{package_name}", :environment => { "DEBIAN_FRONTEND" => "noninteractive" } ) end Provider Example Monday, May 20, 13
  17. Package Resource package "git" { yum install git apt-get install

    git pacman sync git pkg_add -r git Providers are determined by node's platform Monday, May 20, 13
  18. Cookbooks • Cookbooks are collections of Recipes • Cookbooks contain

    related components • Files, Templates, Libraries • A cookbook is responsible for configuring a single thing, e.g. • nova • postgresql • A recipe is responsible for a component, e.g. • api • server • client Monday, May 20, 13
  19. Example cookbook openstack-compute ├── README.md ├── attributes └── default.rb ├──

    files └── default └── nova-compute.conf ├── metadata.rb ├── recipes ├── api-ec2.rb ├── default.rb └── nova-common.rb └── templates └── default └── nova.conf.erb >openstack-compute.tar.gz Monday, May 20, 13
  20. Roles • Roles describe nodes • Roles indicate functionality •

    Roles have a first class API primitive Monday, May 20, 13
  21. Example Role name "os-compute-api" description "OpenStack API for Compute" run_list(

    "role[os-base]", "recipe[openstack-compute]" ) Monday, May 20, 13
  22. http://www.flickr.com/photos/peterrosbjerg/3913766224/ Chef Nodes • Chef nodes do the heavy lifting

    • Authority about themselves • Stored on the server when using Chef Server • Indexed for search Monday, May 20, 13
  23. Search • Search for nodes with Roles • Find Topology

    Data • IP addresses • Hostnames • FQDNs http://www.flickr.com/photos/kathycsus/2686772625 Monday, May 20, 13
  24. OpenStack is complex • Nova (compute) • Swift (object store)

    • Glance (image) • Quantum (network) • Keystone (identity) • Cinder (block storage) • Horizon (dashboard) Monday, May 20, 13
  25. Deploying OpenStack • Start with the Chef Repository • Upload

    Chef components to a Chef Server • "Bootstrap" nodes with the appropriate role • role[os-ops-database] (mysql/postgresql) • role[os-ops-messaging] (rabbitmq) • role[os-identity] (Keystone) • role[os-image] (Glance) • role[os-network] (Quantum) • role[os-compute-scheduler] (Nova) • role[os-compute-api] (Nova) • role[os-block-storage] (Cinder) • role[os-dashboard] (Horizon) Monday, May 20, 13
  26. Chef for OpenStack: Project • Community for the automated deployment

    and management of OpenStack • Reduce fragmentation and encourage collaboration • Deploying OpenStack is not "secret sauce" • Project not a product • Apache 2 license Monday, May 20, 13
  27. Various Projects and Code • Chef Repository for Deploying OpenStack

    • Documentation for Chef for OpenStack • Cookbooks for each component • Knife plugin Monday, May 20, 13
  28. knife openstack % knife openstack knife openstack flavor list knife

    openstack group list knife openstack image list knife openstack server create knife openstack server delete knife openstack server list Monday, May 20, 13
  29. knife openstack server create % grep openstack ~/.chef/knife.rb knife[:openstack_ssh_key_id] =

    ENV['OPENSTACK_SSH_KEY'] knife[:openstack_username] = ENV['OPENSTACK_USERNAME'] knife[:openstack_password] = ENV['OPENSTACK_PASSWORD'] knife[:openstack_tenant] = ENV['OPENSTACK_TENANT'] knife[:openstack_auth_url] = ENV['OPENSTACK_AUTH_URL'] Monday, May 20, 13
  30. knife openstack server create knife openstack server create \ -x

    root \ -f 7 \ -S jtimberman \ -I e0184596-577f-4eb0-9887-d70117c6b77b \ -r 'role[frontend-app]' Monday, May 20, 13
  31. knife openstack server create knife openstack server create \ -x

    root \ -f 7 \ -S jtimberman \ -I e0184596-577f-4eb0-9887-d70117c6b77b \ -r 'role[frontend-app]' SSH as this user Use this SSH keypair Monday, May 20, 13
  32. knife openstack server create knife openstack server create \ -x

    root \ -f 7 \ -S jtimberman \ -I e0184596-577f-4eb0-9887-d70117c6b77b \ -r 'role[frontend-app]' OpenStack "flavor" identifier Monday, May 20, 13
  33. knife openstack server create knife openstack server create \ -x

    root \ -f 7 \ -S jtimberman \ -I e0184596-577f-4eb0-9887-d70117c6b77b \ -r 'role[frontend-app]' OpenStack "image" (e.g., Debian 6.0) Monday, May 20, 13
  34. knife openstack server create knife openstack server create \ -x

    root \ -f 7 \ -S jtimberman \ -I e0184596-577f-4eb0-9887-d70117c6b77b \ -r 'role[frontend-app]' Chef "run list" for the node Monday, May 20, 13
  35. knife openstack server create Instance Name: os-7971334684686417 Instance ID: e609a9bd-8e64-4184-94b6-4ef1ca7b9909

    Waiting for server.................. Flavor: 7 Image: e0184596-577f-4eb0-9887-d70117c6b77b SSH Keypair: jtimberman Public IP Address: 172.31.6.54 Waiting for sshd..done Bootstrapping Chef on 172.31.6.54 172.31.6.54 Setting up chef (11.4.4-2.debian.6.0.5) ... 172.31.6.54 Thank you for installing Chef! 172.31.6.54 Starting Chef Client, version 11.4.4 172.31.6.54 Creating a new client identity for os-7971334684686417 using the validator key. 172.31.6.54 resolving cookbooks for run list: ["apt", "apache2"] Monday, May 20, 13
  36. Chef Client Runs, Victory! 172.31.6.54 Chef Client finished, 18 resources

    updated Instance Name: os-7971334684686417 Instance ID: e609a9bd-8e64-4184-94b6-4ef1ca7b9909 Flavor: 7 Image: e0184596-577f-4eb0-9887-d70117c6b77b SSH Keypair: jtimberman Public IP Address: 172.31.6.54 Environment: _default Run List: role[frontend-app] Monday, May 20, 13
  37. knife node show % knife node show os-7971334684686417 Node Name:

    os-7971334684686417 Environment: _default FQDN: debianx8664.opscode.us IP: 172.31.6.54 Run List: role[frontend-app] Roles: frontend-app Recipes: apt, apache2 Platform: debian 6.0.4 Tags: % knife node show os-7971334684686417 -a cloud.provider os-7971334684686417: cloud.provider: openstack Talks to Chef Server Monday, May 20, 13
  38. knife-openstack compatibility • Uses the OpenStack API • Diablo, Essex,

    Folsom, Grizzly • Cloudscaling • Crowbar • DreamHost • Nebula • Piston • Rackspace Private Cloud Monday, May 20, 13
  39. Chef for Infrastructure Portability • knife openstack • Crowbar, DreamHost,

    Nebula, Piston • knife hp • knife rackspace • knife cloudstack • knife ec2 • knife google • Consolidation coming soon Monday, May 20, 13
  40. Summary • Big players are betting on OpenStack • Many

    orgs are using Chef to deploy OpenStack • There's a growing community of users for Chef + OpenStack http://www.flickr.com/photos/villes/358790270/ Monday, May 20, 13