Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Juergen Brendel: Deploying test and production ...

Juergen Brendel: Deploying test and production systems with Ansible

= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
Juergen Brendel:
Deploying test and production systems with Ansible
= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
@ Kiwi PyCon 2014 - Sunday, 14 Sep 2014 - Track 1
http://kiwi.pycon.org/

**Audience level**

Intermediate

**Description**

This talk presents Ansible as a light-weight, simple and effective configuration management tool. We will see why configuration management is important and how we can use Ansible to automatically deploy and test full production clusters - showcased with a sample Django application - with just a single command and in a number of different environments.

**Abstract**

With modern configuration, deployment and orchestration systems, software developers can easily maintain, bring up and tear down local test systems, staging, demonstration and production systems, which are 100% identical to each other.

This presentation will give an introduction to Ansible, a modern, simple and efficient configuration management system. We will see how we can deploy complete clusters (with load balancers, database and some Django application servers), reliably and fully automated, either on dedicated servers, on local virtual machines or in the cloud.

If all goes well, there will even be a live demo in which we will bring up a complete cluster or two in different environments, to illustrate the power of this approach.

**YouTube**

https://www.youtube.com/watch?v=Dg5TSfLHW3Q

New Zealand Python User Group

September 14, 2014
Tweet

More Decks by New Zealand Python User Group

Other Decks in Programming

Transcript

  1. Automated Deployment with Ansible [email protected] @BrendelConsult Summary • Configuration management

    background • Ansible intro • Unified development, test and deployment environments • Ansible and the cloud • Contributing to Ansible
  2. Automated Deployment with Ansible [email protected] @BrendelConsult Arcane magic Configuring servers

    How do you configure a server? Manual instructions Scripts CM tools Automation!
  3. Automated Deployment with Ansible [email protected] @BrendelConsult Ensure all system packages

    are updated. CM Tools Describe the desired state Ensure that user “xyz” exists. Ensure package “apache” is installed. Have latest sources: Clone repo, update if it exists already. Ensure package “postgres” v9.1 is installed. Ensure DB “app_data” exists with password “****”.
  4. Automated Deployment with Ansible [email protected] @BrendelConsult CM Tools variety •

    Puppet (2005) • Chef (2009) • Salt (2011) • Ansible (2012) • Fabric • Scripts “powerful, feature-rich, enterprisy” “simple, fast, good for most things” “not really CM tools”
  5. Automated Deployment with Ansible [email protected] @BrendelConsult Ansible overview • “Orchestration

    engine” for CM and deployment • Written in Python • Uses YAML • “Playbooks” • Config specs or explicit commands • Linux and Windows
  6. Automated Deployment with Ansible [email protected] @BrendelConsult Ansible simplicity Key points:

    – No central configuration server – No key management – No agent to install on target machine – Explicit order Requirements: – Need SSH access (with key or password) – Need Python installed on target machine
  7. Automated Deployment with Ansible [email protected] @BrendelConsult Modules Hundreds of them.

    They know how to do stuff... - Command - Shell - Script - Copy - Sync - Templates - Line ops - Install packages - Users and groups - Networking - Services - Repositories - Message queues - Monitoring - Notification - Web servers - Database servers - Cloud infrastructure
  8. Automated Deployment with Ansible [email protected] @BrendelConsult How does it work?

    Server Your laptop Python module: “install apache” Run module Delete module on server
  9. Automated Deployment with Ansible [email protected] @BrendelConsult Inventory and groups Define

    hosts, organized in groups • by function • by location • by hosting provider • ...
  10. Automated Deployment with Ansible [email protected] @BrendelConsult Inventory and groups Define

    hosts, organized in groups [europe] server1.somehoster.co.uk server2.otherhoster.de [north­america] host­a.serverhost.com host­b.serverhost.com [frontend] server1.somehoster.co.uk host­b.serverhost.com [backend] server2.otherhoster.de host­a.serverhost.com
  11. Automated Deployment with Ansible [email protected] @BrendelConsult Adhoc commands Single commands,

    applied to groups $ ansible ­i hosts europe ­a “uname ­a” $ ansible ­i hosts frontend ­a “/sbin/reboot” ­f 3
  12. Automated Deployment with Ansible [email protected] @BrendelConsult Playbooks ­­­ ­ hosts:

    frontend sudo: yes tasks: ­ name: Update the system apt: pkg=nginx state=latest ­ name: Create the user account user: name=appuser shell=/bin/bash state=present ­ name: Copy files to remote user's home copy: > src=files/names.txt dst=/home/appuser owner=appuser mode=0644
  13. Automated Deployment with Ansible [email protected] @BrendelConsult Variables ­­­ ­ hosts:

    all sudo: yes vars: username: appuser tasks: ­ name: Create the user account user: > name={{ username }} shell=/bin/bash state=present
  14. Automated Deployment with Ansible [email protected] @BrendelConsult Project layout 1 /

    my_hosts group_vars/ all frontend backend europe north­america site.yml
  15. Automated Deployment with Ansible [email protected] @BrendelConsult Project layout 2 /

    ansible.cfg deploy_hosts staging_hosts group_vars/ all frontend backend europe north­america host_vars/ server1.somehoster.co.uk host­b.serverhost.com site.yml roles/ common/ tasks/ main.yml handlers/ main.yml templates/ sshd_config.j2 files/ my_script.sh vars/ main.yml web/ ... db/ ...
  16. Automated Deployment with Ansible [email protected] @BrendelConsult Dev, Test, Deploy Why

    didn't we catch this bug in testing? How do I setup my development environment? We can't reproduce the issues. I don't have access to our test server. “It works for me...” (shrug)
  17. Automated Deployment with Ansible [email protected] @BrendelConsult Dev, Test, Deploy Why

    didn't we catch this bug in testing? How do I setup my development environment? We can't reproduce the issues. I don't have access to our test server. “It works for me...” (shrug) Wouldn't this be nice instead? Single command: Dev environment created Single command: Test environment created
  18. Automated Deployment with Ansible [email protected] @BrendelConsult Vagrant • Use Vagrant

    to spin-up VMs • local (VirtualBox, VMware, etc.) • cloud (EC2) • Use Ansible as 'provisioner' • Make an inventory file with just your VM • Point at same playbook as before
  19. Automated Deployment with Ansible [email protected] @BrendelConsult The rise and rise

    of APIs APIs Local Infrastructure Services This is really cool!
  20. Automated Deployment with Ansible [email protected] @BrendelConsult Ansible 'cloud' modules Public

    cloud • Amazon AWS • Google Compute • Azure • Digital Ocean • Rackspace • Linode Public cloud • Amazon AWS • Google Compute • Azure • Digital Ocean • Rackspace • Linode Private cloud • OpenStack • Eucalyptus • Vsphere • Docker • libvirt Private cloud • OpenStack • Eucalyptus • Vsphere • Docker • libvirt
  21. Automated Deployment with Ansible [email protected] @BrendelConsult Example: AWS modules EC2

    / infrastructure • Instances • Images • VPCs • Load balancers Services • S3 • Route 53 • Databases • Cache
  22. Automated Deployment with Ansible [email protected] @BrendelConsult Example: Create instances #

    My hostfile: Empty groups [frontend] [backend] [my­server­group]
  23. Automated Deployment with Ansible [email protected] @BrendelConsult Example: Create instances ­

    local_action: module: ec2 key_name: my­key group: my­security­group instance_type: t2.micro image: ami­120abc90 region: us­east­1 count: 3 register: ec2results
  24. Automated Deployment with Ansible [email protected] @BrendelConsult Example: Create instances ­

    local_action: module: ec2 key_name: my­key group: my­security­group instance_type: t2.micro image: ami­120abc90 region: us­east­1 count: 3 register: ec2results { "ec2results": { "changed": true, "instance_ids": [ "i­da9fbbe4", ... ], "instances": [ { "id": "i­da9fbbe4", "instance_type": "t1.micro", "image_id": "ami­0729b73d", "region": "us­east­1", "public_ip": "54.253.65.41", "private_ip": "10.248.39.121", ... }, ...
  25. Automated Deployment with Ansible [email protected] @BrendelConsult Example: Create instances ­

    local_action: module: ec2 key_name: my­key group: my­security­group instance_type: t2.micro image: ami­120abc90 region: us­east­1 count: 3 register: ec2results ­ local_action: module: add_host hostname: {{ item.public_ip }} groupname: my­server­group with_items: ec2results.instances
  26. Automated Deployment with Ansible [email protected] @BrendelConsult Contributing • The perfect

    FOSS project? • Lots of small, independent modules • Written in Python For yourself: Any language, even bash. For contributions: Python preferred
  27. Automated Deployment with Ansible [email protected] @BrendelConsult Getting started $ git

    clone [email protected]:ansible/ansible.git $ source ansible/hacking/env­setup … edit your module … $ ansible/hacking/test­module ­m ./my_module
  28. Automated Deployment with Ansible [email protected] @BrendelConsult Sample module #!/usr/bin/python import

    datetime import json date = str(datetime.datetime.now()) print json.dumps({ “changed” : True, "time" : date }) Catch exceptions All output to stdout JSON
  29. Automated Deployment with Ansible [email protected] @BrendelConsult Sample module #!/usr/bin/python DOCUMENTATION

    = ''' ­­­ module: filewrite short_description: write data to a file description: ­ Write data to start of file. Create if necessary. options: path: description: ­ The full path of the file/object to write to required: true default: null data: description: ­ The data to be written required: false default: some test data '''
  30. Automated Deployment with Ansible [email protected] @BrendelConsult Sample module import os

    from ansible.module_utils.basic import * def main(): module = AnsibleModule( argument_spec = dict( path = dict(required=True), data = dict(default='some test', type='str'), ), ) path = module.params.get('path') data = module.params.get('data') try: if os.path.isfile(path): with open(path, "r") as f: d = f.read(len(data)) if d == data: module.exit_json(changed=False, other_data="Not changed.") with open(path, "w") as f: f.write(data) module.exit_json(changed=True, other_data="Wrote %s characters." % len(data)) except Exception as e: module.fail_json(msg=str(e)) main()
  31. Automated Deployment with Ansible [email protected] @BrendelConsult Thank you very much!

    Questions: [email protected] Ansible docs: http://docs.ansible.com/ Ansible source: https://github.com/ansible/ansible Vagrant: http://www.vagrantup.com/ Example project: https://github.com/jbrendel/ansible_cluster_setup