Immutable AWS Deployments with Packer and Jenkins

In this session I will talk about Immutable Deployments - which have become almost essential in the world of Microservices. As the frequency of deployments across multiple services increases with increasing granularity, it is critical to have repeatable, predictable and immutable deployments serving our customers. In practice, this is achieved via several DevOps tools. We will use Hashicorp Packer (packer.io) and Jenkins to build a simple, immutable AWS deployment of a hello-world microservice. Familiarity with AWS is recommended but not required for this talk.

Transcript

1. Immutable AWS Deployments with Packer and Jenkins Scale by the Bay, 2017 Manish Pandit
2. About Manish Pandit Director of Platform Engineering @Marqeta Twitter : @lobster1234 Blog: lobster1234.github.io
3. Show of hands Deployments DevOps, CI/CD, Tooling,.. AWS (or something similar!)
4. Deployments The process of pushing code beyond the development environment Multi-step Usually (heavily) scripted Complete, or Partial
5. Complete Provisioning the stack from ground up Installation of O/S, Runtime, Application Server, Code, Agents, ... # mkfs –t ext4 /dev/sda0 # mkdir /apps # mount –t ext4 /dev/sda0 /apps
6. Partial Very common across the board In-place deployment “Saves time” Gives a false sense of automation
7. Typical Steps ~ scp builds/myService-1.0.2.war mpandit@prd.example.com:/usr/local/tomcat8/webapps ~ ssh mpandit@prd.example.com ~ sudo /usr/local/tomcat8/bin/catalina.sh restart
8. Typical Steps ~ ssh mpandit@prd.example.com ~ wget https://nx.example.com/com/foo/myService.war -O /usr/local/tomcat8/webapps ~ sudo /usr/local/tomcat8/catalina.sh restart
9. Automation? Script this all Run via a Jenkins job Fabric (Python), Capistrano, etc.
10. What could go wrong?
11. What could go wrong? Unpatched, outdated dependencies Inconsistent app behavior Changes outside of the deployment cycle Human Error(s) Does not scale
12. Immutability Build the entire runtime infrastructure from ground up Automate it!
13. Immutability Build the entire runtime infrastructure from ground up Automate it! Runtime Infrastructure = O/S + Libraries + App Server + Code + Agents
14. AWS AWS is collection of services for.. Compute Storage Databases Messaging + many, many more...
15. AWS AWS helps build architectures that are - Highly Available Fault Tolerant Scalable Cost-efficient
16. AMIs Templates to launch EC2 instances Specify O/S, Virtualization Type, Storage Type, Volume Attachments, etc. Can be shared within accounts, or made public Highest level of deployment abstraction
18. Customize AMIs Trim the fat Configure the libraries, tune the parameters Summary : Make infrastructure, not war* * Java Reference
19. Packer A tool from Hashicorp to create Machine Images Supports multiple providers Supports multiple provisioners
20. Install ~ packer -v 1.1.1 ~ Install via brew, or, Download the binary from the packer.io website
21. Credentials EC2 = Use IAM Role for Packer * Non-EC2 = Use AWS Credentials * Packer website has the IAM role details
22. Builders Define Machine Images for many platforms JSON-based Popular : AWS AMI, VMWare, Docker, Azure, GCP… Custom
23. AWS AMI Builder { "_comment":"Simple Packer Template using Amazon Linux 2017.09.0", "variables":{ "aws_access_key":"", "aws_secret_key":"" }, "builders":[ { "type":"amazon-ebs", "access_key":"{{user `aws_access_key`}}", "secret_key":"{{user `aws_secret_key`}}", "region":"us-east-1", "source_ami":"ami-8c1be5f6", "instance_type":"t2.micro", "ssh_username":"ec2-user", "ami_name":"ScaleByTheBay AMI" } ] }
24. Inspect ~ packer inspect packer.json Optional variables and their defaults: aws_access_key = aws_secret_key = Builders: amazon-ebs Provisioners: <No provisioners> Note: If your build names contain user variables or template functions such as 'timestamp', these are processed at build time, and therefore only show in their raw form here.
25. Build! ~ packer build packer.json amazon-ebs output will be in this color. ==> amazon-ebs: Prevalidating AMI Name: ScaleByTheBay AMI amazon-ebs: Found Image ID: ami-8c1be5f6 ==> amazon-ebs: Launching a source AWS instance... ==> amazon-ebs: Waiting for instance (i-09f4b837ed80a659f) to become ready... ==> amazon-ebs: Waiting for SSH to become available... ==> amazon-ebs: Stopping the source instance... ==> amazon-ebs: Creating the AMI: ScaleByTheBay AMI amazon-ebs: AMI: ami-5b18a121 ==> amazon-ebs: Waiting for AMI to become ready... ==> amazon-ebs: Terminating the source AWS instance... ==> Builds finished. The artifacts of successful builds are: --> amazon-ebs: AMIs were created: us-east-1: ami-5b18a121
28. Provisioners JSON based Install and configure packages and components +many, many more tasks Popular : Ansible, Chef, Puppet, Shell, ..
29. Make our AMI ...useful 1. Apply updates and patches 2. Install OpenJDK 8 3. Install Tomcat 8 4. Download the application artifact, the war 5. Configure Tomcat to run at startup
30. Let’s Provision our AMI "provisioners": [{ "type": "shell", "inline": [ "sudo yum update -y", "sudo yum install java-1.8.0 java-1.8.0-openjdk-devel tomcat8-webapps -y", "sudo yum remove java-1.7.0-openjdk -y", "sudo wget https://github.com/lobster1234/helloworld-api/files/953511/helloworld-api.war.gz -O /usr/share/tomcat8/webapps/helloworld-api.war.gz", "sudo gunzip /usr/share/tomcat8/webapps/helloworld-api.war.gz", "sudo chkconfig tomcat8 on" ] }]
31. { "_comment":"Simple Packer Template using Amazon Linux 2017.09.0", "variables":{ "aws_access_key":"", "aws_secret_key":"" }, "builders":[ { "type":"amazon-ebs", "access_key":"{{user `aws_access_key`}}", "secret_key":"{{user `aws_secret_key`}}", "region":"us-east-1", "source_ami":"ami-8c1be5f6", "instance_type":"t2.micro", "ssh_username":"ec2-user", "ami_name":"ScaleByTheBay AMI with Tomcat8" } ], "provisioners": [{ "type": "shell", "inline": [ "sleep 30", "sudo yum update -y", "sudo yum install java-1.8.0 java-1.8.0-openjdk-devel tomcat8-webapps -y", "sudo yum remove java-1.7.0-openjdk -y", "sudo wget https://github.com/lobster1234/helloworld-api/files/953511/helloworld-api.war.gz -O /usr/share/tomcat8/webapps/helloworld- api.war.gz", "sudo gunzip /usr/share/tomcat8/webapps/helloworld-api.war.gz", "sudo chkconfig tomcat8 on" ] }] }
32. Build! ~ packer build packer.json .... ==> amazon-ebs: Connected to SSH! ==> amazon-ebs: Provisioning with shell script: /var/folders/vf/d0q4kjg964581kjjz4969dbny407x7/T/packer-shell539435218 amazon-ebs: Loaded plugins: priorities, update-motd, upgrade-helper amazon-ebs: Resolving Dependencies amazon-ebs: --> Running transaction check amazon-ebs: ---> Package amazon-ssm-agent.x86_64 0:2.1.4.0-1.amzn1 will be updated amazon-ebs: amazon-ebs: 2017-11-11 07:51:33 (64.0 MB/s) - ‘/usr/share/tomcat8/webapps/helloworld-api.war.gz’ saved [1918559/1918559] amazon-ebs: ==> amazon-ebs: Creating the AMI: ScaleByTheBay AMI with Tomcat8 amazon-ebs: AMI: ami-73ed5509 ==> amazon-ebs: Waiting for AMI to become ready... Build 'amazon-ebs' finished. ==> Builds finished. The artifacts of successful builds are: --> amazon-ebs: AMIs were created: us-east-1: ami-73ed5509
34. Launch the instance
35. Check it out
36. Verify Tomcat
37. Verify the API ~ curl -iv http://ec2-54-88-249-121.compute-1.amazonaws.com:8080/helloworld-api/hello * Trying 54.88.249.121... * TCP_NODELAY set * Connected to ec2-54-88-249-121.compute-1.amazonaws.com (54.88.249.121) port 8080 (#0) > GET /helloworld-api/hello HTTP/1.1 > Host: ec2-54-88-249-121.compute-1.amazonaws.com:8080 > User-Agent: curl/7.54.0 > Accept: */* > < HTTP/1.1 200 HTTP/1.1 200 < Content-Type: text/html;charset=utf-8 < Transfer-Encoding: chunked < Date: Sat, 11 Nov 2017 08:20:09 GMT < * Connection #0 to host ec2-54-88-249-121.compute-1.amazonaws.com left intact Hello World! ~
38. Automate this - Jenkins 1. git clone <repo> 2. mvn clean install test 3. mvn release:prepare release:perform 4. export version=1.0.2 5. packer build packer.json 6. Output this AMI ID to Terraform to launch an Autoscaling Group
39. Summary Do not release code - release runtime infrastructure Automate Everything Legendary = Disable ssh from your AMIs
40. Resources Packer - https://packer.io AWS EC2 - https://aws.amazon.com/documentation/ec2/ My Blog Post - https://tinyurl.com/packer-jenkins
41. Questions Manish Pandit @lobster1234 lobster1234.github.io Like what you saw? Come work with me @Marqeta!