Share
Embed
CodeMesh 2015 : Boot my (secure)->(portable) clouds!
Slides to accompany my talk done at Code Mesh 2015. http://www.codemesh.io/codemesh2015/nicki-watt.
Associated video can be found here: https://www.youtube.com/watch?v=bRZSftRVoAQ
Nicki Watt
November 04, 2015
Transcript
105 slides
-
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2 About Me • Hands on Lead consultant
at OpenCredo • Co-author Neo4j In Action • Currently working with UK gov dept on cloud automation project • Twitter: @techiewatt -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 3 Agenda • What is the problem •
What are the options • How: Principles, challenges, lessons, tools • Conclusion -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 11 • Public cloud workloads • Dev teams
anywhere in the world • No human intervention Initial focus: -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 13 Input: Environment prefix: team1 Number Jenkins Slaves:
3 Environment domain suffix: t1tools.domain.io Initial SSH keys: AAAEFF user1@team1 Cloud: AWS -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 20 • Automate everything • Separate config from
code • API driven clouds & tools • Prefer modular, open source tools ASAP -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 30 Availability Zone #1 Management subnet 10.0.0.32/24 Tooling
subnet 10.0.0.64/27 DNS VPN XXX Jenkins Master Jira Jenkins Slave-n Artifac tory Router Internet Gateway region DMZ subnet 10.0.0.0/27 • Networks • Firewall Rules • Routers • Compute Resources • Public / Floating IP Addresses -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 34 Availability Zone #1 Management subnet 10.0.0.32/24 Tooling
subnet 10.0.0.64/27 DNS VPN XXX Jenkins Master Jira Jenkins Slave-n Artifac tory Router Internet Gateway region DMZ subnet 10.0.0.0/27 Multiple Cloud Providers -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 35 Declarative DSL ## OpenVPN Compute instance resource
"openstack_compute_instance_v2" "ovpn" { name = "${var.env-prefix}-ovpn" image_name = "${var.image_name}" flavor_name = "${var.openvpn-flavour-name}" floating_ip = “${openstack_compute_floatingip_v2. openvpn.address}" ... } ## OpenVPN Public IP resource "openstack_compute_floatingip_v2" "openvpn" { region = "" pool = "${var.public-ip-pool}" ... } terraform.tf -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 36 ## OpenVPN Compute instance resource "openstack_compute_instance_v2" "ovpn"
{ name = "${var.env-prefix}-ovpn" image_name = "${var.image_name}" flavor_name = "${var.openvpn-flavour-name}" floating_ip = “${openstack_compute_floatingip_v2. openvpn.address}" ... } ## OpenVPN Public IP resource "openstack_compute_floatingip_v2" "openvpn" { region = "" pool = "${var.public-ip-pool}" ... } terraform.tf Declarative DSL (OpenStack) -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 37 ## OpenVPN Compute instance resource "openstack_compute_instance_v2" "ovpn"
{ name = "${var.env-prefix}-ovpn" image_name = "${var.image_name}" flavor_name = "${var.openvpn-flavour-name}" floating_ip = “${openstack_compute_floatingip_v2. openvpn.address}" ... } ## OpenVPN Public IP resource "openstack_compute_floatingip_v2" "openvpn" { region = "" pool = "${var.public-ip-pool}" ... } terraform.tf Declarative DSL (OpenStack) -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 38 ## OpenVPN Compute instance resource "openstack_compute_instance_v2" "ovpn"
{ name = "${var.env-prefix}-ovpn" image_name = "${var.image_name}" flavor_name = "${var.openvpn-flavour-name}" floating_ip = “${openstack_compute_floatingip_v2. openvpn.address}" ... } ## OpenVPN Public IP resource "openstack_compute_floatingip_v2" "openvpn" { region = "" pool = "${var.public-ip-pool}" ... } terraform.tf Declarative DSL (OpenStack) -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 39 ## OpenVPN Compute instance resource "openstack_compute_instance_v2" "ovpn"
{ name = "${var.env-prefix}-ovpn" image_name = "${var.image_name}" flavor_name = "${var.openvpn-flavour-name}" floating_ip = “${openstack_compute_floatingip_v2. openvpn.address}" ... } ## OpenVPN Public IP resource "openstack_compute_floatingip_v2" "openvpn" { region = "" pool = "${var.public-ip-pool}" ... } terraform.tf Declarative DSL (OpenStack) -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 40 Declarative DSL (AWS) ## OpenVPN Compute instance
resource "aws_instance" "ovpn" { ami = "${var.ami_name}" instance_type = "${var.openvpn-instance-type}" vpc_security_group_ids = [ "${aws_security_group.ovpn.id}"] subnet_id = “${aws_subnet.dmz.id}" ... } ## DMZ network exposing Public IP resource "aws_subnet" "dmz" { vpc_id = "${aws_vpc.core.id}" cidr_block = "${var.dmz-net-cidr}" map_public_ip_on_launch = 1 ... } terraform.tf -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 41 Availability Zone #1 Management subnet 10.0.0.32/24 Tooling
subnet 10.0.0.64/27 DNS VPN XXX Jenkins Master Jira Jenkins Slave-n Artifac tory Router Internet Gateway region DMZ subnet 10.0.0.0/27 “Other” Infrastructure Providers -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 42 External DNS Availability Zone #1 Management subnet
10.0.0.32/24 Tooling subnet 10.0.0.64/27 DNS VPN XXX Jenkins Master Jira Jenkins Slave-n Artifac tory Router Internet Gateway region DMZ subnet 10.0.0.0/27 “Other” Infrastructure Providers -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 43 External DNS Availability Zone #1 Management subnet
10.0.0.32/24 Tooling subnet 10.0.0.64/27 DNS VPN XXX Jenkins Master Jira Jenkins Slave-n Artifac tory Router Internet Gateway region DMZ subnet 10.0.0.0/27 “Other” Infrastructure Providers -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 44 External DNS Availability Zone #1 Management subnet
10.0.0.32/24 Tooling subnet 10.0.0.64/27 DNS VPN XXX Jenkins Master Jira Jenkins Slave-n Artifac tory Router Internet Gateway region DMZ subnet 10.0.0.0/27 Simultaneous multi-cloud config -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 45 ## OpenVPN Compute instance resource "openstack_compute_instance_v2" "ovpn"
{...} ## OpenVPN Public IP resource "openstack_compute_floatingip_v2" "openvpn" {...} # External DNS (AWS Route 53) resource "aws_route53_record" "openvpn" { zone_id = "${var.route_53_domain_id}" name = "${var.env-prefix}-ovpn.ext.t1tools.domain.io" type = "A" ttl = "60" records = [“${openstack_compute_floatingip_v2. openvpn.address}"] } terraform.tf Declarative DSL -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 46 ## OpenVPN Compute instance resource "openstack_compute_instance_v2" "ovpn"
{ name = "${var.env-prefix}-ovpn" image_name = "${var.image_name}" flavor_name = "${var.openvpn-flavour-name}" floating_ip = “${openstack_compute_floatingip_v2. openvpn.address}" ... } variable "env-prefix" { default = "team1" } variable "image-name" { default = "centos-7-001" } variable "openvpn-flavour-name" { default = "x1.medium" } terraform.tf terraform.tfvars Vars & inter resource refs -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 47 resource "openstack_compute_instance_v2" "ovpn" { floating_ip = “${openstack_compute_floatingip_v2.
openvpn.address}" ... } resource "openstack_compute_floatingip_v2" "openvpn" { pool = "${var.public-ip-pool}" ... } # External DNS (AWS Route 53) resource "aws_route53_record" "openvpn" { name = "${var.env-prefix}-ovpn.ext.t1tools.domain.io" records = [“${openstack_compute_floatingip_v2. openvpn.address}"] } terraform.tf Vars & inter resource refs -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 48 resource "openstack_compute_instance_v2" "ovpn" { floating_ip = “${openstack_compute_floatingip_v2.
openvpn.address}" ... } resource "openstack_compute_floatingip_v2" "openvpn" { pool = "${var.public-ip-pool}" ... } # External DNS (AWS Route 53) resource "aws_route53_record" "openvpn" { name = "${var.env-prefix}-ovpn.ext.t1tools.domain.io" records = [“${openstack_compute_floatingip_v2. openvpn.address}"] } 1 terraform.tf Vars & inter resource refs -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 49 resource "openstack_compute_instance_v2" "ovpn" { floating_ip = “${openstack_compute_floatingip_v2.
openvpn.address}" ... } resource "openstack_compute_floatingip_v2" "openvpn" { pool = "${var.public-ip-pool}" ... } # External DNS (AWS Route 53) resource "aws_route53_record" "openvpn" { name = "${var.env-prefix}-ovpn.ext.t1tools.domain.io" records = [“${openstack_compute_floatingip_v2. openvpn.address}"] } 1 2a 2b terraform.tf Vars & inter resource refs -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 50 ## Jenkins Slave Compute instance resource "openstack_compute_instance_v2"
"js" { count = ${var.jenslave_count} region = "" name = "${var.env-prefix}-js${count.index}” image_name = "${var.image_name}" flavor_name = "${var.jenslave-flavour-name}" ... } terraform.tf Resource Scaling -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 52 + module.mgt.openstack_compute_instance_v2.openvpn name: "" => "team1-ovpn" image_name:
"" => "centos-7-001" floating_ip: "" => "<computed>" flavour_name: "" => "x1.medium" user_data: "" => "b50c352a2e23ceba60e1b10fd3e2" . . . -/+ module.mgt.openstack_compute_instance_v2.ipa name: “team1-ipa" => “team1-ipa" image_name: "centos-7-001" => "centos-7-001" floating_ip: “" => "<computed>" flavour_name: "x1.medium" => “x1.large” (forces new resource) . . . Terraform plan -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 53 • Adhered to all our ASAP principles
• Handle multiple infrastructure providers • Terraform still evolving, not perfect • Other potential alternatives • Tosca/Cloudify IaaS provisioning Summary -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 54 Interlude … consistent cloud images ## OpenVPN
Compute instance resource "openstack_compute_instance_v2" "ovpn" { name = "${var.env-prefix}-ovpn" image_name = "${var.image_name}" flavor_name = "${var.openvpn-flavour-name}" floating_ip = “${openstack_compute_floatingip_v2. openvpn.address}" ... } terraform.tf -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 55 Interlude … consistent cloud images ## OpenVPN
Compute instance resource "openstack_compute_instance_v2" "ovpn" { name = "${var.env-prefix}-ovpn" image_name = "${var.image_name}" flavor_name = "${var.openvpn-flavour-name}" floating_ip = “${openstack_compute_floatingip_v2. openvpn.address}" ... } terraform.tf -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- cloud provider management subnet tooling subnet 10.20.251.36 public
IP Terraform complete … External DNS Provider t1team- ovpn.ext. t1tools. domain.io -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- cloud provider cloud provider metadata service management subnet
tooling subnet 10.20.251.36 Terraform complete … cloud-init takes over public IP -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 64 Accesses user supplied data for VM it
is running on cloud provider metadata service #cloud-config hostname: ${env-prefix}-jm fqdn: ${env-prefix}-jm.${domain} manage_etc_hosts: true puppet: conf: agent: server: "${env-prefix}-ipa.$ {domain}" runcmd: -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 65 Example (user-data) cloud config #cloud-config hostname: ${env-prefix}-jm
fqdn: ${env-prefix}-jm.${domain} manage_etc_hosts: true puppet: conf: agent: server: "${env-prefix}-ipa.${domain}" runcmd: - until curl -ksf https://${env-prefix}-ipa.${domain}:443/ca/ admin/ca/getStatus ; do sleep 30 ; done ; ipa-client-install — domain=${domain} ... --unattended --force-join - export COUNT=0 ; until puppet agent -t ; do echo "`date` - Attempting to run puppet agent for $COUNT time" ; if [[ $COUNT -eq 3 ]] ; then break ; fi ; sleep 30 ; ((COUNT++)) ; done -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 66 #cloud-config hostname: ${env-prefix}-jm fqdn: ${env-prefix}-jm.${domain} manage_etc_hosts: true
puppet: conf: agent: server: "${env-prefix}-ipa.${domain}" runcmd: - until curl -ksf https://${env-prefix}-ipa.${domain}:443/ca/ admin/ca/getStatus ; do sleep 30 ; done ; ipa-client-install — domain=${domain} ... --unattended --force-join - export COUNT=0 ; until puppet agent -t ; do echo "`date` - Attempting to run puppet agent for $COUNT time" ; if [[ $COUNT -eq 3 ]] ; then break ; fi ; sleep 30 ; ((COUNT++)) ; done Example (user-data) cloud config -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 67 #cloud-config hostname: ${env-prefix}-jm fqdn: ${env-prefix}-jm.${domain} manage_etc_hosts: true
puppet: conf: agent: server: "${env-prefix}-ipa.${domain}" runcmd: - until curl -ksf https://${env-prefix}-ipa.${domain}:443/ca/ admin/ca/getStatus ; do sleep 30 ; done ; ipa-client-install — domain=${domain} ... --unattended --force-join - export COUNT=0 ; until puppet agent -t ; do echo "`date` - Attempting to run puppet agent for $COUNT time" ; if [[ $COUNT -eq 3 ]] ; then break ; fi ; sleep 30 ; ((COUNT++)) ; done Example (user-data) cloud config -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 68 #cloud-config hostname: ${env-prefix}-jm fqdn: ${env-prefix}-jm.${domain} manage_etc_hosts: true
puppet: conf: agent: server: "${env-prefix}-ipa.${domain}" runcmd: - until curl -ksf https://${env-prefix}-ipa.${domain}:443/ca/ admin/ca/getStatus ; do sleep 30 ; done ; ipa-client-install — domain=${domain} ... --unattended --force-join - export COUNT=0 ; until puppet agent -t ; do echo "`date` - Attempting to run puppet agent for $COUNT time" ; if [[ $COUNT -eq 3 ]] ; then break ; fi ; sleep 30 ; ((COUNT++)) ; done Example (user-data) cloud config -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 69 #cloud-config hostname: ${env-prefix}-jm fqdn: ${env-prefix}-jm.${domain} manage_etc_hosts: true
puppet: conf: agent: server: "${env-prefix}-ipa.${domain}" runcmd: - until curl -ksf https://${env-prefix}-ipa.${domain}:443/ca/ admin/ca/getStatus ; do sleep 30 ; done ; ipa-client-install — domain=${domain} ... --unattended --force-join - export COUNT=0 ; until puppet agent -t ; do echo "`date` - Attempting to run puppet agent for $COUNT time" ; if [[ $COUNT -eq 3 ]] ; then break ; fi ; sleep 30 ; ((COUNT++)) ; done Example (user-data) cloud config -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ## OpenVPN Compute instance resource "openstack_compute_instance_v2" "jm" {
name = "${var.env-prefix}-jm" image_name = "${var.image_name}" flavor_name = "${var.jm-flavour-name}" user_data = "${template_file.clientconfig.rendered}" ... } ## UserData as input to cloud-init resource "template_file" "clientconfig" { filename = "${path.module}/clientconfig.template" vars { domain = "${var.domain}" env-prefix = "${var.env-prefix}" ... } } 70 Passing user-data via terraform -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 71 ## OpenVPN Compute instance resource "openstack_compute_instance_v2" "jm"
{ name = "${var.env-prefix}-jm" image_name = "${var.image_name}" flavor_name = "${var.jm-flavour-name}" user_data = "${template_file.clientconfig.rendered}" ... } ## UserData as input to cloud-init resource "template_file" "clientconfig" { filename = "${path.module}/clientconfig.template" vars { domain = "${var.domain}" env-prefix = "${var.env-prefix}" ... } } Passing user-data via terraform -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- cloud provider cloud provider metadata service management subnet
tooling subnet 10.20.251.36 Terraform complete … cloud-init takes over public IP -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- cloud provider cloud provider metadata service management subnet
tooling subnet 10.20.251.36 orchestration box: uses supplied token to connect to vault, download keys (github etc) -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- cloud provider cloud provider metadata service management subnet
tooling subnet 10.20.251.36 orchestration box: uses supplied token to connect to vault, download keys (github etc) -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- cloud provider cloud provider metadata service management subnet
tooling subnet 10.20.251.36 orchestration box: downloads puppet source code -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- cloud provider cloud provider metadata service team1-ipa. t1tools.domain.io
management subnet tooling subnet orchestration box: initiates configuration on itself, installs and configures FreeIPA (DNS) -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- cloud provider cloud provider metadata service team1-ovpn. t1tools.domain.io
team1-ipa. t1tools.domain.io team1-jm. t1tools. domain.io team1-js1. t1tools. domain.io team1-js2. t1tools. domain.io team1-jira. t1tools. domain.io team1-elk. t1tools.domain.io management subnet tooling subnet all other boxes: waiting for DNS service to be available, then register & obtain DNS names -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- cloud provider cloud provider metadata service team1-ovpn. t1tools.domain.io
team1-ipa. t1tools.domain.io team1-jm. t1tools. domain.io team1-js1. t1tools. domain.io team1-js2. t1tools. domain.io team1-jira. t1tools. domain.io team1-elk. t1tools.domain.io management subnet tooling subnet all other boxes: Initiate bootstrap puppet run -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 80 Automated Config Mgt Summary • Adhered to
all our ASAP principles • Async bootstrap process • Swapped Ansible and Puppet -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 84 • Secrets Management • For IaaS provisioning
• For Config management • For anything needing access to sensitive stuff … -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 88 • Unified API to access multiple backends
• ACL policies - who can access what • Audit Logs • And more … -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 92 $ vault init -key-shares=3 -key-threshold=2 Key 1:
8573410dd211cc9b5ea6b426b19d6d668e0184c39d4ba Key 2: 3fd762583cc9755222fa20f2a78770aca5ecba3b16d8c Key 3: a0428a6b6681eb15ffcea5be5c787beabcb7599a6fa91 Initial Root Token: 57dfce17-08c4-d042-91a3-68082965367b Vault initialized with 3 keys and a key threshold of 2. Please securely distribute the above keys. When the Vault is re-sealed, restarted, or stopped, you must provide at least 2 of these keys to unseal it again. Vault does not store the master key. Without at least 2 keys, your Vault will remain permanently sealed. $ vault unseal 8573410dd211cc9b5ea6b426b19d6d668e0184c39d4ba Sealed: true Key Shares: 3 Key Threshold: 2 Unseal Progress: 1 $ vault unseal a0428a6b6681eb15ffcea5be5c787beabcb7599a6fa91 Sealed: false Key Shares: 3 Key Threshold: 2 Unseal Progress: 0 Vault init & unseal -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- $ vault unseal 8573410dd211cc9b5ea6b426b19d6d668e0184c39d4ba Sealed: true Key Shares:
3 Key Threshold: 2 Unseal Progress: 1 $ vault unseal a0428a6b6681eb15ffcea5be5c787beabcb7599a6fa91 Sealed: false Key Shares: 3 Key Threshold: 2 Unseal Progress: 0 $ vault init -key-shares=3 -key-threshold=2 Key 1: 8573410dd211cc9b5ea6b426b19d6d668e0184c39d4ba Key 2: 3fd762583cc9755222fa20f2a78770aca5ecba3b16d8c Key 3: a0428a6b6681eb15ffcea5be5c787beabcb7599a6fa91 Initial Root Token: 57dfce17-08c4-d042-91a3-68082965367b Vault initialized with 3 keys and a key threshold of 2. Please securely distribute the above keys. When the Vault is re-sealed, restarted, or stopped, you must provide at least 2 of these keys to unseal it again. Vault does not store the master key. Without at least 2 keys, your Vault will remain permanently sealed. 93 Vault init & unseal -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- $ vault init -key-shares=3 -key-threshold=2 Key 1: 8573410dd211cc9b5ea6b426b19d6d668e0184c39d4ba
Key 2: 3fd762583cc9755222fa20f2a78770aca5ecba3b16d8c Key 3: a0428a6b6681eb15ffcea5be5c787beabcb7599a6fa91 Initial Root Token: 57dfce17-08c4-d042-91a3-68082965367b Vault initialized with 3 keys and a key threshold of 2. Please securely distribute the above keys. When the Vault is re-sealed, restarted, or stopped, you must provide at least 2 of these keys to unseal it again. Vault does not store the master key. Without at least 2 keys, your Vault will remain permanently sealed. $ vault unseal 8573410dd211cc9b5ea6b426b19d6d668e0184c39d4ba Sealed: true Key Shares: 3 Key Threshold: 2 Unseal Progress: 1 $ vault unseal a0428a6b6681eb15ffcea5be5c787beabcb7599a6fa91 Sealed: false Key Shares: 3 Key Threshold: 2 Unseal Progress: 0 94 Vault init & unseal -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Unseal Create env specific mount, add secrets Use
token as input for initial bootstrap Init Acquire policy constrained token -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 96 $ vault mount -path=team1 generic Successfully mounted
'generic' at ‘team1'! $ vault mounts Path Type Default TTL Max TTL Description cubbyhole/ cubbyhole n/a n/a per-token private secr ... secret/ generic system system generic secret storage sys/ system n/a n/a system endpoints used f... team1/ generic system system Vault create new mount -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 97 Vault write, then read back secret $
vault write team1/cloud-provider-password value=ASDKJ234SF*2 Success! Data written to: team1/cloud-provider-password $ vault read team1/cloud-provider-password Key Value lease_duration 2592000 value ASDKJ234SF*2 -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Unseal Use token as input for initial bootstrap
Init Create env specific mount, add secrets Acquire policy constrained token -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 99 $ cat team1-bootstrap.policy path "team1/*" { policy
= "read" } path "auth/token/lookup-self" { policy = "read" } $ vault policy-write team1-puppet team1-bootstrap.policy Policy 'team1-bootstrap' written. Vault create custom policy -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 100 $ vault token-create -policy="team1-bootstrap" Key Value token
4a1cd43a-9206-9f48-caf8-b05e05457271 token_duration 2592000 token_renewable true token_policies [team1-bootstrap] $ vault auth 4a1cd43a-9206-9f48-caf8-b05e05457271 Successfully authenticated! token: 4a1cd43a-9206-9f48-caf8-b05e05457271 token_duration: 2592000 token_policies: [team1-bootstrap] $ vault read team1/cloud-provider-password Key Value lease_duration 2592000 value ASDKJ234SF*2 $ vault write team1/somekey value=somevalue Error writing data to team1/somekey: Error making API request. URL: PUT http://127.0.0.1:8200/v1/team1/somekey Code: 403. Errors: * permission denied Vault validate custom policy -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 102 • Scraped surface • Stuck to ASAP
principles • Centralised Secrets management very handy • Try not to roll your own! Security Summary -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 104 Lesson #4 Baby steps … Initial approach
focuses on more DR (recreate) type scenarios, rather than realtime data migrations -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 106 Conclusion #4 • This approach is limited
• Doesn’t handle realtime migrations • Still work in progress