• Developer -> Release Engineer -> Cloud DevOps • Organiser of the Singapore Atlassian User Group https://www.meetup.com/Atlassian-User-Group-Singapore/ • LinkedIn: https://www.linkedin.com/in/steve-teo-b7988541/
Engineering SME & Product Development Team • Product Engineering Team ◦ On-Prem moving to the Cloud ◦ Mixed Windows & Linux Workloads, Manual Deployment Process ◦ Charge-back was a very important consideration ◦ “Oh btw, you are the Infra Lead now”
Good: No need to link anything back to on-prem • Bad: Multiple VPCs with conflicting or too big CIDRs. E.g. 10.0.0.0/16 • Bad: No clear purpose of various AWS accounts • Bad: Undocumented AWS Accounts • Bad: Unclear chargeback / workload purpose • Bad: Abandoned Workloads • Can’t be helped: Vendors had AWS Management IAM Access • Definite Pain Point: Conflicting or too big CIDRs
our infrastructure and sleep at night • Day 2 Concerns ◦ Creating and Securing Each AWS Account ◦ VPC Connectivity ◦ Visibility of resources ◦ All Others ▪ DNS ▪ Logs, Monitoring, Compliance & Audit ▪ Backup, Patch Management • Good to invest engineering effort to strategize and automate this!
new email address (painful process for us) • Creation was a manual process. Painful and boring • Snooped around ◦ https://github.com/intuit/aws_account_utils ◦ AWS Internal API to automate AWS Account Creation • Have a naming convention to your account • If I had to do it again ◦ Use AWS Organisations ◦ Use Email Addresses with the + trick, eg. [email protected]
Authenticators with Authy - to prevent SPOF • If I had to do it again ◦ Use AWS Organisations ◦ “An new account that is created by using AWS Organizations does not initially have a password assigned to the root user.”
◦ STS AssumeRole from Jump Host Account ◦ AWS Managed AD was not existent yet • Our Approach ◦ AD -> Okta -> SAML Federation to every AWS Account ◦ AD was good for us because of Windows workloads ◦ Okta was good for us to manage complexity with multiple services
as a result of AWS Accounts • We ended up using AD DNS to manage internal DNS across all VPCs, with Route53 Zones for public DNS • Considerations ◦ Route53 doesn’t integrate neatly with resources in different AWS accounts
(CPM) ◦ However: Inflexible licensing around multiple AWS Accounts • Patch Management ◦ Consider using AWS EC2 Systems Manager ◦ Or leverage on Configuration Management with Packer / Shared AMIs
Reserved Instances ◦ Wanted selection of reserved instances specifically not against certain accounts • Reserved Instances and Capacity Reservation ◦ RIs in a central account vs RIs in separate accounts ◦ Capacity Reservation only holds if specific account and specific AZs • Resource Duplication ◦ Eg. Multiple NAT Instances - Considering using Proxy Instances • Support for Multiple AWS Accounts? - We were on Enterprise Support, so this did not bite us
important decision first, eg. VPC Segmentation ◦ “A good architecture allows major decisions to be deferred” • Document or CMDB the following ◦ AWS Accounts ◦ VPCs • Naming Strategy for at least ◦ AWS Accounts • Automate AWS Account Configuration - Terraform / CloudFormation • Centralize Everything • Secure Everything ◦ Use AWS Organisations Service Control Policies
multiple AWS Accounts (eg. Licensing, Automation) • Be aware of the nature of AWS resources (eg. Account-Specific, AZ-Specific, Region-Specific, VPC-Specific) • Be aware of limits which can impact the overall architecture ◦ VPC Peering Limits ◦ Software Licenses • Look at AWS Organisations
any AWS Journey; every organisation can benefit from this • Research far and wide, think of scenarios and do POCs • Good to invest engineering effort to strategize and automate the associated challenges