From Infrastructure as Code to Environment as Code – DevOps Days Boston

Infrastructure as Code(IaC) has made managing infrastructure easier in a lot of ways, but there are many challenges that companies accept as the cost of adopting IaC especially when scaling. IaC is good at provisioning individual resources (or a few of them together) but engineering teams want an entire environment with various components like networking, platform (ec2/eks), database, s3 buckets, etc. to deploy and operate their applications.

To provision and tear down an entire environment, these teams have two options. They can either hand roll pipelines to manage individual resources and then manage complex dependencies between these resources within those pipelines or create a monolith IaC for the entire environment. These approaches are inefficient and slow down feature development and innovation. They also make replicating, visualizing & understanding environments difficult. What if there were a better way?

This talk digs into these challenges to try to better understand them and then look at how to resolve them. We will introduce Environment as Code (abstraction over IaC) that enables teams to provision & teardown entire Environments in an efficient way and promotes best practices like loosely coupled infrastructure resources.

Transcript

1. From Infrastructure as Code to Environment as Code Challenges scaling IaC and how to resolve them Adarsh Shah Engineering Leader, Coach, Public Speaker Founder & CEO at zLifecycle @shahadarsh 
 https://zLifecycle.com @shahadarsh https://bit.ly/eac-dod-boston
2. @shahadarsh https://zLifecycle.com https://bit.ly/eac-dod-boston Terminologies Components: A logical grouping of 1 or more Infrastructure Resources or Applications that get provisioned together. For example, Networking is an Infrastructure Component with various Infrastructure resources like Virtual Private Cloud(VPC), Subnets, Internet Gateways, Route Tables, etc. Environment: A logical grouping of all the Components that are needed to run business applications. The grouping includes components like networking, eks, database, k8s apps, etc.
3. @shahadarsh https://zLifecycle.com https://bit.ly/eac-dod-boston Infrastructure as Code Infrastructure as Code (IaC) is an approach that takes proven coding techniques used by software systems and extends it to infrastructure. It is one of the key DevOps practices that enable teams to deliver infrastructure, and thereby software running on it, rapidly and reliably.
4. @shahadarsh https://zLifecycle.com https://bit.ly/eac-dod-boston Evolution of IaC Setup networking + eks + …. Monolith IaC networking eks website postgres networking networking + eks + …. ec2 website eks postgres ec2 Breaking into Separate & Smaller components + +
5. @shahadarsh https://zLifecycle.com https://bit.ly/eac-dod-boston Evolution of IaC Execution From Engineers machine IaC Pipeline/GitOps execution from a Shared Environment
6. @shahadarsh https://zLifecycle.com https://bit.ly/eac-dod-boston
7. @shahadarsh https://zLifecycle.com https://bit.ly/eac-dod-boston • Option 1 • Create a Monolith IaC • Option 2 • Hand-roll Pipelines • Manage Complex Dependencies Options for Entire Environment Provisioning
8. @shahadarsh https://zLifecycle.com https://bit.ly/eac-dod-boston Other Challenges scaling IaC Replicating Environments is a pain Not easy to visualize/understand Environments Drift Detection Not straightforward to 
 promote changes Dev QA Prod
9. @shahadarsh https://zLifecycle.com https://bit.ly/eac-dod-boston What is Environment as Code?
10. @shahadarsh https://zLifecycle.com https://bit.ly/eac-dod-boston Environment as Code networking eks website postgres networking website eks postgres Declarative Environment as Code +
11. @shahadarsh https://zLifecycle.com https://bit.ly/eac-dod-boston
12. @shahadarsh https://zLifecycle.com https://bit.ly/eac-dod-boston IaC vs EaC Infrastructure as Code Automates various Lego Pieces 
 (i.e. Infrastructure Resources) Environment as Code Automates how those Lego Pieces 
 are connected to make up a Lego Toy 
 (i.e. Entire Environment)
13. @shahadarsh https://zLifecycle.com https://bit.ly/eac-dod-boston Environment as Code Environment as Code (EaC) is an abstraction over Cloud native tools that provides a declarative way of de fi ning an entire Environment. It has a Control Plane that manages the state of the environment, including relationships between various resources, Detects Drift as well enables Reconciliation.
14. @shahadarsh https://zLifecycle.com https://bit.ly/eac-dod-boston Provision Environment Environment as Code Environment State (Manages dependencies, status etc. ) State networking eks State website postgres State networking website eks postgres Provision Reconcile Declarative Detect Drift Control Plane +
15. @shahadarsh https://zLifecycle.com https://bit.ly/eac-dod-boston networking website eks postgres Teardown Environment Environment as Code Environment State (Manages dependencies, status etc. ) State networking State website eks postgres State Teardown Reconcile Declarative Detect Drift Control Plane +
16. @shahadarsh https://zLifecycle.com https://bit.ly/eac-dod-boston Attributes of Environment as Code
17. @shahadarsh https://zLifecycle.com https://bit.ly/eac-dod-boston Ability to de fi ne Entire Environment
18. @shahadarsh https://zLifecycle.com https://bit.ly/eac-dod-boston Environment as Code networking platform-k8s k8s-addons postgres networking eks website postgres Declarative Loosely Coupled
19. @shahadarsh https://zLifecycle.com https://bit.ly/eac-dod-boston Manage State for the entire Environment Example Environment State File
20. @shahadarsh https://zLifecycle.com https://bit.ly/eac-dod-boston Idempotent and Immutable for entire Environment
21. @shahadarsh https://zLifecycle.com https://bit.ly/eac-dod-boston Non-Idempotent Provision VM VM VM { } IaC 3 VMs End State
 Expected = 3 Actual = 6 Idempotent End State
 Expected = 3 Actual = 3 Reapply VM VM VM Provision VM VM VM Reapply No change { } IaC 3 VMs
22. @shahadarsh https://zLifecycle.com https://bit.ly/eac-dod-boston Provision v1 v1 v1 User Mutable Infrastructure { } IaC Apply changes v2 v2 v2 Change { } IaC v1 v1 v1 Provision User Provision v2 v2 v2 User Immutable Infrastructure { } IaC Change { } IaC Deploys v2 to same Infrastructure Provisions new Infrastructure with v2
23. @shahadarsh https://zLifecycle.com https://bit.ly/eac-dod-boston Visualize and Understand Environments
24. @shahadarsh https://zLifecycle.com https://bit.ly/eac-dod-boston Drift Detection and Reconciliation Reconcile (Preferably with Approval step that shows the plan) Desired State Current State Control Loop Observe/Detect Drift
25. @shahadarsh https://zLifecycle.com https://bit.ly/eac-dod-boston Compare and Promote Changes between Environments Compare & promote changes across environments GitOps for Entire Environment
26. @shahadarsh https://zLifecycle.com https://bit.ly/eac-dod-boston Create Pull Request, EaC Validate Validate Approve & 
 Merge PR Control Plane Reconcile GitOps for Environment EaC Trunk networking platform-k8s k8s-addons postgres Observe Branch networking eks website postgres
27. Adarsh Shah Engineering Leader, Coach, Public Speaker Founder & CEO at zLifecycle @shahadarsh 
 https://zLifecycle.com Thank You @shahadarsh https://bit.ly/eac-dod-boston