Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Cloud Infrastructure as Code
Search
Mikhail Shilkov
October 30, 2019
Programming
0
220
Cloud Infrastructure as Code
History and tool comparison of Infrastructure as Code for cloud applications in AWS and Azure
Mikhail Shilkov
October 30, 2019
Tweet
Share
More Decks by Mikhail Shilkov
See All by Mikhail Shilkov
From YAML to TypeScript: Developer’s View on Cloud Automation
mikhailshilkov
0
78
Cloud Management Superpowers with Pulumi
mikhailshilkov
0
480
Cloud Superpowers with Pulumi and F#
mikhailshilkov
1
480
Cloud Management Superpowers with Pulumi and .NET
mikhailshilkov
0
110
Managing Any Cloud with .NET
mikhailshilkov
0
67
Azure Infrastructure as C# and F#
mikhailshilkov
0
240
Infrastructure as Software
mikhailshilkov
0
400
Azure Infrastructure as C# and F#
mikhailshilkov
0
300
Pulumi: Cloud Infrastructure as C# and F#
mikhailshilkov
0
790
Other Decks in Programming
See All in Programming
チームの境界をブチ抜いていけ
tokai235
0
140
CSC509 Lecture 04
javiergs
PRO
0
300
Signals & Resource API in Angular: 3 Effective Rules for Your Architecture @BASTA 2025 in Mainz
manfredsteyer
PRO
0
110
Pull-Requestの内容を1クリックで動作確認可能にするワークフロー
natmark
2
480
育てるアーキテクチャ:戦い抜くPythonマイクロサービスの設計と進化戦略
fujidomoe
1
170
技術的負債の正体を知って向き合う / Facing Technical Debt
irof
0
130
dynamic!
moro
10
7.1k
AI Coding Meetup #3 - 導入セッション / ai-coding-meetup-3
izumin5210
0
660
CSC305 Lecture 01
javiergs
PRO
1
400
Goで実践するドメイン駆動開発 AIと歩み始めた新規プロダクト開発の現在地
imkaoru
4
790
monorepo の Go テストをはやくした〜い!~最小の依存解決への道のり~ / faster-testing-of-monorepos
convto
2
450
組込みだけじゃない!TinyGo で始める無料クラウド開発入門
otakakot
0
170
Featured
See All Featured
The Web Performance Landscape in 2024 [PerfNow 2024]
tammyeverts
9
860
Optimising Largest Contentful Paint
csswizardry
37
3.4k
Scaling GitHub
holman
463
140k
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
507
140k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
48
9.7k
Product Roadmaps are Hard
iamctodd
PRO
54
11k
How to train your dragon (web standard)
notwaldorf
96
6.3k
The Language of Interfaces
destraynor
162
25k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
31
9.7k
For a Future-Friendly Web
brad_frost
180
9.9k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
252
21k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
358
30k
Transcript
Cloud Infrastructure as Code Cloud Republic Event | October 30
| 2019
Agenda
On-Premises
On-premises Load Balancer Web Server Web Server Load Balancer Database
Server Database Server
LB Web Web DB LB DB LB Web Web DB
LB DB Web DB Web + DB
Ad-hoc Management
Issues
Configuration management tools: Puppet/Chef
Benefits of Infrastructure as Code
Cloud
Compute on Demand
REST calls Content-Type: application/x-www-form-urlencoded; charset=UTF-8 X-Amz-Date: 20130813T150211Z Host: ec2.amazonaws.com Authorization:
AWS4-HMAC-SHA256 Credential=AKIDEXAMPLE/20130813/us-east- 1/ec2/aws4_request, SignedHeaders=content-type;host;x-amz- date, Signature=ced6826de92d2bdeed8f846f0bf508e8559e98e4b0194b84example54174deb456c http://ec2.amazonaws.com/?Action=RunInstances ImageId=ami-2bb65342 &MaxCount=3 &MinCount=1 &Monitoring.Enabled=true &Placement.AvailabilityZone=us-east-1a &Version=2016-11-15
var launchRequest = new RunInstancesRequest() { ImageId = amiID, InstanceType
= "t1.micro", MinCount = 1, MaxCount = 1, KeyName = keyPairName, SecurityGroupIds = groups }; var launchResponse = ec2Client.RunInstances(launchRequest); var instances = launchResponse.Reservation.Instances; SDKs to call those REST endpoints
More Cloud services (PaaS, SaaS, FaaS)
Early Cloud API Challenges
Resource Managers
Unified API Layer
az storage account create \ --location westus \ --name samplesa
\ --resource-group myrg \ --sku LRS Unified CLI
Desired State Provisioning
ARM Templates: JSON "resources": [ { "apiVersion": "2016-01-01", "type": "Microsoft.Storage/storageAccounts",
"name": "mystorageaccount", "location": "westus", "sku": { "name": "Standard_LRS" }, "kind": "Storage", "properties": { } } ]
Converted to REST calls PUT https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/ {resourceGroupName}/providers/Microsoft.Storage/storageAccounts/mystorageaccount?api- version=2016-01-01 { "location":
"westus", "properties": { } "sku": { "name": "Standard_LRS" }, "kind": "Storage" }
Tracked Deployment
AWS CloudFormation: YAML Resources: S3BucketForURLs: Type: "AWS::S3::Bucket" DeletionPolicy: Delete Properties:
BucketName: !If [ "CreateNewBucket", !Ref "AWS::NoValue", !Ref S3BucketName ] WebsiteConfiguration: IndexDocument: "index.html" LifecycleConfiguration: Rules: - Id: DisposeShortUrls ExpirationInDays: !Ref URLExpiration Prefix: "u" Status: Enabled
AWS CloudFormation Nice Features
Resource Graphs
Describes a list of Resources
Resource Graphs
Challenges
More Challenges
Terraform
Made by HashiCorp Also: Vagrant, Packer, Vault, Consul Written in
Go, open source MPL 2.0 Since 2014 Terraform
resource "azurerm_storage_account" "sa" { name = "mysa" location = azurerm_resource_group.rg.location
resource_group_name = azurerm_resource_group.rg.name account_tier = "Standard" account_replication_type = "LRS" } Custom DSL: HCL (HashiCorp Configuration Language)
• • • • • Providers • • • •
•
Client-driven Engine Azure Resource Manager Azure Go SDK Azure TF
Provider TF CLI HCL
resource "azurerm_function_app" "app" { name = var.app_name location = azurerm_resource_group.rg.location
resource_group_name = azurerm_resource_group.rg.name app_service_plan_id = azurerm_app_service_plan.asp.id storage_connection_string = azurerm_storage_account.sa.primary_connection_string } Strong in dependency management
terraform plan # Creates a draft of changes, does nothing
to the infra terraform apply # Rolls the infra from its current state to the desired one terraform destroy # Deletes all resources in the current workspace Workflow: Plan, Apply, Destroy
> terraform plan Terraform will perform the following actions: #
azurerm_storage_account.sa must be replaced -/+ resource "azurerm_storage_account" "sa" { ... ~ location = "westus" -> "westus2" # forces replacement ... Preview Changes
Serverless
Serverless URL Shortener AWS DynamoDB Table “URLs” AWS Lambda “Add
URL” AWS Lambda “Open URL”
AWS Resources Lambda “Add URL” Lambda “Open URL” DynamoDB “URLs”
API Gateway S3 Bucket Static site
AWS Resources Lambda “Add URL” Lambda “Open URL” DynamoDB “URLs”
API Gateway S3 Bucket Static site Stage Deployment REST endpoint Permissions Permissions Policy Bucket Objects
Need for higher abstraction and reusable components KV Store Table
“URLs” Function “Add URL” Function “Open URL”
Cloud- Native Ecosystem
How?
Serverless Framework
Made by Serverless, Inc. Tooling for Serverless Applications Since 2015
Serverless Framework
service: serverless-simple-http-endpoint provider: name: aws runtime: nodejs8.10 functions: currentTime: handler:
handler.endpoint events: - http: path: ping method: get YAML Definition Files
Focus on target scenarios
Pulumi
Made by Pulumi Corp. Written in Go, open source Apache
License 2.0 Since 2018 Pulumi
General-Purpose Programming Languages
• • • • • Providers • • • •
•
const storageAccount = new azure.storage.Account("storage", { resourceGroupName: resourceGroup.name, accountReplicationType: "LRS",
accountTier: "Standard", }); TypeScript example
How Pulumi works CLI and Engine Last deployed state index.ts
Language host AWS Azure GCP Kubernetes new Resource() Create, update, delete
Workflow: Preview, Up, Destroy
Workflow: Preview, Up, Destroy
Benefits of Real Code
const counterTable = new aws.dynamodb.Table("url-table", { attributes: [{ name: "id",
type: "S" }], hashKey: "id", readCapacity: 5, }); const lambda = new aws.lambda.Function("mylambda", { code: new pulumi.asset.FileArchive("./app"), handler: "index.handler", }); const endpoint = new awsx.apigateway.API("short-url", { routes: [{ path: "/{route+}", method: "GET", eventHandler: lambda, }], }); URL Shortener
const counterTable = new aws.dynamodb.Table("url-table", { attributes: [{ name: "id",
type: "S" }], hashKey: "id", readCapacity: 5, }); const lambda = new aws.lambda.Function("mylambda", { code: new pulumi.asset.FileArchive("./app"), handler: "index.handler", }); const endpoint = new awsx.apigateway.API("short-url", { routes: [{ path: "/{route+}", method: "GET", eventHandler: lambda, }], }); URL Shortener
const counterTable = new aws.dynamodb.Table("url-table", { attributes: [{ name: "id",
type: "S" }], hashKey: "id", readCapacity: 5, }); const lambda = new aws.lambda.Function("mylambda", { code: new pulumi.asset.FileArchive("./app"), handler: "index.handler", }); const endpoint = new awsx.apigateway.API("short-url", { routes: [{ path: "/{route+}", method: "GET", eventHandler: lambda, }], }); URL Shortener
AWS Cloud Development Kit (CDK)
м TypeScript Python Java .NET Compiles to CloudFormation
const hello = new lambda.Function(this, 'HelloHandler', { runtime: lambda.Runtime.NODEJS_8_10, code:
lambda.Code.asset('lambda'), handler: 'hello.handler' }); new apigw.LambdaRestApi(this, 'Endpoint', { handler: hello }); AWS CDK example
// This is F# let myStorage = storageAccount { name
"mystorage" // set account name sku Storage.Sku.PremiumLRS // use Premium LRS } Azure? ARM Generators (ex: Farmer)
Architecture as Code
Zip Incoming Reports
const tpsReports = new aws.s3.Bucket("tpsReports"); const tpsZips = new aws.s3.Bucket("tpsZips");
tpsReports.onObjectCreated("zipTpsReports", (e) => { for (const rec of e.Records || []) { const [ buck, key ] = [ rec.s3.bucket.name, rec.s3.object.key ]; const data = await s3.getObject({ Bucket: buck, Key: key })); zip.addFile(key, data.Body); await s3.putObject({ Bucket: tpsZips.bucket.get(), Key: `${key}.zip`, Body: zip.toBuffer(), }); } }); Functions = Callbacks
const tpsReports = new aws.s3.Bucket("tpsReports"); const tpsZips = new aws.s3.Bucket("tpsZips");
tpsReports.onObjectCreated("zipTpsReports", (e) => { for (const rec of e.Records || []) { const [ buck, key ] = [ rec.s3.bucket.name, rec.s3.object.key ]; const data = await s3.getObject({ Bucket: buck, Key: key })); zip.addFile(key, data.Body); await s3.putObject({ Bucket: tpsZips.bucket.get(), Key: `${key}.zip`, Body: zip.toBuffer(), }); } }); Functions = Callbacks
Scheduled Tasks
const ecsScheduledTask = new ScheduledEc2Task(stack, 'ScheduledTask', { schedule: events.Schedule.expression('rate(1 minute)')
scheduledEc2TaskImageOptions: { image: ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'), memoryLimitMiB: 256, environment: { name: 'TRIGGER', value: 'CloudWatch Events' }, }, }); Scheduled Tasks
Global Apps
export const functions = new CosmosApp("urls", { resourceGroup, locations: ["WestEurope",
"WestUS", "SouthEastAsia"], factory: ({location, cosmosdb}) => { const app = new azure.appservice.ArchiveFunctionApp("app", { location, archive: new pulumi.asset.FileArchive("./app"), appSettings: { COSMOSDB_ENDPOINT: cosmosdb.endpoint, }, }); return { id: app.functionApp.id }; }}); Global Apps
export const functions = new CosmosApp("urls", { resourceGroup, locations: ["WestEurope",
"WestUS", "SouthEastAsia"], factory: ({location, cosmosdb}) => { const app = new azure.appservice.ArchiveFunctionApp("app", { location, archive: new pulumi.asset.FileArchive("./app"), appSettings: { COSMOSDB_ENDPOINT: cosmosdb.endpoint, }, }); return { id: app.functionApp.id }; }}); Global Apps
Conclusions
USE INFRASTRUCTURE- AS-CODE Making Cloud Apps?
Infrastructure as Code Tools: Tradeoffs
Fine-grained components Better Automation
In Search for Powerful Abstractions
None