Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Cloud Infrastructure as Code
Search
Mikhail Shilkov
October 30, 2019
Programming
0
180
Cloud Infrastructure as Code
History and tool comparison of Infrastructure as Code for cloud applications in AWS and Azure
Mikhail Shilkov
October 30, 2019
Tweet
Share
More Decks by Mikhail Shilkov
See All by Mikhail Shilkov
From YAML to TypeScript: Developer’s View on Cloud Automation
mikhailshilkov
0
64
Cloud Management Superpowers with Pulumi
mikhailshilkov
0
420
Cloud Superpowers with Pulumi and F#
mikhailshilkov
1
420
Cloud Management Superpowers with Pulumi and .NET
mikhailshilkov
0
93
Managing Any Cloud with .NET
mikhailshilkov
0
50
Azure Infrastructure as C# and F#
mikhailshilkov
0
200
Infrastructure as Software
mikhailshilkov
0
340
Azure Infrastructure as C# and F#
mikhailshilkov
0
250
Pulumi: Cloud Infrastructure as C# and F#
mikhailshilkov
0
610
Other Decks in Programming
See All in Programming
Click-free releases & the making of a CLI app
oheyadam
2
120
『ドメイン駆動設計をはじめよう』のモデリングアプローチ
masuda220
PRO
8
540
TypeScriptでライブラリとの依存を限定的にする方法
tutinoko
3
700
watsonx.ai Dojo #4 生成AIを使ったアプリ開発、応用編
oniak3ibm
PRO
1
150
Contemporary Test Cases
maaretp
0
140
Arm移行タイムアタック
qnighy
0
340
카카오페이는 어떻게 수천만 결제를 처리할까? 우아한 결제 분산락 노하우
kakao
PRO
0
110
Flutterを言い訳にしない!アプリの使い心地改善テクニック5選🔥
kno3a87
1
200
CSC509 Lecture 13
javiergs
PRO
0
110
Outline View in SwiftUI
1024jp
1
330
광고 소재 심사 과정에 AI를 도입하여 광고 서비스 생산성 향상시키기
kakao
PRO
0
170
Make Impossible States Impossibleを 意識してReactのPropsを設計しよう
ikumatadokoro
0
240
Featured
See All Featured
Six Lessons from altMBA
skipperchong
27
3.5k
Imperfection Machines: The Place of Print at Facebook
scottboms
265
13k
Done Done
chrislema
181
16k
Why Our Code Smells
bkeepers
PRO
334
57k
GraphQLとの向き合い方2022年版
quramy
43
13k
KATA
mclloyd
29
14k
Designing Experiences People Love
moore
138
23k
The Invisible Side of Design
smashingmag
298
50k
Facilitating Awesome Meetings
lara
50
6.1k
RailsConf 2023
tenderlove
29
900
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
169
50k
Typedesign – Prime Four
hannesfritz
40
2.4k
Transcript
Cloud Infrastructure as Code Cloud Republic Event | October 30
| 2019
Agenda
On-Premises
On-premises Load Balancer Web Server Web Server Load Balancer Database
Server Database Server
LB Web Web DB LB DB LB Web Web DB
LB DB Web DB Web + DB
Ad-hoc Management
Issues
Configuration management tools: Puppet/Chef
Benefits of Infrastructure as Code
Cloud
Compute on Demand
REST calls Content-Type: application/x-www-form-urlencoded; charset=UTF-8 X-Amz-Date: 20130813T150211Z Host: ec2.amazonaws.com Authorization:
AWS4-HMAC-SHA256 Credential=AKIDEXAMPLE/20130813/us-east- 1/ec2/aws4_request, SignedHeaders=content-type;host;x-amz- date, Signature=ced6826de92d2bdeed8f846f0bf508e8559e98e4b0194b84example54174deb456c http://ec2.amazonaws.com/?Action=RunInstances ImageId=ami-2bb65342 &MaxCount=3 &MinCount=1 &Monitoring.Enabled=true &Placement.AvailabilityZone=us-east-1a &Version=2016-11-15
var launchRequest = new RunInstancesRequest() { ImageId = amiID, InstanceType
= "t1.micro", MinCount = 1, MaxCount = 1, KeyName = keyPairName, SecurityGroupIds = groups }; var launchResponse = ec2Client.RunInstances(launchRequest); var instances = launchResponse.Reservation.Instances; SDKs to call those REST endpoints
More Cloud services (PaaS, SaaS, FaaS)
Early Cloud API Challenges
Resource Managers
Unified API Layer
az storage account create \ --location westus \ --name samplesa
\ --resource-group myrg \ --sku LRS Unified CLI
Desired State Provisioning
ARM Templates: JSON "resources": [ { "apiVersion": "2016-01-01", "type": "Microsoft.Storage/storageAccounts",
"name": "mystorageaccount", "location": "westus", "sku": { "name": "Standard_LRS" }, "kind": "Storage", "properties": { } } ]
Converted to REST calls PUT https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/ {resourceGroupName}/providers/Microsoft.Storage/storageAccounts/mystorageaccount?api- version=2016-01-01 { "location":
"westus", "properties": { } "sku": { "name": "Standard_LRS" }, "kind": "Storage" }
Tracked Deployment
AWS CloudFormation: YAML Resources: S3BucketForURLs: Type: "AWS::S3::Bucket" DeletionPolicy: Delete Properties:
BucketName: !If [ "CreateNewBucket", !Ref "AWS::NoValue", !Ref S3BucketName ] WebsiteConfiguration: IndexDocument: "index.html" LifecycleConfiguration: Rules: - Id: DisposeShortUrls ExpirationInDays: !Ref URLExpiration Prefix: "u" Status: Enabled
AWS CloudFormation Nice Features
Resource Graphs
Describes a list of Resources
Resource Graphs
Challenges
More Challenges
Terraform
Made by HashiCorp Also: Vagrant, Packer, Vault, Consul Written in
Go, open source MPL 2.0 Since 2014 Terraform
resource "azurerm_storage_account" "sa" { name = "mysa" location = azurerm_resource_group.rg.location
resource_group_name = azurerm_resource_group.rg.name account_tier = "Standard" account_replication_type = "LRS" } Custom DSL: HCL (HashiCorp Configuration Language)
• • • • • Providers • • • •
•
Client-driven Engine Azure Resource Manager Azure Go SDK Azure TF
Provider TF CLI HCL
resource "azurerm_function_app" "app" { name = var.app_name location = azurerm_resource_group.rg.location
resource_group_name = azurerm_resource_group.rg.name app_service_plan_id = azurerm_app_service_plan.asp.id storage_connection_string = azurerm_storage_account.sa.primary_connection_string } Strong in dependency management
terraform plan # Creates a draft of changes, does nothing
to the infra terraform apply # Rolls the infra from its current state to the desired one terraform destroy # Deletes all resources in the current workspace Workflow: Plan, Apply, Destroy
> terraform plan Terraform will perform the following actions: #
azurerm_storage_account.sa must be replaced -/+ resource "azurerm_storage_account" "sa" { ... ~ location = "westus" -> "westus2" # forces replacement ... Preview Changes
Serverless
Serverless URL Shortener AWS DynamoDB Table “URLs” AWS Lambda “Add
URL” AWS Lambda “Open URL”
AWS Resources Lambda “Add URL” Lambda “Open URL” DynamoDB “URLs”
API Gateway S3 Bucket Static site
AWS Resources Lambda “Add URL” Lambda “Open URL” DynamoDB “URLs”
API Gateway S3 Bucket Static site Stage Deployment REST endpoint Permissions Permissions Policy Bucket Objects
Need for higher abstraction and reusable components KV Store Table
“URLs” Function “Add URL” Function “Open URL”
Cloud- Native Ecosystem
How?
Serverless Framework
Made by Serverless, Inc. Tooling for Serverless Applications Since 2015
Serverless Framework
service: serverless-simple-http-endpoint provider: name: aws runtime: nodejs8.10 functions: currentTime: handler:
handler.endpoint events: - http: path: ping method: get YAML Definition Files
Focus on target scenarios
Pulumi
Made by Pulumi Corp. Written in Go, open source Apache
License 2.0 Since 2018 Pulumi
General-Purpose Programming Languages
• • • • • Providers • • • •
•
const storageAccount = new azure.storage.Account("storage", { resourceGroupName: resourceGroup.name, accountReplicationType: "LRS",
accountTier: "Standard", }); TypeScript example
How Pulumi works CLI and Engine Last deployed state index.ts
Language host AWS Azure GCP Kubernetes new Resource() Create, update, delete
Workflow: Preview, Up, Destroy
Workflow: Preview, Up, Destroy
Benefits of Real Code
const counterTable = new aws.dynamodb.Table("url-table", { attributes: [{ name: "id",
type: "S" }], hashKey: "id", readCapacity: 5, }); const lambda = new aws.lambda.Function("mylambda", { code: new pulumi.asset.FileArchive("./app"), handler: "index.handler", }); const endpoint = new awsx.apigateway.API("short-url", { routes: [{ path: "/{route+}", method: "GET", eventHandler: lambda, }], }); URL Shortener
const counterTable = new aws.dynamodb.Table("url-table", { attributes: [{ name: "id",
type: "S" }], hashKey: "id", readCapacity: 5, }); const lambda = new aws.lambda.Function("mylambda", { code: new pulumi.asset.FileArchive("./app"), handler: "index.handler", }); const endpoint = new awsx.apigateway.API("short-url", { routes: [{ path: "/{route+}", method: "GET", eventHandler: lambda, }], }); URL Shortener
const counterTable = new aws.dynamodb.Table("url-table", { attributes: [{ name: "id",
type: "S" }], hashKey: "id", readCapacity: 5, }); const lambda = new aws.lambda.Function("mylambda", { code: new pulumi.asset.FileArchive("./app"), handler: "index.handler", }); const endpoint = new awsx.apigateway.API("short-url", { routes: [{ path: "/{route+}", method: "GET", eventHandler: lambda, }], }); URL Shortener
AWS Cloud Development Kit (CDK)
м TypeScript Python Java .NET Compiles to CloudFormation
const hello = new lambda.Function(this, 'HelloHandler', { runtime: lambda.Runtime.NODEJS_8_10, code:
lambda.Code.asset('lambda'), handler: 'hello.handler' }); new apigw.LambdaRestApi(this, 'Endpoint', { handler: hello }); AWS CDK example
// This is F# let myStorage = storageAccount { name
"mystorage" // set account name sku Storage.Sku.PremiumLRS // use Premium LRS } Azure? ARM Generators (ex: Farmer)
Architecture as Code
Zip Incoming Reports
const tpsReports = new aws.s3.Bucket("tpsReports"); const tpsZips = new aws.s3.Bucket("tpsZips");
tpsReports.onObjectCreated("zipTpsReports", (e) => { for (const rec of e.Records || []) { const [ buck, key ] = [ rec.s3.bucket.name, rec.s3.object.key ]; const data = await s3.getObject({ Bucket: buck, Key: key })); zip.addFile(key, data.Body); await s3.putObject({ Bucket: tpsZips.bucket.get(), Key: `${key}.zip`, Body: zip.toBuffer(), }); } }); Functions = Callbacks
const tpsReports = new aws.s3.Bucket("tpsReports"); const tpsZips = new aws.s3.Bucket("tpsZips");
tpsReports.onObjectCreated("zipTpsReports", (e) => { for (const rec of e.Records || []) { const [ buck, key ] = [ rec.s3.bucket.name, rec.s3.object.key ]; const data = await s3.getObject({ Bucket: buck, Key: key })); zip.addFile(key, data.Body); await s3.putObject({ Bucket: tpsZips.bucket.get(), Key: `${key}.zip`, Body: zip.toBuffer(), }); } }); Functions = Callbacks
Scheduled Tasks
const ecsScheduledTask = new ScheduledEc2Task(stack, 'ScheduledTask', { schedule: events.Schedule.expression('rate(1 minute)')
scheduledEc2TaskImageOptions: { image: ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'), memoryLimitMiB: 256, environment: { name: 'TRIGGER', value: 'CloudWatch Events' }, }, }); Scheduled Tasks
Global Apps
export const functions = new CosmosApp("urls", { resourceGroup, locations: ["WestEurope",
"WestUS", "SouthEastAsia"], factory: ({location, cosmosdb}) => { const app = new azure.appservice.ArchiveFunctionApp("app", { location, archive: new pulumi.asset.FileArchive("./app"), appSettings: { COSMOSDB_ENDPOINT: cosmosdb.endpoint, }, }); return { id: app.functionApp.id }; }}); Global Apps
export const functions = new CosmosApp("urls", { resourceGroup, locations: ["WestEurope",
"WestUS", "SouthEastAsia"], factory: ({location, cosmosdb}) => { const app = new azure.appservice.ArchiveFunctionApp("app", { location, archive: new pulumi.asset.FileArchive("./app"), appSettings: { COSMOSDB_ENDPOINT: cosmosdb.endpoint, }, }); return { id: app.functionApp.id }; }}); Global Apps
Conclusions
USE INFRASTRUCTURE- AS-CODE Making Cloud Apps?
Infrastructure as Code Tools: Tradeoffs
Fine-grained components Better Automation
In Search for Powerful Abstractions
None