Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Cloud Infrastructure as Code
Search
Mikhail Shilkov
October 30, 2019
Programming
0
220
Cloud Infrastructure as Code
History and tool comparison of Infrastructure as Code for cloud applications in AWS and Azure
Mikhail Shilkov
October 30, 2019
Tweet
Share
More Decks by Mikhail Shilkov
See All by Mikhail Shilkov
From YAML to TypeScript: Developer’s View on Cloud Automation
mikhailshilkov
0
78
Cloud Management Superpowers with Pulumi
mikhailshilkov
0
480
Cloud Superpowers with Pulumi and F#
mikhailshilkov
1
480
Cloud Management Superpowers with Pulumi and .NET
mikhailshilkov
0
110
Managing Any Cloud with .NET
mikhailshilkov
0
65
Azure Infrastructure as C# and F#
mikhailshilkov
0
240
Infrastructure as Software
mikhailshilkov
0
400
Azure Infrastructure as C# and F#
mikhailshilkov
0
300
Pulumi: Cloud Infrastructure as C# and F#
mikhailshilkov
0
780
Other Decks in Programming
See All in Programming
AI時代のドメイン駆動設計-DDD実践におけるAI活用のあり方 / ddd-in-ai-era
minodriven
25
9.6k
Ruby×iOSアプリ開発 ~共に歩んだエコシステムの物語~
temoki
0
120
AIエージェント開発、DevOps and LLMOps
ymd65536
1
370
為你自己學 Python - 冷知識篇
eddie
1
320
プロポーザル駆動学習 / Proposal-Driven Learning
mackey0225
1
120
奥深くて厄介な「改行」と仲良くなる20分
oguemon
1
230
Oracle Database Technology Night 92 Database Connection control FAN-AC
oracle4engineer
PRO
1
360
Google I/O recap web編 大分Web祭り2025
kponda
0
2.9k
Testing Trophyは叫ばない
toms74209200
0
170
Protocol Buffersの型を超えて拡張性を得る / Beyond Protocol Buffers Types Achieving Extensibility
linyows
0
100
Kiroで始めるAI-DLC
kaonash
2
500
tool ディレクティブを導入してみた感想
sgash708
1
160
Featured
See All Featured
A designer walks into a library…
pauljervisheath
207
24k
Building a Scalable Design System with Sketch
lauravandoore
462
33k
The Invisible Side of Design
smashingmag
301
51k
For a Future-Friendly Web
brad_frost
179
9.9k
Facilitating Awesome Meetings
lara
55
6.5k
YesSQL, Process and Tooling at Scale
rocio
173
14k
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
15
1.6k
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
194
16k
Six Lessons from altMBA
skipperchong
28
4k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
48
9.7k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
34
3.1k
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
irinanazarova
8
910
Transcript
Cloud Infrastructure as Code Cloud Republic Event | October 30
| 2019
Agenda
On-Premises
On-premises Load Balancer Web Server Web Server Load Balancer Database
Server Database Server
LB Web Web DB LB DB LB Web Web DB
LB DB Web DB Web + DB
Ad-hoc Management
Issues
Configuration management tools: Puppet/Chef
Benefits of Infrastructure as Code
Cloud
Compute on Demand
REST calls Content-Type: application/x-www-form-urlencoded; charset=UTF-8 X-Amz-Date: 20130813T150211Z Host: ec2.amazonaws.com Authorization:
AWS4-HMAC-SHA256 Credential=AKIDEXAMPLE/20130813/us-east- 1/ec2/aws4_request, SignedHeaders=content-type;host;x-amz- date, Signature=ced6826de92d2bdeed8f846f0bf508e8559e98e4b0194b84example54174deb456c http://ec2.amazonaws.com/?Action=RunInstances ImageId=ami-2bb65342 &MaxCount=3 &MinCount=1 &Monitoring.Enabled=true &Placement.AvailabilityZone=us-east-1a &Version=2016-11-15
var launchRequest = new RunInstancesRequest() { ImageId = amiID, InstanceType
= "t1.micro", MinCount = 1, MaxCount = 1, KeyName = keyPairName, SecurityGroupIds = groups }; var launchResponse = ec2Client.RunInstances(launchRequest); var instances = launchResponse.Reservation.Instances; SDKs to call those REST endpoints
More Cloud services (PaaS, SaaS, FaaS)
Early Cloud API Challenges
Resource Managers
Unified API Layer
az storage account create \ --location westus \ --name samplesa
\ --resource-group myrg \ --sku LRS Unified CLI
Desired State Provisioning
ARM Templates: JSON "resources": [ { "apiVersion": "2016-01-01", "type": "Microsoft.Storage/storageAccounts",
"name": "mystorageaccount", "location": "westus", "sku": { "name": "Standard_LRS" }, "kind": "Storage", "properties": { } } ]
Converted to REST calls PUT https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/ {resourceGroupName}/providers/Microsoft.Storage/storageAccounts/mystorageaccount?api- version=2016-01-01 { "location":
"westus", "properties": { } "sku": { "name": "Standard_LRS" }, "kind": "Storage" }
Tracked Deployment
AWS CloudFormation: YAML Resources: S3BucketForURLs: Type: "AWS::S3::Bucket" DeletionPolicy: Delete Properties:
BucketName: !If [ "CreateNewBucket", !Ref "AWS::NoValue", !Ref S3BucketName ] WebsiteConfiguration: IndexDocument: "index.html" LifecycleConfiguration: Rules: - Id: DisposeShortUrls ExpirationInDays: !Ref URLExpiration Prefix: "u" Status: Enabled
AWS CloudFormation Nice Features
Resource Graphs
Describes a list of Resources
Resource Graphs
Challenges
More Challenges
Terraform
Made by HashiCorp Also: Vagrant, Packer, Vault, Consul Written in
Go, open source MPL 2.0 Since 2014 Terraform
resource "azurerm_storage_account" "sa" { name = "mysa" location = azurerm_resource_group.rg.location
resource_group_name = azurerm_resource_group.rg.name account_tier = "Standard" account_replication_type = "LRS" } Custom DSL: HCL (HashiCorp Configuration Language)
• • • • • Providers • • • •
•
Client-driven Engine Azure Resource Manager Azure Go SDK Azure TF
Provider TF CLI HCL
resource "azurerm_function_app" "app" { name = var.app_name location = azurerm_resource_group.rg.location
resource_group_name = azurerm_resource_group.rg.name app_service_plan_id = azurerm_app_service_plan.asp.id storage_connection_string = azurerm_storage_account.sa.primary_connection_string } Strong in dependency management
terraform plan # Creates a draft of changes, does nothing
to the infra terraform apply # Rolls the infra from its current state to the desired one terraform destroy # Deletes all resources in the current workspace Workflow: Plan, Apply, Destroy
> terraform plan Terraform will perform the following actions: #
azurerm_storage_account.sa must be replaced -/+ resource "azurerm_storage_account" "sa" { ... ~ location = "westus" -> "westus2" # forces replacement ... Preview Changes
Serverless
Serverless URL Shortener AWS DynamoDB Table “URLs” AWS Lambda “Add
URL” AWS Lambda “Open URL”
AWS Resources Lambda “Add URL” Lambda “Open URL” DynamoDB “URLs”
API Gateway S3 Bucket Static site
AWS Resources Lambda “Add URL” Lambda “Open URL” DynamoDB “URLs”
API Gateway S3 Bucket Static site Stage Deployment REST endpoint Permissions Permissions Policy Bucket Objects
Need for higher abstraction and reusable components KV Store Table
“URLs” Function “Add URL” Function “Open URL”
Cloud- Native Ecosystem
How?
Serverless Framework
Made by Serverless, Inc. Tooling for Serverless Applications Since 2015
Serverless Framework
service: serverless-simple-http-endpoint provider: name: aws runtime: nodejs8.10 functions: currentTime: handler:
handler.endpoint events: - http: path: ping method: get YAML Definition Files
Focus on target scenarios
Pulumi
Made by Pulumi Corp. Written in Go, open source Apache
License 2.0 Since 2018 Pulumi
General-Purpose Programming Languages
• • • • • Providers • • • •
•
const storageAccount = new azure.storage.Account("storage", { resourceGroupName: resourceGroup.name, accountReplicationType: "LRS",
accountTier: "Standard", }); TypeScript example
How Pulumi works CLI and Engine Last deployed state index.ts
Language host AWS Azure GCP Kubernetes new Resource() Create, update, delete
Workflow: Preview, Up, Destroy
Workflow: Preview, Up, Destroy
Benefits of Real Code
const counterTable = new aws.dynamodb.Table("url-table", { attributes: [{ name: "id",
type: "S" }], hashKey: "id", readCapacity: 5, }); const lambda = new aws.lambda.Function("mylambda", { code: new pulumi.asset.FileArchive("./app"), handler: "index.handler", }); const endpoint = new awsx.apigateway.API("short-url", { routes: [{ path: "/{route+}", method: "GET", eventHandler: lambda, }], }); URL Shortener
const counterTable = new aws.dynamodb.Table("url-table", { attributes: [{ name: "id",
type: "S" }], hashKey: "id", readCapacity: 5, }); const lambda = new aws.lambda.Function("mylambda", { code: new pulumi.asset.FileArchive("./app"), handler: "index.handler", }); const endpoint = new awsx.apigateway.API("short-url", { routes: [{ path: "/{route+}", method: "GET", eventHandler: lambda, }], }); URL Shortener
const counterTable = new aws.dynamodb.Table("url-table", { attributes: [{ name: "id",
type: "S" }], hashKey: "id", readCapacity: 5, }); const lambda = new aws.lambda.Function("mylambda", { code: new pulumi.asset.FileArchive("./app"), handler: "index.handler", }); const endpoint = new awsx.apigateway.API("short-url", { routes: [{ path: "/{route+}", method: "GET", eventHandler: lambda, }], }); URL Shortener
AWS Cloud Development Kit (CDK)
м TypeScript Python Java .NET Compiles to CloudFormation
const hello = new lambda.Function(this, 'HelloHandler', { runtime: lambda.Runtime.NODEJS_8_10, code:
lambda.Code.asset('lambda'), handler: 'hello.handler' }); new apigw.LambdaRestApi(this, 'Endpoint', { handler: hello }); AWS CDK example
// This is F# let myStorage = storageAccount { name
"mystorage" // set account name sku Storage.Sku.PremiumLRS // use Premium LRS } Azure? ARM Generators (ex: Farmer)
Architecture as Code
Zip Incoming Reports
const tpsReports = new aws.s3.Bucket("tpsReports"); const tpsZips = new aws.s3.Bucket("tpsZips");
tpsReports.onObjectCreated("zipTpsReports", (e) => { for (const rec of e.Records || []) { const [ buck, key ] = [ rec.s3.bucket.name, rec.s3.object.key ]; const data = await s3.getObject({ Bucket: buck, Key: key })); zip.addFile(key, data.Body); await s3.putObject({ Bucket: tpsZips.bucket.get(), Key: `${key}.zip`, Body: zip.toBuffer(), }); } }); Functions = Callbacks
const tpsReports = new aws.s3.Bucket("tpsReports"); const tpsZips = new aws.s3.Bucket("tpsZips");
tpsReports.onObjectCreated("zipTpsReports", (e) => { for (const rec of e.Records || []) { const [ buck, key ] = [ rec.s3.bucket.name, rec.s3.object.key ]; const data = await s3.getObject({ Bucket: buck, Key: key })); zip.addFile(key, data.Body); await s3.putObject({ Bucket: tpsZips.bucket.get(), Key: `${key}.zip`, Body: zip.toBuffer(), }); } }); Functions = Callbacks
Scheduled Tasks
const ecsScheduledTask = new ScheduledEc2Task(stack, 'ScheduledTask', { schedule: events.Schedule.expression('rate(1 minute)')
scheduledEc2TaskImageOptions: { image: ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'), memoryLimitMiB: 256, environment: { name: 'TRIGGER', value: 'CloudWatch Events' }, }, }); Scheduled Tasks
Global Apps
export const functions = new CosmosApp("urls", { resourceGroup, locations: ["WestEurope",
"WestUS", "SouthEastAsia"], factory: ({location, cosmosdb}) => { const app = new azure.appservice.ArchiveFunctionApp("app", { location, archive: new pulumi.asset.FileArchive("./app"), appSettings: { COSMOSDB_ENDPOINT: cosmosdb.endpoint, }, }); return { id: app.functionApp.id }; }}); Global Apps
export const functions = new CosmosApp("urls", { resourceGroup, locations: ["WestEurope",
"WestUS", "SouthEastAsia"], factory: ({location, cosmosdb}) => { const app = new azure.appservice.ArchiveFunctionApp("app", { location, archive: new pulumi.asset.FileArchive("./app"), appSettings: { COSMOSDB_ENDPOINT: cosmosdb.endpoint, }, }); return { id: app.functionApp.id }; }}); Global Apps
Conclusions
USE INFRASTRUCTURE- AS-CODE Making Cloud Apps?
Infrastructure as Code Tools: Tradeoffs
Fine-grained components Better Automation
In Search for Powerful Abstractions
None