Share
Embed
And you thought you knew EC2
EC2 features, best practices, and pro tips. Presented at AWS Community Day 2017 in San Francisco.
Ben Whaley
June 15, 2017
Transcript
39 slides
-
Instance Types Family Intended use C5 Compute-optimized with Intel Skylake*
R4 Memory-optimized, up to 488GiB** I3 High I/O - Watch for sharp corners F1 FPGA P2 GPU - 2x perf of G2 at 1.5x the cost D2 Dense Storage, up to 48TiB HDD & 10GiB network * Coming soon? ** Only 244GiB currently -
4 EBS Volume Types Type Description I/O Throughput Cost io1
Provisioned IOPS SSD Highest High Highest gp2 General purpose SSD High Low(ish) Medium st1 Throughput-optimized HDD Low Highest Low sc1 Non-optimized HDD Low Medium Lowest -
Miscellaneous improvements & features • IPv6! • VPC endpoints for
DynamoDB, S3 • Lightsail - EC2 quick start • Elastic GPUs • New regions • 2016 - Ohio, Canada, London, Mumbia, Seoul • Ningxia, Paris, Stockholm -
EC2 Systems Manager Superpowers for EC2 instances and on-premises systems.
• Remote command execution with Run Commands • Controlled secrets and configuration data with the Parameter Store • Periodic tasks with the State Manager and Maintenance Windows • Stepwise Automation workflows for initializing nodes • Collect and query Inventory and Patch status -
Systems Manager Essentials • The SSM agent • Open source
(Golang) executable for Linux and Windows • Available for cloud and on-premises systems • Assign IAM role with permissions to interface with SSM API • Install at boot or on existing systems • Polls for commands to execute • All actions recorded in CloudTrail (e.g. immutable audit trail) • Trigger SNS, Lambda from Systems Manager events • Store command history and output to S3 • Fine-grained access control to Run Commands • Integration with Config to track changes over time -
Systems Manager Documents JSON Schema describing actions for the systems
manager { "schemaVersion":"2.0", "description":"Run a script", "parameters":{ "commands":{ "type":"String", “description”:"Commands to run" } }, "mainSteps":[ { "action":"aws:runShellScript", "name":"runShellScript", "inputs":{ "runCommand":"{{ commands }}" } } ] } -
Quick & dirty SSM demo: 1. Scale up ASG of
managed instances 2. Run a command on remote agents 3. Store and retrieve a parameter Virtual Private Cloud Private subnet EC2 Instances Systems Manager ssm-agent Workstations -
Five Ways to Provision Instances Configuration management runs at boot,
registers with server, converges a configuration 3 -
Five Ways to Provision Instances Autoscaling lifecycle hook —> CloudWatch
event —> Run Command —> execute provisioning documents 4 (Alternative: CloudWatch event —> Lambda) -
Application Load Balancer • Almost as good as HAProxy or
NGINX. Almost. • Host- and path-based routing • Additional metrics (# active connections, total traffic) • Improved health checks • Websockets • HTTP/2 • Integration with X-Ray (adds X-Amzn-Trace-Id header) • Integration with ECS • Integration with WAF -
One solution: route53 health checks with autoscaling lifecycle events Problems:
Rotating IP addresses Traffic surges ELBs -
ELB IP 1 ELB IP 2 Clients Clients Clients Clients
Moar clients? Feeling… weak… must… scale up. ELB IP 1 ELB IP 2 Clients Clients Couple of clients? Come at me, bro. -
ELB IP 3 ELB IP 4 Clients Clients I sense
a disturbance in the force… as if hundreds of clients suddenly cried out in pain Clients Clients ELB IP 1 ELB IP 2 Bueller? Bueller? … moments later -
Clients Lookup A record for myservice.example.com Route53 IP 1 IP
2 Clients Client Connect to IP 1 Node with IP 1 Client Connect to IP 2 Node with IP 2 health checks ASG -
Clients Connect to IP 1 Clients Connect to IP 2
ASG Node with IP 1 Node with IP 2 Clients Clients Clients Clients Clients Clients 1. Autoscale via custom CloudWatch metric 2. New node boots with IP 3 3. Autoscaling lifecycle hook adds IP 3 to myservice.example.com A record Moar clients! -
Clients Connect to IP 1 Clients Connect to IP 2
ASG Node with IP 1 Node with IP 2 Clients Clients Clients Clients Clients Clients Clients Connect to IP 3 Node with IP 3 Clients Clients Clients -
EC2 Container Service Updates • ECR - best registry to
use for AWS container workloads • Support for volumes • CloudWatch metrics for CPU and memory utilization across the cluster (set alarms for autoscaling) • IAM roles for ECS tasks • Blox allows custom schedulers (github.com/blox/blox) • 3rd party tooling (Convox, Empire) • Integration with ALB • Run tasks on a schedule • Execute tasks in response to CloudWatch events -
Use the BurstBalance CloudWatch metric to monitor I/O credit balance
for gp2, st1, sc1 EBS volumes. -
Network throughput increases substantially with instance type. (Don't forget to
enable enhanced networking) -
Use ssh -D and the SwitchyOmega Chrome extension for convenient
access to services in a private network. -
Running multiple apps per instance? Use AssumeRole to assign granular
permissions to each app. -
Capture userdata output to a file. #!/bin/bash -x exec >
/var/log/userdata.log 2>&1