Share
Embed
How to select the righ EBS?
How to select the right EBS? by Ahmed Shendi during AWS MENA community days 2020.
AWS MENA Community
September 26, 2020
Transcript
43 slides
-
COMMUNITY DAY MENA Agenda What is Amazon EBS Amazon EBS
Volume Types Choosing an Amazon EBS volume type Performance Best Practices for EBS Saving Cost with EBS -
COMMUNITY DAY MENA What is Amazon EBS? o Block storage
as service o Create, attach volumes through an API o Service accessed over network o Volumes persist independent of EC2 o Detach and attach between instances within the same availability zone AWS Cloud Availability Zone NAT gateway Instance Instance Amazon EC2 Auto Scaling Amazon EC2 Amazon Elastic Block Store -
COMMUNITY DAY MENA Amazon EBS Volume Types SSD HDD gp2
General Purpose SSD io1 Provisioned IOPS SSD st1 Throughput Optimized HDD sc1 Cold HDD -
COMMUNITY DAY MENA Amazon EBS volume: General Purpose SSD Baseline:
100 to 16,000 IOPS @ 3 IOPS/GB Burst: 3,000 IOPS (< 1000 GB) Throughput: Up to 256 MiB/s Latency: Single-digit ms Capacity: 1 GiB to 16 TiB I/O Size: Up to 256 KiB (logical merge) gp2 General Purpose SSD Great for boot volumes, low-latency applications, and bursty databases -
COMMUNITY DAY MENA How long can I burst on gp2?
0 0 150 300 450 600 750 Minutes of Burst 200 400 600 800 Volume size in GiB -
COMMUNITY DAY MENA Amazon EBS volume: Provisioned IOPS Baseline: 100
– 64,000 IOPS Throughput: Up to 1000 MiB/s Latency: Single-digit ms Capacity: 4GiB – 16 TiB I/O Size: Up to 256 KiB (logical merge) io1 Provisioned IOPS Ideal for critical applications and databases with sustained IOPS -
COMMUNITY DAY MENA Scaling Provisioned IOPS SSD : io1 0
0 40,000 20,000 30,000 10,000 50,000 64,000 IOPS 1.28 2 6 9 16 -
COMMUNITY DAY MENA Amazon EBS volume: Throughput Optimized Baseline: 40
MiB/s per TiB, up to 500 MiB/s Burst: 250 MiB/s per TiB, up to 500 MiB/s Capacity: 500 GiB to 16 TiB I/O Size: Up to 1 MiB (logical merge) st1 Throughput Optimized HDD Ideal for large-block, high-throughput sequential workloads -
COMMUNITY DAY MENA Amazon EBS volume: Cold HDD Baseline: 12
MiB/s per TiB, up to 192 MiB/s Burst: 80 MiB/s per TiB, up to 250 MiB/s Capacity: 500 GiB to 16 TiB I/O Size: Up to 1 MiB (logical merge) sc1 Cold HDD Ideal for sequential throughput workloads, such as logging and backup -
COMMUNITY DAY MENA Understand your mission Databases PostgreSQL, MySQL Cassandra,
MongoDB Data & Analytics Kafka, Splunk, Hadoop, Data Warehousing Media Transcoding, Encoding, Render Farms File CIFS, NFS Archive -
COMMUNITY DAY MENA Select the right volume • Typically high
performance requirements • Mostly random I/O • Journal is sequential • Highly workload dependent Recommendation: SSD Databases PostgreSQL, MySQL Cassandra, MongoDB -
COMMUNITY DAY MENA Select the right volume • Typically high
throughput requirements • Mostly sequential I/O • Sustained I/O Recommendation: Throughput Optimized HDD (st1) Media Transcoding, Encoding, Render Farms -
COMMUNITY DAY MENA Select the right volume • Typically high
throughput requirements • Mostly sequential I/O • Daily periodicity Recommendation: HDD Data & Analytics Kafka, Splunk, Hadoop, Data Warehousing -
COMMUNITY DAY MENA Select the right volume • Typically low
throughput requirements • Bursty workloads • Cost sensitive Recommendation: Cold HDD (sc1) File CIFS, NFS Archive -
COMMUNITY DAY MENA How do AWS count I/O When possible,
AWS logically merge sequential I/O, to minimize I/O charges on io1, and maximize burst on gp2, sc1, and st1 io1 and gp2 : up to 256 KiB st1 and sc1: up to 1 MiB -
COMMUNITY DAY MENA How do AWS count I/O Example 1:
Random I/Os: • 4 random I/Os (non-sequential) • Each I/O 64 KiB -
COMMUNITY DAY MENA How do AWS count I/O Example 2:
Sequential I/Os: • 4 sequential I/Os • Each I/O 64 KiB -
COMMUNITY DAY MENA How do AWS count I/O Example 2:
Sequential I/Os: • 4 sequential I/Os • Each I/O 64 KiB -
COMMUNITY DAY MENA How do AWS count I/O Example 2:
Sequential I/Os: • 4 sequential I/Os • Each I/O 64 KiB -
COMMUNITY DAY MENA How do AWS count I/O Example 3:
Large I/O: • 1 I/O • 1024 KiB -
COMMUNITY DAY MENA How do AWS count I/O Example 4:
Mixed I/Os: • 2 * 512 KB sequential I/O s • 2 * 64 KB random I/O s • 2 * 128 KB sequential I/O s -
COMMUNITY DAY MENA EBS Optimized • Dedicated EBS bandwidth •
Up to 19 Gbps (2,375 MiB/s) per instance • Burst available for smaller instances Tip: Ensure EBS optimized bandwidth can support your volumes -
COMMUNITY DAY MENA EBS Optimized • Dedicated EBS bandwidth •
Up to 19 Gbps (2,375 MiB/s) per instance • Burst available for smaller instances Tip: Ensure EBS optimized bandwidth can support your volumes -
COMMUNITY DAY MENA Best practices: RAID When to RAID? -
Storage requirements > 16 TiB - Throughput requirement > 1,000 MiB /s - IOPS requirement > 64,000 -
COMMUNITY DAY MENA Best practices: RAID Avoid RAID for redundancy
• EBS data is already replicated • RAID 1 halves available EBS bandwidth • RAID 5 / 6 loses 20 % to 30% of usable I/O to parity -
COMMUNITY DAY MENA Monitor I/O characteristics using CloudWatch You can
monitor the I/O characteristics with each volume's CloudWatch volume metrics. Important metrics to consider include the following: • BurstBalance • VolumeReadBytes • VolumeWriteBytes • VolumeReadOps • VolumeWriteOps • VolumeQueueLength BurstBalance: displays the burst bucket balance for gp2, st1, and sc1 volumes as a percentage of the remaining balance To determine your volume's average I/O size, divide VolumeWriteBytes by VolumeWriteOps. The same calculation applies to read operations. If your I/O latency is higher than you require, check VolumeQueueLength to make sure your application is not trying to drive more IOPS than you have provisioned. -
COMMUNITY DAY MENA EBS Cost gp2 $0.10 per GiB io1
$0.125 per GiB $0.065 per PIOPS st1 $0.045 per GiB sc1 $0.025 per GiB Snapshot storage for all volume types is $0.05 per GiB per month * All prices are per month, prorated to the second, and from the us-west-2 Region as of Nov 2019 -
COMMUNITY DAY MENA Set DeleteOnTermination = TRUE if storage lifecycle
= instance lifecycle DeleteOnTermination = FALSE DeleteOnTermination = TRUE USE WITH CARE – You might delete volumes you need -
COMMUNITY DAY MENA Tier to lower cost volume type if
volume is not in use -
COMMUNITY DAY MENA Don't use io1 unless you are sure
that you need it