Share
Embed
Transcript
47 slides
-
simalexan Aleksandar Simovic Senior Software Engineer @ ScienceExchange AWS Serverless
Hero coauthor of “Serverless Applications with Node.js” book AWS SAM & Lambda Builders Contributor Co-organizer of JS Belgrade, Serverless Belgrade, Wardley Maps Belgrade meetups -
simalexan If relational databases solve all of our needs,
why do we need NoSQL? -
simalexan (1967-71) $1 Million per 1 MB $200 per
92k IPS Cost of Resources (2019) $0.02 per 1 GB $500 per 300 bill. IPS -
simalexan SQL NoSQL Relational Hierarchical (Denormalized) Vertical Scale Horizontal Scale
Queries Instantiated Views Maximize Storage Maximize compute (CPU) -
simalexan Serverless DB (No patches / updates) Amazon DynamoDB Consistent
and Fast (4M transc. / sec) Document or Key-Value Scales per Any Load Access control (Fine grained access table, items, attributes, values) Event Driven Model (Connected to AWS Lambda) -
simalexan • 2007, published paper • One of the authors
is Werner Vogels (CTO of Amazon) • Used by Amazon from day 1 Amazon e-commerce Shopping Cart • 2012, GA Built by AWS / Used By AWS -
simalexan • Hardware provisioning • Cross-availability zone replication (replicas whenever
needed) • Monitoring and handling of failures • Patches / updates / fixes Managed for you -
simalexan • First 25 GB stored per month is free
• $0.25 per GB-month • Write $1.25 / million req units • Read $0.25 / million req units Pay-per-use database! -
simalexan • Automatically partitioned • Partitions completely independent • No
limits on workloads Scaling -
simalexan • Single digit (1-9) millisecond Put / Get •
Custom SSD based platform - performance independent of table size - no need for working set to fit in memory Performance -
simalexan • “Replication? We don’t need stinking replication” • “Automated”
• “multi-region” • "multiple replica” tables (one per region that you choose) • DynamoDB treats as a single unit Global Tables -
simalexan • Scalar data types: - string, - number,
- binary • MultiValue data types (string set, number set, binary set) Data Types Available -
simalexan • Data Index by a Primary Key • Types
of primary keys: - partitionKey (hash) - partitionKey + range Indexing Scientist field year Marie Curie Chemistry 1911 Marie Curie Physics 1903 John Bardeen Physics 1956 Leonid Hurwicz Economics 2007 hash + range example -
simalexan Uniquely identifies a single item Partition Key Unordered hash
index Allows table partitioning for scale (chop up and throw in a storage node - automatic routing for service request) -
simalexan Two attributes to uniquely identify an item Partition Sort
Key Items arranged by the sort key No limits on the number of items per partition key -
simalexan Local index to a single partition key Local Secondary
Index Another sort key attribute (alternate range key) -
simalexan Country Scientist field year topic France Marie Curie Chemistry
1903 Radioactivity France Esther Duflo Economics 2019 Alleviating Global Poverty Get items country = “France” scientist begin_with “Marie” NO Local Secondary Index -
simalexan Get items country = “France” year > 2000
Local Secondary Index Country Scientist field year topic France Marie Curie Chemistry 1903 Radioactivity France Esther Duflo Economics 2019 Alleviating Global Poverty LSI Partition Key: scientist range: year -
simalexan Global Secondary Index (GSI) Index across all partitions
Alternate partition (+sort) key Use composite sort keys for compound indexes -
simalexan Artist Song Year of Release Album AC/DC Shoot to
Thrill 1980 Back in Black Florian Pellieser Quintet Coup de foudre a Thessalonique 2018 Coup de foudre a Thessalonique NO Global Secondary Index Get items artist = “AC/DC” order by song ASC -
simalexan Artist Song Year of Release Album AC/DC Shoot to
Thrill 1980 Back in Black Florian Pellieser Quintet Coup de foudre a Thessalonique 2018 Coup de foudre a Thessalonique Global Secondary Index Get items album = “Back In Black” order by song DESC GSI Partition Key: album range: song * this is why the GSI costs extra * -
simalexan Updating the Global Secondary Index (GSI) Update req
Async updates Table Client GSIs Response -
simalexan 1. Join two columns 2. Use “begins with” Composite
Key is_verified date TRUE 11-05-2019 TRUE 11-05-2019 FALSE is_verified_date TRUE_11-05-2019 TRUE_11-05-2019 FALSE Useful for multivalue filter (and sort sometimes) -
simalexan Limitations Item max size 400kb Max 5 LSI
Initial limit of 20 max GSI Max 32 levels deep Max 40,000 reads and 40,000 write req. units / table 256 tables per Account per AWS Region -
simalexan SQL vs NoSQL SELECT * FROM aircrafts INNER JOIN
bookings WHERE… SELECT * FROM aircrafts -
simalexan Use Case Context Modeling steps Rick Houlihan Review ->
Repeat -> Review Data modeling Avoid relational database patterns, use one table Access patterns • Read Write Workloads • Query dimensions and aggregations • Nature of the app (OLTP, OLAP, DSS) • ER Model • Data Lifecycle (TTL, Backup) • Data Sources • Query aggregations • Document all workflows • 1 application service = 1 table • Identify primary keys • Define indexes for secondary access patterns -
simalexan • As data enters a database, it can also
leave. • AWS DynamoDB Streams • API Gateway -> DynamoDB DynamoDB -> Lambda Event Driven Model -
simalexan Some Use cases Duolingo Amazon Shopping Cart CapitalOne Nordstrom
Snapchat Nike GE Aviation Lyft Netflix Samsung -
simalexan When (not) to use DynamoDB Real-time analytics and queries
Complex queries and joins Bad use cases Good use cases Very high read/write Key-value simple queries Consistently low-latency Autosharding / multiple node scaling No tuning No size throughout put limits OLTP OLAP