Unlock the Value of Your Data with Apache Pinot and AWS (JP Santana & Wahab Syed, AWS) | RTA Summit 2023

A data-centric culture is critical for innovation and growth. Learn how Apache Pinot works with AWS services to unlock the value of your data and gain insights that can help you make smarter, data-driven decisions, no matter where your data lives.

Transcript

1. © 2023, Amazon Web Services, Inc. or its Affiliates. All rights reserved. JP Santana Principal Startup SA Unlock the value of your data with Apache Pinot and AWS Wahab Syed Startup SA
2. © 2023, Amazon Web Services, Inc. or its Affiliates. All rights reserved. What we’re going to cover • Breadth & Depth of AWS services for RT Analytics • Options for running Apache Pinot on AWS • Demo – Apache Pinot on Amazon ECS
3. © 2023, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Breadth & Depth of AWS services for RT Analytics
4. © 2023, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Building streaming data pipelines is easier with AWS Processing Ingestion and storage Amazon Kinesis Data Streams Amazon Managed Streaming for Apache Kafka Apache Pinot Batch Sources IOT sensors Enterprise apps Social media Logs Data lakes Amazon S3
5. © 2023, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Kinesis Data Streams Kinesis Data Analytics Kinesis Video Streams Kinesis Data Firehose Amazon Kinesis COLLECT, PROCESS, AND ANALYZE VIDEO AND DATA STREAMS IN REAL TIME
6. © 2023, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Compatible Fully managed Highly available Secure Kafk a Amazon MSK FULLY MANAGED, HIGHLY AVAILABLE, AND SECURE
7. © 2023, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Built to store and retrieve any amount of data Supports Apache Pinot Auto Ingest and Batch Import Unmatched durability, availability, and scalability Amazon S3 Amazon S3
8. © 2023, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Customer references Online stylist processing 100 million events per day Facilitate communications between 100+ microservices 1 billion events per day from connected devices Migrated data bus to Amazon Kinesis Near real-time home valuation (Zestimates) Billions of events per day from TVs and connected devices 10 TB per day clickstreams from 250+ sites Live clickstream dashboards refreshed under 10s IoT predictive analytics 50 billion daily ad impressions, sub-50 ms responses NORDSTRO M
9. © 2023, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Options for running Apache Pinot on AWS
10. © 2023, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Control Plane Deployment, Scheduling, Scalability & management Data Plane Where containers run Amazon Elastic Container Service Amazon Elastic Container Service for Kubernetes Amazon EC2 AWS Fargate Image Repo Container Image Repository Amazon Elastic Container Registry Container Services on AWS
11. © 2023, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon ECS and Amazon EKS EKS ECS Powerful simplicity Open flexibility
12. © 2023, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Powerful simplicity AWS-opinionated way to run containers at scale Reduce decisions without sacrificing scale or features Reduce time to build, deploy, and migrate applications ECS
13. © 2023, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Open flexibility Gain agility and efficiency with AWS-optimized Kubernetes, and standardize operations everywhere Secure, highly available, with observability across all Kubernetes deployments Build with choice of solutions from the broader community around Kubernetes EKS
14. © 2023, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Serverless Containers Fargate No need to manage compute provisioning, management, and scalability Scale easily and pay only for what you use Natively integrated with VPC, ELB, IAM, CloudWatch, and more
15. © 2023, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Demo: Apache Pinot on Amazon EKS
16. © 2023, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ADD DEMO RECORDING
17. © 2023, Amazon Web Services, Inc. or its Affiliates. All rights reserved. References for Apache Pinot on AWS • Apache Pinot on AWS Quickstart https://docs.pinot.apache.org/basics/getting-started/public-cloud -examples/aws-quickstart • Ingest events from an Amazon Kinesis stream into Pinot https://docs.pinot.apache.org/basics/data-import/pinot-stream-in gestion/amazon-kinesis • JSON Batch Import and Auto Ingest from S3 into Apache Pinot https://youtu.be/1EMBx1XeI9o https://youtu.be/fXraQygBzxg
18. © 2023, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ! Please, don’t forget to rate our session in the app!
19. © 2023, Amazon Web Services, Inc. or its Affiliates. All rights reserved. JP Santana linkedin.com/in/jpsantana Wahab Syed linkedin.com/in/wahab-io Thank you!