Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Database Migration Journey By Diaa Radwan

Database Migration Journey By Diaa Radwan

Database Migration Journey By Diaa Radwan

AWS MENA Community

September 25, 2020
Tweet

More Decks by AWS MENA Community

Other Decks in Technology

Transcript

  1. Agenda Modernize and break free Database Migration Patterns AWS migration

    partners Database freedom program AWS migration resources Q & A
  2. Rapid expansion of data requirements Explosion of data Microservices change

    data and analytics requirements Accelerated rate of change driven by DevOps Data grows 10x every 5 years driven by network- connected smart devices Microservices architecture decreases need for “one size fits all’ databases and increases need for real-time monitoring and analytics Dev Ops Transition from IT to DevOps increases rate of change
  3. Hardware and software installation Database configuration, patching, and backups Cluster

    setup and data replication for high availability Capacity planning, and scaling clusters for compute and storage Managing databases on-premises: Time-consuming and complex
  4. You You Fully managed services on AWS Spend time innovating

    and building new applications, not managing infrastructure AWS Self-managed Fully managed Schema design Query construction Query optimization Automatic failover Backup and recovery Isolation and security Industry compliance Push-button scaling Automated patching Advanced monitoring Routine maintenance Built-in best practices
  5. Broad database and analytics services portfolio Relational databases Nonrelational databases

    Data warehouses Hadoop and Spark Amazon Redshift Amazon EMR Operational analytics Amazon Elasticsearch Service Amazon Aurora Amazon DynamoDB Business Intelligence Amazon QuickSight Amazon RDS Amazon DocumentDB Amazon ElastiCache Real-time analytics Amazon Managed Streaming for Apache Kafka PostgreSQL Logstash Elasticsearch Kibana Microsoft SQL Server Microsoft SQL Server
  6. Amazon Relational Database Service (RDS) Managed relational database service with

    a choice of six popular database engines Easy to administer Available and durable Highly scalable Fast and secure No need for infrastructure provisioning, installing, and maintaining DB software Automatic Multi-AZ data replication; automated backup, snapshots, failover Scale database compute and storage with a few clicks with no app downtime SSD storage and guaranteed provisioned I/O; data encryption at rest and in transit
  7. Amazon RDS database engines Commercial Open source Cloud native Amazon

    Elastic Block Store (Amazon EBS)-based storage Amazon Aurora storage system MySQL compatible PostgreSQL compatible
  8. Rehost Automate Manual Install Config Deploy App code development Refactor

    Redesign ALM/SDLC Integration Migration strategies Retire Retain Re-platform Determine platform Modify infrastructure Determine Discover VALIDATION Transition Production Assess/ Prioritize Repurchase Buy COTS/SaaS Install/setup Relocate (VMware Cloud on AWS) Automate Use migration tools
  9. Discover the right migration pattern for your apps Disconnected and

    incomplete data Discover and organize data 7 Rs Applications Performance Infrastructure Level of effort Retire Retain Relocate Rehost Repurchase Re-platform Refactor Tribal knowledge SLA/OLA App configuration data Asset inventories CMDB Architecture Performance Information Automation and guidance AWS Application Discovery Service Amazon Athena
  10. Database Freedom program: Migration workflow Portfolio discovery Migration Workload analysis

    and pilot ASSESSMENT ANALYSIS AND PILOT WORKLOAD MIGRATIONS Directional business case Rapid discovery Landing zone Skills/CoE Discovery & planning Migration expertise Migration business case Migration plan Migrate Integrate Test Transform Monitor Optimize
  11. Which option is right for you? Rehost: Run servers on

    Amazon Elastic Compute Cloud (Amazon EC2) Replatform: Migrate to a managed service Amazon Relational Database Service (Amazon RDS) Refactor: Modernize application/migrate to cloud-native solution Amazon Elastic Kubernetes Service (Amazon EKS), AWS Lambda, Amazon DynamoDB, and others)
  12. Rehost: Run servers on Amazon EC2 • Familiar administration experience

    • Full control over the environment • COTS applications • For database servers • All features available • All versions supported • Full administrative access Amazon EC2
  13. Typical three tier: Rehost Presentation tier Application tier Database tier

    Presentation tier Application tier Database tier AWS DMS Rehost
  14. Replatform: Migrate to a managed service • Optimized architecture •

    Automated patching • Automated backups • Proven high availability • COTS applications AWS Directory Service Amazon RDS Amazon FSx Replatform
  15. AWS Cloud AWS Region VPC Availability zone 1 Availability zone

    2 Generic database Generic database Generic database On-Premises Replatform Replatform: Databases
  16. Refactor: Modernize app/adopt cloud-native services • Containers/microservices • Amazon EKS

    • Amazon Elastic Container Service (Amazon ECS) • AWS Lambda • Amazon API Gateway • Aurora: SQL/OLTP • DynamoDB: NoSQL AWS Lambda Amazon Elastic Kubernetes Service Amazon Elastic Container Service Amazon API Gateway Amazon Aurora Amazon DynamoDB
  17. Typical three tier: Refactor Presentation tier Application tier Database tier

    Presentation tier Application tier Database tier AWS DMS Amazon API Gateway AWS Lambda Amazon Aurora Amazon DynamoDB Amazon Simple Storage Service (Amazon S3) Containers Containers
  18. Database Migration Service & Schema Conversion Tool AWS SCT Development

    tooling to convert schemas between databases and data warehouses Native tooling Migration options included with the engine Open source Ora2Pg and others Commercial Attunity, Golden Gate, Informatica, Talend, and others Benefit • Automates schema conversion including database structure and code • Minimizes manual effort of performing a schema re-write • Allows conversion from commercial to open source platforms Benefit • Leverage a familiar environment • Full support for native features Benefit • Tailored solution for a specific problem • Multiple tools can offer a more complete solution than any one product • Free Benefit • Commercial solutions are available for a wide range of migration sources and targets including legacy mainframe systems AWS DMS Database migration and replication managed service Benefit • Easily and securely migrate and/or replicate your databases and data warehouses to AWS • Migrate between different database engines • Low cost and global availability
  19. AWS Database Migration Service & AWS Schema Conversion Tool AWS

    Database Migration Service (AWS DMS) easily and securely migrates and/or replicates your databases and data warehouses to AWS AWS Schema Conversion Tool (AWS SCT) converts your commercial database and data warehouse schemas to open-source engines or native AWS services, such as Amazon Aurora and Amazon Redshift Modernize Migrate Replicate
  20. Flexible, powerful migration tooling Most sources and targets, higher conversion

    automation AWS DMS AWS SCT For heterogeneous migrations, such as: • Oracle to Aurora PostgreSQL or Aurora MySQL • Cassandra to Amazon DynamoDB
  21. © 2017, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. AWS Schema Conversion Tool (AWS SCT) • Client-side tool • Database schema and code conversion service • OLTP schema or data warehouse schema • Converts embedded SQL statements too • Creates a conversion report • Helps choose the best target engine
  22. © 2017, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. AWS SCT Assessment Report • Assessment of migration compatibility of source databases with open-source database engines – RDS MySQL, RDS PostgreSQL and Aurora • Recommends best target engine • Provides details level of efforts to complete migration
  23. Database migration process Step 1: Convert or copy your schema

    Source DB or DW AWS SCT Native tool Destination DB or DW Step 2: Move your data Source DB or DW AWS SCT Destination DB or DW AWS DMS Copy or convert Copy DB DW
  24. © 2017, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. AWS DMS Components • Replication instance • Tasks • Endpoints Source Target
  25. Customer Premises Application Users AWS Internet VPN • Start Replication

    Instance • Connect to Source & Destination • Select Tables, Schemas or Databases Keep your apps running during the migration AWS Database Migration Service • Let DMS create tables, load data, and keep them in sync • Switch applications to new target at your convenience
  26. © 2017, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. What else I can do? – Fan In Source Target Source Source
  27. © 2017, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. What else I can do? – Fan Out Source Target Target Target
  28. © 2017, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. What else I can do? – Take it all or not Source Target
  29. © 2017, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. What else I can do? SQL Server MySQL Oracle Oracle Oracle Aurora
  30. © 2017, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. General Workflow—Full Load + CDC • Migrate schema with SCT • Drop all FK constraints/indexes on target • Initiate full load using DMS • Stop task (can do this automatically with task settings) • Create secondary indexes • Resume task to apply cached changes • Enable foreign keys • Start change data capture
  31. Legacy databases to AWS migration playbooks Resources to help guide

    your migration • Topic-by-topic overview of how to migrate databases and data warehouses to AWS services • Covers all proprietary features and the different database objects • Migration best practices • Oracle to Aurora PostgreSQL • SQL Server to Aurora MySQL • SQL Server to Aurora PostgreSQL • Oracle to Aurora MySQL • Cassandra to DynamoDB, data warehouses to Amazon Redshift, and Db2 LUW to Aurora PostgreSQL AWS SCT AWS DMS Playbook Schema Data Best practices
  32. Learn more Purpose-built databases aws.amazon.com/products/databases/ AWS Database Migration Service resources

    aws.amazon.com/dms/resources/ Database Freedom aws.amazon.com/solutions/databasemigrations/database- freedom/ AWS Database Migration Service Partners aws.amazon.com/dms/partners/ Database Migrations Resources aws.amazon.com/solutions/database-migrations/resources/
  33. Learn databases with AWS Training and Certification 25+ free digital

    training courses cover topics and services related to relational and nonrelational databases Resources created by the experts at AWS to help you build and validate database skills Validate expertise with the AWS Certified Database – Specialty exam The classroom offering, Planning and Designing Databases on AWS, features AWS expert instructors and hands-on activities Visit the databases learning path at aws.amazon.com/training/path-databases