2018 - The Future of Storage

Presentation to a group of high level Dell/EMC technology executives (substituting for Carl Ledbetter)

1. The Future of Storage Tom Lyon DriveScale, Inc.
2. Who Is Tom Lyon?  Not Carl Ledbetter  Co-Founder and Chief Scientist, DriveScale  Distinguished Engineer, Cisco  Co-Founder and Chief Scientist, Nuova Systems [-> Cisco UCS]  CTO, Nokia Internet Communications  Founder & CTO, Ipsilon Networks [-> Nokia IP Routing & Security]  Distinguished Engineer, Sun Microsystems [employee #8]  Systems Architect, Amdahl Corp.  BSE in EECS, Princeton Univ.  Hacking on UNIX since 1975  UNIX, UTS, SunOS, NFS, SPARC, SunLink, IP Switching, ATM, UCS, VFIO … 2 ©2018 DriveScale Inc. All Rights Reserved.
3. ©2018 DriveScale Inc. All Rights Reserved. What is DriveScale? 3 DAS Composable DriveScale was born to radically transform how modern applications are deployed in the data center. Seeing how hyperscale companies were reaping the benefits of analytics and AI with massive data sets on their scale-out infrastructure, DriveScale set out to make it viable for any enterprise to deploy this infrastructure on- premises and reap the same reward. DriveScale designed a software platform that creates a way to run these same data-intensive applications unchanged with cloud scale and efficiency at a significantly lower cost.
4. What is Storage?  Storage Systems are: – Lots of software – Servers - commoditized – Devices (HDDs, SSDs) - commoditized  Very rare to have any hardware differentiation  Everything is scale-out / distributed  Disaggregation/Unbundling of these 3 components is the trend – Software-defined storage – Customer gets better choices, efficiency, and flexibility 4 ©2018 DriveScale Inc. All Rights Reserved.
5. Storage Software Evolution  It used to be just blocks or files (SAN/NAS); object storage is exploding – Blocks are for machine – physical and virtual – Files are for people and POSIX – Objects are for network-aware applications  Application Specific Storage Software optimizes for particular use cases – Scale-Out Filesystem, Databases – Moving Compute to the Data (Hadoop) – Storage Services above this level, not below 5 ©2018 DriveScale Inc. All Rights Reserved.
6. Software Best Practices  We’ve learned a lot about distributed software in the last 20 years – State-less micro-services, scale-out, “cloud native” – CI/CD frameworks – CAP Theorem – Concurrency-oriented languages (Go, Erlang) – PAAS, FAAS, Serverless – Cloud Foundry, Kubernetes  What about “State-Full” services?? 6 ©2018 DriveScale Inc. All Rights Reserved.
7. All Software is Stateless  The state is in the storage device – HDDs, SSDs provide the only state-full services  Devices and software captive to a server make the service act state- full  Storage software should be built, bought, and operated just like other software  Monoliths must die  Look at Minio, OpenEBS – micro-storage for micro-services 7 ©2018 DriveScale Inc. All Rights Reserved.
8. Business/Culture Issues: Systems vs Software  Storage Vendors prefer to sell systems, not software  Storage Buyers prefer to buy systems, not software  Worst of all – people expect 1 storage solution to support all workloads  CIOs demand software; the “grunts” still buy the system  Appliances are anti-cloud – Inherent silos, no re-purposing – Software shouldn’t care where it runs – Multi-vendor hardware integration is so much easier than yester- year 8 ©2018 DriveScale Inc. All Rights Reserved.
9. Appliances vs Storage Servers vs Network-Attached Devices  Storage Devices need to live in a chassis somewhere  Used to be bundled with software in appliance  New class of servers: storage servers  But storage devices are too big & costly to be captive to individual servers – Blast-radius problem  Network-Attached Devices – “appliance” to connect devices to servers via network – Highly redundant connectivity/controllers – No “storage” software 9 ©2018 DriveScale Inc. All Rights Reserved.
10. Storage Interface Evolution  Internal – ATA/SCSI -> SATA/SAS -> NVMe (PCIe)  External – SCSI -> Fibre Channel -> iSCSI(Ethernet) -> NVMe Over Fabrics(RDMA) -> NVMe Over TCP  NVMe Over Fabrics/TCP is a huge enabler of disaggregation  But disaggregation requires recomposition: Composable Infrastructure 10 ©2018 DriveScale Inc. All Rights Reserved.
11. Composable Infrastructure: Definition 11 ©2018 DriveScale Inc. All Rights Reserved.
12. Composable Infrastructure: Large System Vendors  Cisco started it, but dropped the ball  HPE is promoting it with Synergy blade system  DellEMC entering market – PowerEdge MX composable system – Kinetic Infrastructure  WDC – OpenFlex Composable Infrastructure 12 ©2018 DriveScale Inc. All Rights Reserved.
13. Composable Infrastructure: Startups  DriveScale: “Software Composable Infrastructure for Your Data Center” – HDD, SSD, NVMe  Liqid: “On-Demand Composable Infrastructure” – NVMe & GPU  H3Platform: NVMe and GPU disaggregation – NVMe & GPU  Other NVMe over Fabrics: – Apeiron, Attala, Excelero, E8 Storage, Pavilion Data 13 ©2018 DriveScale Inc. All Rights Reserved.
14. Thank You!