Upgrade to Pro — share decks privately, control downloads, hide ads and more …

2018 Ceph Day: "NVMe Over Fabrics and Composable Infrastructure"

Tom Lyon
September 21, 2018

2018 Ceph Day: "NVMe Over Fabrics and Composable Infrastructure"

Presentation at Ceph Day 2018: "NVMe Over Fabrics and Composable Infrastructure - What Do They Mean for Software-Defined Storage?"

Tom Lyon

September 21, 2018
Tweet

More Decks by Tom Lyon

Other Decks in Technology

Transcript

  1. NVMe over Fabrics and
    Composable Infrastructure –
    What do they mean for SDS?
    Tom Lyon
    DriveScale, Inc.

    View full-size slide

  2. Who Is Tom Lyon?
    §  Co-Founder and Chief Scientist, DriveScale
    §  Distinguished Engineer, Cisco
    §  Co-Founder and Chief Scientist, Nuova Systems [-> Cisco UCS]
    §  CTO, Nokia Internet Communications
    §  Founder & CTO, Ipsilon Networks [-> Nokia IP Routing & Security]
    §  Distinguished Engineer, Sun Microsystems [employee #8]
    §  Systems Architect, Amdahl Corp.
    §  BSE in EECS, Princeton Univ.
    §  Hacking on UNIX since 1975
    §  UNIX, UTS, SunOS, NFS, SPARC, SunLink, IP Switching, ATM, UCS, VFIO …
    2
    ©2018 DriveScale Inc. All Rights Reserved.

    View full-size slide

  3. Why Software-Defined Storage?
    §  SANs originally provided separation of compute & storage resources
    –  This is a very good thing – consolidation, simpler servers, …
    §  But SANs now are $$$$$! and don’t scale
    §  Centralized storage creates network bottlenecks
    §  SAN hardware evolved to just being servers anyways
    §  But:
    –  there is no right size for an SDS server/storage combo
    –  Every cluster needs different cpu/storage ratios
    –  CPUs and drives have different “natural” life-cycles
    3
    ©2017 DriveScale Inc. All Rights Reserved.

    View full-size slide

  4. NVMe enabled SSDs
    §  Crazy fast SSDs:
    –  500K IOPS
    –  3GB/s
    §  But:
    –  Bottleneck shifts to software
    –  Hard to buy small & fast SSDs
    –  2TB and rising for good performance
    §  NVMe itself is just an interface protocol enabling deep & parallel queues
    4
    ©2017 DriveScale Inc. All Rights Reserved.

    View full-size slide

  5. NVMe Over Fabrics
    §  Substitute network for PCIe – keep NVMe basics
    §  Fibre Channel or RDMA networks
    –  Ethernet RoCE is leader; iWarp has lost out; IB still niche
    –  NVMe over TCP coming because RDMA is “very difficult”
    –  Typically for 25Gb Ethernet and beyond
    §  NVMe-o-F aspects:
    –  Data Plane – highly standardized
    –  Control Plane - some standardization
    –  Manageability aspects – don’t hold your breath
    5
    ©2017 DriveScale Inc. All Rights Reserved.

    View full-size slide

  6. NVMe-o-F Use Cases
    §  1. Just another protocol between servers & SAN
    –  Esp NVMe on FC
    §  2. Protocol embedded in backend network of storage system
    –  NVMe devices not visible to general network
    §  3. Protocol for direct dis-aggregation of drives from servers
    –  Target devices are real network citizens
    –  Discovery, security, monitoring, … are very important
    6
    ©2017 DriveScale Inc. All Rights Reserved.

    View full-size slide

  7. NVMe-o-F Targets
    §  Of course, x86 servers can be NVMe targets
    §  But lots of new SOC chips for more efficient hardware:
    –  Broadcom Stingray
    –  Mellanox Bluefield
    –  Kazan – pure hardware data path
    §  Redundant controllers needed to avoid new SPOFs
    §  So software needs to live on the server, not on the target
    §  Servers become stateless, homogeneous, re-deployable
    7
    ©2017 DriveScale Inc. All Rights Reserved.

    View full-size slide

  8. Server Disaggregation
    §  If SSDs outside the server make sense, why not
    –  Hard Drives
    –  GPUs?
    –  DRAM?
    §  Disaggregation requires re-Composition
    8
    ©2017 DriveScale Inc. All Rights Reserved.

    View full-size slide

  9. Composable Infrastructure: Definition
    9
    ©2018 DriveScale Inc. All Rights Reserved.

    View full-size slide

  10. Composable Infrastructure: Large System Vendors
    §  Cisco started it, but dropped the ball
    §  HPE is promoting it with Synergy blade system
    §  DellEMC entering market
    –  PowerEdge MX composable system
    –  “Kinetic Infrastructure”
    §  WDC – OpenFlex Composable Infrastructure
    10
    ©2018 DriveScale Inc. All Rights Reserved.

    View full-size slide

  11. Composable Infrastructure: Startups
    §  DriveScale: “Software Composable Infrastructure for Your Data Center”
    –  HDD, SSD, NVMe - Ethernet
    §  Liqid: “On-Demand Composable Infrastructure”
    –  NVMe & GPU – External PCIe
    §  H3Platform: NVMe and GPU disaggregation
    –  NVMe & GPU – External PCIe
    §  Other NVMe over Fabrics:
    –  Apeiron, Attala, Excelero, E8 Storage, Pavilion Data, …
    11
    ©2018 DriveScale Inc. All Rights Reserved.

    View full-size slide

  12. ©2018 DriveScale Inc. All Rights Reserved.
    What is DriveScale?
    12
    DAS
    Composable
    DriveScale was born to radically transform how modern
    applications are deployed in the data center.
    Seeing how hyperscale companies were reaping the
    benefits of analytics and AI with massive data sets on their
    scale-out infrastructure, DriveScale set out to make it
    viable for any enterprise to deploy this infrastructure
    on- premises and reap the same reward.
    DriveScale designed a software platform that creates a
    way to run these same data-intensive applications
    unchanged with cloud scale and efficiency at a
    significantly lower cost.

    View full-size slide

  13. DriveScale Composability Platform
    13
    Bare Metal
    DriveScale Composability Platform
    PCIe
    Ethernet
    iSCSI
    RoCE v2 NVME/TCP
    Rack Servers
    HDD SSD NVMe
    Blade Servers
    Modular
    Servers
    GenZ
    NoSQL

    View full-size slide