Concept and Design of SDN-enhanced MPI Framework

My talk at EWSDN2015

Transcript

1. Concept and Design of SDN- enhanced MPI Framework Keichi Takahashi1, Dashdavaa Khureltulga1, Baatarsuren Munkhdorj1, Yoshiyuki Kido1, Susumu Date1, Hiroaki Yamanaka2, Eiji Kawai2 
 and Shinji Shimojo2
 Osaka University1, National Institute of Information Communications Technology2
2. • Modern HPC (High Performance Computing) systems are built on a computer cluster architecture. • Due to the recent rapid scale-out in node number,
 the cost and complexity of HPC interconnects are getting increasingly high. • MPI (Message Passing Interface) is the de facto standard library to develop parallel distributed applications on clusters. Interconnect of HPC systems 2
3. Towards SDN-enhanced MPI 3 SDN$enabled+Interconnect
   Compu5ng+ Nodes
   MPI+ Applica5on
   MPI+Library
   MPI+ Applica5on
   MPI+Library
   MPI+ Applica5on
   MPI+Library
   Applica5on+Informa5on
   MPI+Applica5on$ aware+control+of+ interconnect
   Network+Informa5on
   Leverage SDN architecture to realize tight integration between MPI application and network interconnect; MPI application- aware dynamic interconnect control. Several PoCs: • Offloading MPI broadcast to the network • Dynamic load balancing of traffic load using MPI-layer
 information
4. Proposed Architecture of the Framework 4 SDN$Controller
   SDN$Switch
   MPI$ Applica6on
   MPI$Library
   Tagging$ Module
   MPI$Packet
   Tagged$MPI$Packet
   (3)$Packet$control$based$on$tags
   (1)$PreDinstall$rules
   Untagged$M PI$Packet
   (2)$Packets$are$tagged$with$ applica6on$level$info
   MPI$ Library
   Network$ Stack
   User$MPI$ Applica8on
   Tagging$ Module
   MPI$Packet
   MPI$Packet
   User$Space
   Kernel$Space
   Socket$API
   System$Call
   MPI
   To$NIC
   MPI$Packet
   Tagging Mechanism Overall Architecture