The MLIR Framework: A Brief Introduction to Deep Learning Compilers

Deep learning defines a new programming paradigm. No more coding of predefined algorithmic rules, but rather defining architectures that will allow the data itself to carve the pathways to our desired model.

In addition, Neural Networks train on huge amounts of data that needs to flow through the system as quickly as possible. This demand sets a high bar for hardware performance, and is done by dedicated supercomputers.

This programming paradigm, alongside the dedicated types of hardware creates a new set of problems for compiler engineers. New computing systems require new tools, one of them is MLIR. This talk will introduce MLIR, an open source framework for building compiler infrastructure. There is no need for prior knowledge in compilation or deep learning, just some curiosity and a desire to dive deep.

Transcript

1. The MLIR Framework A Brief Introduction to Deep Learning Compilers Dafna Mordechai 2021
2. Hello! @Dafna_Mordechai • Dafna Mordechai, BSc. in Computer Science, The Hebrew University of Jerusalem (2008) • Senior RT Embedded Software Engineer • LOVE technology, and LOVE sharing it with others
3. AGENDA The LLVM Compiler Deep Learning MLIR
4. The LLVM Compiler
5. Compiler(Source Code) --> Machine Code C/C++ Fortran Rust Swift Go X86 ARM RISC-V AVR PowerPC • • • • • • • • • • Translations + Transformations [semantic preserving] Compiler == ( )
6. C/C++ Fortran Rust Swift Go X86 ARM RISC-V AVR PowerPC LLVM - Low Level Virtual Machine
7. C/C++ Fortran Rust Swift Go X86 ARM RISC-V AVR PowerPC • • • • • • • • • • Hardware agnostic operations Language agnostic operations
8. C/C++ Fortran Rust Swift Go X86 ARM RISC-V AVR PowerPC Front-End Clang / Flang / Gollvm / rustc / … Back-End LLVM - An Open Source Modular Compiler Hardware agnostic operations Language agnostic operations Examples: • Machine instruction selection • Register allocation • Scheduling Examples: • Lexical analysis • Syntax analysis • Semantic analysis
9. C/C++ Fortran Rust Swift Go X86 ARM RISC-V AVR PowerPC Front-End Back-End LLVM - An Open Source Modular Compiler Hardware agnostic operations Language agnostic operations ?
10. C/C++ Fortran Rust Swift Go X86 ARM RISC-V AVR PowerPC LLVM-IR - Standardized Intermediate Representation Hardware agnostic operations Language agnostic operations Middle-End LLVM-IR Standardized IR Front-End High Level Language to LLVM-IR Back-End LLVM-IR to Machine Code
11. ➔ Three-address code ➔ Typed ➔ Static single assignment (SSA) form LLVM-IR - Standardized Intermediate Representation
12. C/C++ Fortran Rust Swift Go X86 ARM RISC-V AVR PowerPC Back-End LLVM - Three-Stage Compiler Hardware agnostic operations Language agnostic operations Front-End Middle-End LLVM Optimizer Examples: • Constant folding • Loop-invariant code motion • Dead code elimination
13. LLVM - Low Level Virtual Machine Three-Stage Compiler ➢ Frontend ➢ Middle-end ➢ Backend ‐ Modularity ‐ Reusability ‐ Standardization of the IR ‐ Open-source ‐ Shared debug tools
14. Deep Learning and MLIR
15. Image by Gordon Johnson from Pixabay Deep Learning - A World of Dataflow Graphs ➔ Training ➔ Inference Time Series Data Numerical Data Categorical Data Images Categorical Data Categorical Data Text
16. Example: GANILLA Image source: GANILLA’s paper: arXiv:2002.05638v2
17. Inference edge-devices Deep Learning - Multiple Frameworks and Multiple Devices CPU GPU TPU ?
18. Example: GANILLA GANILLA’s generator architecture (Image source: GANILLA’s paper: arXiv:2002.05638v2) Code written by Lior Dagan Leib, Liron Soffer and Dafna Mordechai.
19. A Huge Amount of Data but Just A Few Lines of Code... Code written by Lior Dagan Leib, Liron Soffer and Dafna Mordechai.
20. Inference edge-devices Deep Learning - Multiple Frameworks and Multiple Devices CPU GPU TPU Image by Gordon Johnson from Pixabay ? ➔ Operations definition and type handling ➔ Graph optimization ➔ Handling Memory related operation ➔ Handling model / data parallelism ➔ Code-Generation for multiple devices
21. MLIR is a compiler intermediate representation that allows lowering dataflow graphs to target-specific machine code of high-performance data-parallel systems. MLIR: Multi Level Intermediate Representation
22. PyTorch TensorFlow Mxnet Onnx Caffe … CPU GPU TPU Inference edge-devices MLIR - Multi Level Intermediate Representation Middle-End LLVM-IR MLIR LLVM - Instruction Based IR MLIR - Operation Based IR
23. MLIR
24. MLIR: Multi Level Intermediate Representation MLIR’s key concepts are operation, which gathers into Dialects. Compiler passes are used to transform operations, lowering one dialect to another.
25. MLIR Operations An Operation represents a concept. A high-level concept like: function definitions, function calls, buffer allocations, view or slices of buffers, process creation, etc. A low-level concept like: target-independent arithmetic, target-specific instructions, configuration registers, logic gates, etc. Operation semantics can be described abstractly using Traits and Interfaces.
27. MLIR Dialects ➔ Dialects are the mechanism for extending the MLIR ecosystem. ➔ Dialects can define new operations, attributes, and types. ➔ In MLIR, dialects are interleaved. At any given time, the IR can be comprised of multiple dialects. ➔ Dialects don’t necessarily define hierarchies, but rather represent different abstraction levels.
29. Examples: Affine dialect - provides a powerful abstraction for affine operations and analyses. Standard dialect - a collection of operations such as: compare, casts, select, branch, conditional branch, return, assert, Atomic-RMW, fma, div, neg, or, and, etc. math dialect – sin, cos, exp, tanh, atan, pow, log, sqrt, rsqrt, etc. GPU dialect - middle-level abstractions for launching GPU kernels following a programming model that resembles CUDA or OpenCL. Vector dialect – operations such as load, store, shuffle, gather, transpose, reduction, etc. LLVM dialect - maps LLVM IR into MLIR by defining the corresponding operations and types.
30. MLIR - Different Types of Transformation Middle-End LLVM-IR MLIR Importing Exporting Converting Translation Vs. Conversion
31. Dialect Conversion ➔ MLIR support Partial Lowering (Also called Progressive Lowering) ➔ An operation of one dialect can be converted to another operation, or a group of operations, from the same dialect or other dialects. ➔ MLIR supports a pattern matching mechanism
32. Partial Lowering / Progressive Lowering
34. MLIR: Multi Level Intermediate Representation Multi-Stage Compiler ➢ Import to MLIR ➢ Dialect conversion ➢ Export from MLIR ‐ Extendable operation-based IR - Designed for partial lowering ‐ Open source ‐ Part of the LLVM project, and can be easily lowered to LLVM-IR.
35. Thank You! https://www.linkedin.com/in/dafna-mordechai @Dafna_Mordechai