Linux on RISC-V

It is an exciting time for Linux on RISC-V, the open instruction set (ISA) that is quickly gaining critical mass. I will introduce the pieces needed to boot Linux on RISC-V including the Privileged Architecture, OpenSBI and U-Boot, and how that fits into the upcoming RISC-V Platform Specification. I will break down support for existing hardware and current upstreaming efforts. I will also discuss how the arch/riscv maintenance guidelines try to avoid unnecessary churn as the landscape of RISC-V extensions continues to evolve.

Drew Fustini, Baylibre

Transcript

1. Linux on RISC-V Drew Fustini <dfustini@baylibre.com> Slides: tinyurl.com/riscv-kr-22 June 1-3, 2022
2. $ whoami • Linux kernel developer, BayLibre ◦ embedded software consultancy based in Nice, France, with ~50 engineers around the world contributing to open source projects like Linux, U-Boot, Android and Zephyr • Board of Directors, BeagleBoard.org Foundation • Board of Directors, Open Source Hardware Association (OSHWA) ◦ OSHW Certification Program • Ambassador, RISC-V International
3. RISC-V: a Free and Open ISA • Started by a computer architecture research group at University of California Berkeley in 2010 led by Krste Asanovic • V as in the roman numeral five, because it is the 5th RISC instruction set to come out of UC Berkeley • Free and Open because the specifications are published under an open source license: Creative Commons Attribution 4.0 International ◦ Volume 1, Unprivileged Spec v. 20191213 [PDF] ◦ Volume 2, Privileged Spec v. 20211203 [PDF]
4. What is different about RISC-V? • Simple clean-slate design ◦ Avoids any dependencies on microarchitecture style (in-order, out-of-order, etc) • Modular design ◦ Suitable for everything from microcontrollers to supercomputers • Stable base ◦ Base integer ISAs and standard extensions are frozen ◦ Additions via optional extensions, not new versions (source: Instruction Sets Want to be Free, Krste Asanovic)
5. RISC-V base integer ISAs • RV32I: 32-bit ◦ less than 50 instructions needed! • RV64I: 64-bit ◦ Most important for Linux • RV128I: 128-bit ◦ Future-proof address space (source: RISC-V Summit 2019: State of the Union, Krste Asanovic)
6. • XLEN defines the register width ◦ XLEN=32 for RV32I ◦ XLEN=64 for RV64I • 32 registers named x0 to x31 • Dedicated PC register • Base ISA talk by Andrew Waterman explains the instruction encoding scheme RISC-V base integer registers (source: Figure 2.1: RISC-V base unprivileged integer register state)
7. • x1 to x31 are all equally general-use registers as far as the processor is concerned • RISC-V psABI defines standard functions for these registers [PDF] ◦ s0 to s11 are preserved across function calls ◦ argument registers a0 to a7 and the temporary registers t0 to t6 are not RISC-V ABI (source: RISC-V Assembly Programmer's Manual)
8. RISC-V Standard Extensions • M: integer multiply/divide • A: atomic memory operations • F, D, Q: floating point, double-precision, quad-precision • G: “general purpose” ISA, equivalent to IMAFD • C: compressed instructions conserve memory and cache • Most Linux distributions target RV64GC (source: RISC-V Summit 2019: State of the Union, Krste Asanovic)
9. • 15 new specifications representing more than 40 extensions • Vector • Hypervisor • Scalar Cryptography • Bit Manipulation Ratified in 2021
10. RISC-V Profiles • RISC-V is a highly modular and extensible architecture ◦ Flexibility to pick and choose what is right for your processor design, but that flexibility creates a large number of possible combinations • RISC-V Profiles specify a much smaller set of ISA choices that represent the most common use-cases ◦ RVM for microcontrollers intended to run bare-metal code or an RTOS ◦ RVA for application processors designed to run full operating system like Linux • RISC-V Summit talk by Greg Favor [slides]
11. Learn more about RISC-V • Get up-to-speed quick with the RISC-V Reader
12. Learn more about RISC-V • Textbook: Computer Organization and Design, RISC-V Edition
13. RISC-V and Industry • RISC-V International now controls the specifications: riscv.org ◦ Non-profit with 2,700+ members including companies & universities from 70 countries ◦ Become a member - free of cost to individuals and non-profits! ◦ RISC-V International YouTube channel has hundreds of talks • Companies have already shipped billions of RISC-V cores ◦ Nvidia GPUs have RISC-V cores for system management tasks ◦ Seagate and Western Digital are using RISC-V cores in storage controllers (source: State of the Union, Krste Asanovic)
14. • No ISA licensing fees or royalties ◦ Avoid legal costs and delays caused by complex licensing agreements • Freedom to choose microarchitecture implementation ◦ An open ISA means that everyone has an architecture license • Freedom to leverage existing open source implementations ◦ Broad range of open source cores already available from small embedded cores to high-performance out-of-order superscalar designs RISC-V and Industry
15. “Is RISC-V an Open Source processor?” • RISC-V is a set of specifications under an open source license • RISC-V implementations can be open source or proprietary • Open specifications make open source implementations possible • An open ISA makes it possible to design an open source processor
16. RISC-V open source cores • Academia ◦ Rocket and BOOM from Berkeley, PULP family of cores from ETH Zurich • Industry ◦ SweRV created by Western Digital and now developed by CHIPS Alliance ◦ OpenHW Group creating proven verified IP like their Core-V designs ◦ Google OpenTitan silicon root of trust project uses LowRISC Ibex core
17. RISC-V open source cores • FPGA soft-cores ◦ PicoRV32, RVfpga, SERV, and VexRiscV • FOSSi Foundation ◦ El Correo Libre monthly newsletter for the latest on open source cores • Build your own open source SoC with an open source silicon toolchain ◦ Google worked with Skywater to open source their 130 nm PDK (process development kit). Google offers free-of-cost MPW (multi-project wafer) runs to open source projects. Learn to design your own using open source design tools with Zero to ASIC.
18. RISC-V software ecosystem • RISC-V already has a well supported software ecosystem ◦ RISC-V International software committee coordinates efforts of member organizations ◦ RISC-V extension and feature support ◦ PLCT Lab led by Wei Wu at ISCAS does a lot of compiler and runtime work • Operating systems: Linux, BSDs, FreeRTOS, Zephyr • Toolchains & libraries: gcc, glibc, gdb, binutils, clang/llvm, newlib • Languages and Runtimes: V8, Node.js, Rust, Go, OpenJDK, Python
19. RISC-V Privileged Architecture • Three privilege modes ◦ User (U-mode): application ◦ Supervisor (S-mode): OS kernel ◦ Machine (M-mode): firmware • Environment Call (ECALL) instruction ◦ Transfer control to a higher privileged mode ◦ Userspace program (u-mode) uses ECALL to make system call into OS kernel (s-mode)
20. Control and Status Registers (CSRs) • CSR have their own dedicated instructions to read and write • CSR are specific to a mode (e.g. m-mode and s-mode) • Machine Status (mstatus) is an important CSR (source: Introduction to the RISC-V Architecture [PDF],Drew Barbier)
21. RISC-V Virtual Memory • satp CSR (Supervisor Address Translation and Protection) controls supervisor-mode address translation and protection • Sv32: 3 level page table • Sv39: 3 level page table • Sv48: 4 level page table • Sv57: 5 level page table (source: Demystifying the RISC-V Linux software stack, Nick Kossifidis)
22. RISC-V Trap Handling • Exceptions occur synchronously • Interrupts occur asynchronously • <x>cause CSR indicates which interrupt or exception occurred ◦ mcause for m-mode / scause for s-mode • Corresponding bit is set in <x>E/IP CSR (source: RISC-V Privileged Architecture [PDF], Allen Baum)
23. • Hart is a hardware thread • Each RISC-V core contains an independent instruction fetch unit • A RISC-V core with multi-threading (SMT) would contain multiple harts • Each hart is a processor from the perspective of Linux ◦ Imagine a RISC-V laptop which has 2 cores with 2 harts per core ◦ Linux would see 4 processors What is a Hart? (source: Section 1.1 in RISC-V Unprivileged spec)
24. RISC-V Interrupts • Local per-hart interrupts ◦ CLINT (Core Local Interruptor) ◦ CLIC (Core Local Interrupt Controller) • Global interrupts ◦ PLIC (Platform Level Interrupt Controller) (source: RISC-V Fast Interrupts [PDF], Krste Asanovic)
25. • Developed on the AIA SIG mailing list: tech-aia • APLIC (Advanced Platform-Level Interrupt Controller) replaces PLIC • Adds IMSIC (Incoming Message-Signaled Interrupt Controller) for PCIe • AIA is complimented by ACLINT (Advanced Core Local Interruptor) ◦ Developed on the tech-unixplatformspec mailing list ◦ Backwards compatible with the SiFive CLINT but restructured to be more efficient ◦ RISC-V Summit talk by Anup Patel and John Hauser [slides] Advanced Interrupt Architecture (AIA)
26. RISC-V Boot Flow Boot ROM M-mode First stage bootloader (U-Boot SPL or vendor firmware) U-Boot S-mode Linux kernel
27. RISC-V Boot Flow Boot ROM M-mode U-Boot S-mode Linux kernel SBI First stage bootloader (U-Boot SPL or vendor firmware)
28. • Non-ISA RISC-V specification ◦ This means it does not add or modify and RISC-V instructions • The calling convention between S-mode and M-mode ◦ Allows supervisor-mode (s-mode) software like the Linux to be portable across RISC-V implementations by abstracting platform specific functionality Supervisor Binary Interface (SBI)
29. • Required by the UNIX-Class Platform Specification ◦ Mailing list: tech-unixplatformspec ◦ This will be replaced by the upcoming RISC-V Platform Specification • Small core along with a set of optional modular extensions ◦ Base extension - query basic information about the machine ◦ Timer extension - program the clock for the next event ◦ IPI extension - send an inter-processor interrupt to harts defined in mask ◦ RFENCE extension - instructs remote harts to execute FENCE.I instruction Supervisor Binary Interface (SBI)
30. • Hart State Management (HSM) ◦ S-mode can request to stop, start or suspend a hart • System Reset ◦ Supervisor-mode software can request system-level reboot or shutdown • Performance Monitoring Unit ◦ Interface for supervisor-mode to configure and use the RISC-V hardware performance counters with assistance from the machine-mode ◦ ”Performance Monitoring in RISC-V using perf” by Atish Patra SBI Extensions
31. • Hypervisor Supervisor mode (HS-mode) where host kernel runs • Virtualized Supervisor mode (VS-mode) where the guest kernel runs Hypervisor extension
32. • Open source implementation of SBI ◦ Core library ◦ Platform specific libraries ◦ Full reference firmware for some platforms • Provides runtime services to S-mode software ◦ SBI extensions present on a platform define the available runtime services ◦ Unimplemented instructions will trap and OpenSBI can emulate OpenSBI (source: OpenSBI Deep Dive, Anup Patel)
33. • No need to add code to OpenSBI for each new platform ◦ First-stage bootloader, like U-Boot SPL, is expected to pass a Device Tree to OpenSBI which describes all the platform specific functionality • The same OpenSBI binary can be used across platforms ◦ Many RISC-V boards and emulators now use Generic Platform ◦ Linux distros do not need to ship a different OpenSBI build for each board OpenSBI Generic Platform
34. OpenSBI Domain Support • An OpenSBI domain is a system-level partition of underlying hardware having its own memory regions and HARTs • Talk by Anup Patel
35. UEFI Support • UEFI is a standard interface between firmware and operating systems, and it is used on most x86 and arm64 platforms • U-Boot and TianoCore EDK2 both have UEFI implementations on RISC-V • Grub2 can be used as an UEFI payload on RISC-V • UEFI support for RISC-V added in Linux 5.10
36. • Boot hart ID is known only at boot and it is needed before ACPI tables or DT properties can be parsed • Hart ID is passed in the a0 register on non-UEFI systems, but the UEFI application calling conventions do not allow this • RISC-V EFI Boot Protocol allows the OS to discover the boot hart ID • The public review process has completed, and Sunil V L has added support to the Linux kernel for RISCV_EFI_BOOT_PROTOCOL UEFI Support
37. RISC-V Platform Specification • Goal is to support “off-the-shelf” software by standardizing the interface between hardware platforms and operating systems • Created by the Platforms Horizontal Subcommittee (HSC) ◦ Bi-weekly meetings chaired by Kumar Sankaran ◦ Mailing list: tech-unixplatformspec • Platforms talk at RISC-V Summit 2021 ◦ Philipp Tomsich, Chair of Software HSC, and Mark Himelstein CTO RISC-V International
38. RISC-V Platform Specification • OS-A Platform ◦ “A” as in application, this is a category of platforms that support full OS like Linux ◦ OS-A Common Requirements ◦ OS-A Embedded Platform ◦ OS-A Server Platform • RVM-CSI Platform ◦ Bare-metal applications or RTOS running on RISC-V microcontrollers ◦ CSI is common software interface; goal is to ease porting, not binary compatibility
39. RISC-V Platform Specification • OS-A common requirements for Embedded and Server ◦ Must comply with the RVA22U and RVA22S ISA profiles as defined in RISC-V ISA Profiles ◦ Common requirements for Debug, Timers, Interrupt Controllers ◦ Requires serial console with UART 8250 or UART 16550 ◦ Requirements for runtime services such as SBI extensions ◦ Software components must comply with the RISC-V Calling Convention specification and the RISC-V ELF specification
40. RISC-V Platform Specification • OS-A Embedded Platform ◦ Target might be a single board computer or mobile device ◦ PMU counters and events for performance monitoring ◦ Boot process must comply with Embedded Base Boot Requirements (EBBR) spec ◦ EBBR requires a subset of the UEFI spec which U-Boot has implemented ◦ Device Tree (DT) is the required mechanism for the hardware discovery and config ◦ GPT partitioning required for shared storage
41. RISC-V Platform Specification • OS-A Server Platform ◦ Goal is for an enterprise Linux distro like RHEL to “just work” on server-class hardware that complies with this ◦ System peripheral requirements like PCIe, watchdog timer, system date/time ◦ RAS (Reliability, Availability, and Serviceability) requirements like ECC RAM ◦ ACPI is the required mechanism for the hardware discovery and configuration ◦ Must comply with the RISC-V ACPI Platform Requirements Specification
42. RISC-V ACPI Platform Specification • Defines mandatory ACPI tables and objects for RISC-V server platforms • New tables are needed for RISC-V ◦ RISC-V Hart Capabilities Table (RHCT) ◦ RISC-V Timer Description Table (RTDT) • More details in ‘ACPI for RISC-V: Enabling Server Class Platforms’ ◦ Sunil V L from Ventana Microsystems at RISC-V Summit [slides]
43. • Support for RISC-V in mainline QEMU • Boots 32-bit and 64-bit mainline Linux kernel • Machine configs to boot same binaries as some RISC-V dev boards • Supports the new Hypervisor and Vector extensions RISC-V emulation in QEMU
44. RISC-V in the Linux kernel • Initial port by Palmer Dabbelt was merged into Linux 4.15 back in 2018 • “It’s a fun, friendly, and still pretty small community” - Björn Töpel [1] • Palmer continues to maintain the riscv tree • Development happens on the linux-riscv mailing list • View the archive on lore.kernel.org • IRC: #riscv on libera.chat
46. Recently added to Linux • KVM RISC-V support by Anup Patel added in Linux 5.16 ◦ Add KVM support for the Hypervisor specification • SBI SRST extension support by Anup Patel added in Linux 5.17 ◦ Support for the SBI SRST (system reset) extension which allows systems that do not have an explicit driver in Linux to reboot
47. New in Linux 5.18 • Add Sv57 page table support by Qinglin Pan ◦ use 5-level page table to support Sv57 which expands the virtual address space to 57 bits (128 petabytes) • Improve RISC-V Perf support by Atish Patra ◦ Recent talk: Perf on RISC-V: The Past, the Present and the Future (slides)
48. New in Linux 5.18 • RISC-V CPU Idle support by Anup Patel ◦ cpuidle and suspend drivers now support the SBI HSM extension • Provide framework for RISC-V ISA extensions by Atish Patra ◦ Linux was no longer correctly parsing the RISC-V ISA string as the number of RISC-V extensions has grown and extension names are no longer a single character ◦ This series implements a generic framework to parse multi-letter ISA extensions. ◦ Based on initial work by Tsukasa OI
49. Coming in Linux 5.19… • RISC-V Patches for the 5.19 Merge Window, Part 1 - Palmer (2022-05-31) ◦ Support for page-based memory attributes <we’ll dive into that topic in a few slides> ◦ Support for running rv32 binaries on rv64 systems via the compat subsystem ◦ Support for kexec_file() ◦ Support new generic ticket-based spinlocks, which allows us to also move to qrwlock ◦ A handful of cleanups and fixes, include some larger ones around atomics and XIP • Part 2? Follow linux-riscv and look at Palmer’s riscv/for-next branch
50. Work in progress • [PATCH v10 00/16] riscv: Add vector ISA support by Greentime Hu ◦ Vector ISA support based on the ratified Vector 1.0 extension ◦ Defines new structure __riscv_v_state in struct thread_struct to save/restore the vector related registers. It is used for both kernel space and user space. • [PATCH v6 0/7] RISC-V IPI Improvements by Anup Patel ◦ Traditionally, RISC-V S-mode software like the Linux kernel calls to into M-mode runtime firmware like OpenSBI to issue IPIs (inter-processor interrupts) ◦ AIA (advanced interrupt architecture) provides the ability for S-mode to issue IPIs without any assistance from M-mode. This improves efficiency.
51. Work in progress • [PATCH v4 0/4] Add Sstc extension support by Atish Patra ◦ Traditionally, an SBI call is necessary to generate timer interrupts as S-mode does not have access to the M-mode time compare registers. ◦ This results in significant latency for the kernel to generate timer interrupts at kernel ◦ For virtualized environments, it’s even worse as the KVM handles the SBI call and uses a software timer to emulate the time compare register. ◦ Sstc extension allows kernel to program a timer and receive interrupt without supervisor execution environment (M-mode/HS mode) intervention.
52. Linux distro: Fedora • Aims to provide a complete Fedora experience on RISC-V • Talk by Wei Fu, RISC-V Ambassador and Red Hat engineer [slides] • Installation instructions for QEMU and RISC-V dev boards
53. Linux distro: Debian • riscv64 is port of Debian to RISC-V ◦ “a port in Debian terminology means to provide the software normally available in the Debian archive (over 20,000 source packages) ready to install and run” • 95% of packages are built for RISC-V ◦ The Debian port uses RV64GC as the hardware baseline and the lp64d ABI
54. Linux distro: Ubuntu • riscv64 supported since the release of Ubuntu 20.04 LTS. • Ubuntu 22.04 pre-installed SD-card image for SiFive boards and QEMU • Starting with Ubuntu 22.04, a server install image is made available to install Ubuntu on NVMe drive of the SiFive Unmatched board
55. Linux distros • OpenSuSE ◦ RISC-V support is still under development with Tumbleweed images for some boards • Arch Linux ◦ Community effort has 95% of core packages building for RISC-V • Gentoo ◦ riscv64 stages are available on the Gentoo download page
56. OpenEmbedded and Yocto • meta-riscv: general hardware-specific BSP overlay for RISC-V devices ◦ works with different OpenEmbedded/Yocto distributions and layer stacks ◦ Supports both QEMU and RISC-V dev boards
57. BuildRoot • RISC-V port is now supported in the upstream BuildRoot project • “Embedded Linux from scratch in 45 minutes (on RISC-V)” ◦ Tutorial by Michael Opdenacker at FOSDEM 2021 ◦ Use Buildroot to compile OpenSBI, U-Boot, Linux and BusyBox ◦ Boot the system in QEMU
58. • Launched in 2018 as first Linux-capable RISC-V dev board • Exciting to see Fedora GNOME desktop on RISC-V • $999 was too expensive for widespread adoption • FU540 SoC chip was never sold separately SiFive Freedom Unleashed
59. Microchip PolarFire SoC • Same RISC-V cores as the SiFive FU540 SoC but adds a FPGA fabric ◦ FPGA with 25k to 460k logic elements (LEs) ◦ Supports DDR4 and PCIe Gen2 • Full commercial product family ◦ Parts will be available from distributors • Microchip Icicle dev board ($499)
60. Kendryte K210 • 400 MHz dual core RV64GC • 8MB SRAM but no DRAM • Dev boards starting at $14 • Upstream support in Linux and u-boot • Buildroot support by Damien Le Moal can create a busybox-based rootfs
61. SiFive Unmatched • SiFive Freedom FU740 SoC ◦ 4x U74 RV64GC application cores • Mini-ITX PC form factor ◦ 16GB DDR4, 4x USB 3.2, one x16 PCIe slot ◦ M.2 for NVMe SSD and WiFi/Bluetooth • Shipped in 2021 for $665 ◦ Discontinued in 2022 to focus on next-gen
62. T-Head XuanTie C910 • T-Head (“píng tóu gē”) is part of Alibaba • High performance RV64GC with up 16 cores ◦ 12-stage pipeline, out-of-order, multi-issue architecture ◦ comparable to Arm Cortex-A73 • 2 core ‘ICE’ SoC made in low qty for evaluation • T-Head ported Android 10 (AOSP) to RISC-V and showed a demo on the ICE in early 2021
63. Porting Android to RISC-V • Mao Han presented why and how Alibaba T-Head ported Android to RISC-V (jump to 4:32) ◦ PDF of slides • Update on Android 12 from April (jump to 13:23) ◦ PDF of slides • How Alibaba is Porting RISC-V to the Android OS blog post with more technical details
64. • ‘ICE’ SoC featuring dual C910 core at 1.2GHz • 4GB LPDDR4, 16GB eMMC, 7 inch touchscreen, WiFi, Gigabit Ethernet • Produced in limited quantity and available on AliExpress for $399 T-Head RVB-ICE dev board
65. RISC-V Android SIG (Special Interest Group) • GitHub organization riscv-android-src “contains all the modified AOSP(Android open source project) repositories with RISC-V support” • Instruction to build and run Android 12 on RISC-V
66. T-Head XuanTie C906 • single-core RV64GC, only up to 1GHz, simpler 5-stage in-order pipeline
67. Allwinner D1 SoC • mass production low cost SoC with a single T-Head C906 core at 1 GHz
68. Allwinner Nezha D1 dev board • Official D1 board made by Allwinner Online, $115 bundle on AliExpress
69. RISC-V Developer Boards • Initiative from RISC-V International to get Linux-capable boards into the hands of open source developers ◦ Launched in 2021 with the Allwinner D1 Nezha and SiFive Unmatched • Fill out this form to apply ◦ Need to be RISC-V International member (or part of a member organization), but remember that individuals can join RISC-V International free of cost ◦ Explain why you are interested in RISC-V and what you plan to do with dev board ◦ To improve your chances, don’t overestimate your hardware requirements like RAM
70. Allwinner D1 open source community • linux-sunxi: strong open source community for Allwinner SoCs ◦ D1 wiki page and Allwinner Nezha board wiki page • Telegram group: Mainline Linux for D1 (190 members) • Samuel Holland has been working on getting mainline to run ◦ U-Boot SPL: https://github.com/smaeul/sun20i_d1_spl ◦ OpenSBI: https://github.com/smaeul/opensbi ◦ U-Boot: https://github.com/smaeul/u-boot/tree/d1-wip ◦ Linux: https://github.com/smaeul/linux/tree/riscv/d1-wip
71. Fedora on Allwinner D1 • Wei Fu has created a Fedora “Rawhide” image for the Allwinner D1 Nezha dev board that includes the XFCE desktop environment
72. • Allwinner D1 with DDR3 RAM of either 512MB ($22) or 1GB ($32) • 1.14” SPI LCD, USB Type-C OTG, uSD • Lichee RV Dock: HDMI out, USB-A host port, WiFi+BT, mic, audio out • Lichee RV 86 Panel: 8 inch screen • More details on sunxi wiki Lichee RV-Nezha CM
73. Xassette Asterisk • Allwinner F133 combines the Allwinner D1 with 64MB DDR2 in a single package • Board design published as Open Source Hardware under the CERN OHL-w v2 licence • Designed with KiCad (open source CAD sw) • Not available commercially but possible to be hand assembled by hobbyists
74. MangoPi-Nezha MQ • Allwinner F133 (also known as D1s) • WiFi, USB Type C, mic, audio out • DSI and RGB display connectors • Open source hardware: KiCad files on GitHub • $39 on Crowd Supply
75. Allwinner D1 mainline Linux support • Allwinner reused peripheral IP from their existing ARM SoCs and the Linux kernel already has drivers for most of them • T-Head cores like C910 and C906 do have some non-standard functionality for performance but it’s not needed to boot ◦ Instructions to accelerate I-cache and TLB synchronization • T-Head MMU has a non-standard ‘enhanced’ mode that is needed to support DMA with devices on non-coherent interconnects ◦ Linux needs to enable that ‘enhanced’ MMU mode to function properly
76. How to handle non-coherent interconnects? • The original RISC-V Privileged spec stated that “the use of hardware-incoherent regions is discouraged due to software complexity, performance, and energy impacts” • However, non-coherent interconnects are important for low cost SoCs • T-Head designed the C9xx cores in 2019, and there were no RISC-V extensions that provided ability to handle non-coherent devices
77. T-Head PTE format • T-Head used bits in the PTE (Page Table Entry) to specify memory type … but those bits were already marked reserved in RISC-V priv spec | 63 | 62 | 61 | 60 | 59-8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0 SO C B SH RSW D A G U X W R V ^ ^ ^ ^ BIT(63): SO - Strong Order BIT(62): C - Cacheable BIT(61): B - Bufferable BIT(60): SH - Shareable 0000 - NC Weakly-ordered, Non-cacheable, Non-bufferable, Non-shareable 0111 - PMA Weakly-ordered, Cacheable, Bufferable, Shareable 1000 - IO Strongly-ordered, Non-cacheable, Non-bufferable, Non-shareable
78. Page-Based Memory Types extension • Svpbmt proposed by Virtual Memory TG and ratified at the end of 2021 ◦ “S” = supervisor-mode (privileged architecture), “v” for virtual memory Here is the svpbmt PTE format: | 63 | 62-61 | 60-8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0 N MT RSW D A G U X W R V ^ RISC-V Encoding & MemType RISC-V Description ---------- ------------------------------------------------ 00 - PMA Normal Cacheable, No change to implied PMA memory type 01 - NC Non-cacheable, idempotent, weakly-ordered Main Memory 10 - IO Non-cacheable, non-idempotent, strongly-ordered I/O memory 11 - Rsvd Reserved for future standard use
79. Svpbmt support in Linux • [PATCH v10 00/12] riscv: support for Svpbmt and D1 memory types ◦ by Heiko Stuebner based on initial work by Alibaba kernel engineer Guo Ren ◦ Implements the official RISC-V Svpbmt extension ◦ The standard Svpbmt and custom T-Head PTE formats both use the highest bits to determine memory type but the encoding and semantics are different ◦ The custom T-Head PTE format is supported through boot-time code patching using the Linux Alternatives Framework ◦ Expected to land in Linux 5.19 as it is in Palmer’s pull request
80. Cache Management Operations • Instructions to manage cache are important for SoCs which that lack cache coherent interconnects • Zicbom extension (“Z” prefix means Unpriv spec) was ratified at the end of 2021, and it defines cache-block management (CBO) instructions: ◦ CBO.CLEAN guarantee store by hart can be read from mem by non-coherent device ◦ CBO.INVAL guarantee hart can load data written to memory by non-coherent device ◦ CBO.FLUSH guarantees both • Support for Non-Coherent I/O Devices in RISC-V from RV Summit [slides]
81. CMO support in Linux • riscv: implement Zicbom-based CMO instructions + the t-head variant by Heiko Stuebner • Implements Zicbom-extension to handle cache clean, invalidate, flush * cbo.clean rs1 * | 31 - 20 | 19 - 15 | 14 - 12 | 11 - 7 | 6 - 0 | * 0...01 rs1 010 00000 0001111 * * cbo.flush rs1 * | 31 - 20 | 19 - 15 | 14 - 12 | 11 - 7 | 6 - 0 | * 0...10 rs1 010 00000 0001111 * * cbo.inval rs1 * | 31 - 20 | 19 - 15 | 14 - 12 | 11 - 7 | 6 - 0 | * 0...00 rs1 010 00000 0001111 #define CBO_INVAL_A0 ".long 0x15200F" #define CBO_CLEAN_A0 ".long 0x25200F" #define CBO_FLUSH_A0 ".long 0x05200F"
82. CMO support in Linux • T-Head implemented custom cache instructions before Zicbom existed * dcache.ipa rs1 (invalidate, physical address) * | 31 - 25 | 24 - 20 | 19 - 15 | 14 - 12 | 11 - 7 | 6 - 0 | * 0000001 01010 rs1 000 00000 0001011 * dache.iva rs1 (invalida, virtual address) * 0000001 00110 rs1 000 00000 0001011 * * dcache.cpa rs1 (clean, physical address) * | 31 - 25 | 24 - 20 | 19 - 15 | 14 - 12 | 11 - 7 | 6 - 0 | * 0000001 01001 rs1 000 00000 0001011 * dcache.cva rs1 (clean, virtual address) * 0000001 00100 rs1 000 00000 0001011 * * dcache.cipa rs1 (clean then invalidate, physical address) * | 31 - 25 | 24 - 20 | 19 - 15 | 14 - 12 | 11 - 7 | 6 - 0 | * 0000001 01011 rs1 000 00000 0001011 * dcache.civa rs1 (... virtual address) * 0000001 00111 rs1 000 00000 0001011 #define THEAD_INVAL_A0 ".long 0x0265000b" #define THEAD_CLEAN_A0 ".long 0x0245000b" #define THEAD_FLUSH_A0 ".long 0x0275000b"
83. CMO support in Linux • While the Zicbom and T-Head instructions are different, they provide the same functionality, so the T-Head variant handled with the existing alternatives mechanism • Allwinner D1 needs these cache instructions for peripherals like MMC (SD card), USB, and Ethernet to work • Unfortunately, these patches use pre-coded CMO instructions and Palmer would prefer that Linux support wait until the instructions are added to gcc and binutils
84. Allwinner D1 IOMMU support • [PATCH 0/5] iommu/sun50i: Allwinner D1 support by Samuel Holland • IOMMU is not needed for boot • Optional feature for the display engine and video decoder • Without IOMMU support, video/frame buffers have to be contiguous in physical memory, and that requires the user to know how much memory to reserve for them at boot
85. T-Head released RISC-V cores as open source! • OpenE902, OpenE906, OpenC906, and OpenC910 cores on GitHub under permissive Apache 2.0 licence
86. XiangShan (香山) • open source high-performance RISC-V processor project from the Chinese Academy of Science • RISC-V Summit 2021 talk by Professor Yungang Bao (slides) ◦ “Contribute to XiangShan and realize your ideas on real chips! The open-source XiangShan will be taped-out every ~6 months” • Nanhu is the 2nd generation microarchitecture ◦ Target: 2GHz@14nm, SPEC CPU2006 20 marks; 7.81 CoreMark/MHz
87. RISC-V Lab • PLCT Lab at Chinese Academy of Sciences ◦ Status report from lab director Wei Wu • Continuous Integration (CI) for open source software projects on RISC-V hardware ◦ 70+ SiFive Unmatched boards ◦ 100+ Allwinner Nezha D1 boards ◦ Open source devs can request access
88. No hardware? Try Renode! • Emulate physical hardware systems including CPU, peripherals, sensors, and networking • Run the same binaries as the real hardware for over 30 supported dev boards ◦ Microchip PolarFire SoC Icicle Kit ◦ Kendryte K210 ◦ SiFive HiFive Unleashed
89. How to get involved with RISC-V International? • Become a member ◦ Individuals and non-profits can join free of cost • RISC-V Technical wiki landing page is the single best place to visit ◦ Technical Working Groups ◦ Recently Ratified Extensions
90. How to get involved with RISC-V International? • RISC-V mailing list server ◦ Only RISC-V members can participate ◦ Archives of all the lists are public • Technical Meetings Calendar ◦ Many groups have bi-weekly or monthly meetings ◦ ICS File / Google Calendar
91. • YouTube playlist with 93 talks RISC-V Summit 2021
92. RISC-V Spring Week 2022 • Videos on the RISC-V YouTube channel • State of the Union & the Road Ahead • Maturing the RISC-V Ecosystem • Evolving the Role of Software in the RISC-V Ecosystem • RISC-V IOMMU Architecture Overview
93. Embedded Linux Conf 2021 • Initializing RISC-V A Guided Tour for ARM Developers ◦ Ahmad Fatoum & Rouven Czerwinski, Pengutronix • Building a Low-key XIP-enabled RISC-V Linux System ◦ Vitaly Vul, Konsulko AB • Perf on RISC-V: The Past, the Present and the Future ◦ Atish Patra & Anup Patel, Western Digital • "A New user(space): Adding RISC-V Support to Zephyr RTOS" [slides] ◦ Kevin Hilman & Alexandre Mergnat, BayLibre
94. Find more at: community.riscv.org RISC-V meetups around the world
95. RISC-V Open Hours • Bi-weekly virtual meetup for the community to interact in real-time ◦ Primary focus on RISC-V support in open source software and RISC-V dev boards ◦ Call for participation is open! No prepared talk or slides required! • Schedule ◦ Wednesday, June 8, 7:00 PM (US PDT) which is Thursday morning in Asia ◦ Wednesday, June 29, 9:00 AM (US PDT) which is early evening in Europe
96. Linux on RISC-V Drew Fustini <dfustini@baylibre.com> Slides: tinyurl.com/riscv-kr-22 June 1-3, 2022
97. BONUS: What about RISC-V on FPGAs?
98. • “RISC-V and FPGAs: Open Source Hardware Hacking” keynote at Hackaday Supercon 2019 by Dr. Megan Wachs Introduction
99. Open source FPGA toolchains • Project IceStorm for Lattice iCE40 FPGA ◦ “A Free and Open Source Verilog-to-Bitstream Flow for iCE40 FPGAs” by Claire Wolf • Project Trellis for the more capable Lattice ECP5 FPGA ◦ “Project Trellis and nextpnr FOSS FPGA flow for the Lattice ECP5” by Myrtle Shah • Project X-Ray and SymbiFlow for much more capable Xilinix Series 7 ◦ “Xilinx Series 7 FPGAs Now Have a Fully Open Source Toolchain!” by Tim Ansell ◦ “Open Source Verilog-to-Bitstream FPGA synthesis flow, currently targeting Xilinx 7-Series, Lattice iCE40 and Lattice ECP5 FPGAs. Think of it as the GCC of FPGAs”
100. Hackaday Supercon badge • RISC-V “soft” core on ECP5 FPGA • Gigantic FPGA In Game Boy Form Factor
101. Why design an SoC in Python? • Python has advantages over traditional HDL like VHDL and Verilog ◦ Many people are familiar with Python than HDL (hardware description languages) ◦ There are currently more software developers than hardware designers • Migen is a Python framework that can automate chip design ◦ Leverages the object-oriented, modular nature of Python ◦ Produces Verilog code so it can be used with existing chip design workflows • “Using Python for creating hardware to record FOSS conferences!”
102. source: http://goo.gl/mZJvFQ
103. • Based on Migen, builds full SoC that can be loaded into an FPGA LiteX
104. • “LiteX vs. Vivado: First Impressions” • Collection of open cores for DRAM, Ethernet, PCIe, SATA and more... LiteX
105. Linux on LiteX-VexRiscv • VexRiscv: 32-bit Linux-capable RISC-V core ◦ Designed to be FPGA friendly ◦ Written in Spinal HDL (based on Scala) • Builds an SoC using VexRiscv core and LiteX modules ◦ Such as LiteDRAM, LiteEth, LiteSDCard, LitePCIe ◦ “This project demonstrates how high level HDLs (Spinal HDL, Migen) enable new possibilities and complement each other. Results shown here are the results of a productive collaboration between open-source communities” • Supports large number of FPGA dev boards including Digilent Arty A7
111. https://twitter.com/GregDavill/status/1231082623633543168
112. Open Source ECP5 FPGA boards • Radiona.org ULX3S ◦ 32MB SDRAM; ESP32 on-board for WiFi and Bluetooth; $115 on CrowdSupply or Mouser
113. Open Source ECP5 FPGA boards • OrangeCrab by Greg Davill ◦ 128MB DDR RAM; Adafruit Feather form factor; available for $129
114. Want to learn FPGAs? Try Fomu! • Online workshop from Tim Ansell and Sean Cross • $50 on CrowdSupply • Fits inside USB port! • Learn how to use: ◦ MicroPython ◦ Verilog ◦ LiteX