Exploring Reline: Enhancing Command Line Usability

Transcript

1. Mari Imaizumi Exploring Reline Enhancing Command Line Usability RubyConf 2024
3. IRB
4. IRB • Interactive Ruby • Default Ruby REPL (Read-Eval-Print Loop) • Run using the `irb` command • Rails console defaults to IRB 4
5. IRB 5
6. Do you know how the IRB works? 6
7. IRB Reline Terminal Emulator 7
8. 8 Tip: repl_type_completor will become a bundled gem starting with Ruby 3.4.
9. 9 Tip: You can customize the dialog color. See the details in Reline’s README.
10. • Mari Imaizumi (@ima1zumi) • She/her • IRB and Reline committer • STORES, Inc. About me 10
11. Agenda • What is Reline? • Feature Gap between Reline and GNU Readline • Understanding Reline 11
12. What is Reline?
13. Reline is a command line editor • A command line editor runs on terminal emulators, handling user input. • Examples: GNU Readline, libedit, Reline • Reline is compatible with GNU Readline and libedit. • Ruby’s default gem (default before Ruby 3.3, bundled gem from Ruby 3.4). 13 Although not a common term, ‘command line editor’ is used here because there is no general term for software like GNU Readline.
14. Without a Command Line Editor puts "Please enter your name:" text = gets.chomp puts "You entered: #{text}" 14
15. Without a Command Line Editor 15 This is the cursor-back escape sequence (CSI + D)
16. Use Reline require "reline" puts "Please enter your name:" text = Reline.readline puts "You entered: #{text}" 16
17. Use Reline 17
18. Command Line Editor • Provides a set of operations for editing entered text • Command line editors manage key operations in tools like IRB and bash 18
19. Command Line Editors • GNU Readline • Used in bash, IRB (up to Ruby 2.6), GDB • NetBSD Editline Library (libedit) • Utilized in macOS • Reline • Used in IRB (Ruby 2.7 and above), debug, highline, hexapdf 19
20. Capabilities of GNU Readline • Line Editing • History management • Autocompletion • Editing Command • and more... 20
21. Feature Comparison Line Editing History Autocompletion Editing Command Others GNU Readline ✅ ✅ ✅ ✅ ✅ Reline ✅ ✅ ✅ ˚ ˚ 21
22. Capabilities of Reline • Display dialogs • Multiline input including prompts • Extensible with Ruby 22
23. • However, it lacks many features: • Key bindings • Editing commands • Con fi gurable variables • and more • Issues are occasionally reported about functionalities available in Readline but missing in Reline • Indicating user interest in these features Reline is Compatible with GNU Readline 23
24. • From Ruby 3.3, readline-ext removed from default gems • If readline-ext is unavailable, require 'readline' will automatically use Reline. [1][2] • Eliminates installation issues with GNU Readline during Ruby builds. • Increased need for Reline's compatibility features. • Users may unknowingly switch dependency from readline-ext to Reline. require 'readline' loads Reline if missing readline-ext 24
25. My wishes • Hoping for Reline to be used instead of readline-ext. • Hope that users switch to Reline without noticing, ensuring a seamless transition. • Aim for Reline to become the chosen command line editor for users considering alternatives. • Add initial support for Reline on Ruby 3.3 #2298 · pry/pry https:// github.com/pry/pry/pull/2298 25
26. Challenges • Reline aims for GNU Readline compatibility; better compatibility is desired. • Expectation for require 'readline' to seamlessly use Reline. • What speci fi c features are missing? 26
27. Feature Gap between Reline and GNU Readline
28. Have you ever written an .inputrc? 🙋
29. What is .inputrc? • Con fi guration fi le for GNU Readline and Reline • ~/.inputrc or INPUTRC=path/to/ fi le • Con fi gurable options: • Con fi gurable variables • Editing commands • Key bindings • Set editing mode (emacs or vi) 29
30. Emacs mode, Vi mode • Emacs mode (default) • Vi mode • has command mode and insert mode • does not have a full set of vi editing functions 30 I'm a Vimmer but I use Emacs mode. :p
31. .inputrc Variables Key bindings 31
32. Key Bindings • Default key bindings are set in command line editor • For example, in Emacs mode, "\C-a" moves the cursor to the beginning of the line • In Vi mode, there are di ff erent bindings for command mode and insert mode • Customization is possible through .inputrc • Some settings written in .inputrc may not work with Reline 32
33. Configurations usable in Reline • Variables: 12 / 46 (26%) • Editing commands: 53 / 120 (44%) • April 16, 2024 33
34. Importance Assessment • Wanted to know what settings are used how frequently • Conducted investigation using .inputrc fi les pushed to GitHub • https://github.com/search?q=language%3AReadline&type=code 34
35. Usage Analysis • Investigated how many out of 166 con fi gurable items were mentioned in .inputrc • Includes commented-out entries • For items with default key bindings set, the presence in .inputrc indicates usage • Total fi les examined: 6600 35
36. High usage 36
37. Low usage 37
38. Implemented Features Undo, Redo show-all-if-ambiguous vi-editing-mode, emacs-editing-mode mode condition in .inputrc Bracketed paste mode 38
39. Understanding Reline
40. Exploring Reline Implementation • 1. Enter "a" • 2. Enter "C-a" (Control-a) • 3. Implement the undo command 40
41. First Step: Follow the input text require "reline" text = Reline.readline puts text • Enter "a\n" 41
42. First Step: Follow the input text 42
43. Flow Handle input keys Render Wait inputs Initialize Reline.readline 43
44. # reline.rb # Simplified original code private def inner_readline(prompt, add_hist, multiline, &confirm_multiline_termination) l = line_editor l.reset(prompt, encoding: encoding) # l.confirm_multiline_termination_proc = confirm_multiline_termination l.output = output l.completion_proc = completion_proc l.completion_append_character = completion_append_character l.output_modifier_proc = output_modifier_proc l.prompt_proc = prompt_proc l.auto_indent_proc = auto_indent_proc l.dig_perfect_match_proc = dig_perfect_match_proc pre_input_hook&.call @dialog_proc_list.each_pair do |name, d| l.add_dialog_proc(name,d.dialog_proc,d.context) end config.read io_gate.set_default_key_bindings(config) Handle input keys Render Wait inputs Initialize Reline.readline 44
45. # reline.rb private def inner_readline(prompt, add_hist, multiline, &confirm_multiline_termination) # Snip loop do read_io(config.keyseq_timeout) { |inputs| inputs.each { |key| line_editor.update(key) } } if line_editor.finished? line_editor.render_finished break else line_editor.rerender end end end Handle input keys Render Wait inputs Initialize Reline.readline 45
46. # ansi.rb def self.inner_getc(timeout_second) unless @@buf.empty? return @@buf.shift end until @@input.wait_readable(0.01) timeout_second -= 0.01 return nil if timeout_second <= 0 Reline.core.line_editor.handle_signal end c = @@input.getbyte (c == 0x16 && @@input.raw( min: 0, time: 0, &:getbyte)) || c rescue Errno::EIO # Maybe the I/O has been closed. nil rescue Errno::ENOTTY nil end Handle input keys Render Wait inputs Initialize Reline.readline c is 97(Integer) 46
47. # reline.rb private def inner_readline(prompt, add_hist, multiline, &confirm_multiline_termination) # Snip loop do read_io(config.keyseq_timeout) { |inputs| inputs.each { |key| line_editor.update(key) } } if line_editor.finished? line_editor.render_finished break else line_editor.rerender end end end Handle input keys Render Wait inputs Initialize Reline.readline 47
48. Handle input keys Render Wait inputs Initialize Reline.readline # line_editor.rb private def normal_char(key) @buf << key.combined_char if @buf.size > 1 # multi byte keys = @buf.dup.force_encoding(@encoding) process_key(keys, nil) @multibyte_buffer.clear else # single byte return if key.char >= 128 method_symbol = @config.editing_mode.get_method( key.combined_char) process_key(key.combined_char, method_symbol) @multibyte_buffer.clear end end 48
49. Handle input keys Render Wait inputs Initialize Reline.readline method_symbol is :ed_insert # emacs.rb class Reline::KeyActor::Emacs MAPPING = [ # 0 ^@ :em_set_mark, # 1 ^A :ed_move_to_beg, # snip # 97 a :ed_insert, key is 97 -> 49
50. Handle input keys Render Wait inputs Initialize Reline.readline # line_editor.rb private def ed_insert(key) key.chr.encode(Encoding::UTF_8) str = key.chr insert_text(str) end def insert_text(text) current_line = @buffer_of_lines[@line_index] if current_line.bytesize == @byte_pointer current_line += text else current_line = byteinsert(current_line, @byte_pointer, text) end @byte_pointer += text.bytesize process_auto_indent end 50
51. Handle input keys Render Wait inputs Initialize Reline.readline 001* if 1 002* 2 003* 3▪ @bu ff er_of_lines = ["if 1", " 2", " 3"] @line_index = 2 @byte_pointer = 3 @bu ff er_of_lines ←@line_index=2 ↑@byte_pointer=3 51
52. # reline.rb private def inner_readline(prompt, add_hist, multiline, &confirm_multiline_termination) # Snip loop do read_io(config.keyseq_timeout) { |inputs| inputs.each { |key| line_editor.update(key) } } if line_editor.finished? line_editor.render_finished break else line_editor.rerender end end end Handle input keys Render Wait inputs Initialize Reline.readline fi nished? == false fi nished? == true 52
53. Follow the editing command require "reline" text = Reline.readline puts text • Enter "C-a" 53
54. Handle input keys Render Wait inputs Initialize Reline.readline # line_editor.rb private def normal_char(key) @buf << key.combined_char if @buf.size > 1 # multi byte keys = @buf.dup.force_encoding(@encoding) process_key(keys, nil) @multibyte_buffer.clear else # single byte return if key.char >= 128 method_symbol = @config.editing_mode.get_method( key.combined_char) process_key(key.combined_char, method_symbol) @multibyte_buffer.clear end end 54
55. Handle input keys Render Wait inputs Initialize Reline.readline # emacs.rb class Reline::KeyActor::Emacs MAPPING = [ # 0 ^@ :em_set_mark, # 1 ^A :ed_move_to_beg, # snip # 97 a :ed_insert, key is 1 -> :ed_move_to_beg 55
56. Handle input keys Render Wait inputs Initialize Reline.readline # line_editor.rb private def ed_move_to_beg(_key) @byte_pointer = 0 end alias_method :beginning_of_line, :ed_move_to_beg alias_method :vi_zero, :ed_move_to_beg 56
57. Implementing Undo Functionality • Press "C-_" to undo actions. • "C-_" is the default key binding for undo in GNU Readline. 57
58. What is Undo? • Undo reverses previous actions. • Example: Typing "abc" and then undoing reverts it to "ab". • The unit of undo depends on the editor: • GNU Readline likely uses keystroke timing. • Zsh Line Editor considers each input as one unit. • Reline treats each input as one unit. 58
59. Storing Operation History A B C Operation 59
60. Storing Operation History A B Operation C 60
61. Retrieving from Operation History A B Undo C 61
62. Retrieving from Operation History A B Undo C 62
63. "" Current line past_lines 63
64. Input "a" "" Current line past_lines 64
65. "a" Current line "a" "" add 65 past_lines Old line Input
66. "a" "a" "" push "" 66 past_lines Old line Current line Input
67. "a" "b" "" "" 67 past_lines Old line Current line Input
68. "ab" "b" "a" "" add 68 past_lines Old line Current line Input
69. "ab" "b" "a" "" push "a" 69 past_lines Old line Current line Input
70. "ab" "C-_" "a" "" "a" 70 past_lines Old line Current line Input
71. "ab" "C-_" "a" "" "a" Undo 71 past_lines Old line Current line Input
72. "a" "C-_" "a" "" pop 72 past_lines Old line Current line Input
73. Implementation details • https://github.com/ruby/reline/pull/701 • Restores both text and cursor position to previous states. 73
74. Handle input keys Render Wait inputs Initialize Reline.readline # line_editor.rb private def normal_char(key) @buf << key.combined_char if @buf.size > 1 # multi byte keys = @buf.dup.force_encoding(@encoding) process_key(keys, nil) @multibyte_buffer.clear else # single byte return if key.char >= 128 method_symbol = @config.editing_mode.get_method( key.combined_char) process_key(key.combined_char, method_symbol) @multibyte_buffer.clear end end method_symbol is :undo 74
75. Handle input keys Render Wait inputs Initialize Reline.readline # line_editor.rb private def undo(_key) return if @past_lines.empty? @undoing = true target_lines, target_cursor_x, target_cursor_y = @past_lines.last set_current_lines( target_lines, target_cursor_x, target_cursor_y ) @past_lines.pop end 75
76. Undo demo 76
77. Summary • Reline is a pure Ruby gem compatible with GNU Readline and libedit. • It provides line-editing features and is extensible in Ruby. • require "readline-ext" loads Reline if readline-ext is missing. • For feature requests or bug reports, feel free to post on the ruby/reline repo.
78. Footnote • [1] https://bugs.ruby-lang.org/issues/19616 • [2] https://github.com/ruby/ruby/blob/ cb636fe7075a260e4d47327764924a3ad10b502a/lib/readline.rb 78
79. Versions • Ruby 3.3.6 • IRB 1.14.1 • Reline 0.5.11 • GNU Readline 8.2.10 79
80. Enjoy command line coding!