Share
Embed
error_highlight: User-friendly Error Diagnostics
RubyKaigi 2022
https://rubykaigi.org/2022/presentations/mametter.html
Yusuke Endoh
September 16, 2022
Transcript
39 slides
-
Yusuke Endoh / @mametter • A Ruby committer working at
Cookpad w/ @ko1 • My major Ruby contributions: • Keyword arguments • coverage.so • TypeProf • error_highlight Today's topic 3 https://ruby-puzzles-2022.cookpad.tech/ -
error_highlight Ruby 3.1 reports the fine-grained error location 4 $
ruby test.rb test.rb:1:in `<main>': undefined method `time' for 42:Integer (NoMethodError) 42.time { print "Hello" } ^^^^^ Did you mean? times error_highlight -
json is nil ? Or json[:article] is nil ? undefined
method `[]' for nil:NilClass (NoMethodError) How useful? To tell which of the calls failed in a line Typical case: 5 json[:article][:author] ^^^^^^^^^ undefined method `[]' for nil:NilClass (NoMethodError) json[:article][:author] ^^^^^^^^^^ json[:article][:author] json is nil json[:article] is nil -
More useful than theory Developer experience is more improved than
expected • Suggests "how to fix" • Helps you reach the wrong code in the editor 6 $ ruby test.rb test.rb:123:in `<main>': undefined method `gsuub' for ... str.gsuub(/....../, "foobar") ^^^^^^ Did you mean? gsub! gsub -
What will change in Ruby 3.2? • ArgumentError / TypeError
• Ruby 3.1 supported only NameError / NoMethodError 7 $ ruby test.rb test.rb:1:in `+': nil can't be coerced into Integer (TypeError) 1 + nil ^^^ Today's topic: the road to achieve them • Rails' error page -
Agenda • ➔Background • How error_highlight implemented in Ruby 3.1
• Problems and solutions • Conclusion 8 -
Implementation overview • 1. Record the code range in bytecode
• 2. Determine where to underline • 3. Prints an error message 9 3.1 -
1. Record code ranges in byte code 10 compile foo.bar(1)
0..2 0..9 8..8 send "foo" put 1 send "bar" byte code source code code range 0..2 8..8 0..9 -
2. Determine where to underline 11 send "foo" put 1
send "bar" foo.bar(42) 0..9 foo.bar(42) 0..2 error error 0..2 8..8 0..9 This approach is too naïve! Why? -
ary.map do … end.selct do … end Naïve code range
is too wide • error_highlight uses AST analysis & Regexps (!) • AST does not retain a location of punctuation (e.g., period) • Kevin Newton's parser work may improve the situation (related to the next talk in this session!) 12 ary.map do … end.selct do … end expected ary.map do … end.selct do … end naïve Typo -
Support various method calls 13 obj.foo += 1 obj.foo is
not defined obj.foo += 1 obj.foo returned nil obj.foo += 1 obj.foo= is not defined prinnt(str) nil + 1 nil[1] obj&.foo obj. foo obj .foo Suggestion for improvement is welcome -
3. Prints code with the underline • Overrides NameError#message •
Ruby's error printer writes #message to stderr • Ruby 3.1 released this version, but 14 class NameError def message super + "¥n" + code + "¥n" + underline end end This approach had many problems! -
Agenda • Background • ➔Problems and solutions • Less expandable
• Wrong underline location • Poor ecosystem support • Conclusion 15 -
Problem 1: Less expandable • Only NameError / NoMethodError were
supported • Why? • Because an error message has some usages • A. To show an error trace • B. To test error-handling code • C. To log an error • error_highlight breaks B and C 16 -
Problem 1-B: Test compatibility • Many tests in the wild
checks an error message • Changing Exception#message is incompatible • Acceptable to change NameError#message since few test cases check its result 17 expect { … }.to raise_error("foobar") -
Problem 1-C: Logging compatibility Many expect #message to return a
one-line string •"error_highlight makes it difficult to parse a log file!" •"error_highlight makes so confusing!" 18 E, [2022-09-10T10:00:00.000000 #12345] ERROR -- : undefined method `time' for 42:Integer 42.time ^^^^^ Did you mean? times #<NoMethodError: undefined method `time' for 42:Integer 42.time ^^^^^ Did you mean? times> p $! -
Solution for Problem 1 Exception#detailed_message is introduced 19 $!.message undefined
method `time' for 1:Integer $!.detailed_message undefined method `time' for 1:Integer (NoMethodError) 1.time ^^^^^ Did you mean? times -
Request to framework developers • Frameworks may want to use
#detailed_message • Instead of #message • Example: https://github.com/rack/rack/pull/1926 • Sentry and DataDog said they would support this • https://bugs.ruby-lang.org/issues/18438 • thanks to Stan Lo (Sentry) and Ivo Anjo (DataDog) 20 -
Problem 1 is solved ArgumentError / TypeError are now supported
Without significant incompatibility 21 test.rb:1:in `+': nil can't be coerced into Integer (TypeError) 1 + nil ^^^ -
Problem 2: Wrong underline Ruby's error printer "escapes" the message
22 test.rb:1:in `<main>': undefined method `gsuub' for " ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥":String (NoMethodError) " ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥".gsuub("", "") ^^^^^^ " ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥".gsuub("", "") Wrong! Wrong too! -
Solution to Problem 2 Stopped Ruby's error printer from escaping
23 This simple solution took about a month! Why? -
Why did it escape the message? • Because of security
consideration! • "TERMINAL EMULATOR SECURITY ISSUES" [1] • Some terminals had insecure escape sequences • Create a file • Input as if a user typed it 24 [1] https://marc.info/?l=bugtraq&m=104612710031920&w=2 -
Is this concern still valid? • [1] is very old
(in 2003) and also says: • Surveyed the current situation • Rxvt / Eterm / XTerm disabled the dangerous features The terminals mentioned in [1] • Gnome Terminal, Windows Terminal and iTerm2 don't support the dangerous features 25 The responsibility should rest on the actual terminal emulator -
Problem 2 is solved • We agreed for Ruby to
stop escaping • Correct underline in Ruby 3.2 26 test.rb:1:in `<main>': undefined method `gsuub' for " ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥":String (NoMethodError) " ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥".gsuub("", "") ^^^^^^ " ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥".gsuub("", "") -
Support Rails error page • Rails provides a more dedicated
error page • error_highlight provides only Exception#message Parse #message …? 29 -
Export error_highlight API 30 ErrorHighlight.spot( $!, backtrace_location: $!.backtrace_locations[0] ) {
:first_lineno=>2, :first_column=>3, :last_lineno=>2, :last_column=>8, :snippet=>" 1.time¥n", :script_lines=>[…] } -
Problem 3 is solved Rails error page now shows error_highlight
The patch is merged https://github.com/rails/rails/pull/45818 31 -
Ruby 3.2's error_highlight • ArgumentError / TypeError Ruby 3.1 supported
only NameError / NoMethodError 33 $ ruby test.rb test.rb:1:in `+': nil can't be coerced into Integer (TypeError) 1 + nil ^^^ • Rails' error page -
Acknowledgments • @yui-knk made the early prototype of error_highlight •
@ioquatix reported a lot of issues • Those who joined in the discussion on tickets, PRs, or the dev meeting 34 -
Memory overhead of bytecode • Approx. 3% (on scaffold Rails
app) • Record AST node_id instead of raw linenos/columns • Simple compression 36 -
Future work / Known problems • Support CJK full-width characters
• I don't want to do that! • Support more exception classes • User-defined exceptions • Show column number • test.rb:2:2 37 -
Does error_highlight use escape sequence? • It can, but I
have no plan • Why? • Terminal messages are often copy/pasted as a text • Critical use of escape sequence will make people want to use screenshots • ➔ Searching by error message will be difficult 38 -
Lots of considerations and tasks • ANSI escape code (=
terminal font style) • Gems • did_you_mean gem (thanks to Yuki Nishijima) • syntax_suggest gem (thanks to Richard Schneeman) 39 $!.detailed_message(highlight: true) undefined method `time' for 1:Integer (NoMethodError) 1.time ^^^^^ Did you mean? times