Good first issues of TypeProf

@ RubyKaigi 2024

https://rubykaigi.org/2024/presentations/mametter.html

1. Good first issues of TypeProf Yusuke Endoh (@mame) RubyKaigi 2024
2. Yusuke Endoh / @mametter • A Ruby committer working at STORES w/ @ko1 • My recent Ruby change: New error message format 2 test.rb:1:in `func': unhandled exception Old: test.rb:1:in 'Foo#func': unhandled exception New: Backtick → Single quote (markdown friendly!) Class name (not only method name)
3. Yusuke Endoh / @mametter • A Ruby committer working at STORES w/ @ko1 • My recent STORES work (?) 3 1. Ruby "enbugging" quiz https://ruby-quiz-2024.storesinc.tech 2. Ruby "Quine" paper craft A spiral Ruby code Come to the STORES booth!
4. Today's talk • What is TypeProf? • How to play with TypeProf • How to develop TypeProf • TypeProf internals overview • Good first issues 4
5. Today's topic: TypeProf Ruby Editor support • Error report, go to definition, completion, etc. • Without (many) type annotations! 5.ti| 1 + "str" TypeError Do you mean: 5.times
6. Demo Features • Type inference, error report • Go to definition • Completion • (Inline RBS, flow analysis) New Features • Go to references • Go to type references • Automatic rename (method, constant)
8. Today's talk • What is TypeProf? • How to play with TypeProf • How to develop TypeProf • TypeProf internals overview • Good first issues 8
9. How to play with TypeProf: Overview 1. Install Ruby 3.3 2. Install VSCode extension 3. git clone TypeProf 4. Open the directory with VSCode Slide deck: https://speakerdeck.com/mame/good-first-issues-of-typeprof
10. How to play with TypeProf (1/2) • Install Ruby 3.3 • Install VSCode extension (search "typeprof") 1. search "typeprof" from Extensions 2. click
11. How to play with TypeProf (2/2) • git clone https://github.com/ruby/typeprof.git • Open the directory, and "lib/typeprof/core/ast.rb" 1. open a file in lib/typeprof/core 2. You'll see an inferred type signature
12. TypeProf with your code 1. Configure typeprof path 2. Create typeprof configuration file 3. Open your project directory and your file NOTE • It would fail as many Ruby constructs are unsupported yet • The details are tentative
13. TypeProf with your code (1/4) • Configure typeprof path 1. search "typeprof" from Settings 2. click
14. TypeProf with your code (2/4) • Configure typeprof path Write the absolute path to typeprof/bin/typeprof
15. TypeProf with your code (3/4) • Add typeprof.conf.json to your project 1. Create typeprof.conf.json 2. Write this
16. TypeProf with your code (4/4) • Open your Ruby file 1. Open app/test.rb 2. You'll be able to use TypeProf
17. Tips: If TypeProf fails... Keyword argument is not supported yet not supported yet: keyword_hash_node Tips 1. Fix the code 2. Press Ctrl+P 3. Choose "TypeProf: Restart"
18. How to play: Summary • Install VSCode extension • Clone the edge of TypeProf • Add typeprof.conf.json • Open a file (and :pray:) Please play with TypeProf! • Not practical yet, but it works for fun • If you find anything wrong, write a patch!
19. Today's talk • What is TypeProf? • How to play with TypeProf • How to develop TypeProf • TypeProf internals overview • Good first issues 19
20. How to develop TypeProf 1. Write a test scenario 2. Implement! 3. Make the scenario pass
21. Case study: interpolated symbol literal :"symbol#{ 42 }" #=> :symbol42
22. Test scenario DSL ## update: test.rb def foo :"symbol#{ 42 }" end ## assert: test.rb class Object def foo: -> Symbol end scenario/misc/dsym.rb Input: If you write test.rb like this Expected output: The inferred type signature should be like this
23. Why "scenario"? → To describe the edit ## update: test.rb def foo = "symbol#{ 42 }" ## assert: test.rb class Object def foo: -> String end ## update: test.rb def foo = :"symbol#{ 42 }" ## assert: test.rb class Object def foo: -> Symbol end If you write this The inferred type should be String If you edit the file like this (The colon is added) The inferred type should be updated to Symbol
24. Run the test scenario • Use "tool/scenario_runner.rb" $ tool/scenario_runner.rb scenario/misc/dsym.rb Loaded suite tool/scenario_runner Started E =================================================================== Error: test: scenario/misc/dsym.rb(ScenarioCompiler::ScenarioTest): RuntimeError: not supported yet: interpolated_symbol_node … TypeProf does not support the construct yet
25. Implement! diff --git a/lib/typeprof/core/ast.rb b/lib/typeprof/core/ast.rb index 28073bb..b2a3d78 100644 --- a/lib/typeprof/core/ast.rb +++ b/lib/typeprof/core/ast.rb @@ -170,6 +170,7 @@ module TypeProf::Core when :integer_node then IntegerNode.new(raw_node, lenv) when :float_node then FloatNode.new(raw_node, lenv) when :symbol_node then SymbolNode.new(raw_node, lenv) + when :interpolated_symbol_node then InterpolatedSymbolNode.new(raw_node, lenv) when :string_node then StringNode.new(raw_node, lenv, raw_node.content) when :source_file_node then StringNode.new(raw_node, lenv, "") when :interpolated_string_node then InterpolatedStringNode.new(raw_node, lenv) diff --git a/lib/typeprof/core/ast/value.rb b/lib/typeprof/core/ast/value.rb index 8e1fe99..e5c8031 100644 --- a/lib/typeprof/core/ast/value.rb +++ b/lib/typeprof/core/ast/value.rb @@ -69,6 +69,34 @@ module TypeProf::Core def install0(genv) = Source.new(Type::Symbol.new(genv, @lit)) end + class InterpolatedSymbolNode < Node + def initialize(raw_node, lenv) + super(raw_node, lenv) + @parts = [] + raw_node.parts.each do |raw_part| + case raw_part.type + when :string_node + @parts << AST.create_node(raw_part, lenv) + when :embedded_statements_node + @parts << AST.create_node(raw_part.statements, lenv) + else + raise "unknown symbol part: #{ raw_part.type }" + end + end + end + + attr_reader :parts + + def subnodes = { parts: } + + def install0(genv) + @parts.each do |subnode| + subnode.install(genv) + end + Source.new(genv.symbol_type) + end + end + class StringNode < LiteralNode def initialize(raw_node, lenv, content) super(raw_node, lenv, content) https://github.com/ruby/typeprof/pull/170
26. Make the test scenario pass • Use "tool/scenario_runner.rb" $ tool/scenario_runner.rb scenario/misc/dsym.rb Loaded suite tool/scenario_runner Started Finished in 0.002024702 seconds. ------------------------------------------------------------------- 1 tests, 1 assertions, 0 failures, 0 errors, 0 pendings, 0 omission s, 0 notifications 100% passed ------------------------------------------------------------------- 493.90 tests/s, 493.90 assertions/s
27. How to development: Summary 1. Write a test scenario 2. Implement it! 3. Make the scenario pass • Next, explain how to implement Note: PR is welcome to add only test scenarios! • Please add it to scenario/known-issues/
28. How to implement • Read the code!
29. Today's talk • What is TypeProf? • How to play with TypeProf • How to develop TypeProf • TypeProf internals overview • Good first issues 29
30. TypeProf internals (1): Dataflow analysis def foo(n) r = n.to_s r end x = 123 y = x z = foo(y) x y foo .to_s z n r int int int int 123 str str (toplevel) foo( ) int "Source" outputs fixed type(s) "Vertex" outputs input type(s) "Box" (Ruby call): passes args "Box" (RBS call): typechecks args str
31. TypeProf internals (2): Create dataflow graph y = x + 1 Graph AST ChangeSet code + x 1 = x based on Prism What to add to the graph + x y 1 CallBox: Edge: Edge: Edge: x y + + + + 1
32. TypeProf internals (3): Incremental update + x y 1 a.rb Add Box... Add Edge... b.rb Add Box... Add Edge... c.rb Add Box... Add Edge... Add Box... Add Edge... c.rb (2) edit revert this ChangeSet add a new ChangeSet Graph AST ChangeSet code
33. TypeProf source file structure • lib/typeprof/lsp: LSP server • lib/typeprof/core: Type Analyzer • service.rb: Endpoint APIs • ast/: Definitions of AST nodes • graph/: Dataflow graph (Source, Vertex, Box...) • type.rb: Definitions of types • env/: Environments (Class hierarchy, Method definitions, etc.)
34. Tips: How to read TypeProf • service.rb is a good start • Endpoint APIs to the analysis algorithm • Tweak this if you want to add/change LSP features • ast/ is (relatively) easy • Tweak this if you want to support Ruby/RBS constructs • env/ is hard • graph/ is lunatic • Challengers are welcome
35. Today's talk • What is TypeProf? • How to play with TypeProf • How to develop TypeProf • TypeProf internals overview • Good first issues 35
36. Good first issues: Level 1 (easy) • Play with TypeProf • If you find anything weird, add a new test scenario into scenario/known-issues/ • Support more Ruby/RBS constructs • Easy one can be done by copying other similar nodes • (But difficult ones would be Level 3) • Make it type inference command-line tool • Essential feature is already in tool/scenario_runner.rb
37. Good first issues: Level 2 (medium) • Improve the error messages • Many errors and warnings have not implemented yet • There are many comments like "TODO: report ..." • Implement "go to definition" of variables • There are all the parts we need (probably) foo(42) foo(42) failed to resolve overloads expected: String, found: Integer
38. Good first issues: Level 3 (hard) • Make the completion smarter • Currently, "TriggerCharacter" is supported for method name • Need to support other triggers, variable names, … • Improve the flow analysis • Currently, very limited set is supported • "if var", "if var.is_a?(Foo), "if @var", "if @var.is_a?(Foo)" • Need to support "&&", "||", etc. • Make TypeProf a plugin for ruby-lsp • Support rbs-inline • Find tasks!
39. Today's talk • What is TypeProf? • How to play with TypeProf • How to develop TypeProf • TypeProf internals overview • Good first issues • Conclusion 39
40. Future work • This year • Support full Ruby/RBS syntax • Bundle the latest TypeProf with Ruby • Dog-fooding • Next year.. • Implement many features • Improve accuracy and speed of analysis • Start towards Rails apps
41. Conclusion • Introduced how to play with TypeProf • Introduced how to contribute to TypeProf • Please play with TypeProf first • Write a test scenario • If possible, try reading the code Slide deck: https://speakerdeck.com/mame/good-first-issues-of-typeprof