Converting AST

After RubyKaigi 2024 LR Parser Night w/ Asakusa.rb
https://andpad.connpass.com/event/321580/

Transcript

1. Converting AST June 25, 2024 After RubyKaigi 2024 LR Parser Night w/ Asakusa.rb @yui-knk Yuichiro Kaneko
2. About me Yuichiro Kaneko yui-knk (GitHub) / spikeolaf (Twitter) Treasure Data Engineering Manager of Applications Backend CRuby committer, mainly develop parser generator and parser Lrama LALR (1) parser generator (2023, Ruby 3.3) Love LR parser
3. May 15th - 17th, 2024 NAHA CULTURAL ARTS THEATER NAHArt, Okinawa, Japan The Bison Slayer The parser monster Parser界の黎明卿 The world is now in the great age of parsers. People are setting sail into the vast sea of parsers. - RubyKaigi 2023 LT- Yuichiro Kaneko https://twitter.com/kakutani/status/1657762294431105025/ NEW !!! NEW !!!
4. May 15th - 17th, 2024 NAHA CULTURAL ARTS THEATER NAHArt, Okinawa, Japan
5. ast_to_prism https://github.com/yui-knk/ast_to_prism Converts RubyVM::AST to Prism::Node Motivations Provide the compatible layer Clarify the difference between current Node and Prism Node Send feedbacks to current struct RNode Writing codes is easier than writing docs It may heavily depend on ruby master branch
6. They are watching me 2 watching and 3 stars !?
7. Who are Watchers and Stargazers
8. The position of ast_to_prism This is a missing piece Source code RubyVM::AST::Node parse.y struct rb_ast_struct struct RNode ast.c Prism::Node ast_to_prism Parser::AST::Node Prism::Translation::Parser RuboCop <<uses>>
9. How ast_to_prism works “parse” the source code then “convert” nodes to prism nodes
10. How ast_to_prism works “convert” method is a large case statement Some node types are supported, others are not
11. Milestones 1. Add insufficient information to Node 2. Change AST structures 3. Keep some nodes
12. Insuf fi cient information Location information E.g. “if”, “then”, “end” locations are not supported
13. Insuf fi cient information Simply, add them to nodes RNode is already free from 3 fields restriction
14. Change AST structures NODE_RESCUE wraps begin statement
15. Change AST structures NODE_ENSURE wraps NODE_RESCUE
16. Change AST structures Change it to be more simple structure
17. Change AST structures RNode is already free from 3 fields restriction !!!
18. Another example: UNDEF undef with list is expanded to multiple undef(s)
19. Another example: UNDEF Change NODE_UNDEF to holds all items
20. Keep some nodes NODE_BLOCK is omitted for single line body
21. Keep some nodes Right now, “convert_stmts” checks the node is BLOCK or not
22. Tips: range argument Why range optional argument is needed?
23. How “e” is managed Need to manage “e” variable
24. Prism::RescueNode Manages “e” in “reference“ field
25. NODE_RESBODY “e” is managed as the first line of nd_body
26. Need to split nd_body reference = nd_body.children[0] statements = convert_stmts(nd_body, 1..-1)
27. Expand NODE_RESCUE RNode is already free from 3 fields restriction !!!
28. How to develop Add test cases to “spec/basic_spec.rb” Implement it on “lib/ast_to_prism/parser.rb” If test passes OK. Go next NG. “pending” test case Create PR to ruby/ruby
29. “spec/basic_spec.rb” Add test cases
30. “spec/basic_spec.rb” Remove “not_supported” then implement convert logic
31. If you can not implement pending test case Investigate the reason why it can not Create PR to ruby/ruby Implement the conversion logic after merge
32. Next Steps Cover all simple cases in ast_to_prism Cover “test/*” files under ruby/ruby
33. May 15th - 17th, 2024 NAHA CULTURAL ARTS THEATER NAHArt, Okinawa, Japan Thank you !!!