Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Find and Replace Code based on AST

Find and Replace Code based on AST

This is my presentation on RubyKaigi 2023

synvert

flyerhzm

May 13, 2023
Tweet

More Decks by flyerhzm

Other Decks in Technology

Transcript

  1. What is AST? Abstract Syntax Tree class RubyKaigi def year

    2023 end def location 'Matsumoto' end end s(:class, s(:const, nil, :RubyKaigi), nil, s(:begin, s(:def, :year, s(:args), s(:int, 2023)), s(:def, :location, s(:args), s(:str, "Matsumoto"))))
  2. Why based on AST? Precise # match puts puts "hello

    world" puts"hello world" puts("hello world") puts( "hello world" ) puts\ "hello world" # not match puts 'puts "hello world"' <<~HEREDOC puts "hello world" HEREDOC custom_puts "hello world"
  3. Why based on AST? Precise Powerful Find hash pair where

    key and value are identical and delete the value Find old hash rocket syntax and replace with new hash syntax Find minitest setup methods that do not call super and prepend the super
  4. Node Query NQL - css like node query language Node

    Rules - hash object https://github.com/xinminlabs/node-query-ruby
  5. Find debug code puts "hello world" p current_user # NQL

    '.send[receiver=nil][message IN (puts p)]' # Node Rules { node_type: 'send', receiver: nil, message: { in: %w[puts p] } }
  6. Find two string arguments 'slug from title'.gsub(' ', '_') #

    NQL '.send[message=gsub] [arguments.size=2] [arguments.first=.str] [arguments.last=.str]' # Node Rules { node_type: 'send', message: 'gsub', arguments: { size: 2, first: { node_type: 'str' }, last: { node_type: 'str' } } }
  7. Find hash rocket key { :foo => 'bar' } #

    NQL %q{.pair[key=.sym][key=~/\A:([^'"]+)\z/]} # Node Rules { node_type: 'pair', key: /\A:([^'"]+)\z/ }
  8. Find hash pairs where the key and value are identical

    a: a, c: c, some_method( 1 2 b: b + 2, 3 4 d: d + 4 5 ) 6 # NQL '.pair[key="{{value}}"]' # Node Rules { node_type: 'pair', key: "{{value}}" }
  9. Find minitest setup methods that do not call super def

    setup do_something end class TestMeme < Minitest::Test 1 2 3 4 end 5 # NQL '.class[parent_class=Minitest::Test] .def[name=setup]:not_has(> .super)' # Node Rules { node_type: 'class', parent_class: 'Minitest::Test' } # first step { node_type: 'def', name: 'setup' } # second step children.none? { |node| node.type == 'super' } # third step
  10. Find hash value render nothing: true, status: :created render nothing:

    true 1 2 # NQL '.hash[status_value!=nil]' # Node Rules { node_type: 'hash', status_value: { not: nil } }
  11. Node Mutation Provides a set of APIs to rewrite node

    source code https://github.com/xinminlabs/node-mutation-ruby
  12. replace_with errors[:base] = "author not present" 1 2 errors.add(:base, "author

    not present") 3 replace_with node, '{{receiver}}.add({{arguments.0}}, {{arguments.1}})'
  13. replace_with errors[:base] = "author not present" 1 2 errors.add(:base, "author

    not present") 3 errors.add(:base, "author not present") errors[:base] = "author not present" 1 2 3 replace_with node, '{{receiver}}.add({{arguments.0}}, {{arguments.1}})'
  14. replace class Post < ActiveRecord::Base end 1 2 3 class

    Post < ApplicationRecord 4 end 5 replace node, :parent_class, with: 'ApplicationRecord'
  15. replace class Post < ActiveRecord::Base end 1 2 3 class

    Post < ApplicationRecord 4 end 5 class Post < ApplicationRecord end class Post < ActiveRecord::Base 1 end 2 3 4 5 replace node, :parent_class, with: 'ApplicationRecord'
  16. prepend def setup do_something end class TestMeme < Minitest::Test 1

    2 3 4 end 5 6 class TestMeme < Minitest::Test 7 def setup 8 super 9 do_something 10 end 11 end 12 prepend node, 'super'
  17. prepend def setup do_something end class TestMeme < Minitest::Test 1

    2 3 4 end 5 6 class TestMeme < Minitest::Test 7 def setup 8 super 9 do_something 10 end 11 end 12 def setup super do_something end class TestMeme < Minitest::Test 1 def setup 2 do_something 3 end 4 end 5 6 class TestMeme < Minitest::Test 7 8 9 10 11 end 12 prepend node, 'super'
  18. append class TestMeme < Minitest::Test def teardown clean_something end end

    1 2 3 4 5 6 class TestMeme < Minitest::Test 7 def teardown 8 clean_something 9 super 10 end 11 end 12 append node, 'super'
  19. append class TestMeme < Minitest::Test def teardown clean_something end end

    1 2 3 4 5 6 class TestMeme < Minitest::Test 7 def teardown 8 clean_something 9 super 10 end 11 end 12 class TestMeme < Minitest::Test def teardown clean_something super end end class TestMeme < Minitest::Test 1 def teardown 2 clean_something 3 end 4 end 5 6 7 8 9 10 11 12 append node, 'super'
  20. remove puts( "hello"\ "world" ) 1 2 3 4 5

    # nothing 6 # nothing puts( 1 "hello"\ 2 "world" 3 ) 4 5 6 remove node
  21. Hash node after_commit :add_to_index_later, on: :create 1 2 after_create_commit :add_to_index_later

    3 replace node, :message, with: 'after_{{arguments.-1.on_value.to_value}}_commit' delete node, 'arguments.-1.on_pair', and_comma: true
  22. Hash node after_commit :add_to_index_later, on: :create 1 2 after_create_commit :add_to_index_later

    3 after_create_commit :add_to_index_later after_commit :add_to_index_later, on: :create 1 2 3 replace node, :message, with: 'after_{{arguments.-1.on_value.to_value}}_commit' delete node, 'arguments.-1.on_pair', and_comma: true
  23. node_query + node_mutation require 'parser/current' require 'parser_node_ext' require 'node_query' require

    'node_mutation' 1 2 3 4 5 source = "foo\nputs foo\n" 6 node = Parser::CurrentRuby.parse(source) 7 query = NodeQuery.new('.send[receiver=nil][message IN (p puts)]') 8 mutation = NodeMutation.new(source) 9 query.query_nodes(node).each do |matching_node| 10 mutation.remove(matching_node) 11 end 12 result = mutation.process 13 result.new_source # foo\n 14
  24. node_query + node_mutation require 'parser/current' require 'parser_node_ext' require 'node_query' require

    'node_mutation' 1 2 3 4 5 source = "foo\nputs foo\n" 6 node = Parser::CurrentRuby.parse(source) 7 query = NodeQuery.new('.send[receiver=nil][message IN (p puts)]') 8 mutation = NodeMutation.new(source) 9 query.query_nodes(node).each do |matching_node| 10 mutation.remove(matching_node) 11 end 12 result = mutation.process 13 result.new_source # foo\n 14 source = "foo\nputs foo\n" node = Parser::CurrentRuby.parse(source) query = NodeQuery.new('.send[receiver=nil][message IN (p puts)]') query.query_nodes(node).each do |matching_node| end require 'parser/current' 1 require 'parser_node_ext' 2 require 'node_query' 3 require 'node_mutation' 4 5 6 7 8 mutation = NodeMutation.new(source) 9 10 mutation.remove(matching_node) 11 12 result = mutation.process 13 result.new_source # foo\n 14
  25. node_query + node_mutation require 'parser/current' require 'parser_node_ext' require 'node_query' require

    'node_mutation' 1 2 3 4 5 source = "foo\nputs foo\n" 6 node = Parser::CurrentRuby.parse(source) 7 query = NodeQuery.new('.send[receiver=nil][message IN (p puts)]') 8 mutation = NodeMutation.new(source) 9 query.query_nodes(node).each do |matching_node| 10 mutation.remove(matching_node) 11 end 12 result = mutation.process 13 result.new_source # foo\n 14 source = "foo\nputs foo\n" node = Parser::CurrentRuby.parse(source) query = NodeQuery.new('.send[receiver=nil][message IN (p puts)]') query.query_nodes(node).each do |matching_node| end require 'parser/current' 1 require 'parser_node_ext' 2 require 'node_query' 3 require 'node_mutation' 4 5 6 7 8 mutation = NodeMutation.new(source) 9 10 mutation.remove(matching_node) 11 12 result = mutation.process 13 result.new_source # foo\n 14 source = "foo\nputs foo\n" mutation = NodeMutation.new(source) mutation.remove(matching_node) result = mutation.process result.new_source # foo\n require 'parser/current' 1 require 'parser_node_ext' 2 require 'node_query' 3 require 'node_mutation' 4 5 6 node = Parser::CurrentRuby.parse(source) 7 query = NodeQuery.new('.send[receiver=nil][message IN (p puts)]') 8 9 query.query_nodes(node).each do |matching_node| 10 11 end 12 13 14
  26. Support View Files (Erb, Haml, Slim) # <%= h user.login

    %> # => # <%= user.login %> within_files Synvert::RAILS_VIEW_FILES do with_node type: 'send', receiver: nil, message: 'h' do replace_with '{{arguments}}' end end
  27. Synvert Javascript new Synvert.Rewriter("typescript", "array-type", () => { configure({ parser:

    Synvert.Parser.TYPESCRIPT }); // const z: Array<string|number> = ['a', 'b']; // => // const z: (string|number)[] = ['a', 'b']; withinFiles(Synvert.ALL_TS_FILES, function () { findNode( `.TypeReference [typeName.escapedText=Array] [typeArguments.0=.UnionType]`, () => { replaceWith("({{typeArguments}})[]"); } );