Syntax Tree (RubyConf Mini)

Syntax Tree is a new toolkit for interacting with the Ruby parse tree. It can be used to analyze, inspect, debug, and format your Ruby code. In this talk we'll walk through how it works, how to use it in your own applications, and the exciting future possibilities enabled by Syntax Tree.

Transcript

1. Syntax Tree
2. Kevin Newton @kddnewton
3. Syntax Tree
4. What is Syntax Tree?
5. Syntax Tree concepts
6. Parser Syntax Tree concepts
7. Parser Syntax tree Syntax Tree concepts
8. Parser Syntax tree Visitor Syntax Tree concepts
9. Parser · Syntax tree · Visitor
10. Parser · Syntax tree · Visitor Parser
11. Matz is nice so we are nice. Parser · Syntax tree · Visitor
12. Lexical analysis Parser · Syntax tree · Visitor
13. Matz is nice so we are nice. Parser · Syntax tree · Visitor
14. Matz is nice so we are nice . Parser · Syntax tree · Visitor
15. is nice so we are nice . Matz Parser · Syntax tree · Visitor
16. is nice so we are nice . Matz Parser · Syntax tree · Visitor
17. Matz is nice we are nice . so Parser · Syntax tree · Visitor
18. Matz is nice so we are nice . Parser · Syntax tree · Visitor
19. Matz is nice so we are nice . Parser · Syntax tree · Visitor
20. Matz is nice so we are . nice Parser · Syntax tree · Visitor
21. Matz is nice so we are nice . Parser · Syntax tree · Visitor
22. Matz is nice so we are nice . Parser · Syntax tree · Visitor
23. Semantic analysis Parser · Syntax tree · Visitor
24. Parser · Syntax tree · Visitor is nice are nice
25. Grammar Parser · Syntax tree · Visitor
26. Grammar Parser · Syntax tree · Visitor verb-phrase : VERB ADJECTIVE
27. Parser · Syntax tree · Visitor is nice are nice
28. Parser · Syntax tree · Visitor is nice verb phrase verb phrase are nice
29. Parser · Syntax tree · Visitor Matz is nice verb phrase we verb phrase are nice
30. Grammar Parser · Syntax tree · Visitor verb-phrase : VERB ADJECTIVE
31. Grammar Parser · Syntax tree · Visitor verb-phrase : VERB ADJECTIVE 
 subject-phrase : NOUN verb-phrase
32. Parser · Syntax tree · Visitor Matz is nice verb phrase we verb phrase are nice
33. Parser · Syntax tree · Visitor Matz is nice subject phrase subject phrase verb phrase we verb phrase are nice
34. Parser · Syntax tree · Visitor Matz is nice so subject phrase subject phrase verb phrase we verb phrase are nice
35. Grammar Parser · Syntax tree · Visitor verb-phrase : VERB ADJECTIVE 
 subject-phrase : NOUN verb-phrase
36. Grammar Parser · Syntax tree · Visitor verb-phrase : VERB ADJECTIVE 
 subject-phrase : NOUN verb-phrase subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase
37. Parser · Syntax tree · Visitor Matz is nice so subject phrase subject phrase verb phrase we verb phrase are nice
38. Parser · Syntax tree · Visitor Matz is nice so subord. conjunction subject phrase subject phrase verb phrase we verb phrase are nice
39. Parser · Syntax tree · Visitor Matz is nice so . subord. conjunction subject phrase subject phrase verb phrase we verb phrase are nice
40. verb-phrase : VERB ADJECTIVE 
 subject-phrase : NOUN verb-phrase 
 subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase Grammar Parser · Syntax tree · Visitor
41. verb-phrase : VERB ADJECTIVE 
 subject-phrase : NOUN verb-phrase 
 subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase 
 sentence : subordinating-conjunction PERIOD Grammar Parser · Syntax tree · Visitor
42. Parser · Syntax tree · Visitor Matz is nice so . subord. conjunction subject phrase subject phrase verb phrase we verb phrase are nice
43. Parser · Syntax tree · Visitor Matz is nice so . sentence subord. conjunction subject phrase subject phrase verb phrase we verb phrase are nice
44. verb-phrase : VERB ADJECTIVE 
 subject-phrase : NOUN verb-phrase 
 subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase 
 sentence : subordinating-conjunction PERIOD Grammar Parser · Syntax tree · Visitor
45. Parser · Syntax tree · Visitor Syntax tree
46. verb-phrase : VERB ADJECTIVE 
 subject-phrase : NOUN verb-phrase 
 subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase 
 sentence : subordinating-conjunction PERIOD Grammar Parser · Syntax tree · Visitor
47. Parser · Syntax tree · Visitor verb-phrase : VERB ADJECTIVE subject-phrase : NOUN verb-phrase subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase sentence : subordinating-conjunction PERIOD
48. Parser · Syntax tree · Visitor class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end end verb-phrase : VERB ADJECTIVE subject-phrase : NOUN verb-phrase subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase sentence : subordinating-conjunction PERIOD
49. Parser · Syntax tree · Visitor verb-phrase : VERB ADJECTIVE subject-phrase : NOUN verb-phrase subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase sentence : subordinating-conjunction PERIOD class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end end
50. Parser · Syntax tree · Visitor class SubjectPhrase attr_reader :noun, :verb_phrase def initialize(noun:, verb_phrase:) @noun = noun @verb_phrase = verb_phrase end end verb-phrase : VERB ADJECTIVE subject-phrase : NOUN verb-phrase subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase sentence : subordinating-conjunction PERIOD class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end end
51. Parser · Syntax tree · Visitor verb-phrase : VERB ADJECTIVE subject-phrase : NOUN verb-phrase subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase sentence : subordinating-conjunction PERIOD class SubjectPhrase attr_reader :noun, :verb_phrase def initialize(noun:, verb_phrase:) @noun = noun @verb_phrase = verb_phrase end end class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end end
52. Parser · Syntax tree · Visitor verb-phrase : VERB ADJECTIVE subject-phrase : NOUN verb-phrase subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase sentence : subordinating-conjunction PERIOD class SubordinatingConjunction attr_reader :left, :conjunction, :right def initialize(left:, conjunction:, right:) @left = left @conjunction = conjunction @right = right end end class SubjectPhrase attr_reader :noun, :verb_phrase def initialize(noun:, verb_phrase:) @noun = noun @verb_phrase = verb_phrase end end class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end end
53. Parser · Syntax tree · Visitor class SubjectPhrase attr_reader :noun, :verb_phrase def initialize(noun:, verb_phrase:) @noun = noun @verb_phrase = verb_phrase end end verb-phrase : VERB ADJECTIVE subject-phrase : NOUN verb-phrase subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase sentence : subordinating-conjunction PERIOD class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end end class SubordinatingConjunction attr_reader :left, :conjunction, :right def initialize(left:, conjunction:, right:) @left = left @conjunction = conjunction @right = right end end
54. Parser · Syntax tree · Visitor class SubjectPhrase attr_reader :noun, :verb_phrase def initialize(noun:, verb_phrase:) @noun = noun @verb_phrase = verb_phrase end end verb-phrase : VERB ADJECTIVE subject-phrase : NOUN verb-phrase subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase sentence : subordinating-conjunction PERIOD class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end end class SubordinatingConjunction attr_reader :left, :conjunction, :right def initialize(left:, conjunction:, right:) @left = left @conjunction = conjunction @right = right end end class Sentence attr_reader :phrase, :punctuation def initialize(phrase:, punctuation:) @phrase = phrase @punctuation = punctuation end end
55. Parser · Syntax tree · Visitor class SubjectPhrase attr_reader :noun, :verb_phrase def initialize(noun:, verb_phrase:) @noun = noun @verb_phrase = verb_phrase end end verb-phrase : VERB ADJECTIVE subject-phrase : NOUN verb-phrase subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase sentence : subordinating-conjunction PERIOD class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end end class SubordinatingConjunction attr_reader :left, :conjunction, :right def initialize(left:, conjunction:, right:) @left = left @conjunction = conjunction @right = right end end class Sentence attr_reader :phrase, :punctuation def initialize(phrase:, punctuation:) @phrase = phrase @punctuation = punctuation end end
56. Parser · Syntax tree · Visitor class SubjectPhrase attr_reader :noun, :verb_phrase def initialize(noun:, verb_phrase:) @noun = noun @verb_phrase = verb_phrase end end verb-phrase : VERB ADJECTIVE subject-phrase : NOUN verb-phrase subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase sentence : subordinating-conjunction PERIOD class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end end class SubordinatingConjunction attr_reader :left, :conjunction, :right def initialize(left:, conjunction:, right:) @left = left @conjunction = conjunction @right = right end end class Sentence attr_reader :phrase, :punctuation def initialize(phrase:, punctuation:) @phrase = phrase @punctuation = punctuation end end class Verb attr_reader :value def initialize(value:) @value = value end end
57. Parser · Syntax tree · Visitor class SubjectPhrase attr_reader :noun, :verb_phrase def initialize(noun:, verb_phrase:) @noun = noun @verb_phrase = verb_phrase end end verb-phrase : VERB ADJECTIVE subject-phrase : NOUN verb-phrase subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase sentence : subordinating-conjunction PERIOD class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end end class SubordinatingConjunction attr_reader :left, :conjunction, :right def initialize(left:, conjunction:, right:) @left = left @conjunction = conjunction @right = right end end class Sentence attr_reader :phrase, :punctuation def initialize(phrase:, punctuation:) @phrase = phrase @punctuation = punctuation end end class Verb attr_reader :value def initialize(value:) @value = value end end
58. Parser · Syntax tree · Visitor class SubjectPhrase attr_reader :noun, :verb_phrase def initialize(noun:, verb_phrase:) @noun = noun @verb_phrase = verb_phrase end end verb-phrase : VERB ADJECTIVE subject-phrase : NOUN verb-phrase subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase sentence : subordinating-conjunction PERIOD class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end end class SubordinatingConjunction attr_reader :left, :conjunction, :right def initialize(left:, conjunction:, right:) @left = left @conjunction = conjunction @right = right end end class Sentence attr_reader :phrase, :punctuation def initialize(phrase:, punctuation:) @phrase = phrase @punctuation = punctuation end end class Verb attr_reader :value def initialize(value:) @value = value end end class Adjective attr_reader :value def initialize(value:) @value = value end end
59. Parser · Syntax tree · Visitor class SubjectPhrase attr_reader :noun, :verb_phrase def initialize(noun:, verb_phrase:) @noun = noun @verb_phrase = verb_phrase end end verb-phrase : VERB ADJECTIVE subject-phrase : NOUN verb-phrase subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase sentence : subordinating-conjunction PERIOD class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end end class SubordinatingConjunction attr_reader :left, :conjunction, :right def initialize(left:, conjunction:, right:) @left = left @conjunction = conjunction @right = right end end class Sentence attr_reader :phrase, :punctuation def initialize(phrase:, punctuation:) @phrase = phrase @punctuation = punctuation end end class Verb attr_reader :value def initialize(value:) @value = value end end class Adjective attr_reader :value def initialize(value:) @value = value end end
60. Parser · Syntax tree · Visitor class SubjectPhrase attr_reader :noun, :verb_phrase def initialize(noun:, verb_phrase:) @noun = noun @verb_phrase = verb_phrase end end verb-phrase : VERB ADJECTIVE subject-phrase : NOUN verb-phrase subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase sentence : subordinating-conjunction PERIOD class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end end class SubordinatingConjunction attr_reader :left, :conjunction, :right def initialize(left:, conjunction:, right:) @left = left @conjunction = conjunction @right = right end end class Sentence attr_reader :phrase, :punctuation def initialize(phrase:, punctuation:) @phrase = phrase @punctuation = punctuation end end class Verb attr_reader :value def initialize(value:) @value = value end end class Adjective attr_reader :value def initialize(value:) @value = value end end class Noun attr_reader :value def initialize(value:) @value = value end end
61. Parser · Syntax tree · Visitor class SubjectPhrase attr_reader :noun, :verb_phrase def initialize(noun:, verb_phrase:) @noun = noun @verb_phrase = verb_phrase end end verb-phrase : VERB ADJECTIVE subject-phrase : NOUN verb-phrase subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase sentence : subordinating-conjunction PERIOD class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end end class SubordinatingConjunction attr_reader :left, :conjunction, :right def initialize(left:, conjunction:, right:) @left = left @conjunction = conjunction @right = right end end class Sentence attr_reader :phrase, :punctuation def initialize(phrase:, punctuation:) @phrase = phrase @punctuation = punctuation end end class Verb attr_reader :value def initialize(value:) @value = value end end class Adjective attr_reader :value def initialize(value:) @value = value end end class Noun attr_reader :value def initialize(value:) @value = value end end
62. Parser · Syntax tree · Visitor class SubjectPhrase attr_reader :noun, :verb_phrase def initialize(noun:, verb_phrase:) @noun = noun @verb_phrase = verb_phrase end end verb-phrase : VERB ADJECTIVE subject-phrase : NOUN verb-phrase subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase sentence : subordinating-conjunction PERIOD class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end end class SubordinatingConjunction attr_reader :left, :conjunction, :right def initialize(left:, conjunction:, right:) @left = left @conjunction = conjunction @right = right end end class Sentence attr_reader :phrase, :punctuation def initialize(phrase:, punctuation:) @phrase = phrase @punctuation = punctuation end end class Verb attr_reader :value def initialize(value:) @value = value end end class Adjective attr_reader :value def initialize(value:) @value = value end end class Noun attr_reader :value def initialize(value:) @value = value end end class Conjunction attr_reader :value def initialize(value:) @value = value end end
63. Parser · Syntax tree · Visitor class SubjectPhrase attr_reader :noun, :verb_phrase def initialize(noun:, verb_phrase:) @noun = noun @verb_phrase = verb_phrase end end verb-phrase : VERB ADJECTIVE subject-phrase : NOUN verb-phrase subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase sentence : subordinating-conjunction PERIOD class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end end class SubordinatingConjunction attr_reader :left, :conjunction, :right def initialize(left:, conjunction:, right:) @left = left @conjunction = conjunction @right = right end end class Sentence attr_reader :phrase, :punctuation def initialize(phrase:, punctuation:) @phrase = phrase @punctuation = punctuation end end class Verb attr_reader :value def initialize(value:) @value = value end end class Adjective attr_reader :value def initialize(value:) @value = value end end class Noun attr_reader :value def initialize(value:) @value = value end end class Conjunction attr_reader :value def initialize(value:) @value = value end end
64. Parser · Syntax tree · Visitor class SubjectPhrase attr_reader :noun, :verb_phrase def initialize(noun:, verb_phrase:) @noun = noun @verb_phrase = verb_phrase end end verb-phrase : VERB ADJECTIVE subject-phrase : NOUN verb-phrase subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase sentence : subordinating-conjunction PERIOD class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end end class SubordinatingConjunction attr_reader :left, :conjunction, :right def initialize(left:, conjunction:, right:) @left = left @conjunction = conjunction @right = right end end class Sentence attr_reader :phrase, :punctuation def initialize(phrase:, punctuation:) @phrase = phrase @punctuation = punctuation end end class Verb attr_reader :value def initialize(value:) @value = value end end class Adjective attr_reader :value def initialize(value:) @value = value end end class Noun attr_reader :value def initialize(value:) @value = value end end class Conjunction attr_reader :value def initialize(value:) @value = value end end class Period attr_reader :value def initialize(value:) @value = value end end
65. Parser · Syntax tree · Visitor class SubjectPhrase attr_reader :noun, :verb_phrase def initialize(noun:, verb_phrase:) @noun = noun @verb_phrase = verb_phrase end end verb-phrase : VERB ADJECTIVE subject-phrase : NOUN verb-phrase subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase sentence : subordinating-conjunction PERIOD class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end end class SubordinatingConjunction attr_reader :left, :conjunction, :right def initialize(left:, conjunction:, right:) @left = left @conjunction = conjunction @right = right end end class Sentence attr_reader :phrase, :punctuation def initialize(phrase:, punctuation:) @phrase = phrase @punctuation = punctuation end end class Verb attr_reader :value def initialize(value:) @value = value end end class Adjective attr_reader :value def initialize(value:) @value = value end end class Noun attr_reader :value def initialize(value:) @value = value end end class Conjunction attr_reader :value def initialize(value:) @value = value end end class Period attr_reader :value def initialize(value:) @value = value end end
66. Parser · Syntax tree · Visitor class SubjectPhrase attr_reader :noun, :verb_phrase def initialize(noun:, verb_phrase:) @noun = noun @verb_phrase = verb_phrase end end class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end end class SubordinatingConjunction attr_reader :left, :conjunction, :right def initialize(left:, conjunction:, right:) @left = left @conjunction = conjunction @right = right end end class Sentence attr_reader :phrase, :punctuation def initialize(phrase:, punctuation:) @phrase = phrase @punctuation = punctuation end end class Verb attr_reader :value def initialize(value:) @value = value end end class Adjective attr_reader :value def initialize(value:) @value = value end end class Noun attr_reader :value def initialize(value:) @value = value end end class Conjunction attr_reader :value def initialize(value:) @value = value end end class Period attr_reader :value def initialize(value:) @value = value end end
67. class SubjectPhrase attr_reader :noun, :verb_phrase def initialize(noun:, verb_phrase:) @noun = noun @verb_phrase = verb_phrase end end class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end end class SubordinatingConjunction attr_reader :left, :conjunction, :right def initialize(left:, conjunction:, right:) @left = left @conjunction = conjunction @right = right end end class Sentence attr_reader :phrase, :punctuation def initialize(phrase:, punctuation:) @phrase = phrase @punctuation = punctuation end end class Verb attr_reader :value def initialize(value:) @value = value end end class Adjective attr_reader :value def initialize(value:) @value = value end end class Noun attr_reader :value def initialize(value:) @value = value end end class Conjunction attr_reader :value def initialize(value:) @value = value end end class Period attr_reader :value def initialize(value:) @value = value end end Parser · Syntax tree · Visitor
68. class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end end Parser · Syntax tree · Visitor
69. Parser · Syntax tree · Visitor class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end def deconstruct_keys(keys) { verb: verb, adjective: adjective } end def ==(other) other in VerbPhrase[ verb: ^(verb), adjective: ^(adjective) ] end end
70. class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end end Parser · Syntax tree · Visitor
71. Parser · Syntax tree · Visitor class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end def copy(verb: self.verb, adjective: self.adjective) VerbPhrase.new(verb: verb, adjective: adjective) end end
72. class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end end Parser · Syntax tree · Visitor
73. Parser · Syntax tree · Visitor class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end def accept(visitor) visitor.visit_verb_phrase(self) end def child_nodes [verb, adjective] end end
74. Parser · Syntax tree · Visitor class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end def accept(visitor) visitor.visit_verb_phrase(self) end def child_nodes [verb, adjective] end end
75. Visitor Parser · Syntax tree · Visitor
76. class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end def accept(visitor) visitor.visit_verb_phrase(self) end def child_nodes [verb, adjective] end end Parser · Syntax tree · Visitor
77. class VerbPhrase def accept(visitor) visitor.visit_verb_phrase(self) end end Parser · Syntax tree · Visitor
78. class VerbPhrase def accept(visitor) visitor.visit_verb_phrase(self) end end 
 class Visitor def visit_verb_phrase(node) # ... end end node.accept(Visitor.new) Parser · Syntax tree · Visitor
79. class VerbPhrase def accept(visitor) visitor.visit_verb_phrase(self) end end 
 class Visitor def visit_adjective(node) end def visit_conjunction(node) end def visit_noun(node) end def visit_verb(node) end def visit_punctuation(node) end def visit_sentence(node) end def visit_subordinating_conjunction(node) end def visit_subject_phrase(node) end def visit_verb_phrase(node) # ... end end node.accept(Visitor.new) Parser · Syntax tree · Visitor
80. class Visitor def visit_adjective(node) end def visit_conjunction(node) end def visit_noun(node) end def visit_verb(node) end def visit_punctuation(node) end def visit_sentence(node) end def visit_subordinating_conjunction(node) end def visit_subject_phrase(node) end def visit_verb_phrase(node) # ... end end node.accept(Visitor.new) Parser · Syntax tree · Visitor
81. class Visitor def visit(node) node.accept(self) end def visit_adjective(node) end def visit_conjunction(node) end def visit_noun(node) end def visit_verb(node) end def visit_punctuation(node) end def visit_sentence(node) end def visit_subordinating_conjunction(node) end def visit_subject_phrase(node) end def visit_verb_phrase(node) # ... end end node.accept(Visitor.new) Parser · Syntax tree · Visitor
82. Parser · Syntax tree · Visitor class Visitor def visit(node) node.accept(self) end def visit_adjective(node) end def visit_conjunction(node) end def visit_noun(node) end def visit_verb(node) end def visit_punctuation(node) end def visit_sentence(node) end def visit_subordinating_conjunction(node) end def visit_subject_phrase(node) end def visit_verb_phrase(node) node.child_nodes.each { |node| visit(node) } end end node.accept(Visitor.new)
83. Parser · Syntax tree · Visitor class Visitor def visit(node) node.accept(self) end def visit_adjective(node) end def visit_conjunction(node) end def visit_noun(node) end def visit_verb(node) end def visit_punctuation(node) end def visit_sentence(node) end def visit_subordinating_conjunction(node) end def visit_subject_phrase(node) end def visit_verb_phrase(node) node.child_nodes.each { |node| visit(node) } end end
84. Parser · Syntax tree · Visitor class Visitor def visit(node) node.accept(self) end def visit_child_nodes(node) node.child_nodes.each { |node| visit(node) } end def visit_adjective(node) end def visit_conjunction(node) end def visit_noun(node) end def visit_verb(node) end def visit_punctuation(node) end def visit_sentence(node) end def visit_subordinating_conjunction(node) end def visit_subject_phrase(node) end def visit_verb_phrase(node) visit_child_nodes(node) end end
85. Parser · Syntax tree · Visitor class Visitor def visit(node) node.accept(self) end def visit_child_nodes(node) node.child_nodes.each { |node| visit(node) } end def visit_adjective(node) end def visit_conjunction(node) end def visit_noun(node) end def visit_verb(node) end def visit_punctuation(node) end def visit_sentence(node) end def visit_subordinating_conjunction(node) end def visit_subject_phrase(node) end alias visit_verb_phrase visit_child_nodes end
86. Parser · Syntax tree · Visitor class Visitor def visit(node) node.accept(self) end def visit_child_nodes(node) node.child_nodes.each { |node| visit(node) } end alias visit_adjective visit_child_nodes alias visit_conjunction visit_child_nodes alias visit_noun visit_child_nodes alias visit_verb visit_child_nodes alias visit_punctuation visit_child_nodes alias visit_sentence visit_child_nodes alias visit_subordinating_conjunction 
 visit_child_nodes alias visit_subject_phrase visit_child_nodes alias visit_verb_phrase visit_child_nodes end
87. Parser · Syntax tree · Visitor
88. Parser · Syntax tree · Visitor class SentenceCountVisitor < Visitor attr_reader :count def initialize @count = 0 end def visit_sentence(node) @count += 1 super end end visitor = SentenceCountVisitor.new node.accept(visitor) puts visitor.count
89. Parser · Syntax tree · Visitor class PrettyPrintVisitor < Visitor attr_reader :q def initialize @q = PP.new(+"") end def visit_sentence(node) q.group(2, "sentence(", ")") do visit(node.phrase) visit(node.punctuation) end end end visitor = PrettyPrintVisitor.new node.accept(visitor) visitor.q.flush puts visitor.q.output
90. Parser · Syntax tree · Visitor class FormatterVisitor < Visitor attr_reader :q def initialize @q = PP.new(+"") end def visit_sentence(node) q.group do q.format(node.phrase) q.format(node.punctuation) end end end visitor = FormatterVisitor.new node.accept(visitor) visitor.q.flush puts visitor.q.output
91. What is Syntax Tree?
92. Syntax Tree
93. Object layer representing the result of parsing Ruby Syntax Tree
94. Object layer representing the result of parsing Ruby Tools to interact with and manipulate that layer Syntax Tree
95. Build a syntax tree Syntax Tree
96. Build a syntax tree Format the syntax tree Syntax Tree
97. Build a syntax tree Format the syntax tree Create a CLI Syntax Tree
98. Build a syntax tree Format the syntax tree Create a CLI Create a language server (LSP) Syntax Tree
99. Build a syntax tree Format the syntax tree Create a CLI Create a language server (LSP) Translate the syntax tree Syntax Tree
100. Build · Format · CLI · LSP · Translate Syntax Tree
101. Build a syntax tree Build · Format · CLI · LSP · Translate
102. Define each node Build · Format · CLI · LSP · Translate
103. Build · Format · CLI · LSP · Translate class VCall < Node attr_reader :value, :location, :comments def initialize(value:, location:, comments: []) @value = value @location = location @comments = comments end def accept(visitor) visitor.visit_vcall(self) end def child_nodes [value] end alias deconstruct child_nodes def deconstruct_keys(keys) { value: value, location: location, comments: comments } end end
104. Build · Format · CLI · LSP · Translate class VCall < Node attr_reader :value, :location, :comments def initialize(value:, location:, comments: []) @value = value @location = location @comments = comments end def accept(visitor) visitor.visit_vcall(self) end def child_nodes [value] end alias deconstruct child_nodes def deconstruct_keys(keys) { value: value, location: location, comments: comments } end end Named fields
105. Build · Format · CLI · LSP · Translate class VCall < Node attr_reader :value, :location, :comments def initialize(value:, location:, comments: []) @value = value @location = location @comments = comments end def accept(visitor) visitor.visit_vcall(self) end def child_nodes [value] end alias deconstruct child_nodes def deconstruct_keys(keys) { value: value, location: location, comments: comments } end end Named fields #location/#comments
106. Build · Format · CLI · LSP · Translate class VCall < Node attr_reader :value, :location, :comments def initialize(value:, location:, comments: []) @value = value @location = location @comments = comments end def accept(visitor) visitor.visit_vcall(self) end def child_nodes [value] end alias deconstruct child_nodes def deconstruct_keys(keys) { value: value, location: location, comments: comments } end end Named fields #location/#comments #accept/#child_nodes
107. Build · Format · CLI · LSP · Translate class VCall < Node attr_reader :value, :location, :comments def initialize(value:, location:, comments: []) @value = value @location = location @comments = comments end def accept(visitor) visitor.visit_vcall(self) end def child_nodes [value] end alias deconstruct child_nodes def deconstruct_keys(keys) { value: value, location: location, comments: comments } end end Named fields #location/#comments #accept/#child_nodes #deconstruct/ 
 #deconstruct_keys
108. Build · Format · CLI · LSP · Translate class VCall < Node attr_reader :value, :location, :comments def initialize(value:, location:, comments: []) @value = value @location = location @comments = comments end def accept(visitor) visitor.visit_vcall(self) end def child_nodes [value] end alias deconstruct child_nodes def deconstruct_keys(keys) { value: value, location: location, comments: comments } end end Named fields #location/#comments #accept/#child_nodes #deconstruct/ 
 #deconstruct_keys Immutable
109. Define each node Build · Format · CLI · LSP · Translate
110. Ripper::EVENTS.count # => 190 Define each node Build · Format · CLI · LSP · Translate
111. Ripper::EVENTS.count # => 190 SyntaxTree::Node.descendants.count 
 # => 162 Define each node Build · Format · CLI · LSP · Translate
112. Ripper::EVENTS.count # => 190 SyntaxTree::Node.descendants.count 
 # => 162 qwords_new/qwords_add -> SyntaxTree::QWords Define each node Build · Format · CLI · LSP · Translate
113. How we use ripper Build · Format · CLI · LSP · Translate
114. var_ref : user_variable { /*%%%*/ if (!($$ = gettable(p, $1, &@$))) $$ = NEW_BEGIN(0, &@$); /*% if (id_is_var(p, get_id($1))) { $$ = dispatch1(var_ref, $1); } else { $$ = dispatch1(vcall, $1); } %*/ } | keyword_variable { /*%%%*/ if (!($$ = gettable(p, $1, &@$))) $$ = NEW_BEGIN(0, &@$); /*% %*/ /*% ripper: var_ref!($1) %*/ } ; Build · Format · CLI · LSP · Translate
115. var_ref : user_variable { /*%%%*/ if (!($$ = gettable(p, $1, &@$))) $$ = NEW_BEGIN(0, &@$); /*% if (id_is_var(p, get_id($1))) { $$ = dispatch1(var_ref, $1); } else { $$ = dispatch1(vcall, $1); } %*/ } | keyword_variable { /*%%%*/ if (!($$ = gettable(p, $1, &@$))) $$ = NEW_BEGIN(0, &@$); /*% %*/ /*% ripper: var_ref!($1) %*/ } ; Build · Format · CLI · LSP · Translate
116. class Foo def bar baz end end Build · Format · CLI · LSP · Translate
117. class Foo def bar baz end end Build · Format · CLI · LSP · Translate class Parser < Ripper def on_const(value) = [:@const, value] def on_ident(value) = [:@ident, value] def on_const_ref(const) = [:const_ref, const] def on_vcall(ident) = [:vcall, ident] def on_class(const, superclass, bodystmt) [:class, [const, superclass, bodystmt]] end def on_def(name, params, bodystmt) [:def, [name, params, bodystmt]] end end
118. class Foo def bar baz end end Build · Format · CLI · LSP · Translate class Parser < Ripper def on_const(value) = [:@const, value] def on_ident(value) = [:@ident, value] def on_const_ref(const) = [:const_ref, const] def on_vcall(ident) = [:vcall, ident] def on_class(const, superclass, bodystmt) [:class, [const, superclass, bodystmt]] end def on_def(name, params, bodystmt) [:def, [name, params, bodystmt]] end end
119. class Foo def bar baz end end Build · Format · CLI · LSP · Translate class Parser < Ripper def on_const(value) = [:@const, value] def on_ident(value) = [:@ident, value] def on_const_ref(const) = [:const_ref, const] def on_vcall(ident) = [:vcall, ident] def on_class(const, superclass, bodystmt) [:class, [const, superclass, bodystmt]] end def on_def(name, params, bodystmt) [:def, [name, params, bodystmt]] end end
120. class Foo def bar baz end end Build · Format · CLI · LSP · Translate class Parser < Ripper def on_const(value) = [:@const, value] def on_ident(value) = [:@ident, value] def on_const_ref(const) = [:const_ref, const] def on_vcall(ident) = [:vcall, ident] def on_class(const, superclass, bodystmt) [:class, [const, superclass, bodystmt]] end def on_def(name, params, bodystmt) [:def, [name, params, bodystmt]] end end
121. class Foo def bar baz end end Build · Format · CLI · LSP · Translate class Parser < Ripper def on_const(value) = [:@const, value] def on_ident(value) = [:@ident, value] def on_const_ref(const) = [:const_ref, const] def on_vcall(ident) = [:vcall, ident] def on_class(const, superclass, bodystmt) [:class, [const, superclass, bodystmt]] end def on_def(name, params, bodystmt) [:def, [name, params, bodystmt]] end end
122. class Foo def bar baz end end Build · Format · CLI · LSP · Translate class Parser < Ripper def on_const(value) = [:@const, value] def on_ident(value) = [:@ident, value] def on_const_ref(const) = [:const_ref, const] def on_vcall(ident) = [:vcall, ident] def on_class(const, superclass, bodystmt) [:class, [const, superclass, bodystmt]] end def on_def(name, params, bodystmt) [:def, [name, params, bodystmt]] end end
123. class Foo def bar baz end end Build · Format · CLI · LSP · Translate class Parser < Ripper def on_const(value) = [:@const, value] def on_ident(value) = [:@ident, value] def on_const_ref(const) = [:const_ref, const] def on_vcall(ident) = [:vcall, ident] def on_class(const, superclass, bodystmt) [:class, [const, superclass, bodystmt]] end def on_def(name, params, bodystmt) [:def, [name, params, bodystmt]] end end
124. class Foo def bar baz end end Build · Format · CLI · LSP · Translate class Parser < Ripper def on_const(value) = [:@const, value] def on_ident(value) = [:@ident, value] def on_const_ref(const) = [:const_ref, const] def on_vcall(ident) = [:vcall, ident] def on_class(const, superclass, bodystmt) [:class, [const, superclass, bodystmt]] end def on_def(name, params, bodystmt) [:def, [name, params, bodystmt]] end end
125. Build · Format · CLI · LSP · Translate class Parser < Ripper def on_const(value) = [:@const, value] def on_ident(value) = [:@ident, value] def on_const_ref(const) = [:const_ref, const] def on_vcall(ident) = [:vcall, ident] def on_class(const, superclass, bodystmt) [:class, [const, superclass, bodystmt]] end def on_def(name, params, bodystmt) [:def, [name, params, bodystmt]] end end
126. Build · Format · CLI · LSP · Translate class Parser < Ripper def on_const(value) = [:@const, value] def on_ident(value) = [:@ident, value] def on_kw(value) = @stack << [:@kw, value] def on_const_ref(const) = [:const_ref, const] def on_vcall(ident) = [:vcall, ident] def on_class(const, superclass, bodystmt) kw = @stack.rfind { |token| token in [:@kw, "class"] } [:class, [@stack.delete(kw), const, superclass, bodystmt]] end def on_def(name, params, bodystmt) kw = @stack.rfind { |token| token in [:@kw, "def"] } [:def, [@stack.delete(kw), name, params, bodystmt]] end end
127. Build · Format · CLI · LSP · Translate class Parser < Ripper def on_const(value) = [:@const, value] def on_ident(value) = [:@ident, value] def on_kw(value) = @stack << [:@kw, value] def on_const_ref(const) = [:const_ref, const] def on_vcall(ident) = [:vcall, ident] def on_class(const, superclass, bodystmt) kw = @stack.rfind { |token| token in [:@kw, "class"] } [:class, [@stack.delete(kw), const, superclass, bodystmt]] end def on_def(name, params, bodystmt) kw = @stack.rfind { |token| token in [:@kw, "def"] } [:def, [@stack.delete(kw), name, params, bodystmt]] end end
128. Build · Format · CLI · LSP · Translate -> (positional; local1, local2) {}
129. Build · Format · CLI · LSP · Translate -> (positional; local1, local2) {} range = (location.start_char + 1)...location.end_char locals = lambda_locals(source[range]) Paren.new( lparen: params.lparen, contents: LambdaVar.new( params: params.contents, locals: locals, location: location ), location: params.location, comments: params.comments )
130. class Foo def bar baz end end Build · Format · CLI · LSP · Translate
131. Build · Format · CLI · LSP · Translate class Foo # comment def bar baz end end
132. Build · Format · CLI · LSP · Translate class Foo def bar # comment baz end end
133. Build · Format · CLI · LSP · Translate class Foo def bar baz # comment end end
134. Build · Format · CLI · LSP · Translate # comment class Foo def bar baz end end
135. Build · Format · CLI · LSP · Translate class Foo # comment def bar baz end end
136. Build · Format · CLI · LSP · Translate # comment1 class Foo # comment2 # comment3 def bar # comment4 baz # comment5 end end
137. Build · Format · CLI · LSP · Translate # comment1 class Foo # comment2 # comment3 def bar # comment4 baz # comment5 end end [ ]
138. Build · Format · CLI · LSP · Translate # comment1 class Foo # comment2 # comment3 def bar # comment4 baz # comment5 end end [ # comment1, ]
139. Build · Format · CLI · LSP · Translate # comment1 class Foo # comment2 # comment3 def bar # comment4 baz # comment5 end end [ # comment1, # comment2, ]
140. Build · Format · CLI · LSP · Translate # comment1 class Foo # comment2 # comment3 def bar # comment4 baz # comment5 end end [ # comment1, # comment2, # comment3, ]
141. Build · Format · CLI · LSP · Translate # comment1 class Foo # comment2 # comment3 def bar # comment4 baz # comment5 end end [ # comment1, # comment2, # comment3, # comment4, ]
142. Build · Format · CLI · LSP · Translate # comment1 class Foo # comment2 # comment3 def bar # comment4 baz # comment5 end end [ # comment1, # comment2, # comment3, # comment4, # comment5 ]
143. Build · Format · CLI · LSP · Translate [ # comment1, # comment2, # comment3, # comment4, # comment5 ] (program (statements ((class (const_ref (const "Foo")) (bodystmt (statements ((void_stmt), (def (ident "bar") (params) (bodystmt (statements ((vcall (ident "baz")))))))))))))
144. Build · Format · CLI · LSP · Translate (program (statements ((comment "# comment1"), (class (const_ref (const "Foo") ((comment "# comment2"))) (bodystmt (statements ((void_stmt), (comment "# comment3"), (def (ident "bar") (params ((comment "# comment4"))) (bodystmt (statements ((vcall (ident "baz") ((comment "# comment5"))))))))))))))
145. Walking the tree Build · Format · CLI · LSP · Translate
146. Build · Format · CLI · LSP · Translate class Visitor def visit(node) node&.accept(self) end def visit_child_nodes(node) node.child_nodes.map do |node| visit(node) end end end
147. Build · Format · CLI · LSP · Translate class VCall < Node def accept(visitor) visitor.visit_vcall(self) end end class Visitor def visit(node) node&.accept(self) end def visit_child_nodes(node) node.child_nodes.map do |node| visit(node) end end end
148. Build · Format · CLI · LSP · Translate class VCall < Node def accept(visitor) visitor.visit_vcall(self) end end # Visit an ARef node. alias visit_aref visit_child_nodes # Visit an ARefField node. alias visit_aref_field visit_child_nodes # Visit an Alias node. alias visit_alias visit_child_nodes # Visit an ArgBlock node. alias visit_arg_block visit_child_nodes # Visit an ArgParen node. alias visit_arg_paren visit_child_nodes # Visit an ArgStar node. alias visit_arg_star visit_child_nodes # Visit an Args node. alias visit_args visit_child_nodes # Visit an ArgsForward node. alias visit_args_forward visit_child_nodes # Visit an ArrayLiteral node. alias visit_array visit_child_nodes # Visit an AryPtn node. alias visit_aryptn visit_child_nodes # Visit an Assign node. alias visit_assign visit_child_nodes # Visit an Assoc node. alias visit_assoc visit_child_nodes # Visit an AssocSplat node. alias visit_assoc_splat visit_child_nodes # Visit a Backref node. alias visit_backref visit_child_nodes class Visitor def visit(node) node&.accept(self) end def visit_child_nodes(node) node.child_nodes.map do |node| visit(node) end end end
149. Build · Format · CLI · LSP · Translate class VCall < Node def accept(visitor) visitor.visit_vcall(self) end end alias visit_vcall visit_child_nodes # Visit a VoidStmt node. alias visit_void_stmt visit_child_nodes # Visit a When node. alias visit_when visit_child_nodes # Visit a While node. alias visit_while visit_child_nodes # Visit a WhileMod node. alias visit_while_mod visit_child_nodes # Visit a Word node. alias visit_word visit_child_nodes # Visit a Words node. alias visit_words visit_child_nodes # Visit a WordsBeg node. alias visit_words_beg visit_child_nodes # Visit a XString node. alias visit_xstring visit_child_nodes # Visit a XStringLiteral node. alias visit_xstring_literal visit_child_nodes # Visit a Yield node. alias visit_yield visit_child_nodes # Visit a Yield0 node. alias visit_yield0 visit_child_nodes # Visit a ZSuper node. alias visit_zsuper visit_child_nodes # Visit an EndContent node. alias visit___end__ visit_child_nodes class Visitor def visit(node) node&.accept(self) end def visit_child_nodes(node) node.child_nodes.map do |node| visit(node) end end end
150. Build · Format · CLI · LSP · Translate class RegexpVisitor < Visitor def visit_regexp_literal(node) # do something with regexp literal node here node.options # call super to continue walking the tree super end end
151. Build · Format · CLI · LSP · Translate class JSONVisitor < Visitor def visit_aref(node) node(node, "aref") do field("collection", node.collection) field("index", node.index) comments(node) end end def visit_aref_field(node) node(node, "aref_field") do field("collection", node.collection) field("index", node.index) comments(node) end end def visit_alias(node) node(node, "alias") do field("left", node.left) field("right", node.right) comments(node) end end
152. Format the syntax tree Build · Format · CLI · LSP · Translate
153. Algorithm Build · Format · CLI · LSP · Translate
154. prettyprint Algorithm Build · Format · CLI · LSP · Translate
155. prettyprint Build group, text, breakable to layout content Algorithm Build · Format · CLI · LSP · Translate
156. prettyprint Build group, text, breakable to layout content Break outermost groups to fit to print width Algorithm Build · Format · CLI · LSP · Translate
157. Build · Format · CLI · LSP · Translate [foo1, foo2, [bar1, bar2, bar3, bar4], foo3, foo4, [baz1, baz2]]
158. Build · Format · CLI · LSP · Translate [foo1, foo2, [bar1, bar2, bar3, bar4], foo3, foo4, [baz1, baz2]]
159. Build · Format · CLI · LSP · Translate [foo1, foo2, [bar1, bar2, bar3, bar4], foo3, foo4, [baz1, baz2]]
160. Build · Format · CLI · LSP · Translate [ foo1, foo2, [bar1, bar2, bar3, bar4], foo3, foo4, [baz1, baz2] ]
161. Build · Format · CLI · LSP · Translate [ foo1, foo2, [bar1, bar2, bar3, bar4], foo3, foo4, [baz1, baz2] ]
162. Build · Format · CLI · LSP · Translate [ foo1, foo2, [ bar1, bar2, bar3, bar4 ], foo3, foo4, [baz1, baz2] ]
163. breakGroup([group([text(["["]), indent([breakable, text(["foo1", ","]), breakable, text(["foo2", ","]), breakable, group([text(["["]), indent([breakable, text(["bar1", ","]), breakable, text(["bar2", ","]), breakable, text(["bar3", ","]), breakable, text(["bar4"])]), breakable, text(["]"])]), text([","]), breakable, text(["foo3", ","]), breakable, text(["foo4", ","]), breakable, ... Build · Format · CLI · LSP · Translate
164. breakGroup([group([text(["["]), indent([breakable, text(["foo1", ","]), breakable, text(["foo2", ","]), breakable, group([text(["["]), indent([breakable, text(["bar1", ","]), breakable, text(["bar2", ","]), breakable, text(["bar3", ","]), breakable, text(["bar4"])]), breakable, text(["]"])]), text([","]), breakable, text(["foo3", ","]), breakable, text(["foo4", ","]), breakable, ... Build · Format · CLI · LSP · Translate
165. breakGroup([group([text(["["]), indent([breakable, text(["foo1", ","]), breakable, text(["foo2", ","]), breakable, group([text(["["]), indent([breakable, text(["bar1", ","]), breakable, text(["bar2", ","]), breakable, text(["bar3", ","]), breakable, text(["bar4"])]), breakable, text(["]"])]), text([","]), breakable, text(["foo3", ","]), breakable, text(["foo4", ","]), breakable, ... Build · Format · CLI · LSP · Translate
166. breakGroup([group([text(["["]), indent([breakable, text(["foo1", ","]), breakable, text(["foo2", ","]), breakable, group([text(["["]), indent([breakable, text(["bar1", ","]), breakable, text(["bar2", ","]), breakable, text(["bar3", ","]), breakable, text(["bar4"])]), breakable, text(["]"])]), text([","]), breakable, text(["foo3", ","]), breakable, text(["foo4", ","]), breakable, ... Build · Format · CLI · LSP · Translate
167. breakGroup([group([text(["["]), indent([breakable, text(["foo1", ","]), breakable, text(["foo2", ","]), breakable, group([text(["["]), indent([breakable, text(["bar1", ","]), breakable, text(["bar2", ","]), breakable, text(["bar3", ","]), breakable, text(["bar4"])]), breakable, text(["]"])]), text([","]), breakable, text(["foo3", ","]), breakable, text(["foo4", ","]), breakable, ... Build · Format · CLI · LSP · Translate
168. I made too many additions to prettyprint Build · Format · CLI · LSP · Translate 😅😅😅
169. prettyprint Build · Format · CLI · LSP · Translate
170. prettier_print Build · Format · CLI · LSP · Translate
171. Build · Format · CLI · LSP · Translate if foo bar += baz end
172. Build · Format · CLI · LSP · Translate if foo bar += baz end
173. Build · Format · CLI · LSP · Translate bar += baz if foo
174. Build · Format · CLI · LSP · Translate if foo bar else baz end
175. Build · Format · CLI · LSP · Translate if foo bar else baz end
176. Build · Format · CLI · LSP · Translate foo ? bar : baz
177. Build · Format · CLI · LSP · Translate here_is_a_method_call(1, 2, <<~HEREDOC this is the content of the heredoc HEREDOC )
178. Build · Format · CLI · LSP · Translate here_is_a_method_call(1, 2, <<~HEREDOC this is the content of the heredoc HEREDOC )
179. Build · Format · CLI · LSP · Translate here_is_a_method_call(1, 2, <<~HEREDOC) this is the content of the heredoc HEREDOC
180. Build · Format · CLI · LSP · Translate def foo return [1, 2, 3].map { |element| element * 2 } # comment end
181. Build · Format · CLI · LSP · Translate def foo return [1, 2, 3].map { |element| element * 2 } # comment end
182. Build · Format · CLI · LSP · Translate def foo return( [1, 2, 3].map do |element| element * 2 end ) # comment end
183. Build · Format · CLI · LSP · Translate class If < Node def format(q) if node.consequent || node.statements.empty? q.group { format_break(q, force: true) } else q.group do q .if_break { format_break(q, force: false) } .if_flat do Parentheses.flat(q) do q.format(node.statements) q.text(" if ") q.format(node.predicate) end end end end end end
184. Build · Format · CLI · LSP · Translate class If < Node def format(q) if node.consequent || node.statements.empty? q.group { format_break(q, force: true) } else q.group do q .if_break { format_break(q, force: false) } .if_flat do Parentheses.flat(q) do q.format(node.statements) q.text(" if ") q.format(node.predicate) end end end end end end
185. Create a CLI Build · Format · CLI · LSP · Translate
186. Build · Format · CLI · LSP · Translate stree ast [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Print out the AST corresponding to the given files stree check [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Check that the given files are formatted as syntax tree would format them stree debug [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Check that the given files can be formatted idempotently stree doc [--plugins=...] [-e SCRIPT] FILE Print out the doc tree that would be used to format the given files stree expr [-e SCRIPT] FILE Print out a pattern-matching Ruby expression that would match the first expression of the given files stree format [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Print out the formatted version of the given files stree json [--plugins=...] [-e SCRIPT] FILE Print out the JSON representation of the given files stree match [--plugins=...] [-e SCRIPT] FILE Print out a pattern-matching Ruby expression that would match the given files stree help Display this help message stree lsp [--plugins=...] [--print-width=NUMBER] Run syntax tree in language server mode stree search PATTERN [-e SCRIPT] FILE Search for the given pattern in the given files stree version Output the current version of syntax tree stree write [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Read, format, and write back the source of the given files --plugins=... A comma-separated list of plugins to load. --print-width=NUMBER The maximum line width to use when formatting. -e SCRIPT Parse an inline Ruby string.
187. Build · Format · CLI · LSP · Translate stree ast [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Print out the AST corresponding to the given files stree format [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Print out the formatted version of the given files stree match [--plugins=...] [-e SCRIPT] FILE Print out a pattern-matching Ruby expression that would match the given files stree lsp [--plugins=...] [--print-width=NUMBER] Run syntax tree in language server mode stree search PATTERN [-e SCRIPT] FILE Search for the given pattern in the given files
188. Build · Format · CLI · LSP · Translate stree ast [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Print out the AST corresponding to the given files stree format [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Print out the formatted version of the given files stree match [--plugins=...] [-e SCRIPT] FILE Print out a pattern-matching Ruby expression that would match the given files stree lsp [--plugins=...] [--print-width=NUMBER] Run syntax tree in language server mode stree search PATTERN [-e SCRIPT] FILE Search for the given pattern in the given files
189. Build · Format · CLI · LSP · Translate $ stree ast -e '1+2'
190. Build · Format · CLI · LSP · Translate $ stree ast -e '1+2' (program (statements ((binary (int "1") + (int "2")))))
191. Build · Format · CLI · LSP · Translate stree ast [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Print out the AST corresponding to the given files stree format [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Print out the formatted version of the given files stree match [--plugins=...] [-e SCRIPT] FILE Print out a pattern-matching Ruby expression that would match the given files stree lsp [--plugins=...] [--print-width=NUMBER] Run syntax tree in language server mode stree search PATTERN [-e SCRIPT] FILE Search for the given pattern in the given files
192. Build · Format · CLI · LSP · Translate stree ast [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Print out the AST corresponding to the given files stree format [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Print out the formatted version of the given files stree match [--plugins=...] [-e SCRIPT] FILE Print out a pattern-matching Ruby expression that would match the given files stree lsp [--plugins=...] [--print-width=NUMBER] Run syntax tree in language server mode stree search PATTERN [-e SCRIPT] FILE Search for the given pattern in the given files
193. Build · Format · CLI · LSP · Translate $ stree format -e '1+2'
194. Build · Format · CLI · LSP · Translate $ stree format -e '1+2' 1 + 2
195. Build · Format · CLI · LSP · Translate stree ast [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Print out the AST corresponding to the given files stree format [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Print out the formatted version of the given files stree match [--plugins=...] [-e SCRIPT] FILE Print out a pattern-matching Ruby expression that would match the given files stree lsp [--plugins=...] [--print-width=NUMBER] Run syntax tree in language server mode stree search PATTERN [-e SCRIPT] FILE Search for the given pattern in the given files
196. Build · Format · CLI · LSP · Translate stree ast [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Print out the AST corresponding to the given files stree format [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Print out the formatted version of the given files stree match [--plugins=...] [-e SCRIPT] FILE Print out a pattern-matching Ruby expression that would match the given files stree lsp [--plugins=...] [--print-width=NUMBER] Run syntax tree in language server mode stree search PATTERN [-e SCRIPT] FILE Search for the given pattern in the given files
197. Build · Format · CLI · LSP · Translate $ stree match -e '1+2'
198. Build · Format · CLI · LSP · Translate $ stree match -e '1+2' SyntaxTree::Program[ statements: SyntaxTree::Statements[ body: [ SyntaxTree::Binary[ left: SyntaxTree::Int[value: "1"], operator: :+, right: SyntaxTree::Int[value: "2"] ] ] ] ]
199. Build · Format · CLI · LSP · Translate stree ast [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Print out the AST corresponding to the given files stree format [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Print out the formatted version of the given files stree match [--plugins=...] [-e SCRIPT] FILE Print out a pattern-matching Ruby expression that would match the given files stree lsp [--plugins=...] [--print-width=NUMBER] Run syntax tree in language server mode stree search PATTERN [-e SCRIPT] FILE Search for the given pattern in the given files
200. Build · Format · CLI · LSP · Translate stree ast [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Print out the AST corresponding to the given files stree format [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Print out the formatted version of the given files stree match [--plugins=...] [-e SCRIPT] FILE Print out a pattern-matching Ruby expression that would match the given files stree lsp [--plugins=...] [--print-width=NUMBER] Run syntax tree in language server mode stree search PATTERN [-e SCRIPT] FILE Search for the given pattern in the given files
201. Build · Format · CLI · LSP · Translate $ stree search Int -e '1+2'
202. Build · Format · CLI · LSP · Translate $ stree search Int -e '1+2' script:1:0: 1+2 script:1:2: 1+2
203. Build · Format · CLI · LSP · Translate $ stree search Int -e '1+2'
204. Build · Format · CLI · LSP · Translate $ stree search 'Int[value: "2"]' -e '1+2'
205. Build · Format · CLI · LSP · Translate $ stree search 'Int[value: "2"]' -e '1+2' 
 
 script:1:2: 1+2
206. Build · Format · CLI · LSP · Translate $ stree search <(stree match -e '1+2') \ 
 -e '1+2'
207. Build · Format · CLI · LSP · Translate $ stree search <(stree match -e '1+2') \ 
 -e '1+2' 
 
 script:1:0: 1+2
208. Build · Format · CLI · LSP · Translate stree ast [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Print out the AST corresponding to the given files stree format [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Print out the formatted version of the given files stree match [--plugins=...] [-e SCRIPT] FILE Print out a pattern-matching Ruby expression that would match the given files stree lsp [--plugins=...] [--print-width=NUMBER] Run syntax tree in language server mode stree search PATTERN [-e SCRIPT] FILE Search for the given pattern in the given files stree write [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Read, format, and write back the source of the given files
209. Build · Format · CLI · LSP · Translate stree ast [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Print out the AST corresponding to the given files stree format [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Print out the formatted version of the given files stree match [--plugins=...] [-e SCRIPT] FILE Print out a pattern-matching Ruby expression that would match the given files stree lsp [--plugins=...] [--print-width=NUMBER] Run syntax tree in language server mode stree search PATTERN [-e SCRIPT] FILE Search for the given pattern in the given files stree write [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Read, format, and write back the source of the given files
210. Create a language server Build · Format · CLI · LSP · Translate
211. Language server protocol Build · Format · CLI · LSP · Translate
212. Language server protocol Runs a JSON RPC server behind an editor Build · Format · CLI · LSP · Translate
213. Language server protocol Runs a JSON RPC server behind an editor textDocument/didChange textDocument/didOpen textDocument/didClose Build · Format · CLI · LSP · Translate
214. Language server protocol Runs a JSON RPC server behind an editor textDocument/didChange textDocument/didOpen textDocument/didClose textDocument/formatting Build · Format · CLI · LSP · Translate
215. Language server protocol Runs a JSON RPC server behind an editor textDocument/didChange textDocument/didOpen textDocument/didClose textDocument/formatting textDocument/inlayHint Build · Format · CLI · LSP · Translate
216. Build · Format · CLI · LSP · Translate
217. Shopify/ruby-lsp Build · Format · CLI · LSP · Translate
218. Shopify/ruby-lsp Document highlight Build · Format · CLI · LSP · Translate
219. Shopify/ruby-lsp Document highlight Document symbols Build · Format · CLI · LSP · Translate
220. Shopify/ruby-lsp Document highlight Document symbols Folding ranges Build · Format · CLI · LSP · Translate
221. Shopify/ruby-lsp Document highlight Document symbols Folding ranges Formatting Build · Format · CLI · LSP · Translate
222. Shopify/ruby-lsp Document highlight Document symbols Folding ranges Formatting Selection ranges Build · Format · CLI · LSP · Translate
223. Shopify/ruby-lsp Document highlight Document symbols Folding ranges Formatting Selection ranges Semantic highlighting Build · Format · CLI · LSP · Translate
224. Translate the syntax tree Build · Format · CLI · LSP · Translate
225. Translate the syntax tree Build · Format · CLI · LSP · Translate
226. Translate the syntax tree seattlerb/ruby_parser Build · Format · CLI · LSP · Translate
227. Build · Format · CLI · LSP · Translate class RubyParserVisitor < Visitor def visit_binary(node) left = visit(node.left) right = visit(node.right) case node in { operator: :and } s(:and, left, right) in { operator: :!~ } s(:not, s(:call, left, :=~, right)) else s(:call, left, node.operator, right) end end # ... end
228. Translate the syntax tree seattlerb/ruby_parser Build · Format · CLI · LSP · Translate
229. Translate the syntax tree seattlerb/ruby_parser whitequark/parser Build · Format · CLI · LSP · Translate
230. Build · Format · CLI · LSP · Translate class ParserVisitor < Visitor def visit_binary(node) left = visit(node.left) right = visit(node.right) case node in { operator: :and } s(:and, [left, right]) in { operator: :or } s(:or, [left, right]) else s(:send, [left, node.operator, right]) end end # ... end
231. Translate the syntax tree seattlerb/ruby_parser whitequark/parser Build · Format · CLI · LSP · Translate
232. Translate the syntax tree seattlerb/ruby_parser whitequark/parser YARV Build · Format · CLI · LSP · Translate
233. Build · Format · CLI · LSP · Translate class YARVVisitor < Visitor def visit_binary(node) case node.operator when :"&&" # ... when :"||" # ... else visit(node.left) visit(node.right) builder.send( node.operator, 1, VM_CALL_ARGS_SIMPLE ) end end # ... end
234. Translate the syntax tree seattlerb/ruby_parser whitequark/parser YARV Build · Format · CLI · LSP · Translate
235. Translate the syntax tree seattlerb/ruby_parser whitequark/parser YARV Ruby Build · Format · CLI · LSP · Translate
236. Build · Format · CLI · LSP · Translate class MutationVisitor < Visitor def visit_binary(node) node.copy( left: visit(node.left), right: visit(node.right) ) end end
237. Build · Format · CLI · LSP · Translate mutator = SyntaxTree.mutation do |mutator| query = <<~RUBY IfNode[predicate: Assign | OpAssign] | UnlessNode[predicate: Assign | OpAssign] RUBY mutator.mutate(query) do |node| node.copy(predicate: SyntaxTree::Paren.new( lparen: SyntaxTree::LParen.default, contents: node.predicate, location: node.predicate.location )) end end mutated = node.accept(mutator)
238. Translate the syntax tree seattlerb/ruby_parser whitequark/parser YARV Ruby Build · Format · CLI · LSP · Translate
239. Syntax Tree
240. Parser Syntax Tree
241. Parser Syntax tree Syntax Tree
242. Parser Syntax tree Visitor Syntax Tree
243. Syntax Tree
244. Syntax Tree Build a syntax tree
245. Syntax Tree Build a syntax tree Format the syntax tree
246. Syntax Tree Build a syntax tree Format the syntax tree Create a CLI
247. Syntax Tree Build a syntax tree Format the syntax tree Create a CLI Create a language server (LSP)
248. Syntax Tree Build a syntax tree Format the syntax tree Create a CLI Create a language server (LSP) Translate the syntax tree
249. Thank you!
250. Syntax Tree github.com/ruby-syntax-tree Kevin Newton @kddnewton