The test code generator using static analysis and LLM

RubyKaigi2024でのLTです。

Transcript

1. The test code generator using static analysis and LLM Hashino Mikiko (@mikiko_bridge)
2. Agenda - What is “omochi” ? - Motivation - Demo - Architecture - Future Works - Conclusion
3. What is “omochi” ? Detect RSpec files and methods that have not yet been created, and generate sample code for them. feature - List all methods which is not tested by RSpec - Integrate with CI(github action) - Automatically generate RSpec code template
4. What is “omochi” ? feature 1. List all methods which is not tested by RSpec "Verify File List: [\"app/controllers/content_controller.rb\"]" "specファイルなし" ======= RESULT: app/controllers/content_controller.rb ======= - index - show - destroy - edit - update
5. What is “omochi” ? feature 2. Integrate with CI(github action)
6. What is “omochi” ? feature 3. Automatically generate RSpec code template
7. Agenda - What is “omochi” ? - Motivation - Demo - Architecture - Future Works - Conclusion
8. Motivation Omochi confirmed that all methods in PR have RSpec. It is hard to find unverified method by reviewer Some methods don’t have RSpec in your PR.
9. Agenda - What is “omochi” ? - Demo - Architecture - Future Works - Conclusion
11. Agenda - What is “omochi” ? - Demo - Architecture - Future Works - Conclusion
12. Architecture def local_diff_path ...省略 diff_command = 'git diff --diff-filter=d --name-only' diff_output, _diff_error, _diff_status = Open3.capture3(diff_command, chdir: '.') ...省略 diff_output.split("\n") end
13. Architecture def get_ast(diff_path) exprs = [] ast = Parser::CurrentRuby.parse(File.read(diff_path)) exprs << { ast: ast, filename: diff_path } end
14. Architecture
15. Architecture def dfs(node, filename, result) ...省略 case node.type when :def child_value = node.children[0] code = Unparser.unparse(node) result[child_value] = code end node.children.each { |child| dfs(child, filename, result) } end
16. Architecture def find_spec_file(diff_path) spec_path = File.join('spec', diff_path.gsub(/\.rb$/, '_spec.rb').gsub('app/', '')) File.exist?(spec_path) ? spec_path : nil end
17. Architecture def get_ast(diff_path) exprs = [] ast = Parser::CurrentRuby.parse(File.read(diff_path)) exprs << { ast: ast, filename: diff_path } end
18. Architecture def get_ast(diff_path) exprs = [] ast = Parser::CurrentRuby.parse(File.read(diff_path)) exprs << { ast: ast, filename: diff_path } end def dfs_describe(node, filename, def_name_arr) ...省略 case node.type when :send method_node = node.children[1] if node.children[1] == :describe def_name = node.children[2].children[0] # "Omochi::CLI" if !def_name.nil? && def_name.is_a?(String) def_name = get_pure_function_name(def_name) def_name_arr.push(def_name) end end end node.children.each { |child| dfs_describe(child, filename, def_name_arr) } def_name_arr end
19. Architecture def process_spec_file(diff_path, create_spec, perfect) ...省略 method_code = result[spec_def_name] puts '===================================================================' puts "#{spec_def_name} のテストを以下に表示します。 " create_spec_by_bedrock(method_code) end get_ignore_methods(diff_path).each do |def_name| result[def_name] = true if result.key?(def_name) end perfect = false if print_result(diff_path, result).size > 0 perfect end
20. Agenda - What is “omochi” ? - Demo - Architecture - Future Works - Conclusion
21. Future Works - Support request spec etc… - Write test codes for omochi
22. Agenda - What is “omochi” ? - Demo - Architecture - Future Works - Conclusion
23. Conclusion - Please commits to omochi !! - Let's use AST to solve problems using technology👍 https://github.com/mikik0/omochi blog GitHub