Share
Embed
Automatically generating types by running tests
2025/04/16 RubyKaigi Day1
https://rubykaigi.org/2025/presentations/sinsoku_listy.html#day1
Takumi Shotoku
April 16, 2025
Transcript
68 slides
-
About me • Takumi Shotoku • @sinsokuʢGitHubʣ • @sinsoku_listyʢXʣ •
Timee, Inc. and mov inc. • Omotesando.rb, Asakusa-bashi.rbs 2 -
The API is very simple trace = RBS:"Trace.new # Enable
tracing to call methods trace.enable # Call methods user = User.new("Yukihiro", "Matsumoto") user.say_hello # Disable tracing trace.disable # Save RBS type declarations as comments trace.save_comments 6 -
Insert RBS comments to Ruby files class User # @rbs
(String, String) -" void def initialize(first_name, last_name) @first_name = first_name @last_name = last_name end # @rbs () -" String def full_name "#%@first_name} #%@last_name}" end # @rbs () -" void def say_hello puts "hi, #%full_name}." end end 7 -
Integration with RSpec Add the following code to spec_helper.rb. if
ENV["RBS_TRACE"] RSpec.configure do |config| trace = RBS:#Trace.new config.before(:suite) { trace.enable } config.after(:suite) do trace.disable trace.save_comments end end end 8 -
Integration with Minitest Add the following code to test_helper.rb. if
ENV["RBS_TRACE"] trace = RBS:"Trace.new trace.enable Minitest.after_run do trace.disable trace.save_comments end end 9 -
Then, simply run the tests $ RBS_TRACE=1 bin/rails test Automatically
inserts RBS comments into Ruby files. 10 -
Questions • ❓ What is type generation from running tests?
• ❓ Can it be used practically in Rails apps? • ❓ What about performance? 11 -
Tests and type declarations are aligned • No tests, no
type declarations • Not enough tests, not enough type declarations • Tests are wrong, type declarations are wrong 12 -
Only executed methods RSpec.design User do describe "#foo" do it
{ .." } end end class User # @rbs () -" void def foo end def bar end end 13 -
Only arguments passed in tests RSpec.describe "Calculator" do describe ".sum"
do context "when args are Integer" subject { Calculator.sum(1, 2) } it { is_expected.to eq 3 } end end end class Calculator # @rbs (Integer, Integer) -" Integer def self.sum(x, y) x.to_i + y.to_i end end 14 -
Two test cases can be generated RSpec.describe "Calculator" do describe
".sum" do context "when args are Integer" subject { Calculator.sum(1, 2) } it { is_expected.to eq 3 } end context "when args are String" subject { Calculator.sum("1", "2") } it { is_expected.to eq 3 } end end end class Calculator # @rbs (Integer|String, Integer|String) -" Integer def self.sum(x, y) x.to_i + y.to_i end end 15 -
Type declarations are generated as tests RSpec.describe "User" do describe
"#initialize" do subject { User.new(nil) } it { is_expected.to be } end end class User # @rbs (nil) -" void def initialize(name) @name = name end end 16 -
Questions • ✅ What is type generation from running tests?
• ❓ Can it be used practically in Rails apps? • ❓ What about performance? 17 -
It works in the following two Rails apps RED MINE
flexible project management 18 -
See for yourself • Redmine • https://github.com/sinsoku/redmine/pull/2 • Mastodon •
https://github.com/sinsoku/mastodon/pull/4 20 -
Questions • ✅ What is type generation from running tests?
• ✅ Can it be used practically in Rails apps? • ❓ What about performance? 21 -
Test execution time on local machine1 • Redmine - parallelize(workers:
1) • Simply run bin/rails test • Mastodon - using flatware • Run all tests in parallel 1 MacBook Pro (14-inch, 2021) M1 Max 64GB 22 -
Comparison results before after di! slower Redmine 2m14s 12m 7s
9m53s 5.43x Mastodon 45s 2m54s 2m 9s 3.87x 23 -
Organization's Rails apps jobs before after di! slower Timee, Inc.2
35 7m16s 10m40s 3m24s 1.47x mov inc.3 20 15m52s 22m16s 6m24s 1.40x 3 mov: Actions, RSpec, r7kamura/split-tests-by-timings 2 Timee: Actions, RSpec, https://github.com/mtsmfm/split-test 24 -
rbs-trace only needs to be run once. The performance trade-off
is acceptable. 25 -
Questions • ✅ What is type generation from running tests?
• ✅ Can it be used practically in Rails apps? • ✅ What about performance? If you have other questions, ask me after this talk! 26 -
Core design 1. Record class name for arguments and return
values 2. Perform this process on all method calls 3. Insert RBS comments above the method definition 28 -
Core design 1. Record class name for arguments and return
values 2. Perform this process on all method calls 3. Insert RBS comments above the method definition This can be solved with TracePoint. 29 -
Record argument types with TracePoint class User def initialize(first_name, last_name)
p("initialize") end end TracePoint.trace(:call) do |tp| p(defined_class: tp.defined_class, method_id: tp.method_id, parameters: tp.parameters) end User.new("Yukihiro", "Matsumoto") # {defined_class: User, method_id: :initialize, parameters: [[:req, :first_name], [:req, :last_name]]} # "initialize" 30 -
Record argument types with TracePoint class User def initialize(first_name, last_name)
p("initialize") end end TracePoint.trace(:call) do |tp| tp.parameters.each do |_type, name| # [[:req, :first_name], [:req, :last_name]] value = tp.binding.local_variable_get(name) p(name:, value:, class_name: value.class) end end User.new("Yukihiro", "Matsumoto") # {name: :first_name, value: "Yukihiro", class: String} # {name: :last_name, value: "Matsumoto", class: String} # "initialize" 31 -
Record the return type with TracePoint class Calculator def self.sum(x,
y) p("sum") x + y end end TracePoint.trace(:return) do |tp| p(return_value: tp.return_value, class: tp.return_value.class) end Calculator.sum(1, 2) # "sum" # {return_value: 3, class: Integer} 32 -
Issues • NoMethodError occurs in the class method • The
class name becomes ActiveRecord:"Relation • Does not support void type • Does not work in parallel testing 34 -
The class method missing • Redmine has a class that
extends BasicObject • BasicObject does not have a class method class A < BasicObject end obj = A.new p(class: obj.class) # undefined method 'class' for an instance of A (NoMethodError) 35 -
Solved with UnboundMethod class A < BasicObject end obj =
A.new unbound_class = Object.instance_method(:class) p(class: unbound_class.bind_call(obj)) # {class: A} 36 -
Issues • ✅ NoMethodError occurs in the class method •
The class name becomes ActiveRecord:"Relation • Does not support void type • Does not work in parallel testing 37 -
The name method is often overridden users = User.all users.class.name
#"> "ActiveRecord:$Relation" users.class.to_s #"> "User:$ActiveRecord_Relation" 38 -
The name method is often overridden4 class X end X.method(:name)
#"> #<Method: #<Class:X>(Module)#name()> users = User.all users.class.method(:name) #"> #<Method: #<Class:User:$ActiveRecord_Relation> # (ActiveRecord:$Delegation:$ClassSpecificRelation:$ClassMethods)#name() 4 https://github.com/rails/rails/blob/v8.0.2/activerecord/lib/active_record/relation/delegation.rb#L112- L114 39 -
Also resolved by UnboundMethod users = User.all unbound_name = Class.instance_method(:name)
unbound_name.bind_call(users.class) #"> "User:$ActiveRecord_Relation" 40 -
How to get the correct class name UNBOUND_CLASS = Object.instance_method(:class)
UNBOUND_NAME = Class.instance_method(:name) TracePoint.trace(:return) do |tp| klass = UNBOUND_CLASS.bind_call(tp.return_value) class_name = UNBOUND_NAME.bind_call(klass) end 41 -
Issues • ✅ NoMethodError occurs in the class method •
✅ The class name becomes ActiveRecord:"Relation • Does not support void type • Does not work in parallel testing 42 -
What is void type? class UsersController < ApplicationController def new
user = User.new user.set_password render :ok end end class User # @rbs () -" String def set_password @password = SecureRandom.hex(20) end end 43 -
What is void type? class UsersController < ApplicationController def new
user = User.new user.set_password render :ok end end class User # @rbs () -" void def set_password @password = SecureRandom.hex(20) end end 44 -
Use caller_locations class A def foo bar end def bar
nil end end TracePoint.trace(:call) do |tp| p(method_id: tp.method_id, caller_locations:) end A.new.foo # {method_id: :foo, caller_locations: ["example.rb:3:in 'A#foo'", # "example.rb:13:in '<main>'"]} # {method_id: :bar, caller_locations: ["example.rb:6:in 'A#bar'", # "example.rb:3:in 'A#foo'", # "example.rb:13:in '<main>'"]} 45 -
How to determine void type The conditions for void type
are as follows. 1. Method call is directly under the def keyword 2. But exclude the last method call def example foo bar buz # the return value is used end 47 -
Issues • ✅ NoMethodError occurs in the class method •
✅ The class name becomes ActiveRecord:"Relation • ✅ Does not support void type • Does not work in parallel testing 56 -
Configure for parallel testing RSpec.configure do |config| trace = RBS:#Trace.new
config.before(:suite) { trace.enable } config.after(:suite) do trace.disable trace.save_files(out_dir: "tmp/sig-#&ENV.fetch('TEST_ENV_NUMBER', nil)}") end end 61 -
Execute the following commands # Run parallel testings $ bin/flatware
rspec -r ./spec/flatware_helper.rb # Merge RBS files into one $ bundle exec rbs-trace merge -$sig-dir='tmp/sig-*' > tmp/sig/merged.rbs # Insert RBS comments from the merged file $ bundle exec rbs-trace inline -$sig-dir=tmp/sig -$rb-dir=app -$rb-dir=lib 62 -
! RBS files can be generated by CI 1. Add
rbs-trace to Gemfile 2. Configure spec_helper.rb 3. Save RBS files on CI6 4. Download RBS files from CI 5. Use merge and inline commands locally 6 https://github.com/actions/upload-artifact ͳͲ 63 -
Issues • ✅ NoMethodError occurs in the class method •
✅ The class name becomes ActiveRecord:"Relation • ✅ Does not support void type • ✅ Does not work in parallel testing 64 -
Future work 1. Support more types • Block arguments, Generics
types, Interface types • ...etc 2. Update RBS comments each time tests are run 3. Save RBS files for gems not in gem_rbs_collection 66 -
Conclusion • It generally works and is practical • TracePoint,
Prism, caller_locations are very useful • Read the code and the generated type declarations Let's generate type declarations! 67