Share
Embed
Walking around the ruby forest more deeply
my presentation of RubyKaigi2014. for beginners, introducing about how to start to read Ruby implementation source code.
やきとりい
September 20, 2014
Transcript
39 slides
-
YUKI TORII(SASADA) 2014.10.19 RUBYKAIGI WALK AROUND THE RUBY FOREST MORE
DEEPLY. THE GUIDE TO KNOW RUBY IMPLEMENTATION FOR NON-C LANGUAGE PROGRAMERS ϧϏΟϊεεϝ -
WHO AM I? • ௗҪʢాʣઇ / YUKI TORI • Rails
application developer in Japan (company: ສ༿ʣ • RailsGirls in JP organiser • Learned Programming by Rails -
FIRST • This talk is for non C programmers/ beginners
like me. • goal: After go back home tonight, you glance at Ruby’s source code. ͜ͷτʔΫॳ৺ऀ͚Ͱ͢ ʢࢲΈ͍ͨͳʣ -
FIRST • But for beginners, Ruby’s implementation is like a
deep, dark forest… • You must get Maps and Compass. -
PUT ON YOUR BOOTS: • Get a set of Ruby
source • Put a first book in C-language beside of you (ask your friends or Google about good book.) > git clone https://github.com/ruby/ruby ϧϏʔͷιʔείʔυΛखʹೖΕΔ -
MAP1: RUBY UNDER A MICROSCOPE • author: Pat Shaughnessy •
ISBN: 9781593275273 • an Illustrated Guide to Ruby Internals. -
RUBY UNDER A MICROSCOPE WILL GIVE YOU… • a
large scale Map about 2.0 Ruby Implementation. • Parsing, and Compilation of Ruby code • Structures of Ruby Basic objects • and etc… -
MAP1: PARSE-COMPILE Class MyCode deff my … end ! …
end Your Ruby Code Perse Compile AST ByteCode Code-> நߏจ -> ByteCode -
MAP2:STRUCTURE OF OBJECTS String info… Array info… VALUE RSring flags
klass RBasic VALUE RArray flags klass RBasic integer value Flags RObject built-in Object FIXNUM_FLAG : 1 simple ruby value RObject numiv ivptr VALUE flags klass RBasic and RRegexp etc… built-in objects and Symbol ect… Simple values -
JAPANESE VER. IS COMING SOON! • ʰRubyͷ͘͠Έ Ruby Under
a Microscopeʱ • translators: Koji Shimada Shintaro Kakutani • ൃߦɿΦʔϜࣾ։ൃ෦ • ץߦ༧ఆɿ2014ळ • I joins as one of reviewers. * ΖΖௐதͷͨΊ·ͩΦʔϜࣾʹ͓͍߹Θͤ͠ͳ͍Ͱ͍ͩ͘͞ -
MAP2: RUBY HACKING GUIDE(RHG) • author: Minero Aoki •
Original is written in Japanese, and is translated into English. All contents are available at http://ruby-hacking- guide.github.io/ • Talking about very details on Ruby source code and implementation. • about Ruby ver. 1.7.3… old, but Very Very Useful!! -
WHY RHG IS USEFUL? BECAUSE IT’S VERY UNIVERSAL. •
“How to read source codes” • making a method simple by cutting not essential code off. • understanding object structure before. • and many other wisdoms • It was written for C-programmers. But non C-programmer can learn Basic C essence for reading Ruby Implementation. essence -
WHY RHG IS USEFUL? BECAUSE IT’S SOMEHOW OLD. •
Ruby is improving. • While reading Ruby source code with RHG, you will find a difference between the description and the current code. => This is a chance to know Why the source has been changed. It’s a trace of improvement of ruby. You can trace it by ChangeLog and <git log> with its commit message. -
MAP3: SOURCE FILE MAP BSDL COPYING COPYING.ja ChangeLog GPL KNOWNBUGS.rb
LEGAL Makefile.in NEWS README.EXT README.EXT.ja README.ja.md README.md addr2line.c addr2line.h array.c benchmark bignum.c bin boots traptest class.c common.mk compar.c compile.c complex.c configure.in constant.h cont.c cygwin debug.c defs dir.c dln.c dln.h dln_find.c dmydln.c dmyext.c doc enc encoding.c enum.c en umerator.c error.c eval.c eval_error.c eval_intern.h eval_jump.c ext file.c gc.c gc.h gem_prelude.rb golf_prelude.rb goruby.c hash.c ia64.s include inits.c insns.def internal.h io.c iseq.c iseq.h lex.c.blt lib load.c loadpath.c localeinit.c main.c man marshal.c math.c method.h miniinit.c misc missing nacl node.c node.h numeric.c object.c pack.c parse.y prelude.rb probes.d probes_helper.h proc.c process.c random.c range.c rational.c re.c regcomp.c regenc.c regenc.h regerror.c regexec.c regint.h regparse.c regparse.h regsyntax.c ruby.c ruby_atomic.h safe.c sample signal.c siphash.c siphash.h sparc.c spec sprintf.c st.c strftime.c string.c struct.c symbian template test thread.c thread_native.h thread_pthread.c thread_pth read.h thread_win32.c thread_win32.h time.c timev.h tool transcode.c transcode_data.h util.c variable.c version.c version.h vm.c vm_backtrace.c vm_core.h vm_debug.h vm_dump.c vm_eval .c vm_exec.c vm_exec.h vm_insnhelper.c vm_insnhelper.h vm_method.c vm_opts.h vm_trace.c vsnprintf.c win32 > cd ./ruby; ls ઈ….(desperate..) -
MAP3: SOURCE FILE MAP BSDL COPYING COPYING.ja ChangeLog GPL KNOWNBUGS.rb
LEGAL Makefile.in NEWS README.EXT README.EXT.ja README.ja.md README.md addr2line.c addr2line.h array.c benchmark bignum.c bin boots traptest class.c common.mk compar.c compile.c complex.c configure.in constant.h cont.c cygwin debug.c defs dir.c dln.c dln.h dln_find.c dmydln.c dmyext.c doc enc encoding.c enum.c en umerator.c error.c eval.c eval_error.c eval_intern.h eval_jump.c ext file.c gc.c gc.h gem_prelude.rb golf_prelude.rb goruby.c hash.c ia64.s include inits.c insns.def internal.h io.c iseq.c iseq.h lex.c.blt lib load.c loadpath.c localeinit.c main.c man marshal.c math.c method.h miniinit.c misc missing nacl node.c node.h numeric.c object.c pack.c parse.y prelude.rb probes.d probes_helper.h proc.c process.c random.c range.c rational.c re.c regcomp.c regenc.c regenc.h regerror.c regexec.c regint.h regparse.c regparse.h regsyntax.c ruby.c ruby_atomic.h safe.c sample signal.c siphash.c siphash.h sparc.c spec sprintf.c st.c strftime.c string.c struct.c symbian template test thread.c thread_native.h thread_pthread.c thread_pth read.h thread_win32.c thread_win32.h time.c timev.h tool transcode.c transcode_data.h util.c variable.c version.c version.h vm.c vm_backtrace.c vm_core.h vm_debug.h vm_dump.c vm_eval .c vm_exec.c vm_exec.h vm_insnhelper.c vm_insnhelper.h vm_method.c vm_opts.h vm_trace.c vsnprintf.c win32 Don't give up your hope. File names are easy to understand. and README.EXT has “Appendix A. Ruby Source Files Overview” -
MAP3: SOURCE FILE MAP ”Ruby Source Files Overview” in README.EXT
groups source files by their contents. Ruby Language Core class.c :: classes and modules error.c :: exception classes and exception mechanism gc.c :: memory management load.c :: library loading object.c :: objects variable.c :: variables and constants Class Library array.c :: Array bignum.c :: Bignum compar.c :: Comparable complex.c :: Complex cont.c :: Fiber, Continuation dir.c :: Dir enum.c :: Enumerable enumerator.c :: Enumerator file.c :: File hash.c :: Hash io.c :: IO marshal.c :: Marshal math.c :: Math numeric.c :: Numeric, Integer, Fixnum, Float pack.c :: Array#pack, String#unpack proc.c :: Binding, Proc process.c :: Process random.c :: random number range.c :: Range rational.c :: Rational re.c :: Regexp, MatchData signal.c :: Signal sprintf.c :: String#sprintf string.c :: String struct.c :: Struct time.c :: Time Ruby Syntax Parser parse.y :: grammar definition parse.c :: automatically generated from parse.y keywords :: reserved keywords lex.c :: automatically generated from keywords Ruby Evaluator (a.k.a. YARV), Regular Expression Engine (Oniguruma), Utility Functions, Ruby Interpreter Implementation, Multilingualization, goruby Interpreter Implementation and other groups -
MAP3: SOURCE FILE MAP File names are easy to understand.
see the class libraries. Ruby Language Core class.c error.c gc.c load.c object.c variable.c Ruby Syntax Parser parse.y :: grammar definition parse.c :: automatically generated from parse.y keywords :: reserved keywords lex.c :: automatically generated from keywords Ruby Evaluator (a.k.a. YARV) Regular Expression Engine (Oniguruma) Utility Functions Ruby Interpreter Implementation Multilingualization goruby Interpreter Implementation and other groups Class Library array.c :: Array bignum.c :: Bignum compar.c :: Comparable complex.c :: Complex cont.c :: Fiber, Continuation dir.c :: Dir enum.c :: Enumerable enumerator.c :: Enumerator file.c :: File hash.c :: Hash io.c :: IO -
MAP3: SOURCE FILE MAP extension also give us hints. xx.c
=> Of cause c code xx.h => header file. Many definitions such as macros, data-structures. xx.y => only parse.y (grammar definition) xx.rb => Of cause ruby file. xx.def => miscellaneous definitions -
MAP3: SOURCE FILE MAP /include/ … many ruby’s core header
files. /defs/ … definition files such as keywords, errors… /ext/ … c extensions such as json, socket… /lib/ …ruby libraries such as csv, irb, yaml, rake… /missing/ …implements of C functions need for ruby, but not all Systems have. /test/ … test codes for ruby implementations. /spec/ … RubySpec will be downloaded here. Special directories. -
COMPASS:METHOD DEFINITION Because some file names correspond to class names,
it is easy to find where a method is defined. What Hash#to_a does…? 'PSFYBNQMF JGPOFEBZ*TUBSUUPXPOEFSJOH BOEDBOOPUTMFFQXFMM -
COMPASS:METHOD DEFINITION Because some file names correspond to class name,
it is easy to find where a method is defined. What Hash#to_a does…? 'PSFYBNQMF JGPOFEBZ*TUBSUUPXPOEFSJOH BOEDBOOPUTMFFQXFMM See “hash.c”, Of course. and GoodNight. -
COMPASS:METHOD DEFINITION Easy to find the definition of Hash#to_a in
hash.c Because… ! WPJE *OJU@)BTI WPJE \ TOJQ ! SC@EFpOF@NFUIPE SC@D)BTI JOJUJBMJ[F SC@IBTI@JOJUJBMJ[F SC@EFpOF@NFUIPE SC@D)BTI JOJUJBMJ[F@DPQZ SC@IBTI@JOJUJBMJ[F@DPQZ SC@EFpOF@NFUIPE SC@D)BTI SFIBTI SC@IBTI@SFIBTI ! SC@EFpOF@NFUIPE SC@D)BTI UP@IBTI SC@IBTI@UP@IBTI SC@EFpOF@NFUIPE SC@D)BTI UP@I SC@IBTI@UP@I SC@EFpOF@NFUIPE SC@D)BTI UP@B SC@IBTI@UP@B SC@EFpOF@NFUIPE SC@D)BTI JOTQFDU SC@IBTI@JOTQFDU SC@EFpOF@BMJBT SC@D)BTI UP@T lJOTQFDU ! TOJQNBOZPUIFSNFUIPEEFpOJUJPOT built-in class has Init_xx function at bottom of its file. }In the Init_xx, almost all methods are defined by “rb_define_method” -
COMPASS:METHOD DEFINITION rb_define_method takes understandable args. TUBUJD7"-6& SC@IBTI@UP@B 7"-6&IBTI
\ 7"-6&BSZ ! BSZSC@BSZ@OFX@DBQB 3)"4)@4*;& IBTI SC@IBTI@GPSFBDI IBTI UP@B@J BSZ 0#+@*/'&$5 BSZ IBTI ! SFUVSOBSZ ^ SC@EFpOF@NFUIPE SC@D)BTI UP@B SC@IBTI@UP@B klass method_name function init arg function name is put on the head of line to search easily. make new array with hash size convert hash’s each pair to array Macro often defined in ***.h. but I can imagine how work from name… -
NOW, YOU GOT A READY. WOULD YOU LIKE TO WALK
IN RUBY INTERNAL A LITTLE? -
HASH TABLE IS USED EVERYWHERE. • Hash Class • Method
Table • Instance Variables • Constant • Symbol HJUHSFQTU@JOJUcHSFQWEPDcHSFQWUFTU -
WHAT IS A HASH TABLE. The data-structure for keeping key/value
pair. obj of Hash class Method Table Key Value :favorite_animal “rabbit" “scores” [10,3,4] :neighborhood #<Person> Key Value “name” def name … “age” def age… “love?” def love?(me)…. -
OPEN HASH TABLE STRUCTURE bins entries ɾ ! ! ɾ
“entry” under corresponding with hash-value in bins. -
RUBY HASH TABLE STRUCTURE bins entries st_table type num_bins bins
(big) num_entries st_table_entry hash key record next fore back 1 2 3 4 next next fore back (hash is ordered from Ruby 1.9~) 5 6 7 entries_packed (0) back -
RUBY HASH TABLE STRUCTURE entries st_packed_entry hash key value 1
2 3 4 5 st_table type num_bins packed num_entries entries_packed (1) real_entries -
WHEN ENTRIES ARE TOO MANY. Packed -> bins 1 2
3 4 5 6 key hash val -
YOU CAN FIND ALL OF THEM IN • st_table in
“include/ruby/st.h” • st_table_entry /st_packed_entry • add_packed_direct • unpacked • add_direct • rehash in “st.c” -
LAST:DRAW YOUR OWN MAY BY YOUR HAND • The map
of two books are a part of the rich Ruby forest. • You can find your favorite path, or nice views by your self. ࣗͷਤΛ࡞ͬͯΈΑ͏