Perl5の静的解析入門 / The static analysis of Perl5

Perl5の静的解析入門
機械と人間双方の歩み寄りによる平和編

YAPC::Tokyo 2019 Room1 14:00 -

Transcript

1. Perl5ͷ੩తղੳೖ໳ ػցͱਓؒ૒ํͷาΈدΓʹΑΔฏ࿨ฤ YAPC::Tokyo 2019 ໘ന๏ਓΧϠοΫ macopy a.k.a @mackee_w #yapcjapan #yapcjapanRoom1 1
2. ͜ͷεϥΠυ 2
3. ͜ͷτʔΫ ͷ಺༰ 3
4. ಘΒΕΔ஌ࣝɾମݧ (1) •ਖ਼نදݱͰPerlίʔυΛύʔε͢ΔPPR.pm ͷ࢖͍ํ • ۀ຿Ͱͷ۩ମతͳ࢖༻ྫ •ਓؒͱػցʹ෼͔Γʹ͍͘ίʔυͱ௚͠ํ ͷఏҊ 4
5. ಘΒΕΔ஌ࣝɾମݧ (2) •Perlίʔυͷؔ਺ʹܕγάωνϟΛ෇͚Δ •ؔ਺ͷܕγάωνϟΛPPRͰൈ͖ग़͢ 5
6. ࿩ͷྲྀΕ 1. ੩తղੳͱ͸ 2. PPIΛ࢖͏ྫ 3. PPRͷઆ໌ͱྫ 4. ػցͱฏ࿨ͷ࿩ 5. ੩తܕ෇͚ͷೖΓޱ 6
7. ࢲͱ͸ɾɾɾʁ • ໘ന๏ਓΧϠοΫ ιʔγϟ ϧήʔϜࣄۀ෦ • ࢓ࣄݴޠ͸Perl5ͱGo • झຯ͸3DϓϦϯλ࡞Γ • աڈͷτʔΫ: ήʔϜӡ༻ͷ ࿩ͱ͔Ϛελσʔλͷ࿩ͱ͔ WebSocket 7
8. ΠϯλʔωοτͰ͸͜͏͍͏ΞΠ ίϯͰ͢ 8
9. [Ad]͜ͷτʔΫ͸ۀ຿࣌ؒʹॻ͖·ͨ͠ 9
10. ੩తղੳͱ͸Ұ ମɾɾɾʁ 10
11. static analysis ੩తղੳ 11
12. ίʔυΛͨͩͷ จࣈྻͱͯ͠ѻ͏ 12
13. ΍Γํ 1 ਖ਼نදݱͰҾֻ͚ͬͨΓී௨ͷςΩετͱͯ͠ѻ͏ use Path::Tiny; my $target_package = "Example"; my $script = path("$target_package.pm")->slurp; my $is_valid_package = $script =~ /\Apackage $target_package;/; 13
14. ͜͏͍͏ͷ͕ग़ͯ͘Δͱ؆୯ʹ٧Ή use Example; 14
15. ΑΓ࣮֬ͳ΍Γํ͸ͳ͍ͷ͔ʂ 15
16. ΍Γํ 2 perl͕΍Δ͜ͱΛ్த·Ͱ΍ͬͯɺந৅ߏจ໦Λར༻͢Δ 16
17. ίϥϜ: ࠷ۙͷݴޠͩͱݴޠຊମͷASTΛར༻Ͱ͖Δ͜ͱ͕͋Δ •Goͷ go/parser go/token go/types go/ast • GoίϯύΠϥ΋͜ΕΒΛ࢖͍ͬͯΔ • ͦ͏Ͱ͸ͳ͍Β͍͠ •Rubyͷ RubyVM::AST • 2.6.0͔Βར༻Մೳ 17
18. ੩తղੳͷϝϦοτ (1) •ίʔυͷݟͨ··Λѻ͑Δ • ந৅ߏจ໦ҎલͰ͋Ε͹ίϝϯτͷதΛ ѻ͑Δ͜ͱ΋ •ݴޠػೳΛ௒͑ͨදݱྗ • ޙ͔Β੩తܕ෇͚ͱ͔ 18
19. ࣮ߦग़དྷΔঢ়ଶʹͳΔʹͭΕͯιʔείʔυ্ͷ৘ใ͸ൈ͚མ͍ͪͯ͘ 19
20. ࣮ߦ࣌ղੳͰ͸ར༻Ͱ͖ͳ͍৘ใ •ۭന, վߦ, ίϝϯτ • ಛघͳॻ͖ํͰར༻Ͱ͖Δ৔߹΋͋Δ (ྫ: Pythonͷdocstring) 20
21. ࣮ߦ࣌ղੳͰ͸ར༻Ͱ͖ͳ͍৘ใ •τʔΫϯͷҐஔ΍લޙͷτʔΫϯ • ར༻͞Ε͍ͯΔม਺͕࣮ࡍʹ͸Ͳ͜Ͱએ ݴ͞Ε͍ͯΔ͔ͱ͔ • ؔ਺΍จͷҐஔͳͲ͸ελοΫτϨʔε Ͱར༻͢ΔͨΊʹ෇͍͍ͯΔ 21
22. ੩తղੳͷϝϦοτ (2) •࣮ߦͤͣʹίʔυΛѻ͑Δ • BEGINͰϠόΠ͜ͱ͍ͯͯ͠΋େৎ෉ • ٯʹݴ͏ͱBEGINͰେࣄͳ͜ͱ͍ͯͯ͠ ΋ݕ஌͸೉͍͠(ޙड़) 22
23. ੩తղੳͷϝϦοτ (3) •͢΂ͯͷ࣮ߦύεʹ͍ͭͯௐ΂্͛ΒΕΔ ৔߹͕͋Δ • ࣮ߦ࣌ʹܕόϦσʔγϣϯ͢Δ৔߹ɺ࣮ ߦ͢Δ·Ͱ੒ޭ͢Δ͔Ͳ͏͔෼͔Βͳ͍ 23
24. ੩తղੳͷσϝϦοτ •PerlͷύʔεΛࣗલ(Ϟδϡʔϧ࢖͏͕)Ͱ΍ Βͳ͍ͱ͍͚ͳ͍͔Βେม •ͦ΋ͦ΋໦ߏ଄ΛಡΜͰ͍ͬͯͲ͏ʹ͔͢ Δͷେม •perlͷؾ࣋ͪʹͳΒͳ͍ͱ͍͚ͳͯ͘େม 24
25. ࢲ͕੩తղੳ͍ͨ͠ಈػ ػցͷྗΛआΓͯ ؒҧ͍ͷͳ͍ίʔυΛ ॻ͖͍ͨʂ 25
26. Perl5Ͱ੩తղੳ ͢ΔͨΊͷπʔϧ 26
27. Perl5੩తղੳքͷୈҰਓऀ PPI 27
28. PPI Parse, Analyze and Manipulate Perl (without perl) •Pure Perl͚ͩͰPerlίʔυΛύʔεͯ͠੩ తղੳͰར༻Ͱ͖Δܗʹ͢ΔϞδϡʔϧ 28
29. ࢖༻ྫ (1) use PPI; use PPI::Dumper; my $document = PPI::Document->new(\'my $v1 = $v2;'); PPI::Dumper->new($document)->print; 29
30. ࢖༻ྫ (2) PPI::Document PPI::Statement::Variable PPI::Token::Word 'my' PPI::Token::Whitespace ' ' PPI::Token::Symbol '$v1' PPI::Token::Whitespace ' ' PPI::Token::Operator '=' PPI::Token::Whitespace ' ' PPI::Token::Symbol '$v2' PPI::Token::Structure ';' 30
31. ࢖༻ྫ (3) my ($stmt) = $document->children; my $op = $stmt->find_first("PPI::Token::Operator"); $op->content; # => "=" 31
32. PPIͷಛ௃ (1) •PDOM(ύʔεޙͷ໦ߏ଄ͷ͜ͱ)͔Βίʔυ ʹॻ͖໭͢͜ͱ͕ग़དྷΔ $document->prune("PPI::Token::Whitespace"); $document->serialize; # => "my$v1=$v2;" 32
33. PPIͷಛ௃ (2) •ॻ͖໭ͨ͢ΊʹτʔΫϯ΍εϖʔεɺ;ͳͲ ͸อ࣋ͨ͠·· •͋͘·Ͱղੳ͢ΔͨΊͷߏ଄Ͱ͋ͬͯɺ࣮ ߦ͢ΔͨΊͷߏ଄Ͱ͸ͳ͍ 33
34. PPIͷσϝϦοτ •݁ߏ஗͍ • 1175ߦͷ࣮ࡍͷۀ຿ͷίʔυΛύʔε͠ ͨ݁Ռ • 6.80/s1 1 Macbook Pro 15inch mid 2015 Ͱܭଌ 34
35. ͦͷଞͷϞδϡʔϧ •Compiler::Lexer ಠࣗͷCݴޠͰॻ͔Εͨ τʔΧφΠβΛ࢖༻ •Perl::Lexer perlຊମͷ಺෦APIΛୟ͍ͯτʔ ΫϯྻΛऔΓग़͢ 35
36. PPRͷ঺հͱ࢖ ͍ํ 36
37. PPR Pattern-based Perl Recognizer •PerlίʔυͷύλʔϯΛఆٛͯ͠ݕ஌Ͱ͖ Δਖ਼نදݱΛ࡞ΕΔϞδϡʔϧ 37
38. ࢖༻ྫ # my $hoge = "fuga"; ͷม਺໊ΛҾֻ͚ͬΔʹ͸ my $matcher = qr{ my (?&PerlOWS) ((?&PerlVariableScalar)) (?&PerlOWS) (?&PerlAssignmentOperator) (?&PerlOWS) (?&PerlString) (?&PerlOWS) ; $PPR::GRAMMAR }x 38
39. ࢖༻ྫ my ($var) = grep { defined } $script =~ $matcher; ͜ΕͰ $var ʹ "$hoge" ͕ೖΔ 39
40. ศརʂʂʂ 40
41. ʮΘʔʂ͍͖ͳΓ ϠόΠਖ਼نදݱ͕ ग़͖ͯͨͧʂʂʯ 41
42. ଴͍ͬͯͩ͘͞ ղઆ͠·͢ 42
43. PPR͸PerlίʔυͷτʔΫϯʹϚον͢Δਖ਼نදݱͷύλʔϯू Perlίʔυྫ PPRͷύλʔϯ my (?&PerlBuiltinFunction) $var (?&PerlVariableScalar) = (?&PerlAssignmentOperator) "hogehoge" (?&PerlString) 43
44. සग़ύλʔϯ •(?&PerlOWS) • θϩจࣈҎ্ͷPerl্ͷۭനจࣈ • ۭന΍վߦΛؚΉ 44
45. ࣮༻ྫ 45
46. ฐࣾͩͱ͜͏͍͏όϦσʔγϣϯΛΑ͘΍͍ͬͯΔ͚ΕͲ sub do_something { my $self = shift; state $rule = Data::Validator->new( a1 => "Str", b1 => "UInt", ); my $args = $rule->validate(@_); my ($a1, $b1) = $args->{qw/a1 b1/}; # do something... } 46
47. ϓϥΠϕʔτϝιου ͡Όͳ͔ͬͨΒ ͪΌΜͱόϦσʔγϣϯ ͯ͠ΔΑͶνΣοΫ 47
48. εΫϦϓτ͔Βؔ਺ఆٛΛൈ͖ग़͢ my $script = ...; my $sub_matcher = qr{ ((?&PerlSubroutineDeclaration)) $PPR::GRAMMAR }x; my @decls = grep { defined } $script =~ m{$sub_matcher}gx; 48
49. ؔ਺ఆ͔ٛΒؔ਺໊ͱϒϩοΫΛൈ͖ग़͢ my $subname_matcher = qr{ \Asub (?&PerlOWS) ((?&PerlQualifiedIdentifier)) (?&PerlOWS) ((?&PerlBlock)) $PPR::GRAMMAR }x; for my $decl (@decls) { my ($subname, $block) = grep { defined } $decl =~ $subname_matcher; 49
50. ؔ਺໊͕ΞϯμʔείΞ࢝·ΓͳΒεΩοϓ͢Δ for my $decl (@decls) { my ($subname, $block) = grep { defined } $decl =~ $subname_matcher; next if $subname =~ /\A_/; 50
51. validatorͷఆٛ my $validator_matcher = qr{ ( (state) (?&PerlOWS) (?&PerlVariableScalar) (?&PerlOWS) (?&PerlAssignmentOperator) (?&PerlOWS) Data::Validator->new (?&PerlOWS) (?&PerlParenthesesList) (?&PerlOWS) ) $PPR::GRAMMAR }x; 51
52. validatorͷఆ͕ٛ͋Δ͔Λௐ΂Δ for my $decl (@decls) { ... next if $subname =~ /\A_/; my ($validator) = grep { defined } $block =~ $validator_matcher; 52
53. ͋ͱ͸ࣽΔͳΓম͘ͳΓ # warningΛग़͢ͳΓ if (!defined $validator) { warn "$subname Ͱ Data::Validator ͕࢖ΘΕͯͳ͍Αʂ"; } # ςετͰ͚ͤ͜͞ΔͳΓ use Test::More; fail "$subname Ͱ Data::Validator ͕࢖ΘΕͯͳ͍Αʂ"; 53
54. ͜ͷνΣοΫͰൈ͚͍ͯΔͱ͜Ζ •Ҿ਺ΛऔΒͳ͍ؔ਺Λޡݕ஌ •Attribute΍Subroutine signature •Ҿ਺όϦσʔγϣϯҎ֎ͷvalidator 54
55. ׬ᘳΛ໨ࢦͣ͞ʹࣗ෼ͷ࣋ͭίʔυ͚ͩͰ௨Ε͹ྑ͍ ͋Δఔ౓͸αϘΔ 55
56. ͞ΒʹԠ༻Ҋ •Smart::Args Λ࢖͏Α͏ʹஔ׵͢Δ • όϦσʔλͷதͷݸʑͷఆٛ·Ͱόϥ͠ ͯൈ͖ग़ͤΔ •LSPͳͲͰҾ਺ϦετΛग़͢ 56
57. PPR͸ਖ਼نදݱͰίʔυ ͷҰ෦Λൈ͖ग़ͯ͠ ݕࠪ΍ஔ׵ʹ࢖͑Δ 57
58. ͦ͏͍͏໨తͷطଘͷϞδϡʔϧ •Perl::Critic • PPIΛ࢖ͬͨνΣοΧʔ •Perl::Lint • Compiler::LexerΛ࢖ͬͨνΣοΧʔ 58
59. ੩తղੳͰ ίʔυͷ࣏҆Λ Α͍ͯ͘͘͠ʹ͸ʁ 59
60. ػցͱਓؒʹྑ͍ ίʔυͱ ཆ੒Ϊϓεͱͯ͠ ͷ੩తղੳ 60
61. perl͸ೲಘ͕Ͱ͖Δڍಈ΋͢Δ͠ ڻ͖MAXͷڍಈΛى͜͢͜ͱ΋͋Δ 61
62. Only perl can parse Perl (1) •Only perl can parse Perl ͳ෦෼ •PPIͷυΩϡϝϯτΑΓ @result = (dothis $foo, $bar); # Which of the following is it equivalent to? @result = (dothis($foo), $bar); @result = dothis($foo, $bar); 62
63. Only perl can parse Perl (2) $ perl -MO=Deparse -e '@result = (dothis $foo, $bar);' @result = ($foo->dothis, $bar); -e syntax OK $ perl -MO=Deparse -e 'sub dothis {} @result = (dothis $foo, $bar);' sub dothis { } @result = dothis($foo, $bar); -e syntax OK 63
64. ΍Γ͍ͨํʹ໌ࣔ͢Δʂʂʂ @result = dothis($foo, $bar); •͍ΖΜͳॻ͖ํ͕Ͱ͖Δݴޠ͔ͩΒͦ͜ɺ ΍Γ͍ͨ͜ͱΛओு͢Δ 64
65. Only perl can parse Perl (3) sub hoge { "hoge" } sub f { { hoge() => "fuga" } } # hashref or list ? my @result = f(); 65
66. Only perl can parse Perl (4) use DDP; p @result; # [ # [0] "hoge", # [1] "fuga" # ] ϦετʹͳΔͷ͕ਖ਼ղ 66
67. Only perl can parse Perl (5) hashrefʹ͢Δʹ͸ʁ •return Λ෇͚Δ •ϑΝοτΧϯϚͷࠨล஋Λจࣈྻʹ͢Δ •+{} ʹ͢Δ 67
68. ͭ·ΓͪΌΜͱreturnॻ͚͹ղܾ͢Δ sub hoge { "hoge" } sub f { return { hoge() => "fuga" }; } # hashref desu!!! my @result = f(); 68
69. Only perl can parse Perl (6) •1ͭͷه߸͕ෳ਺ͷҙຯʹ࢖ΘΕ͍ͯͯ೉ ͍͠ • είʔϓͱϋογϡϦϑΝϨϯεͷ {} • আࢉͱਖ਼نදݱͷ / 69
70. ʮ͜Ε͔ͩΒPerl͸ʂʯ 70
71. ! ͚ΕͲ ίʔυͷ໎͍͸ࢥߟͷ໎͍ Ұݺٵ͓͍ͯɺ΋͏গ͠ߟ͑Α͏ 71
72. άϨούϏϦςΟ greppability 72
73. greppability •ૂͬͨίʔυΛgrepͰݕ஌Ͱ͖Δ͔Ͳ͏͔ • ΄͔ͷఆ͕ٛ͋ͬͨΒڭ͍͑ͯͩ͘͞ $ hw 'Greeter->say_hello' lib | \ xargs perl -pie 's/Greeter->say_hello/Greeter->send_hello' 73
74. greppability͕௿͍ίʔυ •ϝιου໊Λಈతʹ૊Έཱ͍ͯͯΔ my @params = qr/str vit dex int/; my $total = sum( map { my $method = $_ . "_factor"; $person->$method; } @params ); 74
75. ϝιου໊Λಈతʹ૊Έཱͯͨͱ͖ͷฐ֐ •ʮ͜ͷϝιουͲ͜Ͱ࢖͍ͬͯΔʁʯ͕೉ ͘͠ͳΔ •ϝιουݺͼग़͠Λ੩తղੳͰநग़͢Δͱ ͖ʹ൑ఆͰ͖ͳ͘ͳΔ ͱʹ͔͘ػցʹ༏͘͠ͳ͍ʂʂʂ 75
76. ͤΊͯจࣈྻ࿈݁Λ΍ΊΔ my @methods = qr/ str_factor vit_factor dex_factor int_factor /; my $total = sum(map { $person->$_ } @methods); •ϝιου໊͕ϢχʔΫͰ͋Ε͹grepʹҾͬ ͔͔ͬͯ͘Δ 76
77. ಈతܕ෇͚ݴޠͰ΋৺ʹ੩తܕ෇͚ίϯύΠϥΛ࣋ͭ •ίʔυΛॻ͍ͯܕ͕੩తʹܾఆ͢Δ͔Λߟ ͑Δ •Πϛϡʔλϒϧม਺ͳͲϞμϯݴޠʹ͋Δ ػೳ΍੍ݶΛಋೖ͢Δͱྑ͍ 77
78. ͦͷଞ੩తղੳͰࠔΔ΋ͷ •࣮ߦ࣌ʹuse͢Δ΋ͷ͕ܾ·Δͱ͔ BEGIN ಈ͔͞ͳ͍ͱ֬ఆ͠ͳ͍ͱ͔ •Mo+([uo]se)? ͷ has ΍ method ͸ͦΕΛղऍ ͢Δ΋ͷΛ࡞Ε͹·ͩܕͷ੩తղܾ͕Ͱ͖ Δ৔߹͕͋Δ 78
79. ػցʹ༏͚͠Ε͹ಡΉਓؒʹ΋͍͍ͩͨ༏͍͠ •ॻ͘ਓؒʹ͸ݫ͘͠ͳΔ͜ͱ͕͋Δ • ॻ͘ͷ͸Ұճ͔ͭҰਓɺಡΉͷ͸n(n>=1) ճ͔ͭmਓ •ػցʹ༏͚͠Ε͹ػցͷࢧԉΛड͚Δ͜ͱ ͕ग़དྷΔ 79
80. ػցʹ΋ਓؒʹ΋༏͍͠Perlίʔυ͸PerlͷαϒηοτʹͳΔ => ੩తղੳͰྑ͘ͳ͍ίʔυΛݕ஌Ͱ͖Δ 80
81. ਓؒʹ͸ཉ΍ࡶ೦͕͋ΔͨΊɺࣗݾΛ཯͢Δͷ͸೉͍͠ •ʮλΠϓ਺ଟ͘ͳΔ͔ΒखΛൈ͜͏ʯͬͯ ͳΔ͜ͱ͕͋Δ •ػցʹ஫ҙͯ͠΋Β͏ • returnෆ଍ݕ஌ͷϙϦγʔ͸ Perl::Critic ʹ͋Δ 81
82. ϋ΢εϧʔϧ •Rubyͷࣄྫ Querly •ʮ͜͏͍͏ॻ͖ํ͏ͪͩͱڻ͖͕͍ͬͺ͍ ͔ͩΒνΣοΫ͍ͨ͠ΑͶʯ •PerlͰ΋PPRͰݕ஌Մೳ 82
83. ͦͷଞϋ΢εϧʔϧͷྫ •Plack::RequestͰ $c->req->param(...)͸࢖Θ ͳ͍ •DBIx::Class Ͱ ResultSet->search Λ arrayί ϯςΩετͰड͚ͳ͍ ͳͲͳͲ 83
84. Ұ୴࣭໰λΠϜ 84
85. ͔͜͜Β͸͍࣌ؒͬͺ͍·Ͱ ૸Γൈ͚·͢ 85
86. PPRΛ࢖͍ͬͯͨ͘Ίʹ PPRΛ΋ͬͱਂ͘஌Ζ ͏ʂ 86
87. PPRͷυΩϡϝϯτʹ͸࢖͑Δਖ਼نදݱͷϧʔϧ͕ྻڍ͞Ε͍ͯΔ͕... 87
88. υΩϡϝϯτಡΜͰ΋ ۩ମతʹԿʹϚον ͢Δ͔Θ͔ΒΜͱ͜ΖΛ ίʔυͰಡΜͰΈΔ 88
89. $PPR::GRAMMAR ΛݟͯΈΔ my $stmt_matcher = qr{ \G (?&PerlOWS) ((?&PerlStatement)) $PPR::GRAMMAR # <= ίϨ }x; my @stmts = grep { defined } $script =~ m{$stmt_matcher}gcx; 89
91. ! 1761ߦ ໨ͱ೴ʹ௚઀μϝʔδ͕དྷΔߦͷਖ਼نදݱ 91
92. ਖ਼نදݱ͓͹͚ ! ʹཱͪ޲͔͏ 92
93. ಡΈํ (1) •(?<PerlDocument> ... ) ϧʔϧఆٛ •(?&PerlStatementSequence) ϧʔϧݺͼग़͠ 93
94. ಡΈํ (2) (?<PerlDocument> \x{FEFF}?+ # Optional BOM marker (?&PerlStatementSequence) ) # End of rule Ͷɺ؆୯Ͱ͠ΐʁ 94
95. ಡΈํ (3) ͱ͖Ͳ͖͜͏͍͏পʹग़͘Θ͚͢ΕͲ (?<PerlBuiltinFunction> # Optimized to match any Perl builtin name, without backtracking... (?=[^\W\d]) # Skip if possible (?> s(?>e(?>t(?>(?>(?>(?>hos|ne)t|gr)en|s(?>erven|ockop))t|p(?>r(?>iority|otoent)|went|grp)) | g(?>et(?>p(?>r(?>oto(?>byn(?>umber|ame)|ent)|iority)|w(?>ent|nam|uid)|eername|grp|pid)| | r(?>e(?>ad(?>lin[ek]|pipe|dir)?|(?>quir|vers|nam)e|winddir|turn|set|cv|do|f)|index|mdir | c(?>h(?>o(?>m?p|wn)|r(?>oot)?|dir|mod)|o(?>n(?>tinue|nect)|s)|lose(?>dir)?|aller|rypt) | e(?>nd(?>(?>hos|ne)t|p(?>roto|w)|serv|gr)ent|x(?>i(?>sts|t)|ec|p)|ach|val(?>bytes)?+|of | l(?>o(?>c(?>al(?>time)?|k)|g)|i(?>sten|nk)|(?>sta|as)t|c(?>first)?|ength) | u(?>n(?>(?>lin|pac)k|shift|def|tie)|c(?>first)?|mask|time) | p(?>r(?>ototype|intf?)|ack(?>age)?|o[ps]|ipe|ush) | d(?>bm(?>close|open)|e(?>fined|lete)|ump|ie|o) | f(?>or(?>m(?>line|at)|k)|ileno|cntl|c|lock) | t(?>i(?>mes?|ed?)|ell(?>dir)?|runcate) 95
96. Regexp::OptimizerͰੜ੒͞ΕͯΔΈ͍͔ͨͩ Βେৎ෉ʂ 96
97. ͍͍ײ͡ͷࢹ֮ԽΛ͢Δ •ఆ͕ٛผͷͲΕΛࢀর͍ͯ͠Δ͔ • (?&PerlVariable) ͸ • (?&PerlScalarAccess) ͱ • (?&PerlHashAccess) ͱ • (?&PerlArrayAccess) ͷ OR ͩΑ Έ͍ͨͳ 97
98. ͍͍ײ͡ͷՄࢹԽ (1) •PPRͷਖ਼نදݱͷจࣈྻΛൈ͖ग़͢ • ਖ਼نදݱϦϑΝϨϯεͩͱ͍Ζ͍Ζͬ͘ ͍ͭͯ͘ΔͷͰίʔυ্ͷςΩετͦͷ ··͕΄͍͠ • PPRͰൈ͖ग़͢ 98
99. our $GRAMMAR = qr{ ... }; # ίί ^^^^^^^^^ 99
100. ͍͍ײ͡ͷՄࢹԽ (2) my $stmt_regexp = qr{ our (?&PerlOWS) \$GRAMMAR (?&PerlOWS) = (?&PerlOWS) ((?&PerlRegex)) $PPR::GRAMMAR }x; my ($regexp) = grep { defined } $ppr_script =~ /$stmt_regexp/gcx; # => "qr{ ... }" 100
101. ͍͍ײ͡ͷՄࢹԽ (3) •PPIx::Regexp Ͱ ਖ਼نදݱͷந৅ߏจ໦ʹ͢ Δ use PPIx::Regexp; my $tree = PPIx::Regexp->new($regexp); 101
102. ͍͍ײ͡ͷՄࢹԽ (4) •͋ͱ͸࠶ؼͰτʔΫϯΛऩू͍ͯ͘͠ sub traverse { my ($tree, $bucket, $name) = @_; my $literal = ""; for my $child ($tree->children) { if (ref $child eq "PPIx::Regexp::Token::Literal") { $literal .= $child->content; } elsif (ref $child eq "PPIx::Regexp::Token::Whitespace") { # skip } else { if (length($literal) > 1 && $literal ne '\n') { $bucket->{$name}->{'"' . $literal . '"'} = 1; } $literal = ""; } 102
103. ͍͍ײ͡ͷՄࢹԽ (5) if (ref $child eq "PPIx::Regexp::Token::Recursion") { next if $child->name eq "PerlOWS"; $bucket->{$name}->{$child->content} = 1; } if ($child->can("children")) { my $next_name = $name; if (ref $child eq "PPIx::Regexp::Structure::NamedCapture") { next if $child->name =~ /^PPR_/; $next_name = $name ? $name . " > (?&" . $child->name . ")" : "(?&" . $child->name . ")"; #PPIx::Regexp::Dumper->new($child)->print; } traverse($child, $bucket, $next_name); } } } 103
104. ͍͍ײ͡ͷՄࢹԽ (6) (?&PerlPackageDeclaration) |-- "\}" |-- "package" |-- (?&PerlBlock) |-- (?&PerlNWS) |-- (?&PerlQualifiedIdentifier) `-- (?&PerlVersionNumber) (?&PerlParenthesesList) |-- "\(" `-- (?&PerlExpression) (?&PerlPod) `-- "=cut" 104
105. ࣗ༝ࣗࡏʹτʔΫϯΛൈ͖ग़ͤΔʂ PPRͷத਎΋Θ͔ͬͨʂ 105
106. ͜͜·ͰདྷΔͱܕΛॻ͍ͯ ࢖͍ͨ͘ͳΓ·ͤΜ͔ʁ 106
107. ઴ਐత੩తܕ෇͚ •ಈతܕ෇͚ݴޠʹ෦෼తʹ੩తܕ෇͚Λಋ ೖ͢Δ • TypeScript(JavaScript) ΍ Hack(PHP) ͳ Ͳ 107
108. •ޙ෇͢Δʹ͸ҎԼͷཁૉ͕ඞཁ • ܕ͕ॻ͚ΔΑ͏ʹͳΔ (PythonͷPEP 484 Type Hints) • ੩తղੳͰܕΛௐ΂Δ͜ͱ͕ग़དྷΔ • ܕγεςϜΛ࣮૷͢Δ 108
109. ܕ͸͢Ͱʹॻ͚Δ package Greeter { use Function::Parameters; use Function::Return; use Types::Standard qw/Str Int Bool/; method say_hello(Str :$name, Int :$times) :Return(Bool) { say $self . ": Hello " . $name for 1..$times; return 1; } } Greeter->say_hello(name => "shinpei0213", times => 10); 109
110. ͜ͷؔ਺ͷ γάωνϟΛ PPRͰղੳ͢Δ 110
111. ίʔυͱͯ͠ಡΜͰ͍Ε͹͜͏ग़དྷΔ͚ΕͲ my $pinfo = Function::Parameters::info \&Greeter::say_hello; my $rinfo = Function::Return::info \&Greeter::say_hello; 111
112. ޙʑͷ͜ͱΛߟ͑Δͱ੩తղੳͰऔΓ͍ͨ 112
113. ΍ͬͯΈΑ͏ͱ͢Δ͕ͦ΋ͦ΋Perlͷίʔυͱͯ͠͏·͘ ೝࣝͯ͘͠Εͳ͍ my $doc_matcher = qr{ \A (?&PerlDocument) \Z $PPR::GRAMMAR }x; say !!($script =~ $doc_matcher); # => ""(false) 113
114. ໰୊͸ keyword plugin method say_hello(Str :$name, Int :$times) :Return(Bool) { say $self . ": Hello " . $name for 1..$times; return 1; } keyword plugin ͸ perl 5.12.0 ͔Β࢖͑Δจ ๏֦ுػೳ 114
115. PPR͸ͦ͏͍͏ͷ΋ߟྀग़དྷΔ࡞Γʹͳ͍ͬͯΔ •ͦ΋ͦ΋keyword pluginΛ؆୯ʹॻ͚Δ Keyword::Declare ͷͨΊʹ࡞ΒΕͨϞ δϡʔϧ •keywordͷϧʔϧΛ֦ுͰ͖Δ 115
116. Function::ParametersʹରԠ͢Δ (1) my $FP_GRAMMAR = qr{ (?(DEFINE) (?<PerlFunctionParametersMethod> method (?&PerlOWS) (?&PerlIdentifier) (?&PerlOWS) # => say_hello (?&kw_balanced_parens) (?&PerlOWS) # => (Str :$name, Int :$times) (?: (?&PerlAttributes) (?&PerlOWS) # => :Return(Bool) )?+ (?&PerlBlock) # => { ... } ) (?<PerlKeyword> (?&PerlFunctionParametersMethod) ) (?<kw_balanced_parens> \( (?: [^()]++ | (?&kw_balanced_parens) )*+ \) ) ) $PPR::GRAMMAR }x; 116
117. Function::ParametersʹରԠ͢Δ (2) qr{ (?<PerlFunctionParametersMethod> method (?&PerlOWS) (?&PerlIdentifier) (?&PerlOWS) # => say_hello (?&kw_balanced_parens) (?&PerlOWS) # => (Str :$name, Int :$times) (?: (?&PerlAttributes) (?&PerlOWS) # => :Return(Bool) )?+ (?&PerlBlock) # => { ... } ) } 117
118. Function::ParametersʹରԠ͢Δ (3) qr{ (?<PerlKeyword> # <= keyword֦ுͱͯ͠ొ࿥ (?&PerlFunctionParametersMethod) ) (?<kw_balanced_parens> \( (?: [^()]++ | (?&kw_balanced_parens) )*+ \) ) } 118
119. Function::ParametersʹରԠ͢Δ (4) my $doc_matcher = qr{ \A (?&PerlDocument) \Z $FP_GRAMMAR }x; say !!($script =~ $doc_matcher); # => 1 Perlίʔυͱͯ͠ೝࣝͯ͘͠Εͨʂ ͜ΕͰউ ͭΔʂ 119
120. DEMO 120
121. ܕ͕ݟΕͨ͋ͱͷ࢖͍Έͪ •ܕγεςϜΛ࣮૷ͯ͠઴ਐత੩తܕ෇͚ •Language Server Protocolʹར༻͢Δ • ΤσΟλ/IDEͰิ׬΍ϝιου৘ใ͕ར ༻Ͱ͖ΔΑ͏ʹͳΔ 121
122. PPI౳ͱPPRͷ࢖͍෼͚ •ίʔυͷ͢΂ͯΛར༻͢ΔͳΒPPI •ίʔυͷҰ෦෼Λൈ͖ग़ͨ͠Γ༗Δແ͠Λ ݟΔͳΒPPR • ਖ਼نදݱͳͷͰϦϑΝΫλϦϯάʹ΋ར ༻Մೳ 122
123. ·ͱΊ •Perl5͸े෼ʹ੩తղੳ͕Ͱ͖Δ͜ͱΛࣔ͠ ·ͨ͠ •۩ମతͳ΍Γํ΍࢖͍ಓʹ͍ͭͯ࿩͠·͠ ͨ •ػցͱਓؒͷڞଘʹ͍ͭͯ࿩͠·ͨ͠ 123
124. Any Questions? 124
125. Q: ࣏҆Λྑ͘͢ΔͨΊͷίʔυ͕࣏͕҆ѱ͍ •A1: ਖ਼نදݱ͸ίʔυͷ࣏͕҆ѱ͘ͳΔҰ ͭͷཁૉͰ͕͢ɺίϝϯτ΍ϧʔϧΛ׆༻ ͠·͠ΐ͏ 125
126. Q: ࣏҆Λྑ͘͢ΔͨΊͷίʔυ͕࣏͕҆ѱ͍ my $matcher = qr{ # ͜͜ʹίϝϯτ͕ॻ͚·͢ (?(DEFINE) (?<MyCustomPattern> # ͜͜ʹ޷͖ͳύλʔϯΛ࡞ͬͯύλʔϯʹ໊લΛ͚ͭΑ͏! ) ) }x; 126
127. Q: ͳΜͰͦ͜·Ͱͯ͠PerlΛ࢖͏ΜͰ͔͢ (1) •A1: 10ສߦҎ্͋ΔPerlίʔυ͕͋Γɺࠓ ͪΌΜͱಈ͍͍ͯͯɺͪΌΜͱอक΋͞Ε ͍ͯΔ৔߹ʹଞͷݴޠʹஔ͖׵͑Δ͔Ͳ͏ ͔ • ଞͷݴޠͷ઴ਐతܕ෇͚ͷ࣮૷͢Δಈػ 127
128. Q: ͳΜͰͦ͜·Ͱͯ͠PerlΛ࢖͏ΜͰ͔͢ (2) •A2: Perl5͸͔ͳΓջͷਂ͍ݴޠ • hackable • จ๏͕͔ͳΓࣗ༝ͰɺϞμϯͬΆ͍ॻ͖ ํ͕Ͱ͖Δ • keyword plugin 128