Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
perl for shell, awk and sed programmers
Search
mackee
October 04, 2024
Programming
2
860
perl for shell, awk and sed programmers
YAPC::Hakodate 2024 Track A 12:25〜
mackee
October 04, 2024
Tweet
Share
More Decks by mackee
See All by mackee
今更GoのWebフレームワークを作ろうとしているワケ / Why am I trying to create a Go web framework now?
mackee
1
53
マイクロサービス化を利用した Goへの移行事例
mackee
0
96
PerlでつくるフルスクラッチWebAuthn/パスキー認証 / Demonstration of full-scratch WebAuthn/Passkey Authentication written in Perl
mackee
3
3k
SRE定例やその辺の取り組みをアプリケーションエンジニア目線で語る / "Observe" about SRE Meeting by Application Engineer
mackee
0
1.8k
TinyGoで使えるORM sqllaの 紹介とTinyGoで使えるようにするための工夫
mackee
0
1k
Go向けORM sqllaの紹介と JOINやUNIONを含んだクエリの扱い方
mackee
0
4.1k
デプロイ今昔物語 〜CGIからサーバーレスまで〜 / The deployment technics
mackee
10
11k
E2Eテストから負荷試験シナリオを作ってみた / Why do we make a scenario of load testing from E2E testing scenarios
mackee
3
5.2k
じわじわとPerlからGoに移行しようとしている俺達のマイクロサービシーズの紹介 / The migrating to microservices in Go from Monolith in Perl
mackee
2
3k
Other Decks in Programming
See All in Programming
How to debug Xdebug... or any other weird bug in PHP
dunglas
2
1.4k
cgroup v2 support in Kubeadm
kentatada
0
170
Competitionsだけじゃない! Kaggle Notebooks Grandmasterのすすめ
corochann
2
820
C#および.NETに対する誤解をひも解く
ymd65536
0
320
実践サーバーレスパフォーマンスチューニング ~その実力に迫る~ / Practical Serverless Performance Tuning ~A Close Look at its Power~
seike460
PRO
2
210
20241004 モノタロウ式~ドメインモデリングとリアーキテクチャ
monotaro
PRO
2
480
Scan with Decoupled Look-back and Onesweep Radix Sort
shocker_0x15
0
130
Modern Angular with Lightweight Stores: New Rules and Options
manfredsteyer
PRO
0
130
Mercari AI/LLM Hackathon TeamBの発表資料
imaikosuke
0
140
Why I Choose NetBeans for Jakarta EE
ivargrimstad
0
520
Kubernetes上でOracle_Databaseの運用を楽にするOraOperatorの紹介
nnaka2992
0
160
自分だけの世界を創るクリエイティブコーディング / Creative Coding: Creating Your Own World
chobishiba
2
2k
Featured
See All Featured
Creatively Recalculating Your Daily Design Routine
revolveconf
217
12k
Build your cross-platform service in a week with App Engine
jlugia
229
18k
Bash Introduction
62gerente
608
210k
It's Worth the Effort
3n
183
27k
XXLCSS - How to scale CSS and keep your sanity
sugarenia
246
1.3M
The MySQL Ecosystem @ GitHub 2015
samlambert
250
12k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
126
18k
In The Pink: A Labor of Love
frogandcode
139
22k
How to Ace a Technical Interview
jacobian
275
23k
Web development in the modern age
philhawksworth
205
10k
Keith and Marios Guide to Fast Websites
keithpitt
408
22k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
225
22k
Transcript
perl for shell, awk and sed programmers YAPC::Hakodate 2024 @mackee_w(macopy)
͜ͷτʔΫͰओு͍ͨ͜͠ͱ •օ͞ΜʹPerlΛॻ͍ͯΒ͍͍ͨ!!! • ͔͠͠ैདྷͷ͍ํ(WebͳͲ)ͰଞͷϓϩάϥϛϯάݴޠͰॻ͍͍ͯΔ ͜ͱͰ͠ΐ͏… • ͦ͜Ͱ ֤छshell, awk, sedΛ͍ͬͯΔ໘Ͱ
perlΛͬͯΈͯͲ͏͔ͱ͍͏ఏҊͰ͢
͜ͷτʔΫͰ͢͜ͱ • Ͳ͏͍͏໘Ͱshell,awk,sed͕ΘΕ͍ͯΔ͔ • shell,awk,sedͦΕͧΕͷϝϦοτɾσϝϦοτ • perlͷ͜ΕΒʹର͢Δ༗རͳ • ͍ͭshell,awk,sedΛ͏͖͔ɾ͍ͭperlΛ͏͖͔
͜ͷτʔΫͰ͞ͳ͍͜ͱ • PerlҎ֎ͷεΫϦϓτݴޠͱͷൺֱ • ͠Ӊਓ͕߈ΊࠐΜͰ͖ͯϦϑΝϨϯεΧϯτҎ֎ͷGCΛਓྨʹېࢭͨ࣌͠ͷ͜ͱΛߟ͍͑ͯͩ͘͞… • खʹೃછΉεΫϦϓτݴޠ͕͋ΕͦͪΒΛ͏ͷྑ͍Ͱ͠ΐ͏ • γΣϧةͳ͍ʂͱ͔awkෆศʂͱ͔ͦ͏͍͏ݴઆ •
͜͏͍͏έʔεؒҧ͍͍͢ɾඇޓ͕͋Δݴ͏͚ΕͲਓ͕ؒ ؾΛ͚ͭΕશ෦͑·͢
Δਓ • X: @mackee_w • GitHub: @mackee • PAUSE: MACOPY
• ໘ന๏ਓΧϠοΫ • SRE/αʔόʔαΠυΤϯδχΞ
[AD] ίʔυΰϧϑίϯςετPerlbatrossͬͯ·͢
ऍ: `perl`ͱ`Perl`ͷҧ͍ • ͜ͷεϥΠυͰ`perl`ͱ`Perl`ΛҎԼͷΑ͏ʹ͍͚·͢ • `perl`: perlίϚϯυͷ͜ͱ ྫ: ʮperlίϚϯυͷҾʙʯ •
`Perl`: ϓϩάϥϛϯάݴޠͷ͜ͱ ྫ: ʮPerlͷҰจࣈมʙʯ • ରԠ • bash, zsh, awk, sed => perl • γΣϧεΫϦϓτ, awkεΫϦϓτ => Perl
sed • ඪ४ೖྗͷςΩετΛҰఆͷϧʔϧͰஔ͠ग़ྗ͢ΔίϚϯυ • ਖ਼نදݱͰͷஔʹಛԽͨ͠ίϚϯυͱݴ͑Δ • จࣈྻஔΛ͍ͨ͠߹ʹศརʂ
Α͘Δͭ snake case/camel caseม
Α͘Δͭ snake case/camel caseม gsedͬͯԿʁ
!CAUTION!: sedʹෳͷ࣮͕͋Γ·͢ • BSD sedͱGNU sed • macOSʹBSD sed͕ೖ͍ͬͯΔ •
֦ுਖ਼نදݱͲͪΒ -r ͔ -E Ͱ༗ޮʹͳΔ • খจࣈେจࣈม(\l\u\L\U)GNU sedͷ֦ு • BSD sedͰ-iΦϓγϣϯΛҾແ͠Ͱ͑ͳ͍
!CAUTION!: sedʹෳͷ࣮͕͋Γ·͢ ྫ͑ -i ͷڍಈ͕ҧ͏
ղܾํ๏ • ࣮ߦଆͷsedΛݻఆ͢Δ • macOSͰGNU sedΛೖΕΔͳͲ • ͔͠͠aliasΛషͬͨΓgsedͱ࣮ߦ͢Δ͔Ͳ͏͔ΛܾΊΔϥούʔε ΫϦϓτ͕ඞཁʹ •
͜ͷτʔΫͰͷఏҊ: perlΛ͏ͷͲ͏͔
perlperl͔͠ͳ͍
awk • ඪ४ೖྗ͔ΒϑΟϧλมΛߦͳͬͯඪ४ग़ྗʹग़͢ίϚϯυ • ͦͷมϑΟϧλͷࢦࣔΛawkεΫϦϓτͱݺͿ • ࣮ͬͱڧྗͳػೳ͕͋ΓɺͦΕYAPC::Hiroshima 2024ͷτʔ ΫʮawkͰͭͬͯ͘Θ͔ΔɺWebΞϓϦέʔγϣϯʯʮϓϩάϥϛ ϯάݴޠAWKʯΛಡΈ·͠ΐ͏
awkͷ༻ྫ ߦࢦσʔλͷॲཧʹϐολϦ
awkͷಘҙͳ͜ͱ͍ಓ • ߦ͝ͱʹεϖʔεΧϯϚ۠Γͷೖྗ(CSVͳͲ)Λ౷ܭॲཧ͢Δ • ߦͣͭɺϑΟʔϧυʹର͢Δਖ਼نදݱͳͲͷύλʔϯΛ؆୯ʹ ͔͚Δ • ෳͷϑΟʔϧυʹׂͨ͠ޙʹਖ਼نදݱΛؚΜͩݕࡧΛ͢Δ߹ ϑΟʔϧυͰ؆୯ͳܭࢉूܭΛ͢Δ߹ʹڧ͍ •
sedgrepͳͲʹൺΔͱͱ͍͏
!CAUTION! : awkʹෳͷ࣮͕͋Γ·͢ʂ • macOSʹೖ͍ͬͯΔnawkͱLinuxͳͲͷgawk, mawk, BusyBox awk • Ұ෦ͷมɾ͕ؔgawkҎ֎Ͱ͑ͳ͍
• systime, strftimeͳͲͷ࣌ؒؔͷؔgawkͷΈ • matchͷޙํࢀরΛมʹೖ͢Δetc… • ύλʔϯϚονͰࠩҟ͕͋Δ
ύλʔϯϚονͰͷࠩҟ
gawkͷΈͰಈ͘ྫ
awkͰಈ͍͍ͯͳ͍
ղܾࡦ • ར༻ऀଆͷawkΛݻఆ͢Δ • macOSͷํbrewͰgawkΛೖΕ·͠ΐ͏Ͷͱ͔ • ͔ͦ͠͠ͷ߹ϥούʔεΫϦϓτͰذ͕ඞཁʹͳΓ… • ͜ͷτʔΫͷఏҊ: perlͰಉ༷ͷ͜ͱΛΔͷͲ͏͔ʁ
AWKʹ͔ͳ͍ॲཧ͋Δ • awkΈࠐΈ͕ؔগͳ͘ɺ·ͨϞδϡʔϧͷγεςϜͳ͍ɻ • ୯ͳΔߦ୯Ґͷϧʔϓɺ࢛ଇԋࢉɺਖ਼نදݱʹΑΔՃҎ্ͱͳΔͱ͔ͳΓݫ ͍͠ • JSONͷύʔεͱ͔ • ʮϓϩάϥϛϯάݴޠawkୈ2൛ʯʹawkεΫϦϓτ͔ΒCݴޠͷίʔυ
ʹτϥϯεύΠϧ͢Δawkύʔαʔͷίʔυ͕awkͰॻ͔Ε͍ͯΔ(!) • ࣮༻͢Δʹ͍ͩͿݫ͍͠(ओ؍ͷҙݟͰ͢)
perlʹawkͬΆ͍Ϟʔυ͕͋Δ
perlϫϯϥΠφʔͰΔ͍͍ͱ͜Ζ • ਖ਼نදݱपΓ͕͞Βʹڧྗ • nawkͩͱޙํࢀরͷѻ͍͕͍͠(RSTART,RLENGTH) • PerlͳΒevalͯؔͬͯ͠Ճ·ͰͰ͖ͯ͠·͏ • JSONΒѻ͑ΔΑʂ •
jqΛͬͯϑΟϧλ͢Δέʔε࣮perlϫϯϥΠφʔͰͰ͖·͢ • ෳࡶͳߏΛѻ͑Δ ϋογϡͳͲ
Ұํawkͷྑ͍ͱ͜Ζ • ߦ୯ҐɾϑΟʔϧυ୯ҐͷॲཧΛ؆୯ʹॻ͘ʹൺֱత͘ॻ͚Δ • બతʹผϑΝΠϧग़ྗ͢ΔͳͲͰ͖Δ • PerlϑΝΠϧΛಡΈॻ͖͢Δͷ(ൺֱత)ख͕͔͔ؒΔ
shell • ͜͜Ͱbash, zshͳͲͷPOSIXγΣϧ࣮ͷ૯শͱ͠·͢ • օ͞Μ͕LinuxBSD, macOS্ͰίϚϯυΛଧͭ࣌ʹ͏ͭ • ͍ٞͨ͠ͷshellΛλʔήοτͱͯ͠༻͍ͯهड़ɾ࣮ߦ͢ΔγΣϧ εΫϦϓτ
ྫ: IPv6ͱIPv4Ͱͷifcon fi g.meͷग़ྗͷҧ͍Λgist
γΣϧεΫϦϓτάϧʔݴޠͰ͋Δ • άϧʔݴޠ: ίϯϙʔωϯτಉ࢜Λ݁ͼ͚ͭΔ͜ͱΛओ؟ͱͨ͠ݴޠ • ԿΛ͚ͬͭ͘Δͷ͔ʁ: ίϚϯυͱϑΝΠϧ • ૉͰॻ͍ͨͷίϚϯυ࣮ߦʹͳΔ •
ଞͷϓϩάϥϛϯάݴޠʹͳ͍ಛ • ϦμΠϨΫτͳͲϑΝΠϧͷಡΈॻ͖ʹศརͳػೳ͕͋Δ • ͦͦUNIX͕ϑΝΠϧૢ࡞Λओମͱͨ͠OSͰ͋Δ
shellͲΜͳڥͰೖ͍ͬͯΔ…ͱ͞Ε͍ͯΔʁ • ʢͱ͍͏લఏΛ͓͍ͯྑ͍ͱ͞Ε͍ͯΔʣʢཁग़యʣ • Πϯετʔϧͷ͜ͱΛߟ͑ͳͯ͘ྑ͍͠ɺҰॻ͚ͲΜͳڥͰ ಈ͘ͱ͞Ε͍ͯΔ • ྫ͑։ൃऀͷϩʔΧϧͰಈ͘εΫϦϓτͳͲಈ࡞ڥΛ౷ҰͰ ͖ͳͯ͘γΣϧεΫϦϓτಈ͘Մೳੑ͕ߴ͍ •
ͱ͞Ε͍ͯΔ……ʁ • ͦ͏Ͱͳͦ͞͏
!CAUTION!: γΣϧεΫϦϓτͷ࣮͕ෳ͋Δ • ݱͰ͘ΘΕ͍ͯΔͷͩͱBusyBox ash, dash, bash, zsh etc… •
େମbash͕ͩ…macOSͰzsh, BusyBoxͰash,DebianͰ dash • ͦΕͧΕPOSIXγΣϧʹՃ͑ͯಠࣗͷ֦ுػೳ͕͋Δ
ઌͷγΣϧεΫϦϓτ͕ଞͷ࣮Ͱಈ͔ͳ͍ྫ
bash֦ுͷศརͳػೳ • ϓϩηεஔ • γΣϧมͷஔ • ifͳͲͷࣜධՁ [[ … ]]
ղܾࡦ • POSIXγΣϧͱͯ͠ͷػೳͷΈͰॻ͘ • ࣮ߦଆͷγΣϧΛݻఆ͢Δ • shebangʹ/bin/bashͱॻ͚ྑ͍ • ͕ɺAlpineLinuxͰՃͰೖΕͳ͍ͱ͍͚ͳ͘ͳΔ •
͜͜ͰͷఏҊ: PerlͰॻ͍ͯperlͰಈ͔ͯ͠ΈͯͲ͏͔ʁ
γΣϧεΫϦϓτؒҧ͍͑͢ϙΠϯτ͕͋Δ • վߦεϖʔεͷ͋Δͳ͕͔͠ͳΓݫີ͔ͭɺҧ͏ҙຯʹͳΔ͜ͱ͕ ͋Δ • มͷείʔϓ͕μΠφϛοΫείʔϓ • ୯ޠׂɾύε໊ల։
γΣϧεΫϦϓτؒҧ͍͑͢ϙΠϯτ͕͋Δ
ղܾࡦ • shellcheckΛ͍·͠ΐ͏ 👉 • γΣϧεΫϦϓτ͚ͷLinter • ςετΛॻ͘ • Bats-core,
ShellSpec • ͜͜ͰͷఏҊ: PerlͰॻ͍ͯperl Ͱಈ͔͢ͷͲ͏͔ʁ
Perl/perlͷ͍͍ͱ͜Ζ • perlͱ͍͑ͨͩҰͭͷ࣮Λࢦ͢ ͦͯ͠ѹతͳޙํޓੑ • (POSIXγΣϧͱൺͯ)ਐԽ͠ଓ͚Δݴޠػೳ • Track B 14:50ʙʮྩ࠷৽൛PerlίʔσΟϯάΨΠυʯΛฉ͘ͱྑͦ͞͏
• ༻WebΞϓϦ։ൃ͕͘Ͱ͖Δఔʹ·ͱͳϓϩάϥϛϯάݴޠ • ‘use strict;’ ͳͲͷΨʔυϨʔϧ͋ΔఔΈࠐΈଘࡏ͢Δ • ίΞϞδϡʔϧͰͰ͖Δ͜ͱ͕γΣϧεΫϦϓτ+POSIXίϚϯυΑΓଟ͍ • ͍͟ͱͳΕCPANϞδϡʔϧͰػೳ֦ுͰ͖Δ γΣϧbpkgͱ͍͏ͷ͕͋Γ·͢ • ෳࡶͳσʔλߏΛѻ͑Δ
Perl/perlͷͭΒ͍ͱ͜Ζ • ಡΈॻ͖Ͱ͖Δਓ͕গͳ͘ͳ͍ͬͯΔ • ͜ͷτʔΫͷཪͷతࢲ͕ࣄͰPerlεΫϦϓτΛॻ͍ͯڻ͔Εͳ͍͜ͱͰ͢ • ୯ͳΔίϚϯυ࣮ߦϑΝΠϧͷಡΈॻ͖γΣϧεΫϦϓτͷํ͕؆୯ʹॻ͚ΔՄೳੑ ͕ߴ͍ • Perlͷ߹֎෦ίϚϯυ࣮ߦsystem(...),
`…`, qx(…), ͳͲͱॻ͔ͳ͍ͱ͍͚ͳ͍ • ͜ͷ߹ίϚϯυΑΓϞδϡʔϧΛͬͨ΄͏͕Perlͷར͕ग़Δ • ࣮ߦʹඞཁͳϑΝΠϧαΠζ(perlίϚϯυ)͕େ͖͍ • ͜Εʹ͍ͭͯޙղܾࡦΛϒϩάʹॻ͖·͢
Perlͱͦͷଞͷ͍͚ • ͋͠ͳ͕ͨΓ͍ͨ͜ͱ͕ߦࢦσʔλͷγϯϓϧͳϑΟϧλɾม͔ͭɺෳ ͷϑΝΠϧʹు͖͍ͨͷͰ͋Ε • awkͰΔͱ͙͘͢ॻ͚ΔՄೳੑ͕ߴ͍ • ͋͠ͳ͕ͨΓ͍ͨ͜ͱ͕ίϚϯυ࣮ߦϑΝΠϧૢ࡞͕ओମͰ͋Ε • γΣϧεΫϦϓτͷํ͕ͤʹͳΔՄೳੑ͕ߴ͍
• Γ͍ͨ͜ͱ͕ෳࡶͳσʔλͷมɾՃɺߏԽσʔλͷύʔεͰ͋Ε • PerlͰΔํ͕ͤʹͳΔՄೳੑ͕ߴ͍
͜Εෳࡶͩͳʁ ޓੑΛߟྀ͠ͳͪ͘ΌͱࢥͬͨΒ PerlͰॻ͘͜ͱݕ౼ͯ͠Έ͍ͯͩ͘͞
ࢀߟจݙ • ࡾल໌.৽͍͠γΣϧϓϩάϥϛϯάͷڭՊॻ.SBΫϦΤΠςΟϒ,2017. • Alfred V, Aho; Brian W, Kernighan;
Peter J, Weinberger; ઌॅ ࣏ ༁.ϓϩάϥϛϯάݴޠ AWK ୈೋ൛.ΦϥΠ Ϧʔδϟύϯ,2024 • εΫϦϓτݴޠͱͯ͠Έ֤ͨPOSIXγΣϧͷಛͱޓੑ্ͷҙ·ͱΊ https://qiita.com/ko1nksm/ items/8d28d4f7cb2c325c00fa • sed(1) - FreeBSD Manual Pages https://man.freebsd.org/cgi/man.cgi?sed(1) • GNU sed https://www.gnu.org/software/sed/ • AWKॲཧܥͷൺֱ http://www.yosbits.com/wordpress/?p=2702 • ༣ศ൪߸σʔλμϯϩʔυ https://www.post.japanpost.jp/zipcode/download.html
ఆ࣭ • ͦͷଞඇޓ͕͋ΔίϚϯυ make, grep ͷ߹ʁ • Ͳ͏͢Ε͍͍ΜͰ͠ΐ͏Ͷ͑ʁɹͲ͏ͯ͠·͢ʁɹmakeʹؔͯ͠npm run scriptrakeͳͲͷݴޠσϑΝΫτͷΓ
ํͰஔ͖͑ͨํ͕͍͍ؾ͕͢Δͱ͍͏ͷ͕࠷ۙͷߟ͑Ͱ͢ • ͖ͳperlͷΦϓγϣϯΛڭ͑ͯʂ • `-E` Ͱ͢ɻsayͳͲͷϞμϯͳػೳ͕Ұؾʹ༗ޮʹͳΔ͔Β • ීஈͷΒ͠ͰPerl/perlΛͲ͏͍͏;͏ʹͬͯ·͔͢ • Α͘͏ͷ `rg <searchword> ./ -l | xargs perl -pi -E ’s/…/…/‘` • base64/url encodeͷมͳΜ͔perlίϚϯυͰͬͯ·͢ • JSONܗࣜͷϩάϑΝΠϧ͔ΒಡΈऔͬͯίϚϯυΛෳݸ࣮ߦ͢ΔͷʹPerlεΫϦϓτͰίϚϯυΛྻڍ͢ΔγΣϧε ΫϦϓτΛग़ྗͯ͠ɺҰԠਓͷͰݟ͔ͯΒ࣮ߦͨ͜͠ͱ͕͋Δ
ݕূʹ༻ͨ͠όʔδϣϯ • sed (macOS Sonomaͷͷ) • gsed (GNU sed) 4.9
• awk version 20200816 (nawk) • GNU awk 5.3.1, API 4.0, (GNU MPFR 4.2.1, GNU MP 6.3.0) • shellʹ͍ͭͯϖʔδ27ʹهࡌ • This is perl 5, version 34, subversion 0 (v5.34.0) built for darwin-thread- multi-2level