Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
注意機構を用いた言語創発ゲーム
Search
Ryokan RI
August 19, 2023
Research
0
220
注意機構を用いた言語創発ゲーム
2023 JSAI オーガナイズドセッション
言語とコミュニケーションの創発 ~記号創発システムから共創的言語進化まで~
Ryokan RI
August 19, 2023
Tweet
Share
More Decks by Ryokan RI
See All by Ryokan RI
[論文紹介] Intuitive Fine-Tuning
ryou0634
0
120
[論文紹介] Language is primarily a tool for communication rather than thought
ryou0634
4
1k
マルチリンガルな言語モデル入門:これまでとこれから
ryou0634
4
4.4k
What Are You Token About? Dense Retrieval as Distributions Over the Vocabulary
ryou0634
0
680
人工言語を使った事前訓練:言語間転移が可能なエンコーダの持っている知識とは何か?
ryou0634
0
930
MIROSTAT で意外さを コントロールした文章生成
ryou0634
1
970
Other Decks in Research
See All in Research
SSII2025 [SS1] レンズレスカメラ
ssii
PRO
2
1.1k
2025年度人工知能学会全国大会チュートリアル講演「深層基盤モデルの数理」
taiji_suzuki
25
19k
集合間Bregmanダイバージェンスと置換不変NNによるその学習
wasyro
0
150
AIスパコン「さくらONE」のLLM学習ベンチマークによる性能評価 / SAKURAONE LLM Training Benchmarking
yuukit
2
610
日本語新聞記事を用いた大規模言語モデルの暗記定量化 / LLMC2025
upura
0
230
Combinatorial Search with Generators
kei18
0
860
なめらかなシステムと運用維持の終わらぬ未来 / dicomo2025_coherently_fittable_system
monochromegane
0
3.3k
A scalable, annual aboveground biomass product for monitoring carbon impacts of ecosystem restoration projects
satai
4
290
能動適応的実験計画
masakat0
2
830
Minimax and Bayes Optimal Best-arm Identification: Adaptive Experimental Design for Treatment Choice
masakat0
0
180
単施設でできる臨床研究の考え方
shuntaros
0
2.9k
MIRU2025 チュートリアル講演「ロボット基盤モデルの最前線」
haraduka
15
8.4k
Featured
See All Featured
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
49
3.1k
Mobile First: as difficult as doing things right
swwweet
224
9.9k
Performance Is Good for Brains [We Love Speed 2024]
tammyeverts
12
1.1k
Automating Front-end Workflow
addyosmani
1371
200k
No one is an island. Learnings from fostering a developers community.
thoeni
21
3.5k
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
507
140k
The Language of Interfaces
destraynor
162
25k
Art, The Web, and Tiny UX
lynnandtonic
303
21k
Documentation Writing (for coders)
carmenintech
75
5k
Why Our Code Smells
bkeepers
PRO
339
57k
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
127
53k
Statistics for Hackers
jakevdp
799
220k
Transcript
ҙػߏΛ༻͍ͨ ݴޠൃήʔϜ ཥ ྇פɹ্ా ྄ɹJason Naradowsky ౦ژେֶ
ࢦࣔήʔϜͱʁ - ίϛϡχέʔγϣϯ͔Βࢦࣔͷଆ໘͚ͩΛऔΓग़ͨ͠ήʔϜɻ - ൃίϛϡχέʔγϣϯͷݚڀͰΑ͘ΘΕΔɻ ࢦࣔήʔϜ (referential game)
ࢦࣔήʔϜ (referential game) Speaker 3 14 2 25 ඪΦϒδΣΫτ Listener
ΦϒδΣΫτީิू߹
ࢦࣔήʔϜ (referential game) Speaker 3 14 2 25 Listener ඪΦϒδΣΫτ
ΦϒδΣΫτީิू߹ ᶃ Speaker ͕ ඪΦϒδΣΫτΛݟͯ ϝοηʔδΛੜ
ࢦࣔήʔϜ (referential game) Speaker 3 14 2 25 Listener ඪΦϒδΣΫτ
ΦϒδΣΫτީิू߹ ᶄ Listener ͕ϝοηʔδΛݟͯ ΦϒδΣΫτީิू߹͔Β ඪΦϒδΣΫτΛ༧ଌ͢Δ
ࢦࣔήʔϜ (referential game) Speaker 3 14 2 25 ඪΦϒδΣΫτ Listener
ΦϒδΣΫτީิू߹ ᶅ Listener ͷ༧ଌͱ ඪΦϒδΣΫτ͕Ұக͍ͯ͠Ε ใु͕༩͑ΒΕΔɻ
ࢦࣔήʔϜͷΤʔδΣϯτʹҙػߏΛ࣮͢Δ ຊݚڀͰͬͨ͜ͱ Speaker Listener
ͳͥҙػߏ͔ʁ - ൃݴޠͷߏੑ্͕͕Δͣ - ൃݴޠͷղऍੑ্͕͕Δͣ ຊݚڀͰͬͨ͜ͱ
ͳͥҙػߏ͔ʁ - ൃݴޠͷߏੑ্͕͕Δͣ - ൃݴޠͷղऍੑ্͕͕Δͣ ຊݚڀͰͬͨ͜ͱ
ߏੑͱɿશମ͕෦ͷ૯͔ΒͳΔ͜ͱ ҙػߏൃݴޠͷߏੑΛ্͛Δʁ ߏੑ ߴ🔺 ߏੑ 🔻 5 3 2 3
2 1 6 4
ҙػߏൃݴޠͷߏੑΛ্͛Δʁ 5 Speaker ҙػߏ୯ޠͱΦϒδΣΫτͷ෦ͷ݁ͼ͖ͭ Λֶश͢Δͷʹཱͪͦ͏
ͳͥҙػߏ͔ʁ - ൃݴޠͷߏੑ্͕͕Δͣ - ൃݴޠͷղऍੑ্͕͕Δͣ ຊݚڀͰͬͨ͜ͱ
ҙػߏൃݴޠͷղऍੑΛ্͛Δʁ 5 Speaker ͜ͷ୯ޠ͜ͷҙຯͳͷͩͱΘ͔Δ
࣮ݧઃఆɿࢦࣔήʔϜ Speaker 3 14 2 25 ඪΦϒδΣΫτ Listener ΦϒδΣΫτީิू߹ Fashion
MNIST Λ༻͍ͨࢦࣔήʔϜ ʢεχʔΧʔͱϓϧΦʔόʔʣ
࣮ݧઃఆɿࢦࣔήʔϜ Speaker 3 14 2 25 ඪΦϒδΣΫτ Listener ΦϒδΣΫτީิू߹ Fashion
MNIST Λ༻͍ͨࢦࣔήʔϜ ʢεχʔΧʔͱϓϧΦʔόʔʣ
FashionMNIST - 10 छྨͷΞΠςϜͷը૾ΛؚΉσʔληοτ ࣮ݧઃఆɿࢦࣔήʔϜ
FashionMNIST - 10 छྨͷΞΠςϜͷը૾ΛؚΉσʔληοτ ࣮ݧઃఆɿࢦࣔήʔϜ ̎छྨͷΞΠςϜΛબΜͰ ̍ͭͷΦϒδΣΫτΛ࡞Δ
࣮ݧઃఆɿࢦࣔήʔϜ FashionMNIST Λ༻͍ͨࢦࣔήʔϜ - FashionMNIST த͔Βը૾Λαϯϓϧͯ͠ଟ༷ͳΦϒδΣΫ τΛ࡞ɻ - ಉ͡छྨͷΞΠςϜؚ͕·Ε͍ͯΕಉ͡ΦϒδΣΫτͰ͋Δ ͱΈͳ͢ɻ
- ΦϒδΣΫτͷछྨ - ܇࿅ηοτʹ30छྨɺධՁηοτʹ15छྨ༻ 10 C2 = 45
࣮ݧઃఆɿࢦࣔήʔϜ ਓؒʹ͍ۙݴޠΛֶश͢ΔͨΊʹ ಉ͡छྨͷҟͳΔΦϒδΣΫτΛ Speaker ͱ Listener ʹݟͤΔ ඪΦϒδΣΫτ ΦϒδΣΫτީิू߹
ҙػߏͳ͠ͷ Speaker ҙػߏ͋Γͷ Speaker ࣮ݧઃఆɿϞσϧ ҙػߏͳ͠ͷ Listner ҙػߏ͋Γͷ Listener ×
ͦΕͧΕͷΈ߹ΘͤͰ࣮ݧ ͞ΒʹΤʔδΣϯτΛ࣮͢ΔωοτϫʔΫͱͯ͠ LSTM ͱ Transformer ͦΕͧΕͰ࣮ݧ
࣮ݧઃఆɿϞσϧ ϕʔεϥΠϯɾΤʔδΣϯτ otgt o1 %FDPEFS &ODPEFS m1 m2 m3 ϝοηʔδΦϒδΣΫτ
߹கείΞ m0 m1 m2 m1 m2 m3 Speaker Listener - ҙػߏͳ͠ - ΦϒδΣΫτ୯ҰͷϕΫτϧͰදݱ͞ΕΔ
࣮ݧઃఆɿϞσϧ ҙػߏ͋ΓΤʔδΣϯτ otgt o1 %FDPEFS &ODPEFS m1 m2 m3 "UUFOUJPO
"UUFOUJPO m0 m1 m2 m1 m2 m3 Speaker Listener ϝοηʔδΦϒδΣΫτ ߹கείΞ - ҙػߏ͋Γ - ΦϒδΣΫτෳͷϕΫτϧͰදݱ͞ΕΔ
ҙػߏ͋Γ/ͳ͠ΤʔδΣϯτͷൺֱ - ҙػߏ͋Γͩͱߏੑͷई (TopSim) ্͕͢Δ ҙػߏ͋ΓΤʔδΣϯτͷݴޠͷௐࠪ ࣮ݧͨ͜͠ͱ
ҙػߏ͋Γ/ͳ͠ΤʔδΣϯτͷൺֱ ܇࿅ηοτͷਖ਼ղ ධՁηοτͷਖ਼ղ TopSimʢߏੑͷईʣ - ҙػߏ͕͋Δํ͕ਖ਼ղ͕ߴ͘ͳΔʹ͋Δ - ҙػߏ͕͋Δํ͕ɺTopSim ͕ߴ͘ͳΔʹ͋Δ
ҙػߏ͋ΓΤʔδΣϯτͷ ҙػߏͷՄࢹԽ Speaker Listener Speaker Listener Speaker Listener ޭͨ͠ύλʔϯ ޭͨ͠ύλʔϯ
ࣦഊͨ͠ύλʔϯ
ҙػߏ͋ΓΤʔδΣϯτͷ ൃͨ͠ݴޠͷγϯϘϧͱΦϒδΣΫτͷରԠ Speaker Listener
ҙػߏ͋ΓΤʔδΣϯτͷ ൃͨ͠ݴޠͷγϯϘϧͱΦϒδΣΫτͷରԠ Speaker Listener ্ணܥͷΦϒδΣΫτ ۺܥͷΦϒδΣΫτ
ҙػߏ͋ΓSpeaker ͱ Listener ͷ ҙൣғͷͣΕͱਖ਼ղͷؔ ͣΕ͕গͳ͍΄Ͳਖ਼ղ͍͢͠
ͬͨ͜ͱ - ࢦࣔήʔϜͷΤʔδΣϯτʹҙػߏΛ͚ͭΔ - FashionMNIST ͷը૾ΛͬͨࢦࣔήʔϜͰ࣮ݧ ͔ͬͨ͜ͱ - ҙػߏ͕͋Δͱߏੑ͕ߴ͍ݴޠ͕ൃ͍͢͠
- ҙػߏΛ࣋ͬͨΤʔδΣϯτղऍੑͷ͋ΔݴޠΛൃͤ͞ΒΕΔ ·ͱΊ
࣮͏ͪΐͬͱࣗવͰෳࡶͳը૾ΛͬͨࢦࣔήʔϜͰ ͏·͍ͬͯ͘΄͔͕ͬͨ͠ɺֶश͕͔ͬͨ͠ - ෳࡶͳը૾ͩͱղऍෆೳͳҙͷύλʔϯ͔͠؍͞Εͣ ➡︎ ҙػߏ͕ਓؒͷΑ͏ͳબੑΛ͍࣋ͬͯͳ͍͔Βʁ ਓؒͷ࣋ͭೝػೳͱݴޠͷੑ࣭ͷؔ࿈ͳͲ͕ௐΒΕΔͱ ָͦ͠͏ʁ ࠓޙͷൃల