Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
注意機構を用いた言語創発ゲーム
Search
Ryokan RI
August 19, 2023
Research
0
180
注意機構を用いた言語創発ゲーム
2023 JSAI オーガナイズドセッション
言語とコミュニケーションの創発 ~記号創発システムから共創的言語進化まで~
Ryokan RI
August 19, 2023
Tweet
Share
More Decks by Ryokan RI
See All by Ryokan RI
Language is primarily a tool for communication rather than thought
ryou0634
4
880
マルチリンガルな言語モデル入門:これまでとこれから
ryou0634
4
4.1k
What Are You Token About? Dense Retrieval as Distributions Over the Vocabulary
ryou0634
0
650
人工言語を使った事前訓練:言語間転移が可能なエンコーダの持っている知識とは何か?
ryou0634
0
830
MIROSTAT で意外さを コントロールした文章生成
ryou0634
1
840
Other Decks in Research
See All in Research
Optimal and Diffusion Transports in Machine Learning
gpeyre
0
1.4k
NLP2025 WS Shared Task 文法誤り訂正部門 ehiMetrick
sugiyamaseiji
0
150
資産間の相関関係を頑健に評価する指標を用いたファクターアローケーション戦略の構築
nomamist
0
170
博士学位論文予備審査 / Scaling Telemetry Workloads in Cloud Applications: Techniques for Instrumentation, Storage, and Mining
yuukit
1
1.8k
The Economics of Platforms 輪読会 第1章
tomonatu8
0
160
打率7割を実現する、プロダクトディスカバリーの7つの極意(pmconf2024)
geshi0820
0
410
インドネシアのQA事情を紹介するの
yujijs
0
170
AWS 音声基盤モデル トーク解析AI MiiTelの音声処理について
ken57
0
170
NLP2025SharedTask翻訳部門
moriokataku
0
260
ソフトウェア研究における脅威モデリング
laysakura
0
2k
Bluesky Game Dev
trezy
0
180
言語モデルによるAI創薬の進展 / Advancements in AI-Driven Drug Discovery Using Language Models
tsurubee
1
260
Featured
See All Featured
Building Flexible Design Systems
yeseniaperezcruz
328
38k
Fireside Chat
paigeccino
37
3.3k
4 Signs Your Business is Dying
shpigford
183
22k
It's Worth the Effort
3n
184
28k
Six Lessons from altMBA
skipperchong
27
3.7k
How STYLIGHT went responsive
nonsquared
99
5.4k
Done Done
chrislema
183
16k
Building an army of robots
kneath
304
45k
Measuring & Analyzing Core Web Vitals
bluesmoon
6
320
RailsConf 2023
tenderlove
29
1k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
356
30k
The Myth of the Modular Monolith - Day 2 Keynote - Rails World 2024
eileencodes
22
2.6k
Transcript
ҙػߏΛ༻͍ͨ ݴޠൃήʔϜ ཥ ྇פɹ্ా ྄ɹJason Naradowsky ౦ژେֶ
ࢦࣔήʔϜͱʁ - ίϛϡχέʔγϣϯ͔Βࢦࣔͷଆ໘͚ͩΛऔΓग़ͨ͠ήʔϜɻ - ൃίϛϡχέʔγϣϯͷݚڀͰΑ͘ΘΕΔɻ ࢦࣔήʔϜ (referential game)
ࢦࣔήʔϜ (referential game) Speaker 3 14 2 25 ඪΦϒδΣΫτ Listener
ΦϒδΣΫτީิू߹
ࢦࣔήʔϜ (referential game) Speaker 3 14 2 25 Listener ඪΦϒδΣΫτ
ΦϒδΣΫτީิू߹ ᶃ Speaker ͕ ඪΦϒδΣΫτΛݟͯ ϝοηʔδΛੜ
ࢦࣔήʔϜ (referential game) Speaker 3 14 2 25 Listener ඪΦϒδΣΫτ
ΦϒδΣΫτީิू߹ ᶄ Listener ͕ϝοηʔδΛݟͯ ΦϒδΣΫτީิू߹͔Β ඪΦϒδΣΫτΛ༧ଌ͢Δ
ࢦࣔήʔϜ (referential game) Speaker 3 14 2 25 ඪΦϒδΣΫτ Listener
ΦϒδΣΫτީิू߹ ᶅ Listener ͷ༧ଌͱ ඪΦϒδΣΫτ͕Ұக͍ͯ͠Ε ใु͕༩͑ΒΕΔɻ
ࢦࣔήʔϜͷΤʔδΣϯτʹҙػߏΛ࣮͢Δ ຊݚڀͰͬͨ͜ͱ Speaker Listener
ͳͥҙػߏ͔ʁ - ൃݴޠͷߏੑ্͕͕Δͣ - ൃݴޠͷղऍੑ্͕͕Δͣ ຊݚڀͰͬͨ͜ͱ
ͳͥҙػߏ͔ʁ - ൃݴޠͷߏੑ্͕͕Δͣ - ൃݴޠͷղऍੑ্͕͕Δͣ ຊݚڀͰͬͨ͜ͱ
ߏੑͱɿશମ͕෦ͷ૯͔ΒͳΔ͜ͱ ҙػߏൃݴޠͷߏੑΛ্͛Δʁ ߏੑ ߴ🔺 ߏੑ 🔻 5 3 2 3
2 1 6 4
ҙػߏൃݴޠͷߏੑΛ্͛Δʁ 5 Speaker ҙػߏ୯ޠͱΦϒδΣΫτͷ෦ͷ݁ͼ͖ͭ Λֶश͢Δͷʹཱͪͦ͏
ͳͥҙػߏ͔ʁ - ൃݴޠͷߏੑ্͕͕Δͣ - ൃݴޠͷղऍੑ্͕͕Δͣ ຊݚڀͰͬͨ͜ͱ
ҙػߏൃݴޠͷղऍੑΛ্͛Δʁ 5 Speaker ͜ͷ୯ޠ͜ͷҙຯͳͷͩͱΘ͔Δ
࣮ݧઃఆɿࢦࣔήʔϜ Speaker 3 14 2 25 ඪΦϒδΣΫτ Listener ΦϒδΣΫτީิू߹ Fashion
MNIST Λ༻͍ͨࢦࣔήʔϜ ʢεχʔΧʔͱϓϧΦʔόʔʣ
࣮ݧઃఆɿࢦࣔήʔϜ Speaker 3 14 2 25 ඪΦϒδΣΫτ Listener ΦϒδΣΫτީิू߹ Fashion
MNIST Λ༻͍ͨࢦࣔήʔϜ ʢεχʔΧʔͱϓϧΦʔόʔʣ
FashionMNIST - 10 छྨͷΞΠςϜͷը૾ΛؚΉσʔληοτ ࣮ݧઃఆɿࢦࣔήʔϜ
FashionMNIST - 10 छྨͷΞΠςϜͷը૾ΛؚΉσʔληοτ ࣮ݧઃఆɿࢦࣔήʔϜ ̎छྨͷΞΠςϜΛબΜͰ ̍ͭͷΦϒδΣΫτΛ࡞Δ
࣮ݧઃఆɿࢦࣔήʔϜ FashionMNIST Λ༻͍ͨࢦࣔήʔϜ - FashionMNIST த͔Βը૾Λαϯϓϧͯ͠ଟ༷ͳΦϒδΣΫ τΛ࡞ɻ - ಉ͡छྨͷΞΠςϜؚ͕·Ε͍ͯΕಉ͡ΦϒδΣΫτͰ͋Δ ͱΈͳ͢ɻ
- ΦϒδΣΫτͷछྨ - ܇࿅ηοτʹ30छྨɺධՁηοτʹ15छྨ༻ 10 C2 = 45
࣮ݧઃఆɿࢦࣔήʔϜ ਓؒʹ͍ۙݴޠΛֶश͢ΔͨΊʹ ಉ͡छྨͷҟͳΔΦϒδΣΫτΛ Speaker ͱ Listener ʹݟͤΔ ඪΦϒδΣΫτ ΦϒδΣΫτީิू߹
ҙػߏͳ͠ͷ Speaker ҙػߏ͋Γͷ Speaker ࣮ݧઃఆɿϞσϧ ҙػߏͳ͠ͷ Listner ҙػߏ͋Γͷ Listener ×
ͦΕͧΕͷΈ߹ΘͤͰ࣮ݧ ͞ΒʹΤʔδΣϯτΛ࣮͢ΔωοτϫʔΫͱͯ͠ LSTM ͱ Transformer ͦΕͧΕͰ࣮ݧ
࣮ݧઃఆɿϞσϧ ϕʔεϥΠϯɾΤʔδΣϯτ otgt o1 %FDPEFS &ODPEFS m1 m2 m3 ϝοηʔδΦϒδΣΫτ
߹கείΞ m0 m1 m2 m1 m2 m3 Speaker Listener - ҙػߏͳ͠ - ΦϒδΣΫτ୯ҰͷϕΫτϧͰදݱ͞ΕΔ
࣮ݧઃఆɿϞσϧ ҙػߏ͋ΓΤʔδΣϯτ otgt o1 %FDPEFS &ODPEFS m1 m2 m3 "UUFOUJPO
"UUFOUJPO m0 m1 m2 m1 m2 m3 Speaker Listener ϝοηʔδΦϒδΣΫτ ߹கείΞ - ҙػߏ͋Γ - ΦϒδΣΫτෳͷϕΫτϧͰදݱ͞ΕΔ
ҙػߏ͋Γ/ͳ͠ΤʔδΣϯτͷൺֱ - ҙػߏ͋Γͩͱߏੑͷई (TopSim) ্͕͢Δ ҙػߏ͋ΓΤʔδΣϯτͷݴޠͷௐࠪ ࣮ݧͨ͜͠ͱ
ҙػߏ͋Γ/ͳ͠ΤʔδΣϯτͷൺֱ ܇࿅ηοτͷਖ਼ղ ධՁηοτͷਖ਼ղ TopSimʢߏੑͷईʣ - ҙػߏ͕͋Δํ͕ਖ਼ղ͕ߴ͘ͳΔʹ͋Δ - ҙػߏ͕͋Δํ͕ɺTopSim ͕ߴ͘ͳΔʹ͋Δ
ҙػߏ͋ΓΤʔδΣϯτͷ ҙػߏͷՄࢹԽ Speaker Listener Speaker Listener Speaker Listener ޭͨ͠ύλʔϯ ޭͨ͠ύλʔϯ
ࣦഊͨ͠ύλʔϯ
ҙػߏ͋ΓΤʔδΣϯτͷ ൃͨ͠ݴޠͷγϯϘϧͱΦϒδΣΫτͷରԠ Speaker Listener
ҙػߏ͋ΓΤʔδΣϯτͷ ൃͨ͠ݴޠͷγϯϘϧͱΦϒδΣΫτͷରԠ Speaker Listener ্ணܥͷΦϒδΣΫτ ۺܥͷΦϒδΣΫτ
ҙػߏ͋ΓSpeaker ͱ Listener ͷ ҙൣғͷͣΕͱਖ਼ղͷؔ ͣΕ͕গͳ͍΄Ͳਖ਼ղ͍͢͠
ͬͨ͜ͱ - ࢦࣔήʔϜͷΤʔδΣϯτʹҙػߏΛ͚ͭΔ - FashionMNIST ͷը૾ΛͬͨࢦࣔήʔϜͰ࣮ݧ ͔ͬͨ͜ͱ - ҙػߏ͕͋Δͱߏੑ͕ߴ͍ݴޠ͕ൃ͍͢͠
- ҙػߏΛ࣋ͬͨΤʔδΣϯτղऍੑͷ͋ΔݴޠΛൃͤ͞ΒΕΔ ·ͱΊ
࣮͏ͪΐͬͱࣗવͰෳࡶͳը૾ΛͬͨࢦࣔήʔϜͰ ͏·͍ͬͯ͘΄͔͕ͬͨ͠ɺֶश͕͔ͬͨ͠ - ෳࡶͳը૾ͩͱղऍෆೳͳҙͷύλʔϯ͔͠؍͞Εͣ ➡︎ ҙػߏ͕ਓؒͷΑ͏ͳબੑΛ͍࣋ͬͯͳ͍͔Βʁ ਓؒͷ࣋ͭೝػೳͱݴޠͷੑ࣭ͷؔ࿈ͳͲ͕ௐΒΕΔͱ ָͦ͠͏ʁ ࠓޙͷൃల