Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
注意機構を用いた言語創発ゲーム
Search
Ryokan RI
August 19, 2023
Research
0
220
注意機構を用いた言語創発ゲーム
2023 JSAI オーガナイズドセッション
言語とコミュニケーションの創発 ~記号創発システムから共創的言語進化まで~
Ryokan RI
August 19, 2023
Tweet
Share
More Decks by Ryokan RI
See All by Ryokan RI
[論文紹介] Intuitive Fine-Tuning
ryou0634
0
130
[論文紹介] Language is primarily a tool for communication rather than thought
ryou0634
4
1k
マルチリンガルな言語モデル入門:これまでとこれから
ryou0634
4
4.5k
What Are You Token About? Dense Retrieval as Distributions Over the Vocabulary
ryou0634
0
680
人工言語を使った事前訓練:言語間転移が可能なエンコーダの持っている知識とは何か?
ryou0634
0
940
MIROSTAT で意外さを コントロールした文章生成
ryou0634
1
980
Other Decks in Research
See All in Research
なめらかなシステムと運用維持の終わらぬ未来 / dicomo2025_coherently_fittable_system
monochromegane
0
3.9k
MIRU2025 チュートリアル講演「ロボット基盤モデルの最前線」
haraduka
15
9.1k
RHO-1: Not All Tokens Are What You Need
sansan_randd
1
190
CoRL2025速報
rpc
1
2.2k
[RSJ25] Enhancing VLA Performance in Understanding and Executing Free-form Instructions via Visual Prompt-based Paraphrasing
keio_smilab
PRO
0
150
AWSで実現した大規模日本語VLM学習用データセット "MOMIJI" 構築パイプライン/buiding-momiji
studio_graph
2
770
Pythonでジオを使い倒そう! 〜それとFOSS4G Hiroshima 2026のご紹介を少し〜
wata909
0
990
問いを起点に、社会と共鳴する知を育む場へ
matsumoto_r
PRO
0
660
【輪講資料】Moshi: a speech-text foundation model for real-time dialogue
hpprc
3
760
AI エージェントを活用した研究再現性の自動定量評価 / scisci2025
upura
1
170
PhD Defense 2025: Visual Understanding of Human Hands in Interactions
tkhkaeio
1
260
長期・短期メモリを活用したエージェントの個別最適化
isidaitc
0
200
Featured
See All Featured
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
31
2.5k
Mobile First: as difficult as doing things right
swwweet
225
10k
Build The Right Thing And Hit Your Dates
maggiecrowley
37
2.9k
Connecting the Dots Between Site Speed, User Experience & Your Business [WebExpo 2025]
tammyeverts
10
600
Faster Mobile Websites
deanohume
310
31k
Product Roadmaps are Hard
iamctodd
PRO
55
11k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
52
5.6k
No one is an island. Learnings from fostering a developers community.
thoeni
21
3.5k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
367
27k
Embracing the Ebb and Flow
colly
88
4.9k
The Cost Of JavaScript in 2023
addyosmani
55
9k
Building Adaptive Systems
keathley
44
2.8k
Transcript
ҙػߏΛ༻͍ͨ ݴޠൃήʔϜ ཥ ྇פɹ্ా ྄ɹJason Naradowsky ౦ژେֶ
ࢦࣔήʔϜͱʁ - ίϛϡχέʔγϣϯ͔Βࢦࣔͷଆ໘͚ͩΛऔΓग़ͨ͠ήʔϜɻ - ൃίϛϡχέʔγϣϯͷݚڀͰΑ͘ΘΕΔɻ ࢦࣔήʔϜ (referential game)
ࢦࣔήʔϜ (referential game) Speaker 3 14 2 25 ඪΦϒδΣΫτ Listener
ΦϒδΣΫτީิू߹
ࢦࣔήʔϜ (referential game) Speaker 3 14 2 25 Listener ඪΦϒδΣΫτ
ΦϒδΣΫτީิू߹ ᶃ Speaker ͕ ඪΦϒδΣΫτΛݟͯ ϝοηʔδΛੜ
ࢦࣔήʔϜ (referential game) Speaker 3 14 2 25 Listener ඪΦϒδΣΫτ
ΦϒδΣΫτީิू߹ ᶄ Listener ͕ϝοηʔδΛݟͯ ΦϒδΣΫτީิू߹͔Β ඪΦϒδΣΫτΛ༧ଌ͢Δ
ࢦࣔήʔϜ (referential game) Speaker 3 14 2 25 ඪΦϒδΣΫτ Listener
ΦϒδΣΫτީิू߹ ᶅ Listener ͷ༧ଌͱ ඪΦϒδΣΫτ͕Ұக͍ͯ͠Ε ใु͕༩͑ΒΕΔɻ
ࢦࣔήʔϜͷΤʔδΣϯτʹҙػߏΛ࣮͢Δ ຊݚڀͰͬͨ͜ͱ Speaker Listener
ͳͥҙػߏ͔ʁ - ൃݴޠͷߏੑ্͕͕Δͣ - ൃݴޠͷղऍੑ্͕͕Δͣ ຊݚڀͰͬͨ͜ͱ
ͳͥҙػߏ͔ʁ - ൃݴޠͷߏੑ্͕͕Δͣ - ൃݴޠͷղऍੑ্͕͕Δͣ ຊݚڀͰͬͨ͜ͱ
ߏੑͱɿશମ͕෦ͷ૯͔ΒͳΔ͜ͱ ҙػߏൃݴޠͷߏੑΛ্͛Δʁ ߏੑ ߴ🔺 ߏੑ 🔻 5 3 2 3
2 1 6 4
ҙػߏൃݴޠͷߏੑΛ্͛Δʁ 5 Speaker ҙػߏ୯ޠͱΦϒδΣΫτͷ෦ͷ݁ͼ͖ͭ Λֶश͢Δͷʹཱͪͦ͏
ͳͥҙػߏ͔ʁ - ൃݴޠͷߏੑ্͕͕Δͣ - ൃݴޠͷղऍੑ্͕͕Δͣ ຊݚڀͰͬͨ͜ͱ
ҙػߏൃݴޠͷղऍੑΛ্͛Δʁ 5 Speaker ͜ͷ୯ޠ͜ͷҙຯͳͷͩͱΘ͔Δ
࣮ݧઃఆɿࢦࣔήʔϜ Speaker 3 14 2 25 ඪΦϒδΣΫτ Listener ΦϒδΣΫτީิू߹ Fashion
MNIST Λ༻͍ͨࢦࣔήʔϜ ʢεχʔΧʔͱϓϧΦʔόʔʣ
࣮ݧઃఆɿࢦࣔήʔϜ Speaker 3 14 2 25 ඪΦϒδΣΫτ Listener ΦϒδΣΫτީิू߹ Fashion
MNIST Λ༻͍ͨࢦࣔήʔϜ ʢεχʔΧʔͱϓϧΦʔόʔʣ
FashionMNIST - 10 छྨͷΞΠςϜͷը૾ΛؚΉσʔληοτ ࣮ݧઃఆɿࢦࣔήʔϜ
FashionMNIST - 10 छྨͷΞΠςϜͷը૾ΛؚΉσʔληοτ ࣮ݧઃఆɿࢦࣔήʔϜ ̎छྨͷΞΠςϜΛબΜͰ ̍ͭͷΦϒδΣΫτΛ࡞Δ
࣮ݧઃఆɿࢦࣔήʔϜ FashionMNIST Λ༻͍ͨࢦࣔήʔϜ - FashionMNIST த͔Βը૾Λαϯϓϧͯ͠ଟ༷ͳΦϒδΣΫ τΛ࡞ɻ - ಉ͡छྨͷΞΠςϜؚ͕·Ε͍ͯΕಉ͡ΦϒδΣΫτͰ͋Δ ͱΈͳ͢ɻ
- ΦϒδΣΫτͷछྨ - ܇࿅ηοτʹ30छྨɺධՁηοτʹ15छྨ༻ 10 C2 = 45
࣮ݧઃఆɿࢦࣔήʔϜ ਓؒʹ͍ۙݴޠΛֶश͢ΔͨΊʹ ಉ͡छྨͷҟͳΔΦϒδΣΫτΛ Speaker ͱ Listener ʹݟͤΔ ඪΦϒδΣΫτ ΦϒδΣΫτީิू߹
ҙػߏͳ͠ͷ Speaker ҙػߏ͋Γͷ Speaker ࣮ݧઃఆɿϞσϧ ҙػߏͳ͠ͷ Listner ҙػߏ͋Γͷ Listener ×
ͦΕͧΕͷΈ߹ΘͤͰ࣮ݧ ͞ΒʹΤʔδΣϯτΛ࣮͢ΔωοτϫʔΫͱͯ͠ LSTM ͱ Transformer ͦΕͧΕͰ࣮ݧ
࣮ݧઃఆɿϞσϧ ϕʔεϥΠϯɾΤʔδΣϯτ otgt o1 %FDPEFS &ODPEFS m1 m2 m3 ϝοηʔδΦϒδΣΫτ
߹கείΞ m0 m1 m2 m1 m2 m3 Speaker Listener - ҙػߏͳ͠ - ΦϒδΣΫτ୯ҰͷϕΫτϧͰදݱ͞ΕΔ
࣮ݧઃఆɿϞσϧ ҙػߏ͋ΓΤʔδΣϯτ otgt o1 %FDPEFS &ODPEFS m1 m2 m3 "UUFOUJPO
"UUFOUJPO m0 m1 m2 m1 m2 m3 Speaker Listener ϝοηʔδΦϒδΣΫτ ߹கείΞ - ҙػߏ͋Γ - ΦϒδΣΫτෳͷϕΫτϧͰදݱ͞ΕΔ
ҙػߏ͋Γ/ͳ͠ΤʔδΣϯτͷൺֱ - ҙػߏ͋Γͩͱߏੑͷई (TopSim) ্͕͢Δ ҙػߏ͋ΓΤʔδΣϯτͷݴޠͷௐࠪ ࣮ݧͨ͜͠ͱ
ҙػߏ͋Γ/ͳ͠ΤʔδΣϯτͷൺֱ ܇࿅ηοτͷਖ਼ղ ධՁηοτͷਖ਼ղ TopSimʢߏੑͷईʣ - ҙػߏ͕͋Δํ͕ਖ਼ղ͕ߴ͘ͳΔʹ͋Δ - ҙػߏ͕͋Δํ͕ɺTopSim ͕ߴ͘ͳΔʹ͋Δ
ҙػߏ͋ΓΤʔδΣϯτͷ ҙػߏͷՄࢹԽ Speaker Listener Speaker Listener Speaker Listener ޭͨ͠ύλʔϯ ޭͨ͠ύλʔϯ
ࣦഊͨ͠ύλʔϯ
ҙػߏ͋ΓΤʔδΣϯτͷ ൃͨ͠ݴޠͷγϯϘϧͱΦϒδΣΫτͷରԠ Speaker Listener
ҙػߏ͋ΓΤʔδΣϯτͷ ൃͨ͠ݴޠͷγϯϘϧͱΦϒδΣΫτͷରԠ Speaker Listener ্ணܥͷΦϒδΣΫτ ۺܥͷΦϒδΣΫτ
ҙػߏ͋ΓSpeaker ͱ Listener ͷ ҙൣғͷͣΕͱਖ਼ղͷؔ ͣΕ͕গͳ͍΄Ͳਖ਼ղ͍͢͠
ͬͨ͜ͱ - ࢦࣔήʔϜͷΤʔδΣϯτʹҙػߏΛ͚ͭΔ - FashionMNIST ͷը૾ΛͬͨࢦࣔήʔϜͰ࣮ݧ ͔ͬͨ͜ͱ - ҙػߏ͕͋Δͱߏੑ͕ߴ͍ݴޠ͕ൃ͍͢͠
- ҙػߏΛ࣋ͬͨΤʔδΣϯτղऍੑͷ͋ΔݴޠΛൃͤ͞ΒΕΔ ·ͱΊ
࣮͏ͪΐͬͱࣗવͰෳࡶͳը૾ΛͬͨࢦࣔήʔϜͰ ͏·͍ͬͯ͘΄͔͕ͬͨ͠ɺֶश͕͔ͬͨ͠ - ෳࡶͳը૾ͩͱղऍෆೳͳҙͷύλʔϯ͔͠؍͞Εͣ ➡︎ ҙػߏ͕ਓؒͷΑ͏ͳબੑΛ͍࣋ͬͯͳ͍͔Βʁ ਓؒͷ࣋ͭೝػೳͱݴޠͷੑ࣭ͷؔ࿈ͳͲ͕ௐΒΕΔͱ ָͦ͠͏ʁ ࠓޙͷൃల