Upgrade to Pro — share decks privately, control downloads, hide ads and more …

ヒューマンコンピュテーションによる協調学習支援 / Human Computation for...

Yukino Baba
June 16, 2018
2.4k

ヒューマンコンピュテーションによる協調学習支援 / Human Computation for Collaborative Learning

NEW EDUCATION EXPO 2018

Yukino Baba

June 16, 2018
Tweet

Transcript

  1.  -tap to take a photo.  -tap to begin

    recording your question and again to stop.        side,     User     ? Database - al Client mote Services and Worker Interface ؼُ٦وٝ؝ٝؾُذ٦ءّٝך⢽7J[8J[ ➂䊨濼腉ך➿׻׶ח➂꟦ָ鋔鋙ꥺְָ罏׾佄䴂 Ύءأذيⰻ鿇 ך➂꟦ָ㔐瘶 J. Bigham et al.: VizWiz: Nearly real-time answers to visual questions, In UIST, 2010. ΍ِ٦ؠָ颵㉏׾䫎珲 ⢽չ؝٦ָٝⰅ׏ ׋綸כו׸պ 5/15
  2. 湱✼鐰⣣ ! 欰䖝Ⰻ㆞ָ姻׃ֻ鐰⣣דֹ׷הכꣲ׵זְ ‛ ♧אך瘶周׾醱侧ך欰䖝ָ鐰⣣ծ 鐰⣣罏ך腉⸂׾罋䣁׃姻׃ְ挿侧׾✮庠 ? 瘶周 鐰⣣罏 挿侧

    㔐瘶罏 湱✼鐰⣣穠卓ַ׵姻׃ְ挿侧׾✮庠 Y. Baba and H. Kashima: Statistical quality estimation for general crowdsourcing tasks, In KDD, 2013. 挿 挿 挿 ぐ鐰⣣罏ך䱰挿穠卓ַ׵ 鐰⣣罏ך腉⸂׾罋䣁׃ג挿侧׾✮庠 7/15
  3. 湱✼鐰⣣ٌرٕ ! 㔐瘶罏ָ瘶周׾⡲׷麓玎٥鐰⣣罏ָ瘶周׾䱰挿ׅ׷麓玎׾ ٌرٕ⻉ׅ׷ֿהד瘶周ך溪ך挿侧׾✮庠דֹ׷ o 4UFQ㔐瘶罏ָ㉏겗ח㼎׃$% 挿ך瘶周׾⡲䧭ׅ׷ծ 挿侧$% כ 䎂㖱%

    , ⴓ侔% *ח䖞ֲ 瘶周ך溪ך挿侧 㔐瘶罏ך挿侧ךⴓ侔 㔐瘶罏ך腉⸂ 瘶周 㔐瘶罏 qjk ⇠ N qjk | µk, 2 k $%挿 㔐瘶罏ָ瘶周׾⡲׷麓玎׾ٌرٕ⻉ 8/15
  4. ! 㔐瘶罏ָ瘶周׾⡲׷麓玎٥鐰⣣罏ָ瘶周׾䱰挿ׅ׷麓玎׾ ٌرٕ⻉ׅ׷ֿהד瘶周ך溪ך挿侧׾✮庠דֹ׷ o 4UFQ鐰⣣罏ָ瘶周ח㼎׃-$% 挿׾➰♷ׅ׷ծ 挿侧-$% כ䎂㖱$% + -

    , ⴓ侔- *ח䖞ֲ 湱✼鐰⣣ٌرٕ 鐰⣣罏ך挿侧ךⴓ侔 瘶周ך溪ך挿侧 鐰⣣罏ךغ؎،أ 瘶周 鐰⣣罏 -$%挿 sijk ⇠ N sijk | qjk + ⌘i, 2 i 鐰⣣罏ָ瘶周׾䱰挿ׅ׷麓玎׾ٌرٕ⻉ 9/15
  5. ♧㼎嫰鯰ח״׷湱✼鐰⣣ ! 瘶周ח挿侧׾➰ֽ׷ךָꨇ׃ְ㜥さ׮֮׷ ‛ 瘶周ل،׾嫰鯰ׇׁծ 嫰鯰穠卓ַ׵鐰⣣罏ך腉⸂׾罋䣁׃ג瘶周ך挿侧׾✮庠 ? "ך挿侧 㔐瘶罏 T.

    Sunahase, Y. Baba and H. Kashima, Pairwise HITS: Quality estimation from pairwise comparisons in creator-evaluator crowdsourcing process, In AAAI, 2017 瘶周" 瘶周# " " # # # "    ? #ך挿侧 ぐ鐰⣣罏ך♧㼎嫰鯰穠卓ַ׵ 鐰⣣罏ך腉⸂׾罋䣁׃ג挿侧׾✮庠 ♧㼎嫰鯰穠卓ַ׵姻׃ְ挿侧׾✮庠 10/15
  6. ♧㼎嫰鯰ٌرٕ ! 溪ך挿侧ה鐰⣣罏ך腉⸂׾❛✼ח刿倜 o 4UFQ鐰⣣罏腉⸂- ׾㔿㹀׃溪ך挿侧$ ׾刿倜 o 4UFQ溪ך挿侧$ ׾㔿㹀׃鐰⣣罏腉⸂-

    ׾刿倜 qj qj0 = X i2Vj j0 ri X i2Vj0 j ri 瘶周ח䫎牰׃׋ ➂ך腉⸂ךㄤ ri = |{(j j0) 2 Vi | qj > qj0 }| |Vi | 鐰⣣罏ך 姻׃ְ䫎牰 ךⶴさ 挿侧ה鐰⣣罏腉⸂׾❛✼ח刿倜׃挿侧׾✮庠 11/15 瘶周ה’ך溪 ך挿侧ך䊴 瘶周ˏח䫎牰׃ ׋➂ך腉⸂ךㄤ 溪ך挿侧ָ넝ְ倯ך 瘶周פך鐰⣣罏ך 䫎牰㔐侧 鐰⣣罏ך 䫎牰㔐侧
  7. 湱✼幐⵴ ! ֶ✼ְח䱰挿ׅ׷׌ֽדכזֻ ֶ✼ְח幐⵴׮ׅ׷ֿהד瘶周פךؿ؍٦سغحؙ׾䲿⣘ ! 䱰挿ה幐⵴ך♧顐䚍׾ⵃ欽׃ג幐⵴罏腉⸂׾䱿㹀ծ 姻׃ְ挿侧ך✮庠ח崞欽 o 䱰挿ה幐⵴ָ♧顐׃זְ⢽ չ挿侧כ⡚ְךח幐⵴ָⰋ搫זְպ

    չ挿侧ָ넝ְךח幐⵴׾׋ֻׁ׿ׅ׷պ We observe that sometimes there is inconsistency between a grade and a correction; for example, a grader provides a high grade with a submission but she corrects many errors. We observed the occasional inconsistency between a grade and the correction; for example, a grader provides a high grade for a submission, but many errors were corrected.  挿 T. Sunahase, Y. Baba and H. Kashima, Statistical modeling of peer correction and peer assessment, submitted. 䱰挿ה幐⵴ך♧顐䚍׾ⵃ欽׃ג鐰⣣罏腉⸂׾䱿㹀 12/15
  8. 湱✼⡲㉏ ! 侄䌌ך➿׻׶ח欰䖝ָذأزך㉏겗׾⡲䧭 o ぐ欰䖝כ➭ך欰䖝ָ⡲׏׋㉏겗׾鍑ֻ ! 欰䖝ָ⡲׏׋㉏겗ח㼎ׅ׷㔐瘶ַ׵ぐ欰䖝ך统擾䏝׾✮庠 o 葺ְ㉏겗׾⡲׷欰䖝׮ְ׸ל׉ֲׄׯזְ欰䖝׮ְ׷ Ԃ欰䖝׀הך⡲㉏腉⸂׾罋䣁

    A. Taniguchi and S. Inoue, A method for automatic assessment of user-generated tests and its evaluation, In UbiComp/ISWC Adjunct, 2015. ،وبٝך،ؚٓ٘ٔ屎 崧㚖ד饯ֿ׷˘ "ؾٗٗحؕ #ػٗٗحؕ $هٗٗحؕ %لٗٗحؕ 欰䖝ָ㉏겗׾⡲䧭 ➭ך欰䖝ָ㔐瘶 欰䖝ָ✼ְח㉏겗׾⳿׃さֲהֹך⡲㉏腉⸂׾䱿㹀 13/15
  9. Ⱏずⶼ⡲ L. Yu and J. V. Nickerson: Cooks or cobblers?

    crowd creativity through combination, In CHI, 2011. ⢽喱㶨ךرؠ؎ٝ 4UFQ痥⚅➿ך⦐⡤׾ぐ荈ָ欰䧭 4UFQ䫎牰ד⮚葺⦐⡤ ׾鼅䫙׃如⚅➿ך⦐⡤ ׾❛⿷ח״׶欰䧭 4UFQ鼅䫙٥❛⿷׾粸׶鵤ׅ 鼋⠗涸،ٕ؞ٔؤيח⦺ְⰟずⶼ⡲׾佄䴂 ! ✼ְך涪䟝׾⤛׃׫׿זד⡲ㅷ׾峤箺ׇׁגְֻ 14/15