Share
Embed
ヒューマンコンピュテーションによる協調学習支援 / Human Computation for Collaborative Learning
NEW EDUCATION EXPO 2018
Yukino Baba PRO
June 16, 2018
Transcript
15 slides
-
ؼُ٦وٝ؝ٝؾُذ٦ءّٝך⢽SF$"15$)" ➂䊨濼腉ך➿ח➂ָ俑㶵钠陎 ˑNPSPJEH˒ ˑNPSSJOH˒ ˑNPSOJOH˒ 4UFQ剅硂⚥ך俑㶵אך0$3ءأذيח钠陎ׇׁ 4UFQ穠卓ָ♶♧荜ךהֹ➂ח㉏ְさׇ L. von Ahn
et al.: reCAPTCHA: Human-based character recognition via web security measures, In Science, 2008. 4/15 -
-tap to take a photo. -tap to begin
recording your question and again to stop. side, User ? Database - al Client mote Services and Worker Interface ؼُ٦وٝ؝ٝؾُذ٦ءّٝך⢽7J[8J[ ➂䊨濼腉ך➿ח➂ָ鋔鋙ꥺְָ罏佄䴂 Ύءأذيⰻ鿇 ך➂ָ㔐瘶 J. Bigham et al.: VizWiz: Nearly real-time answers to visual questions, In UIST, 2010. ِ٦ؠָ颵㉏䫎珲 ⢽չ؝٦ָٝⰅ 綸כוպ 5/15 -
ؼُ٦وٝ؝ٝؾُذ٦ءّٝה侄肪 ! ؼُ٦وٝ؝ٝؾُذ٦ءّٝך侄肪פך崞欽⢽䱰挿 o 侄䌌ח״ٖه٦ز瘝ך䱰挿 ➂䊨濼腉ח״荈⹛䱰挿ח縧ֹ䳔ְִָ劢㔭ꨇ ☞ ➂䊨濼腉ך➿ח欰䖝ず㡦ד䱰挿湱✼鐰⣣ ! 湱✼鐰⣣כ
欰䖝ず㡦ֶָ✼ְך㷕统⸔ֽさֲ⼿锃㷕统 ך⤛鹌חאזָ ➂䊨濼腉ך➿ח欰䖝ָ侄䌌佄䴂 6/15 -
湱✼鐰⣣ ! 欰䖝Ⰻ㆞ָ姻׃ֻ鐰⣣דֹהכꣲזְ ‛ ♧אך瘶周醱侧ך欰䖝ָ鐰⣣ծ 鐰⣣罏ך腉⸂罋䣁׃姻׃ְ挿侧✮庠 ? 瘶周 鐰⣣罏 挿侧
㔐瘶罏 湱✼鐰⣣穠卓ַ姻׃ְ挿侧✮庠 Y. Baba and H. Kashima: Statistical quality estimation for general crowdsourcing tasks, In KDD, 2013. 挿 挿 挿 ぐ鐰⣣罏ך䱰挿穠卓ַ 鐰⣣罏ך腉⸂罋䣁׃ג挿侧✮庠 7/15 -
湱✼鐰⣣ٌرٕ ! 㔐瘶罏ָ瘶周⡲麓玎٥鐰⣣罏ָ瘶周䱰挿ׅ麓玎 ٌرٕ⻉ֿׅהד瘶周ך溪ך挿侧✮庠דֹ o 4UFQ㔐瘶罏ָ㉏겗ח㼎׃$% 挿ך瘶周⡲䧭ׅծ 挿侧$% כ 䎂㖱%
, ⴓ侔% *ח䖞ֲ 瘶周ך溪ך挿侧 㔐瘶罏ך挿侧ךⴓ侔 㔐瘶罏ך腉⸂ 瘶周 㔐瘶罏 qjk ⇠ N qjk | µk, 2 k $%挿 㔐瘶罏ָ瘶周⡲麓玎ٌرٕ⻉ 8/15 -
! 㔐瘶罏ָ瘶周⡲麓玎٥鐰⣣罏ָ瘶周䱰挿ׅ麓玎 ٌرٕ⻉ֿׅהד瘶周ך溪ך挿侧✮庠דֹ o 4UFQ鐰⣣罏ָ瘶周ח㼎׃-$% 挿➰♷ׅծ 挿侧-$% כ䎂㖱$% + -
, ⴓ侔- *ח䖞ֲ 湱✼鐰⣣ٌرٕ 鐰⣣罏ך挿侧ךⴓ侔 瘶周ך溪ך挿侧 鐰⣣罏ךغ؎،أ 瘶周 鐰⣣罏 -$%挿 sijk ⇠ N sijk | qjk + ⌘i, 2 i 鐰⣣罏ָ瘶周䱰挿ׅ麓玎ٌرٕ⻉ 9/15 -
♧㼎嫰鯰ח״湱✼鐰⣣ ! 瘶周ח挿侧➰ֽךָꨇ׃ְ㜥さ֮ ‛ 瘶周ل،嫰鯰ׇׁծ 嫰鯰穠卓ַ鐰⣣罏ך腉⸂罋䣁׃ג瘶周ך挿侧✮庠 ? "ך挿侧 㔐瘶罏 T.
Sunahase, Y. Baba and H. Kashima, Pairwise HITS: Quality estimation from pairwise comparisons in creator-evaluator crowdsourcing process, In AAAI, 2017 瘶周" 瘶周# " " # # # " ? #ך挿侧 ぐ鐰⣣罏ך♧㼎嫰鯰穠卓ַ 鐰⣣罏ך腉⸂罋䣁׃ג挿侧✮庠 ♧㼎嫰鯰穠卓ַ姻׃ְ挿侧✮庠 10/15 -
♧㼎嫰鯰ٌرٕ ! 溪ך挿侧ה鐰⣣罏ך腉⸂❛✼ח刿倜 o 4UFQ鐰⣣罏腉⸂- 㔿㹀׃溪ך挿侧$ 刿倜 o 4UFQ溪ך挿侧$ 㔿㹀׃鐰⣣罏腉⸂-
刿倜 qj qj0 = X i2Vj j0 ri X i2Vj0 j ri 瘶周ח䫎牰׃ ➂ך腉⸂ךㄤ ri = |{(j j0) 2 Vi | qj > qj0 }| |Vi | 鐰⣣罏ך 姻׃ְ䫎牰 ךⶴさ 挿侧ה鐰⣣罏腉⸂❛✼ח刿倜׃挿侧✮庠 11/15 瘶周ה’ך溪 ך挿侧ך䊴 瘶周ˏח䫎牰׃ ➂ך腉⸂ךㄤ 溪ך挿侧ָ넝ְ倯ך 瘶周פך鐰⣣罏ך 䫎牰㔐侧 鐰⣣罏ך 䫎牰㔐侧 -
湱✼幐 ! ֶ✼ְח䱰挿ֽׅדכזֻ ֶ✼ְח幐ֿׅהד瘶周פךؿ؍٦سغحؙ䲿⣘ ! 䱰挿ה幐ך♧顐䚍ⵃ欽׃ג幐罏腉⸂䱿㹀ծ 姻׃ְ挿侧ך✮庠ח崞欽 o 䱰挿ה幐ָ♧顐׃זְ⢽ չ挿侧כ⡚ְךח幐ָⰋ搫זְպ
չ挿侧ָ넝ְךח幐ֻׁׅպ We observe that sometimes there is inconsistency between a grade and a correction; for example, a grader provides a high grade with a submission but she corrects many errors. We observed the occasional inconsistency between a grade and the correction; for example, a grader provides a high grade for a submission, but many errors were corrected. 挿 T. Sunahase, Y. Baba and H. Kashima, Statistical modeling of peer correction and peer assessment, submitted. 䱰挿ה幐ך♧顐䚍ⵃ欽׃ג鐰⣣罏腉⸂䱿㹀 12/15 -
湱✼⡲㉏ ! 侄䌌ך➿ח欰䖝ָذأزך㉏겗⡲䧭 o ぐ欰䖝כ➭ך欰䖝ָ⡲㉏겗鍑ֻ ! 欰䖝ָ⡲㉏겗ח㼎ׅ㔐瘶ַぐ欰䖝ך统擾䏝✮庠 o 葺ְ㉏겗⡲欰䖝ְלֲׄׯזְ欰䖝ְ Ԃ欰䖝׀הך⡲㉏腉⸂罋䣁
A. Taniguchi and S. Inoue, A method for automatic assessment of user-generated tests and its evaluation, In UbiComp/ISWC Adjunct, 2015. ،وبٝך،ؚٓ٘ٔ屎 崧㚖ד饯ֿ˘ "ؾٗٗحؕ #ػٗٗحؕ $هٗٗحؕ %لٗٗحؕ 欰䖝ָ㉏겗⡲䧭 ➭ך欰䖝ָ㔐瘶 欰䖝ָ✼ְח㉏겗⳿׃さֲהֹך⡲㉏腉⸂䱿㹀 13/15 -
Ⱏずⶼ⡲ L. Yu and J. V. Nickerson: Cooks or cobblers?
crowd creativity through combination, In CHI, 2011. ⢽喱㶨ךرؠ؎ٝ 4UFQ痥⚅➿ך⦐⡤ぐ荈ָ欰䧭 4UFQ䫎牰ד⮚葺⦐⡤ 鼅䫙׃如⚅➿ך⦐⡤ ❛⿷ח״欰䧭 4UFQ鼅䫙٥❛⿷粸鵤ׅ 鼋⠗涸،ٕ؞ٔؤيח⦺ְⰟずⶼ⡲佄䴂 ! ✼ְך涪䟝⤛׃זד⡲ㅷ峤箺ׇׁגְֻ 14/15