Share
Embed
Transcript
14 slides
-
HCOMP 2020 CrowDEA: Multi-view Idea Prioritization with Crowds Yukino Baba
(University of Tsukuba) Jiyi Li (University of Yamanashi ) Hisashi Kashima (Kyoto University) -
We can collect a large number of ideas from crowds
What’s the best emblem design for the next Olympic Games? How can we prevent cheating in exams? Impose severe penalties … Have proctors watch students … Prepare di erent examination … -
How can we prioritize the collected ideas? • Criteria for
ideas are usually multi-faced • There rarely exists the single winner in terms of all criteria Modern Traditional Low-cost E ective Impose severe penalties … Prepare di erent examination … -
CrowDEA finds promising ideas in diverse viewpoints Target ideas INPUT:
Pairwise comparison ≻ ≻ -
CrowDEA finds promising ideas in diverse viewpoints Target ideas OUTPUT:
Priority map INPUT: Pairwise comparison ≻ ≻ -
CrowDEA finds promising ideas in diverse viewpoints Target ideas OUTPUT:
Priority map Priority INPUT: Pairwise comparison ≻ ≻ -
CrowDEA finds promising ideas in diverse viewpoints Target ideas OUTPUT:
Priority map Viewpoints INPUT: Pairwise comparison ≻ ≻ -
CrowDEA finds promising ideas in diverse viewpoints Target ideas OUTPUT:
Priority map Size Color INPUT: Pairwise comparison ≻ ≻ -
CrowDEA finds promising ideas in diverse viewpoints Target ideas OUTPUT:
Priority map Size Color Promising ideas INPUT: Pairwise comparison ≻ ≻ -
CrowDEA estimates embeddings and best viewpoints [Constraint 1] from the
best viewpoint of idea , it should be most valuable among all ideas: i vi ⊤xi > vi ⊤xj xi vi Best viewpoint Embedding Idea parameters wk Personal viewpoint Evaluator parameter -
CrowDEA estimates embeddings and best viewpoints [Constraint 2] if evaluator
prefers idea over idea : k i j wk ⊤xi > wk ⊤xj [Constraint 1] from the best viewpoint of idea , it should be most valuable among all ideas: i vi ⊤xi > vi ⊤xj xi vi Best viewpoint Embedding Idea parameters wk Personal viewpoint Evaluator parameter -
Case1 : Olympic emblem designs Top-5 ideas ranked by BT
Promising ideas detected by CrowDEA Modern Traditional -
Case 2: “How to prevent cheating in exams?” Impose severe
penalties for cheating, such as cancellation of modules for an entire year. Prepare two types of examination sheets with di erently ordered items, and distribute one to every student such that neighboring students have di erent exam sheets. Have proctors watch students from the back of an examination room. Instead of multiple-choice questions or short answer questions, use essay questions to make it di cult to copy the answers of other students. Promising ideas detected by CrowDEA -
Result: Ranking accuracy BT Crowd BT Blade-Chest BPR CrowDEA Bike
0.772 0.779 0.757 0.827 0.833 Cheat 0.768 0.767 0.813 0.789 0.893 Meeting 0.817 0.815 0.829 0.800 0.877 Night 0.790 0.790 0.903 0.853 0.917 Visitor 0.818 0.825 0.868 0.933 0.938 Character 0.902 0.912 0.866 0.929 0.930 Olympics 0.926 0.926 0.920 0.940 0.936 Dataset Method Average of nDCG@5 among various viewpoints