Share
Embed
Transcript
35 slides
-
/35 )VNBOJOUIF-PPQػցֶशɿਓ͕ؒࢀՃ͢Δػցֶशϓϩηε 2 1BSU ܈ऺʹΑΔ ܇࿅σʔλ࡞ 1BSU
܈ऺ͔Βͷֶश 1BSU ػցֶशͷ༷ʑͳ໘Ͱͷ ܈ऺ׆༻ Ϟσϧ adelie gentoo chinstrap ܇࿅σʔλ Ϟσϧ ߹ͤ ڭࢣϥϕϧ ͷఏڙ ߹ͤ ༷ʑͳݟ ͷఏڙ 2ΑΓྑ͍ϞσϧΛޮతʹֶश͢ΔͨΊʹਓؒΛͲ͏׆༻͢Δ͔ʁ ڭࢣϥϕϧ ͷఏڙ ෆಛఆଟͷਓʢ܈ऺʣΛఆ -
/35 ˔ ܈ऺʹϥϕϧΛ͍߹Θͤ܇࿅σʔλΛ࡞ ˔ ෆಛఆଟͷਓ͕ࢀՃ͢ΔͨΊڭࢣϥϕϧͷ࣭͕՝ ৴པੑ͕͍ࢀՃऀ͍Δ ϥϕϧ͚ཁ͕݅ेʹΘΒͳ͍͜ͱ͕͋Δ
େྔͷϥϕϧ͚݁ՌΛґཔऀ͕શͯݕূ͢Δͷࠔ ࣭อূ ڭࢣϥϕϧͷ࣭͕՝ 5 Choose the correct category Adelie Chinstrap Gentoo “This was the best book I ever read!!! Thank you so much! :)” What emotion does this text convey? Anger Disgust Fear Happiness Sadness Surprise -
/35 ࢀՃऀબൈฒྻԽɾྻԽͰ࣭อূ 6 ࢀՃऀબൈ ฒྻԽ ྻԽ ࣄલςετͰબൈ ճ Chinstrap
Adelie Adelie ਖ਼ղطͷΛࠞͥͯબൈ ճ Gentoo Adelie Gentoo ·ͨ ͷਖ਼ղ"EFMJF ˠޡͨ͠ɹɹͷճશͯআ֎ ࣭อূ Adelie Adelie Adelie Gentoo ಉ͡ͷෳਓͷճΛ౷߹ Adelie Chinstrap Gentoo ଟܾͰ ౷߹ ͋ΔਓͷճΛଞऀ͕ݕূɾमਖ਼ ճ Gentoo Adelie Gentoo ճ Gentoo Adelie Gentoo ❌ ⭕ ❌ ݕূ ࣮͕༰қͳฒྻԽ͕ ͘༻͍ΒΕ͍ͯΔ -
/35 ฒྻԽʹΑΔ࣭อূ ଟܾͰ֤ࣗͷ৴པੑΛߟྀͰ͖ͳ͍ 7 NO YES YES YES YES YES
NO YES YES YES YES NO YES YES YES ଟܾ֤ࣗͷճͷॏΈ͕͍͠ͱΈͳ͕͢ ճऀ͝ͱʹ৴པੑҟͳΔͣ ଟܾͰ ༧ଌͨ͠ਖ਼ղ “Is a bird in the picture?" ❌ ⭕ ⭕ -
/35 ฒྻԽʹΑΔ࣭อূ ֤ࣗͷ৴པੑ͕ෆ໌ͳͨΊॏΈ͖ଟܾ͑ͳ͍ 8 NO YES YES YES YES
YES NO YES YES YES YES NO ɹɹɹ YES ɹɹɹ YES ɹɹɹ NO ॏΈ ॏΈ ॏΈ ॏΈ :&4ථ /0ථ :&4ථ /0ථ :&4ථ /0ථ ॏΈ͖ଟܾͰ ༧ଌͨ͠ਖ਼ղ ࣮ࡍʹॏΈΘ͔Βͳ͍ “Is a bird in the picture?" ⭕ ⭕ ⭕ -
/35 ˔ ճ͔Β֤ࣗͷ৴པੑͳͲΛਪఆ͠ਖ਼ղͷ༧ଌʹ༻͍Δ ˔ ੜెͷճ͚͔ͩΒࢼݧͷਖ਼ղΛ༧ଌ͢ΔΑ͏ͳͷ ਅ࣮ൃݟ ਅ࣮ൃݟ 5SVUIEJTDPWFSZ ɿෳਓͷճ͔Βͷਖ਼ղ༧ଌ 9
ਖ਼ղ NO YES YES YES YES YES NO YES YES YES YES NO ? ? ? “Is a bird in the picture?" -
/35 ਅ࣮ൃݟ %BXJE4LFOFճऀͷ৴པੑΛࠞಉߦྻͰදݱ 10 A. P. Dawid and A. M.
Skene: Maximum likelihood estimation of observer error-rates using the EM algorithm. Journal of the Royal Statistical Society, Series C (Applied Statistics), 1979. ɿճऀ ͕ਖ਼ղ͕YESͷʹYESͱ͑Δ֬ αj j ɿճऀ ͕ਖ਼ղ͕NOͷʹNOͱ͑Δ֬ βj j ճऀͷ৴པੑύϥϝʔλʢࠞಉߦྻʣ ti ਖ਼ղ YES ti = NO ti = yij βj ճ ճϞσϧʢ ʹର͢Δճऀ ͷճʣ i j αj ճ YES NO ਖ਼ ղ YES NO αj βj 1 − αj 1 − βj ࠞಉߦྻ Pr[yij ∣ ti = 1] = αyij j (1 − αj )(1−yij ) Pr[yij ∣ ti = 0] = β(1−yij ) j (1 − βj )yij -
/35 ਅ࣮ൃݟ %BXJE4LFOFճऀͷ৴པੑΛࠞಉߦྻͰදݱ 11 qi = Pr[ti = 1 ∣
{yij }] ∝ p∏ j αyij j (1 − αj )1−yij αj = ∑ i qi yij ∑ i qi , βj = ∑ i (1 − qi )yij ∑ i (1 − qi ) , p = ∑ i qi N ਖ਼ղͷࣄޙ֬ ˔ Λͬͯܭࢉ͠ Λࢉग़ ˔ Pr[ti = 0 ∣ {yij }] βj qi p = Pr[ti = 1] ਖ਼ղ͕YESͷͰͷਖ਼ͷΑ͏ͳͷ &TUFQճऀ৴པੑΛݻఆͯ͠ਖ਼ղΛਪఆ .TUFQਖ਼ղΛݻఆͯ͠ճऀ৴པੑΛਪఆ ˞ N &.ΞϧΰϦζϜΛ͍ճऀ৴པੑͱਖ਼ղΛަޓʹਪఆ -
/35 ਅ࣮ൃݟ %BXJE4LFOFʢͷվྑ൛ʣ"NB[PO.FDIBOJDBM5VSLͰར༻Մೳ 12 Amazon SageMaker GroundTruth "NB[PO.FDIBOJDBM5VSLΛར༻ͨ͠܇࿅σʔλ࡞Λࢧԉ https://aws.amazon.com/sagemaker/groundtruth/
https://aws.amazon.com/jp/blogs/news/use-the-wisdom-of-crowds-with-amazon-sagemaker-ground-truth-to-annotate-data-more-accurately/ ฒྻλεΫΛࣗಈൃߦɺ %BXJE4LFOFͰ ਖ਼ղΛ༧ଌͯ͠ग़ྗ ฒྻΛࢦఆ ୯ՁΛࢦఆ λεΫͷछྨΛࢦఆ -
/35 ਅ࣮ൃݟ $PNNVOJUZ#$$ࠞಉߦྻΛෳਓͰڞ௨Խ 13 M. Venanzi, J. Guiver, G. Kazai,
P. Kohli, and M. Shokouhi: Community-based bayesian aggregation models for crowdsourcing. WWW 2014. Truth Predicted Truth Predicted Truth Predicted 0 1 234 4 3 2 3 1 0 0 1 234 4 3 2 3 1 0 0 1 234 4 3 2 3 1 0 0.8 0.4 0 0.8 0.4 0 0.8 0.4 0 Decisive (5%) Conservative (4%) Calibrated (91%) ճऀ͕அʹج͍ͮͯ ίϛϡχςΟΛܗ͍ͯ͠Δͱ͢Δ ίϛϡχςΟͷࠞಉߦྻ͔Β ֤ࣗͷࠞಉߦྻ͕ੜ͞ΕΔ ࠞಉߦྻ ࠞಉߦྻͷਪఆྫɿίϛϡχςΟ͝ͱʹҟͳΔஅΛଊ͍͑ͯΔ 0: Negative 1: Neutral 2: Positive 3: Not-related 4: Unknown ࠞಉߦྻ -
/35 ˔ ྫɿ܈ऺʹΑΔ࠶൜ϦεΫ༧ଌಛఆͷਓछʹରͯ͠ෆެฏ ˔ ճऀͷόΠΞεΛਪఆɾআڈ͠ͳ͕Βެฏͳਖ਼ղΛਪఆ ਅ࣮ൃݟ 'BJS5%ෆެฏͳճ͔Βͷެฏͳਅ࣮ൃݟ 14 Y. Li,
H. Sun, W. H. Wang: Towards fair truth discovery from biased crowdsourced answers. KDD 2020. yij = ti + 𝒩 (0,σ2 j ) + bA=a j “The defendant is a [RACE] [SEX] aged [AGE]. They have been charged with: [CRIME CHARGE]. This crime is classi fi ed as a [CRIMINAL DEGREE]. They have been convicted of [NON-JUVENILE PRIOR COUNT] prior crimes. They have [JUVENILE- FELONY COUNT] juvenile felony charges and [JUVENILE-MISDEMEANOR COUNT] juvenile misdemeanor charges on their record.” Do you think this person commit another crime within 2 years? J. Dressel and H. Farid: The accuracy, fairness, and limits of predicting recidivism. Science advances, 2018. ճϞσϧ อޢάϧʔϓʹର͢ΔόΠΞε ΞϧΰϦζϜ ਪఆͨ͠ਖ਼ղͷόΠΞεΛࢉग़ɺ ٯ͖ͷόΠΞεΛ࣋ͭճऀΛҰਓબ બͨ͠ճऀҎ֎ͷόΠΞεΛআڈͯ͠ਖ਼ղΛਪఆ ਪఆͨ͠ਖ਼ղΛ༻͍ͯ Λߋ৽ bA=a j , σj -
/35 අ༻ͷޮԽ Ұఆਓ͕ಉ͡ճΛͨ࣌͠Ͱ߹ͤΛதࢭ 15 L. von Ahn, B. Maurer, C.
McMillen, D. Abraham, and M. Blum: reCAPTCHA: Human-based character recognition via web security measures. Science, 2008. morning morninq morning ̎ਓ͕ಉ͑͡Λฦͨ͠ͷͰ morning Λೝࣝ݁Ռͱͯ͠࠾༻ͯ͠ऴྃ Type the word Type the word SF$"15$)"ҹͷจࣈΛਓؒʹೝࣝͤ͞ΔϓϩδΣΫτ -
/35 ˔ ݱࡏ·Ͱͷ:&4/0ͷճʹԠͯ͡߹ͤͷՃΛܾఆ ˔ ޡఆͷ͕֬ҰఆҎ্ͷ߹ʹ߹ͤΛՃ͢Δ අ༻ͷޮԽ ෆ࣮֬ੑʹԠͯ͡৽ͨͳ߹ͤΛՃ 16 V. S.
Sheng, F. Provost, and P. G. Ipeirotis: Get another label? Improving data quality and data mining using multiple, noisy labelers. KDD 2008. NO YES YES YES NO :&4ͷਓ /0ͷਓ n = 3 ¯ n = 2 ผͷਓʹ͍߹ΘͤΔ͔ʁ ਖ਼ղ͕YESͰ͋Δࣄޙ֬ ͕ ʹै͏ͱ͢Δ q Beta(n + 1,¯ n + 1) I0.5 (n, ¯ n) NOͱఆ YESͱఆ ਖ਼ղ͕YESͳΒޡఆͷ֬I0.5 (n, ¯ n) ਖ਼ղ͕NO ͳΒޡఆͷ֬ 1 − I0.5 (n, ¯ n) ͷͱ͖߹ͤՃ min{I0.5 (n, ¯ n),1 − I0.5 (n, ¯ n)} > ϵ -
/35 ଟܾͷ͑Λਖ਼ղͱݟͳͯ͠ ֤ࣗͷਖ਼֬ͱͦͷ৴པ۠ؒΛਪఆ ৴པ۠ؒͷ্ݶ͕ߴ͍ճऀɹɹɹΛ࠾༻ ˔ ৴པੑͷߴ͍ճऀʹ͍߹ΘͤΔ͜ͱͰ༧ࢉΛޮར༻͍ͨ͠ ˔ ୭͕৴པͰ͖Δ͔ࣄલʹΘ͔Βͳ͍ͷͰ
ଟόϯσΟοτʹ͓͚Δ6$#ઓུΛར༻ͯ͠ ৴པͰ͖Δճऀͷ߹ͤʢ׆༻ʣͱಉ࣌ʹ৴པͰ͖ΔճऀΛ୳ࡧ අ༻ͷޮԽ *&5ISFTI৴པͰ͖ΔճऀΛ୳ࡧɾ׆༻ 17 P. Donmez, J. G. Carbonell, and J. Schneider: E ff i ciently learning the accuracy of labeling sources for selective sampling. KDD 2009. ਖ਼֬ ਖ਼͕֬ߴ͍ˠ׆༻ ୳ࡧ͕ෆेɹˠ୳ࡧ { -
/35 ܈ऺ͔Βͷֶश ܈ऺͷճ͔ΒػցֶशϞσϧΛֶश 19 ௨ৗͷֶश ܈ऺ͕࡞ͨ͠ ܇࿅σʔλ͔Βͷֶश ܈ऺ͔Βͷֶश adelie
chinstrap chinstrap chinstrap gentoo adelie gentoo adelie chinstrap gentoo gentoo gentoo adelie gentoo chinstrap ਖ਼ղ ֶश adelie gentoo chinstrap ֶश ਖ਼ղΛਪఆ adelie chinstrap chinstrap chinstrap gentoo adelie gentoo adelie chinstrap gentoo gentoo gentoo ֶश Ϟσϧ -
/35 ܈ऺ͔Βͷֶश ճϞσϧ͔Βਪఆͨ͠ਖ਼ղΛར༻ͯ͠ྨϞσϧΛֶश 20 V. C. Raykar, S. Yu, L.
H. Zhao, G. H. Valadez, C. Florin, L. Bogoni, and L. Moy: Learning from crowds. Journal of Machine Learning Research, 2010. ྨϞσϧ fw (xi ) = Pr [yi = 1 ∣ xi] = 1 1 + exp(−w⊤xi ) ྨϞσϧͷֶशखॱʢऩଋ͢Δ·Ͱ܁Γฦ͢ʣ &TUFQ ճऀࠞಉߦྻΛݻఆͯ͠ਖ਼ղΛਪఆʢ%BXJE4LFOFͱಉ༷ʣ .TUFQ ਪఆͨ͠ਖ਼ղΛ༻͍ͯճऀࠞಉߦྻͱྨϞσϧΛߋ৽ ௨ৗͷ܇࿅σʔλ {(xi , yi )} {(xi , yi1 , yi2 , …)} ܈ऺ͔Βͷֶशͷ܇࿅σʔλ ճϞσϧ Pr[yij ∣ yi = 1] = αyij j (1 − αj )(1−yij ) Pr[yij ∣ yi = 0] = β(1−yij ) j (1 − βj )yij -
/35 ܈ऺ͔Βͷֶश "HH/FUճϞσϧ͔Βਪఆͨ͠ਖ਼ղΛར༻ͯ͠ਂֶशϞσϧΛֶश 21 S. Albarqouni, C. Baur, F. Achilles,
V. Belagiannis, S. Demirci, and N. Navab: AggNet: Deep learning from crowds for mitosis detection in breast cancer histology images. IEEE Transactions on Medical Imaging, 2016. ਂֶशʹΑΔྨϞσϧ ճϞσϧ &TUFQͰਪఆͨ͠ਖ਼ղЖΛ ༻͍ͯྨϞσϧΛߋ৽ ALBARQOUNI et al.: AGGNET: DEEP LEARNING FROM CROWDS FOR MITOSIS DET Fig. 2. AggNet architecture: The same CNN architecture is used for different w e t b a g f m -
/35 ܈ऺ͔Βͷֶश ڞ௨ɾݸผͷࠞಉߦྻΛ༻ҙͯ͠ਪఆΛޮԽ 22 Z. Chu, J. Ma, and H.
Wang: Learning from crowds by modeling common confusions. AAAI 2021. model parameter estimation. We define cross-entropy loss on the observed annotations and use error back-propagation to update the classifier’s output and the network parameters simultaneously. e 1:R xi Classifier Aux.Net Wa 1:R i 1:R i (1 1:R i ) + = f i Wgf i W1:Rf i h1:R i input parallel noise adaptation layers backbone model predicted anno. dist. where (W terms fo moid fun fer the p the mag or small we norm before c Based modelin the netw ing the feature v ing the c and pred ڞ௨ࠞಉߦྻ ݸผࠞಉߦྻ ࣮ࡍͷճͱ ൺֱ͠ଛࣦΛܭࢉ ྨϞσϧ ೖྗ ճऀಛ ڞ௨ͷࠞಉߦྻΛ͏֬ ิॿϞσϧ -
/35 ೳಈֶश ֶशʹ༗༻ͳαϯϓϧΛબͼϥϕϧ͚ճΛݮΒ͢ 23 ֶशʹ༗༻ͳαϯϓϧΛબΜͰϥϕϧ͚ ˠ৽͍͠ϥϕϧͰྨϞσϧΛߋ৽ adelie ༗༻ͳ
αϯϓϧΛબ ϞσϧΛߋ৽ ϥϕϧ͚ ೳಈֶश "DUJWFMFBSOJOH ༗༻ੑͷࢦඪ ༧ଌΛ༻͍ͯෆ࣮֬ੑΛଌΔ ʢΤϯτϩϐʔΛར༻ʣ adelie chinstrap gentoo 0 0.25 0.5 0.75 1 adelie chinstrap gentoo 0 0.25 0.5 0.75 1 ༧ଌ ༧ଌ͕࣮֬ ༧ଌ͕ෆ࣮֬ -
/35 ܈ऺ͔Βͷೳಈֶश ༗༻ͳαϯϓϧʹ৴པੑͷߴ͍ճऀΛׂΓͯΔ 24 Y. Yan, R. Rosales, G. Fung,
and J. G. Dy: Active learning from crowds. ICML 2011. fw (x) = Pr [y = 1 ∣ x] = 1 1 + exp(−w⊤x) ྨϞσϧ ηwj (x) = 1 1 + exp(−w⊤ j x) ճϞσϧʢਖ਼֬ʣ ೳಈֶशͷखॱ ྨϞσϧʹΑΔ༧ଌ͕࠷ෆ࣮֬ͳαϯϓϧ ΛબͿ ਖ਼֬ ͕࠷େͷճऀΛબͼճΛऔಘ ಘΒΕͨճΛ༻͍ͯྨϞσϧͱճϞσϧ Λߋ৽ x* ηwj (x*) w, {wj } -
/35 ܈ऺ͔Βͷೳಈֶश ਂֶशʹ༗༻ͳαϯϓϧʹ৴པੑͷߴ͍ճऀΛׂΓͯΔ 25 J. Yang, T. Drake, A. Damianou,
and Y. Maarek: Leveraging crowdsourcing data for deep active Learning an application: Learning intents in Alexa. WWW 2018. ϞϯςΧϧϩυϩοϓΞτʹΑΔෆ࣮֬ੑධՁ ˔ ਂֶशωοτϫʔΫͷҰ෦ΛϥϯμϜʹ ܽམͤͨ͞ͷΛෳ༻ҙ ˔ ༧ଌͷฏۉΛ༻͍ͯෆ࣮֬ੑΛධՁ ճϞσϧʢਖ਼֬ʣ ηwj (x) = 1 1 + exp(−w⊤ j Fx) , wj ∈ Rk, F ∈ Rk×d, x ∈ Rd Λ࣍ݩʹຒΊࠐΉ x ˞ೳಈֶशͷखॱલεϥΠυͷख๏ͱಉ༷ -
/35 ಛநग़ ܭࢉػͰଊ͑ΒΕͳ͍ಛΛਓ͕ؒநग़ 27 S. Branson, C. Wah, F. Schro
ff , B. Babenko, P. Welinder, P. Perona, S. Belongie: Visual recognition with humans in the loop. ECCV 2010. Visual Recognition with Humans in the Loop 3 mputer vision is helpful Computer vision is not helpful mputer vision is helpful Computer vision is not helpful The bird is a Black‐footed Albatross Is the belly white? yes Are the eyes white? yes Th bi d i Is the beak cone‐shaped? yes Is the upper‐tail brown? yes Is the breast solid colored? no Is the breast striped? yes I h h hi ? The bird is a Parakeet Auklet Is the throat white? yes The bird is a Henslow’s Sparrow 2. Examples of the visual 20 questions game on the 200 class Bird dataset. an responses (shown in red) to questions posed by the computer (shown in blue) sed to drive up recognition accuracy. In the left image, computer vision algorithms uess the bird species correctly without any user interaction. In the middle image, uter vision reduces the number of questions to 2. In the right image, computer n provides little help. 2ෲ͕ന͍ʁ 2͕ന͍ʁ 2͕ͪ͘͠ίʔϯܕʁ YES YES NO ಛ̍ ಛ ಛ 1 1 0 ػցֶशͰಛઃܭ͕ॏཁ ˠࣄલʹ࣭Λ༻ҙ͓͖ͯ͠܈ऺʹճͤͯ͞ಛΛநग़ -
/35 ಛઃܭ "EB'MPDLಛநग़͚ͩͰͳ͘ಛઃܭਓ͕࣮ؒࢪ 28 R. Takahama, Y. Baba, N. Shimizu,
S. Fujita, and H. Kashima: AdaFlock: Adaptive feature discovery for human-in-the-loop predictive modeling. AAAI 2018. B ਖ਼ྫɾෛྫΛ ਓؒʹఏࣔ C ਖ਼ྫɾෛྫΛ ۠ผ͢Δ࣭จΛ ਓ͕ؒੜ ʢಛઃܭʣ D ਓ͕࣭ؒʹճ ʢಛநग़ʣ E ྨثΛߋ৽͠ ಛઃܭʹ͏ αϯϓϧΛબ ྫɿϞωͱγεϨʔͷֆͷྨ -
/35 Ϟσϧݕࠪ ෆదͳಛΛਓؒʹআڈͤͯ͞ϞσϧΛվળ 29 M. T. Ribeiro, S. Singh, and
C. Guestrin: "Why should I trust you?": Explaining the predictions of any classi fi er. KDD 2016. https://drive.google.com/ fi le/d/0ByblrZgHugfYZ0ZCSWNPWFNONEU/view ྫɿफڭؔ࿈ϝʔϧͷఆ Ϟσϧͷ அࠜڌΛఏࣔ )PTU 1PTUJOH //51ͷ ແؔͳ୯ޠΛॏࢹ͢Δͷෆద -
/35 ղऍੑ ਓؒʹΑΔஅ࣌ؒΛߟྀͯ͠ղऍੑͷߴ͍ϞσϧΛൃݟ 30 I. Lage, A. Ross, S. J.
Gershman, B. Kim, and F. Doshi-Velez: Human-in-the-loop interpretability prior. NeurIPS 2018. ਫ਼ͷߴ͍ϞσϧΛީิͱͯ͠ྻڍ ਓؒʹධՁͤ͞ΔϞσϧ Λબ M ਓ͕ؒϞσϧ ͷղऍੑ ΛධՁ M p(M) ࠷ྑͷϞσϧΛܾఆ p(M) ≈ 1 N ∑ x 𝖧 𝖨𝖲 (x, M) 𝖧𝖨𝖲 (x, M) = max{0, 𝗆𝖺 𝗑 𝖱𝖳 − 𝗆𝖾𝖺 𝗇𝖱𝖳 (x, M)} ฏۉஅ࣌ؒ (a) An example of our interface with a tree trained on Ϟσϧͷઆ໌ʹج͍ͮͯਓؒʹ༧ଌΛͤ͞Δ ˠॴཁ͕͍࣌ؒ΄Ͳղऍੑͷߴ͍Ϟσϧ -
/35 ఢରతੜωοτϫʔΫʢ("/ʣ )VNBO("/ਓؒΛ("/ͷࣝผثʹͯ͠ਓؒͷײੑΛऔΓࠐΉ 31 K. Fujii, Y. Saito, S. Takamichi,
Y. Baba, and H. Saruwatari: HumanGAN: Generative adversarial network with human-based discriminator and its evaluation in speech perception modeling. ICASSP 2020. Pertur- bation Crowd- workers Backpropagation using approximated Worker’s answer to “to what degree are two samples different?” [times] 2 Fig. 2. Generator training procedure of proposed HumanGAN. Crowdworkers state a perceptual difference (i.e., difference of pos- terior probabilities) of two perturbed samples. Answer and perturba- tion are used for backpropagation to train generator. University of Tokyo, Japan. ty of Tsukuba, Japan. minator Natu- ral -based ata Distr. of human perception of basic GAN and proposed HumanGAN. rator by fooling DNN-based discriminator scriminator), and generator finally represents n. In comparison, HumanGAN trains gener- ࣝผثͱͯ͠ਓؒΛ༻͍Δ ਓؒΛ᱐͢ੜثΛֶश͢Δ͜ͱͰ ਓؒͷײੑΛऔΓࠐΉ -
/35 ਓؒͱܭࢉػͷڠௐϞσϧ ਓؒͷ͍߹ΘͤΛؚΊͨϞσϧΛֶश 32 B. Wilder, E. Horvitz, and E.
Kamar: Learning to complement humans. IJCAI 2020. ਓؒͱܭࢉػͷڠௐϞσϧ ೖྗ x ࣭ثq(x) ਓؒʹ͍߹Θͤͳ͍ ਓؒʹ͍߹ΘͤΔ h ྨث ̂ y = f(x) ̂ y = f(x, h) ڠௐϞσϧͷֶश ܇࿅σʔλ Λ༻͍ͯ ͕࠷େʹͳΔΑ͏ʹ ྨثͱ࣭ثΛֶश {(x, y, h)} q(x){u(y, f(x, h)) − c} + (1 − q(x))u(y, f(x)) ༧ଌͷਖ਼͠͞ͷޮ༻ؔ Λग़ྗ Λग़ྗ අ༻ -
/35 )VNBOJOUIF-PPQػցֶशɿਓ͕ؒࢀՃ͢Δػցֶशϓϩηε 34 1BSU ܈ऺʹΑΔ ܇࿅σʔλ࡞ 1BSU
܈ऺ͔Βͷֶश 1BSU ػցֶशͷ༷ʑͳ໘Ͱͷ ܈ऺ׆༻ Ϟσϧ adelie gentoo chinstrap ܇࿅σʔλ Ϟσϧ ߹ͤ ڭࢣϥϕϧ ͷఏڙ ߹ͤ ༷ʑͳݟ ͷఏڙ ڭࢣϥϕϧ ͷఏڙ ˔ ਅ࣮ൃݟ ˔ අ༻ͷޮԽ ˔ ܈ऺ͔Βͷਂֶश ˔ ೳಈֶश ˔ ಛநग़ɾઃܭ ˔ ղऍੑ্ ˔ ਓؒͱܭࢉػͷڠௐϞσϧ ˔ FUD -
/35 35 ࣛౡٱ࢚ খࢁ૱ അઇ೫ ώϡʔϚϯίϯϐϡςʔγϣϯͱΫϥιʔγϯά ߨஊࣾ
Robert (Munro) Monarch. Human-in-the-Loop Machine Learning: Active learning and annotation for human-centered AI. Manning Publications, 2021. ࢀߟจݙ