$30 off During Our Annual Pro Sale. View Details »

Coco-Nut: 自由記述文による声質制御に向けた多話者音声・声質自由記述ペアデータセット

Aya Watanabe
September 28, 2023

Coco-Nut: 自由記述文による声質制御に向けた多話者音声・声質自由記述ペアデータセット

日本音響学会2023秋季大会口頭発表資料

Aya Watanabe

September 28, 2023
Tweet

Other Decks in Research

Transcript

  1. $PDP/VUࣗ༝هड़จʹΑΔ੠੍࣭ޚ
    ʹ޲͚ͨଟ࿩ऀԻ੠ɾ੠࣭ࣗ༝هड़
    ϖΞσʔληοτ
    ೔ຊԻڹֶձୈճʢ೥ळقʣݚڀൃදձɹ
    ˑ౉ᬑဌᒀ ߴಓ৻೭հ ᜊ౻༎थ ਏ℄ହ Ԑ౉༸ ౦େӃɾ৘ใཧ޻

    View Slide

  2. ࣗ༝هड़ʹΑΔ੠੍࣭ޚͷՄೳੑ
    w ࣗ༝౓͕ߴ͍ɼ௚ײతͳૢ࡞ΛՄೳʹ
    w େن໛ͰΦʔϓϯͳίʔύε͕ඞཁ
    Ի੠੠࣭දݱจϖΞίʔύεͷ࡞੒
    w ଟ࿩ऀԻ੠σʔλͷࣗಈऩूɾॻ͖ى͜͠
    w Ի੠΁ͷ੠࣭දݱจ෇༩ϑϩʔཱ֬
    w ໿ηάϝϯτɼ࣌ؒڧͷԻ੠ͱ੠࣭දݱจΛؚΉίʔύε͕׬੒
    ۙ೔ެ։༧ఆ
    ֓ཁ

    ςΩετ
    Ի੠߹੒
    Ϟσϧ
    ੠੍࣭ޚ
    Ϟσϧ
    ಡΈ্͛จ
    ੠࣭දݱจ
    ࣗ༝هड़ʹΑΔ
    ੠੍࣭ޚγεςϜ
    ྫେਓͼͨ௿͍੠Ͱམͪண͍ͯ࿩͢ɼͳͲ

    View Slide

  3. ֓ཁ
    "436൛ POBS9JW

    w ରরֶशϞσϧʹΑΔఆྔධՁ͋Γ

    View Slide

  4. ؔ࿈ݚڀ
    ଟ࿩ऀɾଟελΠϧςΩετԻ੠߹੒ʹ͓͚Δ੠੍࣭ޚ
    w ࿩ऀબ୒<)PKPFUBM >ɼύϥϝʔλʢײ৘<-JVFUBM >౳ʣௐ੔௿ࣗ༝౓
    w ࣗ༝هड़ʹΑΔ੠੍࣭ޚ<(VPFUBM :BOHFUBM >
    w Ի੠߹੒༻ʹऩ࿥ͨ͠গ਺࿩ऀίʔύε͕ϕʔεɾඇެ։
    ˠଟ༷ͳ࿩ऀɾ୭Ͱ΋ݚڀʹ࢖͑Δίʔύεʢ͓Αͼߏஙख๏ʣ͕ඞཁ
    5FYUUP*NBHFʹ͓͚Δଟ༷ɾެ։ίʔύεͷߩݙ
    w .4$0$0<-JOFUBM >
    w ਓखΞϊςʔγϣϯ͞ΕͨΩϟϓγϣϯը૾ϖΞίʔύε
    w ʮPCKFDUDBUFHPSJFTʯͰʮ,ʯͷը૾ʹΩϟϓγϣϯ෇༩
    w ςΩετը૾ੜ੒ʹ͓͚Δར༻ FH %"--&<3BNFTIFUBM >

    ˠଟ༷σʔλʹਓखΞϊςʔγϣϯ͞Εͨίʔύεͷެ։͕ݚڀΛଅਐ

    View Slide

  5. ίʔύεߏ੒ཁૉ
    Ի੠ॻ͖ى͜͠จʢ಺༰ϓϩϯϓτʣ੠࣭දݱจʢ੠࣭ϓϩϯϓτʣ

    த೥ͷஉੑ͕ɺ
    Θ͔Γ΍͍͢ޱௐͰ
    ஸೡʹ࿩Λ͍ͯ͠Δɻ
    ืू͍ͤͯͨͩ͘͜͞ͱͱͳ
    Γ·ͨ͠ɻࠓճ͸:PV5VCFͳ
    ͲͷϝσΟΞ࿐ग़Մೳͳʜʜ

    View Slide

  6. ίʔύεߏஙख๏

    View Slide

  7. ίʔύεߏங֓ཁ

    σʔλऩू
    ಈըϑΟϧλ
    ඼࣭อূ
    ਓखΞϊςʔγϣϯ
    w 8FC্ͷಈըΛࣗಈऩूʢଟ਺ͷ࿩ऀͷԻ੠Λऩूʣ
    w ੠ʹؔ܎͢ΔΩʔϫʔυͰಈըݕࡧ
    w Ի੠Λؚ·ͳ͍ɾԻ੠ΛؚΉ͕ಛ௃ͷബ͍੠ͷಈըΛআ֎
    w ੠ʹؔ܎͢Δࢹௌऀίϝϯτͷ਺͕ج४
    w ࣗಈऩूͨ͠Ի੠ΛԻڹతɾݴޠతʹ඼࣭อূ͠ɼίʔύε
    ͷ࣭Λ୲อ
    w ऩूԻ੠ʹ੠࣭දݱจΛ෇༩
    w Ϋϥ΢υιʔγϯάΛ௨ͨ͡ෆಛఆଟ਺͔Βͷ੠࣭දݱจऩ
    ूϑϩʔཱ֬

    View Slide

  8. ίʔύεߏஙσʔλऩूɾಈըϑΟϧλ
    ੠ʹؔ܎͢ΔϑϨʔζΛ༻͍ͨ:PV5VCFݕࡧ
    w 8JLJQFEJB಺੠ؔ܎ΧςΰϦهࣄΛ࢖༻ͨ͠ϑϨʔζऩू<౉ᬑFUBM >
    ੠ؔ܎ίϝϯτࣝผ<౉ᬑFUBM >ʹجͮ͘੠ؔ܎ಈըநग़
    w ੠ʹؔ܎͢Δίϝϯτ͕෇͘ʢݴٴͨ͘͠ͳΔಛ௃ͷ͋ΔʣԻ੠ΛؚΉ
    w Ұఆ਺Ҏ্ͷʮ੠ʹؔ܎͢Δʯࢹௌऀίϝϯτ͕෇͍ͨಈըͷΈ࢖༻
    w ʮ੠ʹؔ܎͢Δʯίϝϯτͷࣗಈࣝผ<౉ᬑFUBM >

    ˓˓
    8JLJQFEJB
    $BUFHPSZ
    🔍
    ˚˚ ˑˑ


    View Slide

  9. ίʔύεߏங඼࣭อূ
    Ի੠඼࣭อূԻͱͯ͠ͷ඼࣭อূ
    w Ի੠۠ؒηάϝϯτݕग़ɾ෼ׂɼԻ੠ڧௐɼ௿Ի࣭Ի੠আڈɼ୹͗͢Δ௕͗͢ΔԻ੠আ
    ڈɼෳ਺࿩ऀԻ੠আڈɼՎ੠আڈɼ੠࣭ଟ༷ੑ࠷େԽʹΑΔൃ࿩બ୒
    w ੠࣭ଟ༷ੑ࠷େԽʹΑΔൃ࿩બ୒YWFDUPS<4OZEFSFUBM >ʹΑΔ੠࣭ϕΫτϧԽ
    ʹج͖ͮɼ΢Υʔυ๏ʹجͮ͘֊૚తΫϥελϦϯά<8BSE+SFUBM >ͷ͋ͱ֤Ϋϥ
    ελ͔ΒϥϯμϜʹൃ࿩Λબ୒
    ಺༰඼࣭อূൃ࿩಺༰ͷ඼࣭อূ
    w ॻ͖ىࣗ͜͠ಈԻ੠ೝࣝखಈमਖ਼
    w ඇର৅ݴޠԻ੠আڈɼෆద੾දݱʢ/4'8ʣআڈɼϊϯόʔόϧԻ੠আڈ
    Ի੠ɾ಺༰྆ଆ໘͔Βͷ඼࣭อূʹΑΓɼࣗಈऩूͨ͠Ի੠ͷ࣭Λอূ

    View Slide

  10. ίʔύεߏஙਓखΞϊςʔγϣϯ
    Ϋϥ΢υιʔγϯάʹΑΔ੠࣭දݱจऩू

    จࣈ਺Լݶఏࣔ
    ྫจఏࣔ

    View Slide

  11. ίʔύε෼ੳ

    View Slide

  12. ίʔύε෼ੳσʔλऩू֓ཁ

    ݕࡧʹΑΔ֫ಘಈը਺ ໿ສ<౉ᬑFUBM >
    ಈըϑΟϧλޙͷಈը਺
    ηάϝϯτ਺
    ߹ܭܧଓ௕ ໿࣌ؒ൒
    ੠࣭දݱจ෇༩਺Ի੠ ֶशηοτݕূɼධՁηοτ
    ܧଓ௕ൣғ ඵ

    View Slide

  13. ίʔύε෼ੳੑผͷ෼෍

    0 20 40 60
    Number of utterances [%]
    Not indicated
    Non-binary
    Female
    Male
    Male
    Female
    Non-binary
    Not indicated
    ੠࣭දݱϓϩϯϓτ಺Ͱͷʮஉʯʮঁʯͷදهʹج͖ͮ෼ྨɾूܭ
    ྆ํͷදه͕͋Ε͹/POCJOBSZɼ྆ํͳ͚Ε͹/PUJOEJDBUFE
    ੑผ͝ͱͷԻ੠ηάϝϯτ਺ ੑผ͝ͱͷYWFDUPS෼෍ʢU4/&ͰՄࢹԽʣ
    େ෦෼͕෼ྨՄೳ
    ʢҰ෦/POCJOBSZ౳ʣ
    உঁ͸Ϋϥελ͋Γ
    ʢ/POCJOBSZ౳͸Ϋϥελແ͘෼෍ʣ

    View Slide

  14. ίʔύε෼ੳ੠࣭දݱจྫ

    ୅͘Β͍ͷஉੑͷ੠ɻΏͬ͘ΓͱԺ΍͔ͳ࿩͠ํͰͨ͠ɻۤ೰ʹຬͪ
    ͨɺ͚ͩΔͦ͏ͳ੠Ͱͨ͠ɻ
    ݩؾͳஉੑ͕໌Δ͍੠Ͱɺςϯγϣϯߴ͘ൃදΛ͢ΔΑ͏ʹ஻͍ͬͯΔɻ
    ए͍ঁੑ͕ɺ཈༲ͷ͋Δ੠ͰɺΏͬ͘Γͱ஻͍ͬͯΔɻ

    View Slide

  15. ίʔύε෼ੳऩू੠࣭
    プ
    ϩϯ
    プ
    τͷසग़୯ޠ෼ੳ

    ୯ޠͷग़ݱස౓ΛϫʔυΫϥ΢υͰਤࣔʢେ͖͘ॻ͔Εͨ୯ޠ΄Ͳසग़ʣ
    සग़୯ޠ͸සग़ଐੑɾೝࣝ͞Ε΍͍͢ཁૉΛࣔ͢
    ໊ࢺ ܗ༰ࢺ ಈࢺ
    w :PV5VCFυϝΠϯʹ͸ૣޱͷ࿩ऀ͕ଟ͍
    w ࿩଎͸੠࣭ͷதͰ΋ೝࣝ͠΍͍͢ཁૉͰ͋Δ

    View Slide

  16. ·ͱΊ
    ࣗ༝هड़Ͱ੠࣭ίϯτϩʔϧՄೳͳςΩετԻ੠߹੒ͷͨΊͷίʔύε࡞੒
    w Ի੠੠࣭ࣗ༝هड़จͷର
    w ެ։͞Ε͓ͯΓɼଟ༷ͳ੠࣭ͷ࿩ऀΛؚΈɼਓखʹΑΔʢࣗಈऩूͰ͸ͳ͘ɼ࣭͕ߴ͍ʣ
    ࣗ༝هड़͕෇༩͞Εͨίʔύε
    :PV5VCFͰͷԻ੠σʔλऩूࣗಈɾਓख඼࣭อূ
    w ੠ʹؔ܎͢Δಈըͷબผ
    w Ի੠ɾ಺༰྆ଆ໘͔Βͷ඼࣭อূ
    w ਓखʹΑΔࣗ༝هड़෇༩
    w ࠓޙɼຊίʔύεΛ༻͍ɼ੠࣭දݱจͰ੠੍࣭ޚՄೳͳςΩετԻ੠߹੒ϞσϧͷߏஙΛ
    ߦ͏༧ఆ

    View Slide

  17. "QQFOEJYಈըϑΟϧλͷઃఆ

    ੠ؔ܎ίϝϯτ਺ᮢ஋ ݅
    ΩʔϫʔυϚονϯά
    ࢖༻୯ޠ
    ੠ɼ
    ボ
    ΠεɼԻɼௌɼฉɼՎ
    λΠτϧซ༻ ͋Γ
    #&35ϕʔεࣝผػ࢖༻
    ୯ޠ૊Έ߹Θͤ
    ʮϘΠεɾฉʯʮԻʯʮ੠ʯʮԻɾฉʯ
    ʮ੠ɾϘΠεɾԻʯʮϘΠεɾԻʯʮ੠ɾԻɾՎʯ

    View Slide

  18. "QQFOEJYԻ੠඼࣭อূ
    w Ի੠۠ؒݕग़ɾηάϝϯτநग़
    w Ի੠ڧௐ
    w ௿Ի࣭Ի੠আڈ
    w ܧଓ௕ϑΟϧλ
    w ௿ԻྔԻ੠আڈ
    w ෳ਺࿩ऀԻ੠আڈ
    w Վ੠আڈ
    w ੠࣭ଟ༷ੑ࠷େԽʹΑΔൃ࿩બ୒

    JOB4QFFDI4FHNFOUFSʹΑΔ7"%
    %FNVDT
    /*42"ʹΑΔࣗಈ.04ධՁ
    dඵ
    ਖ਼نԽޙͷԻྔEC
    ਓख඼࣭อূ
    ਓख඼࣭อূ
    TDJLJUMFBSOΛ࢖༻

    View Slide

  19. "QQFOEJY಺༰඼࣭อূ
    w ඇର৅ݴޠԻ੠আڈ
    w ෆద੾දݱʢ/4'8ʣআڈ
    w ϊϯόʔόϧԻ੠আڈ

    8IJTQFSʹΑΔ࢖༻ݴޠਪఆ
    ਓख඼࣭อূ
    /4'8ࣙॻʹΑΔΩʔϫʔυݕࡧ
    ਓख඼࣭อূ
    #&35ʹΑΔ.-.είΞධՁ
    ʢϊϯόʔόϧԻ੠͸.-.είΞ͕
    ۃ୺ʹߴ͘ͳΔ܏޲͕͋Δ FH l͋
    ͋͋͋z
    ʣ

    View Slide

  20. "QQFOEJYਓख඼࣭อূͷͨΊͷΫϥ΢υιʔγϯά6*

    डௌධՁ࣮ݧ
    Ի੠Λ࠷ޙ·Ͱ࠶ੜ͠ɺ֤Ի੠͕ҎԼͷ৚݅Λຬ͍ͨͯ͠Δ͔ɺͦΕͧΕνΣοΫ͍ͯͩ͘͠͞ɻ
    ஻͍ͬͯΔ಺༰͕ೝࣝͰ͖ΔͳΒ͹ػցͷ੠΋ൃ࿩ऀͱͯ͠ѻ͍ͬͯͩ͘͞ɻ
    1. ެংྑଏʹ൓͍ͯ͠Δ
    2. ೔ຊޠ͚͕ͩ࢖ΘΕ͍ͯΔʢ೔ຊޠจதʹ୯ޠ୯ҐͰ֎ࠃޠ͕ݱΕΔ৔߹͸೔ຊޠ͚ͩͱΈͳ͢ʣ
    3. ൃ࿩ऀ͸1ਓ͚ͩʢਓͷ੠͕ೖͬͨBGMɺ૬Ṁ΋ͳ͍ʣ
    4. Վ͕ೖ͍ͬͯΔʢBGMͷϘʔΧϧ΋ؚΉʣ
    Ի੠͝ͱʹશ߲໨ʹ͍ͭͯೖྗ͕ऴΘͬͨΒ֬ఆϘλϯΛԡ͠ɺ࣍ʹਐΜͰ͍ͩ͘͞ɻ
    ͳ͓ɺಉ͡Ի੠͕࿈ଓͯ͠ྲྀΕΔ͜ͱ΋͋Γ·͢ɻ
    ࡞ۀ࣌ؒ͸3෼ఔ౓Λ༧ఆ͍ͯ͠·͢ɻ
    ެংྑଏʹ൓͍ͯ͠Δʁ
    ͸͍ ͍͍͑
    ೔ຊޠ͚͕ͩ࢖ΘΕ͍ͯΔʁʢ೔ຊޠจதʹ୯ޠ୯ҐͰ֎ࠃޠ͕ݱΕΔ৔߹͸೔ຊޠ͚ͩͱΈͳ͢ʣ
    ͸͍ ͍͍͑
    ൃ࿩ऀ͸1ਓ͚ͩʁʢਓͷ੠͕ೖͬͨBGMɺ૬Ṁ΋ͳ͍ʣ
    ͸͍ ͍͍͑
    Վ͕ೖ͍ͬͯΔʁʢBGMͷϘʔΧϧ΋ؚΉʣ
    ͸͍ ͍͍͑
    ֬ఆ
    -0:02

    View Slide

  21. "QQFOEJYਓख඼࣭อূͷͨΊͷΫϥ΢υιʔγϯάઃఆ

    ϓϥοτϑΥʔϜ
    -BODFST
    IUUQTXXXMBODFSTKQ

    ධՁ͢ΔԻ੠ϑΝΠϧ਺
    ϫʔΧʔ

    ༧૝ॴཁ࣌ؒ
    ϫʔΧʔ

    ใु
    ϫʔΧʔ
    =

    View Slide

  22. "QQFOEJYਓखΞϊςʔγϣϯͷͨΊͷΫϥ΢υιʔγϯάઃఆ

    ϓϥοτϑΥʔϜ
    -BODFST
    IUUQTXXXMBODFSTKQ

    ධՁ͢ΔԻ੠ϑΝΠϧ਺
    ϫʔΧʔ

    ༧૝ॴཁ࣌ؒ
    ϫʔΧʔ

    ใु
    ϫʔΧʔ
    =

    View Slide

  23. "QQFOEJYॻ͖ى͜͠จͷਓखमਖ਼ྫ
    w ࣗಈॻ͖ى͜͠͸8IJTQFSͰߦ͏
    w 5ZQPɼ8IJTQFSϞσϧߏ଄༝དྷͷϧʔϓɼ੾Ε͍ͯΔ෦෼ͷࣗಈ༧ଌ͞Εͯ
    ͠·ͬͨՕॴͳͲΛਓखमਖ਼

    άϦʔϯं͸଍ݩͷॏྔ৐ंޱ߸ंͱ
    ߸ंͰ͓଴͍ͪͩ͘͞
    άϦʔϯं͸଍ݩͷ྆৐ंޱ߸ं
    ͱ߸ंͰ͓଴͍ͪͩ͘͞

    View Slide

  24. "QQFOEJYಈըΧςΰϦͷ෼෍

    0 10 20 30 40
    Number of utterances [%]
    Sports
    Science & Technology
    Nonprofits & Activism
    Travel & Events
    News & Politics
    Comedy
    Howto & Style
    Music
    Autos & Vehicles
    Film & Animation
    People & Blogs
    Gaming
    Education
    Entertainment Entertainment
    Education
    Gaming
    People & Blogs
    Film & Animation
    Autos & Vehicles
    Music
    Howto & Style
    Comedy
    News & Politics
    Travel & Events
    Nonprofits & Activism
    Science & Technology
    Sports
    ಈըΧςΰϦ͝ͱͷԻ੠ηάϝϯτ਺ ಈըΧςΰϦ͝ͱͷYWFDUPS෼෍
    :PV5VCFͰઃఆ͞Ε͍ͯΔಈըΧςΰϦʹج͖ͮ෼ྨ͠ूܭ
    ΧςΰϦ͝ͱͷ੠࣭ͷಛ௃ΛYWFDUPSʹΑͬͯఆྔԽ͠ɼU4/&ͰՄࢹԽ
    ໿
    ΧςΰϦ ΫϥελʢΧςΰϦʹಛ௃తͳ੠࣭ʣ

    View Slide