Upgrade to Pro — share decks privately, control downloads, hide ads and more …

나만의 프로필 이미지를 만드는 Personlized T2I 모델 개발기

kakao
November 01, 2024

나만의 프로필 이미지를 만드는 Personlized T2I 모델 개발기

#Kanana #MultiModal #T2I #이미지생성

카카오의 프로필 이미지 생성 기술이 어떻게 탄생했는지 Personalized Text to Image 연구 과정을 소개합니다. 실제 촬영 사진과 같은 고품질 인물 이미지 생성을 목표로 Diffusion 모델의 학습 과정과 평가 방법을 공유하고, 이 과정에서 마주한 도전 과제와 한계점, 향후 목표를 알아봅니다.

발표자 : ted.chl
카카오의 카나나 알파 조직에서 생성형 모델을 이용해 AI 프로필을 연구하는 테드입니다. 이미지, 비디오 생성에 초점을 맞춰 다양한 응용 기술을 연구하고 개발하고 있습니다. 세상을 변화시키는 AI 모델과 함께 다양한 기술에 관심이 많습니다.

kakao

November 01, 2024
Tweet

More Decks by kakao

Other Decks in Programming

Transcript

  1. 5FYU - UP - *NBHF%JGGVTJPOݽ؛ ੉ۆ  95 9U 9U

    9 pθ (xt−1 |xt ) q(xt |xt−1 ) q(xt−1 |xt )  )P +POBUIBOFUBMt%FOPJTJOH%JGGVTJPO1SPCBCJMJTUJD.PEFMTu /FVS*14 
  2. 5FYU - UP - *NBHF%JGGVTJPOݽ؛੉ۆ  %JGGVTJPO.PEFM 5FYU&ODPEFS "QIPUPPGDBUj 95

    9 ࠄ੗ܐীחࢤࢿഋ"*Ѿҗޛ੉ੌࠗನೣغয੓णפ׮ 9U -> 9U
  3. ,PMMBHF1SP fi MFݽ؛੄ҳઑ *OTUBOU#PPUI ӝ߈੄*EFOUJUZ੿ࠁ೟ण t"QIPUPPG QFSTPOu $-*15FYU &ODPEFS $-*1*NBHF

    &ODPEFS 1SPKFDUJPO 1SPKFDUJPO  +JOH4IJ FUBMt*OTUBOU#PPUI1FSTPOBMJ[FE5FYU - UP - *NBHF(FOFSBUJPOXJUIPVU5FTU - 5JNF'JOFUVOJOHu   $IFSUJ .FIEJFUBMt3FQSPEVDJCMFTDBMJOHMBXTGPSDPOUSBTUJWFMBOHVBHF - JNBHFMFBSOJOHu 
  4. ,PMMBHF1SP fi MFݽ؛੄ҳઑ t"QIPUPPG QFSTPOu $-*15FYU &ODPEFS $-*1*NBHF &ODPEFS 1SPKFDUJPO

    1SPKFDUJPO 1BUDIFNCFEEJOH੄೟ण (BUFE4FMG"UUFOUJPO  :VIFOH-J FUBMt(-*(&/0QFO - 4FU(SPVOEFE5FYU - UP - *NBHF(FOFSBUJPOu 
  5. ,PMMBHF1SP fi MFݽ؛੄ؘ੉ఠࣇ ҕѐػੋޛؘ੉ఠࣇ੄ೠ҅ 113L%BUBTFU -"*0/ - 'BDF%BUBTFU  -JBOH

    +JFFUBMt113,"-BSHF - 4DBMF1PSUSBJU1IPUP3FUPVDIJOH%BUBTFUXJUI)VNBO - 3FHJPO.BTLBOE(SPVQ - -FWFM$POTJTUFODZu   ;IFOH :JOHMJOFUBMt(FOFSBMGBDJBMSFQSFTFOUBUJPOMFBSOJOHJOBWJTVBM - MJOHVJTUJDNBOOFSu 
  6. 1SPT $POT ,PMMBHF1SP fi MFݽ؛੄ؘ੉ఠࣇ  ׮নೠఫझ౟೐܁೐౟оನೣػ੉޷૑  ׮নೠੋޛ੉ನೣػ੉޷૑ 

    ࠺Ү੸ए਍ؘ੉ఠࣇഛࠁ  Ѿҗ੉޷૑੄ ࠂࢎࠢৈ֍ӝ ޙઁߊࢤ  ׮নೠҳب߂ನૉ੄੉޷૑ࠗ઒  ,0--"(&1SP fi MFݽ؛੄ࢿמࠗ઒ 4*4*ؘ੉ఠࣇ੄੢ױ੼
  7. ,PMMBHF1SP fi MFݽ؛੄ؘ੉ఠࣇ 4JOHMF*EFOUJUZ.VMUJ*NBHF 4*.* A woman seated in an

    empty classroom with rows of white desks and chairs. She is working on a laptop, smiling gently at the camera.
  8. ,PMMBHF1SP fi MFݽ؛੄ؘ੉ఠࣇ 1SPT $POT  ೠࢎۈী؀ೠ׮নೠпب৬಴੿੄੉޷૑  ೠࢎۈী؀ೠৈ۞੢੄੉޷૑ 

    ,PMMBHF1SP fi MFݽ؛੄ࢿמೱ࢚  ҕѐػؘ੉ఠࣇ੄ࠗ઒  ؘ੉ఠࣇҳ୷੄दр߂࠺ਊ  زੌੋޛী؀ೠ੿೤ࢿ 4*.*ؘ੉ఠࣇ੄੢ױ੼
  9. ,PMMBHF1SP fi MFݽ؛੄೟ण ੉޷૑ࢤࢿߔࠄݽ؛੄ഝਊ  IUUQT DJWJUBJDPNJNBHFT  IUUQT DJWJUBJDPNJNBHFT

    ࠄ੗ܐীחࢤࢿഋ"*Ѿҗޛ੉ੌࠗನೣغয੓णפ׮ Ҋಿ૕੄ )VNBO1PSUSBJU੉޷૑ ࢤࢿ 1MBZHSPVOEW 4UBCMF%JGGVTJPO9-
  10. ,PMMBHF1SP fi MFݽ؛੄೟ण )VNBOQPSUSBJU੉޷૑ܳࢤࢿೡࣻ੓ب۾ߔࠄݽ؛੄'JOF - 5VOJOH ࠄ੗ܐীחࢤࢿഋ"*Ѿҗޛ੉ੌࠗನೣغয੓णפ׮ )( - %10

     4BOHIZFPO/BFUBMt#PPTU:PVS0XO)VNBO*NBHF(FOFSBUJPO.PEFMWJB%JSFDU1SFGFSFODF0QUJNJ[BUJPOXJUI"*'FFECBDLu 
  11. ,PMMBHF1SP fi MFݽ؛੄೟ण ࠄ੗ܐীחࢤࢿഋ"*Ѿҗޛ੉ੌࠗನೣغয੓णפ׮ 8JOOJOH -PTJOH  ݽ؛ӝ߈ಿ૕ಣо  ؀ӏݽؘ੉ఠࣇഛࠁ

     %JGGVTJPO - %10 ੄೟ण  :VWBM,JSTUBJO/BFUBMt1JDL - B - 1JD"O0QFO%BUBTFUPG6TFS1SFGFSFODFTGPS5FYU - UP - *NBHF(FOFSBUJPOu   #SBN8BMMBDFFUBMt%JGGVTJPO.PEFM"MJHONFOU6TJOH%JSFDU1SFGFSFODF0QUJNJ[BUJPOu $713 
  12. ,PMMBHF1SP fi MFݽ؛੄೟ण XP%10 X%10 ࠄ੗ܐীחࢤࢿഋ"*Ѿҗޛ੉ੌࠗನೣغয੓णפ׮ 1SPNQU GSPOUBMQPSUSBJU PGBNBO 

    MPPLJOHGSPOU  MJHIUCSPXORVJGGIBJSTUZMF GSPNT ESFTTFEJO BSFUSPEFOJNKBDLFUBOEBCMBDLTIJSU  WJWJESFECBDLESPQ
  13. ,PMMBHF1SP fi MFݽ؛੄೟ण XP%10 X%10 ࠄ੗ܐীחࢤࢿഋ"*Ѿҗޛ੉ੌࠗನೣغয੓णפ׮ 1SPNQUDFOUFSFEDPNQPTJUJPO  DMPTF -

    VQVQQFSCPEZQPSUSBJU TUSBJHIUQPTUVSF GSPOU - GBDJOH TJUUJOHPOBDPVDI XFBSJOHBSFE TXFBUFSXJUIUFEEZCFBS POJU  HJGUCPYFTBT CBDLHSPVOE BMBSHF$ISJTUNBTUSFFJTQSPNJOFOUMZQPTJUJPOFEBUUIFMFGUFEHF
  14. ,PMMBHF1SP fi MFݽ؛੄೟ण 4*4*4*.*ؘ੉ఠࣇ ,PMMBHF1SP fi MF.PEFM %JGGVTJPO-PTT'VODUJPO  #BDLCPOF

    *%.PEVMF  )P +POBUIBOFUBMt%FOPJTJOH%JGGVTJPO1SPCBCJMJTUJD.PEFMTu /FVS*14 
  15. ,PMMBHF1SP fi MFݽ؛੄೟ण *EFOUJUZ-PTT'VODUJPO   %FOH +JBOLBOHFUBMt"SD'BDF"EEJUJWF"OHVMBS.BSHJO-PTTGPS%FFQ'BDF3FDPHOJUJPOu $713 

     1BSLIJ 0NLBS.FUBM%FFQ'BDF3FDPHOJUJPOu #.7$  #BDLCPOF *%.PEVMF ,PMMBHF1SP fi MF.PEFM 4*4*4*.*ؘ੉ఠࣇ
  16. ੗୓ಣо࠺Ү ೧࢚ب ࢤࢿदр ݽ؛౵ۄ޷ఠࣻ ,PMMBHF/BOP1SP fi MF Y _TFD _রѐ

    ,PMMBHF&TTFODF1SP fi MF Y _TFD _রѐ /7*%*""((16ীࢲஏ੿
  17. ,PMMBHF1SP fi MFݽ؛੄Ѿҗ ,PMMBHF&TTFODF1SP fi MF ,PMMBHF/BOP1SP fi MF 1SPNQU

    GSPOUBMNFEJVNTIPUPGCVTJOFTTXPNBO  MPPLJOHGSPOU  HSFZCBDLHSPVOE XFBSJOHB QSFNJVNIBOENBEFCFJHFXPPMCMB[FSXJUIBXIJUFCMPVTF  QPTJOHTUSBJHIU DBMN  TUBUJD  WJFXJOHGSPOU VMUSBIJHIRVBMJUZ TMJHIUTNJMF  UBLFOPOB$BOPO&043XJUIBNNMFOT 6)% )%3 EFQUIPG fi FME DFOUFSFEDPNQPTJUJPO  GBDFDMPTFVQ ࠄ੗ܐীחࢤࢿഋ"*Ѿҗޛ੉ੌࠗನೣغয੓णפ׮
  18. ,PMMBHF1SP fi MFݽ؛੄Ѿҗ ,PMMBHF&TTFODF1SP fi MF ,PMMBHF/BOP1SP fi MF 1SPNQU"DPO

    fi EFOUGFNBMFDIFGXJUICMBDLIBJSUJFECBDL XFBSJOHBXIJUFDIFGVOJGPSN4IFIBTBTFSJPVT GPDVTFEFYQSFTTJPO TUBOEJOHBHBJOTUBEBSLHSFZCBDLHSPVOEXJUITVCUMFUFYUVSFT5IF MJHIUJOHJTDJOFNBUJD IJHIMJHIUJOHIFSQSPGFTTJPOBMEFNFBOPSBOEUIFEFUBJMTPGUIFDIFGsTVOJGPSN ࠄ੗ܐীחࢤࢿഋ"*Ѿҗޛ੉ੌࠗನೣغয੓णפ׮
  19. ,PMMBHF1SP fi MFݽ؛੄Ѿҗ ,PMMBHF&TTFODF1SP fi MF ,PMMBHF/BOP1SP fi MF 1SPNQU'SFTIQPSUSBJUPGBZPVOH"TJBONBOXJUIBDBMNBOEUIPVHIUGVMFYQSFTTJPO

    SFTUJOHIJTIBOEOFBSIJTGBDF)FIBTTIPSUCMBDLIBJSBOEJTXFBSJOHBXIJUF DBTVBMTIJSU5IFCBDLHSPVOEJTB TNPPUI QBTUFMQJOL QSPWJEJOHBTPGUBOENJOJNBMJTUBFTUIFUJD5IFMJHIUJOHJTTPGU XJUIBOBUVSBM DMFBOMPPLUIBUFNQIBTJ[FTUIFTVCKFDUsTDMFBSTLJOBOESFMBYFEQPTF ࠄ੗ܐীחࢤࢿഋ"*Ѿҗޛ੉ੌࠗನೣغয੓णפ׮
  20. ,PMMBHF1SP fi MFݽ؛੄Ѿҗ ,PMMBHF&TTFODF1SP fi MF ,PMMBHF/BOP1SP fi MF 1SPNQU1PSUSBJUPGBZPVOH,PSFBONBOXJUIBTFSJPVTFYQSFTTJPO

    TUBOEJOHDPO fi EFOUMZXJUIBSNTDSPTTFE XFBSJOHBEBSLCMVFTVJUBOEXIJUFTIJSU5IFCBDLHSPVOEJTBTPGUHSBEJFOUXJUIB NJOJNBMJTU MJHIU - HSBZUPOF5IFNBOIBTTIPSUCMBDLIBJS QBSUFEUPUIFTJEF XJUIBDMFBOBOEQPMJTIFEBQQFBSBODF QSPGFTTJPOBMMJHIUJOH ࠄ੗ܐীחࢤࢿഋ"*Ѿҗޛ੉ੌࠗನೣغয੓णפ׮
  21. ੗୓ಣо࠺Ү 7((@EJTU ­ "SD'BDF@EJTU ­ $-*1@5 ¬ ,PMMBHF/BOP1SP fi MF

    XP%10    ,PMMBHF/BOP1SP fi MF X%10    ,PMMBHF&TTFODF1SP fi MF    7((@EJTUBODF "SD'BDF@EJTUBODF઱য૓঴ҷ੉޷૑৬ࢤࢿ੉޷૑о঴݃ա׵਷૑ܳஏ੿ೞח૑಴ $-*1@5઱য૓ఫझ౟೐܁೐౟ܳࢤࢿ੉޷૑о঴݃աੜ߈৔ೞח૑ஏ੿ೞח૑಴
  22. ੗୓ಣо࠺Ү 7((@EJTU ­ "SD'BDF@EJTU ­ $-*1@5 ¬ ,PMMBHF/BOP1SP fi MF

    XP%10    ,PMMBHF/BOP1SP fi MF X%10    ,PMMBHF&TTFODF1SP fi MF    7((@EJTUBODF "SD'BDF@EJTUBODF઱য૓঴ҷ੉޷૑৬ࢤࢿ੉޷૑о঴݃ա׵਷૑ܳஏ੿ೞח૑಴ $-*1@5઱য૓ఫझ౟೐܁೐౟ܳࢤࢿ੉޷૑о঴݃աੜ߈৔ೞח૑ஏ੿ೞח૑಴
  23. ੗୓ಣо࠺Ү 7((@EJTU ­ "SD'BDF@EJTU ­ $-*1@5 ¬ ,PMMBHF/BOP1SP fi MF

    XP%10    ,PMMBHF/BOP1SP fi MF X%10    ,PMMBHF&TTFODF1SP fi MF    7((@EJTUBODF "SD'BDF@EJTUBODF઱য૓঴ҷ੉޷૑৬ࢤࢿ੉޷૑о঴݃ա׵਷૑ܳஏ੿ೞח૑಴ $-*1@5઱য૓ఫझ౟೐܁೐౟ܳࢤࢿ੉޷૑о঴݃աੜ߈৔ೞח૑ஏ੿ೞח૑಴
  24. ,PMMBHF1SP fi MFݽ؛੄ҳઑ t"QIPUPPG QFSTPOu $-*15FYU &ODPEFS $-*1*NBHF &ODPEFS 1SPKFDUJPO

    1SPKFDUJPO 1BUDIFNCFEEJOH੄೟ण (BUFE4FMG"UUFOUJPO  :VIFOH-J FUBMt(-*(&/0QFO - 4FU(SPVOEFE5FYU - UP - *NBHF(FOFSBUJPOu 
  25. ,PMMBHF1SP fi MFݽ؛੄Ѿҗ (MPCBM&NCFEEJOH0 1BUDI&NCFEEJOH0 1SPNQUDFOUFSFEDPNQPTJUJPO  DMPTF - VQVQQFSCPEZQPSUSBJU

    TUSBJHIUQPTUVSF GSPOU - GBDJOH TJUUJOHPOBDPVDI XFBSJOHBSFE TXFBUFSXJUIUFEEZCFBS POJU  HJGUCPYFTBT CBDLHSPVOE BMBSHF$ISJTUNBTUSFFJTQSPNJOFOUMZQPTJUJPOFEBUUIFMFGUFEHF "XJOEPXXJUIBWJFXPGUIFOJHIUTLZ  DISJTUNBTMJHIUT HMJUUFSJOHPSOBNFOU ࠄ੗ܐীחࢤࢿഋ"*Ѿҗޛ੉ੌࠗನೣغয੓णפ׮ (MPCBM&NCFEEJOH9 1BUDI&NCFEEJOH0 (MPCBM&NCFEEJOH0 1BUDI&NCFEEJOH9
  26. ,PMMBHF1SP fi MFݽ؛੄Ѿҗ 1SPNQUNBO BQIPUPHSBQIZPGIBOETPNFNBO TJNQMF-JHIUHSBZCBDLHSPVOE  QPSUSBJUTIPU  IFBETIPU

     FOPVHIIFBESPPN VOQBUUFSOFEQMBJOTUSFFUXFBS UPOFPOUPOF  TNJMF ࠄ੗ܐীחࢤࢿഋ"*Ѿҗޛ੉ੌࠗನೣغয੓णפ׮ (MPCBM&NCFEEJOH0 1BUDI&NCFEEJOH0 (MPCBM&NCFEEJOH9 1BUDI&NCFEEJOH0 (MPCBM&NCFEEJOH0 1BUDI&NCFEEJOH9
  27. ъੋೠߔࠄݽ؛੄ഝਊ ࠄ੗ܐীחࢤࢿഋ"*Ѿҗޛ੉ੌࠗನೣغয੓णפ׮ '-69130 '-69%&7 '-694$)/&-- 1SPNQU$3&"5&"%&5"*-&%$-04& - 61*."(&0'")"/%&-&("/5-:"%03/&%8*5)&926*4*5&+&8&-3:5)&)"/%4)06-%#&'&.*/*/& 8*5)1&3'&$5-:."/*$63&%/"*-4*5*4"%03/&% 8*5)"41"3,-*/(%*".0/%3*/(0/5)&3*/('*/(&3

    "%&-*$"5&(0-%#3"$&-&50/5)&83*45 "/%"'&8&-&("/54*-7&33*/(40/05)&3'*/(&345)&4,*/50/&4)06-%#&8"3."/% /"563"- "/%5)&#"$,(306/%4)06-%#&"40'5 /&653"-$0-0350&.1)"4*;&5)&(-*55&3"/%4)*/&0'5)&%*".0/%4"/%+&8&-3:5)&-*()5*/(4)06-%#&40'5 )*()-*()5*/(5)& #3*--*"/$&0'5)&%*".0/%4"/%5)&'*/&%&5"*-40'5)&+&8&-3:
  28. 2"

  29. ଵҊޙ೴  )P +POBUIBOFUBMt%FOPJTJOH%JGGVTJPO1SPCBCJMJTUJD.PEFMTu /FVS*14   +JOH4IJ FUBMt*OTUBOU#PPUI1FSTPOBMJ[FE5FYU -

    UP - *NBHF(FOFSBUJPOXJUIPVU5FTU - 5JNF'JOFUVOJOHu   $IFSUJ .FIEJFUBMt3FQSPEVDJCMFTDBMJOHMBXTGPSDPOUSBTUJWFMBOHVBHF - JNBHFMFBSOJOHu   :VIFOH-J FUBMt(-*(&/0QFO - 4FU(SPVOEFE5FYU - UP - *NBHF(FOFSBUJPOu   -JBOH +JFFUBMt113,"-BSHF - 4DBMF1PSUSBJU1IPUP3FUPVDIJOH%BUBTFUXJUI)VNBO - 3FHJPO.BTLBOE(SPVQ - -FWFM$POTJTUFODZu   ;IFOH :JOHMJOFUBMt(FOFSBMGBDJBMSFQSFTFOUBUJPOMFBSOJOHJOBWJTVBM - MJOHVJTUJDNBOOFSu   IUUQT DJWJUBJDPNJNBHFT  IUUQT DJWJUBJDPNJNBHFT  1BUSJDL&TTFSFUBMt4DBMJOH3FDUJ fi FE'MPX5SBOTGPSNFSTGPS)JHI - 3FTPMVUJPO*NBHF4ZOUIFTJTu   IUUQT TUBCJMJUZBJOFXTTUBCMF - EJGGVTJPO  4BOHIZFPO/BFUBMt#PPTU:PVS0XO)VNBO*NBHF(FOFSBUJPO.PEFMWJB%JSFDU1SFGFSFODF0QUJNJ[BUJPOXJUI"*'FFECBDLu   :VWBM,JSTUBJO/BFUBMt1JDL - B - 1JD"O0QFO%BUBTFUPG6TFS1SFGFSFODFTGPS5FYU - UP - *NBHF(FOFSBUJPOu   #SBN8BMMBDFFUBMt%JGGVTJPO.PEFM"MJHONFOU6TJOH%JSFDU1SFGFSFODF0QUJNJ[BUJPOu $713   %FOH +JBOLBOHFUBMt"SD'BDF"EEJUJWF"OHVMBS.BSHJO-PTTGPS%FFQ'BDF3FDPHOJUJPOu $713   1BSLIJ 0NLBS.FUBM%FFQ'BDF3FDPHOJUJPOu #.7$ 8JMMJBN1FFCMFTFUBMt4DBMBCMF%JGGVTJPO.PEFMTXJUI5SBOTGPSNFSTu   8JMMJBN1FFCMFTFUBMt4DBMBCMF%JGGVTJPO.PEFMTXJUI5SBOTGPSNFSTu   IUUQT CMBDLGPSFTUMBCTBJBOOPVODJOH - CMBDL - GPSFTU - MBCT  IUUQT QJYBSU - BMQIBHJUIVCJP1JY"SU - TJHNB - QSPKFDU  -WNJO;IBOHFUBMt"EEJOH$POEJUJPOBM$POUSPMUP5FYU - UP - *NBHF%JGGVTJPO.PEFMTu *$$7   &EXBSE+)VFUBMt-P3"-PX - 3BOL"EBQUBUJPOPG-BSHF-BOHVBHF.PEFMTu   IUUQT XXXBJUJNFTLSOFXTBSUJDMF7JFXIUNM JEYOP  IUUQT XXXJEBJMZOFXTDPLSOFXTBSUJDMF7JFXIUNM JEYOP