Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Synthesizing Human Images in SIGGRAPH Asia 2021
Search
Udon
February 27, 2022
Technology
0
170
Synthesizing Human Images in SIGGRAPH Asia 2021
SIGGRAPH Asia 2021の「Synthesizing Human Images」セッションに採択された5本の論文を紹介します.
Udon
February 27, 2022
Tweet
Share
More Decks by Udon
See All by Udon
MIRU2024_招待講演_RALF_in_CVPR2024
udonda
1
330
[CVPR24 Oral] Retrieval-Augmented Layout Transformer for Content-Aware Layout Generation
udonda
0
280
Survey of Image Editing with GANs in SIGGRAPH'21
udonda
2
840
Network-to-Network Translation with Conditional Invertible Neural Networks
udonda
1
230
DARTS: Differentiable Architecture Search
udonda
0
110
Other Decks in Technology
See All in Technology
OCI Network Firewall 概要
oracle4engineer
PRO
0
4.2k
飲食店データの分析事例とそれを支えるデータ基盤
kimujun
0
180
OCI 運用監視サービス 概要
oracle4engineer
PRO
0
4.8k
Shopifyアプリ開発における Shopifyの機能活用
sonatard
4
260
New Relicを活用したSREの最初のステップ / NRUG OKINAWA VOL.3
isaoshimizu
3
630
SREが投資するAIOps ~ペアーズにおけるLLM for Developerへの取り組み~
takumiogawa
1
450
The Rise of LLMOps
asei
8
1.7k
The Web Performance Landscape in 2024 [PerfNow 2024]
tammyeverts
0
110
ノーコードデータ分析ツールで体験する時系列データ分析超入門
negi111111
0
420
個人でもIAM Identity Centerを使おう!(アクセス管理編)
ryder472
4
230
AI前提のサービス運用ってなんだろう?
ryuichi1208
8
1.4k
初心者向けAWS Securityの勉強会mini Security-JAWSを9ヶ月ぐらい実施してきての近況
cmusudakeisuke
0
130
Featured
See All Featured
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
sstephenson
159
15k
Designing Dashboards & Data Visualisations in Web Apps
destraynor
229
52k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
364
24k
4 Signs Your Business is Dying
shpigford
180
21k
Done Done
chrislema
181
16k
Reflections from 52 weeks, 52 projects
jeffersonlam
346
20k
How GitHub (no longer) Works
holman
310
140k
Site-Speed That Sticks
csswizardry
0
29
Speed Design
sergeychernyshev
25
620
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
27
840
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
irinanazarova
4
370
Understanding Cognitive Biases in Performance Measurement
bluesmoon
26
1.4k
Transcript
%BJDIJ)PSJUBൃද 4*((3"1)"TJB Synthesizing Human Images Session
୭ʁ 2
Synthesizing Human Images Session • URL: https://sa2021.siggraph.org/jp/attend/technical-papers/8/session/73 • 5ຊͷจ͕࠾ •
Barbershop • EyelashNet • Neural Actor • Pose with Style • SketchHairSalon 3
4
Barbershop ઃఆ — GAN inversion-based hair editing 5 Face identity
Generated image Hair color condition Hair texture condition Hair shape condition
• ͱإͷؒͷϒϨϯυΞʔςΟϑΝΫτΛੜͤ͡͞ͳ͍ੜ Barbershop ݁Ռ 6
Barbershop ख๏ 7 ͷલΛ” ” W+ F ͷޙΛ” ” W+
S C = (F, S) Face Hair ೖྗ ࠶ߏ Swap S ֤ೖྗ݅ ͷద༻ Blend ݟͨͷ࠷దԽ Face Identity Hair style
Barbershop ੍ݶ 8 • ᶃϚεΫ͕ͳ͍෦͋·Γ៉ྷʹ ͳΒͳ͍ • ᶄᶆϐΞεͷΑ͏ͳαϯϓϧ ແཧ •
ᶅᶇإΛःΔΑ͏ͳੜແཧ
9
EyelashNet ಈػ 10 ·ͭ͛ͷ3D࠶ߏࠔ ʢमਖ਼ʹ5࣌ؒఔඞཁʣ ·ͭ͛ͷMattingΛߦ͍ আڈ͔ͯ͠Βͷ3D࠶ߏ៉ྷʹͰ͖Δ with ·ͭ͛ w/o
·ͭ͛
EyelashNet ·ͭ͛σʔληοτ࡞ͷ 11 • ·ͭ͛άϦʔϯόοΫࡱӨͷΑ͏ͳํ๏ͰࡱӨͰ͖ͳ͍ • എܠʹͰͳ͘ɼલܠʹʢ·ͭ͛ʹܬޫృྉΛృͬͯࡱӨʣ
EyelashNet ·ͭ͛σʔληοτ࡞ͷ 12 • 2ຕͷࡱӨͷؒʹਓಈ͍ͯ͠·͏ͷͰ…ʁʁ • Optical flowͰҐஔ߹Θͤ XPҐஔ߹Θͤ XҐஔ߹Θͤ
EyelashNet ݁Ռ 13
14
Neural Actor ֓ཁ • ҙͷࢹɾ੍ޚՄೳͳϙʔζͰͷߴ࣭ͳਓؒ߹ͷͨΊͷख๏ • ϙʔζ<->ඪ४ۭؒͷมΛֶश 15
16 Neural Actor ܇࿅
NeuralActor ৽نࢹ߹࣮ݧ 17 ఆྔൺֱ ʢFID͕ۃʹྑ͍; γʔέϯεͰݟͨ࣌ʹ༏ΕΔʣ ఆੑൺֱ
18
Pose with Style ֓ཁ 19 • UVϚοϓ͕݅ͳStyleGAN2ʹΑΔશը૾ੜ ೖྗ ग़ྗ ਓੜ
Ծࢼண ೖྗ ग़ྗ
Pose with Style ܇࿅ 20 Ґஔ߹Θͤ 5BSHFUDPPSE HFOFSBUPS
Pose with Style ࣮ݧ — vs. Img-to-imgมϞσϧͱͷൺֱ 21 • ߴप៉ྷʹ
ੜ • ͷ༷ • إͷৄࡉ
Pose with Style ࣮ݧ — vs. StyleGAN-basedϞσϧͱͷൺֱ 22 • StylePoseGAN[Liu+
arXiv21] • Pose with StyleͷUVϚοϓʹର͢ ΔΛ΄ͱΜͲແͨ͘͠Α͏ͳ ख๏ • UVϚοϓΛ࠷େݶར༻͢Εߴ࣭
23
SketchHairSalon σʔληοτ࡞ 24 ೖྗ soft mask ࣗಈੜ खಈ •
soft maskMattingख๏Ͱ ਪ • ࣗಈੜͰฤΈࠐΈͳͲΛ දݱͰ͖ͳ͍ • खಈσʔληοτ͕ඞਢ
SketchHairSalon σʔληοτ࡞ 25 • ਪ࣌ʹ{ฤΈࠐΈ, ςΫενϟ}Λॻ͔ͤΔͷඇৗʹ໘͍͘͞ͷͰ…? • ύϥϝτϦοΫͳදݱؔΛఏҊ ฤΈࠐΈ ඇฤΈࠐΈ
SketchHairSalon ख๏ 26
SketchHairSalon ࣮ݧ 27
SketchHairSalon ੍ݶ 28 • Ԟߦ͖͕ඞཁͳܕݫ͍͠ • ແݶໟଋܕϑϥοτ ͳੜ • רࢴֶशαϯϓϧͱಉ͡
ੜ • Ԟߦ͖ߟྀͨ͠ੜͰղܾ(?)
·ͱΊ • ਓυϝΠϯͷΛѻ͏5ຊͷจΛհ • ฤू • Animatable NeRF • ·ͭ͛
• શੜ • ݻ༗ͷΛͲ͏ղܾ͢Δ͔ʁΛ͔ͳΓ۩ମతͳΞϓϩʔνͰղܾ͍ͯ͠Δ จ͕ଟ͔ͬͨ 29 Synthesizing Human Images Session