Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Synthesizing Human Images in SIGGRAPH Asia 2021
Search
Udon
February 27, 2022
Technology
0
170
Synthesizing Human Images in SIGGRAPH Asia 2021
SIGGRAPH Asia 2021の「Synthesizing Human Images」セッションに採択された5本の論文を紹介します.
Udon
February 27, 2022
Tweet
Share
More Decks by Udon
See All by Udon
MIRU2024_招待講演_RALF_in_CVPR2024
udonda
1
370
[CVPR24 Oral] Retrieval-Augmented Layout Transformer for Content-Aware Layout Generation
udonda
0
310
Survey of Image Editing with GANs in SIGGRAPH'21
udonda
2
880
Network-to-Network Translation with Conditional Invertible Neural Networks
udonda
1
290
DARTS: Differentiable Architecture Search
udonda
0
140
Other Decks in Technology
See All in Technology
ワールドカフェI /チューターを改良する / World Café I and Improving the Tutors
ks91
PRO
0
120
ブラウザのレガシー・独自機能を愛でる-Firefoxの脆弱性4選- / Browser Crash Club #1
masatokinugawa
1
480
AI Agentを「期待通り」に動かすために:設計アプローチの模索と現在地
kworkdev
PRO
2
460
白金鉱業Meetup_Vol.18_AIエージェント時代のUI/UX設計
brainpadpr
0
110
勝手に!深堀り!Cloud Run worker pools / Deep dive Cloud Run worker pools
iselegant
2
380
DuckDB MCPサーバーを使ってAWSコストを分析させてみた / AWS cost analysis with DuckDB MCP server
masahirokawahara
0
1.3k
Goの組織でバックエンドTypeScriptを採用してどうだったか / How was adopting backend TypeScript in a Golang company
kaminashi
6
6.1k
品質文化を支える小さいクロスファンクショナルなチーム / Cross-functional teams fostering quality culture
toma_sm
0
120
Conquering PDFs: document understanding beyond plain text
inesmontani
PRO
0
110
The Tale of Leo: Brave Lion and Curious Little Bug
canalun
1
120
AIでめっちゃ便利になったけど、結局みんなで学ぶよねっていう話
kakehashi
PRO
0
180
システムとの会話から生まれる先手のDevOps
kakehashi
PRO
0
290
Featured
See All Featured
The Illustrated Children's Guide to Kubernetes
chrisshort
48
49k
A designer walks into a library…
pauljervisheath
205
24k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
29
9.4k
Statistics for Hackers
jakevdp
798
220k
Why You Should Never Use an ORM
jnunemaker
PRO
55
9.3k
Unsuck your backbone
ammeep
670
57k
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
31
1.1k
Making the Leap to Tech Lead
cromwellryan
133
9.2k
jQuery: Nuts, Bolts and Bling
dougneiner
63
7.7k
KATA
mclloyd
29
14k
Designing for humans not robots
tammielis
252
25k
Large-scale JavaScript Application Architecture
addyosmani
512
110k
Transcript
%BJDIJ)PSJUBൃද 4*((3"1)"TJB Synthesizing Human Images Session
୭ʁ 2
Synthesizing Human Images Session • URL: https://sa2021.siggraph.org/jp/attend/technical-papers/8/session/73 • 5ຊͷจ͕࠾ •
Barbershop • EyelashNet • Neural Actor • Pose with Style • SketchHairSalon 3
4
Barbershop ઃఆ — GAN inversion-based hair editing 5 Face identity
Generated image Hair color condition Hair texture condition Hair shape condition
• ͱإͷؒͷϒϨϯυΞʔςΟϑΝΫτΛੜͤ͡͞ͳ͍ੜ Barbershop ݁Ռ 6
Barbershop ख๏ 7 ͷલΛ” ” W+ F ͷޙΛ” ” W+
S C = (F, S) Face Hair ೖྗ ࠶ߏ Swap S ֤ೖྗ݅ ͷద༻ Blend ݟͨͷ࠷దԽ Face Identity Hair style
Barbershop ੍ݶ 8 • ᶃϚεΫ͕ͳ͍෦͋·Γ៉ྷʹ ͳΒͳ͍ • ᶄᶆϐΞεͷΑ͏ͳαϯϓϧ ແཧ •
ᶅᶇإΛःΔΑ͏ͳੜແཧ
9
EyelashNet ಈػ 10 ·ͭ͛ͷ3D࠶ߏࠔ ʢमਖ਼ʹ5࣌ؒఔඞཁʣ ·ͭ͛ͷMattingΛߦ͍ আڈ͔ͯ͠Βͷ3D࠶ߏ៉ྷʹͰ͖Δ with ·ͭ͛ w/o
·ͭ͛
EyelashNet ·ͭ͛σʔληοτ࡞ͷ 11 • ·ͭ͛άϦʔϯόοΫࡱӨͷΑ͏ͳํ๏ͰࡱӨͰ͖ͳ͍ • എܠʹͰͳ͘ɼલܠʹʢ·ͭ͛ʹܬޫృྉΛృͬͯࡱӨʣ
EyelashNet ·ͭ͛σʔληοτ࡞ͷ 12 • 2ຕͷࡱӨͷؒʹਓಈ͍ͯ͠·͏ͷͰ…ʁʁ • Optical flowͰҐஔ߹Θͤ XPҐஔ߹Θͤ XҐஔ߹Θͤ
EyelashNet ݁Ռ 13
14
Neural Actor ֓ཁ • ҙͷࢹɾ੍ޚՄೳͳϙʔζͰͷߴ࣭ͳਓؒ߹ͷͨΊͷख๏ • ϙʔζ<->ඪ४ۭؒͷมΛֶश 15
16 Neural Actor ܇࿅
NeuralActor ৽نࢹ߹࣮ݧ 17 ఆྔൺֱ ʢFID͕ۃʹྑ͍; γʔέϯεͰݟͨ࣌ʹ༏ΕΔʣ ఆੑൺֱ
18
Pose with Style ֓ཁ 19 • UVϚοϓ͕݅ͳStyleGAN2ʹΑΔશը૾ੜ ೖྗ ग़ྗ ਓੜ
Ծࢼண ೖྗ ग़ྗ
Pose with Style ܇࿅ 20 Ґஔ߹Θͤ 5BSHFUDPPSE HFOFSBUPS
Pose with Style ࣮ݧ — vs. Img-to-imgมϞσϧͱͷൺֱ 21 • ߴप៉ྷʹ
ੜ • ͷ༷ • إͷৄࡉ
Pose with Style ࣮ݧ — vs. StyleGAN-basedϞσϧͱͷൺֱ 22 • StylePoseGAN[Liu+
arXiv21] • Pose with StyleͷUVϚοϓʹର͢ ΔΛ΄ͱΜͲແͨ͘͠Α͏ͳ ख๏ • UVϚοϓΛ࠷େݶར༻͢Εߴ࣭
23
SketchHairSalon σʔληοτ࡞ 24 ೖྗ soft mask ࣗಈੜ खಈ •
soft maskMattingख๏Ͱ ਪ • ࣗಈੜͰฤΈࠐΈͳͲΛ දݱͰ͖ͳ͍ • खಈσʔληοτ͕ඞਢ
SketchHairSalon σʔληοτ࡞ 25 • ਪ࣌ʹ{ฤΈࠐΈ, ςΫενϟ}Λॻ͔ͤΔͷඇৗʹ໘͍͘͞ͷͰ…? • ύϥϝτϦοΫͳදݱؔΛఏҊ ฤΈࠐΈ ඇฤΈࠐΈ
SketchHairSalon ख๏ 26
SketchHairSalon ࣮ݧ 27
SketchHairSalon ੍ݶ 28 • Ԟߦ͖͕ඞཁͳܕݫ͍͠ • ແݶໟଋܕϑϥοτ ͳੜ • רࢴֶशαϯϓϧͱಉ͡
ੜ • Ԟߦ͖ߟྀͨ͠ੜͰղܾ(?)
·ͱΊ • ਓυϝΠϯͷΛѻ͏5ຊͷจΛհ • ฤू • Animatable NeRF • ·ͭ͛
• શੜ • ݻ༗ͷΛͲ͏ղܾ͢Δ͔ʁΛ͔ͳΓ۩ମతͳΞϓϩʔνͰղܾ͍ͯ͠Δ จ͕ଟ͔ͬͨ 29 Synthesizing Human Images Session