バイラテラルアップサンプリング

高解像度なディスプレイに対して効率よくリアルタイムレンダリングを行う為の工夫を紹介します
これは2025年5月11日に行われた Kernel/VM探検隊@関西 11回目 での発表動資料です
発表動画 : https://www.youtube.com/watch?v=tGUp17w6Bbs
ソースコード : https://github.com/Fadis/gct/tree/kernelvm_20250511

Transcript

1. όΠϥςϥϧΞοϓαϯϓϦϯά Bilateral Upsampling NAOMASA MATSUBAYASHI
2. 2023೥࣌఺ https://github.com/Fadis/gct/
3. 2025೥࣌఺ https://github.com/Fadis/gct/
4. 2023೥࣌఺ https://github.com/Fadis/gct/
5. 2025೥࣌఺ https://github.com/Fadis/gct/
6. ௖఺୯Ґͷॲཧ ϥελϥΠζ ϐΫηϧ୯Ґͷॲཧ Graphics Pipeline
7. ௖఺୯Ґͷॲཧ ϥελϥΠζ ϐΫηϧ୯Ґͷॲཧ ը໘্ͷ͜ͷࡾ֯ܗͷൣғΛృΖ͏ Graphics Pipeline
8. ௖఺୯Ґͷॲཧ ϥελϥΠζ ϐΫηϧ୯Ґͷॲཧ ը໘্ͷ͜ͷࡾ֯ܗͷൣғΛృΖ͏ ࡾ֯ܗͷൣғ಺ʹ͋ΔϐΫηϧΛݟ͚ͭΑ͏ Graphics Pipeline
9. ௖఺୯Ґͷॲཧ ϥελϥΠζ ϐΫηϧ୯Ґͷॲཧ ը໘্ͷ͜ͷࡾ֯ܗͷൣғΛృΖ͏ ࡾ֯ܗͷൣғ಺ʹ͋ΔϐΫηϧΛݟ͚ͭΑ͏ ࡾ֯ܗͷൣғ಺ͷϐΫηϧͷ ৭ΛܾΊΑ͏ Graphics Pipeline
10. ௖఺୯Ґͷॲཧ ϥελϥΠζ ϐΫηϧ୯Ґͷॲཧ ը໘্ͷ͜ͷࡾ֯ܗͷൣғΛృΖ͏ ࡾ֯ܗͷൣғ಺ʹ͋ΔϐΫηϧΛݟ͚ͭΑ͏ ࡾ֯ܗͷൣғ಺ͷϐΫηϧͷ ৭ΛܾΊΑ͏ Graphics Pipeline ܭࢉίετ͸γʔϯͷ௖఺਺ͱ εΫϦʔϯͷϐΫηϧ਺ʹൺྫ
11. ௖఺୯Ґͷॲཧ ϥελϥΠζ ϐΫηϧͷ৭ΛܾΊΔͷʹ ඞཁͳ৘ใΛ Πϝʔδʹॻ͘ Graphics Pipeline Compute Pipeline ΠϝʔδΛಡΜͰ ϐΫηϧͷ৭ΛܾΊΔ ஗ԆϨϯμϦϯά
12. ௖఺୯Ґͷॲཧ ϥελϥΠζ ϐΫηϧͷ৭ΛܾΊΔͷʹ ඞཁͳ৘ใΛ Πϝʔδʹॻ͘ Graphics Pipeline Compute Pipeline ΠϝʔδΛಡΜͰ ϐΫηϧͷ৭ΛܾΊΔ ஗ԆϨϯμϦϯά GόοϑΝ
13. ௖఺୯Ґͷॲཧ ϥελϥΠζ ϐΫηϧͷ৭ΛܾΊΔͷʹ ඞཁͳ৘ใΛ Πϝʔδʹॻ͘ Graphics Pipeline Compute Pipeline ΠϝʔδΛಡΜͰ ϐΫηϧͷ৭ΛܾΊΔ ஗ԆϨϯμϦϯά ར఺ ྡ઀͢ΔϐΫηϧͷ৘ใΛಡΊΔ GόοϑΝ ಉظ
14. ௖఺୯Ґͷॲཧ ϥελϥΠζ ϐΫηϧͷ৭ΛܾΊΔͷʹ ඞཁͳ৘ใΛ Πϝʔδʹॻ͘ Graphics Pipeline Compute Pipeline ΠϝʔδΛಡΜͰ ϐΫηϧͷ৭ΛܾΊΔ ஗ԆϨϯμϦϯά ར఺ ྡ઀͢ΔϐΫηϧͷ৘ใΛಡΊΔ ෳࡶͳϐΫηϧॲཧΛෳ਺ͷγΣʔμʹ෼͚ΒΕΔ GόοϑΝ ಉظ
15. ௖఺୯Ґͷॲཧ ϥελϥΠζ ϐΫηϧͷ৭ΛܾΊΔͷʹ ඞཁͳ৘ใΛ Πϝʔδʹॻ͘ Graphics Pipeline Compute Pipeline ΠϝʔδΛಡΜͰ ϐΫηϧͷ৭ΛܾΊΔ ஗ԆϨϯμϦϯά ར఺ ྡ઀͢ΔϐΫηϧͷ৘ใΛಡΊΔ ෳࡶͳϐΫηϧॲཧΛෳ਺ͷγΣʔμʹ෼͚ΒΕΔ Compute Pipelineઐ༻ͷػೳ͕࢖͑Δ GόοϑΝ
16. γϟυ΢Ϛοϓͷੜ੒ Graphics Pipeline GόοϑΝͷੜ੒ Graphics Pipeline ௚઀ޫ Compute Pipeline τʔϯϚοϐϯά Compute Pipeline ΨϯϚ Compute Pipeline ϒϧʔϜ Compute Pipeline
17. froxelαϯϓϦϯά Compute Pipeline γϟυ΢Ϛοϓͷੜ੒ Graphics Pipeline GόοϑΝͷੜ੒ Graphics Pipeline ௚઀ޫ Compute Pipeline τʔϯϚοϐϯά Compute Pipeline ΨϯϚ Compute Pipeline ΦΫϧʔδϣϯΧϦϯά Graphics Pipeline ϨϯζϑϨΞ Compute Pipeline ΞϯϏΤϯτΦΫϧʔδϣϯ Compute Pipeline froxelͷੜ੒ Compute Pipeline CoCͷੜ੒ Compute Pipeline ΦʔτϑΥʔΧε Compute Pipeline ϏωοςΟϯά Compute Pipeline ඃࣸքਂ౓ޮՌ Compute Pipeline ϒϧʔϜ Compute Pipeline ৭ۭؒͷม׵ Compute Pipeline ൒ಁ໌ͷղܾ Compute Pipeline ۭͷੜ੒ Compute Pipeline
18. froxelαϯϓϦϯά Compute Pipeline γϟυ΢Ϛοϓͷੜ੒ Graphics Pipeline GόοϑΝͷੜ੒ Graphics Pipeline ௚઀ޫ Compute Pipeline τʔϯϚοϐϯά Compute Pipeline ΨϯϚ Compute Pipeline ΦΫϧʔδϣϯΧϦϯά Graphics Pipeline ϨϯζϑϨΞ Compute Pipeline ΞϯϏΤϯτΦΫϧʔδϣϯ Compute Pipeline froxelͷੜ੒ Compute Pipeline CoCͷੜ੒ Compute Pipeline ΦʔτϑΥʔΧε Compute Pipeline ϏωοςΟϯά Compute Pipeline ඃࣸքਂ౓ޮՌ Compute Pipeline ϒϧʔϜ Compute Pipeline ৭ۭؒͷม׵ Compute Pipeline ൒ಁ໌ͷղܾ Compute Pipeline ۭͷੜ੒ Compute Pipeline 1ϑϨʔϜຖʹܭࢉ͢Δඞཁ͕͋Γ ܭࢉίετ͕ϐΫηϧ਺ʹൺྫ͢Δॲཧ
19. ΦΫϧʔδϣϯΧϦϯάΛ௨ա͢Δ௖఺਺ 1080pશը໘දࣔͷϐΫηϧ਺ ਺ઍ͔Β਺ສఔ౓ 2,073,600
21. 60FPS
22. 60FPS 1080p
23. 60FPS 1080p 2160p
24. ͳΜͱ͔ΠϯνΩͰ͖Μͷ͔
25. 1080pͰϨϯμϦϯάͨ݁͠ՌΛ ิؒͯ͠2160pʹ͢Ε͹ྑ͍ͷͰ͸
27. ઢܗิؒ
28. 60FPS 1080p 2160p
29. 60FPS 1080p 2160p
32. 1080pͰϨϯμϦϯάͯ͠ิؒ 2160pͰϨϯμϦϯά
33. 1080pͰϨϯμϦϯάͯ͠ิؒ 2160pͰϨϯμϦϯά
34. 1080pͰϨϯμϦϯάͯ͠ิؒ 2160pͰϨϯμϦϯά ͜ͷ͋ͨΓͷҧ͍͸ Θ͔Γ΍͍͢
35. 1080pͰϨϯμϦϯάͯ͠ิؒ 2160pͰϨϯμϦϯά ͜ͷ͋ͨΓͷҧ͍͸ Θ͔Γʹ͍͘
36. 1080pͰϨϯμϦϯάͯ͠ิؒ 2160pͰϨϯμϦϯά ৭͕ٸܹʹมԽ͢Δͱ͜Ζ͕΅΍͚Δͱ ҧ͍ʹؾ෇͖΍͘͢ͳΔ
37. 1080pͰϨϯμϦϯάͯ͠ิؒ 2160pͰϨϯμϦϯά ৭͕ٸܹʹมԽ͢Δͱ͜Ζ͕΅΍͚Δͱ ҧ͍ʹؾ෇͖΍͘͢ͳΔ Τοδͷγϟʔϓ͚ͩ͞Λอޢ͢ΔΑ͏ʹิؒ͢Ε͹ όϨͳ͍ͷͰ͸
38. C. Tomasi and R. Manduchi, “Bilateral fi ltering for gray and color images,” in Sixth International Conference on Computer Vision, Jan 1998, pp. 839–846. https://ieeexplore.ieee.org/document/710815 όΠϥςϥϧ ϑΟϧλ
39. C. Tomasi and R. Manduchi, “Bilateral fi ltering for gray and color images,” in Sixth International Conference on Computer Vision, Jan 1998, pp. 839–846. https://ieeexplore.ieee.org/document/710815 όΠϥςϥϧ ϑΟϧλ ৭ͷมԽͷܹ͠͞Ͱ ϑΟϧλͷॏΈΛม͑Δ Τοδ͚ͩ࢒ͯ͠ ϊΠζΛআڈ͢Δ ը૾ॲཧख๏
40. όΠϥςϥϧ ΞοϓαϯϓϦϯά ௿ղ૾౓ͷ ը૾
41. όΠϥςϥϧ ΞοϓαϯϓϦϯά ௿ղ૾౓ͷ ը૾ ߴղ૾౓ͷ Τοδ৘ใ
42. όΠϥςϥϧ ΞοϓαϯϓϦϯά ௿ղ૾౓ͷ ը૾ ߴղ૾౓ͷ Τοδ৘ใ ΤοδҎ֎͸ઢܗิؒ ΤοδͰ͸ۙ๣ͷϐΫηϧͷ͏ͪ ৚͕͍݅ۙϐΫηϧͷ৭Λ࠾༻
43. όΠϥςϥϧ ΞοϓαϯϓϦϯά ௿ղ૾౓ͷ ը૾ ߴղ૾౓ͷ Τοδ৘ใ ΤοδҎ֎͸ઢܗิؒ ߴղ૾౓ͷ ը૾ ΤοδͰ͸ۙ๣ͷϐΫηϧͷ͏ͪ ৚͕͍݅ۙϐΫηϧͷ৭Λ࠾༻
44. όΠϥςϥϧ ΞοϓαϯϓϦϯά ௿ղ૾౓ͷ ը૾ ߴղ૾౓ͷ Τοδ৘ใ ΤοδҎ֎͸ઢܗิؒ ߴղ૾౓ͷ ը૾ ΤοδͰ͸ۙ๣ͷϐΫηϧͷ͏ͪ ৚͕͍݅ۙϐΫηϧͷ৭Λ࠾༻ ໰୊͸ߴղ૾౓ͷΤοδ৘ใΛ Ͳ͏΍ͬͯ༻ҙ͢Δ͔
45. froxelαϯϓϦϯά Compute Pipeline γϟυ΢Ϛοϓͷੜ੒ Graphics Pipeline GόοϑΝͷੜ੒ Graphics Pipeline ௚઀ޫ Compute Pipeline τʔϯϚοϐϯά Compute Pipeline ΨϯϚ Compute Pipeline ΦΫϧʔδϣϯΧϦϯά Graphics Pipeline ϨϯζϑϨΞ Compute Pipeline ΞϯϏΤϯτΦΫϧʔδϣϯ Compute Pipeline froxelͷੜ੒ Compute Pipeline CoCͷੜ੒ Compute Pipeline ΦʔτϑΥʔΧε Compute Pipeline ϏωοςΟϯά Compute Pipeline ඃࣸքਂ౓ޮՌ Compute Pipeline ϒϧʔϜ Compute Pipeline ৭ۭؒͷม׵ Compute Pipeline ൒ಁ໌ͷղܾ Compute Pipeline ۭͷੜ੒ Compute Pipeline bilateral offsetͷੜ੒ Compute Pipeline όΠϥςϥϧΞοϓαϯϓϦϯά Compute Pipeline GόοϑΝΛ2160pͰ࡞Δ GόοϑΝ͔ΒΤοδΛಘΔ 1080pͰϨϯμϦϯά͢Δ
46. ਂ౓ Ξϧϕυ ΞϧϑΝ ๏ઢ ൃޫ ःṭ౓ ϝλϦοΫ ϥϑωε ΠϯελϯεID ϓϦϛςΟϒID ΦϓςΟΧϧϑϩʔ 0 16 32 48 64bit GόοϑΝͷத਎
47. ਂ౓ Ξϧϕυ ΞϧϑΝ ๏ઢ ൃޫ ःṭ౓ ϝλϦοΫ ϥϑωε ΠϯελϯεID ϓϦϛςΟϒID ΦϓςΟΧϧϑϩʔ 0 16 32 48 64bit ਂ౓ Ξϧϕυ ΞϧϑΝ ๏ઢ ൃޫ ःṭ౓ ϝλϦοΫ ϥϑωε ΠϯελϯεID ϓϦϛςΟϒID ΦϓςΟΧϧϑϩʔ ਂ౓ Ξϧϕυ ΞϧϑΝ ๏ઢ ൃޫ ःṭ౓ ϝλϦοΫ ϥϑωε ΠϯελϯεID ϓϦϛςΟϒID ΦϓςΟΧϧϑϩʔ ਂ౓ Ξϧϕυ ΞϧϑΝ ๏ઢ ൃޫ ःṭ౓ ϝλϦοΫ ϥϑωε ΠϯελϯεID ϓϦϛςΟϒID ΦϓςΟΧϧϑϩʔ ϨΠϠʔͷॱং ϨΠϠʔ0 ϨΠϠʔ1 ϨΠϠʔ2 ϨΠϠʔ3
48. A B ྡ઀͢ΔϐΫηϧAͱϐΫηϧB͸Τοδ͔
49. A B ྡ઀͢ΔϐΫηϧAͱϐΫηϧB͸Τοδ͔ AͱBͷΠϯελϯεID·ͨ͸ϓϦϛςΟϒID͕ҟͳΔ ҟͳΔ2ͭͷ෺ମͷڥքͳͷͰ ͨͿΜΤοδ
50. A B ྡ઀͢ΔϐΫηϧAͱϐΫηϧB͸Τοδ͔ AͱBͷΠϯελϯεID·ͨ͸ϓϦϛςΟϒID͕ҟͳΔ ҟͳΔ2ͭͷ෺ମͷڥքͳͷͰ ͨͿΜΤοδ AͱBͷਂ౓͕େ͖͘ҟͳΔ ۙ͘ʹ͋Δ໘ͱԕ͘ʹ͋Δ໘ͷڥքͳͷͰ ͨͿΜΤοδ
51. A B ྡ઀͢ΔϐΫηϧAͱϐΫηϧB͸Τοδ͔ AͱBͷΠϯελϯεID·ͨ͸ϓϦϛςΟϒID͕ҟͳΔ ҟͳΔ2ͭͷ෺ମͷڥքͳͷͰ ͨͿΜΤοδ AͱBͷਂ౓͕େ͖͘ҟͳΔ ۙ͘ʹ͋Δ໘ͱԕ͘ʹ͋Δ໘ͷڥքͳͷͰ ͨͿΜΤοδ AͱBͷ๏ઢͷ಺ੵ͕খ͍͞ ޲͖͕େ͖͘ҟͳΔ2ͭͷ໘ͷڥքͳͷͰ ͨͿΜΤοδ
52. ͜ͷ෦෼͸ ಉ͡ੴͷபͷҟͳΔ໘ͳͷͰ ΠϯελϯεIDͱϓϦϛςΟϒID͸ಉ͡ ਂ౓΋ຆͲಉ͡ ๏ઢ͸େ͖͘ҟͳ͍ͬͯΔͷͰ ͨͿΜେ͖ͳ໌Δ͞ͷ มԽ͕ੜ͡Δ
53. fid (x, y) = { 1 xpid = ypid ∧ xiid = yiid 0 xpid ≠ ypid ∨ xiid ≠ yiid fdepth (x, y) = min(1.3 − |xdepth , ydepth |,1) fnormal (x, y) = min(1.3max(xnormal ⋅ ynormal ,0),1) f(x, y) = fid (x, y)fdepth (x, y)fnormal (x, y)
54. A B P C D Pͷ৭ΛܾΊ͍ͨͱ͖ ɺ ɺ ɺ ɺ ΛٻΊ ࠷΋େ͖ͳ஋Λฦͨ͠ϐΫηϧͷ৭ΛPͷ৭ͱ͢Δ f(A, P) f(B, P) f(C, P) f(D, P) f(E, P) fid (x, y) = { 1 xpid = ypid ∧ xiid = yiid 0 xpid ≠ ypid ∨ xiid ≠ yiid fdepth (x, y) = min(1.3 − |xdepth , ydepth |,1) fnormal (x, y) = min(1.3max(xnormal ⋅ ynormal ,0),1) f(x, y) = fid (x, y)fdepth (x, y)fnormal (x, y)
55. A B P C D Pͷ৭ΛܾΊ͍ͨͱ͖ E = A + B + C + D 4 ɺ ɺ ɺ ɺ ΛٻΊ ࠷΋େ͖ͳ஋Λฦͨ͠ϐΫηϧͷ৭ΛPͷ৭ͱ͢Δ f(A, P) f(B, P) f(C, P) f(D, P) f(E, P) ͕࠷େͩͬͨ৔߹͸ઢܗิؒ͢Δ f(E, P) fid (x, y) = { 1 xpid = ypid ∧ xiid = yiid 0 xpid ≠ ypid ∨ xiid ≠ yiid fdepth (x, y) = min(1.3 − |xdepth , ydepth |,1) fnormal (x, y) = min(1.3max(xnormal ⋅ ynormal ,0),1) f(x, y) = fid (x, y)fdepth (x, y)fnormal (x, y)
56. A B P C D 35 36 11 12 ϐΫηϧPͷ৭͸ ϐΫηϧCͷ৭ʹ͢΂͖ͱ൑அ͞Εͨ৔߹ Pͷ৭͸ઢܗิؒͳΒ ͔Β αϯϓϦϯά͢΂͖ॴΛ όΠϥςϥϧΞοϓαϯϓϦϯάͰ͸ ͔ΒαϯϓϦϯά͢Δ (35.5,11.5) (35.0,12.0)
57. A B P C D 35 36 11 12 (−0.5,0.5) ϐΫηϧPͷ৭͸ ϐΫηϧCͷ৭ʹ͢΂͖ͱ൑அ͞Εͨ৔߹ Pͷ৭͸ઢܗิؒͳΒ ͔Β αϯϓϦϯά͢΂͖ॴΛ όΠϥςϥϧΞοϓαϯϓϦϯάͰ͸ ͔ΒαϯϓϦϯά͢Δ (35.5,11.5) (35.0,12.0) ͜ͷࠩ ͕ όΠϥςϥϧΦϑηοτ (−0.5,0.5) όΠϥςϥϧΦϑηοτΛٻΊͯΠϝʔδʹه࿥͓͚ͯ͠͹ ΤοδΛอޢͯ͠ิ͕ؒͰ͖Δ
58. A B C D
59. A B C D ͜ͷϐΫηϧ͸ৗʹA
60. A B C D ͜ͷϐΫηϧ͸ ɺ ɺ ͷ͍ͣΕ͔ʹͳΔ A B A + B 2 2bit ͜ͷϐΫηϧ͸ৗʹA
61. A B C D ͜ͷϐΫηϧ͸ ɺ ɺ ͷ͍ͣΕ͔ʹͳΔ A B A + B 2 ͜ͷϐΫηϧ͸ ɺ ɺ ɺ ɺ ͷ͍ͣΕ͔ʹͳΔ A B C D A + B + C + D 4 2bit 3bit ͜ͷϐΫηϧ͸ৗʹA
62. A B C D ͜ͷϐΫηϧ͸ ɺ ɺ ͷ͍ͣΕ͔ʹͳΔ A B A + B 2 ͜ͷϐΫηϧ͸ ɺ ɺ ͷ͍ͣΕ͔ʹͳΔ A C A + C 2 ͜ͷϐΫηϧ͸ ɺ ɺ ɺ ɺ ͷ͍ͣΕ͔ʹͳΔ A B C D A + B + C + D 4 2bit 3bit 2bit ͜ͷϐΫηϧ͸ৗʹA
63. A B C D ͜ͷϐΫηϧ͸ ɺ ɺ ͷ͍ͣΕ͔ʹͳΔ A B A + B 2 ͜ͷϐΫηϧ͸ ɺ ɺ ͷ͍ͣΕ͔ʹͳΔ A C A + C 2 ͜ͷϐΫηϧ͸ ɺ ɺ ɺ ɺ ͷ͍ͣΕ͔ʹͳΔ A B C D A + B + C + D 4 2bit 3bit 2bit ͜ͷϐΫηϧ͸ৗʹA 7bit͋Ε͹ ϐΫηϧͷ όΠϥςϥϧΦϑηοτΛه࿥Ͱ͖Δ 2 × 2
64. const auto bilateral_weight_desc = builder.call( builder.get_image_io_create_info( std::make_shared< gct::compute >( gct::compute_create_info() .set_allocator_set( res.allocator_set ) .set_shader( CMAKE_CURRENT_BINARY_DIR "/bilateral_weight/bilateral_weight.comp.spv" ) .set_scene_graph( sg->get_resource() ) .add_resource( { "global_uniforms", global_uniform } ) ), gct::image_io_plan() .add_input( "src" ) .add_input( "depth" ) .add_output( "dest", "src", { 0.5f, -1.f }, vk::Format::eR8Uint ) .set_dim( "src", { 0.5f, -1.f } ) .set_node_name( "bilateral_weight" ) ) )( gct::shader_graph::vertex::combined_result_type() .add( "src", extended_gbuffer_desc.linear ) .add( "depth", extended_depth_desc.linear ) ); const auto lighting_desc = builder.call( builder.get_image_io_create_info( std::make_shared< gct::compute >( gct::compute_create_info() .set_allocator_set( res.allocator_set ) .set_shader( CMAKE_CURRENT_BINARY_DIR "/lighting/lighting.comp.spv" ) bilateral offsetͷܭࢉΛ ೖྗΠϝʔδ(GόοϑΝ)ͷ൒෼ͷղ૾౓Ͱߦ͏ ग़ྗΠϝʔδ͸ೖྗͷ൒෼ͷղ૾౓Ͱ ϑΥʔϚοτ͸8bitූ߸ͳ͠੔਺
65. const auto lighting_desc = builder.call( builder.get_image_io_create_info( std::make_shared< gct::compute >( gct::compute_create_info() .set_allocator_set( res.allocator_set ) .set_shader( CMAKE_CURRENT_BINARY_DIR "/lighting/lighting.comp.spv" ) .set_scene_graph( sg->get_resource() ) .add_resource( { "global_uniforms", global_uniform } ) ), gct::image_io_plan() .add_input( "src" ) .add_input( "depth" ) .add_output( "dest", "src", { 0.5f, -4.f }, vk::Format::eR16G16B16A16Sfloat ) .set_dim( "src", { 0.5f, -4.f } ) .set_node_name( "lighting" ) ) .set_push_constant( "unproject", *unproject_to_world_desc ) )( gct::shader_graph::vertex::combined_result_type() .add( "src", extended_gbuffer_desc.linear ) .add( "depth", extended_depth_desc.linear ) ); ௚઀ޫͷܭࢉΛ ೖྗΠϝʔδ(GόοϑΝ)ͷ൒෼ͷղ૾౓Ͱߦ͏ ग़ྗΠϝʔδ͸ೖྗͷ൒෼ͷղ૾౓Ͱ ϑΥʔϚοτ͸൒ਫ਼౓ුಈখ਺఺਺4ݸͷϕΫλ
66. const float depth_similarity_01 = min( 1.3 - abs( normal_depth[ pos0 ].w - normal_depth[ pos1 ].w ), 1.0 ); const float depth_similarity_a1 = min( 1.3 - abs( ( normal_depth[ pos0 ].w + normal_depth[ pos2 ].w ) * 0.5 - normal_depth[ pos1 ].w ), 1.0 ); const float depth_similarity_12 = min( 1.3 - abs( normal_depth[ pos1 ].w - normal_depth[ pos2 ].w ), 1.0 ); const float depth_similarity_b1 = min( 1.3 - abs( ( normal_depth[ pos3 ].w + normal_depth[ pos4 ].w ) * 0.5 - normal_depth[ pos1 ].w ), 1.0 ); const float depth_similarity_13 = min( 1.3 - abs( normal_depth[ pos1 ].w - normal_depth[ pos3 ].w ), 1.0 ); const float depth_similarity_14 = min( 1.3 - abs( normal_depth[ pos1 ].w - normal_depth[ pos4 ].w ), 1.0 ); const float normal_similarity_01 = min( 1.3 * max( dot( normal_depth[ pos0 ].xyz, normal_depth[ pos1 ].xyz ), 0.0 ),1.0 ); const float normal_similarity_a1 = min( 1.3 * max( dot( normalize( normal_depth[ pos0 ].xyz + normal_depth[ pos2 ].xyz ), normal_depth[ pos1 ].xyz ), 0.0 ), 1.0 ); const float normal_similarity_12 = min( 1.3 * max( dot( normal_depth[ pos1 ].xyz, normal_depth[ pos2 ].xyz ), 0.0 ), 1.0 ); const float normal_similarity_b1 = min( 1.3 * max( dot( normalize( normal_depth[ pos3 ].xyz + normal_depth[ pos4 ].xyz ), normal_depth[ pos1 ].xyz ), 0.0 ), 1.0 ); const float normal_similarity_13 = min( 1.3 * max( dot( normal_depth[ pos1 ].xyz, normal_depth[ pos3 ].xyz ), 0.0 ), 1.0 ); const float normal_similarity_14 = min( 1.3 * max( dot( normal_depth[ pos1 ].xyz, normal_depth[ pos4 ].xyz ), 0.0 ), 1.0 ); const float similarity_01 = instance_similarity_01 * depth_similarity_01 * normal_similarity_01; const float similarity_a1 = instance_similarity_02 * instance_similarity_01 * depth_similarity_a1 * normal_similarity_a1; const float similarity_12 = instance_similarity_12 * depth_similarity_12 * normal_similarity_12; const float similarity_b1 = instance_similarity_34 * instance_similarity_13 * depth_similarity_b1 * normal_similarity_b1; const float similarity_13 = instance_similarity_13 * depth_similarity_13 * normal_similarity_13; const float similarity_14 = instance_similarity_14 * depth_similarity_14 * normal_similarity_14; const float similarity_center = max( similarity_a1, similarity_b1 ); float highest = similarity_center; xy = 4; xy = ( highest < similarity_01 ) ? 0 : xy; highest = max( highest, similarity_01 ); xy = ( highest < similarity_12 ) ? 3 : xy; highest = max( highest, similarity_12 ); xy = ( highest < similarity_13 ) ? 1 : xy; highest = max( highest, similarity_13 ); xy = ( highest < similarity_14 ) ? 2 : xy; highest = max( highest, similarity_14 ); ίϯϐϡʔτγΣʔμͰ όΠϥςϥϧΦϑηοτΛٻΊΔ
67. if( src_offset.x != 0 && src_offset.y == 0 ) { linear = mix( p0, p1, ( ( ( bw >> 5 ) & 0x3 ) == 0 ) ? 0.0 : ( ( ( ( bw >> 5 ) & 0x3 ) == 1 ) ? 1.0 : 0.5 ) ); } else if( src_offset.x == 0 && src_offset.y != 0 ) { linear = mix( p0, p2, ( ( ( bw >> 3 ) & 0x3 ) == 0 ) ? 0.0 : ( ( ( ( bw >> 3 ) & 0x3 ) == 1 ) ? 1.0 : 0.5 ) ); } else if( src_offset.x != 0 && src_offset.y != 0 ) { if( ( bw & 0x7 ) == 0 ) linear = p0; else if( ( bw & 0x7 ) == 1 ) linear = p1; else if( ( bw & 0x7 ) == 2 ) linear = p2; else if( ( bw & 0x7 ) == 3 ) linear = p3; else linear = ( p0 + p1 + p2 + p3 ) * 0.25; } else { linear = p0; } όΠϥςϥϧΦϑηοτʹج͍ͮͯ ิؒํ๏Λมߋ͢Δ
69. 60FPS 1080p 2160p
71. ઢܗิؒ 2160p όΠϥςϥϧ
72. ઢܗิؒ 2160p όΠϥςϥϧ ͷϐΫηϧ͔͠ܭࢉ͍ͯ͠ͳׂ͍ʹ͸2160pʹ͍ۙֆ໘ʹͳΔ 1 4
74. ઢܗิؒ 2160p όΠϥςϥϧ ͷϐΫηϧ͔͠ܭࢉ͍ͯ͠ͳׂ͍ʹ͸2160pʹ͍ۙֆ໘ʹͳΔ 1 4
75. ͜ͷࠩ͸ԿͰੜͨ͡?
76. froxelαϯϓϦϯά Compute Pipeline γϟυ΢Ϛοϓͷੜ੒ Graphics Pipeline GόοϑΝͷੜ੒ Graphics Pipeline ௚઀ޫ Compute Pipeline τʔϯϚοϐϯά Compute Pipeline ΨϯϚ Compute Pipeline ΦΫϧʔδϣϯΧϦϯά Graphics Pipeline ϨϯζϑϨΞ Compute Pipeline ΞϯϏΤϯτΦΫϧʔδϣϯ Compute Pipeline froxelͷੜ੒ Compute Pipeline CoCͷੜ੒ Compute Pipeline ΦʔτϑΥʔΧε Compute Pipeline ϏωοςΟϯά Compute Pipeline ඃࣸքਂ౓ޮՌ Compute Pipeline ϒϧʔϜ Compute Pipeline ৭ۭؒͷม׵ Compute Pipeline ൒ಁ໌ͷղܾ Compute Pipeline ۭͷੜ੒ Compute Pipeline bilateral offsetͷੜ੒ Compute Pipeline όΠϥςϥϧΞοϓαϯϓϦϯά Compute Pipeline ͜ͷ෦෼͸ ઢܗิؒͱಉ͡ͳͷͰ ݪҼͰ͸ͳ͍
77. froxelαϯϓϦϯά Compute Pipeline γϟυ΢Ϛοϓͷੜ੒ Graphics Pipeline GόοϑΝͷੜ੒ Graphics Pipeline ௚઀ޫ Compute Pipeline τʔϯϚοϐϯά Compute Pipeline ΨϯϚ Compute Pipeline ΦΫϧʔδϣϯΧϦϯά Graphics Pipeline ϨϯζϑϨΞ Compute Pipeline ΞϯϏΤϯτΦΫϧʔδϣϯ Compute Pipeline froxelͷੜ੒ Compute Pipeline CoCͷੜ੒ Compute Pipeline ΦʔτϑΥʔΧε Compute Pipeline ϏωοςΟϯά Compute Pipeline ඃࣸքਂ౓ޮՌ Compute Pipeline ϒϧʔϜ Compute Pipeline ৭ۭؒͷม׵ Compute Pipeline ൒ಁ໌ͷղܾ Compute Pipeline ۭͷੜ੒ Compute Pipeline bilateral offsetͷੜ੒ Compute Pipeline όΠϥςϥϧΞοϓαϯϓϦϯά Compute Pipeline ͜ͷ෦෼͸ ઢܗิؒͱಉ͡ͳͷͰ ݪҼͰ͸ͳ͍ ΞοϓαϯϓϦϯάΛઢܗิؒʹͨ͠ΒͲ͏ͳΔ͔
78. ΞοϓαϯϓϦϯάࣗମͷ ܭࢉίετ͸ޡࠩ
79. froxelαϯϓϦϯά Compute Pipeline γϟυ΢Ϛοϓͷੜ੒ Graphics Pipeline GόοϑΝͷੜ੒ Graphics Pipeline ௚઀ޫ Compute Pipeline τʔϯϚοϐϯά Compute Pipeline ΨϯϚ Compute Pipeline ΦΫϧʔδϣϯΧϦϯά Graphics Pipeline ϨϯζϑϨΞ Compute Pipeline ΞϯϏΤϯτΦΫϧʔδϣϯ Compute Pipeline froxelͷੜ੒ Compute Pipeline CoCͷੜ੒ Compute Pipeline ΦʔτϑΥʔΧε Compute Pipeline ϏωοςΟϯά Compute Pipeline ඃࣸքਂ౓ޮՌ Compute Pipeline ϒϧʔϜ Compute Pipeline ৭ۭؒͷม׵ Compute Pipeline ൒ಁ໌ͷղܾ Compute Pipeline ۭͷੜ੒ Compute Pipeline bilateral offsetͷੜ੒ Compute Pipeline όΠϥςϥϧΞοϓαϯϓϦϯά Compute Pipeline ͜ͷ෦෼͸ ઢܗิؒͱಉ͡ͳͷͰ ݪҼͰ͸ͳ͍ Φϑηοτͷੜ੒Λ΍ΊͨΒͲ͏ͳΔ͔
80. όΠϥςϥϧΦϑηοτͷੜ੒΋ޡࠩ
81. froxelαϯϓϦϯά Compute Pipeline γϟυ΢Ϛοϓͷੜ੒ Graphics Pipeline GόοϑΝͷੜ੒ Graphics Pipeline ௚઀ޫ Compute Pipeline τʔϯϚοϐϯά Compute Pipeline ΨϯϚ Compute Pipeline ΦΫϧʔδϣϯΧϦϯά Graphics Pipeline ϨϯζϑϨΞ Compute Pipeline ΞϯϏΤϯτΦΫϧʔδϣϯ Compute Pipeline froxelͷੜ੒ Compute Pipeline CoCͷੜ੒ Compute Pipeline ΦʔτϑΥʔΧε Compute Pipeline ϏωοςΟϯά Compute Pipeline ඃࣸքਂ౓ޮՌ Compute Pipeline ϒϧʔϜ Compute Pipeline ৭ۭؒͷม׵ Compute Pipeline ൒ಁ໌ͷղܾ Compute Pipeline ۭͷੜ੒ Compute Pipeline bilateral offsetͷੜ੒ Compute Pipeline όΠϥςϥϧΞοϓαϯϓϦϯά Compute Pipeline ͜ͷ෦෼͸ ઢܗิؒͱಉ͡ͳͷͰ ݪҼͰ͸ͳ͍ ΋͔ͯ͠͠ݪҼ͸͍͔ͭ͜?
82. 2ϑϨʔϜ໨Ҏ߱͸ 1ϑϨʔϜ໨ͷgbufferΛ࠶ར༻
83. 2ϑϨʔϜ໨Ҏ߱͸ 1ϑϨʔϜ໨ͷgbufferΛ࠶ར༻ linearͱbilateralͷ ܭࢉ͕࣌ؒಉ͡ʹͳΔ
84. ௖఺୯Ґͷॲཧ ϥελϥΠζ ϐΫηϧ୯Ґͷॲཧ Graphics Pipeline ͜ͷ෦෼͸ϐΫηϧ਺ʹґଘ͠ͳ͍ ͜ͷ෦෼͸ॲཧ͢Δσʔλ͕ ϐΫηϧ਺ʹൺྫͯ͠૿͑Δ
85. ௖఺୯Ґͷॲཧ ϥελϥΠζ ϐΫηϧ୯Ґͷॲཧ Graphics Pipeline ϑϥάϝϯτγΣʔμͷத਎Λ શͯίϝϯτΞ΢τͯ͠ ϥελϥΠβʹΑΔ૿෼͚ͩΛऔΔ
87. ϥελϥΠβ3 : ϑϥάϝϯτγΣʔμ5
88. void main() { primitive_value p = read_primitive( push_constants.primitive, input_position, input_normal, input_tangent, input_texcoord, input_optflow, input_previous_position ); if( p.albedo.a <= 0.0 ) discard; const uint visibility_index = instance_resource_index[ push_constants.instance ].visibility; visibility_pool[ visibility_index ] = 1; const ivec2 image_pos = ivec2( gl_FragCoord.x, gl_FragCoord.y ); beginInvocationInterlockARB(); update_kplus_buffer16( global_uniforms.gbuffer, global_uniforms.depth, image_pos, p, gl_FragCoord.z, input_id.xy, vec4( 0.0, 0.0, 0.0, 0.0 ) ); endInvocationInterlockARB(); } gbufferͷੜ੒ͷϑϥάϝϯτγΣʔμ Πϝʔδʹॻ͘ ௖఺ଐੑΛಡΜͰ
89. void main() { primitive_value p = read_primitive( push_constants.primitive, input_position, input_normal, input_tangent, input_texcoord, input_optflow, input_previous_position ); if( p.albedo.a <= 0.0 ) discard; const uint visibility_index = instance_resource_index[ push_constants.instance ].visibility; visibility_pool[ visibility_index ] = 1; const ivec2 image_pos = ivec2( gl_FragCoord.x, gl_FragCoord.y ); beginInvocationInterlockARB(); update_kplus_buffer16( global_uniforms.gbuffer, global_uniforms.depth, image_pos, p, gl_FragCoord.z, input_id.xy, vec4( 0.0, 0.0, 0.0, 0.0 ) ); endInvocationInterlockARB(); } gbufferͷੜ੒ͷϑϥάϝϯτγΣʔμ ௖఺ଐੑΛಡΜͰ Πϝʔδʹॻ͘ ΄΅ಡΜͰॻ͚ͩ͘ = Memory Bound άϩʔόϧϝϞϦͷόε͕٧·͍ͬͯΔͷͰ͸
90. άϩʔόϧϝϞϦ ͕͜͜Α͘ौ଺͢Δ GPU
91. void main() { primitive_value p = read_primitive( push_constants.primitive, input_position, input_normal, input_tangent, input_texcoord, input_optflow, input_previous_position ); if( p.albedo.a <= 0.0 ) discard; const uint visibility_index = instance_resource_index[ push_constants.instance ].visibility; visibility_pool[ visibility_index ] = 1; const ivec2 image_pos = ivec2( gl_FragCoord.x, gl_FragCoord.y ); beginInvocationInterlockARB(); update_kplus_buffer16_reduced( global_uniforms.gbuffer, global_uniforms.depth, image_pos, p, gl_FragCoord.z, input_id.xy, vec4( 0.0, 0.0, 0.0, 0.0 ) ); endInvocationInterlockARB(); } image_posͷ஋Λݟͯ όΠϥςϥϧΦϑηοτͷܭࢉʹ࢖Θͳ͍஋͸ 4ϐΫηϧʹ͖ͭ1͔ͭ͠ॻ͔ͳ͍Α͏ʹมߋ
92. ޮՌͳ͠
93. άϩʔόϧϝϞϦ όε෯ 384bit͔Β512bit GPU
94. άϩʔόϧϝϞϦ όε෯ 384bit͔Β512bit GPU 1ϐΫηϧ=float16 bitͳͷͰ ϞʔτϯίʔυॱͰྡ઀͢Δෳ਺ͷϐΫηϧ͕ ·ͱΊͯϝϞϦʹॻ͔ΕΕΔ × 4 = 64 όεʹͱͬͯ͸Կͷઅ໿ʹ΋ͳ͍ͬͯͳ͍ͷͰ͸
95. ਂ౓ ๏ઢ ϝλϦοΫ ϥϑωε ΠϯελϯεID ϓϦϛςΟϒID ΦϓςΟΧϧϑϩʔ 0 16 32 48 64bit Ξϧϕυ ΞϧϑΝ ൃޫ ःṭ౓ ΦϓςΟΧϧϑϩʔ 2160pͰه࿥ 1080pͰه࿥ GόοϑΝΛόΠϥςϥϧΦϑηοτͷܭࢉʹඞཁͳ෺ͱ ͦ͏Ͱͳ͍෺ʹ෼ׂͯ͠ ޙऀΛ൒෼ͷαΠζͷΠϝʔδʹີʹه࿥͢Ε͹ ੑೳ͕ྑ͘ͳΔ͔΋?
96. ݱ୅ͷϦΞϧλΠϜϨϯμϦϯάͷܭࢉ࣌ؒ͸ ΄΅ϐΫηϧ਺ʹൺྫ ·ͱΊ ϐΫηϧ਺͕ଟ͗ͯ͢ܭࢉ͕ؒʹ߹Θͳ͍࣌͸ Ұ෦ͷϐΫηϧ͚ͩͰϨϯμϦϯάΛߦ͍ิؒ ิؒΛ࢖͏ͱ૬ରతʹGόοϑΝੜ੒࣌ͷ ϥελϥΠζͱϑϥάϝϯτγΣʔμ͕઎ΊΔ͕࣌ؒେ͖͘ͳΓ ࣍ͷॏཁͳ࠷దԽର৅ʹͳΔ
97. https://fadis.booth.pm/ ٕज़ॻయͰຊग़ͯ͠·͢ /&8 ϝογϡ γΣʔμʔ FADIS PRESS Ver. 1.0 ࣥචத