【CEDEC2024】Application of USD to Game Assets

This is the English translation of CEDEC2024 "Application of USD to Game Assets".

This section explains the technique of using USD for game assets, for which json and other formats are often used.
It also explains the advantages and disadvantages of using USD for game assets.

1. Application of USD to Game Assets Bandai Namco Studios Inc. Wataru Tada
2. Self-introduction Past Projects ・Ace Combat 6 ・Ace Combat: Assault Horizon ・Tales Of Card Evolve ・THE IDOLM@STER MILLION LIVE! ・Pokkén Tournament ・New Pokémon Snap Past talks ・CEDEC2011 Continuous integration of ACE COMBAT ASSAULT HORIZON ・CEDEC2011 In-game camera production example in “ACE COMBAT ASSAULT HORIZON” in-game camera production example ・CEDEC2015 Game editor design and implementation in Pokkén Tournament ・CEDEC2019 Basic design of game editor and stable and fast asset management in console video game development ・CEDEC2019 Implementation of Light Baker using DirectX Raytracing ・CEDEC2021 New Pokémon Snap Lighting Overview ・CEDEC2022 Application of MaterialX and Standard Surface to Video Game Graphics ・CEDEC2023 Wide gamut texture workflow using DCC tools Wataru Tada
3. Contents • Overview • Adding USD Libraries to the tool • Tool Example • Using the USD Toolset • Conclusion
4. USD(Universal Scene Description) • USD is a file format and control library for scene graph description developed by Pixar. • Although USD is suitable for graphics applications, it is designed as a general-purpose file format and is not limited to graphics applications. • This talk explains how to apply USD to game assets, where JSON and other file formats are often used. https://openusd.org
5. Game asset flow using JSON Game Runtime Development environment (Game Editor, etc.) { "Mission": [ { "name":"Mission1", “Comment”:“Test Mission", "ID":1 } ] } Binary file JSON Binary file
6. Game asset flow with USD Development environment (Game Editor, etc.) #usda 1.0 def "Mission" { def "Mission_1" { custom string Comment = " Test Mission " custom int ID = 1 } } Binary file USD Game Runtime Binary file
7. Advantages of using USD for game assets • Game assets can use USD features • Composition • Interoperability with DCC tools • USD Toolset • …
8. Composition • Composition can compose multiple USD files. • Asset Converter composes USD and outputs to the binary file. Game Asset A.USD Game Asset B.USD Game Asset C.USD Game Asset Composition Binary file Asset Converter
9. Using Composition • Compositions can be used if game assets are file-separated according to the composition arcs. • subLayers • inherits • variantSets • references • payloads • specializes Character common parameters.USD Individual character parameters.USD references Composition Example
10. Interoperability with DCC tools • USD Input/Export functionality in DCC tools can handle game assets. Houdini Solaris, Maya's USD Importer/Exporter, etc. DCC tool Game Asset USD
11. USD Toolset • USD offers the USD Toolset as an official tool. • The USD Toolset allows the use of game assets created in USD. usdview usdcat https://github.com/PixarAnimationStudios/OpenUSD/tree/release/pxr/usd/bin
12. Contents • Overview • Adding USD Libraries to the tool • Tool Example • Using the USD Toolset • Conclusion
13. USD library • USD provides a library to handle USD as a file format (.usd, etc.) • We use the USD 24.05 Windows version built as a Monolithic Library. (No console porting) →USD is only used in tools, not game runtime. https://github.com/PixarAnimationStudios/OpenUSD USD Windows Development Environment Game Runtime Binary file
14. USD access from C# • The USD library supports C++ and Python, but we decided to access USD from C#. →We created a C++ DLL for marshalling the USD library and a C# library to handle each USD class as an object. (For accessing USD in C#, some examples, like usd-unity-sdk, provide a parsing mechanism for the USD library.) USD Access Library C# (DLL) C++ (DLL) usd_ms.dll C# Tools Library for marshalling Interface
15. C# Tools • The USD Access library allows reading and writing USD files from C# tools. • USD loads what is needed and when needed, but the USD Access library loads all attributes at once when loading USD. C# Object (C# classes are defined for each USD schema) USD
16. USD Serializer/Deserializer • When outputting C# objects to USD, we consider the USD schema design for each asset type. • When the number of asset types increases, we decide to create a serializer to simplify the USD schema design. →Using reflection, C# objects are automatically serialized to USD. C# Object USD Serializer USD schema design C# Object UsdGeomXform? UsdGeomScope? UsdPrim?
17. C# Object • C# objects are represented using Prim. • Recording the class name in the USD attribute allows the class to be recovered from the Attribute when deserializing. →This function is similar to $type in Json.NET. C# USD Class name
18. Built-in types • USD provides basic types so C# types can be converted. C# USD bool bool char uchar int,sbyte,short int uint,byte,ushort uint long,decimal,nint int64 ulong,nuint uint64 float float double double string string C# USD bool[] pxr::VtArray<bool> char[] pxr::VtArray<uchar> int[],sbyte[],short[] pxr::VtArray<int> uint[],byte[],ushort[] pxr::VtArray<uint> long[],decimal[],nint[] pxr::VtArray<int64> ulong[],nuint[] pxr::VtArray<uint64> float[] pxr::VtArray<float> double[] pxr::VtArray<double> string[] pxr::VtArray<string> Built-in types Array
19. Type not provided in USD • Since USD does not provide Prim arrays, C# object arrays are represented by adding sequential numbers to Prim's SdfPath. (It is possible to represent an array by using Relationship.) • We create our own Prim for types not provided in USD. List、ObservableCollection、Dictionary、HashSet... Array type C# USD Value_Number C# Dictionary type Key Value USD Reserved words (. and /) cannot be used in SdfPath
20. Serialization example • Serializers can serialize C# objects to USD. • Use the serializer for simple assets that do not require USD schema design. (Using a serializer reduces interoperability with DCC tools compared to manually design the USD schema.) C# Object USD
21. Contents • Overview • Adding USD Libraries to the tool • Tool Example • Using the USD Toolset • Conclusion
22. USD support for tools • Now that we can read and write USD from C# and provide a serializer, tools can use USD. • Excel • Model Converter • Collision Converter • Game Editor
23. Excel • Excel (spreadsheets) when dealing with large amounts of data, such as game parameters. • C# tools read Excel files and output USD files. USD file Binary file Excel
24. Game Parameters Excel • Excel can represent rows as Prim and columns as Attribute (or Custom Data) in USD. USD (Parameters are represented by Attribute) Excel USD (Parameters are represented in Custom Data) Parameter
25. Game Parameters USD • Attribute and Custom Data can also contain JSON. (USD features such as composition cannot be used for strings that are JSON.) USD Excel Custom Data with JSON
26. Game Parameters USD • Setting up a schema in USD improves interoperability with DCC tools. (DCC tools can create functions using attributes of the schema) • UsdGeomScope can be used for generic parameters. USD Excel UsdGeomScope
27. Game Parameters USD • UsdGeomXform can be used if game parameters contain position. USD Excel UsdGeomXform Position
28. Game Parameters USD USD (using serializer) JSON • The USD serializer produces JSON-like output.
29. Composition (references) • Compositions allow game parameters to refer to other parameters. • For example, you can set common parameters for characters, and then individual character information can refer to the common parameters. Character common parameters Individual character parameters USD file USD file reference Binary file reference
30. USD with references Character common parameters USD Individual character parameters USD Composition result (preview in usdcat) Binary file reference Character common parameter values are referenced
31. Camera Bookmark Excel • Outputting an Excel file containing camera information to a USD file allows DCC tools and game editors to reference the camera information in the Excel file. →This can be used to automatically take screenshots, etc. Camera Bookmark Excel DCC tool Game Editor USD file
32. Example of using camera bookmarks in Houdini • Asset Reference node applies camera using camera bookmark. UsdGeomCamera Camera position Houdini Camera Bookmark Excel USD
33. Lighting Presets Excel • By writing HDRI and sunlight information in Excel as lighting for Lookdev, the Excel lighting information can be referenced from DCC tools and game editors via USD. Lighting Presets Excel DCC tool Game Editor USD file
34. Example of using lighting presets in Houdini • Asset Reference node applies lights using lighting presets. • Houdini cannot change the lighting preset Excel, so artists can prevent unintended lighting changes. HDRI (UsdLuxDomeLight) Sunlight(UsdLuxDistantLight) USD Lighting Presets Excel HDRI (UsdLuxDomeLight) Sunlight(UsdLuxDistantLight) Houdini
35. Model converter • By outputting the USD from Houdini and converting it to a binary file in the Model Converter, the USD can be used as an exporter in Houdini. →The Model Converter reads the USD and puts the UsdGeomMesh into a binary file. Houdini USD Model converter Game faceVertexCounts is set to 3 using the divide node Binary file
36. Using Custom Data • Use Custom Data to embed parameters not included in the USD schema, such as game-specific parameters, into USD. →By using the Model Converter to embed Custom Data into binaries, you can use parameters set in Houdini from the game. Set RenderPass (rendering path) string in Custom Data using ConfigurePrimitive node Games are rendered using the model rendering path set up in Houdini. USD
37. Data Debugging • If USD is used for model data, data debugging can be done in Houdini's Scene Graph Tree and usdview. Houdini usdview
38. Collision converter • Using USD for collision assets makes them editable assets in game editors and DCC tools. Collision Asset USD Houdini Game Editor Maya Binary file The collision model uses UsdGeomMesh Parameters used in the game are set in Custom Data (e.g., collision types such as grass and ground) faceVertexCounts set to 3 for polygons
39. File splitting for collision asset USD • Using compositions, collision asset USD can be split into multiple files, allowing multiple people to edit simultaneously. Houdini Collision Asset USD Composition Result Composition with sublayers Binary file
40. Combination of multiple tools • USD can be output from multiple tools, so USD parameters output from DCC tools can be overwritten by USD output from Excel. Houdini USD Excel USD Composition Result Binary file Overwrite collision type values with Excel values
41. Level Asset USD • If USD is available in the level editor, USD can be applied to level assets Level Asset USD Level Editor Preview in usdview
42. Level Asset USD • The level structure is a tree structure of sublevels and Entity, so use the tree structure of USD. Root Level Sub Level Sub Level Entity Entity Level Tree Entity Entity USD Layer Layer Layer Prim Prim Prim Prim Level data USD
43. Sub Level Reference • Composition reference features can reference sub levels of other files. Sub Level
44. Game Parameter Reference • Compositions can refer to game parameters in USD from level data in USD. Game parameter USD Composition Result Level Editor Excel Level data USD reference Binary file
45. Contents • Overview • Adding USD Libraries to the tool • Tool Example • Using the USD Toolset • Conclusion
46. USD Toolset • USD offers the USD Toolset (official USD tools), which can be used for data debugging, etc. Name Overview usdedit Editing with a text editor usdcat Checking .usda usddiff Diff display usdview Hierarchy preview usdrecord usdresolve usdtree Hierarchical text display Name Overview usdzip Creating .usdz usdchecker usdstitch Merging multiple USDs usdstitchclips usddumpcrate sdfdump Debugging SdfPath and TfToken sdffilter
47. Using .usda with usdedit or usdcat • Convert .usd files to .usda files using usdedit or usdcat when you want to do a string search within an asset or check variables that cannot be displayed in usdview. usda usdview Custom Data is not partially displayed usdcat
48. Merge using usdstich • usdstitch can be used to merge multiple USDs. • The order of the arguments determines the priority of merging USDs. usdstich test1.usda priority test2.usda priority add overwrite add
49. Diff display using usddiff • Register usddif with Perforce's Diff tool to see USD object diffs. Perforce usddiff
50. Contents • Overview • Adding USD Libraries to the tool • Tool Example • Using the USD Toolset • Conclusion
51. Disadvantages of using USD for game assets • Need to develop an environment such as python. • Assets are subject to the USD specification restrictions. • Since USD is developed based on the Linux version, problems will likely occur when building for Windows and using MSVC. → Report on github as needed. https://github.com/PixarAnimationStudios/OpenUSD
52. Assets are subject to the USD specification restrictions • Prim must be set to a unique path (string), so ID-based assets require a path-generation process. • USD is difficult to represent things that do not exist in the USD schema. For example, there are no UsdGeomLines, so use UsdGeomBasisCurves. (Custom schemas are difficult to use because they reduce forward compatibility of assets and interoperability of DCC tools.) • Variant can be used for the function of switching something, but it is difficult to get the information of a Prim that is not selected in Variant (it is necessary to switch Variant).
53. Conclusion • By creating tools using the USD library, game assets can be used as USD. • Using game assets as USD has some advantages, such as the ability to use compositions and interoperability with DCC tools, but it also has disadvantages, such as assets being subject to the restrictions of the USD specifications. • Game assets that are suitable for USD include assets that perform compositions and assets that interoperate with DCC tools. • Conventional file formats such as JSON and XML are suitable for assets that have little advantage of using USD, such as simple string list files.
54. References • https://openusd.org • https://github.com/PixarAnimationStudios/OpenUSD • Academy Software Foundation Slack • [CEDEC 2017] Introduction to Pixar USD: Building a New Content Pipeline • [CEDEC 2022] Example of USD in “Gran Turismo 7” Development
55. Questions?