Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Let's make an Immersive Video with APMP
Search
Shingo Tamaki
August 10, 2025
Technology
0
19
Let's make an Immersive Video with APMP
This document summarizes a brief explanation of APMP, which was announced at WWDC25.
Shingo Tamaki
August 10, 2025
Tweet
Share
More Decks by Shingo Tamaki
See All by Shingo Tamaki
Use Gemini CLI from Claude Code as part of Sub Agent
tamaki
1
160
Firebase Studioで始めるモバイルアプリ開発入門
tamaki
0
18
Introduction to Claude Code Action
tamaki
0
690
AIエージェントを使ったiOSアプリ開発を試してみた
tamaki
0
120
沖縄モバイルアプリ開発勉強会#1
tamaki
0
120
iOSアプリ開発を始めよう
tamaki
0
200
詳解xcresult.pdf
tamaki
0
380
メルペイでのリグレッションテスト自動化推進のこれまでとこれから
tamaki
0
770
What do you want to test with UI Test v2
tamaki
2
900
Other Decks in Technology
See All in Technology
API提供者のためのMCPサーバー設計ガイド / MCP Server Design Guide for API Providers
yokawasa
0
240
analysis パッケージの仕組みの上でMulti linter with configを実現する / Go Conference 2025
k1low
1
250
VCC 2025 Write-up
bata_24
0
140
AWSのProductのLifecycleについて
stknohg
PRO
0
300
BtoBプロダクト開発の深層
16bitidol
0
140
Oracle Base Database Service 技術詳細
oracle4engineer
PRO
11
77k
kaigi_on_rails_2025_設計.pdf
nay3
8
4k
【新卒研修資料】LLM・生成AI研修 / Large Language Model・Generative AI
brainpadpr
21
14k
C# 14 / .NET 10 の新機能 (RC 1 時点)
nenonaninu
1
1.2k
Modern_Data_Stack最新動向クイズ_買収_AI_激動の2025年_.pdf
sagara
0
150
Azure SynapseからAzure Databricksへ 移行してわかった新時代のコスト問題!?
databricksjapan
0
110
バイブコーディングと継続的デプロイメント
nwiizo
2
360
Featured
See All Featured
Optimizing for Happiness
mojombo
379
70k
Intergalactic Javascript Robots from Outer Space
tanoku
273
27k
Six Lessons from altMBA
skipperchong
28
4k
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
194
16k
Rebuilding a faster, lazier Slack
samanthasiow
84
9.2k
4 Signs Your Business is Dying
shpigford
185
22k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
285
14k
What's in a price? How to price your products and services
michaelherold
246
12k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
132
19k
Raft: Consensus for Rubyists
vanstee
139
7.1k
The Pragmatic Product Professional
lauravandoore
36
6.9k
Learning to Love Humans: Emotional Interface Design
aarron
274
40k
Transcript
͓खݩͷಈըΛAPMPʹ ม͓ͯ͠खܰʹΠϚʔ γϰͳମݧΛ࡞Ζ͏ʂ
None
visionOSͰݟΔ͜ͱ͕Ͱ͖Δಈըͷछྨ
Apple Projected Media Profile
Apple Projected Media Profile 360°ಈըͷΑ͏ͳӨܕϝσΟΞΛѻ͏ͨΊʹISOBMFFͷ֦ு ͱͯ͠࡞ΒΕͨϓϩϑΝΠϧ
ISOBMFF (ISO Base Media File Format) ߏ ISOBMFF File ftyp
(File Type Box) ϑΝΠϧλΠϓɺϒϥϯυɺޓੑใ moov (Movie Box) - ϝλσʔλίϯςφ mvhd (Movie Header) શମͷ࣌ؒใ trak (Track Box) τϥοΫใ mvex (Movie Extends) ϑϥάϝϯτ༻֦ு tkhd ϔομʔ mdia ϝσΟΞ mdhd hdlr minf mdat (Media Data Box) ࣮ࡍͷϝσΟΞσʔλʢԻɾө૾ɾࣈນͳͲʣ ௨ৗϑΝΠϧͷେ෦ΛΊΔ free/skip (Free Space) ະ༻ྖҬ udta (User Data) Ϣʔβʔσʔλ ͦͷଞͷBox ֦ுɾΧελϜBox ɿBox ֊ߏΛ࣋ͪɺ֤Box size (4bytes) + type (4bytes) + data Ͱߏ͞ΕΔ ISOBMFFɺQTFFͱ • ISOBMFFϏσΦΦʔσΟΦͳͲ ͷϚϧνϝσΟΞσʔλΛؚΉϑΝΠ ϧͷҰൠతͳߏ͕ఆٛ͞Εͨࠃࡍ ج४ͷϑΥʔϚοτ • QTFF͕ϕʔε • MP4ISOBMFFʹج͍࣮ͮͨࡍͷ ϑΝΠϧܗࣜ
ISOBMFFͷ֦ுʹͭ ͍ͯ • Apple͕ۭؒϏσΦͷͨΊʹՃ͠ ֦ͨு: vexu(Video Extended Usage)
APMPʹରԠ͢ΔͱͲ͏ͳΔͷ͔ʁ • visonOSͷඪ४ͷࣸਅΞϓϦࣗͷΞϓϦ͔ΒΠϚʔγϒ ͳಈըͷ࠶ੜ͕Ͱ͖ΔΑ͏ʹͳΔ
APMPܗࣜͷಈըͷ࡞Γํ 1. CLIπʔϧʢavconvertʣ·ͨmacOSʢbetaʣͷػೳΛͬ ͯม͢Δ 2. طଘͷҰ෦ͷΧϝϥͰͬͨಈըࣗಈͰมͰ͖Δ 3. APIΛͬͯม͢Δ(Appleͷαϯϓϧίʔυ͋Γ)
1. avconvert
2. ࣗಈม 1. ·ͣInsta360͔Go ProΛߪೖ͠·͠ΐ͏ 2. Insta360 StudioͰMP4ग़ྗ͠·͢ 3. AirDropͰAVPʢvisionOS
26 betaʣૹΔ 4. visionOSͰ֘ϑΝΠϧΛ։͘ࡍʹม͢Δ͔Ͳ͏͔μΠΞ ϩά͕ग़ͯ͘ΔͷͰબͿ
3. APIΛͬͯม͢Δ AppleͷαϯϓϧίʔυProjectedMediaConversionΛ༻͠· ͢ https://developer.apple.com/documentation/AVFoundation/ converting-projected-video-to-apple-projected-media-profile
ॲཧͷྲྀΕ 1. ݩಈըಡΈࠐΈ 2. ύοΩϯάઃఆ 3. ѹॖϓϩύςΟઃఆ 4. Τϯίʔμʔͷઃఆ 5.
ॻ͖ग़͠
ύοΩϯά ཱମࢹʢstereoscopicʣͷ߹ύοΩ ϯά͕ҎԼͷ͍ͣΕ͔Ͱ͋Δ͔Λࢦఆ͠ ·͢ • SideBySide • OverUnder
ύοΩϯά if let viewPackingKind = projectedMediaMetadata.viewPackingKind { isFramePacked = true
if viewPackingKind.caseInsensitiveCompare("SideBySide") == .orderedSame { horizontalScale = 2.0 isSideBySide = true } else if viewPackingKind.caseInsensitiveCompare("OverUnder") == .orderedSame { verticalScale = 2.0 } } let eyeFrameSize = CGSize(width: sourceVideoFrameSize.width / horizontalScale, height: sourceVideoFrameSize.height / verticalScale) ... let cropRectDict = [ kCVImageBufferCleanApertureHorizontalOffsetKey: apertureHorizontalOffset, kCVImageBufferCleanApertureVerticalOffsetKey: apertureVerticalOffset, kCVImageBufferCleanApertureWidthKey: eyeFrameSize.width, kCVImageBufferCleanApertureHeightKey: eyeFrameSize.height ]
ѹॖϓϩύςΟͷઃఆ let MVHEVCVideoLayerIDs = [0, 1] let MVHEVCViewIDs = [0,
1] let MVHEVCLeftAndRightViewIDs = [0, 1] ... let stereoCompressionProperties: [CFString: Any] = [ kVTCompressionPropertyKey_MVHEVCVideoLayerIDs: MVHEVCVideoLayerIDs, kVTCompressionPropertyKey_MVHEVCViewIDs: MVHEVCViewIDs, kVTCompressionPropertyKey_MVHEVCLeftAndRightViewIDs: MVHEVCLeftAndRightViewIDs, kVTCompressionPropertyKey_HasLeftStereoEyeView: true, kVTCompressionPropertyKey_HasRightStereoEyeView: true ]
ѹॖϓϩύςΟͷઃఆʢ2ʣ let projectionKind = projectedMediaMetadata.projectionKind ... compressionProperties[kVTCompressionPropertyKey_ProjectionKind] = kCMFormatDescriptionProjectionKind_HalfEquirectangular ...
let baselineInMicrometers = UInt32(1000.0 * baselineInMillimeters) compressionProperties[kVTCompressionPropertyKey_StereoCameraBaseline] = baselineInMicrometers ... let encodedHorizontalFOV = UInt32(1000.0 * horizontalFOV) compressionProperties[kVTCompressionPropertyKey_HorizontalFieldOfView] = encodedHorizontalFOV
Τϯίʔμʔͷઃఆ let outputSettings: [String: Any] = [ AVVideoCodecKey: AVVideoCodecType.hevc, AVVideoWidthKey:
eyeFrameSize.width, AVVideoHeightKey: eyeFrameSize.height, AVVideoCompressionPropertiesKey: compressionProperties ]
ॻ͖ग़͠ for (layerID, eye) in zip(MVHEVCVideoLayerIDs, eyes) { let pixelBuffer
= try pixelBufferPool.makeMutablePixelBuffer() ... ΓऔΓൣғࢉग़ॲཧʢলུʣ ... CVBufferSetAttachment(imageBuffer, kCVImageBufferCleanApertureKey, cropRectDict as CFDictionary, CVAttachmentMode.shouldPropagate) VTSessionSetProperty(session, key: kVTPixelTransferPropertyKey_ScalingMode, value: kVTScalingMode_CropSourceToCleanAperture) pixelBuffer.withUnsafeBuffer { cvPixelBuffer in guard VTPixelTransferSessionTransferImage(session, from: imageBuffer, to: cvPixelBuffer) == noErr else { fatalError("Error during pixel transfer session for layer \(layerID)") } } // Create and append a tagged buffer for this eye. let tags: [CMTag] = [.videoLayerID(Int64(layerID)), .stereoView(eye)] taggedBuffers.append(.init(tags: tags, content: .pixelBuffer(.init(pixelBuffer)))) }
ॻ͖ग़͠ // Create and append a tagged buffer for this
eye. let tags: [CMTag] = [.videoLayerID(Int64(layerID)), .stereoView(eye)] taggedBuffers.append(.init(tags: tags, content: .pixelBuffer(.init(pixelBuffer))))
ग़དྷ্͕Γʂ
!
ࣗݾհ ۄ৴ޛ iOS DeveloperʢϑϦʔϥϯεʣ ԭೄࡏॅ ίϛϡχςΟ׆ಈ - try! Swift TokyoӡӦ
- ԭೄϞόΠϧΞϓϦ։ൃษڧձӡӦ
Okinawa.swiftͲ͏Ͱ͠ΐ͏ʁ
Ҏ্