Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Let's make an Immersive Video with APMP
Search
Shingo Tamaki
August 10, 2025
Technology
0
22
Let's make an Immersive Video with APMP
This document summarizes a brief explanation of APMP, which was announced at WWDC25.
Shingo Tamaki
August 10, 2025
Tweet
Share
More Decks by Shingo Tamaki
See All by Shingo Tamaki
Use Gemini CLI from Claude Code as part of Sub Agent
tamaki
1
180
Firebase Studioで始めるモバイルアプリ開発入門
tamaki
0
20
Introduction to Claude Code Action
tamaki
0
700
AIエージェントを使ったiOSアプリ開発を試してみた
tamaki
0
130
沖縄モバイルアプリ開発勉強会#1
tamaki
0
120
iOSアプリ開発を始めよう
tamaki
0
200
詳解xcresult.pdf
tamaki
0
390
メルペイでのリグレッションテスト自動化推進のこれまでとこれから
tamaki
0
770
What do you want to test with UI Test v2
tamaki
2
910
Other Decks in Technology
See All in Technology
dbtとBigQuery MLで実現する リクルートの営業支援基盤のモデル開発と保守運用
recruitengineers
PRO
3
150
ローカルLLMとLINE Botの組み合わせ その2(EVO-X2でgpt-oss-120bを利用) / LINE DC Generative AI Meetup #7
you
PRO
0
140
事業開発におけるDify活用事例
kentarofujii
4
1.2k
サイバーエージェント流クラウドコスト削減施策「みんなで金塊堀太郎」
kurochan
4
2.2k
だいたい分かった気になる 『SREの知識地図』 / introduction-to-sre-knowledge-map-book
katsuhisa91
PRO
0
770
ソフトウェアエンジニアの生成AI活用と、これから
lycorptech_jp
PRO
0
790
Introduction to Sansan for Engineers / エンジニア向け会社紹介
sansan33
PRO
5
43k
Introduction to Sansan Meishi Maker Development Engineer
sansan33
PRO
0
310
Azureコストと向き合った、4年半のリアル / Four and a half years of dealing with Azure costs
aeonpeople
1
250
ソースを読むプロセスの例
sat
PRO
15
9.7k
生成AIを安心して活用するために──「情報セキュリティガイドライン」策定とポイント
gree_tech
PRO
0
140
RDS の負荷が高い場合に AWS で取りうる具体策 N 連発/a-series-of-specific-countermeasures-available-on-aws-when-rds-is-under-high-load
emiki
7
4.5k
Featured
See All Featured
Typedesign – Prime Four
hannesfritz
42
2.8k
Speed Design
sergeychernyshev
32
1.2k
Building an army of robots
kneath
306
46k
Measuring & Analyzing Core Web Vitals
bluesmoon
9
630
Why You Should Never Use an ORM
jnunemaker
PRO
59
9.6k
Why Our Code Smells
bkeepers
PRO
340
57k
How to Ace a Technical Interview
jacobian
280
24k
Improving Core Web Vitals using Speculation Rules API
sergeychernyshev
21
1.2k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
367
27k
10 Git Anti Patterns You Should be Aware of
lemiorhan
PRO
657
61k
The Cult of Friendly URLs
andyhume
79
6.6k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
46
7.7k
Transcript
͓खݩͷಈըΛAPMPʹ ม͓ͯ͠खܰʹΠϚʔ γϰͳମݧΛ࡞Ζ͏ʂ
None
visionOSͰݟΔ͜ͱ͕Ͱ͖Δಈըͷछྨ
Apple Projected Media Profile
Apple Projected Media Profile 360°ಈըͷΑ͏ͳӨܕϝσΟΞΛѻ͏ͨΊʹISOBMFFͷ֦ு ͱͯ͠࡞ΒΕͨϓϩϑΝΠϧ
ISOBMFF (ISO Base Media File Format) ߏ ISOBMFF File ftyp
(File Type Box) ϑΝΠϧλΠϓɺϒϥϯυɺޓੑใ moov (Movie Box) - ϝλσʔλίϯςφ mvhd (Movie Header) શମͷ࣌ؒใ trak (Track Box) τϥοΫใ mvex (Movie Extends) ϑϥάϝϯτ༻֦ு tkhd ϔομʔ mdia ϝσΟΞ mdhd hdlr minf mdat (Media Data Box) ࣮ࡍͷϝσΟΞσʔλʢԻɾө૾ɾࣈນͳͲʣ ௨ৗϑΝΠϧͷେ෦ΛΊΔ free/skip (Free Space) ະ༻ྖҬ udta (User Data) Ϣʔβʔσʔλ ͦͷଞͷBox ֦ுɾΧελϜBox ɿBox ֊ߏΛ࣋ͪɺ֤Box size (4bytes) + type (4bytes) + data Ͱߏ͞ΕΔ ISOBMFFɺQTFFͱ • ISOBMFFϏσΦΦʔσΟΦͳͲ ͷϚϧνϝσΟΞσʔλΛؚΉϑΝΠ ϧͷҰൠతͳߏ͕ఆٛ͞Εͨࠃࡍ ج४ͷϑΥʔϚοτ • QTFF͕ϕʔε • MP4ISOBMFFʹج͍࣮ͮͨࡍͷ ϑΝΠϧܗࣜ
ISOBMFFͷ֦ுʹͭ ͍ͯ • Apple͕ۭؒϏσΦͷͨΊʹՃ͠ ֦ͨு: vexu(Video Extended Usage)
APMPʹରԠ͢ΔͱͲ͏ͳΔͷ͔ʁ • visonOSͷඪ४ͷࣸਅΞϓϦࣗͷΞϓϦ͔ΒΠϚʔγϒ ͳಈըͷ࠶ੜ͕Ͱ͖ΔΑ͏ʹͳΔ
APMPܗࣜͷಈըͷ࡞Γํ 1. CLIπʔϧʢavconvertʣ·ͨmacOSʢbetaʣͷػೳΛͬ ͯม͢Δ 2. طଘͷҰ෦ͷΧϝϥͰͬͨಈըࣗಈͰมͰ͖Δ 3. APIΛͬͯม͢Δ(Appleͷαϯϓϧίʔυ͋Γ)
1. avconvert
2. ࣗಈม 1. ·ͣInsta360͔Go ProΛߪೖ͠·͠ΐ͏ 2. Insta360 StudioͰMP4ग़ྗ͠·͢ 3. AirDropͰAVPʢvisionOS
26 betaʣૹΔ 4. visionOSͰ֘ϑΝΠϧΛ։͘ࡍʹม͢Δ͔Ͳ͏͔μΠΞ ϩά͕ग़ͯ͘ΔͷͰબͿ
3. APIΛͬͯม͢Δ AppleͷαϯϓϧίʔυProjectedMediaConversionΛ༻͠· ͢ https://developer.apple.com/documentation/AVFoundation/ converting-projected-video-to-apple-projected-media-profile
ॲཧͷྲྀΕ 1. ݩಈըಡΈࠐΈ 2. ύοΩϯάઃఆ 3. ѹॖϓϩύςΟઃఆ 4. Τϯίʔμʔͷઃఆ 5.
ॻ͖ग़͠
ύοΩϯά ཱମࢹʢstereoscopicʣͷ߹ύοΩ ϯά͕ҎԼͷ͍ͣΕ͔Ͱ͋Δ͔Λࢦఆ͠ ·͢ • SideBySide • OverUnder
ύοΩϯά if let viewPackingKind = projectedMediaMetadata.viewPackingKind { isFramePacked = true
if viewPackingKind.caseInsensitiveCompare("SideBySide") == .orderedSame { horizontalScale = 2.0 isSideBySide = true } else if viewPackingKind.caseInsensitiveCompare("OverUnder") == .orderedSame { verticalScale = 2.0 } } let eyeFrameSize = CGSize(width: sourceVideoFrameSize.width / horizontalScale, height: sourceVideoFrameSize.height / verticalScale) ... let cropRectDict = [ kCVImageBufferCleanApertureHorizontalOffsetKey: apertureHorizontalOffset, kCVImageBufferCleanApertureVerticalOffsetKey: apertureVerticalOffset, kCVImageBufferCleanApertureWidthKey: eyeFrameSize.width, kCVImageBufferCleanApertureHeightKey: eyeFrameSize.height ]
ѹॖϓϩύςΟͷઃఆ let MVHEVCVideoLayerIDs = [0, 1] let MVHEVCViewIDs = [0,
1] let MVHEVCLeftAndRightViewIDs = [0, 1] ... let stereoCompressionProperties: [CFString: Any] = [ kVTCompressionPropertyKey_MVHEVCVideoLayerIDs: MVHEVCVideoLayerIDs, kVTCompressionPropertyKey_MVHEVCViewIDs: MVHEVCViewIDs, kVTCompressionPropertyKey_MVHEVCLeftAndRightViewIDs: MVHEVCLeftAndRightViewIDs, kVTCompressionPropertyKey_HasLeftStereoEyeView: true, kVTCompressionPropertyKey_HasRightStereoEyeView: true ]
ѹॖϓϩύςΟͷઃఆʢ2ʣ let projectionKind = projectedMediaMetadata.projectionKind ... compressionProperties[kVTCompressionPropertyKey_ProjectionKind] = kCMFormatDescriptionProjectionKind_HalfEquirectangular ...
let baselineInMicrometers = UInt32(1000.0 * baselineInMillimeters) compressionProperties[kVTCompressionPropertyKey_StereoCameraBaseline] = baselineInMicrometers ... let encodedHorizontalFOV = UInt32(1000.0 * horizontalFOV) compressionProperties[kVTCompressionPropertyKey_HorizontalFieldOfView] = encodedHorizontalFOV
Τϯίʔμʔͷઃఆ let outputSettings: [String: Any] = [ AVVideoCodecKey: AVVideoCodecType.hevc, AVVideoWidthKey:
eyeFrameSize.width, AVVideoHeightKey: eyeFrameSize.height, AVVideoCompressionPropertiesKey: compressionProperties ]
ॻ͖ग़͠ for (layerID, eye) in zip(MVHEVCVideoLayerIDs, eyes) { let pixelBuffer
= try pixelBufferPool.makeMutablePixelBuffer() ... ΓऔΓൣғࢉग़ॲཧʢলུʣ ... CVBufferSetAttachment(imageBuffer, kCVImageBufferCleanApertureKey, cropRectDict as CFDictionary, CVAttachmentMode.shouldPropagate) VTSessionSetProperty(session, key: kVTPixelTransferPropertyKey_ScalingMode, value: kVTScalingMode_CropSourceToCleanAperture) pixelBuffer.withUnsafeBuffer { cvPixelBuffer in guard VTPixelTransferSessionTransferImage(session, from: imageBuffer, to: cvPixelBuffer) == noErr else { fatalError("Error during pixel transfer session for layer \(layerID)") } } // Create and append a tagged buffer for this eye. let tags: [CMTag] = [.videoLayerID(Int64(layerID)), .stereoView(eye)] taggedBuffers.append(.init(tags: tags, content: .pixelBuffer(.init(pixelBuffer)))) }
ॻ͖ग़͠ // Create and append a tagged buffer for this
eye. let tags: [CMTag] = [.videoLayerID(Int64(layerID)), .stereoView(eye)] taggedBuffers.append(.init(tags: tags, content: .pixelBuffer(.init(pixelBuffer))))
ग़དྷ্͕Γʂ
!
ࣗݾհ ۄ৴ޛ iOS DeveloperʢϑϦʔϥϯεʣ ԭೄࡏॅ ίϛϡχςΟ׆ಈ - try! Swift TokyoӡӦ
- ԭೄϞόΠϧΞϓϦ։ൃษڧձӡӦ
Okinawa.swiftͲ͏Ͱ͠ΐ͏ʁ
Ҏ্