Share
Embed
Let's make an Immersive Video with APMP
This document summarizes a brief explanation of APMP, which was announced at WWDC25.
Shingo Tamaki
August 10, 2025
Transcript
25 slides
-
ISOBMFF (ISO Base Media File Format) ߏ ISOBMFF File ftyp
(File Type Box) ϑΝΠϧλΠϓɺϒϥϯυɺޓੑใ moov (Movie Box) - ϝλσʔλίϯςφ mvhd (Movie Header) શମͷ࣌ؒใ trak (Track Box) τϥοΫใ mvex (Movie Extends) ϑϥάϝϯτ༻֦ு tkhd ϔομʔ mdia ϝσΟΞ mdhd hdlr minf mdat (Media Data Box) ࣮ࡍͷϝσΟΞσʔλʢԻɾө૾ɾࣈນͳͲʣ ௨ৗϑΝΠϧͷେ෦ΛΊΔ free/skip (Free Space) ະ༻ྖҬ udta (User Data) Ϣʔβʔσʔλ ͦͷଞͷBox ֦ுɾΧελϜBox ɿBox ֊ߏΛ࣋ͪɺ֤Box size (4bytes) + type (4bytes) + data Ͱߏ͞ΕΔ ISOBMFFɺQTFFͱ • ISOBMFFϏσΦΦʔσΟΦͳͲ ͷϚϧνϝσΟΞσʔλΛؚΉϑΝΠ ϧͷҰൠతͳߏ͕ఆٛ͞Εͨࠃࡍ ج४ͷϑΥʔϚοτ • QTFF͕ϕʔε • MP4ISOBMFFʹج͍࣮ͮͨࡍͷ ϑΝΠϧܗࣜ -
2. ࣗಈม 1. ·ͣInsta360͔Go ProΛߪೖ͠·͠ΐ͏ 2. Insta360 StudioͰMP4ग़ྗ͠·͢ 3. AirDropͰAVPʢvisionOS
26 betaʣૹΔ 4. visionOSͰ֘ϑΝΠϧΛ։͘ࡍʹม͢Δ͔Ͳ͏͔μΠΞ ϩά͕ग़ͯ͘ΔͷͰબͿ -
ύοΩϯά if let viewPackingKind = projectedMediaMetadata.viewPackingKind { isFramePacked = true
if viewPackingKind.caseInsensitiveCompare("SideBySide") == .orderedSame { horizontalScale = 2.0 isSideBySide = true } else if viewPackingKind.caseInsensitiveCompare("OverUnder") == .orderedSame { verticalScale = 2.0 } } let eyeFrameSize = CGSize(width: sourceVideoFrameSize.width / horizontalScale, height: sourceVideoFrameSize.height / verticalScale) ... let cropRectDict = [ kCVImageBufferCleanApertureHorizontalOffsetKey: apertureHorizontalOffset, kCVImageBufferCleanApertureVerticalOffsetKey: apertureVerticalOffset, kCVImageBufferCleanApertureWidthKey: eyeFrameSize.width, kCVImageBufferCleanApertureHeightKey: eyeFrameSize.height ] -
ѹॖϓϩύςΟͷઃఆ let MVHEVCVideoLayerIDs = [0, 1] let MVHEVCViewIDs = [0,
1] let MVHEVCLeftAndRightViewIDs = [0, 1] ... let stereoCompressionProperties: [CFString: Any] = [ kVTCompressionPropertyKey_MVHEVCVideoLayerIDs: MVHEVCVideoLayerIDs, kVTCompressionPropertyKey_MVHEVCViewIDs: MVHEVCViewIDs, kVTCompressionPropertyKey_MVHEVCLeftAndRightViewIDs: MVHEVCLeftAndRightViewIDs, kVTCompressionPropertyKey_HasLeftStereoEyeView: true, kVTCompressionPropertyKey_HasRightStereoEyeView: true ] -
ѹॖϓϩύςΟͷઃఆʢ2ʣ let projectionKind = projectedMediaMetadata.projectionKind ... compressionProperties[kVTCompressionPropertyKey_ProjectionKind] = kCMFormatDescriptionProjectionKind_HalfEquirectangular ...
let baselineInMicrometers = UInt32(1000.0 * baselineInMillimeters) compressionProperties[kVTCompressionPropertyKey_StereoCameraBaseline] = baselineInMicrometers ... let encodedHorizontalFOV = UInt32(1000.0 * horizontalFOV) compressionProperties[kVTCompressionPropertyKey_HorizontalFieldOfView] = encodedHorizontalFOV -
Τϯίʔμʔͷઃఆ let outputSettings: [String: Any] = [ AVVideoCodecKey: AVVideoCodecType.hevc, AVVideoWidthKey:
eyeFrameSize.width, AVVideoHeightKey: eyeFrameSize.height, AVVideoCompressionPropertiesKey: compressionProperties ] -
ॻ͖ग़͠ for (layerID, eye) in zip(MVHEVCVideoLayerIDs, eyes) { let pixelBuffer
= try pixelBufferPool.makeMutablePixelBuffer() ... ΓऔΓൣғࢉग़ॲཧʢলུʣ ... CVBufferSetAttachment(imageBuffer, kCVImageBufferCleanApertureKey, cropRectDict as CFDictionary, CVAttachmentMode.shouldPropagate) VTSessionSetProperty(session, key: kVTPixelTransferPropertyKey_ScalingMode, value: kVTScalingMode_CropSourceToCleanAperture) pixelBuffer.withUnsafeBuffer { cvPixelBuffer in guard VTPixelTransferSessionTransferImage(session, from: imageBuffer, to: cvPixelBuffer) == noErr else { fatalError("Error during pixel transfer session for layer \(layerID)") } } // Create and append a tagged buffer for this eye. let tags: [CMTag] = [.videoLayerID(Int64(layerID)), .stereoView(eye)] taggedBuffers.append(.init(tags: tags, content: .pixelBuffer(.init(pixelBuffer)))) } -
ॻ͖ग़͠ // Create and append a tagged buffer for this
eye. let tags: [CMTag] = [.videoLayerID(Int64(layerID)), .stereoView(eye)] taggedBuffers.append(.init(tags: tags, content: .pixelBuffer(.init(pixelBuffer)))) -
ࣗݾհ ۄ৴ޛ iOS DeveloperʢϑϦʔϥϯεʣ ԭೄࡏॅ ίϛϡχςΟ׆ಈ - try! Swift TokyoӡӦ
- ԭೄϞόΠϧΞϓϦ։ൃษڧձӡӦ