Share
Embed
Macとオーディオ再生 2024/11/02
Yusuke Ito
November 02, 2024
Transcript
29 slides
-
About me • Yusuke Ito • Web (TypeScript, React, Node.js)
macOS ωΠςΟϒΞϓϦ (Minutes) iOS ωΠςΟϒΞϓϦ (App Store Best of 2014) • ϝΠϯWeb͕ͩɺΦʔσΟΦػثͷ։ൃɾ +macOSΞϓϦ https://audio.current.directory/ -
PCM (LPCM) Linear Pulse Code Modulation Ϗοτਂ=ৼ෯ (16, 24bit) αϯϓϦϯάप(1ඵ͋ͨΓͷׂ)
44.1kHz(CD) 48kHz, 96kHz (Ի੍ָ࡞) Իͷੜσʔλ 0 -32768 32767 -
LPCMͷσʔλྔ • 44.1kHz (CD) αϯϓϦϯάप • 16bit (CD) Ϗοτਂ •
3 (180ඵ) • 44100 (sample/sec) × 2 (byte, 16bit) × 2 (channel, L R) × 180 (secs) • = 31,752,000 (bytes) ≈ 30 MB (megabytes) -
ίϯςφͱίʔσοΫ ίϯςφ: QuickTime (.mov) MPEG-4 (.m4a, .m4v) Ի AAC ө૾
H.264 ϝλσʔλ ΞʔςΟετ໊… ΞϧόϜ໊… ϝλσʔλ ίʔσοΫ: H.264 AAC Apple Lossless MP3, ATRAC (MD) JPEG PNG ϑΝΠϧͷ෦ -
Mixing & Playback ԻσʔλͷྲྀΕ Χʔωϧ ϛΩαʔ σόΠε 🔊 App A
App B γεςϜܯࠂԻ App C ഉଞϞʔυ OR (αϯϓϦϯάपม) 96kHz 48kHz 44.1kHz 96kHz LPCM LPCM -
Audio APIs on Mac • AVFoundation: ؆୯ʹ͑ΔΫϥε܈ • AVAudioPlayerͳͲ, ϑΝΠϧ(M4A…)ϨϕϧͷΓऔΓ
• Audio Toolbox: • σίʔυɾΤϯίʔυɾαϯϓϦϯάपม(SRC)ͳͲ • Իσʔλ(PCM)ϨϕϧͷΓऔΓ • Core Audio: ϋʔυΣΞͱͷ௨৴ • ϋʔυΣΞΛར༻͢Δ߹ Core Audio Hardware Audio Toolbox AVFoundation -
Core Audio Λ͏ ࠷ϨΠϠʔͷAPI • ΦʔσΟΦσόΠεʹग़ྗ͞ΕΔσʔλΛࣗͰίϯτϩʔϧͰ͖Δ • ඞཁͳλΠϛϯάͰίʔϧόοΫ͕ݺΕΔͷͰԻσʔλ(PCMੜσʔλ)Λ ͢ •
σʔλͷ४උ͕࠶ੜʹؒʹ߹͏Α͏ʹ͢Δ • ΠϯλʔϑΣʔεCݴޠ • Audio Toolbox, Core Audio -
ԻָPlayerΛ࡞Δ σʔλͷྲྀΕ αϯϓϦϯά पม (SRC) σίʔμʔ ϑΝΠϧ .m4a Apple Lossless
όοϑΝ Core Audio 🔊 औΓʹ͍͘ Audio Toolbox Core Audio LPCM LPCM LPCM Apple Lossless App C ഉଞϞʔυ -
ഉଞϞʔυ • API: Hog Mode • ΧʔωϧϛΩαʔΛհ͞ͳ͍ • →αϯϓϦϯάपมͳͲ͕ߦΘΕͳ͍ •
→ଞͷΞϓϦέʔγϣϯͷԻग़ྗ͞Εͳ͍ • ΦʔσΟΦϋʔυΣΞʹΞΫηε • Ի(LPCM)σʔλΛૹͬͯ࠶ੜ -
Swift & Realtime • ϦΞϧλΠϜॲཧͷอূ͞Ε͍ͯͳ͍(ͣ) • Swiftಛ༗ͷॲཧͷΦʔόʔϔου͕͋ΔՄೳੑ • ΫϦςΟΧϧͳͷ࠷ޙͷίʔϧόοΫՕॴͷΈ •
࣮ࡍSwiftͰͳ͠ • ͋ΓͳΒCͰॻ͘ • ͦΕҎ֎ͷՕॴઈରϦΞϧλΠϜͰͳͯ͘ͳ͠ -
Concurrency • (Իָ࠶ੜPlayerͰ) ฒߦॲཧͷඞཁੑബ͍ • ͨͩ͠… • ಉ͡ΦϒδΣΫτΛڞ༗͢Δ • όοϑΝʔ(ϝϞϦɾϑΝΠϧ),
ΦʔσΟΦσόΠε, σίʔμʔ • ಉ࣌ʹΞΫηε͠ͳ͍Έඞཁ -
ഉଞॲཧ • ର͕ಉظؔͷ߹ • NIOLock - swift-nio (All Platforms) •
OSAllocatedUnfairLock (macOS 13~) -
struct AudioSytemObject { let id: AudioObjectID = ... let lock
= NIOLock() func get(addr: T) throws -> [T.DataType] { try lock.withLock { let memory = ... let statusData = AudioObjectGetPropertyData(id, &propAddr, 0, nil, &propSize, memory) return (0..<count).map { memory[$0] } } } func set(values: [T.DataType], forAddr addr: T) throws { try lock.withLockVoid { ... let status = AudioObjectSetPropertyData(id, &propAddr, 0, nil, UInt32(propSize), memory) } } } कΓ͍ͨΦϒδΣΫτ -
final class PlaybackBuffer { var buffer = Data() let lock
= NIOLock() func write(_ data: Data) { lock.withLockVoid { buffer.append(data) } } func read() -> Data? { return lock.withLock { if buffer.isEmpty { return nil } let buf = buffer buffer.removeAll() // clear data return buf } } } -
Actor vs Lock • Actorͩͱisolated context (isolation boundary)Λ͑ΒΕͳ͍ • ݺͼग़͠ݩ͕CͷίʔϧόοΫ(ಉظ)ͷ߹ͳͲ
• nonisolated ʹ͢Δ? →ࣗલͷഉଞॲཧ͕ඞཁ • await ͢Δ? →ಉظઐ༻ίʔϧόοΫͳͷͰෆՄ func AudioIOProc(buffer: PlaybackBuffer, outOutputData: UnsafeMutablePointer<AudioBufferList>) -> OSStatus { let data = buffer.read() memcpy(outOutputData, data, length) ... -
ඇಉظؔͷഉଞॲཧ • ϑΝΠϧΛόοϑΝʔͱͯ࣋ͭ͠ • ಡΈࠐΈதॻ͖ࠐ·ͳ͍ (ಡΈࠐΈ͕ऴΘΔ·Ͱͭ) • ॻ͖ࠐΈதಡΈࠐ·ͳ͍ (ॻ͖ࠐΈ͕ऴΘΔ·Ͱͭ) •
ϩοΫ(ͦͷ··)͑ͳ͍ (σουϩοΫ͢Δ) actor FileCache { func write(_ data: Data) async func read() async -> Data? } -
actor FileCache { let queue = DispatchQueue(label: "FileCacheQueue", qos: .utility)
let semaphore = DispatchSemaphore(value: 1) func write(_ data: Data) async { await withCheckedContinuation { continuation in queue.async { self.semaphore.wait() Task { await self.write_(data) continuation.resume() self.semaphore.signal() } } } } func read() async -> Data? { await withCheckedContinuation { continuation in queue.async { self.semaphore.wait() Task { let data = await self.read_() continuation.resume(returning: data) self.semaphore.signal() } } } } • ඇಉظؔ Serial Queue & Semaphore(or Mutex) • Actor͚ͩͰಉ࣌ಡΈॻ͖Λ͛ͳ͍߹͕͋Δ • อޢ͍ͨ͠Օॴ͕ඇಉظؔ -
εϨουB εϨουA ϑΝΠϧ ҙεϨου ࣌ؒ ॻ͖ࠐΈத Semaphore 1 0 wait()
wait() DispatchQueue signal() Task ҙεϨου 1 ಡΈࠐΈ 0 signal() write() read() 1 Queue Block… -
ࢀߟจݙ • Core Audio Overview (Apple) https://developer.apple.com/library/archive/documentation/MusicAudio/Conceptual/ CoreAudioOverview/WhatisCoreAudio/WhatisCoreAudio.html • Xcode
16 & Swift 6 ΩϟονΞοϓ: Swift Concurrencyͷجૅͱ࠷ॏཁϙΠϯτΛ૯෮श (QonceptInc) https://www.youtube.com/watch?v=jJgEtjx8KHY • ʲSwiftʳGrand Central Dispatch (GCD)ͱOperationQueue ·ͱΊ (@shiz) https://qiita.com/shiz/items/693241f41344a9df6d6f • Linuxͷsemaphoreͱmutexͷ࣮ (@watatuki) https://www.docswell.com/s/watatuki/Z7VNJG-2023-08-24-014308