Upgrade to PRO for Only $50/Year—Limited-Time Offer! 🔥

Macとオーディオ再生 2024/11/02

Yusuke Ito
November 02, 2024

Macとオーディオ再生 2024/11/02

macOS native Symposium #10 での発表資料です。

https://macos-native.github.io/backnumbers.html

Yusuke Ito

November 02, 2024
Tweet

More Decks by Yusuke Ito

Other Decks in Programming

Transcript

  1. About me • Yusuke Ito • Web (TypeScript, React, Node.js)

    macOS ωΠςΟϒΞϓϦ (Minutes) iOS ωΠςΟϒΞϓϦ (App Store Best of 2014) • ϝΠϯ͸Web͕ͩɺΦʔσΟΦػثͷ։ൃɾ ੡଄+macOSΞϓϦ https://audio.current.directory/
  2. LPCMͷσʔλྔ • 44.1kHz (CD) αϯϓϦϯάप೾਺ • 16bit (CD) Ϗοτਂ౓ •

    3෼ (180ඵ) • 44100 (sample/sec) × 2 (byte, 16bit) × 2 (channel, L R) × 180 (secs) • = 31,752,000 (bytes) ≈ 30 MB (megabytes)
  3. ίϯςφͱίʔσοΫ ίϯςφ: QuickTime (.mov) MPEG-4 (.m4a, .m4v) Ի੠ AAC ө૾

    H.264 ϝλσʔλ ΞʔςΟετ໊… ΞϧόϜ໊… ϝλσʔλ ίʔσοΫ: H.264 AAC Apple Lossless MP3, ATRAC (MD) JPEG PNG ϑΝΠϧͷ಺෦
  4. Mixing & Playback Ի੠σʔλͷྲྀΕ Χʔωϧ ϛΩαʔ σόΠε 🔊 App A

    App B γεςϜܯࠂԻ App C ഉଞϞʔυ OR (αϯϓϦϯάप೾਺ม׵) 96kHz 48kHz 44.1kHz 96kHz LPCM LPCM
  5. Audio APIs on Mac • AVFoundation: ؆୯ʹ࢖͑ΔΫϥε܈ • AVAudioPlayerͳͲ, ϑΝΠϧ(M4A…)Ϩϕϧͷ΍ΓऔΓ

    • Audio Toolbox: • σίʔυɾΤϯίʔυɾαϯϓϦϯάप೾਺ม׵(SRC)ͳͲ • Ի੠σʔλ(PCM)Ϩϕϧͷ΍ΓऔΓ • Core Audio: ϋʔυ΢ΣΞͱͷ௨৴ • ϋʔυ΢ΣΞΛ௚઀ར༻͢Δ৔߹ Core Audio Hardware Audio Toolbox AVFoundation
  6. ԻָPlayerΛ࡞Δ σʔλͷྲྀΕ αϯϓϦϯά प೾਺ม׵ (SRC) σίʔμʔ ϑΝΠϧ .m4a Apple Lossless

    όοϑΝ Core Audio 🔊 औΓʹ͍͘ Audio Toolbox Core Audio LPCM LPCM LPCM Apple Lossless App C ഉଞϞʔυ
  7. ഉଞϞʔυ • API: Hog Mode • ΧʔωϧϛΩαʔΛհ͞ͳ͍ • →αϯϓϦϯάप೾਺ม׵ͳͲ͕ߦΘΕͳ͍ •

    →ଞͷΞϓϦέʔγϣϯͷԻ͸ग़ྗ͞Εͳ͍ • ΦʔσΟΦϋʔυ΢ΣΞʹ௚઀ΞΫηε • Ի੠(LPCM)σʔλΛૹͬͯ࠶ੜ
  8. struct AudioSytemObject { let id: AudioObjectID = ... let lock

    = NIOLock() func get(addr: T) throws -> [T.DataType] { try lock.withLock { let memory = ... let statusData = AudioObjectGetPropertyData(id, &propAddr, 0, nil, &propSize, memory) return (0..<count).map { memory[$0] } } } func set(values: [T.DataType], forAddr addr: T) throws { try lock.withLockVoid { ... let status = AudioObjectSetPropertyData(id, &propAddr, 0, nil, UInt32(propSize), memory) } } } कΓ͍ͨΦϒδΣΫτ
  9. final class PlaybackBuffer { var buffer = Data() let lock

    = NIOLock() func write(_ data: Data) { lock.withLockVoid { buffer.append(data) } } func read() -> Data? { return lock.withLock { if buffer.isEmpty { return nil } let buf = buffer buffer.removeAll() // clear data return buf } } }
  10. Actor vs Lock • Actorͩͱisolated context (isolation boundary)Λ௒͑ΒΕͳ͍ • ݺͼग़͠ݩ͕CͷίʔϧόοΫ(ಉظ)ͷ৔߹ͳͲ

    • nonisolated ʹ͢Δ? →ࣗલͷഉଞॲཧ͕ඞཁ • await ͢Δ? →ಉظઐ༻ίʔϧόοΫͳͷͰෆՄ func AudioIOProc(buffer: PlaybackBuffer, outOutputData: UnsafeMutablePointer<AudioBufferList>) -> OSStatus { let data = buffer.read() memcpy(outOutputData, data, length) ...
  11. actor FileCache { let queue = DispatchQueue(label: "FileCacheQueue", qos: .utility)

    let semaphore = DispatchSemaphore(value: 1) func write(_ data: Data) async { await withCheckedContinuation { continuation in queue.async { self.semaphore.wait() Task { await self.write_(data) continuation.resume() self.semaphore.signal() } } } } func read() async -> Data? { await withCheckedContinuation { continuation in queue.async { self.semaphore.wait() Task { let data = await self.read_() continuation.resume(returning: data) self.semaphore.signal() } } } } • ඇಉظؔ਺͸ Serial Queue & Semaphore(or Mutex) • Actor͚ͩͰ͸ಉ࣌ಡΈॻ͖Λ๷͛ͳ͍৔߹͕͋Δ • อޢ͍ͨ͠Օॴ͕ඇಉظؔ਺
  12. εϨουB εϨουA ϑΝΠϧ ೚ҙεϨου ࣌ؒ ॻ͖ࠐΈத Semaphore 1 0 wait()

    wait() DispatchQueue signal() Task ೚ҙεϨου 1 ಡΈࠐΈ 0 signal() write() read() 1 Queue Block…
  13. ࢀߟจݙ • Core Audio Overview (Apple) https://developer.apple.com/library/archive/documentation/MusicAudio/Conceptual/ CoreAudioOverview/WhatisCoreAudio/WhatisCoreAudio.html • Xcode

    16 & Swift 6 ΩϟονΞοϓ: Swift Concurrencyͷجૅͱ࠷ॏཁϙΠϯτΛ૯෮श (QonceptInc) https://www.youtube.com/watch?v=jJgEtjx8KHY • ʲSwiftʳGrand Central Dispatch (GCD)ͱOperationQueue ·ͱΊ (@shiz) https://qiita.com/shiz/items/693241f41344a9df6d6f • Linuxͷsemaphoreͱmutexͷ࣮૷ (@watatuki) https://www.docswell.com/s/watatuki/Z7VNJG-2023-08-24-014308