Upgrade to Pro — share decks privately, control downloads, hide ads and more …

What_s_new_in_voice_processing他What_s_new_in_Au...

entaku
June 22, 2023

 What_s_new_in_voice_processing他What_s_new_in_Audio.pdf

entaku

June 22, 2023
Tweet

More Decks by entaku

Other Decks in Technology

Transcript

  1. https://developer.apple.com/videos/play/wwdc2023/10235/ What’s new in voice processing Learn how to use

    the Apple voice processing APIs to achieve the best possible audio experience in your VoIP apps. We'll show you how to detect when someone is talking while muted, adjust ducking behavior of other audio, and more.
  2. let audioEngine = AVAudioEngine() let inputNode = audioEngine.inputNode // 11:08

    - Other audio ducking do { try inputNode.setVoiceProcessingEnabled(true) } catch { print("Could not enable voice processing \(error)") } let duckingConfig = AVAudioVoiceProcessingOtherAudioDuckingConfiguration(enableAdvancedDucking: false, duckingLevel: .max) inputNode.voiceProcessingOtherAudioDuckingConfiguration = duckingConfig https://developer.apple.com/videos/play/wwdc2023/10235/ Other audio ducking
  3. // 12:31 - Voice activity detection - implementation with HAL

    APIs let listener = { (event : AVAudioVoiceProcessingSpeechActivityEvent) in if (event == AVAudioVoiceProcessingSpeechActivityEvent.started) { // User has started talking while muted. Prompt the user to un- mute } else if (event == AVAudioVoiceProcessingSpeechActivityEvent.ended) { // User has stopped talking while muted } } inputNode.setMutedSpeechActivityEventListener(listener) // When user mutes inputNode.isVoiceProcessingInputMuted = true https://developer.apple.com/videos/play/wwdc2023/10235/ Muted talker detection
  4. Add SharePlay to your app Discover how your app can

    take advantage of SharePlay to turn any activity into a shareable experience with friends! We'll share the latest updates to SharePlay, explore the bene fi ts of creating shared activities, dive into some exciting use cases, and take you through best practices to create engaging and fun moments of connection in your app. https://developer.apple.com/videos/play/wwdc2023/10239/
  5. Add SharePlay to your app struct OrderTogether: GroupActivity { //

    Define a unique activity identifier for system to reference static let activityIdentifier = "com.example.apple- samplecode.TacoTruck.OrderTogether" // App-specific data so your app can launch the activity on others' devices let orderUUID: UUID let truckName: String var metadata: GroupActivityMetadata { var metadata = GroupActivityMetadata() metadata.title = "Order Tacos Together" metadata.subtitle = truckName metadata.previewImage = UIImage(named: "ActivityImage")?.cgImage metadata.type = .shopTogether return metadata } } https://developer.apple.com/videos/play/wwdc2023/10239/
  6. Enhance your app’s audio experience with AirPods Discover how you

    can create transformative audio experiences in your app using AirPods. Learn how to incorporate AirPods Automatic Switching, use AVAudioApplication to support Mute Control, and take advantage of Spatial Audio to create immersive soundscapes in your app or game. https://developer.apple.com/videos/play/wwdc2023/10233/
  7. Enhance your app’s audio experience with AirPods // Adopting AVAudioApplication

    into your App import AVFAudio // Get the started instance let instance = AVAudioApplication.shared // Register for mute gesture notifications on Notification Center AVAudioApplication.inputMuteStateChangeNotification // Key for mute state AVAudioApplication.muteStateKey // Updating AVAudioApplication’s mute state instance.setInputMuted(...) // Reading AVAudioApplication’s mute state instance.isInputMuted https://developer.apple.com/videos/play/wwdc2023/10233/
  8. instance.setInputMuteStateChangeHandler { isMuted in //... return didSucceed } // Optional:

    let CoreAudio mute your input for you (macOS only) // Define the Core Audio property var inputeMutePropertyAddress = AudioObjectPropertyAddress( mSelector: kAudioHardwarePropertyProcessInputMute, mScope: kAudioObjectPropertyScopeInput, mElement:kAudioObjectPropertyElementMain) // Enable this property when you want to mute your input UInt32 isMuted = 1; // 1 = muted, 0 = unmuted AudioObjectSetPropertyData(kAudioObjectSystemObject, &inputeMutePropertyAddress, 0, nil, UInt32(MemoryLayout.size(ofValue: isMuted), &isMuted) Enhance your app’s audio experience with AirPods https://developer.apple.com/videos/play/wwdc2023/10233/
  9. Explore AirPlay with interstitials https://developer.apple.com/videos/play/wwdc2023/10238/ Learn how you can upgrade

    your app's AirPlay audio experience to be more robust and responsive. We'll show you how to adopt enhanced audio buffering with AVQueuePlayer, explore alternatives when building a custom player in your app, and share best practices.
  10. let audioSession = AVAudioSession.sharedInstance() try audioSession.setCategory(. playback ,xmode: . default

    , policy:.longFormAudio ) let player = AVQueuePlayer() let url = URL(string: "http://www.examplecontenturl.com") let asset = AVAsset(url: url) let item = AVPlayItem(asset: asset) player.insert(item, after: nil) player.play() Explore AirPlay with interstitials https://developer.apple.com/videos/play/wwdc2023/10238/
  11. ·ͱΊ • AVAudioEngineͷڧԽͰ௨࿩தͷϛϡʔτ΍ԻΛࡌͤΒΕΔ • Share PlayͷڧԽ • αʔυύʔςΟ͔Β࢖͑Δ • AirDropͰ͖Δ

    • AirPods/AirPlayར༻࣌ͷΞοϓσʔτ • ϛϡʔτ͕ઃఆ΍ݕ஌ʹΑͬͯΑΓγʔϜϨεͳମݧʹͳΔʁ
  12. ࣗݾ঺հ Name: ԕ౻୓໻(͑Μͨ͘) Job: iOS / AndroidΤϯδχΞͳͲ Career: SIer໿6೥ 2018/3~

    εϙʔπϚονϯάΞϓϦ 2019/3~ CBcloud ෺ྲྀITαʔϏε 2021/12~ Voicy