今日から分かる Android Audioの全て

KSA Online Meetup #1
https://student-kyushu.connpass.com/event/191541/

Transcript

1. ࠓ೔͔Β෼͔Δ Android Audioͷશͯ 2020.10.26 ߹ಉձࣾφΫα
 ઎෦ ॡฏ
2. ࣗݾ঺հ ઎෦ ॡฏ • ߹ಉձࣾφΫαɹ୅ද • ΤϯλϝͷϥΠϒ഑৴Λࢧ͍͑ͯ ·͢ • झຯ͸࡞ۂͱωοτϫʔΫӡ༻ • Android։ൃͰҰ൪࢖͍ͬͯΔ΋ͷ ͸MediaCodecͱVulkan
3. Pixel 3 ໰୊ w 1JYFM
   Ͱ͸ɺ""VEJP͕ݪҼͰ࠶ੜ͞Ε ͳ͍໰୊͕͋ͬͨ
4. ձࣾ঺հ 1.
5. w ೥݄ʹઃཱ
   w ຊࣾ͸౦ژ౎ઍ୅ా۠ਆాۋொ
   w ΦϑΟεʹϥοΫ͕͋Δ
   w ϝϯόʔ਺͸໊
   w ࠾༻ืूதʂ ߹ಉձࣾφΫα
6. w &/5&35"*.&/5
   ʷ
   5&$)/0-0(:
   w ΤϯλʔςΠϝϯτͷ෼໺Λٕज़Λ࢖ͬͯ໰୊ղܾΛ͢Δ ձࣾ
   w ΢Σϒ੍࡞͔Βಈը഑৴·ͰԿͰ΋΍Γ·͢ φΫαͷࣄۀ
7. w ϥΠϒ഑৴αʔϏε
   w ిࢠॻ੶αʔϏε
   w ͳͲͳͲ ࣄۀྫ
8. w /-*7&͸ɺφΫα͕ఏڙ͍ͯ͠ΔιϦϡʔγϣϯͷҰͭɻ
   w /-*7&͸ɺΫϥΠΞϯτ͔ΒΠϯϑϥ·ͰશͯφΫα͕Χ όʔ͢Δ͜ͱ͕ग़དྷ·͢
   w ͦͷଞɺ%3.ͷରԠͳͲ΋αϙʔτ͢Δ͜ͱ͕ՄೳͰ͢ ϥΠϒ഑৴ιϦϡʔγϣϯ NLIVE
9. w ڵຯΛ࣋ͬͨਓ͸๻ʹ࿈བྷ͍ͯͩ͘͠͞
   w ۀ຿ҕୗɾΞϧόΠτ΋େ׻ܴͰ͢ We're Hiring!
10. AndroidͷAudioͷྺ࢙ 2.
11. w "VEJP'MJOHFS͸"VEJP4FSWFSͷͭͰ͋ΓɺΞϓϦͷԻΛ ؅ཧ͢Δαʔό
    w "VEJP'MJOHFS͸"35 "OESPJE
    3VOUJNF ্Ͱಈ࡞͢Δ
    w "VEJP'MJOHFS͔Β)"- )BSEXBSF
    "CTUSBDU
    -BZFS ʹॲ ཧ͞Εͯɺ࿥Ի΍࠶ੜΛߦ͏ AudioFlinger
12. w "VEJP'MJOHFS͕"OESPJEͷ஗Ԇ໰୊ͷ தͰ͸ಛʹӨڹ͕େ͖͍
    w ͦͷཧ༝͸"VEJP'MJOHFSͰόοϑΝΛ ࡞Γ௚͔͢ΒͰ͋Δ AudioFlingerͷ10ms໰୊
13. w "VEJP'MJOHFSͷલʹ͸)"-͕͋Γɺ )"-͕"-4"ͱ΍ΓͱΓΛͯ͠σʔλΛ ૹड৴͢Δ
    w "-4"͸ϦϯάόοϑΝΛ࢖ͬͯɺυϥ Πόʔ͔Β"VEJP'MJOHFSʹૹ͍ͬͯΔ AudioFlingerͷ10ms໰୊
14. w "VEJP'MJOHFSͷޙʹ͸"VEJP3FDPSE ΍"VEJP5SBDL͕͋Δ
    w ͪ͜Β΋#ZUF#VGGFSΛ࢖ͬͯɺόο ϑΝΛ࡞ͬͯॻ͖ࠐΉॲཧ͕͋Δͷ Ͱɺ஗Ԇ͕͋Δ AudioFlingerͷ10ms໰୊
15. w "VEJP
    "1*ͷඪ४࣮૷͕0QFO4-
    &4
    w 0QFO4-
    &4Λ࢖͏͜ͱͰɺ"VEJP'MJOHFSΑΓ΋௿஗Ԇʹ ࿥Ի΍࠶ੜ͕Ͱ͖Δ OpenSL ES
16. w 0QFO4-
    &4ͷ໰୊఺͸ɺσόΠεʹΑͬͯ͸ඇରԠͳ͜ ͱ͕ଟ͍ʢυϥΠόʔͷඇαϙʔτʣ
    w ·ͨɺ0QFO4-
    &4ͷ࣮૷΋͔ͳΓεύήοςΟίʔυʹ ͳ͓ͬͯΓɺ࣌ʑෆ҆ఆʹͳΔ͜ͱ΋Α͋͘Δ OpenSL ES͕࢖͍෺ʹͳΒͳ͍
17. w "OESPJE
    ΑΓɺ0QFO4-
    &4ͷ୅ସͱͯ͠""VEJP͕ొ ৔ͨ͠
    w ""VEJPͰ͸ɺ͜Ε·Ͱ"VEJP'MJOHFSͷϥούʔతͳ໾ׂ Λ͍ͯͨ͠0QFO4-
    &4ͱ͸ҧ͍ɺ௚઀)"-ʹΞΫηεͰ ͖Δ AAudio
18. w ""VEJP͸
    0QFO4-
    &4
    ಉ༷ʹ$ Ͱهड़Λߦ͏ͨΊɺ "35ͷ($֎Ͱ͋ΔʢͦͷͨΊϝϞϦղ์͸ࣗ෼Ͱ΍Δඞ ཁ͕͋Δʣ
    w ""VEJPͰ͸ɺ֤ϕϯμʔͱڠྗ͠ɺ҆ఆͯ͠ಈ࡞͢Δ͜ ͱΛ໨ࢦͨ͠ AAudio
19. Implementaion 3.
20. w (PPHMF͕։ൃͨ͠ϥΠϒϥϦ
    w 0QFO4-
    &4ͱ""VEJPͷ૬ޓར༻Λ ߦ͏ͨΊͷϥούʔ
    w ࣗ෼Ͱॻ͘ͷ͕ͭΒ͍ਓ͸0CPFΛ ࢖͏ͱ؆୯ʹهड़Ͱ͖Δ Oboe
21. w 0QFO4-
    &4͸ࠓߋ࢖͏ඞཁ΋ͳ͍ͷͰࠓճ͸""VEJPΛ࣮ ૷ͯ͠Έ·͠ΐ͏ ࣗ෼Ͱ࣮૷ͯ͠ΈΔ
22. dlsymͰAAudioΛಡΈࠐΉ w MJCBBVEJPTP͸ NJO4EL7FSTJPO
    ͔Βೖ ͍ͬͯΔ
23. InputStreamΛ४උ͢Δ void AAudio_IO::setupInputStream( AAudioStreamBuilder* builder ) { AAudioStreamBuilder_setDeviceId ( builder, _inputDeviceId ); AAudioStreamBuilder_setSampleRate ( builder, _sampleRate ); AAudioStreamBuilder_setChannelCount( builder, _inputChannelCount ); AAudioStreamBuilder_setFormat ( builder, _sampleFormat ); AAudioStreamBuilder_setSharingMode ( builder, AAUDIO_SHARING_MODE_EXCLUSIVE ); AAudioStreamBuilder_setPerformanceMode( builder, AAUDIO_PERFORMANCE_MODE_LOW_LATENCY ); AAudioStreamBuilder_setDirection ( builder, AAUDIO_DIRECTION_INPUT ); AAudioStreamBuilder_setErrorCallback ( builder, ::errorCallback, this ); }
24. outputStreamΛ४උ͢Δ void AAudio_IO::setupOutputStream( AAudioStreamBuilder* builder ) { AAudioStreamBuilder_setDeviceId ( builder, _outputDeviceId ); AAudioStreamBuilder_setFormat ( builder, _sampleFormat ); AAudioStreamBuilder_setChannelCount( builder, _outputChannelCount ); AAudioStreamBuilder_setSharingMode ( builder, AAUDIO_SHARING_MODE_EXCLUSIVE ); AAudioStreamBuilder_setPerformanceMode( builder, AAUDIO_PERFORMANCE_MODE_LOW_LATENCY ); AAudioStreamBuilder_setDirection ( builder, AAUDIO_DIRECTION_OUTPUT ); AAudioStreamBuilder_setDataCallback ( builder, ::dataCallback, this ); AAudioStreamBuilder_setErrorCallback ( builder, ::errorCallback, this ); }
25. inputStreamΛ࡞Δ void AAudio_IO::createInputStream() { AAudioStreamBuilder* builder = createStreamBuilder(); setupInputStream( builder ); aaudio_result_t result = static_cast<aaudio_result_t>(AAudioStreamBuilder_openStream( builder, &_inputStream )); AAudioStreamBuilder_delete( builder ); }
26. outputStreamΛ࡞Δ void AAudio_IO::createOutputStream() { AAudioStreamBuilder* builder = createStreamBuilder(); setupOutputStream( builder ); aaudio_result_t result = static_cast<aaudio_result_t>(AAudioStreamBuilder_openStream( builder, &_outputStream )); if ( result == AAUDIO_OK && _outputStream != nullptr ) { _sampleRate = AAudioStream_getSampleRate( _outputStream ); _framesPerBurst = AAudioStream_getFramesPerBurst( _outputStream ); AudioEngineProps::SAMPLE_RATE = _sampleRate; updateBufferSizeInFrames( AAudioStream_setBufferSizeInFrames( _outputStream, _framesPerBurst )); _underrunCountOutputStream = AAudioStream_getXRunCount( _outputStream ); } else { return; } AAudioStreamBuilder_delete( builder ); }
27. streamΛ։࢝ɾఀࢭ͢Δ void AAudio_IO::startStream( AAudioStream* stream ) { aaudio_result_t result = static_cast<aaudio_result_t>(AAudioStream_requestStart( stream )); if ( result != AAUDIO_OK ) { return; } } void AAudio_IO::stopStream( AAudioStream* stream ) { if ( stream == nullptr ) { return; } aaudio_result_t result = static_cast<aaudio_result_t>(AAudioStream_requestStop( stream )); }
28. w ""VEJP4USFBN#VJMEFS@TFU1FSGPSNBODF.PEFͰɺ-08@-"5&/$:ʹઃఆ͢Δͱ௒௿஗ԆʹͳΔ͕ όοςϦʔ͕ݮΔɻ͔͠΋ɺ͜Ε͚ͩͩͱ͋·Γҙຯ͕ͳ͍ʢޙड़ʣ
    w ""VEJP͸5ISFBE
    4BGFͰ͸ͳ͍ɻͦͷͨΊɺผεϨου͔Βಉ࣌ʹ""VEJPΛݺͼग़͢͜ͱ͸Ͱ͖ͳ ͍ɻ
    w ίʔϧόοΫؔ਺Λ࢖͍ɺඇಉظͰॲཧΛ͢Δ͜ͱΛਪ঑ɻ४උ͕Ͱ͖ͨΒίʔϧόοΫͰಡΈࠐΈɾ ॻ͖ࠐΈ͕Ͱ͖Δɻ
    w όοϑΝαΠζΛௐ੔͢Δ΂͖ɻ࠷ॳ͸5$1ͷ8JOEPX
    4J[F
    0QUJNJ[FͷΑ͏ʹঃʑʹόοϑΝαΠ ζΛେ͖͍ͯ͘͘͠ํ͕Α͍ɻ AAudioͷͭ·͖ͮϙΠϯτ
29. w ""VEJPͷ-PX
    -BUFODZ
    .PEF͚ͩͩͱ௒௿஗Ԇʹ͸ͳΒͳ͍
    w "DUJWJUZͷ8JOEPX.BOBHFSʹ
    8JOEPX.BOBHFS-BZPVU1BSBNT'-"(@,&&1@4$3&&/@0/
    Λ ηοτ͢Δඞཁ͕͋Δ
    w BDUJWJUZHFU8JOEPX TFU4VTUBJOFE1FSGPSNBODF.PEF
    USVF
    
    ͰɺύϑΥʔϚϯεϞʔυΛ༗ޮʹ ͢Δ WindowManagerΛϋοΫ͢Δ
30. inline void optimizeThreadPerformance( const std::vector<int>& cpuIds ) { cpu_set_t mask; pid_t current_thread_id = gettid(); cpu_set_t cpu_set; CPU_ZERO( &cpu_set ); if ( cpuIds.empty() ) { int current_cpu_id = sched_getcpu(); CPU_SET( current_cpu_id, &cpu_set ); } else { for ( size_t i = 0; i < cpuIds.size(); i++ ) { int cpu_id = cpuIds.at( i ); CPU_SET( cpu_id, &cpu_set ); } } } CPUΞϑΟχςΟΛεϨουʹઃఆ͢Δ w ΞϑΟχςΟΛઃఆ͢Δ͜ͱͰɺ$16ύϑΥʔ ϚϯεΛۃݶ·Ͱ޲্ͤ͞Δ͜ͱ͕Ͱ͖Δ
    w ϓϩηεɾεϨου͸04ʹΑͬͯͲΕ͔ͷ $16ίΞ্Ͱ࣮ߦ͞ΕΔɻ$16ΞϑΟχςΟΛ ઃఆ͢Δ͜ͱͰɺͦͷ࣮ߦ͞ΕΔ$16ίΞΛ೚ ҙʹ੍ݶɾࢦఆ͢Δ͜ͱ͕Ͱ͖Δɻ
31. ·ͱΊ 4.
32. w "OESPJEͷΦʔσΟΦͷྺ࢙͸௕͘ɺෳࡶͰ͋Δ
    w 0CPFΛͱΓ͋͑ͣ࢖͓͚ͬͯ͹Α͍
    w ۃݶ·ͰΧελϚΠζ͍ͨ͠ਓ͸ຊηογϣϯͷ಺༰Λࢀ ߟʹɺ࣮૷ͯ͠Έ͍ͯͩ͘͞ ·ͱΊ