Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Video Processing on Android

Video Processing on Android

An increasing number of social media apps such as Instagram, Snapchat and Messenger are using video features. With modern Android devices having enough processing power to run video processing algorithms locally, there’s no need to send videos through expensive internet bandwidth to be processed on the backend. Overlays, transcoding and cropping are just a few of the challenges you face when working with video on-device. In this talk I will go through my experience experimenting with built-in as well as third party libraries for video manipulation on Android. We will look at the benefits of two leading media processing libraries, and how you can leverage them to enhance Android’s native MediaCodec API to accomplish these tasks. You will walk away with a head start on how to tackle the most common challenges with videos on Android.

Namrata Bandekar

July 13, 2017
Tweet

More Decks by Namrata Bandekar

Other Decks in Education

Transcript

  1. V I D E O P R O C E

    S S I N G O N A N D R O I D N A M R ATA B A N D E K A R
  2. V I D E O P R O C E

    S S I N G I N T R O D U C T I O N T O
  3. M O B I L E A P P S

    P O P U L A R U S E I N
  4. C O M M O N F E AT U

    R E S • Special video effects • Merging videos • Cropping and trimming
  5. V I D E O E D I T I

    N G O N D E V I C E
  6. A N D R O I D S U P

    P O R T • MediaCodec • MediaExtractor • MediaMuxer
  7. W H Y F F M P E G ?

    • Swiss Army Knife for video • Filters • Subtitles
  8. F F M P E G O N A N

    D R O I D • Executable binary • Shared object library
  9. F F M P E G E X E C

    U TA B L E B U I L D I N G T H E
  10. D E P E N D E N C I

    E S • autoconf • automake • libtool
  11. G E T T I N G S TA R

    T E D Step 1: Download and unpack NDK
  12. G E T T I N G S TA R

    T E D Step 2: Download FFmpeg source code https://github.com/FFmpeg/FFmpeg
  13. G E T T I N G S TA R

    T E D Step 3: Download sources for other libraries like yasm and libvpx
  14. G E T T I N G S TA R

    T E D Step 4: Put FFmpeg and other libraries under ndk/sources
  15. F F M P E G E X E C

    U TA B L E I N Y O U R A N D R O I D P R O J E C T
  16. C H A N G E P E R M

    I S S I O N S
  17. S H A R E D O B J E

    C T L I B R A RY B U I L D I N G F F M P E G A S A
  18. S E T U P F O R F F

    M P E G . S O Step 1: JNI project same folder as NDK
  19. S E T U P F O R F F

    M P E G . S O Step 2: Android.mk file
  20. S E T U P F O R F F

    M P E G . S O Step 3: Application.mk file
  21. S E T U P F O R F F

    M P E G . S O Step 4: Place native code in app/jni
  22. L O A D S H A R E D

    L I B R A RY
  23. L I C E N S E C O M

    P L I A N C E
  24. L I C E N S E C O M

    P L I A N C E • Dynamic linking • FFmpeg source • Configure script
  25. L I C E N S E C O M

    P L I A N C E
  26. L I C E N S E C O M

    P L I A N C E
  27. F F M P E G P R O S

    • Modular • Support for codecs • Versatile • Well documented
  28. F F M P E G C H A L

    L E N G E S • Licensing • Slow • Large
  29. M P 4 PA R S E R F E

    AT U R E S • Concatenation • Trimming • Muxing • Demuxing
  30. M P 4 PA R S E R P R

    O S • Java library • Fast • Clean API
  31. M P 4 PA R S E R C H

    A L L E N G E S • Limited functionality • No encoding and decoding • Concatenates only if inputs have same format
  32. R E S O U R C E S •

    https://github.com/guardianproject/android- ffmpeg • https://github.com/Yelp/ffmpeg-android • https://github.com/inFullMobile/videokit- ffmpeg-android • https://www.ffmpeg.org/legal.html • https://github.com/sannies/mp4parser • https://github.com/TeamWanari/RxMp4Parser
  33. O A N D A I S H I R

    I N G ! jobs.oanda.com
  34. T H A N K Y O U ! Q

    U E S T I O N S ? @ n a m r a t a C o d e s https://github.com/namrata-b/talks/