Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Moodstocks - Paris Tech Talks #6

Moodstocks - Paris Tech Talks #6

An overview of Mobile Image Recognition, a presentation of Moodstocks' approach and a focus on Moodstocks SDK core library for Paris Tech Talks Meetup #6.

* https://moodstocks.com/
* http://www.meetup.com/Paris-Tech-Talks/events/165819522/
* https://github.com/Moodstocks/jpec

Cédric Deltheil

March 11, 2014
Tweet

More Decks by Cédric Deltheil

Other Decks in Technology

Transcript

  1. Mobile Image Recognition Your Mobile App Reference Images + ID-s

    Image Recognition Platform query match ID = 1234 index user developer ID = 1234
  2. The old way: thin client Mobile App Image Recognition API

    client server reference images send JPEG image via HTTP match ID or nil
  3. Our approach: thick client Mobile App Moodstocks API client server

    reference images up to millions images Moodstocks SDK image signatures up to 10k images query match ID or nil server-side search for large DB sync
  4. Design Goal #1 Portability • core library written in C99

    • +99% code in common between iOS / Android • e.g supporting Intel CPU-s on Android was… no effort!
  5. Design Goal #2 Efficiency • in-memory data structures for maximum

    speed • leverage NEON capabilities (SIMD) where applicable • a typical search runs in ~ 30 ms / 10k images (iPhone 5S)
  6. Design Goal #3 Low footprint • 100% home made, lightweight

    implementation • implement & embed only what we need! • around 800 kB for the iOS ARMv7s release build (-Os)
  7. Image Recognition Pipeline • full-stack: the whole recognition occurs on-device!

    • robust to camera noises (blur, lighting, …) • supports 1D and 2D barcodes too Image Recognition Pipeline
  8. Internal Modules (1/4) • master-slave sync engine for image signatures

    • efficient: works with diff-s & data bundles (less requests) • fail-safe: retries + the client always stays consistent Sync Storage Logs API client
  9. Internal Modules (2/4) Sync Storage Logs API client • collects

    search results & metadata (OS, etc) along time • a long-running thread posts batches every 30s • tolerant to network errors -> HTTP 409 (Already Exists)
  10. Internal Modules (3/4) • persists image signatures on-disk • persists

    search results events on-disk • used as a temp on-disk buffer at sync time Sync Storage Logs API client
  11. Internal Modules (4/4) • used for server-side recognition (send image

    via HTTP) • includes a pool to re-use connections • also used to post logs batches to the server (HTTP) Sync Storage Logs API client
  12. 3rd Party Libraries (1/4) libcurl Tokyo Cabinet jpec msgpack •

    the +500M users file transfer library • multi API -> multiple transfers in the same thread • we use it as the backend of our sync engine + HTTP reqs.
  13. 3rd Party Libraries (2/4) libcurl Tokyo Cabinet jpec msgpack •

    key / value embedded database (hash / table / B+tree) • … and also a fantastic C library (see tcutil.h) • we use it as the backend of our storage engine
  14. 3rd Party Libraries (3/4) libcurl Tokyo Cabinet jpec msgpack •

    our home-made, tiny JPEG encoder (600 LOC-s) • we open sourced it -> github.com/Moodstocks/jpec • we use it for server-side requests (send image)
  15. 3rd Party Libraries (4/4) libcurl Tokyo Cabinet jpec msgpack •

    efficient binary serialization format • we use it to pack the data we synchronize
  16. Building • we use simple Makefile-s and shell scripts •

    we cross-compile w/ the iOS and Android toolchains • we combine all the arch-s into a fat static lib (armv7, …)
  17. Testing • we write unit tests with ct (Easy Unit

    Testing) by @krarick • we run them on device! e.g jailbreaked iPhone + openssh • we use a mock to test the sync engine
  18. Quality • no warnings: we compile with -Wall -Werror •

    static analysis with Clang Static Analyzer • memory leak checks with Valgrind