Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Image Processing with Vision Framework

Image Processing with Vision Framework

Shohei Yokoyama

June 19, 2019
Tweet

More Decks by Shohei Yokoyama

Other Decks in Technology

Transcript

  1. CA.swift CyberaAgent Presents iOS Developer’s Meetup Image Processing with Vision

    Framework shoheiyokoyama CA.swift CyberaAgent Presents iOS Developer’s Meetup
  2. CA.swift CyberaAgent Presents iOS Developer’s Meetup Saliency Saliency refers to

    what’s noticeable or important in an image. 
 ը૾಺Ͱ໨ཱͭ΋ͷ΍ॏཁͳ΋ͷ https://developer.apple.com/documentation/vision/cropping_images_using_saliency. Cropping Images Using Saliency, Apple Developer Document by Apple Inc,
  3. CA.swift CyberaAgent Presents iOS Developer’s Meetup Saliency ɾ object-based saliency

    ɾ attention-based saliency ਓʑ͕ݟ͍ͯΔՄೳੑ͕ߴ͍ / ௿͍ ඃࣸମ΍લܠͷΦϒδΣΫτͷϋΠϥΠτ౓߹͍
  4. CA.swift CyberaAgent Presents iOS Developer’s Meetup Attention-based saliency ɾίϯτϥετ ɾإ

    ɾඃࣸମ ɾ஍ฏઢ ɾޫ ɾ… ༷ʑͳཁҼ͕ݦஶੑΛܾఆ͚ͮΔͨΊෳࡶͰ͋Δ
  5. CA.swift CyberaAgent Presents iOS Developer’s Meetup MFUSFRVFTU)BOEMFS7/*NBHF3FRVFTU)BOEMFS DW1JYFM#V⒎FSTBNQMF#V⒎FS PSJFOUBUJPOSJHIU PQUJPOT<>

     MFUSFRVFTU7/3FRVFTU7/(FOFSBUF"UUFOUJPO#BTFE4BMJFODZ*NBHF3FRVFTU  USZ SFRVFTU)BOEMFSQFSGPSN <SFRVFTU>  MFUPCTFSWBUJPOSFRVFTUSFTVMUT pSTUBT 7/4BMJFODZ*NBHF0CTFSWBUJPO MFUQJYFM#V⒎FSPCTFSWBUJPO QJYFM#V⒎FS Attention-based saliency
  6. CA.swift CyberaAgent Presents iOS Developer’s Meetup MFUSFRVFTU)BOEMFS7/*NBHF3FRVFTU)BOEMFS DW1JYFM#V⒎FSTBNQMF#V⒎FS PSJFOUBUJPOSJHIU PQUJPOT<>

     MFUSFRVFTU7/3FRVFTU7/(FOFSBUF0CKFDUOFTT#BTFE4BMJFODZ*NBHF3FRVFTU  USZ SFRVFTU)BOEMFSQFSGPSN <SFRVFTU>  MFUPCTFSWBUJPOSFRVFTUSFTVMUT pSTUBT 7/4BMJFODZ*NBHF0CTFSWBUJPO MFUQJYFM#V⒎FSPCTFSWBUJPO QJYFM#V⒎FS Object-based saliency
  7. CA.swift CyberaAgent Presents iOS Developer’s Meetup MFUSFRVFTU)BOEMFS7/*NBHF3FRVFTU)BOEMFS VSMJNBHF63- NBLFSFRVFTUXJUIJNBHF63- MFUSFRVFTU7/3FRVFTU7/(FOFSBUF0CKFDUOFTT#BTFE4BMJFODZ*NBHF3FRVFTU

     USZ SFRVFTU)BOEMFSQFSGPSN <SFRVFTU>  MFUPCTFSWBUJPOSFRVFTUSFTVMUT pSTUBT 7/4BMJFODZ*NBHF0CTFSWBUJPO MFUQJYFM#V⒎FSPCTFSWBUJPO QJYFM#V⒎FS Object-based saliency
  8. CA.swift CyberaAgent Presents iOS Developer’s Meetup MFUPCTFSWBUJPOTUSZ7/$MBTTJGZ*NBHF3FRVFTULOPXO$MBTTJpDBUJPOT GPS3FWJTJPO7/$MBTTJGZ*NBHF3FRVFTU3FWJTJPO  QSJOU

    PCTFSWBUJPOTDPVOU  PCTFSWBUJPOTGPS&BDI\QSJOU JEFOUJpFS ^ BCBDVT BDDPSEJPO BDPSO BDSPCBU BEVMU BEVMU@DBU ɻɻɻ ZPHVSU ZPML [FCSB [PP [VDDIJOJ
  9. CA.swift CyberaAgent Presents iOS Developer’s Meetup MFUIBOEMFS7/*NBHF3FRVFTU)BOEMFS VSMJNBHF63-  MFUSFRVFTU7/$MBTTJGZ*NBHF3FRVFTU

     USZ IBOEMFSQFSGPSN <SFRVFTU> 
 MFUPCTFSWBUJPOTSFRVFTUSFTVMUTBT <7/$MBTTJpDBUJPO0CTFSWBUJPO> Sample code
  10. CA.swift CyberaAgent Presents iOS Developer’s Meetup MFUIBOEMFS7/*NBHF3FRVFTU)BOEMFS VSMJNBHF63-  MFUSFRVFTU7/$MBTTJGZ*NBHF3FRVFTU

     USZ IBOEMFSQFSGPSN <SFRVFTU> 
 MFUPCTFSWBUJPOTSFRVFTUSFTVMUTBT <7/$MBTTJpDBUJPO0CTFSWBUJPO> Sample code < BOJNBM   DBU   NBNBM   ʜ > *EFOUJpFS$POpEFODF
  11. CA.swift CyberaAgent Presents iOS Developer’s Meetup Precision and recall ɾPresision

    ɾRecall ݁Ռͷ͏ͪɺͲͷ͘Β͍݁Ռ͕ਖ਼͍͔͠ ਖ਼ղͷதͰɺ݁ՌͷதʹͲΕ͘Β͍ਖ਼ղؚ͕·Ε͍ͯΔ͔
  12. CA.swift CyberaAgent Presents iOS Developer’s Meetup Precision and recall ݕࡧΤϯδϯͷྫ

    ɾݕࡧΤϯδϯͰ30݅ώοτ ɾͦͷ͏ͪ20͕݅ؔ࿈ ɾؔ࿈͍ͯ͠Δ40͕݅ώοτ͠ͳ͔ͬͨ
  13. CA.swift CyberaAgent Presents iOS Developer’s Meetup Precision and recall ɾPresision

    ݕࡧΤϯδϯͷྫ ɾݕࡧΤϯδϯͰ30݅ώοτ ɾͦͷ͏ͪ20͕݅ؔ࿈ ɾؔ࿈͍ͯ͠Δ40͕݅ώοτ͠ͳ͔ͬͨ 20/30 = 2/3 ɾRecall 20/60 = 1/3
  14. CA.swift CyberaAgent Presents iOS Developer’s Meetup Precision and recall ݕࡧΤϯδϯͷྫ

    ɾݕࡧΤϯδϯͰ30݅ώοτ ɾͦͷ͏ͪ20͕݅ؔ࿈ ɾؔ࿈͍ͯ͠Δ40͕݅ώοτ͠ͳ͔ͬͨ A B C A: ݕࡧ݁Ռ B: શମͷݕࡧཁٻΛຬͨ͢ϖʔδ C: ݕࡧ݁ՌͷதͷݕࡧཁٻΛຬͨ͢ϖʔδ
  15. CA.swift CyberaAgent Presents iOS Developer’s Meetup Precision and recall A:

    ݕࡧ݁Ռ B: શମͷݕࡧཁٻΛຬͨ͢ϖʔδ C: ݕࡧ݁ՌͷதͷݕࡧཁٻΛຬͨ͢ϖʔδ ɾPresision C / A ɾRecall C / B A B C
  16. CA.swift CyberaAgent Presents iOS Developer’s Meetup Precision and recall A:

    ݕࡧ݁Ռ B: શମͷݕࡧཁٻΛຬͨ͢ϖʔδ C: ݕࡧ݁ՌͷதͷݕࡧཁٻΛຬͨ͢ϖʔδ ɾPresision ɾRecall A B C ͲΕ͚ͩແؔ܎ͳ݁ՌΑΓ΋ ؔ࿈ੑͷߴ͍݁ՌΛฦ͔͢ ͲΕ͚ͩΞϧΰϦζϜ͕ؔ࿈ੑ ͷ͋Δ݁ՌΛฦ͔͢
  17. CA.swift CyberaAgent Presents iOS Developer’s Meetup The PR Curve Presision

    Recall 1 1 0 ɾҰํ্͕͕Ε͹ଞํ͕௿Լ͢Δ ɾΫϥεʹΑͬͯݻ༗ͷΧʔϒΛඳ͘
  18. CA.swift CyberaAgent Presents iOS Developer’s Meetup MFUIBOEMFS7/*NBHF3FRVFTU)BOEMFS VSMJNBHF63-  MFUSFRVFTU7/$MBTTJGZ*NBHF3FRVFTU

     USZ IBOEMFSQFSGPSN <SFRVFTU> 
 MFUPCTFSWBUJPOTSFRVFTUSFTVMUTBT <7/$MBTTJpDBUJPO0CTFSWBUJPO> MFUpMUFSFEPCTFSWBUJPOT pMUFS\IBT.JOJNVN1SFDJTJPO  GPS3FDBMM ^ Sample code ݁ՌΛϑΟϧλϦϯά
  19. CA.swift CyberaAgent Presents iOS Developer’s Meetup Filtering the PR Curve

    Presision Recall 1 1 0 observations?.filter { $0.hasMinimumPrecision(0.4, forRecall: 0.5) }
  20. CA.swift CyberaAgent Presents iOS Developer’s Meetup Filtering the PR Curve

    Presision Recall 1 1 0 observations?.filter { $0.hasMinimumPrecision(0.4, forRecall: 0.5) } Precision > 0.4
  21. CA.swift CyberaAgent Presents iOS Developer’s Meetup Filtering the PR Curve

    Presision Recall 1 1 0 observations?.filter { $0.hasMinimumPrecision(0.4, forRecall: 0.5) } Precision > 0.4 Recall = 0.5
  22. CA.swift CyberaAgent Presents iOS Developer’s Meetup Filtering the PR Curve

    Presision Recall 1 1 0 observations?.filter { $0.hasMinimumPrecision(0.4, forRecall: 0.9) } Precision > 0.4 Recall = 0.9
  23. CA.swift CyberaAgent Presents iOS Developer’s Meetup Filtering the PR Curve

    Presision Recall 1 1 0 observations?.filter { $0.hasMinimumPrecision(0.4, forRecall: 0.9) } Precision > 0.4 Recall = 0.9 ΞϓϦέʔγϣϯͷ ಛੑʹ͋ͬͨᮢ஋Λઃఆ͢Δ
  24. CA.swift CyberaAgent Presents iOS Developer’s Meetup F-measure f − measure

    = 2 × precision × recall precision + recall F஋͕ߴ͍΄ͲɺόϥϯεΑ͘૒ํͷ஋͕ߴ͍
  25. CA.swift CyberaAgent Presents iOS Developer’s Meetup MFUIBOEMFS7/*NBHF3FRVFTU)BOEMFS VSMJNBHF63-  MFUSFRVFTU7/(FOFSBUF*NBHF'FBUVSF1SJOU3FRVFTU

     USZ IBOEMFSQFSGPSN <SFRVFTU> 
 MFUDPOUFTUBOU'10SFRVFTUSFTVMUTpSTUBT 7/'FBUVSF1SJOU0CTFSWBUJPO WBSEJTUBODF'MPBU   USZDPOUFTUBOU'10DPNQVUF%JTUBODF EJTUBODF UPPSJHJOBM'10 Sample code
  26. CA.swift CyberaAgent Presents iOS Developer’s Meetup Other New API ɾFace

    capture quality ɾFace Landmarks Enhancements ɾHuman detector ɾCat and Dog detector ɾNew Object Tracker ɾVision and CoreML Integration Enhancements