Upgrade to Pro — share decks privately, control downloads, hide ads and more …

SuperGlue; Learning Feature Matching with Graph Neural Networks (CVPR'20)

July 04, 2020

SuperGlue; Learning Feature Matching with Graph Neural Networks (CVPR'20)

第三回 全日本コンピュータビジョン勉強会(前編)

SuperGlue;Learning Feature Matching with Graph Neural Networks の解説


July 04, 2020

More Decks by yu4u

Other Decks in Technology


  1. Mobility Technologies Co., Ltd. SuperGlue Learning Feature Matching with Graph

    Neural Networks 株式会社Mobility Technologies 内⽥ 祐介 JapanCV CVPR2020読み会(前編)
  2. Mobility Technologies Co., Ltd. Detection, Description, and Feature Matching 5

    ① Detec%on: Extract local regions (patches) from images ② Descrip%on: Describe the patches by d-dimensional vectors ③ Matching: Make correspondences between similar patches Position (x, y) Orientation θ Scale σ Feature vector f (e.g., 128-dim SIFT) Local feature
  3. Mobility Technologies Co., Ltd. “Classical” Detector 6 Hessian Beaudet’78 Harris

    Harris’88 LoG Lindeberg’98 DoG Lowe’99 SURF Bay’06 Harris-Laplace Mikolajczyk’01 Hessian-Affine Mikolajczyk’04 Harris-Affine Mikolajczyk’02 FAST Rosten’05 Affine-invariant Scale-invariant Rota0on-invariant LoG scale sele0on Affine adapta0on Multi-scale + Box filter acceleration LoG approxima0on Hessian-Laplace Mikolajczyk’01 Oriented FAST Rublee’11 SUSAN Smith’97 Simplifica0on + tree accelera0on OrientaGon Corner-like Blob-like (SIFT) (ORB)
  4. Mobility Technologies Co., Ltd. “Classical” Descriptor 7 SIFT Lowe’99 SURF

    Bay’06 BRIEF Calonder’10 ORB Rublee’11 GLOH Mikolajczyk’05 FREAK Alahi’12 A-KAZE Alcantarilla’13 LDB Yang’12 LATCH Levi’16 BRISK Leutenegger’11 Real-valued Binary (0.56, 0.22, -0.10, …, 0.96) (1, 0, 0, …, 1) RootSIFT Arandjelovic’12 画像検索今昔物語 h@ps://www.slideshare.net/ren4yu/image-retrieval-overview-from-tradi0onal-local-features-to-recent-deep-learning-approaches Local Feature Detectors, Descriptors, and Image Representa0ons: A Survey h@ps://arxiv.org/abs/1607.08368
  5. Mobility Technologies Co., Ltd. n単純なdescriptorの最近傍マッチングでは⼤量の誤対応が発⽣ nHeuristics • 閾値︓単純に⼀定距離のマッチングのみを残す • Ratio

    test: 最近傍と第⼆近傍の距離の⽐を利⽤ • Mutual check: お互いがお互いの最近傍となるマッチングのみを残す nLearned • e.g. Learning to Find Good Correspondences, CVPRʼ18. Outlier Filtering 9
  6. Mobility Technologies Co., Ltd. nSelf layer • 同⼀画像内のdescriptorで 完全グラフを構成 nCross

    layer • 画像間のdescriptorで 完全⼆部グラフを構成 Self layerとcross layer 14 画像引⽤元 h@ps://ja.wikipedia.org/wiki/%E5%AE%8C%E5%85%A8%E3%82%B0%E3%83%A9%E3%83%95 h@ps://ja.wikipedia.org/wiki/%E5%AE%8C%E5%85%A82%E9%83%A8%E3%82%B0%E3%83%A9%E3%83%95 画像Aのdescriptor node 画像Bのdescriptor node 画像Aのdescriptor node 画像Bのdescriptor node
  7. Mobility Technologies Co., Ltd. Message Passing with Self Attention 16

    self or cross qとkの類似度に 基づいた重み vの重み付け和
  8. Mobility Technologies Co., Ltd. nScore matrixから実際のマッチング結果(割当)を求めたい nSoft assignment problem nSinkhorn

    algorithm(row, columnを交互に正規化することを 繰り返す)により⼆重確率⾏列化 Sinkhorn algorithm 20
  9. Mobility Technologies Co., Ltd. npose error is the maximum of

    the angular errors in rotation and translation Wide-baseline indoor pose estimation 23