Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Search fashion items by colors

shoby
March 22, 2016

Search fashion items by colors

shoby

March 22, 2016
Tweet

More Decks by shoby

Other Decks in Technology

Transcript

  1. Shota Izaka • Fablic, inc. • Developing Fril • iOS

    / Ruby on Rails / Image Processing • Search by colors / Apple Watch App / Swift migration  !TIPCZTIPCZ
  2. What is Fril? • Fashion market place app • Similar

    to Poshmark • Buy and sell fashion items • Fril mediates trading 
  3. Outline • What is color search? • Why we implemented

    it? • Implementation • Create color palette • Color detection by image processing • Result 
  4. 

  5. What is color search? • Narrow down items by colors

    • e.g. One-piece dress & white • e.g. Coat & brown • Detect colors automatically • Sellers input nothing • Color detection from item images 
  6. Why we implemented it? • Most users choosing clothing colors

    • Questionnaire • Interview • Search word logs • To specialize in fashion • Fashion specialized search features 
  7. Why 12 colors? • From human color recognition research[1] •

    All languages have basic 11 color terms • 11 colors are universal • Red, yellow, green, blue, orange, purple, pink, brown, gray, black, white • From search queries including color names • Listed queries in descending order • Merged into basic 11 colors (e.g camel is brown) • Leaved beige because it can't merge  [1] Berlin B.and Kay P, “Basic Color Terms:Their Universality and Evolution.”
  8. Color detection by image processing • Color correction by white

    balancing • Remove background • Detect a mode color • Choose nearest color palette 
  9. Color correction by white balancing • Correct colors of images

    taken under fluorescent lamp • Typical fluorescent lights make images greenish • Fashion images usually taken inside a room • Correct colors with Retinex[2] • A method modeled human visual features • Do not affect photoshopped images  [2] E. Land and J. McCann, “Lightness and Retinex theory,”
  10. require 'RMagick' image = ImageList.new('image.jpg') red_extrema = image.channel_extrema(Magick::RedChannel) green_extrema =

    image.channel_extrema(Magick::GreenChannel) blue_extrema = image.channel_extrema(Magick::BlueChannel) red_max = red_extrema.last.to_f green_max = green_extrema.last.to_f blue_max = blue_extrema.last.to_f white_balanced_image = image white_balanced_image = white_balanced_image.convolve_channel(1, [green_max / red_max], Magick::RedChannel) white_balanced_image = white_balanced_image.convolve_channel(1, [green_max / blue_max], Magick::BlueChannel)  Retinex using RMagick
  11. Remove background • Eliminate the effect of background colors •

    Make background mask to remove • Make background mask • Detect edges • Flood magenta from corners • Extract magenta regions as background mask 
  12. Detect a mode color • Detect a mode color from

    the central portion • Fashion items usually appear in center of photo • Central portion is less affected by background colors • Detect a mode color • Crop the central portion • Reduce colors into 8 • Composite background mask • Make histogram and detect a mode 
  13. Choose nearest color palette • RGB to 12 colors •

    255*255*255ʢ16,581,375ʣ → 12 • Use color name survey[3] • 196,608 RGB values are named • We converted to HSL and used the range of each color names • Choose nearest color palette • Conver RGB to HSL • Choose nearest palette considering HSL range  IUUQCMPHYLDEDPNDPMPSTVSWFZSFTVMUT
  14. Accuracy of color detection • Checked 30 items visually in

    major categories • Average is 48.36% • Well appeared colors are well • Black: 90.67% • Red: 79.33% • Pink: 72.67% • White: 56.67% • Less appeared colors affected by false detection of other colors 
  15. Item found ratio in search result • We defined item

    found proportion • Percentage of users who opened a item in search result • Item found • Keyword search: about 69% • Color search: about 75% • Color search become easier to find items than keyword search 
  16. Summary • We implemented color search • Researched user buying

    behavior and implemented fashion specialized search • Color palette were created from human recognition research and search queries • Correcting color, removing background, cropping center and detect nearest color palette • Color search become easier to find items than keyword search