Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
Missspell Detection
Search
bk
February 10, 2020
Science
180
1
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
Missspell Detection
bk
February 10, 2020
Transcript
44 slides
ฤूڑʹΑΔจࣈྻޡදهݕ ϨʔϕϯγϡλΠϯڑͱδϟϩɾΟϯΫϥʔڑ
࣍ 1. ՝……………………………………p.3-10 2. ࡞ͬͨͷ……………………………p.11-16 3. ฤूڑ………………………………p.17-39 4. ݁Ռ……………………………………p.40-41 5.·ͱΊ…………………………………p.42
6.ࢀߟจݙ………………………………p.43
՝
՝ ϒϥϯυͷࡏݿ
՝ flea ख࡞ۀͰग़
՝ flea
՝ GUCCI Tote Bag Black Leather flea ग़লྗԽ
՝ GUCCHI Tote Bag Black Leather flea
՝ GUCCHI Tote Bag Black Leather flea • ग़औΓফ͠ •
ग़ऀධՁԼ • ΞΧϯτఀࢭ ϒϥϯυ໊ޡදهͷ ϖφϧςΟ
՝ AIͰͳΜͱ͔ͯ͠ Python ࣗવݴޠॲཧ
࡞ͬͨͷ
࡞ͬͨͷ ग़λΠτϧϦετ GUCCHI Tote Bag Black Leather ɾɾɾ ɾɾɾ ɾɾɾ
ɾɾɾ
ग़λΠτϧϦετ GUCCHI Tote Bag Black Leather ɾɾɾ ɾɾɾ ɾɾɾ ɾɾɾ
୯ޠʹղ ग़୯ޠϦετ GUCCHI Tote Bag Black Leather ࡞ͬͨͷ
ग़λΠτϧϦετ GUCCHI Tote Bag Black Leather ɾɾɾ ɾɾɾ ɾɾɾ ɾɾɾ
୯ޠʹղ ग़୯ޠϦετ GUCCHI Tote Bag Black Leather ਖ਼ϒϥϯυ໊Ϧετ GUCCI VUITTON ɾɾɾ ɾɾɾ ɾɾɾ ࡞ͬͨͷ
ग़λΠτϧϦετ GUCCHI Tote Bag Black Leather ɾɾɾ ɾɾɾ ɾɾɾ ɾɾɾ
୯ޠʹղ ग़୯ޠϦετ GUCCHI Tote Bag Black Leather ਖ਼ϒϥϯυ໊Ϧετ GUCCI VUITTON ɾɾɾ ɾɾɾ ɾɾɾ ૯ͨΓ ࣅͨ୯ޠΛग़ྗ ࡞ͬͨͷ
ग़λΠτϧϦετ GUCCHI Tote Bag Black Leather ɾɾɾ ɾɾɾ ɾɾɾ ɾɾɾ
୯ޠʹղ ग़୯ޠϦετ GUCCHI Tote Bag Black Leather ਖ਼ϒϥϯυ໊Ϧετ GUCCI VUITTON ɾɾɾ ɾɾɾ ɾɾɾ ૯ͨΓ ࣅͨ୯ޠΛग़ྗ ࡞ͬͨͷ
ฤूڑ
ฤूڑ 1. ϨʔϕϯγϡλΠϯڑ (Levenshtein Distance) 2. δϟϩɾΟϯΫϥʔڑ (Jaro-Winkler Distance) GUCCHI
GUCCI
ฤूڑ 1. ϨʔϕϯγϡλΠϯڑ (Levenshtein Distance) 2. δϟϩɾΟϯΫϥʔڑ (Jaro-Winkler Distance) GUCCHI
GUCCI 1. ϨʔϕϯγϡλΠϯڑ (Levenshtein Distance)
ฤूڑʢϨʔϕϯγϡλΠϯڑʣ ͋Δจࣈྻ ൺֱ͢Δจࣈྻ จࣈΛૢ࡞ͯ͠Ұகͤ͞Δ
͋Δจࣈྻ ൺֱ͢Δจࣈྻ จࣈΛૢ࡞ͯ͠Ұகͤ͞Δ ૢ࡞ ஔ আ ૠೖ ૢ࡞ճ=ڑ ฤूڑʢϨʔϕϯγϡλΠϯڑʣ
ஔ ݩͷจࣈྻ G U T T I ൺֱ͢Δจࣈྻ G U
C C I ஔ ૢ࡞ճ = ڑ = 2 ฤूڑʢϨʔϕϯγϡλΠϯڑʣ
ஔ আ ૠೖ GUTTI GUCCI GUCCHI GUCCI GUCI GUCCI ฤूճʢڑʣ
2 1 1 ݩͷจࣈྻ ൺֱ͢Δจࣈྻ ฤूํ๏ ฤूڑʢϨʔϕϯγϡλΠϯڑʣ
ฤूڑ 1. ϨʔϕϯγϡλΠϯڑ (Levenshtein Distance) 2. δϟϩɾΟϯΫϥʔڑ (Jaro-Winkler Distance) GUCCHI
GUCCI
Dj = 1 3 * ( m |s1 | +
m |s2 | + m − t 2 m ) s1, s2 ɿจࣈྻͷ͞ mɿ۠ؒͷҰகจࣈ tɿҰகจࣈͷஔ δϟϩڑɿ จࣈྻͷ෦తͳҰக߹͍ΛଌΔ ͕େ͖͍ํ͕ڑ͕͍ۙ ฤूڑʢδϟϩɾΟϯΫϥʔڑʣ
Dj = 1 3 * ( m |s1 | +
m |s2 | + m − t 2 m ) m m m m s1, s2 ɿจࣈྻͷ͞ mɿ۠ؒͷҰகจࣈ tɿҰகจࣈͷஔ δϟϩڑɿ จࣈྻͷ෦తͳҰக߹͍ΛଌΔ ͕େ͖͍ํ͕ڑ͕͍ۙ ฤूڑʢδϟϩɾΟϯΫϥʔڑʣ
mɿ۠ؒͷҰகจࣈ max(|s1 |, |s2 |) 2 − 1 ݩͷจࣈྻɿGCCUHI →
6 ൺֱ͢ΔจࣈྻɿGUCCI → 5 max(6,5) 2 − 1 = 2 ฤूڑʢδϟϩɾΟϯΫϥʔڑʣ
mɿ۠ؒͷҰகจࣈ ݩͷจࣈྻ G C C U H I ൺֱ͢Δจࣈྻ G
U C C I ۠ؒͰҰகจࣈΛݕࡧ Ұகจࣈ͕͋ΕΧϯτ ฤूڑʢδϟϩɾΟϯΫϥʔڑʣ
mɿ۠ؒͷҰகจࣈ ݩͷจࣈྻ G C C U H I ൺֱ͢Δจࣈྻ G
U C C I m = 5 ฤूڑʢδϟϩɾΟϯΫϥʔڑʣ
Dj = 1 3 * ( m |s1 | +
m |s2 | + m − t 2 m ) t s1, s2 ɿจࣈྻͷ͞ mɿ۠ؒͷҰகจࣈ tɿҰகจࣈͷஔ จࣈྻͷ෦తͳҰக߹͍ΛଌΔ ͕େ͖͍ํ͕ڑ͕͍ۙ ฤूڑʢδϟϩɾΟϯΫϥʔڑʣ
tɿҰகจࣈͷஔ ݩͷจࣈྻ G C C U H I ൺֱ͢Δจࣈྻ G
U C C I Ұகͨ͠จࣈΛநग़ ݩͷจࣈྻ G C C U I ൺֱ͢Δจࣈྻ G U C C I ฤूڑʢδϟϩɾΟϯΫϥʔڑʣ
tɿҰகจࣈͷஔ ݩͷจࣈྻ G C C U I ൺֱ͢Δจࣈྻ G U
C C I t = 2 ฤूڑʢδϟϩɾΟϯΫϥʔڑʣ ಉҰͷจࣈྻʹ͢ΔҝʹԿจࣈஔ͢Δͷ͔
Dj = 1 3 * ( m |s1 | +
m |s2 | + m − t 2 m ) s1, s2 ɿจࣈྻͷ͞ mɿ۠ؒͷҰகจࣈ tɿҰகจࣈͷஔ = 1 3 * ( 5 6 + 5 5 + 5 − 2 2 5 ) = 79 90 = 0.8777... ฤूڑʢδϟϩɾΟϯΫϥʔڑʣ
Djw = Dj + l * 1 10 * (1
− Dj ) Dj ɿJaro Distance lɿઌ಄͔ΒͷҰகจࣈʢl <= 4ʣ δϟϩɾΟϯΫϥʔڑɿ ઌ಄จࣈͷҰகॏΈΛ͚ͭͯධՁ ฤूڑʢδϟϩɾΟϯΫϥʔڑʣ
Djw = Dj + l * 1 10 * (1
− Dj ) Dj ɿJaro Distance lɿઌ಄͔ΒͷҰகจࣈʢl <= 4ʣ l δϟϩɾΟϯΫϥʔڑɿ ઌ಄จࣈͷҰகॏΈΛ͚ͭͯධՁ ฤूڑʢδϟϩɾΟϯΫϥʔڑʣ
lɿઌ಄͔ΒͷҰகจࣈʢl <= 4ʣ ݩͷจࣈྻ G C C U H I
ൺֱ͢Δจࣈྻ G U C C I l = 1 ฤूڑʢδϟϩɾΟϯΫϥʔڑʣ
Djw = Dj + l * 1 10 * (1
− Dj ) Dj ɿJaro Distance lɿઌ಄͔ΒͷҰகจࣈʢl <= 4ʣ = 79 90 + 1 * 1 10 * (1 − 79 90 ) = 801 900 = 0.89 ฤूڑʢδϟϩɾΟϯΫϥʔڑʣ
* https://github.com/ztane/python-Levenshtein/ **https://github.com/nap/jaro-winkler-distance Levenshteinɿখ͍͞΄Ͳ͍ۙ Jaro-Winklerɿେ͖͍΄Ͳ͍ۙ ݩͷจࣈྻ ൺֱ͢Δ จࣈྻ *Levenshtein **Jaro-Winkler
GUCCHI GUCCI 1 0.97 GUTTI 2 0.79 GCCUHI 3 0.89 άον༟ࡾ 5 0.00 ฤूڑ
ݩͷจࣈྻ ൺֱ͢Δ จࣈྻ *Levenshtein **Jaro-Winkler GUCCHI GUCCI 1 0.97 GUTTI
2 0.79 GCCUHI 3 0.89 άον༟ࡾ 5 0.00 Jaro-WinklerҰக͢Δจࣈ͕ ଘࡏ͍ͯ͠Δ͜ͱΛධՁ͍ͯ͠Δɻ LevenshteinͱJaro-WinklerͰ ۙ͞ͷॱং͕ҟͳΔɻ ฤूڑ * https://github.com/ztane/python-Levenshtein/ **https://github.com/nap/jaro-winkler-distance
݁Ռ
.py ͳΜ͔ಈ͍ͯΔ͔Βྑ͠ ݁Ռ * https://github.com/bk-18/Misspelled-Brand-Name-Detector
·ͱΊ • ग़࣌ͷϒϥϯυ໊ޡදهͱ͍͏՝ • Ϧετ૯ͨΓʹΑΔޡදهݕ • ϨʔϕϯγϡλΠϯڑ • δϟϩɾΟϯΫϥʔڑ
ࢀߟจݙ • ̎ͭͷจࣈྻͷྨࣅΛԽɹϨʔϕϯγϡλΠϯڑͱδϟϩɾΟ ϯΫϥʔڑͷղઆ, ਓೳͰ͋ͦͿ, http://nkdkccmbr.hateblo.jp/entry/ 2016/08/18/102727 • ฤूڑ (Levenshtein
Distance), naoyaͷͯͳμΠΞϦʔ, https:// naoya-2.hatenadiary.org/entry/20090329/1238307757 • จࣈྻྨࣅධՁ ϨʔϕϯγϡλΠϯڑ / δϟϩɾΟϯΫϥʔڑ, ਓೳͯ͠ΈΔ, http://grahamian.hatenablog.com/entry/word_similarity • Yaoshu Wang(B) , Jianbin Qin, and Wei Wang,: Efficient Approximate Entity Matching Using Jaro-Winkler Distance, Univeristy of New South Wales, http://qinjianbin.com/files/wise2017-wang.pdf
ENJOY! ENJAY! EMJOY! ENJOI! ENZYOI!
More Decks by bk
See All by bk
Befriending Kurtosis with R
bk_18
1
1k
tidy_rpart
bk_18
1
1.8k
dotdotdot_in_predict_function
bk_18
1
1.2k
Introduction_of_GoogleAnalytics_with_R
bk_18
2
1k
web scraping with polite package
bk_18
2
860
start-salesforce-with-r
bk_18
0
940
About Missing Values
bk_18
1
430
Other Decks in Science
See All in Science
CAD / B-rep の ML 解析と DreamCAD — ECCV 2026 読み会
daigo0927
0
380
Засідання студентського наукового гуртка «Садово-паркове господарство»
mariiabarkan12
0
100
データベース11: 正規化(1/2) - 望ましくない関係スキーマ
trycycle
PRO
0
1.8k
Path integral polarons for real semiconductors
jarvist
0
130
Utiliser Bitcoin sans Internet
rlifchitz
0
460
[Webinaire InnOvin] Coup de Chaud : Comment préserver la santé des animaux
institutdelelevage
PRO
0
280
Build your own LLM, Live, with MicroGPT
ianozsvald
0
180
Génétique moléculaire de la mortalité
institutdelelevage
PRO
0
370
チュートリアル:世界モデル
hf149
0
2.5k
20260820_アウトカムが二値のデータに対するCausal Impact@LINEヤフー Data Science Share #2 / Causal Impact for Binary Outcomes
brainpadpr
3
1.6k
AI(人工知能)の過去・現在・未来 ~AIは人類を越えるのか~
tagtag
PRO
0
190
Conwayの法則を"ちゃんと"使うために — 原典でConwayは何を言っていたのか
bonotake
10
7.9k
Featured
See All Featured
For a Future-Friendly Web
brad_frost
184
11k
HDC tutorial
michielstock
2
1k
The Anti-SEO Checklist Checklist. Pubcon Cyber Week
ryanjones
0
300
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
287
14k
Tips & Tricks on How to Get Your First Job In Tech
honzajavorek
1
850
Sam Torres - BigQuery for SEOs
techseoconnect
PRO
0
610
Building AI with AI
inesmontani
PRO
1
1.4k
Are puppies a ranking factor?
jonoalderson
2
4.2k
The Pragmatic Product Professional
lauravandoore
37
7.5k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
52
6.2k
sira's awesome portfolio website redesign presentation
elsirapls
0
510
The Hidden Cost of Media on the Web [PixelPalooza 2025]
tammyeverts
2
580