Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
About Missing Values
Search
bk
August 09, 2019
Science
430
1
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
About Missing Values
bk
August 09, 2019
Transcript
51 slides
ܽଛॲཧʹ͍ͭͯ dࣦΘΕͨΛٻΊͯd
࣍ 1. ܽଛͱ 2. ܽଛআڈ 3. ܽଛλΠϓ 4. ୯Ұೖ๏ 5.
ଟॏೖ๏ 6. ࠓճհ͠ͳ͔ͬͨͷ 7. ࢀߟจݙ
ܽଛͱ Missing Valueʢܽଛɺܽଌʣ In statistics, missing data, or missing values,
occur when no data value is stored for the variable in an observation. Missing data are a common occurrence and can have a significant effect on the conclusions that can be drawn from the data. ʢhttps://en.wikipedia.org/wiki/Missing_dataʣ
ܽଛ͕͋Δͱ Կ͕ࠔΔͷ͔ʁ ܽଛͱ
ܭࢉग़དྷͳ͍ Q,10, 20, ܽଛͷฏۉʁ ܽଛͱ > mean(c(10, 20, NA)) [1]
NA
Ͳ͏͢Δͷ͔ʁ ܽଛͱ
1. ফ͢ 2. ຒΊΔ ܽଛͱ
1. ফ͢ 2. ຒΊΔ 1. ফ͢ ܽଛͱ
ܽଛͷ͋ΔϨίʔυΛফ͢ ମॏ ੑผ 50 ঁ 70 உ 55 NA NA
உ ܽଛআڈ
ମॏ ੑผ 50 ঁ 70 உ 55 NA NA உ
ମॏ ੑผ 50 ঁ 70 உ ܽଛͷ͋ΔϨίʔυΛফ͢ ܽଛআڈ
ମॏ ੑผ 50 ঁ 70 உ 55 NA NA உ
ମॏ ੑผ 50 ঁ 70 உ ϦετϫΠζ๏ ܽଛͷ͋ΔϨίʔυΛফ͢ ܽଛআڈ
ফ͍͍ͯ͠ͷʁ 1. ফͯ͠ྑ͍߹ 2. ফͯ͠ବͳ߹ ܽଛআڈ
શʹϥϯμϜʹܽଛ ܽଛύλʔϯੑผ ମॏͱແؔ ܽଛআڈ
શʹϥϯμϜʹܽଛ ܽଛύλʔϯੑผ ମॏͱແؔ ফͯ͠ͳ͍ ͨͩ͠ɺޮԼ͕Δ ܽଛআڈ
؍ଌมʹґଘͯܽ͠ଛ ঁੑͷํ͕ଟܽ͘ଛ ܽଛআڈ
؍ଌมʹґଘͯܽ͠ଛ ফ͢ͱঁੑͷσʔλͷΈݮগ ภΔ ঁੑͷํ͕ଟܽ͘ଛ ܽଛআڈ
ܽଛมࣗମʹґଘͯܽ͠ଛ ମॏͷॏ͍ํ͕ଟ͘ ܽଛ ܽଛআڈ
ܽଛมࣗମʹґଘͯܽ͠ଛ ফ͢ͱମॏͷߴ͍σʔλ͕ݮগ ภΔ ମॏͷॏ͍ํ͕ଟ͘ ܽଛ ܽଛআڈ
‣ શʹϥϯμϜʹܽଛˠMCAR ʢMissing Completely At Randomʣ ‣ ؍ଌมʹґଘͯܽ͠ଛˠMAR ʢMissing At
Randomʣ ‣ ܽଛมࣗମʹґଘͯܽ͠ଛˠNMAR ʢNot Missing At Randomʣ ܽଛλΠϓ
‣ શʹϥϯμϜʹܽଛˠMCAR ʢMissing Completely At Randomʣ ‣ ؍ଌมʹґଘͯܽ͠ଛˠMAR ʢMissing At
Randomʣ ‣ ܽଛมࣗମʹґଘͯܽ͠ଛˠNMAR ʢNot Missing At Randomʣ ܽଛλΠϓ
‣ શʹϥϯμϜʹܽଛˠMCAR ʢMissing Completely At Randomʣ ‣ ؍ଌมʹґଘͯܽ͠ଛˠMAR ʢMissing At
Randomʣ ‣ ܽଛมࣗମʹґଘͯܽ͠ଛˠNMAR ʢNot Missing At Randomʣ ফͯ͠ྑ͍ ܽଛλΠϓ
‣ શʹϥϯμϜʹܽଛˠMCAR ʢMissing Completely At Randomʣ ‣ ؍ଌมʹґଘͯܽ͠ଛˠMAR ʢMissing At
Randomʣ ‣ ܽଛมࣗମʹґଘͯܽ͠ଛˠNMAR ʢNot Missing At Randomʣ ܽଛλΠϓ
‣ શʹϥϯμϜʹܽଛˠMCAR ʢMissing Completely At Randomʣ ‣ ؍ଌมʹґଘͯܽ͠ଛˠMAR ʢMissing At
Randomʣ ‣ ܽଛมࣗମʹґଘͯܽ͠ଛˠNMAR ʢNot Missing At Randomʣ ରॲࠔ ܽଛλΠϓ
‣ શʹϥϯμϜʹܽଛˠMCAR ʢMissing Completely At Randomʣ ‣ ؍ଌมʹґଘͯܽ͠ଛˠMAR ʢMissing At
Randomʣ ‣ ܽଛมࣗମʹґଘͯܽ͠ଛˠNMAR ʢNot Missing At Randomʣ ܽଛλΠϓ
‣ શʹϥϯμϜʹܽଛˠMCAR ʢMissing Completely At Randomʣ ‣ ؍ଌมʹґଘͯܽ͠ଛˠMAR ʢMissing At
Randomʣ ‣ ܽଛมࣗମʹґଘͯܽ͠ଛˠNMAR ʢNot Missing At Randomʣ ܽଛλΠϓ
1. ফ͢ 2. ຒΊΔ 1. ফ͢ 2. ຒΊΔ ୯Ұೖ๏
ͲΜͳͰຒΊΔͷ͔ʁ ୯Ұೖ๏
1. ฏۉʢฏۉೖ๏ʣ 2. ༧ଌʢ֬ఆతճؼೖ๏ʣ 3. ༧ଌ + ࠩʢ֬తճؼೖ๏ʣ + ୯Ұೖ๏
1. ฏۉʢฏۉೖ๏ʣ 2. ༧ଌʢ֬ఆతճؼೖ๏ʣ 3. ༧ଌ + ࠩʢ֬తճؼೖ๏ʣ + ୯Ұೖ๏
ฏۉΛܭࢉ͠ೖ ୯Ұೖ๏
ฏۉΛܭࢉ͠ೖ ฏۉೖ๏ ฏۉ ≒ உੑͷମॏͷฏۉ ୯Ұೖ๏
1. ฏۉʢฏۉೖ๏ʣ 2. ༧ଌʢ֬ఆతճؼೖ๏ʣ 3. ༧ଌ + ࠩʢ֬తճؼೖ๏ʣ + ୯Ұೖ๏
ܽଌΛ༧ଌ͢ΔϞσϧΛ࡞Δ *ิॿม ܽ ଛ ͷ ͋ Δ ม *ิॿม:ܽଛͷ༧ଌʹΘΕΔมɻඞͣ͠తมͷ༧ଌʹΘΕΔΘ͚Ͱͳ͍ɻ
୯Ұೖ๏
ܽଌΛ༧ଌ͢ΔϞσϧΛ࡞Δ ิॿม ܽ ଛ ͷ ͋ Δ ม ୯Ұೖ๏
ܽଌΛ༧ଌ͢ΔϞσϧΛ࡞Δ ܽ ଛ ͷ ͋ Δ ม ʢ֬ఆతʣ ճؼೖ๏
ิॿม ୯Ұೖ๏
ܽ ଛ ͷ ͋ Δ ม ิॿม ೖ͕શͯઢ্ʹ ୯Ұೖ๏
ܽ ଛ ͷ ͋ Δ ม ༧ଌͷޡࠩΛաখධՁ ิॿม ೖ͕શͯઢ্ʹ
୯Ұೖ๏
1. ฏۉʢฏۉೖ๏ʣ 2. ༧ଌʢ֬ఆతճؼೖ๏ʣ 3. ༧ଌ + ࠩʢ֬తճؼೖ๏ʣ 1. ฏۉʢฏۉೖ๏ʣ
2. ༧ଌʢ֬ఆతճؼೖ๏ʣ 3. ༧ଌ + ࠩʢ֬తճؼೖ๏ʣ + ୯Ұೖ๏
ೖϞσϧʹޡ߲ࠩΛՃ ܽ ଛ ͷ ͋ Δ ม ิॿม ޡ߲ࠩ
୯Ұೖ๏
ܽ ଛ ͷ ͋ Δ ม ิॿม ೖϞσϧʹޡ߲ࠩΛՃ Β͖ͭΛө
୯Ұೖ๏
ܽ ଛ ͷ ͋ Δ ม ิॿม ೖϞσϧʹޡ߲ࠩΛՃ ֬తճؼೖ๏
୯Ұೖ๏
ೖϞσϧͷޡ߲ࠩөग़དྷͨ ܽ ଛ ͷ ͋ Δ ม ิॿม ୯Ұೖ๏
ೖϞσϧͦͷͷͷෆ࣮֬ੑʁ ܽ ଛ ͷ ͋ Δ ม ิॿม ʁ
ʁ ʁ ୯Ұೖ๏
1ͭͷΛೖʢ୯Ұೖ๏ʣ ෳͷΛೖ ʢଟॏೖ๏ʣ ଟॏೖ๏
ೖ ੳ ౷߹ ෳͷೖΛਪఆ͠ෳͷೖࡁΈσʔλΛੜ ೖࡁΈσʔλΛ༻ͯ͠ਪఆ ਪఆΛ౷߹͠ɺ࠷ऴ݁Ռͱ͢Δ ଟॏೖ๏
ܽଛσʔλ ೖࡁΈσʔλ1 ೖࡁΈσʔλ2 ೖࡁΈσʔλ3 ੳ݁Ռ1 ੳ݁Ռ2 ੳ݁Ռ3 ࠷ऴ݁Ռ ೖ ੳ
౷߹ ଟॏೖ๏
ܽଛσʔλ ೖࡁΈσʔλ1 ೖࡁΈσʔλ2 ೖࡁΈσʔλ3 ੳ݁Ռ1 ੳ݁Ռ2 ੳ݁Ռ3 ࠷ऴ݁Ռ ೖ ੳ
౷߹ ଟॏೖ๏
‣ DA๏ : Data Augmentationʢσʔλ֦େ๏ʣ ‣ FCS๏ : Fully Conditional
Specificationʢ શ͖݅ࢦఆʣ ‣ EMB๏ : Expectation-Maximization with Bootstrapping ଟॏೖ๏
‣ ܽଛআڈɿϖΞϫΠζ๏ɺมআڈ ‣ ୯Ұೖ๏ɿൺೖ๏ɺϗοτσοΫ๏ɺίʔ ϧυσοΫ๏ɺLOCFɺNOCB ‣ ଟॏೖ๏ɿೖஅɺײੳ ‣ શใ࠷๏ ‣
LightGBMͳͲͷܽଛࣗಈॲཧ ࠓճհ͠ͳ͔ͬͨͷ
‣ ߴڮকٓ,ลඒஐࢠ, ܽଌσʔλॲཧ: RʹΑΔ୯Ұೖ๏ͱ ଟॏೖ๏ (౷ܭֶOne Point), ڞཱग़൛, 2017/12/9, 192ϖʔ
δ ‣ ܽଛ͕͋Δσʔλͷੳ, Sunny side up!, http:// norimune.net/1811 ‣ RͰ࣮ફʂܽଛσʔλੳೖʲ1ʳ, NHN TECHORUS Tech Blog, https://techblog.nhn-techorus.com/archives/6573 ‣ R ܽଛͷରԠʢmissing value treatmentʣ, ౷ܭֶͱӸֶͱ ࣌ʑɺॿڭੜ׆, http://jojoshin.hatenablog.com/entry/ 2017/02/03/220118 ࢀߟจݙ
ENJOY! A
More Decks by bk
See All by bk
Befriending Kurtosis with R
bk_18
1
1k
tidy_rpart
bk_18
1
1.8k
dotdotdot_in_predict_function
bk_18
1
1.2k
Introduction_of_GoogleAnalytics_with_R
bk_18
2
1k
web scraping with polite package
bk_18
2
860
start-salesforce-with-r
bk_18
0
940
Missspell Detection
bk_18
1
180
Other Decks in Science
See All in Science
「念のためのログ保存」を組織全体でやめるためのポリシーと仕組み作り
i2tsuki
4
430
JSAI2026企画セッションKS-14 インタビュー集『⼈⼯知能と哲学と四つの問い』が提起する⼈⼯知能のこれからの課題 趣旨説明 / JSAI2026 Special Session: A Collection of Interviews, “Artificial Intelligence, Philosophy, and Four Questions”
ykiyota
0
500
生成AI時代のインタラクションサイエンス(招待講演)/interaction-science
okana2ki
0
130
JPA2026_NetworkSymposium_Kashihara
junkashihara
0
130
Génétique moléculaire de la mortalité
institutdelelevage
PRO
0
370
「AIエージェントを作る」とは、何を設計することか
salt2
0
270
Note agro-climatique et prairies - Numéro 7
institutdelelevage
PRO
0
740
プレイがつなぐ研究と社会:市⺠‧クリエイター‧SF作家による実践を通じて
hayataka88
1
140
生成AIが科学とRAにもたらしていること:メタサイエンスの視点から
rmaruy
0
180
データベース05: SQL(2/3) 結合質問
trycycle
PRO
0
1.4k
JPA2026_NetworkSymposium_Shimotsukasa
junkashihara
0
120
JPA2026_NetworkSymposium_Overview
junkashihara
0
130
Featured
See All Featured
Fashionably flexible responsive web design (full day workshop)
malarkey
409
67k
Making the Leap to Tech Lead
cromwellryan
135
10k
Music & Morning Musume
bryan
48
7.5k
What the history of the web can teach us about the future of AI
inesmontani
PRO
3
810
Chasing Engaging Ingredients in Design
codingconduct
0
400
Navigating Weather and Climate Data
rabernat
0
590
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
50
10k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
230
23k
Pawsitive SEO: Lessons from My Dog (and Many Mistakes) on Thriving as a Consultant in the Age of AI
davidcarrasco
1
300
Game over? The fight for quality and originality in the time of robots
wayneb77
1
330
A Guide to Academic Writing Using Generative AI - A Workshop
ks91
PRO
1
570
10 Git Anti Patterns You Should be Aware of
lemiorhan
PRO
659
62k