ABテストが上手く行かなかった話

Transcript

1. 鸐濼鏩閉桦ぢ♳ך׋׭ך倵瘻ך
   "#ذأزדֲתְַֻזַ׏׋鑧 傈劤穗幥倜耀爡խ⚥䊛
   ♰蛁㣽 Otemachi.swift #02
2. Marina Nakagawa @marina10172013 ೔ຊܦࡁ৽ฉࣾͷiOSΤϯδχΞ
 ిࢠ൛ΞϓϦɼࢴ໘ϏϡʔΞʔΞϓϦͷ։ൃ
3. ຊ೔࿩͢͜ͱ ࣮ࡍʹऔΓ૊Μͩ͜ͱ ௨஌ڐ୚཰Λ޲্ͤ͞ΔͨΊͷࢪࡦͷ֓ཁ 1 2 3 ݁Ռͱ·ͱΊ
4. ௨஌ڐ୚཰Λ޲্ͤ͞ΔͨΊͷࢪࡦͷ֓ཁ 1
5. ೔ܦిࢠ൛ͷ௨஌ڐ୚཰ʹ͍ͭͯ • ిࢠ൛ͷ௨஌ͷڐ୚ͷΞϥʔτΛग़͢ͷ͸ىಈ࣌ • ςετ։࢝࣌ͷڐ୚཰͸໿52ˋҐ Կ΋͠ͳͯ͘΋໿52%͘Β͍ͷਓ͸ ڐ୚ͯ͘͠Ε͍ͯΔ
6. ڐ୚཰52%ͷײ֮ New data shows up to 60% of users opt-out of push notifications (Guest Post)ΑΓҾ༻
 (https://andrewchen.co/why-people-are-turning-off-push/)
7. ڐ୚཰52%ͷײ֮ New data shows up to 60% of users opt-out of push notifications (Guest Post)ΑΓҾ༻
 (https://andrewchen.co/why-people-are-turning-off-push/)
8. ڐ୚཰52%ͷײ֮ New data shows up to 60% of users opt-out of push notifications (Guest Post)ΑΓҾ༻
 (https://andrewchen.co/why-people-are-turning-off-push/) ҰൠతͳχϡʔεͷΞϓϦͱ
 ൺֱ͢Δͱଟ͍
9. ௨஌Λ։͘ͱ௚઀هࣄʹඈͿΑ͏ʹͳͬͨ
10. ࠓͦ͜௨஌Λ࢖ͬͯ΄͍͠ɾɾɾ
11. ௨஌ڐ୚཰Λ΋ͬͱ΋ͬͱ্͛ͯ! ϢʔβʔʹχϡʔεΛಧ͚͍ͨ
12. ௨஌ΛڐՄͯ͠΋Β͏ʹ͸ ৐Γӽ͑Δඞཁ͕͋Δน͕͋Δ
14. Ծઆ ΞϓϦͷॳճΠϯετʔϧ࣌ʹ
 ϓογϡ௨஌ͷҰྫΛࣔ͠༗༻Ͱ͋Δ͜ͱΛࣔͤΕ͹
 ௨஌ͷڐ୚཰্͕͕ΔͷͰ͸ͳ͍͔
15. ࣮ࡍʹऔΓ૊Μͩ͜ͱ 2
16. ը໘ભҠਤ
17. ϙοϓΞοϓը໘ͷৄࡉ ͜ͷϘλϯΛԡͨ͠ਓ͚͕ͩ
 OSͷڐ୚௨஌ͷϙοϓΞοϓ͕Ͱͯ͘Δ
18. ϙοϓΞοϓը໘ͷৄࡉ
19. ϙοϓΞοϓը໘ͷৄࡉ ࠓ͸΍Ί͓ͯ͘ϘλϯΛԡͨ͠ਓ ࿮֎λοϓΛͨ͠ਓ ԼεϫΠϓΛͨ͠ਓ
20. ϙοϓΞοϓը໘ͷৄࡉ
21. ݁Ռͱ·ͱΊ 3
22. ▪ ϙοϓΞοϓͳ͠ɿ໿40% ▪ ϙοϓΞοϓ͋Γɿ໿21% ௨஌ڐ୚཰:52% ࣮ࢪޙ ࣮ࢪલ A/Bςετͷ݁Ռ
23. A/Bςετͷ݁Ռ ϙοϓΞοϓΛදࣔͨ͠σόΠεͷํ͕
 ௨஌ONͷׂ߹͕൒਺ఔ౓ͩͬͨ
24. A/Bςετͷ݁Ռ ϙοϓΞοϓΛදࣔͨ͠σόΠεͷํ͕
 ௨஌ONͷׂ߹͕൒਺ఔ౓ͩͬͨ མͪͯΔɾɾɾɾɾ "
25. ݪҼ͸ʁ Ϣʔβʔ͕཭୤͢Δͱ͜ΖΛ࡞ͬͯ͠·ͬͨ 1 2 ϝϦοτΛϢʔβʔʹ఻͖͑Ε͍ͯͳ͔ͬͨ
26. Ϣʔβʔ͕཭୤͢Δͱ͜ΖΛ࡞ͬͯ͠·ͬͨ 1
27. Ϣʔβʔ͕཭୤͢Δͱ͜ΖΛ࡞ͬͯ͠·ͬͨ 1
28. Ϣʔβʔ͕཭୤͢Δͱ͜ΖΛ࡞ͬͯ͠·ͬͨ 1 ͜ͷϘλϯΛԡͨ͠ਓ͚͕ͩ
 OSͷڐ୚௨஌ͷϙοϓΞοϓ͕Ͱͯ͘Δ
29. ࠓ͸΍Ί͓ͯ͘ϘλϯΛԡͨ͠ਓ ࿮֎λοϓΛͨ͠ਓ ԼεϫΠϓΛͨ͠ਓ ໿70%ͷਓ͕OSͷڐ୚௨஌Λ
 ݟͳ͍͜ͱʹͳͬͯ͠·ͬͨ Ϣʔβʔ͕཭୤͢Δͱ͜ΖΛ࡞ͬͯ͠·ͬͨ 1
30. ◦ཁૉΛͻͱ͚ͭͩม͑ͯςετ͢Δͷ͕A/Bςετͳͷʹɺ ෳ਺ཁૉͰൺֱͤͯ͞͠·ͬͨ
 ◦ࣗ෼͕ίϯςϯπͷத਎(ಈըͷ੍࡞΍දهͷ಺༰)ʹूத͠ ͯ͠·͍ભҠͷ͜ͱ·Ͱߟ͍͑ͯͳ͔ͬͨ ◦ԼϑϦοΫɺແࢹ͢Δͱ͍͏બ୒ࢶ͸ແͯ͘͠Α͔ͬͨ ഊҼ Ϣʔβʔ͕཭୤͢Δͱ͜ΖΛ࡞ͬͯ͠·ͬͨ 1
31. Ծઆ ϓογϡ௨஌ͷҰྫΛࣔ͠༗༻Ͱ͋Δ͜ͱΛࣔͤΕ͹ ௨஌ͷڐ୚཰্͕͕Δ͔͔֬ΊΔ ࢼ͢͜ͱ OSͷ௨஌ڐ୚͕ग़ΔલʹϙοϓΞοϓը໘Ͱ
 ༗༻Ͱ͋Δ͜ͱΛઆ໌͢Δ͜ͱͰ௨஌ڐ୚͸มΘΔͷ͔ Ϣʔβʔ͕཭୤͢Δͱ͜ΖΛ࡞ͬͯ͠·ͬͨ 1
32. Ծઆ ϓογϡ௨஌ͷҰྫΛࣔ͠༗༻Ͱ͋Δ͜ͱΛࣔͤΕ͹ ௨஌ͷڐ୚཰্͕͕Δ͔͔֬ΊΔ ࢼ͢͜ͱ OSͷ௨஌ڐ୚͕ग़ΔલʹϙοϓΞοϓը໘Ͱ
 ༗༻Ͱ͋Δ͜ͱΛઆ໌͢Δ͜ͱͰ௨஌ڐ୚͸มΘΔͷ͔ Ϣʔβʔ͕཭୤͢Δͱ͜ΖΛ࡞ͬͯ͠·ͬͨ 1 มߋ͢Δ΂͖͸ϙοϓΞοϓը໘Λग़͔͢ग़͞ͳ͍͔Ͱ͋Γ
 OSͷ௨஌ڐ୚ը໘Λग़͔͢ग़͞ͳ͍͔มߋΛՃ͑Δ΂͖Ͱ͸ͳ͍
33. ʮࠓ஌Δ΂͖ʯΛಀ͞ͳ͍ ศརͳϓογϡ௨஌ 2 ϝϦοτΛϢʔβʔʹ఻͖͑Ε͍ͯͳ͔ͬͨ ◦ҙࢥܾఆΛઌʹ͖͚࣋ͬͯͨͲ৘ใ͕গͳ͍͔Β͔΋͠Εͳ͍
 ◦ಈը͸ॏཁͰ͸ͳ͍͔΋͠Εͳ͍
 ഑ஔͷ໰୊ͰจݴͱϘλϯΛ͚ۙͮͨ΄͏͕Α͔͔ͬͨ΋ ◦Ϙλϯͷબ୒ࢶͰ͋Δ
 ʮ͍·͸΍Ί͓ͯ͘ʯ͸·ͩνϟϯε͕༗ΔΑ͏ʹݟ͑Δ
34. ·ͱΊ A/BςετΛ࣮ࢪ͢Δ࣌͸ɺ
 ຊ౰ʹཁૉΛͻͱ͚ͭͩม͑ͯൺֱͯ͠Δͷ͔֬ೝ͢Δ ϙοϓΞοϓʹදࣔͨ͠ίϯςϯπ͕ຊ౰ʹྑ͔ͬͨͷ͔Ͳ͏͔͸Θ͔Βͳ͍
 จݴ΍ίϯςϯπʹΑͬͯҧ͍͕͋Δͷ͔ͱ͍͏ͷ΋ࢼͯ͠Έ͍ͨ
35. ͋Γ͕ͱ͏͍͟͝·ͨ͠