Back of First Screen AB test of Hatena Bookmark iOS app with Firebase AB Testing

https://google-admob-japan.connpass.com/event/138083/

Transcript

1. 2019/07/22 @Google AdMob Japan Meetup #5 Yutaro Muta @yutailang0119 ͸ͯͳϒοΫϚʔΫiOSΞϓϦͷ First Screen A/Bςετͷཪଆ with Firebase A/B Testing
2. • Yutaro Muta @yutailang0119 • Hatena Co., Ltd. @Kyoto • Conference Staff • builderscon 2017, 2018 • try! Swift Tokyo 2019, 2020 • ࠷ۙͷਪ͠͸Combine framework Who am I ?
3. ࿩͢͜ͱ • ͸ͯͳϒοΫϚʔΫͷiOSΞϓϦͰߦͬͨA/Bςετͷ֓ཁ • ্هʹ͓͚ΔFirebase A/B Testingͷ໾ׂ • ઃఆํ๏ • Tips
4. ࿩͞ͳ͍͜ͱ • AdMobʹΑΔΞϓϦ಺޿ࠂ
5. ͸ͯϒͷεϚϗΞϓϦΛ ࢖͍ͬͯΔਓʙ
6. ͸ͯϒεϚϗΞϓϦ • https:b.hatena.ne.jp ͷεϚϗΞϓϦ • ίϝϯτ෇͖ͰϒοΫϚʔΫͰ͖Δ • ײ૝Λڞ༗Ͱ͖Δ • ࿩୊ͷϖʔδΛൃݟͰ͖Δ • iOS / Android
7. ࠷ۙ͸ͯϒΞϓϦͷ ϗʔϜը໘͕มΘͬͨͳʔ ͱؾ͍ͮͨਓʙ
8. Before
9. After
10. Before After
11. Before After
12. มߋ఺ • First ScreenͰΧςΰϦຖͷਓؾʹ੾Γସ͑ΒΕΔ • ݩʑ͸૯߹ͷਓؾͱ৽ணͷΈ͕දࣔ͞Ε͍ͯͨ • ΧςΰϦͷฒͼॱ͸ɺ೚ҙʹฒͼସ͑ΒΕΔ
13. Ͱ΋ɺ First ScreenΛΤΠϠοͱ มߋ͢Δͷ͸ා͍ͳ͊
14. ͦ͜ͰA/Bςετ
15. A/BςετπʔϧʹٻΊͨ͜ͱ • [Must] A or Bͷ੾Γସ͑Λ֎෦ (ΞϓϦϦϦʔεαΠΫϧ֎) ͔Βߦ͑Δ • ঢ়ଶʹ߹Θͤͯը໘ଆΛ੾Γସ͑Δ • ςετதͷෆ۩߹΍݁Ռʹ໌Β͔ʹେ͖ͳ͕ࠩग़ͨࡍʹɺׂ߹Λมߋ • Ϗδωεతͳ͋Ε͜ΕʹରԠͰ͖ΔΑ͏ • [Better] ෼ੳπʔϧͱͷ૬ੑ΍ݕূͷ͠΍͢͞
16. Firebase A/B Testing
17. Firebase A/B Testing • ͢ͰʹFirebaseϓϥοτϑΥʔϜΛར༻͍ͯ͠Δͱɺಋೖίετ͕গͳ͍ • Google Analytics for Firebaseͱ࿈ܞ͕Մೳ • Remote Configͱͯ͠ѻ͏ͷͰɺखܰ
18. Firebase A/B Testingͷઃఆ
19. ઃఆελʔτ
20. λʔήοτͷઃఆ
21. λʔήοτͷઃఆ Analytics.logEvent(_:, parameters:) ͰૹΔΠϕϯτΛ࢖ͬͯɺ ಛఆͷૢ࡞Λͨ͠ϢʔβʔͷΈΛλʔήοτʹࢦఆՄೳ
22. ໨ඪઃఆ
23. ໨ඪઃఆ
24. όϦΞϯτͷઃఆ
25. ΞϓϦͷ࣮૷
26. Remote Config͔Β஋Λऔಘ // Fetch from Remote Config func fetch() { let expiration: TimeInterval = (60 * 60 * 3) remoteConfig.fetch(withExpirationDuration: expiration, completionHandler: nil) } // Activate fetched Remote Config settings func activateFetched() { remoteConfig.activateFetched() }
27. Remote ConfigͰऔಘͨ͠ઃఆ஋͔Βը໘Λग़͠෼͚Δ let viewController: UIViewController switch remoteConfig.viewSetting { case .a: viewController = AViewController() case .b: viewController = BViewController() } // Add to ContainerView viewController.view.translatesAutoresizingMaskIntoConstraints = false addChild(viewController) containerViewController.addSubview(viewController.view) viewController.didMove(toParent: self) NSLayoutConstraint.activate([NSLayoutConstraint]) // Switch ViewController by Remote Config let viewController: UIViewController switch remoteConfig { case .a: viewController = AViewController() case .b: viewController = BViewController() default: viewController = DefaultViewController() } // Add to ContainerView viewController.view.translatesAutoresizingMaskIntoConstraints = false addChild(viewController) containerViewController.addSubview(viewController.view) viewController.didMove(toParent: self) NSLayoutConstraint.activate([NSLayoutConstraint]())
28. ⷁ ⷂ
29. A/Bςετ։࢝
30. ~͕࣌ྲྀΕͯ~
31. ςετ݁Ռ • ͓Αͦ1ϲ݄΄ͲͷςετΛ࣮ࢪͨ͠ • ςετ࣮ࢪલ͔Β΄΅૝ఆ௨Γͷظؒ • ৽ը໘ͷํ͕ɺچը໘ΑΓ΋ઃఆͨ͠ࢦඪʹ͓͍ͯ༏ҐͳࠩΛಘΒΕͨ • ࢦඪ͸ൿີ
32. ίϯιʔϧը໘
33. ςετ׬ྃ
34. ςετ݁ՌΛݩʹ... • ຬΛ࣋ͯ͠ɺ৽ը໘Λ100%ʹมߋ • AndroidͰ΋৽ը໘Λ࣮૷ɺϦϦʔε • ݁ՌɺAndroidͰ΋ྑ͍݁ՌΛಘΒΕͨ
35. A/B Testing Q&A
36. Q. ॳճىಈ࣌ʹ΋ A/BςετΛ൓ө͍ͨ͠
37. Q. ॳճىಈ࣌ʹ΋A/BςετΛ൓ө͍ͨ͠ • Remote Config͸ɺωοτϫʔΫ௨৴Λ௨ͯ͡ઃఆ஋Λऔಘ͢Δ • ॳճFetch࣌ʹ͸ઃఆ஋͸ແ͍ • ίʔυ্Ͱͷdefault஋͸༻ҙͰ͖Δ • (ಛʹ) First ScreenͷA/BςετͰ͸ɺ൓өλΠϛϯάʹ޻෉͕ඞཁʹͳΔ
38. ॳճFetchͷ޻෉Ҋ 1. Fetch͕׬ྃ͢Δ·Ͱɺը໘Λwait͢Δ Pros: ॳճ΋A/Bςετͷઃఆ஋Λ൓өͰ͖Δ Cons: ଴͕ͪ࣌ؒൃੜɺFetch͕ࣦഊͨ࣌͠͸Ͳ͏͢Δʁ 2. Fetch͕׬ྃͨ͠Βଈ࠲ʹ൓өͤ͞Δ -> ࠷ॳ͸default஋Ͱը໘Λදࣔ -> Fetchޙ͙͢ʹ RemoteConfig.activateFetched() -> ը໘ͷߋ৽ Pros: ࣮૷্ͷߟྀ఺͸গͳ͍ Cons: Ϣʔβʔ͸࢖༻தʹUI͕มߋ͞ΕΔͷͰɺڻ͔͘΋ 3. ࣍ճىಈ࣌·Ͱ଴ͭ -> ىಈ࣌ʹͷΈ RemoteConfig.activateFetched() Pros: ࣮૷্ͷߟྀ఺͸গͳ͍ Cons: 2ճ໨Ҏ߱ͷىಈʹͨͲΓண͚ͳ͍ͱɺςετͰ͖ͳ͍
39. Q. A/Bςετͷσόοά େม͡Όͳ͍ʁ
40. Q. A/Bςετͷσόοάେม͡Όͳ͍ʁ • betaϏϧυͰͷΈ Khan/SwiftTweaks Λ࢖͍ɺ੾Γସ͑Մೳʹͨ͠ • A/BςετҎ֎ʹ΋௨৴΍ͦͷଞઃఆͷ੾Γସ͑ɺσόοάʹ΋࢖͍ͬͯΔ
41. &OKPZ
    "#
    5FTUJOH
    5IBOLT w NVUBZVUBSP!HNBJMDPN
    w IUUQTUXJUUFSDPNZVUBJMBOH
    w IUUQTHJUIVCDPNZVUBJMBOH