iOSDC20200921: Feature Flagを適切に分類することでA/Bテストの運用コストを下げる

https://fortee.jp/iosdc-japan-2020/proposal/cfd5bd01-c940-42e6-b06c-66edf3bef6f8

Transcript

1. Feature FlagΛద੾ʹ෼ྨ͢Δ ͜ͱͰA/Bςετͷӡ༻ίετ ΛԼ͛Δ iOSDC 2020 2020/09/21 Takeshi Ihara (ABEMA)
2. A/Bςετ ίϯςϯπ഑৴ܕαʔϏεʹ͓͍ͯɺ৽ػೳͷϦϦʔεͷࡍɺKPIͷม Խ͕৽ػೳʹΑΔӨڹ͔ίϯςϯπʹΑΔӨڹ͔ͷ۠ผ͕ࠔ೉ ৽ػೳͷޮՌଌఆͷਫ਼౓޲্ͷͨΊɺ৽ػೳͷ༗ແҎ֎ͷཁҼΛۃྗഉ আ͢Δ͜ͱ͕Ͱ͖ΔA/Bςετ͕༻͍ΒΕΔ
3. ABEMAʹ͓͚Δ৽ػೳͷA/Bςετ • ৽ػೳͷදࣔͷ੾Γସ͑ͷΈ • ϦϦʔεͷલʹଌఆࢦඪͱωΫετΞΫγϣϯΛࡦఆ A. ৽ػೳͷImp/Click (= දࣔ਺/ར༻਺) B. ؔ࿈ػೳͷImp/Click C. ࢹௌ࣌ؒ΍ࢹௌຊ਺ (KPI)
4. Feature Flag • ίʔυ্Ͱಛఆػೳͷදࣔͷ੾Γସ͑ʹ༻͍ΒΕΔ • Feature Toggle, Feature Switchͱ΋ݺ͹ΕΔ • ؅ཧαʔϏεͷྫ A. Firebase A/B Testing B. Launch Darkly
5. A/Bςετ + Feature Flagͷεςʔλε؅ཧ • A/Bςετͷ਺ = Feature Flagͷ਺͕૿͍͑ͯ͘ʹैͬͯɺ֤A/Bςε τʹΑΔίʔυ্ͷ෼ذʹΑΓঢ়ଶͷ਺͕ංେԽ͍ͯ͘͠ • A/Bςετ + Feature FlagͷεςʔλεΛ؅ཧ͠ͳ͍ͱɺະ࣮ࢪͷA/ Bςετ͕ຊ൪ʹ࿐ग़ͨ͠ΓɺݕূࡁͷA/Bςετ͕ίʔυ্ʹ࢒Γ ଓ͚Δ໰୊͕ൃੜ͠͏Δ
6. https://martinfowler.com/articles/feature-toggles.html Feature toggles can be categorized across two major dimensions: how long the feature toggle will live and how dynamic the toggling decision must be.
7. Feature Flagͷ෼ྨ (ΫϥΠΞϯτ) • Release Flag • Ops Flag • Permission Flag • Experiment Flag
8. Release Flag • ຊ൪؀ڥͰܾͯ͠༗ޮʹͳΒͳ͍࣮૷த΍ະςετͷػೳʹରͯ͠ͷ Feature Flag • τϥϯΫϕʔε։ൃͱ૬ੑ͕ྑ͍ • ੜଘظؒ) ୹͍ • มߋن໛) খ ~ த • ྫ) ։ൃதͷ৽ػೳ
9. Ops Flag • γεςϜͷಈ࡞Λ੍ޚ͢ΔͨΊͷFeature Flag • ੜଘظؒ) ௕͍ • มߋن໛) খ ~ த • ྫ) ಈըϓϨΠϠʔͷΤϥʔ࣌ͷϦτϥΠִؒ
10. Permission Flag • ಛఆͷϢʔβʔ͕ड͚औΔػೳ΍੡඼ମݧΛมߋ͢ΔFeature Flag • ੜଘظؒ) ୹͍ • มߋن໛) େ • ྫ) VoiceOverͷମݧ޲্ͷͨΊͷϢʔβʔςετ
11. Experiment Flag • ύϑΥʔϚϯεଌఆ΍A/BςετͷͨΊͷFeature Flag • ੜଘظؒ) ୹͍ • มߋن໛) த ~ େ • ྫ) ϦϦʔεࡁΈͷ৽ػೳ
12. ੜଘظؒ มߋن໛ 3FMFBTF
    'MBH &YQFSJNFOU
    'MBH 0QT
    'MBH 1FSNJTTJPO
    'MBH
13. Code of Feature Flag enum FeatureFlag<Value> { case release(id: String, defaultValue: Value) case ops(id: String, defaultValue: Value) case permission(id: String, defaultValue: Value) case experiment(id: String, defaultValue: Value) var value: Value { #if DEBUG_MENU // DEBUGϝχϡʔͰઃఆ͞Εͨ஋͕͋Ε͹ฦ٫͢Δ #endif if case let .release(_, defaultValue) = self { return defaultValue } else { // ؅ཧπʔϧ͔Βऔಘͨ͠஋͕͋Ε͹ฦ٫͢Δɺͳ͚Ε͹ defaultValueΛฦ٫͢Δ } } } ΫϥΠΞϯτଆͰͷ࣮૷
14. ·ͱΊ • ΫϥΠΞϯτଆͰFeature FlagΛ4छྨʹ෼ྨ A. ؅ཧπʔϧʹґଘ͠ͳ͍ B. ຊ൪؀ڥͰͷ։ൃ్தͷ࿐ग़Λ๷͛Δ • ఆظతʹFeature Flagͷ୨Է͕͠ඞཁ C. ߴස౓) Release Flag, Experiment Flag D. ௿ස౓) Ops Flag, Permission Flag