Extreme testing of WebRTC Applications

This material is presented at WebRTC Meetup Japan:
https://44145dc9ce5b767cd2eb8b7683.doorkeeper.jp/events/33154

You can see video from:
https://youtu.be/02wAz0Ii-ZU?t=1h33m21s

Transcript

1. "Extreme" Tes+ng of WebRTC Applica+ons
 2015/11/4(Wed) @WebRTC Meetup Japan iwashi86(Yoshimasa Iwase) atakigawa(Akinori Takigawa)
2. ▪Name Yoshimasa Iwase / @iwashi86 ▪Work NTT Communications, DevOps Engineer of SkyWay ▪Name Akinori Takigawa / @kenkenkenken8 ▪Work CTO of Nefrock Inc.
3. Copyright © NTT Communica+ons Corpora+on. All rights reserved. 3 SkyWay = WebRTC plaUorm (CPaaS)
4. TL; DR •  Tes+ng WebRTC plaUorm/app is super hard WebRTCプラットフォーム/アプリのテストは超大変 •  Let’s automate it! and we did it! そこで自動化したく、やってみた！
5. TL; DR •  Tes+ng WebRTC plaUorm/app is super hard WebRTCプラットフォーム/アプリのテストは超大変 •  Let’s automate it! and we did it! そこで自動化したく、やってみた！
6. Why so tough? なぜ大変？
7. Reason 1 : Manual opera+on (手動) Video 1 Video2 ① ② ③
8. Reason 1+α : Tes+ng signaling is not enough Video 1 Video2 　　Signaling Succeeded 　　Video s+ll black シグナリング単体じゃ不十分
9. Reason 2 : Various condi+ons (Clients) + iOS, Android, Na+ve … 条件多すぎ（クライアントだけで)
10. Reason 2+α : Aggressive update of clients like, gUM on HTTP just doesn’t work アップデート早すぎ(クライアントだけで) gUMはHTTPで動かなくなったり 6-8 weeks update!
11. Reason3 : Various condi+ons (Network) NAT-X ① Endpoint Independent Filtering X Endpoint Independent Mapping … over 80 cases (NATタイプだけで発着を考えると80種類超) 条件多すぎ（ネットワークの種類も)
12. Reason3 : Various condi+ons (Network) NAT-X NAT-Y UDP blocked! ① Endpoint Independent Filtering X Endpoint Independent Mapping … over 80 cases (NATタイプだけで発着を考えると80種類超) 条件多すぎ（ネットワークの種類も)
13. Reason3 : Various condi+ons (Network) NAT-X NAT-Y UDP blocked! ① Endpoint Independent Filtering X Endpoint Independent Mapping … over 80 cases (NATタイプだけで発着を考えると80種類超) 条件多すぎ（ネットワークの種類も) Just tes+ng in the same machine isn’t enough
14. TL; DR •  Tes+ng WebRTC plaUorm/app is super hard WebRTCプラットフォーム/アプリのテストは超大変 •  Let’s automate it! and we did it! そこで自動化したく、やってみた！
15. Ubuntu with Xvn Architecture Overview Ubuntu with Xvn NAT instance NAT instance Intranet1 Intranet2 Web Server (hosts test app page) Virtual Internet VPC Network Segment TURN server
16. Browser1 DataChannel Tes+ng Browser2
17. MediaChannel Tes+ng Browser1 Browser2 video (gUM) video (gUM)
18. Browser x Browser x NAT type x NAT type ・Browsers ・ Chrome Stable, Chrome Beta, Chrome Unstable ・ Firefox Stable (, Firefox Unstable) ・NAT types ・Theore+cally 9 types (RFC4787) ・some types not configurable with iptables -> wrote it from scratch ・4 * 4 * 9 * 9 = 1296 uh…
19. NAT types hup://www.netmanias.com/en/?m=view&id=techdocs&no=6065
20. Ubuntu with Xvn Architecture Overview Ubuntu with Xvn NAT instance NAT instance Intranet1 Intranet2 Web Server (hosts test app page) Virtual Internet VPC Network Segment TURN server
21. Ubuntu with Xvn Architecture Overview (Detailed ver.) Intranet1 Intranet2 Web Server (hosts test app page) Virtual Internet VPC Network Segment Ubuntu with Xvn Ubuntu with Xvn Ubuntu with Xvn Ubuntu with Xvn 16 clients NAT Type 9 NAT Type 8 NAT Type 7 NAT Type 6 NAT Type 5 NAT Type 4 NAT Type 3 NAT Type 2 NAT Type 1 Ubuntu with Xvn Ubuntu with Xvn Ubuntu with Xvn Ubuntu with Xvn Ubuntu with Xvn 16 clients NAT Type 9 NAT Type 8 NAT Type 7 NAT Type 6 NAT Type 5 NAT Type 4 NAT Type 3 NAT Type 2 NAT Type 1 TURN server
22. Test result and Packet visualiza+on see -> hup://status.skyway.io
23. Thanks!!