ビデオ通話アプリのお話

Skywayサービスを利用して、iOSでWebRTCを実装した話です。

Transcript

1. ϏσΦ௨࿩ΞϓϦͷ͓࿩ Cocoaษڧձؔ੢ #72 2017/04/22 େౡ ޫو@taminif
2. ࣗݾ঺հ • ΢ΣϒϦΦגࣜձࣾWEBΤϯδχΞ • ΞϓϦΤϯδχΞ
 ʢࣗশʣ • ࠓ೔ͷͨΊʹ
 ͍͖ͭͬ͞൅੾͖ͬͯ·ͨ͠ʂ
3. Weblio ͝ଘ஌Ͱ͔͢ʁ
4. http://ejje.weblio.jp/content/Cocoa?dictCode=WEJTY
5. ΦϯϥΠϯࣙॻΛఏڙ͢Δ WebαʔϏε
6. ࣮͸ࣙॻҎ֎ʹ΋ ͜ΜͳαʔϏεΛ΍͍ͬͯ ·͢
7. https://eikaiwa.weblio.jp/
8. ΦϯϥΠϯӳձ࿩αʔϏε
9. Skypeӳձ࿩ͱݴ͑͹ ฉ͍ͨ͜ͱ͋Δํ΋ ଟ͍ͷͰ͸ͳ͍Ͱ͠ΐ͏͔
10. ͦΜͳWeblioӳձ࿩Ͱ͕͢
11. ஫໨ʂ
12. 3݄1೔ΑΓɺWebRTCΛ ࢖༻ͨ͠Ϩοεϯʹ Ҡߦ͠·ͨ͠ɻ
13. WebRTCͱ͸ • W3C͕ఏএ͢ΔP2PϦΞϧλΠϜίϛϡχέʔ γϣϯΛ࣮ݱ͢Δٕज़ • ϓϥάΠϯͳ͠Ͱϒϥ΢βؒͷ΍ΓͱΓ͕Մೳ • ຊ೔͸࢓૊Έʹ͍ͭͯ͸ৄ͘͠΍Γ·ͤΜʂ ->αʔό͕ඞཁͰ೉ͦ͠͏ʹݟ͑·͕͢
 ͜ΕΒΛఏڙ͢ΔαʔϏε͕͋Γ·͢ʂ
14. WebRTCΛ؆୯ʹ࣮ݱ͢Δ αʔϏε ొ࿥͢Ε͹ແྉͰ࢖༻Մೳʂ https://nttcom.github.io/skyway/
15. ֤ϒϥ΢βରԠঢ়گʢPCʣ • Google Chrome • Firefox • Opera • Edge(Creators UpdateͰରԠ༧ఆ) • Safari(ରԠத) ※࠷৽όʔδϣϯΛର৅
16. ֤ϒϥ΢βରԠঢ়گʢεϚϗʣ • Android
 -> ChromeରԠࡁΈʢͦͷଞະݕূʣ • iOS
 -> ඇରԠɾɾɾ
17. iPhone΍iPadͰ͸ Weblioӳձ࿩͕࢖͑ͳ͍
19. ࿩͸มΘͬͯɺ Weblioӳձ࿩͸ֶߍ΍ اۀʹ΋ఏڙ͍ͯ͠·͢
20. ·ͨɺֶߍͷதʹ͸ iPadΛतۀʹ࢖༻͢Δॴ΋ ૿͖͑ͯ·ͨ͠
21. ʮWeblioӳձ࿩ΛiPadͰ ࢖༻Ͱ͖ͳ͍ʁʯ
23. ͔͜͜Β͕ຊ୊Ͱ͢
24. ʮiOSΞϓϦʹWebRTCΛ ࣮૷ͨ͠࿩ʯ
25. https://itunes.apple.com/jp/app/weblio๏ਓӳձ࿩/id1215209006?mt=8
26. ϦϦʔε͠·ͨ͠
27. SkyWayΛ࢖ͬͯ ϏσΦΞϓϦΛ࣮૷ͨ͠ ஌ݟΛڞ༗͠·͢
28. 1. ಋೖฤ
29. ωΠςΟϒΞϓϦ༻ͷ SDKΛ௥Ճ͠·͢
30. ެ͔ࣜΒϥΠϒϥϦ͕഑෍ https://webrtc.org/native-code/ios/
31. SkyWayͰ΋ϥΠϒϥϦ͕഑෍ ʢϥοϐϯάͨ͠ϥΠϒϥϦʣ https://nttcom.github.io/skyway/documentation.html
32. SkyWay iOS SDK • αΠτ͔Βμ΢ϯϩʔυ
 or GithubͰclone
 or CocoaPodsͰΠϯετʔϧ • Objective-CͰॻ͔Ε͍ͯΔͷͰ
 Bridging-HeaderΛ௥Ճͯ͠ಡΈࠐΈ
33. ͦͷଞɺ͍͔ͭ͘ frameworkΛಡΈࠐΉඞཁ ͕͋Γ·ׂ͕͢Ѫ ʢެࣜͷυΩϡϝϯτࢀরʣ
34. 2. νϟωϧฤ
35. ೋͭͷνϟωϧ • ϝσΟΞνϟωϧ
 ө૾΍Ի੠ͷStreamΛ΍ΓͱΓ͢Δνϟωϧ • σʔλνϟωϧ
 ςΩετ΍όΠφϦ౳σʔλΛ΍ΓͱΓ͢Δ νϟωϧ
36. SkyWayͰ͸ mediaConnectionΫϥεͱ dataConnectionΫϥε
37. Connectionʹ Stream΍σʔλΛ౉ͯ͠ ΍ΓͱΓ͠·͢
38. ϏσΦ௨࿩ʹ͸ϝσΟΞνϟ ωϧΛ࢖༻͠·͢
39. mediaConnection // ૬खͷϝσΟΞνϟωϧΛड͚औΔCALLBACKΛઃఆ peer?.on(SKWPeerEventEnum.PEER_EVENT_CALL, callback: {(obj) -> Void in let mediaConnection:SKWMediaConnection = obj as! SKWMediaConnection mediaConnection.answer(self.localMediaStream) }) ※্هҎ֎ʹ΋peer.callWithId(callId, localStream)Ͱ૬खʹͭͳ͙͜ͱ͕Ͱ͖Δ
40. mediaConnectionʹ localStreamΛ౉͠ remoteStreamΛ ड͚औΓ·͢
41. localStreamΛऔಘ // peerΠχγϟϥΠζ SKWNavigator.initialize(peer) // constraintsઃఆ let constraints:SKWMediaConstraints = SKWMediaConstraints.init() // ΧϝϥϞʔυઃఆ constraints.cameraMode = .CAMERA_MODE_ADJUSTABLE // ΧϝϥͷҐஔઃఆ constraints.cameraPosition = .CAMERA_POSITION_FRONT // localMediaStreamऔಘ localMediaStream = SKWNavigator.getUserMedia(constraints)
42. remoteStreamΛऔಘ // MediaConnectionͷCALLBACKΛઃఆ mediaConnection.on(SKWMediaConnectionEventEnum.MEDIACONNECTION_EVENT _STREAM, callback: {(obj) -> Void in self.remoteMediaStream = obj as? SKWMediaStream DispatchQueue.global(qos: .default).async { let remoteVideo:SKWVideo = self.view.viewWithTag(1002) as! SKWVideo remoteVideo.isUserInteractionEnabled = true remoteVideo.addSrc(self.remoteMediaStream, track: 0) } })
43. ΍͓ͬͯ͘͜ͱ
44. ΧϝϥͱϚΠΫͷڐՄΛऔΔ • Info.plistʹઆ໌จΛ༻ҙ͢Δ • ༻ҙͨ͠จ͕σόΠεڐՄऔಘ࣌ʹදࣔ͞ΕΔ • NSCameraUsageDescription • NSMicrophoneUsageDescription • ΧϝϥͱϚΠΫͷڐՄΛऔΔ • AVCaptureDevice • Χϝϥ͕࢖༻ෆՄͩͱgetUserMedia͕ࣦഊ͢Δ͕ɺ
 ϚΠΫ͕࢖༻ෆՄͰ΋getUserMedia͸ࣦഊ͠ͳ͍ • ϚΠΫσόΠεͷ੾Γସ͑͸Ͱ͖ͳ͍
45. σόΠεڐՄऔಘ let cameraStatus = AVCaptureDevice.authorizationStatus(forMediaType: AVMediaTypeVideo) switch (cameraStatus) { case .authorized: // ࢖༻Մೳ case .restricted, .notDetermined: // ࢖༻ڐՄ AVCaptureDevice.requestAccess(forMediaType: AVMediaTypeVideo, completionHandler: {(granted: Bool) in // OK or CancelͰgrantedͷBool͕มΘΔ }) break case .denied: // ࢖༻ෆՄ break }
46. 3. ϋϚͬͨ͜ͱฤ
47. ϋϚͬͨ͜ͱ A. localMediaStream͕औΕͳ͍ B. remoteMediaStream͕өΒͳ͍ C. ϑϨʔϜϨʔτ͕มߋͰ͖ͳ͍
48. A. localMediaStream͕औΕͳ͍
49. localStream͕औΕͳ͍ // peerΠχγϟϥΠζ SKWNavigator.initialize(peer) // constraintsઃఆ let constraints:SKWMediaConstraints = SKWMediaConstraints.init() // ΧϝϥϞʔυઃఆ constraints.cameraMode = .CAMERA_MODE_ADJUSTABLE // ΧϝϥͷҐஔઃఆ constraints.cameraPosition = .CAMERA_POSITION_FRONT // localMediaStreamऔಘ localMediaStream = SKWNavigator.getUserMedia(constraints) 4USFBNܗࣜͰऔಘ͸Ͱ͖Δ͚Ͳ ࣗ෼ͷө૾͕දࣔ͞Εͳ͔ͬͨ ౰ॳ͜ͷ෦෼ͷهड़͕ൈ͚͍ͯͨ
50. localStream͕औΕͳ͍ // peerΠχγϟϥΠζ SKWNavigator.initialize(peer) // constraintsઃఆ let constraints:SKWMediaConstraints = SKWMediaConstraints.init() // ΧϝϥϞʔυઃఆ constraints.cameraMode = .CAMERA_MODE_ADJUSTABLE // ΧϝϥͷҐஔઃఆ constraints.cameraPosition = .CAMERA_POSITION_FRONT // localMediaStreamऔಘ localMediaStream = SKWNavigator.getUserMedia(constraints) ಛʹฦΓ஋΋ͳ͍ؔ਺͕ͩ
    هࡌ͠ͳ͍ͱ໰୊͋Γ
51. B. remoteMediaStream͕өΒͳ͍
52. remoteMediaStream͕өΒͳ͍ // MediaConnectionͷCALLBACKΛઃఆ mediaConnection.on(SKWMediaConnectionEventEnum.MEDIACONNECTION_EVENT _STREAM, callback: {(obj) -> Void in self.remoteMediaStream = obj as? SKWMediaStream DispatchQueue.global(qos: .default).async { let remoteVideo:SKWVideo = self.view.viewWithTag(1002) as! SKWVideo remoteVideo.isUserInteractionEnabled = true remoteVideo.addSrc(self.remoteMediaStream, track: 0) } }) ૬ख͔ΒૹΒΕ͍ͯΔ͸ͣͷө ૾͕өΒͳ͔ͬͨ
53. remoteMediaStream͕өΒͳ͍ // MediaConnectionͷCALLBACKΛઃఆ mediaConnection.on(SKWMediaConnectionEventEnum.MEDIACONNECTION_EVENT _STREAM, callback: {(obj) -> Void in self.remoteMediaStream = obj as? SKWMediaStream DispatchQueue.global(qos: .default).async { let remoteVideo:SKWVideo = self.view.viewWithTag(1002) as! SKWVideo remoteVideo.isUserInteractionEnabled = true remoteVideo.addSrc(self.remoteMediaStream, track: 0) } }) ඳըॲཧ͕ඞཁʹͳΔͷͰ
    NBJO2VFVFͰઃఆ͢Δඞཁ͕͋ͬͨ
54. C. ϑϨʔϜϨʔτ͕มߋͰ͖ͳ͍
55. ܦҢ • ϏσΦ௨࿩Λߦ͏ʹ͸ɺܭࢉ্ԼهͷଳҬ͕ ඞཁʢ1ΫϥΠΞϯτʣ
 Ի੠ɿ500Kbps
 ө૾ɿ2Mbps • ʢಛʹʣֶߍͩͱڞ༗ωοτϫʔΫͷͨΊɺ
 1ΫϥεͰडߨ͢Δͱ஗Ԇ͕ଟ͘ൃੜͨ͠
56. ʮ࢖༻ଳҬΛݮΒͤͳ͍ʁʯ
57. ௐࠪ͢Δ΋ SkyWayͷυΩϡϝϯτʹ ͸هࡌͳ͠ɾɾɾ
58. ͔͠͠ʂ
59. SKMediaConstraintsͷ ϓϩύςΟʹ maxFrameRate͕ ͋Γ·ͨ͠ʂ
60. ࢖͑·ͤΜͰͨ͠
61. ެࣜͰඇରԠͱճ౴͞Ε͍ͯ·ͨ͠ https://support.skyway.io/hc/ja/community/posts/115000352647-ϑϨʔϜϨʔτͷมߋ͕൓ө͞Ε·ͤΜ
62. ϝδϟʔόʔδϣϯͰ ରԠ͞ΕΔ͜ͱΛ ظ଴͍ͨ͠Ͱ͢ʂ
63. ·ͱΊ
64. ·ͱΊ • iOSͰϏσΦ௨࿩ΞϓϦ͸ҙ֎ͱ؆୯ʹ࡞ΕΔ • ҙ֎ͱ͋ͬ͞Γ৹ࠪ΋௨Δ • ϒϥ΢β <-> ΞϓϦؒ΋࢖༻Մೳ • WebRTC͸ଟछଟ༷ͳ༻్͕͋Δʂ
65. Weblioӳձ࿩͸ Web΋ΞϓϦ΋ ·ͩ·ͩWebRTCʹ ྗΛೖΕ͍͖ͯ·͢ʂ
66. ͋Γ͕ͱ͏͍͟͝·ͨ͠