WebRTC 101 - How to get started building your first WebRTC application

How to get started with WebRTC, what are all the terms you have to care about? When do you need to care about them?

Transcript

1. WebRTC 101 for Developers How to get started building your first WebRTC application
2. Dan Jenkins @dan_jenkins Google Developer Expert (Web Technologies specialising in WebRTC) Founder Nimble Ape Web Developer - not a typical telecommunications developer General Geek Lego Geek
3. Nimble Ape @nimbleapeltd WebRTC Development & Consulting IoT | Microservices | Docker Web APIs & Scalable Web Services Asterisk Applications Node.js Development & Consulting
4. Real Time Communication What is WebRTC? Web
5. Who's built something with WebRTC?
6. Draft or Spec? It's currently a Working Draft and not a Specification. Expect a Specification soon. Don't let this stop you building things with it. Both the IETF & W3C involved. https://www.w3.org/TR/webrtc/
7. Peer to Peer P2P?
8. Encrypted by design Secure!
9. Media Channel
10. Data Channel { "code": 200, "status": "Ok", "copyright": "Ž 2016 MARVEL", "attributionText": "Data provided by Marvel. Ž 2016 MARVEL", "attributionHTML": "<a href=\"http://marvel.com\">Data provided by Marvel. Ž 2016 MARVEL</a>", "etag": "051a4391a8d0f1ca2c9a8da7bf3345e2e8a47f0f", "data": { "offset": 0, "limit": 20, "total": 1, "count": 1, "results": [ { "id": 1009220, "name": "Captain America", "description": "Vowing to serve his country any way he could, young Steve Rogers took the super soldier serum to become America's one-man army. Fighting for the red, white and blue for over 60 years, Captain America is the living, breathing symbol of freedom and liberty.", "modified": "2014-06-10T16:13:04-0400", "thumbnail": { "path": "http://i.annihil.us/u/prod/marvel/i/mg/3/50/537ba56d31087", "extension": "jpg" }, "resourceURI": "http://gateway.marvel.com/v1/public/characters/1009220" } ] } } Strings
11. Data Channel Files
12. Data Channel Pretty much anything...
13. getUserMedia RTCPeerConnection RTCDataChannel Core APIs
14. Signalling Server
15. How your WebRTC endpoints communicate vital bits of information to one another WebRTC is a NEW Web API so you'd think you'd have a nice JSON blob explaining all the important things so a connection can be made. SDP (Session Description Protocol)
16. WRONG! SDP (Session Description Protocol)
17. a=candidate:4022866446 1 udp 2113937151 192.168.0.197 36768 typ host generation 0 a=candidate:4022866446 2 udp 2113937151 192.168.0.197 36768 typ host generation 0 a=candidate:2706108158 1 tcp 1509957375 192.168.0.197 0 typ host generation 0 a=candidate:2706108158 2 tcp 1509957375 192.168.0.197 0 typ host generation 0 a=candidate:1853887674 1 udp 1845501695 46.2.2.2 36768 typ srflx raddr 192.168.0.197 rport 36768 generation 0 a=candidate:1853887674 2 udp 1845501695 46.2.2.2 36768 typ srflx raddr 192.168.0.197 rport 36768 generation 0 a=candidate:2157334355 1 udp 33562367 180.6.6.6 54278 typ relay raddr 46.2.2.2 rport 38135 generation 0 a=candidate:2157334355 2 udp 33562367 180.6.6.6 54278 typ relay raddr 46.2.2.2 rport 38135 generation 0 SDP (Session Description Protocol)
18. Offer pc.createOffer(function(offer) { pc.setLocalDescription(new RTCSessionDescription(offer), function() { // send the offer to a server to be forwarded to the friend you're calling. }, error); }, error);
19. Answer pc.setRemoteDescription(new RTCSessionDescription(offer), function() { pc.createAnswer(function(answer) { pc.setLocalDescription(new RTCSessionDescription(answer), function() { // send the answer to a server to be forwarded back to the caller (you) }, error); }, error); }, error);
20. Audio Codecs We Care About OPUS G711 (PSTN interop)
21. Video Codecs We Care About VP8 VP9 H.264
22. ICE (Interactive Connectivity Establishment) (ICE Candidates) How to talk to me
23. STUN (Session Traversal Utilities for NAT) What's my public IP address?
24. TURN (Traversal Using Relay around NAT) Bugger, a nasty firewall is stopping us connecting P2P ... Let's relay our media instead
25. No Plugins...
26. Browsers
27. Apple iPhone Native Apps only (Cordova/Phonegap Supported)
28. Android Native Apps & Web View
29. Other Devices Anything that can run a WebRTC stack (or subset of it)
30. An open source adapter to level out all of the browsers to the "standard" Use it somewhere within your codebase https://github.com/webrtc/adapter adapter.js
31. You'll hear about ORTC But don't worry about it. There's an adapter! (https://github.com/twilio/ortc-adapter) But Microsoft have committed to introducing WebRTC "1.0" into Edge... so expect that soon! ORTC (Object Real Time Communications)
32. getUserMedia() now requires a trusted origin Localhost or HTTPS (But not self-signed TLS certs - letsencrypt.org) Trusted Origins
33. Getting Started with WebRTC
34. What can I use WebRTC for?
35. Or rather... What should I not use WebRTC for? What can I use WebRTC for?
37. Don't build phone calling
38. you can...
39. But don't... Build something awesome
40. Platform as a Service Use one! PaaS
41. Twilio var accessManager = new Twilio.AccessManager(token); conversationsClient = new Twilio.Conversations.Client(accessManager); conversationsClient.listen().then(clientConnected, function (error) { log('Could not connect to Twilio: ' + error.message); }); function clientConnected() { conversationsClient.on('invite', function (invite) { invite.accept().then(conversationStarted); }); } function conversationStarted(conversation) { activeConversation = conversation; if (!previewMedia) { conversation.localMedia.attach('#local-media'); } conversation.on('participantConnected', function (participant) { participant.media.attach('#remote-media'); }); conversation.on('disconnected', function (conversation) { conversation.localMedia.stop(); conversation.disconnect(); activeConversation = null; }); } https://www.twilio.com/docs/api/video/guide/quickstart-js
42. Tokbox <!DOCTYPE HTML> <html> <body> <script src="https://static.opentok.com/v2/js/opentok.js" charset="utf-8"></script> <script charset="utf-8"> var apiKey = 'YOUR-API-KEY'; var sessionId = 'YOUR-SESSION-ID'; var token = 'YOUR-TOKEN'; var session = OT.initSession(apiKey, sessionId) .on('streamCreated', function(event) { session.subscribe(event.stream); }) .connect(token, function(error) { var publisher = OT.initPublisher(); session.publish(publisher); }); </script> </body> </html> https://tokbox.com/developer/quickstart/
43. Respoke // App ID from the Respoke Dashboard for your App var appId = "c10a2075-3f3d-466f-82f9-d2285e64c5d4"; // The unique username identifying the user var endpointId = "spock@enterprise.com"; // Create an instance of the Respoke client using your App ID var client = respoke.createClient({ appId: appId, developmentMode: true }); // "connect" event fired after successful connection to Respoke client.listen("connect", function(e) { console.log("Connected to Respoke!", e); }); // Execute some signin event, then connect to Respoke with client.connect({ endpointId: endpointId }); https://docs.respoke.io/client/javascript/getting-started.html
44. Xura (Was Forge, Acision, Crocodile? I lose track...) apidaze plivo kandy.io sinch Cisco Frozen Mountain Voxbone And many more...
45. Vanila JS 'use strict'; var startButton = document.getElementById('startButton'); var callButton = document.getElementById('callButton'); var hangupButton = document.getElementById('hangupButton'); callButton.disabled = true; hangupButton.disabled = true; startButton.onclick = start; callButton.onclick = call; hangupButton.onclick = hangup; var startTime; var localVideo = document.getElementById('localVideo'); var remoteVideo = document.getElementById('remoteVideo'); localVideo.addEventListener('loadedmetadata', function() { trace('Local video videoWidth: ' + this.videoWidth + 'px, videoHeight: ' + this.videoHeight + 'px'); }); remoteVideo.addEventListener('loadedmetadata', function() { trace('Remote video videoWidth: ' + this.videoWidth + 'px, videoHeight: ' + this.videoHeight + 'px'); }); remoteVideo.onresize = function() { trace('Remote video size changed to ' + remoteVideo.videoWidth + 'x' + remoteVideo.videoHeight); // We'll use the first onsize callback as an indication that video has started // playing out. if (startTime) { var elapsedTime = window.performance.now() - startTime; trace('Setup time: ' + elapsedTime.toFixed(3) + 'ms'); startTime = null; } }; var localStream; var pc1; var pc2; var offerOptions = { offerToReceiveAudio: 1, offerToReceiveVideo: 1 }; function getName(pc) { return (pc === pc1) ? 'pc1' : 'pc2'; } function getOtherPc(pc) { return (pc === pc1) ? pc2 : pc1; } function gotStream(stream) { trace('Received local stream'); localVideo.srcObject = stream; localStream = stream; callButton.disabled = false; } function start() { trace('Requesting local stream'); startButton.disabled = true; navigator.mediaDevices.getUserMedia({ audio: true, video: true }) .then(gotStream) .catch(function(e) { alert('getUserMedia() error: ' + e.name); }); } function call() { callButton.disabled = true; hangupButton.disabled = false; trace('Starting call'); startTime = window.performance.now(); var videoTracks = localStream.getVideoTracks(); var audioTracks = localStream.getAudioTracks(); if (videoTracks.length > 0) { trace('Using video device: ' + videoTracks[0].label); } if (audioTracks.length > 0) { trace('Using audio device: ' + audioTracks[0].label); } var servers = null; pc1 = new RTCPeerConnection(servers); trace('Created local peer connection object pc1'); pc1.onicecandidate = function(e) { onIceCandidate(pc1, e); }; pc2 = new RTCPeerConnection(servers); trace('Created remote peer connection object pc2'); pc2.onicecandidate = function(e) { onIceCandidate(pc2, e); }; pc1.oniceconnectionstatechange = function(e) { onIceStateChange(pc1, e); }; pc2.oniceconnectionstatechange = function(e) { onIceStateChange(pc2, e); }; pc2.onaddstream = gotRemoteStream; pc1.addStream(localStream); trace('Added local stream to pc1'); trace('pc1 createOffer start'); pc1.createOffer(onCreateOfferSuccess, onCreateSessionDescriptionError, offerOptions); } 209 lines of code function onCreateSessionDescriptionError(error) { trace('Failed to create session description: ' + error.toString()); } function onCreateOfferSuccess(desc) { trace('Offer from pc1\n' + desc.sdp); trace('pc1 setLocalDescription start'); pc1.setLocalDescription(desc, function() { onSetLocalSuccess(pc1); }, onSetSessionDescriptionError); trace('pc2 setRemoteDescription start'); pc2.setRemoteDescription(desc, function() { onSetRemoteSuccess(pc2); }, onSetSessionDescriptionError); trace('pc2 createAnswer start'); // Since the 'remote' side has no media stream we need // to pass in the right constraints in order for it to // accept the incoming offer of audio and video. pc2.createAnswer(onCreateAnswerSuccess, onCreateSessionDescriptionError); } function onSetLocalSuccess(pc) { trace(getName(pc) + ' setLocalDescription complete'); } function onSetRemoteSuccess(pc) { trace(getName(pc) + ' setRemoteDescription complete'); } function onSetSessionDescriptionError(error) { trace('Failed to set session description: ' + error.toString()); } function gotRemoteStream(e) { remoteVideo.srcObject = e.stream; trace('pc2 received remote stream'); } function onCreateAnswerSuccess(desc) { trace('Answer from pc2:\n' + desc.sdp); trace('pc2 setLocalDescription start'); pc2.setLocalDescription(desc, function() { onSetLocalSuccess(pc2); }, onSetSessionDescriptionError); trace('pc1 setRemoteDescription start'); pc1.setRemoteDescription(desc, function() { onSetRemoteSuccess(pc1); }, onSetSessionDescriptionError); } function onIceCandidate(pc, event) { if (event.candidate) { getOtherPc(pc).addIceCandidate(new RTCIceCandidate(event.candidate), function() { onAddIceCandidateSuccess(pc); }, function(err) { onAddIceCandidateError(pc, err); } ); trace(getName(pc) + ' ICE candidate: \n' + event.candidate.candidate); } } function onAddIceCandidateSuccess(pc) { trace(getName(pc) + ' addIceCandidate success'); } function onAddIceCandidateError(pc, error) { trace(getName(pc) + ' failed to add ICE Candidate: ' + error.toString()); } function onIceStateChange(pc, event) { if (pc) { trace(getName(pc) + ' ICE state: ' + pc.iceConnectionState); console.log('ICE state change event: ', event); } } function hangup() { trace('Ending call'); pc1.close(); pc2.close(); pc1 = null; pc2 = null; hangupButton.disabled = true; callButton.disabled = false; } https://github.com/webrtc/samples/blob/gh-pages/src/content/peerconnection/pc1/js/main.js
46. Vanila JS OK, slight exaggeration You can probably do it in around 70 lines of really bad JavaScript...
47. Resources https://webrtchacks.com/sdp-anatomy/ https://webrtc.github.io/samples/ https://www.webrtc-experiment.com/ https://github.com/webrtc/apprtc https://appr.tc/ http://iswebrtcreadyyet.com/ https://webrtcweekly.com/ https://bloggeek.me/
48. IoT Workshop Come join the IoT workshop tomorrow! It's FREE!
49. Thanks! dan@nimblea.pe @dan_jenkins @nimbleapeltd Link to slides will be tweeted soon!
50. images 
 used http://img14.deviantart.net/9aad/i/2015/147/5/3/the_flash_silhouette_wallpaper_by_speedaroo-d8v1jo7.jpg https://upload.wikimedia.org/wikipedia/commons/thumb/3/31/ProhibitionSign2.svg/2000px-ProhibitionSign2.svg.png https://upload.wikimedia.org/wikipedia/commons/e/e2/Captain-america_serial_poster.jpg