Using GStreamer to build real-time applications with Golang

Everyone knows how to build up a GStreamer pipeline on the CLI - give gst-launch-1.0 a source and a sink and some steps in between and you've got yourself a pipeline doing something. But building an application with GStreamer is much more complex when it comes to building using bindings to the C level api for GStreamer.

Avi started building the Golang bindings back in 2020 but they'd started to become a little unloved with Pull Requests and issues becoming stale. Now the bindings have moved to their own github organisation and are getting some all important bug fixes and a new semver release of 1.0 coming soon.

Building a solution doesn't always fit into what's available with GStreamer's plugins - enter appsrc and appsink, your best friends in building custom solutions with GStreamer. In this session you'll hear about using appsrc and appsink to build custom real time applications, as well as the updates coming to GStreamer's Golang binding.

Transcript

1. Using GStreamer to build real-time applications with Golang Dan Jenkins Nimble Ape / Everycast Labs FOSDEM, Brussels, February 2024
2. A Little About Me
3. • Founder of Everycast Labs, Nimble Ape • Creator of CommCon • Was the fi rst "Google Developer Expert" in WebRTC • Loves LEGO • Loves Real-Time Media • Loves developing for "the web" • Over 10 years in Open Source VoIP • @dan_jenkins / @danjenkins@fosstodon.org Dan Jenkins
4. • Real-Time Communication consultancy • Based in the UK • Work with Open Source Real-Time Comms • VoIP, WebRTC, Broadcast • Got a problem you want help with? Let us know • hello@nimblea.pe Nimble Ape
5. • Creators of Broadcast Bridge (broadcastbridge.app) • A Platform as a Service for bringing in remote talent into production AV work fl ows • We work with WebRTC / SRT / NDI / Decklink & AJA cards. • hello@everycastlabs.uk Everycast Labs
6. • "Open Media" - Real-Time or not • Top quality production values • 5 Years worth of content on our YouTube Channel • commcon.xyz • News about 2024 coming SOON! We'll be going on tour! CommCon
7. Using GStreamer to build real-time applications with Golang
8. We're going to talk about...
9. GStreamer
10. Golang
11. go-gst
12. Pion
13. https://gstreamer.freedesktop.org/
14. "open source multimedia framework"
15. You might know GStreamer as this...
16. gst-launch-1.0 rtspsrc location="rtsp://192.168.5.90/axis-media/media.amp? videocodec=h264&resolution=1280x720&fps=25&videobitrate=4000&compression=50" is- live=true latency=0 protocols=tcp ! rtph264depay ! video/x-h264,stream- format=avc,alignment=au,profile=baseline ! h264parse config-interval=-1 ! queue silent=true ! rtph264pay mtu=1400 config-interval=-1 ! application/x- rtp,media=video,clock-rate=${$channels.video.clockRate},encoding- name=${$channels.video.encodingName},ssrc=(uint)${$channels.video.SSRC} ! queue silent=true ! udpsink host=127.0.0.1 port=${$pipeline.port} sync=false async=true
17. GStreamer is incredibly powerful
18. Ingress ↓ Do Something ↓ Egress
19. And GStreamer can do it all!
20. NDI
21. WebRTC
22. SRT
23. RTP
24. HLS/DASH
25. RTMP/RTSP
26. But GStreamer has a real super power.
27. AppSink & AppSrc
28. This is what we use in Broadcast Bridge
29. But we don't write C like this
30. /* Initialize GStreamer */ gst_init (&argc, &argv); /* Build the pipeline */ pipeline = gst_parse_launch ("playbin uri=https://gstreamer.freedesktop.org/data/media/sintel_trailer- 480p.webm", NULL); /* Start playing */ gst_element_set_state (pipeline, GST_STATE_PLAYING); /* Wait until error or EOS */ bus = gst_element_get_bus (pipeline); msg = gst_bus_timed_pop_filtered (bus, GST_CLOCK_TIME_NONE, GST_MESSAGE_ERROR | GST_MESSAGE_EOS); /* See next tutorial for proper error message handling/parsing */ if (GST_MESSAGE_TYPE (msg) == GST_MESSAGE_ERROR) { g_error ("An error occurred! Re-run with the GST_DEBUG=*:WARN environment " "variable set for more details."); }
31. We write Go like this
32. gst.Init(&os.Args) // Let GStreamer create a pipeline from the parsed launch syntax on the cli. pipeline, err := gst.NewPipelineFromString(strings.Join(os.Args[1:], " ")) if err != nil { return err } // Add a message handler to the pipeline bus, printing interesting information to the console. pipeline.GetPipelineBus().AddWatch(func(msg *gst.Message) bool { switch msg.Type() { case gst.MessageEOS: // When end-of-stream is received stop the main loop pipeline.BlockSetState(gst.StateNull) mainLoop.Quit() case gst.MessageError: // Error messages are always fatal err := msg.ParseError() fmt.Println("ERROR:", err.Error()) if debug := err.DebugString(); debug != "" { fmt.Println("DEBUG:", debug) } mainLoop.Quit() default: // All messages implement a Stringer. However, this is // typically an expensive thing to do and should be avoided. fmt.Println(msg) } return true }) // Start the pipeline pipeline.SetState(gst.StatePlaying)
33. And that's because of the go-gst bindings
34. Originally created by tinyzimmer
35. Now in their own GitHub Organisation https://github.com/go-gst/go-gst
36. Under the new GitHub Org we have 3 main contributors
38. Less forks = great for everyone
39. Broadcast Bridge uses a mixture of SRT, NDI and WebRTC among others...
40. So why would we need to use AppSrc and AppSink?
41. Greater Control
42. For us, we use Pion to "do" WebRTC https://github.com/pion
43. This means we're handling WebRTC in a language we know
44. Pion is hugely powerful
45. And easily upgradeable
46. Unlike GStreamer (for us and our team's skillset)
47. If we found a bug in "WebRTC" in GStreamer, getting it fixed and released might be a multi week/month process
48. With Pion handling WebRTC we're better in control.
49. We then use AppSrc to pass in the resulting RTP & RTCP
50. We let GStreamer do its thing...
51. And use AppSink to get the media back out...
52. Is it the most optimised way of doing it?
53. Absolutely not.
54. But it gives us huge flexibility.
55. Move fast. Add new features. Win business.
56. So let's take a look at AppSrc
57. https://gstreamer.freedesktop.org/documentation/
58. GStreamer can ask you for data or you can just push it in
59. src.SetCallbacks(&app.SourceCallbacks{ NeedDataFunc: func(self *app.Source, _ uint) { // If we've reached the end of the palette, end the stream. if i == len(palette) { src.EndStream() return } fmt.Println("Producing frame:", i) // Create a buffer that can hold exactly one video RGBA frame. buffer := gst.NewBufferWithSize(videoInfo.Size()) // For each frame we produce, we set the timestamp when it should be displayed // The autovideosink will use this information to display the frame at the right time. buffer.SetPresentationTimestamp(gst.ClockTime(time.Duration(i) * 500 * time.Millisecond)) // Produce an image frame for this iteration. pixels := produceImageFrame(palette[i]) buffer.Map(gst.MapWrite).WriteData(pixels) buffer.Unmap() // Push the buffer onto the pipeline. self.PushBuffer(buffer) i++ }, }) need-data signal
60. And AppSink is no different
61. https://gstreamer.freedesktop.org/documentation/
62. You get pushed your data
63. sink.SetCallbacks(&app.SinkCallbacks{ // Add a "new-sample" callback NewSampleFunc: func(sink *app.Sink) gst.FlowReturn { // Pull the sample that triggered this callback sample := sink.PullSample() if sample == nil { return gst.FlowEOS } // Retrieve the buffer from the sample buffer := sample.GetBuffer() if buffer == nil { return gst.FlowError } samples := buffer.Map(gst.MapRead).AsInt16LESlice() defer buffer.Unmap() // Calculate the root mean square for the buffer // (https://en.wikipedia.org/wiki/Root_mean_square) var square float64 for _, i := range samples { square += float64(i * i) } rms := math.Sqrt(square / float64(len(samples))) fmt.Println("rms:", rms) return gst.FlowOK }, }) new-sample signal
64. For us, we need to handle RTP & RTCP
65. But GStreamer makes that easy
66. RTPBin
67. https://gstreamer.freedesktop.org/documentation/
68. RTPBin implements everything you need to handle RTP & RTCP jitter bu ff er, ssrc demuxer, payload type demuxer, rtcp, rtp depayloading
69. Connect the AppSrc sink Pad(s) to RTPBin src pads
70. rtcpSinkPad := rtpbin.GetRequestPad("recv_rtcp_sink_%u") rtcpSrcPad := rtpbin.GetRequestPad("send_rtcp_src_%u") rtcpAppSink, err := app.NewAppSink() if err != nil { log.Warnf("Error creating Gstreamer app sink %s", err) return } rtcpAppsrc, err := app.NewAppSrc() if err != nil { log.Warnf("Error creating Gstreamer app src %s", err) return } err = pipeline.Add(rtcpAppsrc.Element) if err != nil { log.Warnf("Error adding rtcp src to pipeline, %s", err) return } err = pipeline.Add(rtcpAppSink.Element) if err != nil { log.Warnf("Error adding rtcp src to pipeline, %s", err) return } linkedRTCPSrc := rtcpAppsrc.GetStaticPad("src").Link(rtcpSinkPad) if linkedRTCPSrc.String() != "ok" { log.Warnf("Error connecting RTCP sink to rtpbin, %s", err) return }
71. And you'll get RTP in, RTCP in and out!
72. Don't forget about the RTCP!
73. And you end up with something looking like this
74. <GstPipeline> pipeline81e63a7f-7870-446e-8e14-b1d89cf0afc2 [>] GstDecodeBin decodebin-audio [>] caps=video/x-raw(ANY); audio/x-raw(ANY); text/x-raw(ANY); subpicture/x-dvd; subpictur… GstOpusDec opusdec0 [>] GstRTPOpusDepay rtpopusdepay0 [>] GstTypeFindElement typefind [>] caps=application/x-rtp, encoding-name=(string)OPUS, payload=(int)111, media=(string)a… GstDecodeBin decodebin-video [>] caps=video/x-raw(ANY); audio/x-raw(ANY); text/x-raw(ANY); subpicture/x-dvd; subpictur… GstTypeFindElement typefind [>] caps=application/x-rtp, media=(string)video, encoding-name=(string)VP8-DRAFT-IETF-01,… GstVP8Dec vp8dec0 [>] GstRtpVP8Depay rtpvp8depay0 [>] wait-for-keyframe=TRUE GstQueue queue5 [>] GstVideoScale videoscale0 [>] qos=TRUE GstIdentity identity0 [>] GstQueue queue4 [>] GstTee tee-audio [>] num-src-pads=1 GstTee tee-video [>] num-src-pads=1 GstAppSrc appsrc0 [>] format=time is-live=TRUE emit-signals=FALSE GstRtpBin rtpbin0 [>] sdes=application/x-rtp-source-sdes, cname=(string)\"user244345186\\@host-c701ca24… fec-decoders=application/x-rtp-fec-decoders; fec-encoders=application/x-rtp-fec-encoders; GstRtpSession rtpsession0 [>] sdes=application/x-rtp-source-sdes, cname=(string)\"user244345186\\@host-c701ca24… num-sources=3 num-active-sources=3 internal-session=((RTPSession*) 0x12f878120) GstRtpPtDemux rtpptdemux1 [>] GstRtpJitterBuffer rtpjitterbuffer1 [>] GstRtpPtDemux rtpptdemux0 [>] GstRtpJitterBuffer rtpjitterbuffer0 [>] percent=52 GstRtpStorage rtpstorage0 [>] internal-storage=((RtpStorage*) 0x108404cc0) GstRtpSsrcDemux rtpssrcdemux0 [>] GstQueue queue3 [>] current-level-buffers=35 current-level-bytes=192740 current-level-time=1003854167 GstAudioRate audiorate0 [>] in=207360 out=225936 add=31056 drop=9600 GstQueue queue2 [>] current-level-buffers=48 current-level-bytes=184320 current-level-time=633876406 GstVideoConvert videoconvert0 [>] qos=TRUE GstQueue queue1 [>] current-level-buffers=4 current-level-bytes=5529600 current-level-time=67487391 GstAppSink video-ndi-appsink [>] last-sample=((GstSample*) 0x13b87e470) caps=video/x-raw, format=(string)UYVY, interlace-mode=(string)progressive, width=(int… eos=FALSE GstAppSink appsink0 [>] last-sample=((GstSample*) 0x13b87e550) eos=FALSE GstAppSrc appsrc1 [>] GstAudioConvert audioconvert0 [>] GstQueue queue0 [>] current-level-buffers=37 current-level-bytes=188932 current-level-time=984020833 GstAppSink audio-ndi-appsink [>] last-sample=((GstSample*) 0x13b87e630) caps=audio/x-raw, format=(string)F32LE, layout=(string)interleaved, channels=(int)2 eos=FALSE Legend Element-States: [~] void-pending, [0] null, [-] ready, [=] paused, [>] playing Pad-Activation: [-] none, [>] push, [<] pull Pad-Flags: [b]locked, [f]lushing, [b]locking, [E]OS; upper-case is set Pad-Task: [T] has started task, [t] has paused task sink [>][bfb] src [>][bfb][T] sink [>][bfb] video/x-raw format: I420 width: 1280 height: 720 interlace-mode: progressive multiview-mode: mono multiview-flags: 0:ffffffff:/right-view... pixel-aspect-ratio: 1/1 framerate: 0/1 src_0 [>][bfb] sink [>][bfb] src [>][bfb] video/x-raw format: I420 width: 1280 height: 720 interlace-mode: progressive multiview-mode: mono multiview-flags: 0:ffffffff:/right-view... pixel-aspect-ratio: 1/1 framerate: 0/1 sink [>][bfb] src [>][bfb] video/x-raw format: I420 width: 640 height: 360 interlace-mode: progressive multiview-mode: mono multiview-flags: 0:ffffffff:/right-view... pixel-aspect-ratio: 1/1 framerate: 0/1 sink [>][bfb] src [>][bfb][T] video/x-raw format: I420 width: 640 height: 360 interlace-mode: progressive multiview-mode: mono multiview-flags: 0:ffffffff:/right-view... pixel-aspect-ratio: 1/1 framerate: 0/1 sink [>][bfb] src [>][bfb][T] sink [>][bfb] audio/x-raw format: S16LE layout: interleaved rate: 48000 channels: 2 channel-mask: 0x0000000000000003 src_0 [>][bfb] sink [>][bfb] src [>][bfb] audio/x-raw format: S16LE layout: interleaved rate: 48000 channels: 2 channel-mask: 0x0000000000000003 sink [>][bfb] src [>][bfb][T] audio/x-raw format: S16LE layout: interleaved rate: 48000 channels: 2 channel-mask: 0x0000000000000003 sink [>][bfb] src [>][bfb] sink [>][bfb] video/x-raw width: 1280 height: 720 interlace-mode: progressive multiview-mode: mono multiview-flags: 0:ffffffff:/right-view... pixel-aspect-ratio: 1/1 framerate: 0/1 format: UYVY sink [>][bfb] src [>][bfb][T] video/x-raw format: I420 width: 1280 height: 720 interlace-mode: progressive multiview-mode: mono multiview-flags: 0:ffffffff:/right-view... pixel-aspect-ratio: 1/1 framerate: 0/1 sink [>][bfb] src [>][bfb][T] recv_rtcp_sink_0 [>][bfb] application/x-rtcp application/x-srtcp ANY proxypad3 [>][bfb] sink [>][bfb] src [>][bfb] sink [>][bfb] audio/x-raw rate: 48000 format: F32LE channels: 2 layout: interleaved channel-mask: 0x0000000000000003 sink [>][bfb] src [>][bfb][T] audio/x-raw format: S16LE layout: interleaved rate: 48000 channels: 2 channel-mask: 0x0000000000000003 audio/x-raw format: S16LE layout: interleaved rate: 48000 channels: 2 channel-mask: 0x0000000000000003 video/x-raw format: I420 width: 1280 height: 720 interlace-mode: progressive multiview-mode: mono multiview-flags: 0:ffffffff:/right-view... pixel-aspect-ratio: 1/1 framerate: 0/1 src [>][bfb][T] recv_rtp_sink_0 [>][bfb] application/x-rtp application/x-srtp ANY proxypad2 [>][bfb] send_rtcp_src_0 [>][bfb] recv_rtp_sink [>][bfb] application/x-rtp ANY recv_rtcp_sink [>][bfb] application/x-rtcp ANY proxypad4 [>][bfb] application/x-rtcp proxypad5 [>][bfb] recv_rtp_src_0_1129573010_111 [>][bfb] sink [>][bfb] application/x-rtp encoding-name: OPUS payload: 111 media: audio clock-rate: 48000 proxypad6 [>][bfb] recv_rtp_src_0_3085994635_96 [>][bfb] sink [>][bfb] application/x-rtp media: video encoding-name: VP8-DRAFT-IETF-01 payload: 96 clock-rate: 90000 sink [>][bfb] src_111 [>][bfb] application/x-rtp encoding-name: OPUS payload: 111 media: audio clock-rate: 48000 sink [>][bfb] src [>][bfb][T] sink_rtcp [>][bfb] application/x-rtp sink [>][bfb] src_96 [>][bfb] application/x-rtp media: video encoding-name: VP8-DRAFT-IETF-01 payload: 96 clock-rate: 90000 sink [>][bfb] src [>][bfb][T] sink_rtcp [>][bfb] application/x-rtp sink [>][bfb] src [>][bfb] sink [>][bfb] application/x-rtp src_3085994635 [>][bfb] rtcp_sink [>][bfb] application/x-rtp rtcp_src_3085994635 [>][bfb] application/x-rtcp src_1129573010 [>][bfb] application/x-rtp rtcp_src_1129573010 [>][bfb] application/x-rtcp ssrc: 1129573010 recv_rtp_src [>][bfb] application/x-rtp sync_src [>][bfb] application/x-rtcp send_rtcp_src [>][bfb] application/x-rtcp proxypad1 [>][bfb] src_0 [>][bfb] proxypad0 [>][bfb] src_0 [>][bfb] sink [>][bfb] application/x-rtp encoding-name: OPUS payload: 111 media: audio clock-rate: 48000 proxypad8 [>][bfb] audio/x-raw format: S16LE layout: interleaved rate: 48000 channels: 2 channel-mask: 0x0000000000000003 sink [>][bfb] src [>][bfb] audio/x-raw format: S16LE layout: interleaved rate: 48000 channels: 2 channel-mask: 0x0000000000000003 sink [>][bfb] src [>][bfb] audio/x-opus channel-mapping-family: 0 channels: 2 rate: 48000 src [>][bfb] application/x-rtp encoding-name: OPUS payload: 111 media: audio clock-rate: 48000 sink [>][bfb] application/x-rtp media: video encoding-name: VP8-DRAFT-IETF-01 payload: 96 clock-rate: 90000 proxypad7 [>][bfb] video/x-raw format: I420 width: 640 height: 360 interlace-mode: progressive multiview-mode: mono multiview-flags: 0:ffffffff:/right-view... pixel-aspect-ratio: 1/1 framerate: 0/1 sink [>][bfb] src [>][bfb] video/x-raw format: I420 width: 640 height: 360 interlace-mode: progressive multiview-mode: mono multiview-flags: 0:ffffffff:/right-view... pixel-aspect-ratio: 1/1 framerate: 0/1 sink [>][bfb] src [>][bfb] video/x-vp8 framerate: 0/1 height: 360 width: 640 profile: 0 src [>][bfb] application/x-rtp media: video encoding-name: VP8-DRAFT-IETF-01 payload: 96 clock-rate: 90000
75. GstAppSrc appsrc0 [>] format=time is-live=TRUE emit-signals=FALSE GstAppSrc appsrc1 [>] Legend Element-States: [~] void-pending, [0] null, [-] ready, [=] paused, [>] playing Pad-Activation: [-] none, [>] push, [<] pull Pad-Flags: [b]locked, [f]lushing, [b]locking, [E]OS; upper-case is set Pad-Task: [T] has started task, [t] has paused task src [>][bfb][T] recv_rtcp_sink_0 [>][bfb] application/x-rtcp application/x-srtcp ANY proxypad3 [>][bfb] src [>][bfb][T] recv_rtp_sink_0 [>][bfb] application/x-rtp application/x-srtp ANY proxypad2 [>][bfb] applica ANY applicat ANY
76. GstRtpBin rtpbin0 [>] sdes=application/x-rtp-source-sdes, cname=(string)\"user244345186\\@host fec-decoders=application/x-rtp-fec-decoders; fec-encoders=application/x-rtp-fec-encoders; GstRtpSession rtpsession0 [>] sdes=application/x-rtp-source-sdes, cname=(string)\"user244345186\\@host-c701ca24… num-sources=3 num-active-sources=3 internal-session=((RTPSession*) 0x12f878120) GstRtpStorage rtpstorage0 [>] internal-storage=((RtpStorage*) 0x108404cc0) GstRtpSsrcDem rtpssrcdemux [>] recv_rtp_sink [>][bfb] lication/x-rtp recv_rtcp_sink [>][bfb] ication/x-rtcp sink [>][bfb] src [>][bfb] sink [>][bfb] application/x-rtp src rtcp_sink [>][bfb] rtcp_ src rtcp_ recv_rtp_src [>][bfb] application/x-rtp sync_src [>][bfb] application/x-rtcp send_rtcp_src [>][bfb]
77. GstRtpBin rtpbin0 [>] s, cname=(string)\"user244345186\\@host-c701ca24… s=application/x-rtp-fec-decoders; s=application/x-rtp-fec-encoders; GstRtpPtDemux rtpptdemux1 [>] GstRtpJitterBuffer rtpjitterbuffer1 [>] GstRtpPtDemux rtpptdemux0 [>] GstRtpJitterBuffer rtpjitterbuffer0 [>] percent=52 GstRtpSsrcDemux rtpssrcdemux0 [>] sink [>][bfb] src_111 [>][bfb] application/ encod cl sink [>][bfb] src [>][bfb][T] sink_rtcp [>][bfb] application/x-rtp sink [>][bfb] src_96 [>][bfb] application/x-rtp med encoding-na paylo clock-ra sink [>][bfb] src [>][bfb][T] sink_rtcp [>][bfb] application/x-rtp sink [>][bfb] rtp src_3085994635 [>][bfb] rtcp_sink [>][bfb] application/x-rtp rtcp_src_3085994635 [>][bfb] application/x-rtcp src_1129573010 [>][bfb] application/x-rtp rtcp_src_1129573010 [>][bfb] application/x-rtcp ssrc: 1129573010 application/x-rtcp
78. GstRtpPtDemux rtpptdemux1 [>] GstRtpPtDemux rtpptdemux0 [>] GstAppSink appsink0 [>] last-sample=((GstSample*) 0x13b87e550) eos=FALSE sink [>][bfb] send_rtcp_src_0 [>][bfb] proxypad4 [>][bfb] application/x-rtcp proxypad5 [>][bfb] recv_rtp_src_0_1129573010_111 [>][bfb] sink [>][bfb] application/x-rtp encoding-name: OPUS payload: 111 media: audio clock-rate: 48000 proxypad6 [>][bfb] recv_rtp_src_0_3085994635_96 [>][bfb] sink [>][bfb] application/x-rtp media: video encoding-name: VP8-DRAFT-IETF-01 payload: 96 clock-rate: 90000 sink [>][bfb] src_111 [>][bfb] application/x-rtp encoding-name: OPUS payload: 111 media: audio clock-rate: 48000 -rtp sink [>][bfb] src_96 [>][bfb] application/x-rtp media: video encoding-name: VP8-DRAFT-IETF-01 payload: 96 clock-rate: 90000 -rtp proxypad1 [>][bfb] proxypad0 [>][bfb]
79. GstT caps=application/x-rtp, encoding-name= GstT caps=application/x-rtp, media=(string)vid GstAppSink appsink0 [>] last-sample=((GstSample*) 0x13b87e550) eos=FALSE sink [>][bfb] send_rtcp_src_0 [>][bfb] proxypad4 [>][bfb] application/x-rtcp proxypad5 [>][bfb] recv_rtp_src_0_1129573010_111 [>][bfb] sink [>][bfb] application/x-rtp encoding-name: OPUS payload: 111 media: audio clock-rate: 48000 proxypad6 [>][bfb] recv_rtp_src_0_3085994635_96 [>][bfb] sink [>][bfb] application/x-rtp media: video encoding-name: VP8-DRAFT-IETF-01 payload: 96 clock-rate: 90000 TF-01 proxypad1 [>][bfb] proxypad0 [>][bfb] sink [>][bfb] application/x-rtp encoding-name: OPUS payload: 111 media: audio clock-rate: 48000 sink [>][bfb] application/x-rtp media: video encoding-name: VP8-DRAFT-IETF-01 payload: 96 clock-rate: 90000
80. GstDeco decodebin [>] caps=video/x-raw(ANY); audio/x-raw(ANY); tex GstTypeFindElement typefind [>] caps=application/x-rtp, encoding-name=(string)OPUS, payload=(int)111, media=(string)a… GstDeco decodebin [>] caps=video/x-raw(ANY); audio/x-raw(ANY); tex GstTypeFindElement typefind [>] caps=application/x-rtp, media=(string)video, encoding-name=(string)VP8-DRAFT-IETF-01,… GstAppSink appsink0 [>] last-sample=((GstSample*) 0x13b87e550) eos=FALSE sink [>][bfb] sink [>][bfb] sink [>][bfb] F-01 proxypad1 [>][bfb] proxypad0 [>][bfb] sink [>][bfb] application/x-rtp encoding-name: OPUS payload: 111 media: audio clock-rate: 48000 src [>][bfb] application/x-rtp encoding-name: OPUS payload: 111 media: audi clock-rate: 4800 sink [>][bfb] application/x-rtp media: video encoding-name: VP8-DRAFT-IETF-01 payload: 96 clock-rate: 90000 src [>][bfb] application/x-rtp media: video encoding-name: VP8-DRAFT- payload: 96 clock-rate: 90000
81. <GstPipeline> pipeline81e63a7f-7870-446e-8e14-b1d89cf0afc2 [>] GstDecodeBin decodebin-audio [>] x-raw(ANY); text/x-raw(ANY); subpicture/x-dvd; subpictur… GstOpusDec opusdec0 [>] GstRTPOpusDepay rtpopusdepay0 [>] GstDecodeBin decodebin-video [>] x-raw(ANY); text/x-raw(ANY); subpicture/x-dvd; subpictur… GstVP8Dec vp8dec0 [>] GstRtpVP8Depay rtpvp8depay0 [>] wait-for-keyframe=TRUE src_0 [>][bfb] src_0 [>][bfb] proxypad8 [>][bfb] sink [>][bfb] src [>][bfb] audio/x-raw format: S16LE layout: interleaved rate: 48000 channels: 2 channel-mask: 0x0000000000000003 sink [>][bfb] src [>][bfb] audio/x-opus channel-mapping-family: 0 channels: 2 rate: 48000 tp -name: OPUS yload: 111 media: audio -rate: 48000 proxypad7 [>][bfb] vid p sink [>][bfb] src [>][bfb] video/x-raw format: I420 width: 640 height: 360 interlace-mode: progressive multiview-mode: mono multiview-flags: 0:ffffffff:/right-view... pixel-aspect-ratio: 1/1 framerate: 0/1 sink [>][bfb] src [>][bfb] video/x-vp8 framerate: 0/1 height: 360 width: 640 profile: 0 video VP8-DRAFT-IETF-01 96 90000
82. We're using Go purely because of Pion
83. Pion gives us control.
84. WebRTC in Pure Golang
85. But you can do this with any of the GStreamer bindings
86. Go isn't on the list! (yet) https://gitlab.freedesktop.org/gstreamer/www/-/merge_requests/92
87. Got a problem and no plugin available?
88. Build it yourself with AppSrc and AppSink!
89. So... Why GStreamer?
90. Why not FFmpeg?
91. GStreamer does everything we need.
92. It has a great community
93. A Super friendly community
94. Gstreamer is super flexible and easier for us to work with
95. GStreamer FTW
96. Don't wait for others.
97. Build with GStreamer and AppSrc and AppSink
98. Thanks for Having Me!
99. Thanks! nimblea.pe everycastlabs.uk broadcastbridge.app commcon.xyz @dan_jenkins @danjenkins@fosstodon.org @nimbleape@nimblea.pe @everycastlabs@everycastlabs.uk @commcon@commcon.xyz