Mike Howsden - Zen of Quality - How PBS measures QoS for digital viewers
It is extremely important to PBS that digital viewers have an awesome experience when viewing online videos. In this talk, we explain how PBS built a system to collect, analyze, and measure who's getting a good experience -- and who's not.
boastful mind, which thus obtains the reputation of creative genius by demanding what it cannot itself supply, by censuring what it cannot improve, and by proposing what it knows not where to find. Immanuel Kant - Prolegomena (1783)
Chromecast, AppleTV, etc.) and mobile application development for PBS • We serve videos and lists of videos to as many platforms as possible • Disclaimer - PBS does not endorse any product or service mentioned and opinions expressed here are my own
video streaming system • It’s hard to know what is at fault when a user has a bad streaming experience • We need insight into user experiences around the country, end user bandwidth, and ISP service degradation in relation to our CDNs (net neutrality?) • Complaints are inherently anecdotal and generally don’t come with enough data to make a good diagnosis
are very expensive • Integrations have a non-trivial development cost • We want an independent way to evaluate our vendors’ players • No good open source solutions (yet?)
but is specifically tuned to video QoS • Similar frameworks exist for other business applications • The goal is to be modular and clearly define interfaces • This is one of the projects at PBS Digital that we consider “big data” • 70 million events -> 7 million stream profiles, daily* * I just picked a day and did very rough math
fires events like MediaStarted, BufferingStart/Stop, MediaPaused, etc. • These events need to be collected to represent a stream • They then need to be interpreted to create a stream profile • These profiles need to be analyzed and summarized to answer our business questions
• Dashboard style live visualizations for operational monitoring • Alerts and automated decisioning • Continued flexibility for changing business needs
- python library we used for agora-proc • http://github.com/pbs/agora-proc - batch analyzer utilizes Amazon EMR cluster to reduce video events into defined characteristics of a particular stream (this repo also documents the player events) • http://www.postgresql.org/docs/9.4/static/ddl-partitioning.html - for more on postgreSQL table partitioning