Greg Philpott is the founder and president of mDialog, an innovator in the fast-evolving mobile video sector. Greg has worked in media and technology for the past 18 years, and is pioneering work in Internet video and Digital Asset Management across a number of companies in the media and entertainment industry. Though H.264-based MPEG-4 is currently the video format of choice for big video publishers and distributors such as MLB, iTunes and Netflix, premium video publishers and broadcasters generally don’t deliver their video assets via an H.264 progressive download. They require secure streaming protocols to deliver or stream their video assets to the video player.
At a high level it goes something like this: Video can be encoded into different formats such as H.264-based MPEG-4 or VP8 based WebM; the video is delivered or streamed using one of several possible streaming formats/protocols such as HTTP Live Streaming, Smooth Streaming and HTTP Dynamic Streaming; finally, players implemented in HTML5, Flash and Silverlight receive these streams and play them out for the viewer.
In this post we’ll focus on the approach to Adaptive Streaming that
Apple is supporting. The technology, called HTTP Live Streaming (HLS), allows you stream live or on-demand video to devices like Apple’s
iPad,
iPhoneand
iPod Touch without the need for a specialized video streaming server — any HTTP web server will do the trick. While created by Apple, the specification has been submitted to the Internet Engineering Task Force (IETF), the first step on its path to becoming an Internet standard.
Here’s a partial list of key features that benefit video publishers, advertisers, viewers and even mobile carriers.
The Basics
HTTP Live Streaming is
backed by Apple, Smooth Streaming is
backed by Microsoft and HTTP Dynamic Streaming is
backed by Adobe, but they all have one thing in common: They each use the MPEG-4 H.264 as their inputs. I believe the real story isn’t which video format will be supported by HTML5, but more importantly, when it comes to premium video, which video
streaming format will be supported.
All three delivery methods are forms of Adaptive Bit Rate Streaming. This delivery method will have a massive impact on every aspect of Internet video delivery because it allows the stream to actually adapt the video experience to the quality of the network and the device’s CPU.
In other words, the video stream can increase or decrease the bit rate and resolution of the video (its quality) in real time so that it’s always streaming the best possible quality the available network connection can support. The better the network connection, the better the video image quality. The fact that the stream handles all of this complexity means the mobile video viewer doesn’t have to do anything; everything is left to the stream and the player.
So how does this all work? To prep your video content for HLS, you start off with a high quality version of your video and encode multiple copies of it using MPEG-4 H.264. These copies are at various bit rates and resolutions ranging from lower quality renditions appropriate for slower 3G connections, up to extremely high quality renditions suitable for fast devices on fast networks. The renditions are then wrapped into MPEG-2 Transport Streams and chopped up into 10 second segments or chunks. It’s these segments that are eventually streamed to an HTML5 Video Player on a mobile device, browser or set-top box, and because the player receives the video in 10 second chunks and can detect the quality of the network connection, it can switch to a higher or lower quality video segment every ten seconds if bandwidth conditions change.
Real Cost Savings
HLS saves the video publisher and the receiving viewer’s data plan money. HLS is a true video stream, which means it only delivers a few segments of video as it plays. For example, if a viewer watched five minutes of streamed video, their data plan is only dinged for five minutes of data consumption. In contrast, a progressive download of a video could result in only five minutes of actual view time, but 20 minutes of downloaded video passed to their iPhone.
This is a big problem for everybody involved. The viewer’s data plan gets charged for video they didn’t actually see, the publisher pays their CDN for video delivery that’s never consumed (or monetized!), and the carrier’s network gets congested delivering video that it really didn’t have to.
Another major cost consideration for the video publisher is they don’t have to pay for and maintain specialized and proprietary video streaming servers. HLS uses an ordinary HTTP server, and if they are publishing a website, they already have one.
Security
The HLS specification has provisions to ensure security of the stream, which is great news for broadcasters or publishers who want to stream licensed content. The entire HLS streams can be encrypted using AES-128; you can even encrypt each rendition or even a group of segments using unique keys. Because the streams themselves are encrypted, you can deliver the content over normal HTTP, rather than SSL, which keeps the speed fast. The keys can be hidden and protected using authentication, tokens and signing. All of this makes it much more difficult for people to use your content in ways you don’t want them to, while ensuring that the stream is efficient and the goals of HLS are still met.
Live Streaming
Another big advantage of HLS is you can stream live video content to iOS devices. Streaming live video to tablet devices like the iPad is very compelling. The iPad is basically a personal TV that you can take anywhere. The tablet as a TV is an idea that’s not lost on the major broadcasters and cable companies. Now that premium Video on Demand is established on the iPad as a viable business model, the live re-transmission of linear broadcasts to the iPad via HLS is a natural progression.
Live HLS encoding requires an additional step in the publisher’s workflow (to encode the live feed in real time). That’s where encoding appliance companies like Inlet Technologies or Elemental Technologies come it. They have developed turnkey products that handle all of the complexities of creating a HLS stream and delivering it to the cloud.
Real-Time In-Stream Ad Delivery
Last, but certainly not least, is how HLS provides the publisher with a unique opportunity when it comes to in-stream ad insertion. Recall how HLS is a stream of ten-second segments of video; those segments can be created in such a way that a video ad can be injected into the content stream so that it plays out seamlessly for the viewer. The viewing experience is basically identical to watching a commercial on broadcast TV.
Injecting the “ad stream” into the “content stream” is a new concept several major broadcasters such as Canadian Broadcaster Global TV have adopted. The traditional delivery mechanism for pre- or mid-roll video ads is to perform a (disruptive) video player switch.
You see this a lot with your typical web video players: While watching a video, the stream goes blank while another player is presented, and a spinner or buffer is shown until the video ad stream is cued up and ready for playback. Then when the ad is finished, that player is closed and the feature content player is opened and a spinner presented until the feature content is cued up and ready for playback.
With HLS, you have the opportunity to do all your video content switching in the stream rather than within the video player. Additionally, you can take advantage of the adaptive nature of HLS to further ensure the video ad gets presented so the viewer never sees spinners or buffers as the ad loads. The ad experience is seamless.
Disclosure: The author’s company develops in-stream advertising technology.
The Future is Adapting
All of this is a bit reminiscent of the “video codec” wars of a few years ago. Back then, Apple, Microsoft and Adobe were all promoting different video encoding technologies (MPEG-4 H.264, VC1 and On2). Each produced good looking video, but it was a major headache for video publishers. The industry eventually settled on H.264 once it reached mass adoption, but it’s still a horse race when it comes to Adaptive Video Streaming. One thing is for sure: Adaptive Bit Rate Streaming is the future of video delivery across all device types, from PCs to smartphones to set-top devices.
Will HLS win out over other Adaptive Streaming solutions? It’s still early, but one would think that the open spec and submission to the IETF would give it a leg up on the competition, just as it did for H.264. If HLS is going to take off as an Internet video streaming standard, it has to expand beyond just iOS devices.
Android, Google’s mobile OS is now capable of playing HLS video thanks to a company called Nextreaming, which has built a robust HLS player SDK for the Android platform. And HLS is no longer a just a mobile video play. Roku, which makes one of the leading set-top streaming players, now supports HLS. Perhaps, most importantly, several major video publishers such as ABC, Netflix and Hulu are taking advantage of HLS in their iPad Apps. One thing is certain: You will be seeing a lot more adaptive video in 2011.