What is Video Streaming?

With all of the buzz around HTML5 and the iPad, there's been a lot of talk about the technologies underlying digital video. Besides the inevitable codecs (H264 & VP8), experts are discussing video delivery mechanisms, using indecipherable acronyms like RTMP, CDN and HLS. This blog post will give an overview of the various video streaming methods in plain English and bring the all-round developer and publisher up to date.

In a nutshell, there are three widely used ways to stream a video: Progressive Download, RTMP/RTSP Streaming, and Adaptive HTTP Streaming. We'll look at the three in detail here, describing their pros, cons, and various technologies that support each.

Note: more streaming methods (like MultiCast or Peer 2 Peer) exist, but are of little use to the regular publisher. They are not widely adopted and/or only useful in specific situations.

Progressive Download

Progressive Download is the most widely used video delivery method by far (in part because it's what YouTube uses). It's also easiest to implement: just put a video on your webserver and point your player to the URL. Once a user hits play, the player immediately starts downloading the file. The player will start video playback as soon as it has enough data to do so, but it will continue to download until it has received the whole file (hence the progressive).

visualization of Progressive Download

Progressive Download is supported by Flash, HTML5 browsers, the iPad/iPhone and Android. On the server side, every regular webhoster supports downloads, as does every CDN (Content Delivery Network; webhosters that special in large-scale delivery). In most cases (Flash needs a small server module), it is possible to seek in a player to a not-yet-downloaded part of the video. At that point, the player re-downloads the video, starting at the seek offset instead of at the beginning. We call that feature pseudo-streaming.

The simplicity of Progressive Download also has its downsides. For one, bandwidth is wasted on data downloaded but not watched. Consider a user watching a ten minute video. They may leave the page after having watched only one minute of the video, but at that point the other nine minutes have already been downloaded. This means that the publisher has paid to transfer nine times as much data as the user actually watched – an expensive proposition on a large scale.

Another downside is the inability to change the quality of the video mid-stream: once the download starts, the video quality is locked. After switching a player to fullscreen, you generally see a blurry video, because it was intended to be watched at a much smaller size. Or, when you watch video on an iPad, your connection may switch from WiFi to 3G. Playback then stutters, because the download speeds are much lower on 3G.

In sum, Progressive Download works fine for short clips (a few minutes). For longer videos, the downsides start to impact playback too much. Plus, live streaming is not possible, as there's no downloadable file.

RTSP/RTMP Streaming

Because of the downsides of Progressive Download, RTMP/RTSP Streaming is widely used by professional media organizations like Hulu. This method uses specialized webservers that only deliver the frames of a video the user is currently watching. No data is downloaded in advance and data a user has seen is immediately discarded.

visualization of RTMP RTSP Streaming

The most widely solution is used is RTMP (Real Time Messaging Protocol), the streaming protocol of Flash. It is supported by servers such as FMS and Wowza and most CDNs (but not by regular webhosters). HTML5 does not include a dedicated streaming protocol, nor does the iPad/iPhone. Android has support, for RTSP (Real Time Streaming Protocol). Unfortunately, RTSP is not widely supported by servers and CDNs.

This lack of support, especially on the server side, is the biggest drawback of RTMP/RTSP Streaming. Most publishers do not want to maintain expensive, dedicated servers to stream their videos. Additionally, the dedicated protocols (RTMP and RTSP) are often blocked by corporate firewalls.

On the plus side, RTMP streaming can change video quality mid-stream. This allows for optimal playback quality in the fullscreen and WiFi/3G scenarios described above. However, if the connection speed drops below the minimum bandwidth needed for the video, playback will be continuously interrupted. Unlike progressive download, users cannot pause a video and wait for enough data to download to ensure smooth playback.

In sum, RTMP/RTSP Streaming works great even for long-form or live video. It has specific server and protocol requirements, which makes it less accessible and adds significant complexity and cost as compared to Progressive Download.

Adaptive HTTP Streaming

Adaptive HTTP Streaming is a fairly new streaming format. It attempts to join the merits of RTMP/RTSP Streaming (bandwidth efficiency, quality switching) with those of Progressive Download (no special servers or protocol needed). Adaptive HTTP Streaming works by storing your videos on the server in small fragments (a few seconds each). The player then glues these fragments together into a continuous stream.

visualization of Adaptive HTTP Streaming

At present, Adaptive HTTP Streaming is supported by both Flash and the iPad/iPhone. Android supports it as of version 3 and support in HTML5 is currently under development. Since Adaptive HTTP Streaming leverages standard webservers, it is supported by webhosters and CDNs alike.

Although Adaptive HTTP Streaming eliminates many of the downsides of RTMP/RTSP Streaming and Progressive Download, it still has issues of its own, the biggest being the lack of standardization. Because it is a new technology, there is no single, widely used implementation. The most popular is currently Apple's HLS (HTTP Live Streaming), which is supported by the iPad/iPhone and Android 3.0. However, both Adobe and Microsoft have competing offerings (Zeri & Smooth) and the MPEG consortium is working on a standard named DASH.

It's also worth noting that none of the Adaptive HTTP Streaming implementations work with regular MP4 files. They all require your files to be converted from a regular MP4 into a specific fragmented format. Apple, Microsoft and Adobe each supply a tool for this, but support for these formats doesn't exist in regular video editors and transcoding tools (yet).

In summary, while Adaptive HTTP Streaming will likely become the single video streaming method over time, the technology is still fragmented (no pun intended) and ecosystem support is only beginning to arrive.

Summary: Streaming Support

This table sums up support for the various streaming methods across devices and servers.

Devices Progressive Download RTMP/RTSP Streaming Adaptive HTTP Streaming
Adobe Flash Player MP4, FLV RTMP HLS, Zeri, Smooth
HTML5 (Safari & IE9) MP4 - -
HTML5 (Firefox & Chrome) WebM - -
iOS (iPad/iPhone) MP4 - HLS
Android Devices MP4, WebM RTSP HLS (as of 3.0)
CDNs (e.g. CloudFront) MP4, FLV, WebM RTMP HLS
Web Servers (e.g. S3) MP4, FLV, WebM - HLS

23 Comments

  1. Ethan LongTail May 26, 2011 - 11:18 EDT

    @Matt Cosentino – You can contact us here – http://www.longtailvideo.com/support/contact-us

  2. JeroenW August 15, 2011 - 07:01 EDT

    I do think we’ll keep this as a separate plugin for now (next 6 months?). We’re holding off until there’s a good breakthrough in HLS support (e.g. FMS supporting it, or Android 3 becoming the “main” version).

    At present, there’s also still quite some people with Flash Player 10.0 (~15%). Since the plugin needs 10.1, they won’t be able to see HLS streams.

  3. JeroenW January 30, 2012 - 09:08 EDT

    JW Player can play HLS streams. By default, this only works on iOS, but you can use this plugin to also play the streams in Flash:

    http://developer.longtailvideo.com/trac/browser/branches/adaptive

  4. JeroenW October 17, 2011 - 02:21 EDT

    Solely using HTTP does not work, since you’re then missing out on a chunk (15%) of desktop users that don’t have Flash 10.1+ installed. You should use RTMP for the desktop and Android, plus HLS for iOS.

  5. Clive Davis December 10, 2012 - 06:03 EDT

    Great info thanks Jerone.

  6. JeroenW February 29, 2012 - 02:37 EDT

    It’s as simple as this:

  7. Gilles January 27, 2012 - 11:24 EDT

    Hi,

    I have a streaming server, playing Apple HLS ans Adobe HDS.
    Is JWPlayer playing those kind of streams ?

    Gilles

  8. D July 8, 2012 - 01:17 EDT

    easy to understand explanation. Thanks.

  9. yash February 18, 2013 - 08:31 EDT

    thanks for the detailed info.

  10. SK February 15, 2012 - 12:06 EDT

    Does Adaptive stream works on Android devices? if so, Please provide me some tips/instructions to build HLS

  11. Bob October 16, 2011 - 07:36 EDT

    My understanding is that Apple’s HLS system on works on OS X and iOS. If someone wants to publish and allow playback on streaming media using HTTP, and wishes to reach OS X, iOS, Windows, and Android clients, any ideas which route(s) to go?

  12. Rishabh Parekh October 16, 2012 - 10:16 EDT

    very clear explanation! Thanks for grt tutorial

  13. JeroenW February 15, 2012 - 08:22 EDT

    HLS streaming should work on Android 4.0 devices, but we don’t have a device to confirm this yet. According to the specs, HLS also works on Android 3.x devices, but on our test device (Galaxy Tab 10.1), this consistently crashes. Android 2.x devices (the far majority) do not support HLS.

    I’d wait a little while until it’s confirmed Android 4.x does play HLS correctly and it has some market share.

  14. Dan August 14, 2011 - 08:39 EDT

    @Jeroenw: Thank you for the great work to support HLS natively in flash. When do you think this will be a supported feature in the released version of jwplayer? This will be key so we can have a standard encoding solution across client platforms.
    Thanks and keep up the great work!

  15. cholla February 28, 2012 - 09:22 EDT

    Thanks for Article.
    Can any one provide sample code on video streaming using adaptive http streaming in Android.

  16. Matt Cosentino May 26, 2011 - 10:35 EDT

    For my company’s new service we have been trying to get adaptive streaming working for Android and iOS devices. We originally were using Smooth Streaming for desktops and were able to use Microsoft software to convert this to HTTP Live Streaming files. This works great on iOS devices, but I’ve found that Android’s support is very spotty. It actually works much better in Android 2.3 than it does in 3.0. So, I’ve looked into your adaptive media provider that you linked to, and it seems to be exactly what I need, but it doesn’t quite work. There are some minor issues that I was able to work around, but the big problem is that it is unable to transition between fragments on Android. I’d like to offer some help in ironing out these issues, but couldn’t figure out how to post bug reports or submit patches, and this is the only way I could find to contact you.

  17. Shalom Ormsby April 29, 2011 - 12:47 EDT

    Very helpful overview. Thanks!

  18. Dennis Lundin April 29, 2011 - 06:55 EDT

    This is so good Jeroen, excellent.

  19. JeroenW April 29, 2011 - 07:23 EDT

    CPU usage for loading should be negligible compared to CPU for playback on all devices. However, there is one exception: inside Flash, for progressive downloads, CPU usage for loading is quite high. It’s really strange and we never got why that is the case, but it is definitely happening.

  20. Ajay Karat April 29, 2011 - 08:01 EDT

    Extremely helpful insight on how this fucntions. For a layman like me, this technology is beginning to make sense

  21. Carlos Rocha May 17, 2011 - 05:46 EDT

    Nice explanation. Thanks.

    Can JWPlayer play HLS HTTP Live Streaming such as

    http://demo.anevia.com/hls/hls/redbull_linecatcher.m3u8

    What are the subtitle formats supported by JWPlayer?

  22. Erick Rugerio April 28, 2011 - 08:40 EDT

    Excellent your streaming note!

    Can you explain more about CPU Usage when loading a video? Does it depends of size of the video file? I think would be a nice theme to talk about.

    Thanks!

  23. JeroenW May 18, 2011 - 02:01 EDT

    The JW Player can indeed play HLS streams. With the regular player, this works only on an iPhone/iPad or with Safari. We also have an in-development version that allows playing HLS streams in Flash though. It can be found here:

    http://developer.longtailvideo.com/trac/browser/branches/adaptive

    For subtitles, the player supports SRT, DFXP and MP4 TimedText:

    http://www.longtailvideo.com/support/addons/captions-plugin/14974/

Post a Comment

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>