HTML5 Video: Still not there yet, but making headway

Last year, we declared that HTML5 video was not quite there yet. Well, it’s nearly 18 months since that post, so what’s happened in the intervening time?

Let’s start off with the positive: compared to 18 months ago, the browsers’ implementations of the <video> tag have become much more stable. They also all have built-in support for pseudo-streaming (i.e. the ability to seek to an unbuffered position in the video file). The tools surrounding HTML5 video — streaming servers, transcoders, etc. — have also improved.

On the other hand, many of the issues we pointed to in our last post have not yet been resolved. HTML5 video is still "not quite there yet", but we’ll take a look at the developments over the past year and make some predictions on where this is all headed.

Answering some open questions

In our last post on the subject, Jeroen mentioned three major issues which prevented HTML5 video from taking off. Let’s see where these issues are today.

Codec support

The major question surrounding codecs revolves around whether open video codecs such as Google’s WebM and Ogg Theora would overtake proprietary formats such as H.264. As before, this still comes down to browser and device support. Though the open-source browsers have decided on an open format — WebM — no current mass-market mobile device has hardware drivers which can decode VP8 (the video codec used in WebM).

On the desktop, the sides seem firmly entrenched. Apple has no interest in moving away from H.264, considering the investment they’ve made in the technology. Microsoft doesn’t support WebM natively (users need to manually install the codecs). Firefox and Opera still have no desire (or, possibly, ability) to support H.264. And although Google announced in January that H.264 support would be removed from Chrome, more than 8 months later, there’s been no indication when (or even if) this will happen.

The codec wars are still being fought full-steam. And the end result of the lack of a standard video format is that many publishers simply encode once — in H.264 — and rely on Flash for support in browsers that don’t support it.

Streaming

We predicted that the browsers would choose to implement their own streaming mechanism to compete with Apple’s HTTP Live Streaming (HLS) format (built into desktop and mobile Safari). The question would be what open standard would be chosen to build upon?

What happened instead is that this area was more or less overlooked. Safari Desktop and Mobile, and Android 3.0+ are still the only browsers with built-in support for an HTML5 streaming protocol. Publishers who require that their content be streamed (rather than simply downloaded) on any other device or browser must fall over to a Flash or Silverlight player.

Moreover, a common streaming format has still not been agreed upon. The browser vendors are reluctant to hitch their wagon to any one method of HTTP streaming — for technical reasons, and because they’re waiting to see which formats users begin implementing themselves. Of course, this is something of a chicken-or-egg proposition.

Fullscreen

A draft standard for adding fullscreen support to browsers has been written up and more or less agreed upon. Safari has already implemented support for this in version 5.1. While no other browsers have implemented the necessary APIs, Chrome (which uses the same rendering engine as Safari, called Webkit) is probably the closest to making this a reality. Although Mozilla wrote the spec, there’s currently no word on when Firefox will implement it Firefox has implemented fullscreen support in their nightly build, but the feature is disabled by default. In short, true fullscreen functionality is still lacking on most browsers.

It’s worth mentioning that on most mobile phones (not tablets), HTML5 videos will always play in fullscreen mode. This is partially due to limitations in hardware, but it also makes sense from a usability perspective — if you’re watching video on a web page on a 3.5-inch screen, do you really want to see anything other than the video itself?

What’s coming up

So what can we expect within the next 6-12 months? We recently attended the Open Video Conference in New York, and rubbed shoulders with the people who are making HTML5 Video happen — browser engineers from Apple, Google, Mozilla and Opera were all in attendance (not to mention YouTube, Netflix, Internet Archive and many more). Everyone had a ton to bring to the table (Silvia Pfeiffer, who organized the conference’s Open Media Developer track, has written a great summary of the work that went on). And while there’s a lot of exciting technology being built, the core problems I mentioned earlier which are preventing HTML5 from becoming the de facto standard in web video (format compatibility and streaming) are still not being addressed in a unified way.

That being said, some of that new stuff is pretty cool, and may end up moving the needle. Let’s take a look.

The <track> Element: Adding Text Elements to Media

The HTML5 <track> element is for adding timed textual metadata to a media element. This enables video players to add functionality to support subtitles, closed captions, chaptering and more. While none of the browsers currently support <track> natively, we got to see a cool JavaScript demo which added captioning functionality into a <video> tag.

Once this becomes a reality, text tracks will be an exciting new development in HTML5. It will help make video accessible to the seeing- and hearing-impaired, enable subtitles for foreign language speakers, and make video files indexable (and therefore searchable). All of these will pave the way for video and audio to become first-class citizens on the web.

MediaSource API: Direct access to video data

Google is working on a feature for Chrome, called the MediaSource API, which will allow JavaScript developers to manipulate video data at the lowest possible level (the byte stream). One application of this mechanism would be for video players to piece together small bits of video, serving as the basis for an adaptive streaming application.

This approach puts the onus on JavaScript video players, such as the JW Player, and on video content providers, to come up with their own solutions for streaming video in HTML5. While I’m sure there will be attempts at making this happen, the jury’s still out on whether this method can become the basis for HTML5 video streaming, for the usual reasons — browser fragmentation, and the ever-present format wars (Chrome’s implementation will only support WebM at first). My personal opinion is that until the browsers present a standard for HTTP streaming to developers and publishers, you’ll see a couple of cool demos, maybe even an end-to-end solution (think Netflix or YouTube), but not an overall standard.

Conclusions

Yes, progress has been made. The tools are better (for example, the popular desktop encoder Miro Video Converter now includes support for WebM). The servers are better (Wowza Media Server and Adobe Flash Media Server (FMS) support both Apple HLS for HTML5 streaming and RTMP for Flash streaming). And the browsers are better, especially the mobile ones (many of the HTML5 video bugs in Android and iOS were fixed in versions 2.3 and 4.0, respectively).

So why did I say at the beginning of this article that HTML5 still isn’t there yet? Why isn’t Flash losing more ground, considering its own drawbacks (poor resource utilization, lack of support on iOS, to name a couple)? The simple answer is that Flash offers a safer, more uniform cross-platform environment for video developers and publishers, while HTML5 presents a dizzying array of browsers to test against, and competing formats to support. You still need both Flash and HTML5 to cover all of your bases, but most publishers are still choosing Flash as their primary video delivery platform for these reasons.

The JW Player will continue to stay close to the bleeding edge of HTML5 video, and we’ll continue integrating both Flash and HTML5 together as seamlessly as possible. We’re rooting for HTML5 and its promise of open standards, better accessibility and better performance. But it’ll still be some time before HTML5 has everything it needs to take its place at the head of the table.

14 Comments

  1. genefer February 16, 2012 - 05:18 EDT

    Nice post. It would be even better if this site featured stand demos of these features so that users can get a clean look at the code. Please add a demo section containing a working implementation plus an explanation of the code.Buying and taking that plugin and integrating it into WP is really good for beginers . I would just like to share with everyone html5 video player

  2. stardust2810 January 31, 2012 - 05:44 EDT

    Hi!

    Regarding “play videos inline on iPhones”, jplayer seems to be able to do so while jwplayer will pop up the native player.

    May i know the reasons?

    Appreciated!

  3. tracey jaquith January 6, 2012 - 05:46 EDT

    Hi Pablo,
    (I’ve met you (or Zach?) briefly at OVC…)

    nice post.

    for chrome, we’re doing something like this in dev. underway on ww.archive.org
    hooking into the user initiated “click” (w/ your JW Player v5.8)
    and flipping into it’s pretty awesome pseudo-full screen (faux screen? 8-)

    whereas for safari, your 5.8 player is *already* (on a mac) using native fullscreen
    (not sure why you didn’t mention that? 8-)

    something close to what we are doing for chrome follows below.

    best regards,
    -Tracey Jaquith, Internet Archive

    var videoFS=false;

    jwplayer().onFullscreen(function(obj){
    var v=$(‘video’).get(0);
    if (v && v.webkitSupportsFullscreen)
    {
    if (!videoFS)
    {
    videoFS=true;
    v.webkitEnterFullscreen();
    }
    else
    {
    videoFS=false;
    v.webkitExitFullscreen();
    }
    }

  4. PabloS January 31, 2012 - 06:33 EDT

    @Stardust2810 -

    jplayer puts their own controlbar below the

  5. PabloS January 25, 2012 - 09:00 EDT

    Hi, Tracey!

    We’ve definitely met. Glad to hear you guys are using the JW Player. :-)

    I didn’t go into much detail about the JW Player fullscreen implementation in this post, since I wanted to discuss the state of HTML5 video in more of a player-agnostic way. But yes, the player code does something very similar to your example, specifically for Safari. In Chrome,

    In the future, we’ll definitely be moving towards complete fullscreen functionality in all the browsers that support it, but so far it’s been a bit of a moving target.

  6. Victor October 10, 2011 - 04:17 EDT

    Poor resource utilization? Maybe you guys could look into using StageVideo.

  7. PabloS October 10, 2011 - 06:16 EDT

    @Victor -

    We’ve looked into Stage Video in the past, and may integrate it in a future version. However, it doesn’t really make too much of a dent in the JW Player itself, since the major use case Stage Video solves (compositing video elements with other Flash display objects) isn’t much of an issue in the player, which fades out its player controls during playback.

    The resource utilization I was referring to was the overhead of running a separate runtime inside the browser in a plugin model. Even if Flash were much more lightweight (which it’s not) you’d still have to allocate a sizable chunk of additional memory to run it.

  8. Miller Medeiros October 10, 2011 - 07:27 EDT

    Forgot to talk about the iOS bugs and limitations: http://blog.millermedeiros.com/2011/03/html5-video-issues-on-the-ipad-and-how-to-solve-them/ and http://blog.millermedeiros.com/2011/04/unsolved-html5-video-issues-on-ios/ – Cheers.

  9. PabloS October 10, 2011 - 07:00 EDT

    @Miller -

    Some good links there. We briefly touched upon the iOS bugs – many of the big ones were solved in iOS 4.x, which is much more stable / consistent. The iOS limitations, such as the inability to autostart videos without user input, change volume control in JavaScript or play videos inline on iPhones, are less about HTML5 in general and more about designing players around Apple’s specific browser limitations.

  10. Jerome October 11, 2011 - 01:44 EDT

    Hi Pablo,

    Great article. It’s nice to see some progress being made in this very complex and segmented space!

    Just a comment on StageVideo: I think implementing it into your very popular player should be a priority. Yes StageVideo is great for compositing but that’s only because the video decoding and rendering is pushed to the GPU. So once Flash is done rasterizing the display list, it just has to composite 2 bitmaps on the GPU. OSMF 1.6 implemented StageVideo with Seamless software fallback and so should you. The trend in devices is bigger or more dense screens with less CPU power and better GPUs. To ensure a much better experience on supported hardware StageVideo should now be at the core of any video player.

    Bigger and higher quality videos, less CPU, more battery life… Why exactly are you not implementing it?

    ;)

  11. Chris Double November 1, 2011 - 09:44 EDT

    Regarding “Although Mozilla wrote the spec, there’s currently no word on when Firefox will implement it”, Mozilla Developer Chris Pearce provided details on the state of the Mozilla implementation here: http://blog.pearce.org.nz/2011/09/mozilla-full-screen-api-progress-update.html

  12. Janni October 11, 2011 - 12:58 EDT

    Great article, and good to see that longtail are watching how this evolve closely.

  13. PabloS October 11, 2011 - 02:21 EDT

    @Jerome -

    Good points.

    The reason Stage Video is not already implemented in the JW Player is that the control flow surrounding it is a bit awkward, and would require a fair amount of refactoring throughout the player to make it work. That being said, Stage Video on our roadmap for the next major release of the player (v6), where we’ll be able to build it in from the start, rather than try to bolt it on after the fact.

    But here we are talking about helping Flash to run more efficiently — weren’t we supposed to be talking about HTML5?

  14. PabloS November 1, 2011 - 03:35 EDT

    @Chris -

    Thanks for the update; I hadn’t seen that. It looks like they’re not too far away!

Post a Comment

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>