Dynamic Streaming & Stream Switching with Flash Video

By Larry B, March 26, 2010

My latest article, How to do Dynamic Streaming with Flash Media Server, has been published at streamingmedia.com. There’s a page with code examples and demos of dynamic streaming on this site, as well.

I’d expected that writing this article was going to be an easy, quick process of explaining how to encode multiple files and set up a playlist in a couple of popular Flash video players (JW FLV Player 5.1 and Flowplayer 3.1.5). My problems started when I decided to actually test out the process as documented by Adobe and the player vendors by creating actual bandwidth fluctuations and watching the behavior of the player.

Imagine my surprise when things didn’t always work very well. I started to wonder if the dynamic streaming technology wasn’t really ready for prime-time. Some research led to finding and testing the 1080p player (from flashstreamworks.com), which had simply outstanding switching performance – on the same videos from the same server. So the technology definitely works. [whew!]

After much research, talking to the vendors, and more testing, I came up with some guidelines that make things work pretty well in all the players. Still, there remains some room for improvement in the implementations of dynamic streaming in the popular players I tested, particularly when it comes to detecting bandwidth changes and then smoothly switching streams during playback.

Testing Adaptive Streaming by Controlling Bandwidth

By Larry B, March 17, 2010

In the course of researching my article on Dynamic Streaming in Flash, I ended up doing way more testing than I’d initially intended. But things didn’t work the way I expected right away, and being the way I am (foolish? glutton for punishment?), I had to find out why.

There’ll be more on that in the article when it comes out on streamingmedia.com, but for now, I wanted to make a note about how to simulate fluctuating bandwidth conditions.

On Windows, Netlimiter 3 Lite works OK, especially if you’re just doing bandwidth detection to select the appropriate stream at startup. Shunra VE Desktop seemed to create more realistic test conditions for fluctuating bandwidth and stream-switching during playback, an impression that was validated by colleagues I spoke with. At $850 a pop, it certainly ought to be better than the $20 NetLimiter.

But on the Mac, it all worked for free. It’s already built in to the OS’s Unix roots.  It’s in the ipfw command.  You set it up by creating filters with bandwidth limits, then associating those filters with the ports you want limited.  Here’s how to set up a bandwidth limiter for testing streaming over all ports. Note that if you’re not logged in as root, you will need to use sudo to run these:

sudo ipfw pipe 1 config bw 400kbps
sudo ipfw add 10 pipe 1 tcp from any to me
sudo ipfw add 11 pipe 1 tcp from any to me

Change it at will by issuing the pipe command again…

sudo ipfw pipe 1 config bw 1400kbps

Or remove the filters like this…

sudo ipfw delete 10
sudo ipfw delete 11

You can also introduce simulated network latency, control outbound bandwidth separately from inbound, and control bandwidth to or from a single IP address or subnet.  There’s great documentation at Luigi Rizzo’s Dummynet site.  Thanks also to Ask Bjorn Hansen for his mini-tutorial on this.

A Second Test of YouTube’s Captioning

comments Comments Off
By Larry B, March 12, 2010

Audio quality being very important to the success of speech-recognition, I’ve re-recorded the video from my YouTube speech-recognition auto-captioning test. This time I used a high quality condenser mic plugged into a good mixer, and recorded in an acoustically good space.

With good quality audio, YouTube made a much better caption file. To be fair, in the beginning I throw around a few company names which aren’t real words, and I didn’t expect those to be right in the caption. But YouTube seems to be unable to recognize “YouTube”, which is kind of funny in its own way.

The other issue is the awful audio/video sync problem I’ve had recording direct from Webcam into YouTube. Oddly, I downloaded the video and corrected the problem using QT Sync. When I re-uploaded the corrected file to YouTube, the sync was off again.

Anyway, the captions are the interesting part. Here’s the clip:

YouTube Offers Speech-Recognition Captioning

By Larry B, March 5, 2010

It was only a matter of time. YouTube is bringing the speech recognition technology from Google Voice to bear on all the video in its vast library.

The industry has seen a variety of solutions for using speech-recognition to create a transcript of a video or podcast. Virage, Pictron, Streamsage, Podzinger all have done this. Only Pictron is more or less the same company it was at the start. Virage was acquired by Autonomy and has languished there as a Web product, Streamsage was acquired by Comcast and turned into an internal division, Podzinger has become Ramp…I’m not sure what they do, at this point, but it’s not the podcast transcription service they used to be. Virage and Streamsage go back almost ten years in this space, but their systems are still running in various enterprise and educational settings.

But back to YouTube… I use Google Voice, and the speech recognition is pretty good.  I rarely have to actually listen to a voice mail, since it shows up in my email as a text message that’s almost always easily decipherable, if not perfect. So just for fun, I tried YouTube’s captioning. Here’s the result.

Usually, speech-recognition provides a good set of words for searching, if nothing else. I’ve used speech-to-text to create searchable text from a video with very good results. It makes the video file, which is essentially opaque to a search engine, into something transparent. OK…in this case, maybe translucent.

I’m sure this would do better with better audio, and I will test that. In the meantime, YouTube does provide the means to download and edit the caption file, which is probably what this is best suited for, anyway. It’s a head start on a caption file, complete with time markers already in place. For those of us who are not professional transcriptionists, that has to beat making one from scratch.

Low-tech high-value instructional video

comments Comments Off
By Larry B, March 4, 2010

Lots of us involved in instructional technology content development are rightly cognizant of high production values and a carefully edited script.  In my prior job as director of educational technology development at Harvard Business School, we were very focused on making highly-designed instructional products that looked great, sounded great, and didn’t waste a syllable in their tightly edited, word-crafted voiceover.

Nothing wrong with that, if that’s your target market and you’re planning to productize the content at a high price.  But there’s another way, as Jon Udell highlights in his conversation with Sal Khan, principal of Khan Academy.org.  Khan (interestingly, an MBA graduate of Harvard Business School) has created over one thousand instructional videos aimed primarily at middle/high school and college students on topics from Biology to Physics to Economics to Mathematics.

Khan uses nothing more complicated than a screen capture program like Camtasia, a Wacom tablet and a $20 headset to create powerful, explanatory tutorials that give the feel of looking-over-the-expert’s-shoulder.  Khan’s videos are posted to YouTube, which has granted khanacademy an exception to the ten-minute limit that applies to conventional YouTube channels.

What’s amazing is the scalability of this approach. Khan has been able to create this vast collection of material because he’s found the right combination for effective teaching while having a scalable process. You might think that reaching kids today means competing with video games, high-def TV, sophisticated animations and graphics by trying to beat those formats on their terms.  Khan’s gone the other way, and hit a home run, as evidenced by the popularity of his site and the feedback coming from kids, parents, and teachers.

There’s lots of rich detail in Jon’s interview with Sal on IT Conversations and it’s worth a listen (even if you don’t usually find podcast interviews compelling – this one is worth the download). Khan is working on analytics, assessment, and other innovation around the library of content he’s creating.

But at its heart, the lesson I see in this is that it’s not always about having the most advanced technology and picture-perfect production. Figuring out how to reach your audience and be effective, might mean going decidely low-tech.

Flash Video Performance on the Mac – Finally Some Real Data

comments Comments Off
By Larry B, March 1, 2010

Is Flash video on a Mac a CPU hog?  More than on Windows? If so, why?

Thankfully, someone’s finally done a test to put some data behind the anecdotes.  (Doh!  Why I didn’t think of doing that?!?) Jan Ozer over at the Streaming Learning Center hastested Flash video vs. HTML5 video, covering all the browsers on both Windows and (Intel) Mac, and Flash versions 10.0 and (the new, performance-optimized) 10.1.

It’s hard to summarize the findings without leaving out important detail, so I recommend looking at Jan’s data directly.  The tables are revealing.  But in a nutshell, Jan found that where the video decoder can access hardware acceleration, performance is excellent, and where it can’t…not so much.  This means that on Windows, Flash is actually slightly more CPU-efficient than HTML5. On the Mac, where Apple has not made API hooks to its graphics hardware acceleration available to software developers, Flash and HTML5 are both hogs – unless you’re using HTML5 in Safari.  It suggests that Apple is using graphics acceleration APIs that it’s keeping from others who are developing applications for the Mac. (Kinda smells like what Microsoft was accused of years ago – keeping various Windows APIs secret so that its non-OS products would always have an advantage over  competitors. Microsoft has denied this. )  

Is it fair – or smart – to withhold powerful APIs from the devleopers who create the applications that make your computer useful and relevant to users?  At best. it’s disingenuous for Apple to criticize Adobe for Flash performance on the Mac while keeping access to hardware acceleration under wraps.  

In any case, Jan’s tests show that Adobe is continuing to work on this (to the extent that it can).  Video performance in Flash 10.1 is improved over 10.0 on both Mac and Windows. On Windows, the difference is dramatic.