Comments (4)

By Larry B, May 27, 2010

Our adventures in live mobile streaming continue. If anyone should happen to read this post on Thursday May 27, you can see the results of this effort at http://harvard.edu/commencement2010/

So what are the lessons learned so far? Here’s a preliminary list in no particular order:

• Setting up the server side of things is the easiest part. Configuring for FMS delivery from Limelight, and for Wowza on Amazon EC2 was a breeze. Multiple bitrates, the RSS playlist for JW Player, the SMIL playlist for Wowza….once you figure out the moving parts, it works almost just like it’s supposed to.
• Adaptive streaming from Limelight and other CDNs that use the ‘fcsubscribe’ method for load-balancing can cause a problem when switching to a streams that comes from a new edge node. More on this later…
• Mobile devices: Make sure you’re encoding H.264 with Baseline profile level as low as you can go.  iPhones and iPads turned out to be the easiest to support fully.  Blackberries and Droids work…or they don’t. It seems to depend on the model phone, and on the network you’re on.  My personal Blackberry gets the RTSP stream just fine.  Others around the office with different Blackberries can’t play the stream. Same with Droids – some people are able to play it, some not. I haven’t discovered why just yet.  Codec issues are a likely possibility, but it’ll take some digging to find out. I have not found any useful documentation on the differences between Blackberry models, in terms of live video streaming support.
• Encoders – this has been the headache of all headaches and took many many man-hours to get right.
  • Encoding three bitrates (100k, 500k, 1000k) to two different CDNs (Limelight, Wowza/EC2) takes a lot of horsepower.
  • One brand new 8-core Cisco machine with a brand new Osprey 240 proved unsuitable for capturing video at all.
  • Adobe Flash Media Live Encoder (FMLE) and Telestream Wirecast on Windows both depend on your display hardware and drivers. If you’re planning a headless encoding system, plan extra time to get it all working.
  • A 2-core IBM/Windows/Osprey system running FMLE gave us better encoding performance than an 8-core Mac Pro/AJA system running Wirecast.
  • All of the above systems had issues with audio/video sync, either being off from the start, or drifting as the webcast went on. Only on the Mac/AJA system were we able to resolve these in time for a successful webcast.
  • Ordinary desktop PC running consumer USB video capture devices are easiest to set up and are the machines most likely to work right off the bat.  No audio/video sync issues occurred with these, even though we were capturing video on one of a couple $50 USB
    devices and audio using the built-in audio support on the PC.  The more expensive and industrial-grade the hardware, the more trouble it gave us.
  • Our final encoding configuration included an 8-core MacPro/Wirecast for the 1Mbps and 500kbps streams, a single-core desktop PC running FMLE for the 100k streams, and a dual-core desktop PC with FMLE for capturing a 1.2Mbps H.264 archive file.
  • Some of our partner schools are using our infrastructure for mobile streaming. They’ve got Digital Rapids TouchStream appliances, and have had no encoding issues doing multiple bitrates from HD down to 3G/mobile. I’m quickly becoming a big fan of purpose-built appliances for encoding.

That’s about it for now…I’ll follow up on some of these as we do some analysis and learn more.

Bookmark to:

Comments (2)

By Larry B, May 18, 2010

For an upcoming university commencement, I’ve been looking into doing live streaming in H.264/Flash, as well as http streaming to Apple iPhone/iPad/iPod devices (herein referred to as iP* devices) and rtsp streaming to Droids and Blackberrys. It’s been an experience piecing it all together, and I’ll be writing about some of the surprises and pitfalls as we figure out how to best do it.

In a nutshell, we’re using Limelight Networks’ Flash Media Server 3.5 for delivery to browsers on PCs and Macs. For mobile streaming, I provisioned and started up an instance of Wowza on Amazon EC2. One stream in (or several, for multiple bitrate support) via RTMP, and Wowza delivers in all the right formats – whether it’s chunked HTTP (Apple devices), RTSP (Droids and Blackberrys), or RTMP (Flash).  Setting that up involved an awful lot of moving parts, but half a day later, it was up and running and has been flawless in testing. We’ve been streaming multiple bitrates (100kbps, 500kbps, 900kbps) from Adobe Flash Media Live Encoder on a PC, as well as from Telestream Wirecast on a Mac.

We’ve developed a page that uses the JW Player (Flash) as the default, and falls back to HTML5 if it’s an iP* device, or provides an rtsp:// link if it’s a Droid or a Blackberry. Yes…Flash is the default for all browsers that will allow it, as it provides a uniform experience for all users, and a single thing to worry about from a user-support perspective.

What’s been interesting to me is how quickly it all went together. In a couple of days, starting with no deep mobile experience, we’ve provisioned infrastructure in the cloud, configured it, and are up and running with live Flash and mobile streaming for short money. More details to follow in the coming days…

Bookmark to:

Comments (5)

By Larry B, May 2, 2010

Flash security constraints can prevent a SWF hosted on one domain from reading data hosted on another domain. Users trying out the SlideSync and SlideScroller plugins might encounter this issue if the XML data file that contains the slide URLs and timing is on a different website from the one that hosts the JW FLV Player itself.

The solution is to add a crossdomain.xml file to the root directory of the web server that hosts the XML file. There’s official Adobe docs on crossdomain policy files, and here’s a pretty good tutorial on crossdomain.xml files.

In short, you create this simple file using any text editor, and set up a rule describing which other sites may access data hosted on the site where the crossdomain.xml file lives.

<?xml version="1.0"?>
<!DOCTYPE cross-domain-policy SYSTEM "http://www.adobe.com/xml/dtds/cross-domain-policy.dtd">
<cross-domain-policy>
   <allow-access-from domain="*.learningapi.com" />
</cross-domain-policy>

sbs!

Bookmark to:

Comments (9)

By Larry B, April 6, 2010

For years I’ve been hearing and reading about demand for a simple synchronized slides plugin for the JW Flash Video player. Sure, you can do it with some Javascript: add event listeners to track the play-head position and use that to trigger image loads in a separate DIV. But that requires page scripting and introduces dependencies that might not always be do-able.

But I always thought there oughtta be a simpler way. So I made one. Honestly, I didn’t know if it was possible using the JW Plugin API, and while I’m a pretty good Java/Web programmer, I’m definitely not a Flash/ActionScript ace. So I decided to give it a try as a learning experience. The result is two plugins for the JW FLV Player: SlideSync and SlideScroller. These are free for commercial and non-commercial use.

You can see an example of the SlideSync and SlideScroller plugins in action, or look at documentation of the options and parameters, or go to Longtail Video’s plugin pages for the SlideSync and SlideScroller plugins.

There’s a lot of room for improvement and growth in this. It’s really a first-effort, but should be useful anyway in some cases. There are still reasons to use the Javascript event-listener model as well, which offers lots of flexibility and control you won’t get from the this plugin. But for simplicity, this is a good start. Feedback is welcome. Improvements welcome, too! The source is linked on the documentation page.

Bookmark to:

Comments (1)

By Larry B, March 17, 2010

In the course of researching my article on Dynamic Streaming in Flash, I ended up doing way more testing than I’d initially intended. But things didn’t work the way I expected right away, and being the way I am (foolish? glutton for punishment?), I had to find out why.

There’ll be more on that in the article when it comes out on streamingmedia.com, but for now, I wanted to make a note about how to simulate fluctuating bandwidth conditions.

On Windows, Netlimiter 3 Lite works OK, especially if you’re just doing bandwidth detection to select the appropriate stream at startup. Shunra VE Desktop seemed to create more realistic test conditions for fluctuating bandwidth and stream-switching during playback, an impression that was validated by colleagues I spoke with. At $850 a pop, it certainly ought to be better than the $20 NetLimiter.

But on the Mac, it all worked for free. It’s already built in to the OS’s Unix roots.  It’s in the ipfw command.  You set it up by creating filters with bandwidth limits, then associating those filters with the ports you want limited.  Here’s how to set up a bandwidth limiter for testing streaming over all ports. Note that if you’re not logged in as root, you will need to use sudo to run these:

Change it at will by issuing the pipe command again…

Or remove the filters like this…

You can also introduce simulated network latency, control outbound bandwidth separately from inbound, and control bandwidth to or from a single IP address or subnet.  There’s great documentation at Luigi Rizzo’s Dummynet site.  Thanks also to Ask Bjorn Hansen for his mini-tutorial on this.

Bookmark to:

Comments (0)

By Larry B, March 12, 2010

Audio quality being very important to the success of speech-recognition, I’ve re-recorded the video from my YouTube speech-recognition auto-captioning test. This time I used a high quality condenser mic plugged into a good mixer, and recorded in an acoustically good space.

With good quality audio, YouTube made a much better caption file. To be fair, in the beginning I throw around a few company names which aren’t real words, and I didn’t expect those to be right in the caption. But YouTube seems to be unable to recognize “YouTube”, which is kind of funny in its own way.

The other issue is the awful audio/video sync problem I’ve had recording direct from Webcam into YouTube. Oddly, I downloaded the video and corrected the problem using QT Sync. When I re-uploaded the corrected file to YouTube, the sync was off again.

Anyway, the captions are the interesting part. Here’s the clip:

Bookmark to:

Comments (1)

By Larry B, March 5, 2010

It was only a matter of time. YouTube is bringing the speech recognition technology from Google Voice to bear on all the video in its vast library.

The industry has seen a variety of solutions for using speech-recognition to create a transcript of a video or podcast. Virage, Pictron, Streamsage, Podzinger all have done this. Only Pictron is more or less the same company it was at the start. Virage was acquired by Autonomy and has languished there as a Web product, Streamsage was acquired by Comcast and turned into an internal division, Podzinger has become Ramp…I’m not sure what they do, at this point, but it’s not the podcast transcription service they used to be. Virage and Streamsage go back almost ten years in this space, but their systems are still running in various enterprise and educational settings.

But back to YouTube… I use Google Voice, and the speech recognition is pretty good.  I rarely have to actually listen to a voice mail, since it shows up in my email as a text message that’s almost always easily decipherable, if not perfect. So just for fun, I tried YouTube’s captioning. Here’s the result.

Usually, speech-recognition provides a good set of words for searching, if nothing else. I’ve used speech-to-text to create searchable text from a video with very good results. It makes the video file, which is essentially opaque to a search engine, into something transparent. OK…in this case, maybe translucent.

I’m sure this would do better with better audio, and I will test that. In the meantime, YouTube does provide the means to download and edit the caption file, which is probably what this is best suited for, anyway. It’s a head start on a caption file, complete with time markers already in place. For those of us who are not professional transcriptionists, that has to beat making one from scratch.

Bookmark to:

Comments (0)

By Larry B, March 1, 2010

Is Flash video on a Mac a CPU hog?  More than on Windows? If so, why?



Thankfully, someone’s finally done a test to put some data behind the anecdotes.  (Doh!  Why I didn’t think of doing that?!?) Jan Ozer over at the Streaming Learning Center hastested Flash video vs. HTML5 video, covering all the browsers on both Windows and (Intel) Mac, and Flash versions 10.0 and (the new, performance-optimized) 10.1.



It’s hard to summarize the findings without leaving out important detail, so I recommend looking at Jan’s data directly.  The tables are revealing.  But in a nutshell, Jan found that where the video decoder can access hardware acceleration, performance is excellent, and where it can’t…not so much.  This means that on Windows, Flash is actually slightly more CPU-efficient than HTML5. On the Mac, where Apple has not made API hooks to its graphics hardware acceleration available to software developers, Flash and HTML5 are both hogs – unless you’re using HTML5 in Safari.  It suggests that Apple is using graphics acceleration APIs that it’s keeping from others who are developing applications for the Mac. (Kinda smells like what Microsoft was accused of years ago – keeping various Windows APIs secret so that its non-OS products would always have an advantage over  competitors. Microsoft has denied this. )  



Is it fair – or smart – to withhold powerful APIs from the devleopers who create the applications that make your computer useful and relevant to users?  At best. it’s disingenuous for Apple to criticize Adobe for Flash performance on the Mac while keeping access to hardware acceleration under wraps.  



In any case, Jan’s tests show that Adobe is continuing to work on this (to the extent that it can).  Video performance in Flash 10.1 is improved over 10.0 on both Mac and Windows. On Windows, the difference is dramatic.

Bookmark to:

Comments (0)

By Larry B, February 26, 2010

I had the distinct pleasure of attending Lawrence Lessig’s talk at Harvard Law School on behalf of the Open Video Alliance.  It was a terrific event, simulcast worldwide to dozens of screening locations using entirely open technologies; in particular, HTML5 and the Ogg Theora video codec.

Interesting side note that the session was funded in part by iCommons, the open standards/knowledge/software advocate; while I work at iCommons, Harvard University’s academic computing team.  I kept hearing “iCommons” mentioned, and it took a moment to recognize that it was another iCommons.  But I digress….

Lessig’s talk was great. The parts about open software, open standards, and the architecture (both legal and technical) of the read/write culture mirror closely the points in his books, and are eye-opening.  If you’ve not read Remix: Making Art and Commerce Thrive in the Hybrid Economy, you ought to.

But what I was really wondering about most during this talk was the open video concept and mention of the “Flash problem”.  Seems the Ogg Theora codec and HTML5 are seen as potential resolutions of a huge problem — the problem of proprietary video codecs and players. But as someone who builds, buys, deploys, and manages streaming video platforms and content, I couldn’t quite come to terms with all that I’d have to give up if I replaced the Flash Player in my solutions with HTML5 and Ogg.  Flash’s universality has been a tremendous boon to online video. Those of us who remember the format wars — Real vs Windows Media vs Quicktime, platform-specific plugins, single-platform codecs, browser incompatibilities — Flash is a breath of fresh air compared to that.  Being able to support a single set of APIs and codecs for all my users has been huge.  And, using a mature player such as the JW FLV Player,  being able to do stuff like:

• support for rtmp or http streaming
• callback event-based client-side scripting
• playlist support (RSS, ATOM, XSPF)
• bandwidth switching/adaptive streaming
• plugins for screengrabs or stats collection
• subtitles
• control over buffering

I can create an outstanding user experience using these tools, and do it for more than the degenerate case of simply putting a video in the page.  All sorts of interactive behavior can be easily layered into my video apps, and with no browser dependencies to worry about.

Contrast that with my first experience showing HTML5 video to a non-techie, my wife.  At the end of Lessig talk, the Open Video Alliance announced the winners of the Open Video in 60 Seconds contest, which gave contributors 60 seconds to explain open video using video.  One of the entries, (not the winner, although IMHO it should have been) was by Rafaella, a teacher from Italy who did an outstanding job showing that all creativity is but a link in a long chain of the creative contributions of others.  I came home eager to show my wife, who I thought would really appreciate it.

At the conference, it was played in Quicktime with English subtitles. At home, I quickly found it on the Web, in the HTML5 player….with no subtitles.  Huh?  You gotta be kidding me!   Helpfully, a download link is provided to the .srt file containing the subtitles. That’s helpful. After all, of course I’d want to read this in an open texteditor alongside my video:

Thankfully, a link was also provided to the original source site for the video, which offered a subtitled version, in…..you guessed it…Flash.

So….the Flash-based video world is seen as proprietary, which it is.  But as an applications guy, what makes a platform proprietary to me?  Vendor lock-in. Platform lock-in. Client-server dependencies.  I don’t really see this as a huge problem in Flash video.  I can deliver videos in Sorenson, On2, or MPEG4 codecs. I can use players by numerous vendors, or roll my own for free with the Flex SDK. I can serve video from any server, from FMS 3.5 to Apache to Wowza. I can switch from rtmp to http, or from Akamai to a free server under my desk. Or I can dump Flash and play the same MP4 content in Quicktime, RealPlayer, or Silverlight.  I’m not getting that proprietary locked-in feeling, really.

So what’s my point?  Not that Open Video is a bad idea….I think it’s wonderful. I love vendors with an open mindset and open products, like Kaltura.  Imagine when browser support for video approaches the support now universal for DOM scripting, Javascript, AJAX, etc. Powerful toolkits like JQuery and ExtJS are only possible because of the support for standards in the browser.  And, crucially, these toolkits have made it possible to do things in the browser that previously could only be done with Flash. There are some demos that show the promise of attractive, plugin-free Web video, although compatibility and functionality are still in a nascent stage.

But as a real working technologist solving problems on the ground every day, I don’t entirely understand the “Flash problem”.  I don’t want to employ closed technologies that narrow my options and lock me in, but I’m not seeing Flash as being that way very much when it comes to video.  But I’m eager to be educated.

Remix

Lawrence Lessig. Penguin Press HC, The 2008, Hardcover, 352 pages, $3.89

Bookmark to:

Comments (0)

By Larry B, January 6, 2010

A test of the Wordpress Kaltura Plugin. Interesting how they’ve implemented this. I love the idea, and the design of the API.

Interesting, too, that my first upload, an h.264/AAC Quicktime file (.mov) will not work. Kaltura lets me upload the file, name it, and insert into the page. At that point, what you see here is what there is.  No error message, no indication of what’s wrong. Just no video.

With a little more digging, it appears to be an issue with the Kaltura WordPress plugin rather than the Kaltura server-side app. I can log in to the Kaltura Management Console and play the video there, but the EMBED statement Kaltura plugged into this page is lacking some necessary parameters. I has worked in the past….so it’ll take some investigation to see what’s broken, exactly.

[ADDITION: 2.26.2010]

OK…so it turns out not to be a Kaltura problem at all. The problem was the WYSIWYG editor in Wordpress itself. When Kaltura inserts the video player tag into the blog entry editor, it does so in raw HTML mode, and it does so correctly.  Problem is that if you happen to switch back to WYSIWYG, the editor clobbers the video tag Kaltura inserted, clearing out a bunch of important tag attributes. Mystery solved. Kaltura is not at fault!

Bookmark to: