January 16, 2009

Streaming Video Playback Speed Controls - Two Innovative Methods

One of the coolest playback features for online video, especially academic video, is a player with the ability to speed up (or slow down) the playback speed of a streaming video.  Way back in the early 2000's there was a tool called Enounce that acted as a plugin to RealPlayer or Windows Media Player and would add a control slider to the player.  Everything from half-speed to 5x playback, with no pitch change on the audio.  It was very effective for watching lectures or news content - for much material, you can really absorb it much faster than it's spoken.  Turns out that Enounce is still available, and works pretty well, and they announced a version called MySpeed which supports embedded Flash video.   

End-users can buy and install Enounce and use it on their systems.  It's a native Windows-only application and must be installed individually on each system.

OK, that's great, but I want this as a feature of my website - I want all my Flash videos to appear with a speed control for all users.  To date, I'd been unable to find any way to do this - no one I've spoken with seems to know how to write code for Flash Player that will permit a speed control.  I'm told it's currently not possible.  

Then I came upon Bloggingheads.tv.  Bloggingheads.tv includes a Flash-based player (derived from the JW Media Player 3.2)  that has a "1.4x" button that bumps up the playback speed -- perfectly intelligible, but much quicker playback for taking in a long talk in a jiffy.  They did the impossible!

I had to know how they did it, so I did some poking around. Turns out they didn't do the impossible, they did an end-run around it.  The playlist that their flash player reads for each video program references two media files.  Here's the relevant code snippet from the XSPF-format playlist:

<location>
    rtmp://mirror-image.bloggingheads.tv/bloggingheads/flash
</location>
<identifier>bhtv-2009-01-13-pb-jg-100x.flv</identifier>
<meta rel="alternate">
   rtmp://mirror-image.bloggingheads.tv/bloggingheads/flash/bhtv-2009-01-13-pb-jg-140x.flv
</meta>

So, they created an alternate encoding of each video, one with the 1.4x timeline baked right in.  The player needed some modification to play this, but only so that the time, duration, and the location bar all showed an appropriately scaled value as this video played. After all, a 30 minute video encoded to play at 1.4x is actually only a 21 minute file, but the timeline still needs to show it like it's the 30 minute length of the original content.

When you switch from one speed to another while playing, the stream rebuffers and seeks to the same spot in the video, so there's just a momentary pause in playback switching from one stream to another.

It's a great workaround - although for my purposes (user-generated content, thousands of contributors) I'd still prefer a player-based way to do it so it can apply equally to video from all sources without requiring added backed processing.  Still...this is the only solution I've ever seen to this issue a) for Flash video, and b) not requiring an additional plugin.

September 11, 2008

Tools for testing streaming media

So here's a neat trick for testing how streaming media and web applications perform for users with limited bandwidth connections.  OK...I'm getting some rolling eyes at the premise. "Does anyone have limited bandwidth connections anymore?"  Turns out that, "Yup...they do."  

Two cases in point:  
  • Last fall, the Harvard Alumni Association offered an all-online version of the popular undergraduate course, "Justice" to Harvard alumni and their invited guests.  Over 3000 participants signed up for the term-length 24-lecture course, delivered via Flash streaming (rtmp) video encoded for broadband (~400kbps). Most common technical complaint from users: video performance and rebuffering due to insufficient bandwidth. 

  • So, I've got one of those Verizon Wireless Cellular Modems, a little USB one. Great 500kbps+ broadband in the city, but when I go on one of my frequent trips to coastal Maine, I'm lucky if I get 80kbps.  Actually, I really am lucky, as the alternative is 56k dial-up.  Wide swaths of the geography are not covered by cable or DSL, and cellular is the best I can do.  People have to drive into town to get Wifi at the local cafe, or suffer with a slow connection from home.
So, even among a net-savvy demographic of people who otherwise have resources, there remains a small but significant need for low-bitrate video solutions.  Currently, we're encoding a new set of programs for multiple bitrates all the way from HD to dial-up, and testing has been an issue.  Two techniques have been lifesavers.

For Windows, Netlimiter, an inexpensive bandwidth simulator utility I wrote about on streamingmedia.com (and commented on here) a while back, lets me set my throughput to whatever I like. I can pretend I'm on my coastal-Maine cell-modem, or a dialup, or anything else easily.

For the Mac, it's already built in to the OS's Unix roots.  It's in the ipfw command.  You set up the bandwidth limits by creating filters with bandwidth limits, then associating those filters with the ports you want limited.  Here's how to set up a bandwidth limiter for testing rtmp Flash streaming (port 1935). Note that if you're not logged in as root, you will need to use sudo to run these:

sudo ipfw pipe 1 config bw 400kbps
sudo ipfw add 10 pipe 1 tcp from any to me 1935
sudo ipfw add 11 pipe 1 tcp from any 1935 to me

Change it at will by issuing the pipe command again...

sudo ipfw pipe 1 config bw 1400kbps

You can also introduce simulated network latency, control outbound bandwidth separately from inbound, and control bandwidth to or from a single IP address or subnet.  There's great documentation at Luigi Rizzo's Dummynet site.  Thanks also to Ask Bjorn Hansen for his mini-tutorial on this.

December 03, 2007

Video Transcript Browsing Interface

CNN has presented a unique and powerful UI for viewing and navigating the video of one of the recent Presidential debates.  Aside from having done a great job presenting the transcript alongside the video (with appropriate click-to-play-from-here functionality), as well as a table-of-contents by topic; CNN has created a unique "map" of the debate, allowing a user to single out a moment, a particular speaker, or the results of a search by spoken word in a brilliant, graphical display.  

What's also interesting is the implementation:  a client-side Flash applet handles the whole thing by reading a single XML file that contains the entire contents of the debate in text form.  

It's one of the finest examples of this kind of thing that I've seen.  I'd love to know if anyone has thoughts about other situations in which this kind of interface could really add value.  The cost would be an issue - transcripts are expensive, as is massaging a transcript into the descriptive XML required for this tool.  Automation using tools like the Virage VideoLogger and Pictron's Audio Gateway can identify speakers and generate text from speech - the accuracy would certainly be far less than what CNN has done here, but for some purposes, would it be "good enough"?

June 26, 2007

The New RealPlayer 11 - A First Look

The new, "web-video-download-center" version of RealPlayer (Is RealPlayer Going to Make a Comeback?) is out this morning. [Download RealPlayer 11]  Some first impressions:
  • Download: A 13MB download, but no questions asked and no prompting for personal info
  • Installation: At about 80 seconds on my MacBook Pro (WinXP under Parallels), still not zippy, but it asked just a few simple questions about configuration on a single screen. Much simpler than prior versions.
  • Loading: Loads up quickly - much more quickly than older RealPlayers.  Feels fast.  The UI still includes tabs for managing your media library, looking at the RealGuide, etc.  But interestingly, the RealGuide, rather than be full of links to teenybopper pop stars, is full of links to top content on YouTube, Metacafe, and the like. Screenshot of RealPlayer download button on Brightcove page
  • Downloading video from the Web: Of course, the feature everyone's talking about, RP11 adds a "Download This" link to videos that appear on any Web site with video.  Shown is a screenshot showing the button added to a Brightcove page:
  • Clicking the download button gets you a download manager that resembles Firefox's download manager. 
  • Videos go into your video library in their native format -- no conversions or obfuscating file names -- it's just there in plain view in the home directory (\My Documents\My Videos\RealPlayer Downloads by default)
One interesting thing about RP11 is what it will not download:
  • RTMP Streaming Flash video (from a Flash Media Server or equivalent) will not download.  Only HTTP-delivered FLV will work. 
  • DRM-protected video will not download.
The built-in "Share This" button on RealPlayer sends a link to the original video content.

All in all, I found that RP11 downloads video successfully from a wide variety of sites, including YouTube, Metacafe, Brightcove, CNN and others.  My first "cannot record this" hit came from Harvard@Home's Human Systems Explorer site, which uses true Flash rtmp streaming from Akamai.

Not a bad first impression. Some things need further exploration.  For example, I haven't yet found out how to hide, when the player first opens, the window that shows the RealGuide, the Library, and other stuff that's not "just the video, please".  Maybe this fits with Real's apparent strategy to make the standalone player into a good media download/management center and leave the embedded player for "video only" uses.  I'll post more info as I learn more.

June 21, 2007

Is RealPlayer going to make a comeback?

Real Networks' newest RealPlayer player appears to be a huge departure from their earlier client-side products.  While the Helix server technology and the RealVideo codecs have been ones-to-beat in streaming media technology, the RealPlayer has been the face of the company to the user community - and it hasn't always been a pretty face.  Real's marketing folks, in the heat of their battle for survival with Microsoft, saw RealPlayer as a the company's direct pipeline to users' pocketbooks.  

Let's face it - the RealPlayer, despite its technological excellence (SMIL 1.0 & 2.0, universal format support, the industry's best codecs, and support for nearly every OS and browser out there) became an abomination - big heavy download, cumbersome registration required, ads and eye candy all over the place, "notifications" that pop up and annoy with marketing messages.

Fast-forward to today -- In the new world of the Web, Flash is taking over because its player is everywhere and its user experience is simple, unmarred by distractions, and an easy download in the unlikely case you need it.  I've even been able to install the Flash player using Firefox's XPI Flash installer - no UI whatsoever, just one click and it's in.

So with many of Real's remaining customers (there are many, especially in the higher-education industry) avidly looking for alternatives to RealPlayer, and Real rapidly approaching irrelevance in the video technology space, RealNetworks has come up with a new approach. Real's new player (RealPlayer 11) boasts two major innovations:
  1. The player is small and unobtrusive, with a quick, easy install that asks no questions, takes over nothing, and generally leaves you alone.
  2. In what could be a stunning new capability, RP11 will download non-DRM-protected video from any website, in any format (Flash. Real, QuickTime, WindowsMedia, etc).  While you're watching that video on YouTube, Google, Metacafe, Brightcove, or anywhere else, RP11 will add a little "save this" button to the video itself. 
The idea is that RealPlayer becomes the base of your personal video library. You can share (by sending around links to the original source), or with a $30 upgrade, burn to DVD disc. Presumably, one of these options will let you easily flip content to your iPod.  There's a pretty good video demo given by Real VP Jeff Chasen at Scobleizer.com.

Dan Rayburn at the Business of Online Video blog wonders what's the business advantage to Real?

Now aside from the obvious idea that content owners may revolt at the idea of people being able to save their content whether they want them to or not, I just don't see the value to RealNetworks in a new player. Why offer it?  

And I think the bigger question is, do we really need more players in the industry? Isn't it already hard enough for consumers? How many more players and plugins are we going to try and force viewers to have to download?

He's not alone.  Real's CEO Rob Glaser makes his case in his post The World Isn't Flat, and responds directly to Rafat Ali's "Open Questions to Rob Glaser" in his own RealNetworks Blog post.

The new RealPlayer gives the users lots of control over Internet video -- watching it offline, burning it to CD or DVDs, storing it in a library, etc. Sharing content links directly from the RealPlayer library can be really useful. A number of people who’ve tested the pre-beta have told me that they love watching a few seconds of a video on a web site, then using RealPlayer to download a copy for later viewing.

My take?  Looking at the education industry, up to now I see a large investment in Real's technology that's been feeling more and more like a liability, strictly because of the horrendous RealPlayers of the RealONE/RP10 generation.  Folks are looking at costly switches to Flash video infrastructure not because the video or server technology is so great, but mainly because the player has mindshare and doesn't do anything to piss-off its users.  

So, if Real's new player is something that a) is a no-brainer to install and use; and b) provides truly useful functionality on top of the enormous-and-growing world of online video content, it may just become relevant again to online users. And that's good for Real's existing customers, for sure.  How that helps Real acquire new paying customers isn't clear to me, but I'd guess that anything that makes RealPlayer more relevant in the marketplace has to be a good first step.

May 01, 2007

Is Amazon's S3 the cheapest streaming video hosting out there?


[EDIT Feb 26, 2010] There's a new how-to article on this topic. How to Get Started with Cloudfront Streaming is the example page for the article How to Get Started with Cloudfront Streaming[/EDIT]

While researching CDNs for storage and delivery of digital video, I found that at least one major user-generated video website provider is using Amazon's S3 service for delivery of Flash video over HTTP. For http delivery of lots of clips on-demand, S3 is apparently doing the job.  The costs are astonishingly low:
  • $0.15/GB stored per month
  • $0.18/GB delivered (or less)
A look at Dan Rayburn's recent blog post listing streaming CDN vendors shows some of the streaming heavyweights, with broad networks of origin and edge servers optimized for real-time media delivery.  Akamai, for example, pre-caches media content close to the network's edges, making videos load quicker. For a site that's advertising-funded, fast loading pages can lead to more page views, which equals more revenue. 

But, S3 is an option I hadn't thought of for online video.  There's no support for the RTMP or RTSP streaming protocols, but many sites are just fine with HTTP download delivery these days. (Streaming vs. Downloading - What's the Difference?)  While it may not be optimized around realtime delivery, it certainly offers unlimited scalability at rock-bottom prices.  And options like its rich developer API and BitTorrent integration could be an asset to a comprehensive media delivery strategy.  It's certainly an option worth looking at.    

April 13, 2007

Image, Audio & Video Search - Reading Content and Context

In his article, Improving Image Search, Harvard's Michael Hemment writes about a research project at UC San Diego that uses human-generated sample data to train an engine that analyses images to extract searchable metadata. 

 Supervised Multiclass Labeling (SML), automatically analyses the content of images, compares it to various “learned” objects and classes, and then assigns searchable labels or keywords to the images. SML can also be used to identify content and generate keywords for different parts of the same image.

This is an interesting topic. I'm reminded of several related topics -- all involved in extracting useful metadata from binary media objects :
  • The Music Genome Project and their Pandora site. Uses human-generated metadata to describe the music, but using fields very similar in concept to the data in VIA or the seed data used in SML. 
  • Using OCR tools to identify and index text that appears in an image. Google's Orcopus project is an open-source way to do this, although commercial products like Pictron do it for images and video. 
  • Speech-recognition on audio/video content is similarly a way to try to index the otherwise opaque contents of a binary media file. What's odd is how little use this has gotten in the real world, even though the technology has been around for quite some years.

    I read somewhere on the web recently, (can't recall the source) the correct observation that hugely popular video sites like YouTube are built on making video findable by using very primitive metadata combined with the all-important context. Who else likes this? What else has this person created/bookmarked/shared? What comments and tags have users applied? All have turned out to be far more useful than a full transcript or speech-recognition search. 
One burning question for me is, why is searching inside a PDF massively useful, but searching inside a video just doesn't quite hit the mark?  What's holding video or image searching back?  Is it the quality of the metadata we extract and index?  Does video simply contain less information density (in its transcript) than a written article (i.e. have you ever read the transcript of a half-hour program, only to realize that you can read/skim it in less than 3 minutes?)? Or do people simply use these kinds of assets differently than they do text-based documents, so different rules and benefits apply when searching?  


April 06, 2007

Online Video and Web 2.0 - What's missing?

Dan Rayburn points out in his Business of Online Video blog that streaming video isn't a Web 2.0 technology.  But while Dan's point is that streaming video has been around way too long to be considered part of the Web 2.0 "fad", I think the relationship between video and Web 2.0 is more complicated than that.  

The key ingredient of "Web 2.0" technologies that makes them worthy of that label is that they have open APIs and are freeform platforms that allow user behavior to define and create value.  Harvard Business School professor Andrew McAfee says it well...

...the use of technology platforms that are initially freeform (meaning that they don't specify up front roles, identities, workflows, or interdependencies) and eventually emergent (meaning that they come over time to contain patterns and structure that can be exploited by their members).  Email is a channel, not a platform; groupware is not freeform and typically not emergent; and knowledge management systems were essentially the opposite of freeform --  they presupposed the structure of the knowledge they were meant to capture. 

...so, to build a Web 2.0 service, Andy says, 
  • Build platforms, not channels
  • Make sure they're initially freeform
  • Build in mechanisms for emergence.  These mechanisms include links, tags, powerful search...
...and, I'd add, simple APIs for combining and syndicating content from one site to another.  Sites like YouTube are on the edge of Web 2.0 because of the ease with which users can publish their content not just to YouTube, but to other sites.  Web 2.0 facilitates video mashups: videos can be embedded across sites, search results can be published as RSS, users can "mash-up" collections of video with photos from Flickr and maps from Google or Yahoo.

But, Dan's right - video isn't really Web 2.0 enough, yet.  As Microsoft's Jon Udell points out

The kinds of standard affordances that we take for granted on the textual web — select, copy, reorganize, link, paste — are missing in action on the audio-visual web. The lack of such affordances in our current crop of (mostly) proprietary media players suggests that open source and open standards can help move things along. But nobody in the open world or in the proprietary world has really figured out what those affordances need to be in the first place.

Standard ways to search within video, associate a video timeline with other media, and deep-link into video content simply don't exist.  RealPlayer and WindowsMedia always did offer a way to deep link using start parameters in the .ram or .asx file URLs, but the endless variety of custom Flash video players (since there isn't really an official, usable "standard" one) means that even that simple method is no longer available on most sites.  And as for search -- while web search engines crawling into a Word document or a PDF file is routine, video content search hasn't caught on, even though the technology, from (the defunct) Virage, Streamsage (now part of Comcast) , Pictron, Podzinger, and others, has been around for years.  

So, Online Video 2.0 is yet to be born - while video is a part of the Web 2.0 ecosystem that generates value from unscripted user behavior on freeform platforms, it's not yet ready to BE one of those freeform platforms.

April 04, 2007

Fundamentals of Website Development - Course Resources

Last night I had the privilege of giving a guest lecture on streaming video in Dave Heitmeyer's Harvard course, Fundamentals of Web Development.  As a follow-up, here are some further information and references on topics that came up in the class.
Anything I forgot..?  Leave me a comment and I'll update this as needed...

March 27, 2007

A Full-Featured Flash Video Player

Flash video is great for users, but the player lacks easy, built-in features Web developers have come to expect.  But now, Jeroen Wijering has developed the full-featured Flash Video Player 3.6 which finally makes all the features of a "real" video player available to Web developers using Flash video on their sites.

The standard video players - RealPlayer, Quicktime and WindowsMedia - all have APIs that help make it easy to embed interactive video onto a Web page.  The major video platforms provided simple run-time customization capabilities that developers have come to expect from video platforms.  By setting values in either the web page or the metafile (.ram, .asx, .qtl), you could accomplish a lot:
  • support for metafiles that can be generated on-the-fly
  • playlists
  • background colors and logos
  • captioning
  • control over the appearance of the player controls
  • fullscreen mode
  • autostart and repeat behavior
Naturally, if you are a Flash developer, you can make a player that handles all of this.  Indeed, unless you're simply hard-coding an .flv URL into the stock Flash video player, you have to do Flash development to make a more capable player.  Jeroen's Flash Video Player 3.6 solves all that.  With an elegant API that works through metafiles or FlashVars, you can customize the playback experience without having to do a lick of Flash development.  What's more, a full Javascript API includes controls (playlist navigation, play/pause, scrub and seek, volume control, and movie loading), Javascript callbacks, and metadata extraction.

This player covers all the important bases in terms of the video player capabilities Web developers need, and makes publishing Flash video as easy as publishing Real, QT or Windows Media.  It's distributed under a Creative Commons License, free for non-commercial use, and nearly free for commercial use.

February 16, 2007

Online Video Industry Index

The folks at Read/WriteWeb have put together a terrific Online Video Industry Index that provides a snapshot of the current online video marketplace.  It's not about hardware or software vendors, like Sorenson, Accordent, or Adobe, but rather sums up the online services space very well.  This list breaks the industry down into categories that include Video Sharing, Video Search, Video Editing & Creation, Video eCommerce, Video Streaming, and others.  

I've been working in this space a lot lately, and have worked with many of the providers on this list, including some as-yet little-known ones.  While the authors disclaim the index as "not complete", to my eye it looks pretty comprehensive.   One useful addition would be Sorenson Media, which is getting into the space with its Squishnet video management service.

This is a great resource. In particular, many of the vendors in this index, as well as being consumer-oriented destinations for video, offer enterprise services and developer APIs that let companies and educational institutions build custom, branded video services on hosted infrastructure. Because of this, the consumer video Web is relevant far beyond the entertainment focus that's driving its growth.

January 17, 2007

Flash streaming encoding settings - what are sites like YouTube doing? (and how can you find out?)

I'm trying to do something simple - find out the codec and bitrate information for Flash FLV videos found on the Web.  In RealPlayer, the "Statistics" menu option reveals everything you need to know, and shows even more if you launch it in "Debug" mode (holding the crtl-shift keys as you select 'Statistics" from the menu).  WindowsMedia provides similar info if you select "Properties" while it's playing. But Flash?  You're out of luck unless you do some extra work.  

My solution may not be the slickest one, but it's one I could work out in a few minutes of Googling around.  You need two things:
  1. First, a tool to download the FLV file to your local system so you can open it with a program that will expose the technical details you want.
  2. A program that analyzes the clip and outputs the codec and bitrate details of the file. 
Stumbling around, I found two options for downloading:
  • The Firefox extension Video Downloader.  This grabs embedded media from a page and saves it locally.  For video, it only works if the video is delivered via http download (not streaming). 
  • A Windows tool called WM Recorder 11.  This one hooks into your TCP/IP stack to automatically record any streaming content coming to your computer, including rtsp, mms, and rtmp streams. As a stream recorder, it's a very cool new option in addition to the old standby I wrote about in 2003, Cucusoft's StreamDown [buy].    
For examining the contents of a file, Moyea FLV Converter [buy] seems to be the best one of those I tried for extracting all the information from files encoded with both the H.263/Spark and On2 VP6 codecs. 

Some interesting findings?
  • YouTube, Metacafe, and Google are all doing Flash 7 - Sorenson Spark via http download.  Spark, based on H.263 isn't so great by today's standards, which explains why video on these sites often looks so bad. On the other hand it is compatible with Flash 7, making the installed base of players close to 100%.   It can also be encoded from a command-line (or from website code) using free or open-source tools like ffmpeg and Riva. Some video/audio bitrates I sampled are:
    • YouTube - 240kbps/64kbps
    • Google - 282kbps/64kbps
    • Metacafe - 330kbps/48kbps
  • Brightcove is rtmp streaming using the excellent Flash 8 VP6 codec, bitrates range from 308k/64k to 640k/96k.
  • Feedroom (responsible for video news sites like New York Times and USAToday is using rtmp streaming, but it's using another level of indirection in its site that makes its streams  not capturable by WMRecorder.


Disclosure: Some of the links above are affiliate links. Any of the pennies that trickle in from purchases of these products from these links will help support the ongoing costs of maintaining this site.

  

January 12, 2007

Video encoding tools, codec comparison

Which Web video format is the best? Which encoding tool is the best one?  Which tools handle high-action video best?  If part of your job involves encoding digital video for Web delivery, you must read streamingmedia.com's research reports on codecs and encoders.  Some of the key points are summarized in  Jan Ozer's article, Choosing a Codec. Some highlights for me:
  • RealVideo is the best overall codec of all the tested choices, and became the benchmark against the others were compared.
  • Differerent codecs were best at handling each combination of encoding bitrate and content type (e.g. talking head vs. sports video) 
  • Some tools encode some formats and content types exceptionally well, while doing a poor job on others.  
  • VBR (variable bitrate) won't always offer an improvement over CBR (constant bitrate), even for high-motion content. 
The two reports, Proprietary Codecs, 2006: Choosing and Using the Optimal Video Codec and  Flash Codecs, 2006: Choosing and Using the Optimal Flash Codec are, unfortunately, not free,  but they are treasure-troves of information about codecs and encoding software.  I recently bought a copy of each, and plan to site-license them for all the folks involved in encoding digital media at Harvard University, where I'm the University's Multimedia Technology Architect.  

Although I've been a contributing editor to streamingmedia.com and have friends there, I have no financial interest in these reports. I just think there's awfully helpful!

January 04, 2007

User-Generated Media - Challenges & Solutions for Business and Academia

Social networking and user-generated content (UGC) sites present unique technical challenges, which lead to unique business challenges.  While unexpected growth is a potential problem for any online site, it is both the holy grail and (in the spirit of "be careful what you wish for") a ticking time bomb for social networking sites. 

A new whitepaper from Akamai (also available free from streamingmedia.com) goes into some depth about the special factors that affect social networking sites.  Some highlights:
  • User-generated content sites are the fastest-growing category of web site (by unique visitors) on the Net, showing, in some cases, triple digit year-over-year growth. Of the ten fastest growing web brands, five are UGC sites (for example, Flickr and Wikipedia). 
  • Social networking/UGC sites have, by definition, unpredictable storage and bandwidth needs, making technical infrastructure (and therefore, budget and capital expense) planning a crap shoot.  Outsourced capacity on-demand is an important option to consider before you're faced with site-crippling runaway success. 
  • Success is tied closely to having a fast innovation cycle -- try stuff out, see how it works for your users.  Continually sense-and-respond to user needs to find that sweet spot of simplicity, functionality, and sustainability that makes your site sticky and social.  One way to do this is to minimize the time and effort you put into infrastructure build-out and put it into more creative endeavors. 
  • If you're an ad-driven site, performance is directly tied to revenue, as faster loading pages keep eyeballs on the site, lead to more page views per user, and therefore register more ad impressions.  When Friendster moved to Akamai's delivery network in March 2006, they saw an immediate 33% decrease in page load times, and a threefold uptick in page views.
Even for an educational institution, outsourcing certain infrastructure is appealing.  With service-oriented Web APIs, it can be easier now to work with a vendor/partner than it is to build it myself.  If I want to put up a quick video recording/encoding/sharing service for my users, I can:
  • Build it myself - not always a bad idea, and definitely a quick-and-dirty solution for a pilot or proof-of-concept, provided I have to staff and the time to move it from P-O-C to production-ready if the need arises.  
  • Acquire and deploy an inexpensive product.  I was surprised to find YouTube clones like Clip'Share and Altrasoft VideoShare for a few hundred bucks or less.  Again - good for a proof-of-concept.  May or may not offer enough for coping with real success.
  • Use a Web Service API like that from Video Egg or JumpCut to handle all the media operations, while you focus just on your website.  These services handle media input (in the case of Video Egg, from webcam and cell phone, as well as file upload). transcoding, online editing and delivery.  It can provide a platform for rapid development of your own custom solutions, as well as a scalable solution in case your solution takes off.  
I'm generally a big fan of institutions building their media solutions in-house, but the combination of the unpredictable needs of user-generated media, the ease and excellence of some of the vendor service-based APIs, and the need to be able to innovate quickly without up-front investment in big infrastructure creates some interesting possibilities.  

The Akamai white paper, Successful Social Networking and User-Generated-Content Applications: What You Need to Know, (which, by the way, I wrote) addresses some other challenges of social and UGC sites -- how edge-caching works with dynamic content, how to control costs when growth is unpredictable, options for exercising editorial control over UGC sites, and some examples of how social networking is being used by businesses to build revenue and create new opportunities.  

November 15, 2006

Is Learning Online Like Watching Football on TV?

The challenge of effective eLearning is finding ways to leverage the medium that simply can't be equaled in solely traditional teaching environments.  Can students learn better from online-instruction than from in-person instruction?  One example pointed out to me was the wildly different experience of listening to a string quartet play live in a real space as compared to listening to the radio or even a CD on a good audio system.  Does the presence, energy, acoustic power, and ambiance of that live performance extend through the electronic realm?  Sort of, but it's just not the same.  Would you willingly deny anyone the option of the in-person experience without good reason?

To me, the alternate example is the professional football game.  Sure, sitting at the top section of a stadium with 85,000 of your closest friends is a social experience with an energy that's hard to beat; but for actually watching a game, nothing beats a TV (even a small one) with instant replay, close-ups of the action, and that bright yellow line that marks the yardage for a first down.  

Which led me to this:  The challenge for eLearning and distance education is to identify the "yellow lines" of the medium -- those things that represent something inherently valuable but simply not possible in the traditional-teaching realm.  Maybe eLearning's real advantage will remain rooted in the fact not that it competes with in-person teaching, but that it allows learning where in-person teaching is not possible or practical.  But I think there's also some "yellow-line" capabilities waiting to be explored, even where educational technology supports (rather than supplants) in-person learning.  

One example of a genuinely new and interesting capability is the digital pen note-taking integration done by Tegrity in their classroom capture system.  I've long been a user of Logitech's digital pen.  The pen allows you to write on special notebook paper, and captures everything you write to your computer as a perfect digital image of the page you wrote.  You can print pages, share them via email, as well as add text and drawing to the page in the computer, making pages indexable and searchable.  What Tegrity has done is to tie the note-taking with the digital pen to the timeline of the video/slides (marketing demo video) captured during a live lecture.  Students who took notes during the class can, at their own PCs, bring up their notes on-screen alongside the lecture video. The lecture video, the instructor's notes, and the student's notes all become part of a synchronized presentation.  Notes can go from being a one-shot chance to get the main points down (sometimes at the expense of really listening) to being a guide to review and further exploration.  I don't know if it will transform teaching and learning, but it struck me as an example of a stunningly clever and useful application of technology to do something that was previously quite impossible.

There's a lot of activity in researching the effect of these technologies.  One interesting study is Lonnie Harvel's dissertation Using Student-Generated Notes as an Interface to a Digital Repository (pdf).  Harvel explores the surprisingly low use of digital repositories in education by experimenting with methods to integrate lectures, student notes, and external resources in deeply integrated ways. 
  

October 11, 2006

Video Editing Online - A Keyframe Extraction Script

Video processing and editing online is becoming a more common occurrence as video sharing and hosting sites are finally catching on.  YouTube, of course, is the one getting all the press this week with Google's $1.6B acquisition.  But other sites have begun offering some very impressive video editing capabilities on the Web.  EyeSpot and JumpCut (owned by Yahoo!) both offer simple, but capable, video editing, including some combination of cuts, remixes, transitions, effects, and audio tracks.  

What's new about this is that it's all done over the Web.  Tools like Apple's iMovie or Final Cut Pro, Adobe Premiere, Avid Liquid or Pinnacle Studio and others are more powerful, sure.  But being able to do this on the Web from any computer at any time, with no software to buy or install, is very cool.

That got me to wondering about the engines behind these sites -- is it all custom code or are there vendors developing and separately selling parts of these solutions?  My initial digging around didn't answer that question, but it led me to one rather simple, but very interesting video manipulation tool called VideoScript.  

Available on Windows and MacOS, VideoScript is a free tool that lets you write simple Basic-like code that manipulates, analyzes, assembles and edits video.  Record time-lapse movies, detect motion in video frames, subtract backgrounds, extract keyframes, blend and composite frames...it's all here and it's suprisingly simple to do.  It's not entirely bug-free - I found that my own first script, which extracts keyframes from a QuickTime movie (based on diff'ing frames and extracting as a JPG any frame that differs more than 25% with its predecessor) and writes an HTML page to look at them, tended to hang the program upon completion.  But it's a neat tool and sheds some light on how folks who aren't Google or Yahoo can do some Web-based video manipulation of their own.

My first VideoScript program, its source code, and its output is here in the extended entry:

Extracting keyframes from movie using VideoScript

Link to original movie on Harvard University's Video Archives



This is a movie assembled from just these keyframes

And here is the relevant part of the source code:
 set frame_count to (length of m) - 1;
 set new_movie to movie;
 set keyIndex to 0;
 set keyFrame to m[0];

 set n to 0;
 set i to 1;
set myHTML to "><html><head><title>Video Keyframe Extraction with VideoScript</title></head><body><br><br>";
    repeat frame_count times increment i
    begin
      set currentFrame to m[i];
      set diff to Math.Difference(currentFrame, keyFrame);
         if (diff > 0.25) then
         begin
  	 set n to n+1;
         append keyFrame to new_movie;
         set file "frame"+n+".jpg" to keyFrame;
	set myHTML to myHTML + "<img src='frame"+n+".jpg' style='border: 4px grey outset;width: 90px; height: 70px;'>\n";          
        set keyIndex to i;
        set keyFrame to m[i];
         end
    end
set file "keyframes.html" to myHTML as text;     
set file "keyframes_only.mov" to new_movie;


October 09, 2006

Apple Fixes Video Playback Bug in iPod

With Apple's firmware release 1.2 for the video iPod (5G), one significant video playback bug has been fixed.  When conducting research for the Akamai whitepaper, Creating & Delivering Podcasts & Other Downloadable Media, I found a critical iPod video bug related to playback of lengthy H.264 files on the 60GB iPod (the 30GB model was not affected).  The problem was that video files larger than about 100MB which were not created with QuickTime Pro (Sorenson, ffmpeg-based encoders, and others all suffered from the problem) would play fine under iTunes on the desktop, but on the iPod they'd play for about 30 seconds, stall for a moment, then continue to play without audio.  
 
With the 1.2 firmware update, files that previously did not play correctly are now working.  Apple's Release Notes for the update are vague (and, apparently, impossible to find again after seeing them once) so it's not clear exactly what else was fixed. I've not done exhaustive testing to see if there are circumstances in which videos will not play, but the specific failure conditions I identified previously have been resolved.

July 24, 2006

Creating & Delivering Podcasts & Other Downloadable Media

Article thumbnailA new white paper from Akamai, Creating & Delivering Podcasts & Other Downloadable Media, tells you (nearly) everything you need to know to create, encode, and publish podcasts.  Not just a basic "how-to" on podcasting, this paper explains the real details of codecs, formats, and gotcha's for delivering audio and video podcasts.  Akamai has deep knowledge of storing and distributing digital media. I was surprised to see how easy it is to replicate your digital media across Akamai's Edge distribution network (my own subdomain, media.learningapi.com, lives on the Akamai network).  Akamai knows that it's good for them to help people do more with digital media.  In turn, white papers like this are good for everyone who wants to know more, whether they are Akamai customers or not. 

The paper includes details on the differences between the different (and incompatible) flavors of MPEG4, the best way to encode  podcasts and vodcasts for the Creative Zen Vision and Sony PSP as well as the ubiquitous iPod, some player bugs (even the iPod has 'em) you need to know about, maintaining RSS feeds, and even creating downloadable media for "alternative" devices like the TiVo and XBox.  The article also includes quick-start directions for using Sorenson Squeeze for all your encoding needs, and of course, how to leverage Akamai's distribution network for handling the storage needs and load that podcasting can generate.

I should note that I wrote this paper.  I think the value-add of articles like this lies in describing the difference between how it's supposed to work, and how it really does work.  As is usually the case, when researching this piece I came across numerous technical procedures and processes that are supposed to be straightforward, but that don't really work as documented (or that simply aren't documented in one place where you need it).  As such, this paper, while brief, reflects the way it really works and will hopefully save some people a bunch of that "figuring it out" time!

July 16, 2006

One-click iTunes Podcast Subscriptions

While building out the podcasting features of the Videotools Media Content Mangement system for Harvard Business School, we were trying to figure the simplest way to do a "one-click" RSS subscription for iTunes users.  Before we explored it, we had some open questions: do you have to register you feed with the iTunes podcast directory for it to work?  Will it work without complicated client-side Javascript?  With all the "iTunes U" university stuff out there ( proprietary as it is), will a totally standards-based RSS feed work?

Turns out that it's easier than we expected, although surprisingly, there were few documented examples on the Web showing how simple it really is.  

To make a one-click iTunes subscription link, just link the RSS feed from your Website using a URL with the protocol prefix itpc://   So, a one-click iTunes subscription link to a podcast would look like this:

itpc://www.learningapi.com/rss/podcast.xml

That's all there is to it.   No registration required, no Javascript, nothing special.  This will not enter your feed into the iTunes Music Store's directory, so you won't get rankings, etc from iTunes. For our purposes, which are really intranet-oriented podcasts, we don't want publicity beyond our own user population, so that's a bonus. When a user who has iTunes installed clicks this link, it will automatically subscribe them to the RSS feed as an iTunes podcast.  While it was tempting to explore many of the other RSS one-click options (noticing the Odeo and PodNova options on my local NPR station), we determined that for our users, offering a one-click for  iTunes along with a plain RSS link for manual copy/paste was the sweet spot of user choice and simplicity.  

One other nice touch that's become common for making RSS feeds more friendly to new users - if you click on the RSS feed link (for either podcast or simple RSS news  feeds), it's styled in the browser using XSL, so that it's human-readable, with some helpful instructions for what to do next. 


June 13, 2006

Podcasting and MPEG4 video -- the PSP problem

My prior post on the travails of podcasting, MPEG4, and supporting multiple devices detailed the differences between MPEG4 as it's supported by three of the most popular portable digital media devices: the iPod, the Creative Zen Vision, and the Sony PSP.  After further exploration, I've learned a few new things.

MPEG4 Woes
The MPEG4 format supported by the PSP is structurally the same as the iPod format: MPEG4 H.264 (AVC) w/ AAC audio.  For some reason that escapes me, Sony employs a customized header within the file that makes it look different -- and incompatible.  To get an iPod-format MPEG4 to play on a PSP, you have to either convert the file using software like the free PSP Video 9 or Sony's own PSP Media Manager; or use a utility that is supposed to flip the header bits: AtomChanger.  I didn't have any luck making AtomChanger work, but truthfully, I didn't spend a lot of time working at it.  
  
Podcasting
For podcasts, PSP Media Manager software is excellent and makes it easy for the user.  Although Sony sells it separately, it should be included with the device, in my opinion.  It handles RSS subscriptions, automatically does any file format conversions necessary for the PSP, also manages photos, music and games on the device, and makes the process seamless for the user.  Alternatively, PSP Video 9 combined with Videora provides a no-cost, although less seamless, solution for podcasting and file conversions.

For those of us producing podcast content for these devices,  I think the best answer is still to encode for the iPod.  Audio, of course, should be MP3 - then you support everyone. For video, software like PSP Video 9 and PSP Media Manager mean that PSP users can use the same media that iPod users can.  But if you're looking to deliver video content directly online to PSP Web surfers (see the next paragraph), you'll need to provide MPEG4  files in the PSP format.

PSP for Browsing the Web

The Sony PSP is a fine wireless Web device in its own right.  It took me about ten minutes to get it up and running on my home WiFi network (802.11g), complete with WEP authentication.  The internal Web browser is adequate, although the way you "type" text  (a URL, for example) on the device is clunky.  You can surf the Web, and download audio and video files directly to the device.

April 05, 2006

MPEG4 format madness - iPod and Creative Zen Vision and Sony PSP

In the process of  trying to figure out video podcasting (vodcasting) for  the most popular portable devices (ipods, Creative Zen Vision, and Sony PSP), I've discovered that MPEG4 as it's supported in the real world is a mess of incompatibilities.  In my streamingmedia.com article on interactive MPEG4 authoring a few years ago, I noted that interactivity in the MPEG4 spec was not really implemented by  any major vendor.   Now I see that almosr three years later, even the basic video codec profiles are not  supported in a meaningful way.  

I've been trying to find the codec/format combination that will work on an iPod, a Creative Zen Vision, and a Sony PSP.  All of these purport to support MPEG4.
  • iPod handles MPEG4 SP and AVC in an .mp4 container
  • Creative Zen Vision handles  MPEG4 SP, but only in an AVI container. Leading tools like Sorenson don't even bother to support this archaic variant.  I might as well use the WM format's better compression and drop MPEG4 from the process. 
  • Sony PSP - I've just ordered one for testing, but in advance of its arrival, I've already noticed that the .mp4 AVC files that Google Video makes available for the PSP are different than the .mp4 AVC files they make available for the iPod.  And tests reveal that the PSP-targeted file will not play on an iPod.  
I guess I  thought there'd be more support for common standards in 2006 than there was in 2003.  When I'm done testing, I'll post the recommendations on what codecs/formats will work on most of the popular devices.  Seems that no one has really tried 'em all (and written about it) from the content publisher's perspective - "what's the simplest process for supporting the most users?"  

March 27, 2006

iPod Bug - Video Plays on 30GB iPod but not 60GB

If you publish video destined for the iPod, you should know that there's a problem with the iPod 60GB Firmware Version 1.1 that affects playback of some H.264 videos.  I found this myself when I had encoded a SageTV-recorded episode of Battlestar Galactica.  My son popped it over to his 30GB iPod where it worked fine.  I moved a copy to my 60GB iPod, and found that it would play for about 30seconds, and then begin to skip and drop audio.    

Anyway, a little sniffing around reveals that very large H.264 files (over 100MB) that are encoded with some third-party MPEG encoders (the problem does not affect encodes using Quicktime Pro).  A discussion on FEWL.net summarizes the condition:
  • This problem only seems to affect the 5G 60Gb iPod
  • There have been no reports of this problem occurring on the 5G 30Gb iPod
  • The problem does not occur with content purchased from iTunes
  • The problem does not occur with content encoded with QuickTime and/or iTunes.
  • The problem only seems to occur with large video files (in excess of 100Mb)
  • The problem seems to occur most frequently with content encoded with Videora on PC, although other applications have shown this problem as well.
  • There have been few to no reports of this problem occurring for Mac users encoding with Handbrake.
  • Turns out that both Videora and the utility I used to convert the SageTV MPEG2 file to an H.264 are based on the open-source ffmpeg encoder.  Other encoders may be affected as well, so be sure to test your longer-format video on the 60GB iPod/Firmware 1.1.  Apparently, one workaround is to open an affected file with Quicktime Pro and re-export it as H.264.  

    February 06, 2006

    User-Driven Innovation in Television - the creative ecosystem around SageTV

    Want to slip TV programs over to your iPod (or other portable media viewer) automatically?  Read on...

    PC Magazine last month published a feature called TV Transformed - Watch Anytime, Anywhere, on Any Device.  It's a great piece on the options now available for digital distribution and consumption of TV and video content.  One solution they didn't cover in their article is called  SageTV.  In the process of getting ready to buy a new home computer for the family, I'd done some research on Windows Media Center Edition and found the presence of DRM restrictions on recorded content to be unnecessary and unacceptable.  

    I ended up deciding on Sage TV, bundled with the Haupaugge PVR350 video tuner card.  SageTV is like Tivo, but runs on your computer.  It's got all the usual Personal Video Recorder (PVR) features, like interactive program guide, recording of individual shows or whole seasons, recording things it think you might like, and pause/instant replay of live TV.   The hardware includes a remote control and audio/video outputs that let you use your computer like a TV and your TV like a computer - but that's just the tip of the iceberg. 

    The delightful thing about SageTV is that it's architected to be a platform for user innovation.  It comes with a set of  published APIs for everything from controlling it via command-line scripts to full Java and native C/C# APIs for customizing the system or writing your own applications.  A wide range of tools and utilities have sprung up around SageTV as users leverage the power an open platform gives them. The development community has a wiki and busy discussion forums where users and developers share ideas, code, and tips.

    For example, Geoff Gerhardt at the InveterateDIY Blog has created Sage-To-iPod, a terrific utility that will automatically take your chosen selection of recorded TV programming, convert it to MPEG4/H.264 and sync it to your iPod.  Now you can go to bed early and still get The Daily Show on your iPod in time for the morning commute on the train.  There are other examples: the UI tweaks on the Ruel.net PC-TV page,  or these custom modules to tie in imdb.com movie lookup, RSS feeds, or control SageTV via a web interface.  And of course, anything you record can be burned to DVD.  

    I know there are other options - including the open source MythTV.  On the scale of effort  required to get up and running, MythTV requires more of an investment in time than many people are willing to make. The sweet spot for me is that SageTV combined the ease of a commercial product with the open interfaces and invitation to tinker that makes good software great.

    January 01, 2006

    Converting video - DVD to iPod

    Moving audio around - from CD to PC to MP3 to iPod - is an simple task, with lots of free tools to make it easy.  This week, I needed something to get one of my DVDs over to my iPod, and found one useful tool - Cucusoft DVD to iPod Converter [download link - 5.4MB].
    [Drag the slider to enlarge the screenshot] - [details on this]

    It's a super-easy way to rip a DVD into iPod-compatible MP4, H.264 at up to 768 Kbps or MPEG4 at up to 2.5 Mbps.  Options for various input and output formats let you choose details about the audio downsampling, frame rates, codecs, how aspect ratio is handled, and more.  On my 2.2GHz Pentium 4 laptop, conversion is slightly slower than real-time, and the output looks great.

    Cucusoft's iPod Video Converter [download link - 4.3MB] does the same to-iPod video conversions, but from a variety of sources, including AVI, MPEG, Windows Media, RealVideo, DivX, and others.

    Each product retails for $29.95, but the two are available as a "suite" for $39.95 [download link - iPod Video Converter Suite 7.1MB].  The downloadable trial versions are fully functional, but leave a "Trial Version" watermark on the converted video image.

    I haven't done an exhaustive review of products available in this category, but in a pinch, the Cucusoft toolset was exactly what I needed to get the job done with little ramp-up time and minimal effort.  I recommend giving it a try.

    [Disclosure - the download links on this page are affiliate links. If you register the software by clicking "Upgrade" from one of these downloads, a small payment will go to help support this website.]

    December 19, 2005

    Time-shifting streaming audio - podcasts the hard way

    This week, Jon Udell writes a piece about his efforts to get an archived radio program from the program's web site over to his MP3 player.  Sounds easy enough - that's what podcasting is for.  But in this case, the show, NPR's Fresh Air, has only been made available as streaming audio in Real format.  

    Jon's got a workable solution going with shell scripts, mplayer and lame on his Mac OSX system.  His screencast of making it work is terrific.  I've done the same thing - it was my primary use of my iPod even before podcasting was in broad use.  For me, it was trying to listen to a streaming National Public Radio program in the bustle of the workplace that caused me to want to time-shift the interview--record it, slip it over to my iPod, and let it entertain my evening commute.

    My process was just about as clunky, but it worked: my streamingmedia.com article on Stream Recorders reviews Streamdown - give it the URL to a SMIL, RAM, or ASX file and it will promptly retrieve all the media files to your desktop.  Then I'd use RM To MP3 Converter from Boilsoft to convert to something my iPod can deal with.  And finally, as the third step, use iTunes to sync the file to the iPod.   For a while, I used RealPlayer to manage my files and synchronization with the iPod. It saved steps, since the RM to MP3 conversion would be handled automatically by RealPlayer. But it seems that iTunes and RealPlayer would end up fighting over control of my iPod and I tired of dealing with it. I returned to using iTunes alone, for its predictability (and despite its dreadful performance).

    But Jon's real question in all this was, "am I doing anything wrong" by downloading and recording this audio.  To me, the answer is clear.  Jon's not redistributing the program, he's timeshifting it.  Fair use.  Settled law.  Closed issue.  

    December 15, 2005

    Managing Video Content  - "Like Netflix, Only Better?"

    The Videotools Video Content Management System, which my team developed at Harvard Business School, is a first-place winner of the 9th Annual Process Innovation Award by Kinetic Information.  Videotools was one of six winners in the Innovative Solutions Category (Recognizing Superior Solutions for their Creativity and Effectiveness ).  Specifically, they look for process improvement -- those applications that best exemplify how technology can be used for business benefit.  

    A recent Campus Technology article on Digital Libraries by Matt Villano profiled Videotools, introducing it as "Like Netflix, Only Better."  It's flattering, even if that's a bit of a stretch!  But, Videotools does make an impact on the institution, by providing three services:
    • Managing and automating the encoding, metadata extraction and collection, and publishing of digital video in various (and multiple) formats and bitrates.
    • Managing permissions, roles and collections, and providing users with a video and media portal where they can search, organize, and share video content.
    • Providing delivery management that allows a unique URL for each video clip which applies rulesets to seamlessly determine a user's permission to view a video, detect their network location and preferred format/bitrate/size, and generates a metafile (.ram, .asx, etc) that gets the right video to the user quickly. (i.e. The same URL that  opens a 1.5Mbps RealVideo at full-screen when accessed from a classroom may provide 300kbps Real SureStream or 250kbps Flash video via http when accessed from home.)
    More information about how we designed and built Videotools, along with our  philosophy of how to think about these kinds of projects, can be found in:

    November 16, 2005

    Digital Asset Management - Some Advice

    In preparation for my conference session today at the DAM (Digital Asset Management) Symposium, I was asked to summarize my thoughts on what's most important for people who are implementing a content management system to know.  

    My personal experience in DAM is mostly centered around building systems for managing streaming media content (search, delivery management, metadata extraction, etc) and other kinds of multimedia materials.  Having done several generations of  a video content management system, as well as several multimedia authoring and asset management systems, here are a few points I think are important:
    1. Buy vs. Build --
      1. Realistically, it's not build vs. buy, it's "build" vs. "buy & build".  These implementations take a great deal of analysis of your business problem, copious customization, and require a strong internal team.  You cannot outsource success.
      2. Integration of a vendor solution can take as long as a custom build.  Be sure your vendor's direction and your implementation will let you take advantage of the vendor's upgrade path, otherwise you may have been better off building.
    2. Plan for change - Don't expect to get everything right in the specification stage.  When you define your business problem and its solution, find the right balance between up-front analysis/specification and leaving room for the system to evolve as its users begin using it.  Follow a path-based development model in which you break the big problem into a bunch of small ones and tackle each incrementally; because;
    3. "If a project team can eat more than two pizzas, it's too large."  This week's Baseline Magazine profiles Amazon.com CTO Verner Vogels and his approach to running Amazon's software development operation.  Small problems are easier to grasp, examine, and solve than big ones.  Small solutions are easier to explain, understand, test, and implement.  Small teams need less process, have few communications challenges, and lower overhead than larger ones.  Small teams can get real work done while large ones are still trying to find common understanding about the problem. 
    4. Retain internal development capacity -- in order to have the system evolve, you need to have internal expertise in modifying it.  
    5. Be ruthless about insisting on the use of open, flexible standards and APIs - Using a system based on open interfaces and standards gives you flexibility to create new things you didn't even dream of when you began.  Information "stovepipes" can be OK...as long as there are simple hooks between them.
    More info can be found in the slides from my Gilbane Conference Presentation on content management systems for video and multimedia.

    May 17, 2005

    3D Modeling in Flash

    Apparently, a little basic mathematics is all it takes to create real 3D wireframe rendering in Macromedia Flash.  Oliver Knill of the Harvard University Department of Mathematics has developed a set of animated 3D models for teaching about vectorfields.  

    What's really cool is the use of Flash.  The .FLA file contains only the blank stage and a reference to the ActionScript.  A single ActionScript file contains the entire rendering and interactive behavior (the ability to rotate the model around all three axes using the keyboard) for the model.   What's more, source code for all examples is GPL'd and is freely available on the Flash/Actionscript project for Multivariable calculus site.

    May 09, 2005

    Gilbane 2005 Content Management Conference Presentation:
       Media Content and Delivery Management at HBS: Videotools

    On April 12, I gave the keynote address at the Gilbnane Conference on Content Management in San Francisco.  The session description included "We'll examine actual business and IT planning scenarios and identify the characteristics associated with successful content technology deployment."  

    I previously  mentioned that organic solutions created from individual components that have flexible APIs can be a better approach than the monolithic "ERP" model of Content Management.  More to come on that later, but for now, here are my slides from the presentation: Media Content and Delivery Management at HBS: Videotools [bouthillier_gilbane2005.pdf 1.6MB]. They describe the video content management implementation we did at HBS, and some lessons learned from that project.

    January 26, 2005

    Google Video Search - Video's not opaque anymore!

    Today, Google announced its Video Search feature - the ability to search into the Closed Caption transcript of various TV broadcasts.  My first test search included some of the optional parameters you can include in your search.  A search for "scientific" on "channel: PBS" led me to these results for PBS' Scientific American series -- including links to the series homepage and upcoming scheduled broadcasts.   I couldn't play the video from the search results - there's no asusmption that this content is being made available via streaming by the broadcasters, but it's a great search.  

    I first built a system that does this back in 1998 - the marriage of a video delivery management system with the data from a Virage VideoLogger created what Google has done, but with the ability to work also from speech-recognition of the program's audio track.  Systems like StreamSage and Virage are capable of providing and processing raw data for these kinds of searches.  In a recent project, we're adding Virage-generated data to a system based on Clearstory Systems ActiveMedia back-end, and integrating Jakarta Lucene to create deep contextual search of Closed-Captioning, speech-to-text, character recognition, speaker ID (by voice), and more.  

    Google is moving along its strategy of making knowledge searchable.  Web pages are just a tiny fraction of the knowledge and information (which are not necessarily the same thing) in the world. Search engines - even the "video search" available from Yahoo and others, only searches metadata associated with a video clip, not the video itself. The video is an opaque object to the search indexer.  Other content that was previously opaque to web search - like that from books and library archives (or even shopping catalogs) - is being made searchable by Google's efforts to index the world's knowledge.  Adding video to the mix is a terrific new step in that direction.  

    March 04, 2004

    VCRs for streaming media

    This week's entry on streamingmedia.com - a review of two products for recording streams - VCRs for Streaming.  This may be a controversial topic, as evidenced by RealNetworks'  lawsuit against StreamBox. That was settled out of court in September 2000 when StreamBox backed off and killed its "Ripper" and "VCR" products.  Neither of the companies in the article responded to requests to be interviewed for this article.  Keeping a low profile, I guess.

    Still, if recording off the radio is legal, and my home VCR and PVR are legal, then there's no reason for stream recorders to be anything less.  These two utilities are rough around the edges, but they get the job done. 

    Streaming media is a great way to receive multimedia over the Internet. But it doesn't help you much when you're on a plane, in traffic, or sitting in your ice-fishing hut in Minnesota. There's help available - StreamDown and SDP are utilities that can record streaming media programs, letting you watch or listen at your convenience.


    February 25, 2004

    All the many MPEGs - from MPEG1 to MPEG21

    From streamingmedia.com this week, trying to make some sense out of the many MPEG standards:

    There's more to MPEG than just audio and video compression. There are five MPEG standards–MPEG1, MPEG2, MPEG4, MPEG7, and MPEG21–spanning all aspects of compressing, authoring, identifying, and delivering multimedia. Here's a quick look at each one and where it fits in the digital media landscape.


    February 12, 2004

    DRM turns computers against their owners

    I haven't had my new iPod long enough to have found all the limitations imposed by Apple's Digital Rights/Restrictions Management scheme, except for one - it's a read-only device.  Once a music file's been copied onto it, I can't copy it form the iPod to any other device.  I can only delete it. 

    Hey, I work for a major content-producing entity, so I'm not in favor of people violating legal and reasonable copyright claims.  It's the unreasonable (and those counter to legal precedent) ones I have a problem with.  About DRM in general, my discomfort can be summed up in this quote from Annalee Newitz's great piece in the San Francisco Bay Guardian about DRM and the entertainment industry:

    "DRM turns computers against their owners. I don't want a Disney security guard sitting in my living room watching my every move." -- Ian Clarke


    January 12, 2004

    Open source TV

    Want a TiVo-like box that you can actually control, free of DRM, forced ads, and monthly fees?  As noted by Dan Gillmor:  the open-source Linux-based MythTV PVR project looks awfully cool, if you've got the time and energy to build it. 

    December 17, 2003

    MPEG4 XMT tutorial

    After my own tutorial on using XMT (XMT-O, actually, a SMIL derivative) and BIFS to author interactive MPEG4 videos (along with the accompanying MPEG4 tutorial article), I found this excellent primer on using XMT-A  to create simple MPEG4 scenes.  XMT-A is lower-level than XMT-O - more control and flexibility, but more complex and more code to write, too.

    The author, Jean Le Feuvre, has created a good MPEG4 reference site around the GPAC project - a GNU-licensed implementation that will:

    at term provide a 2D/3D core player, complete MPEG-4 Systems encoders and publishing tools for content distribution.


    October 21, 2003

    Creative Commons Blogs Digital Media Article

    The Creative Commons folks just blogged my CC Licenses for Digital Media (audio and video) article.

    It was also a hoot to see an article in yesterday's Wall Street Journal - "Can Copyright Be Saved?" (subscribers only) that echoes my CC piece.  I'd argue that it's a bit of a watering-down of the truth, though.  When they say "the digital age hasn't changed anything in terms of the rights of artists and entertainment companies to control the distribution of their creations", I beg to differ.  The entertainment companies, by and large, have seen the digital age as the opportunity to vastly expand their control over what used to be consumer rights taken for granted. 

    The challenge is finding a way out of this mess. Efforts fall broadly into two camps. On one side, generally speaking, are those who revel in the freedom that technology has brought to the distribution of creative material, and who believe that copyright law should reflect this newfound freedom.

    On the other side are those who believe that the digital age hasn't changed anything in terms of the rights of artists and entertainment companies to control the distribution of their creations and to be paid for them -- the essence of copyright law. For them, the answer is to leave copyright law intact, and to use technology to make it harder for people to make digital copies.


    September 16, 2003

    Microsoft's WM9 strategy

    The big news last week is that Microsoft has submitted WM9 codec specification to SMPTE for consideration as a standard.  It's all over the press, including my own article on StreamingMedia.com, Microsoft Opens Windows Media 9 Codec to SMPTE.  Stefanie Olsen also writes a great and informative piece on the topic at ZDnet. 

    The surprising addition to the news and perspective came via Good Morning Silicon Valley yesterday:

    HomeTheaterHiFi.com reports that Redmond is plotting an end run around the consumer tech industry that may bring HD-DVD to market within months and in the process make Windows Media 9 video format the standard for HD-DVD media, set-top boxes, and video-editing systems.

    What's interesting here is the way it suggests that Microsoft's submission to SMPTE is not a big change in strategy, but may be what was planned all along.  First, develop technology that offers something of value to the motion picture and broadcast industries; then seed the marketplace with a bit of it here and there (the T2 DVD, Sigma chip designs).  Then, when the market value is demonstrated and there's some product on the shelves, submit it as a standard to "grease the wheels" of further adoption. 

    In the article, Tod DeBie notes that WM9 can get HDTV DVDs and other media into the market sooner and better than standards-body driven alternatives, which are still tied up in process.  If WM9 does it faster and better, and MS has opened the technology to SMPTE, then industry adoption just might take off. 

    Finally, Tod says this:

    There are many possibilities for Microsoft here, and considering their obvious commitment to video quality, the results are bound to be good for the consumer.

    I'm not certain that Microsoft dominance over its markets is ever particularly good for the consumer in the long run.  For that matter, neither is anyone's total dominance over any market.  WM9 is great technology, but codec technology is a very fickle thing. Competing technologies leapfrog each other regularly, with the "best" codec only remaining "best" for a short while before having its title taken away by the competition. I only hope that the decisions and deals made in the next few months by the consumer electronics and broadcast industries are based on solid long-term business strategy. Whatever they choose, they (and we) will have to live with for a very long time.

    August 31, 2003

    Serious video publishing

    I've been looking at Serious Magic's "Visual Communicator" product for a possible review on streamingmedia.com, where I write a regular column on streaming media technology.  I've only just started playing with the product, but two things strike me right away as notable.  First, the quality of the printed (yes...on actual paper that you can read in the bathroom!) manual is superb.  It covers not only the software product, but also the most basic basics of video production in general, and does so in an effective, lighthearted tone. 

    The second striking thing is this paragraph in the last section, A brief history of broadcast communication:

    Television has forever altered our society, but here at Serious Magic, we don't believe TV has yet reached its full potiential to inform, educate, entertain,.and enrich our lives.  To reach its full potential we believe that the power to make television (or television-like communications) must be in the hands of millions of people (not just a few thousand broadcasters and TV producers).  ...  Perhaps someday Visual Comunicator will be rememberd as a major step forward in democratizing the tools of television. 

    Lofty, yes.  A bit overblown?  Perhaps.  But I like the premise - that the tools of personal publishing will make a difference in video media as it has in text.  I'd love to hear from anyone who's used it.  What did you use it for?  Was it effective?  I'm as interested in the ways its "broadcast" presentation styles are used in the real world as in the technical details of the software. 

    August 28, 2003

    SMIL for professional video production

    The usual method of video production is "destructive", which means that when you're done editing, compositing, "effecting", and synchronizing your media elements, you haven't preserved continuity with  the original source materials.  You've made something new and separate from the original source. What's more, the only descriptive path between the two is, at best, the proprietary EDL format from your editing system of choice. 

    This isn't a huge problem in itself - we're used to that just being the way it is.  But with all the media increasingly available on the network, is there a way to do more editing, compositing, "effecting", and synchronizing by reference rather than destuctively?  What efficiencies will this enable in the creation, delivery and consumer usage of complex media content?

    Well, Sony is driving an effort to examine whether SMIL has a role in professional video production.  From the W3C Note on SMIL 2.0 Extension for Professional Multimedia Authoring:

    Most of the professional content production environment has been replaced with digital systems and networked with each other. Some of the consumer environment has also already been replaced with digital systems for quite a number of regional services. Such change grows expectations that both content production and delivery can be connected seamlessly from professional systems to consumer systems.

    August 05, 2003

    Helix puts (grant) money where its mouth is

    RealNetworks just announced the Helix Community Grant Program.  Real is giving away $75,000 this year in grant money to support research and development in basic digital media technology as well as functional extensions to streaming media delivery systems.  The projects with the best chances will be the ones that fertilize the digital media ecosystem, have open-source appeal, and have some conceivable future commercial application. 

    The open-source Helix Community has already brought in a legion of contributors, many from companies with commercial interests in the Helix platform.  This effort tries to bring more of the best creative energy into the Helix ecosystem by funding the best academic- and research-oriented ideas which might otherwise go undone or unknown.
    In Helix's own words:

    Enhancing our efforts to build the first open multi-format platform for digital media delivery, the Helix Community Grant Program will support the most promising innovations from developers and the global research community.

    The grant program will ensure the Helix DNA platform incorporates cutting edge research advances and has the widest extensibility. This program welcomes ground-breaking research proposals, inventive implementations and creative project ideas from independent developers, the academic and research communities or any non-profit or commercial enterprise engaged in digital media research and development.


    An open way to be closed...

    From Steve Mirsky's lighthearted column in last month's Scientific American...

    True or False:   In April the CIA decided to classify a report on a January session in which microbiologists and the CIA's strategic assessment group discussed scientific openness.
    True. That popping sound was your head exploding.

    What does this have to do with streaming?  Nothing, but it's what came to mind when I read about the OpenIPMP initiative, an
    Open Intellectual Property Management and Protection effort from ObjectLab.  At first it appears to be a contradiction in terms - an open way to lock down content.  But then, what it really is is a way to lock down content that keeps control of the content-protection technology out of the hands of a single vendor.  You might imagine (if you really stretch) a vendor taking advantage of a monopoly position on content distribution to exert inordinate influence in every aspect of the media business.  ObjectLab describes their effort like this:

    Open IPMP offers content owners protection, rights management, and persistent association of certified meta data of their media assets. ... Objectlab does not force-feed its clients proprietary software or confuse the industry with another turn-key DRM solution. Because we adhere to MPEG standards, use an Open Source toolset and deliver finished software and source-code to our client,  the content owners -- not the technologists -- control distribution. And in a networked economy, to control distribution is to control content.

    They appear to be doing all the right things - MPEG4, ISMA, MPEG-REL, etc. I'm curious about who is funding the effort, though.

    July 16, 2003

    Happy Birthday to SMIL!

    The SMIL language just had its 5th birthday!  On June 15, 1998 SMIL 1.0 became a W3C recommendation.  I have fond memories of that, since it was the SMIL tutorial article, Synchronized Multimedia On The Web, that was my first published article.  If you haven't seen the example presentation, What I did Last Summer..., you should take a look some of the neat tricks SMIL (even the old 1.0) makes possible.  But with respect to SMIL's birthday, Philip Hoschka from the W3C pointed out a few notable notes:
    Of these, I think the MPEG4/XMT connection is perhaps the most important for the continued growth and influence of SMIL in the communications industry.

    Some cool SMIL 2.0 examples can be found at the French National Research Institute INRIA's site.  INRIA also is the source of the LimSee2 SMIL authoring tool.  I've been meaning to give that a try...

    July 07, 2003

    Licensing your video content - digital copyright and the Creative Commons

    Addition (October 19, 2003) - I've just posted my new tutorial article entitled Creative Commons Licensing For Digital Media (audio and video) on streamingmedia.com. How-to, and why-to use CC for your audio and video projects.

    These days, just about everyone that creates content is concerned about copyright.  Likewise, content creators and consumers alike know that copyright is as much about allowing use of content as it is about restricting it.  After all, there's not much use in publishing your stuff if no one's allowed to make use of it (RIAA/MPAA, take notice)!  The Creative Commons, a project of Stanford Law School, Harvard's Berkman Center for Internet & Society and others, aims to help.  From the CC website:

    Creative Commons has developed a Web application that helps people dedicate their creative works to the public domain -- or retain their copyright while licensing them as free for certain uses, on certain conditions. ... Creative Commons licenses are not designed for software, but rather for other kinds of creative works: websites, scholarship, music, film, photography, literature, courseware, etc.  [more explanation of licenses]

    While initially developed for Web pages, Creative Commons licenses can now be tagged to MP3 files.  The CC technique not only tags content with an approriate license, but it also provides a way to drive traffic to the content creator's website.  The technique here can easily be applied to video and audio content encoded using RealNetworks, Windows Media, or QuickTime tools.  There's never been a better way to both protect copyright and advertise public access to digital video.  And there's no excuse not to do it. Protect your content, empower your viewers, and contribute to the success of CC...all at once!

    June 16, 2003

    Video blogging - ready for prime time? Depends what 'ready' means...

    After my notes about Google CEO Eric Schmidt's betting that video and multimedia blogging will become Big,  I began looking for examples.  This DailyWireless piece covers the growing number of options for personal publishing of audio and video.  Something clearly is happening here, although I agree with Matt May that it's subject to the golden rule: professional video production requires video professionals.  Part of his summary:
    • People can scan textual content meaningfully. In fact, scanning is the dominant information-gathering mechanism in Web browsing. They couldn't scan in audio or video players if they tried.
    • Text blog reading is done actively, as users scan and read the information that interests them. Audio and video content is inherently passive; therefore, the only value you can provide is active and engaging personality and good quality imagery and sound.
    • ...which you won't provide, because you're not as gregarious or photogenic as you think you are, your lighting sucks, and your bare walls look just like mine. The ability to buy a camera and some software does not help you in this department.
    Both Matt and Eric are right...personal video publishing will continue to grow, but as with any creative or expressive endeavor, having good skills will continue to be important to reaching and keeping an audience.  

    Matt goes on to point out that the key ingredients required for personal video publishing to take off (in the blogging sense) will be a robust ability for cross-linking and the ability to browse and scan video information  the way you can with text.  The technical underpinnings for this are SMIL, SVG, SlideML, and other open, text-based standards.  The right creative approach to content....I haven't seen that yet.