LearningAPI has moved to a new blog!

The learningapi blog has moved to a new URL. These posts will remain here, but all new content has moved to learningAPI.com: Digital Media, Streaming Video & Educational Technology. You may also subscrdibe to the RSS feed for the new learningAPI.com blog.

Optimizing real-time video encoding performance

Steve Mack's Streaming Media Bible proved its worth last weekend at a gig doing real-time high-bandwidth encoding.  The gig was streaming 1024kbps RealVideo9 to a half-dozen sites on the local intranet.  We had a number of high-horsepower encoders running in redundant-mode, serving redundant Helix Universal servers.  It was a nice example of the kind of fault-tolerant enterprise-class architecture you can build with the Helix product line.  I wrote about it in my 2001 article Corporate Streaming with RealSystem iQ (600k pdf available).  

Anyway, the encoding process was CPU-bound.  I was trying for 30fps, 640x480 at 1024kbps, but the encoder CPU was running at 100% and my framerate was still too low.  While examining the settings of my Osprey 220 video capture cards, I found the Color Format pulldown set to its default - RGB 15.  I knew that the selection here affects encoding performance.  Different video codecs employ different color formats, and conversions from one to another suck up CPU.  So the goal is to make the capture format as close to the encoding target format as possible.  It reduces the work the encoder has to do to make it all happen.  

Fortunately, I had Steve's book handy, and looked up the specs on these color formats.  Right there on page 55 I found a table listing the various color formats and corresponding data rates:  At 640x480, RGB24 creates 26.6MBytes/sec, YUV4:2:2 creates 17.6 MBytes/sec, down to YUV9, which creates only 9.9MBytes/sec.  Sure enough, selecting YUV9 created a much smaller datastream for the Helix Producer to deal with and increased my framerate by about 20%.  That 20% made the difference I was looking for.

It doesn't end there.  I later mentioned this to Steve, who pointed me at a post to the StreamingMedia.com advanced mailing list  by Karl Lillevold, a codec engineer at RealNetworks and one of the brains behind RealVideo9.  In it he states the following:


and then goes on to say:


I tried this again on different machines with different video capture cards and different target audiences.  When encoder CPU is the factor limiting framerate, YUV9 consistently offered about a 20% frame rate increase over other color formats.   Of course, with sufficient CPU (either more horsepower or a lower-quality target bitrate or transcoding from another file rather than doing real-time encoding), perhaps using other color formats (such as I420) will create better-looking video than YUV9.