For software companies that make consumer and professional video applications, keeping up with a hardware platform ecosystem that’s constantly evolving is an essential yet resource draining task. To ensure that applications deliver the best possible performance, developers must maintain a deep understanding of the complexities of a wide variety of hardware architectures and produce different code paths that are optimized and fine-tuned to support each. This time-consuming programming exercise diverts valuable resources and keeps development teams from focusing on adding new features, enhancing existing capabilities, and designing new products that meet customer needs and expectations.
The Intel® Media Software Development Kit 2.0 (Intel® Media SDK 2.0) takes the tedium out of supporting multiple graphics platforms by providing a single, unified solution for handling video encoding, preprocessing, decoding, and transcoding. With support for current and future Intel® Graphics processing components, the Intel Media SDK’s applications programming interface (API) enables developers to optimize their code for both software-only and hardware acceleration. In addition, the API is open and extensible, so user-defined plug-ins and filters, and additional codecs can be supported.
New enhancements in the Intel Media SDK 2.0 streamline the development process further and include workload balancing between the CPU and processor graphics, additional video preprocessing functions, support for user-defined video plug-ins, and additional encoding control. This solutions brief outlines the features and benefits of this powerful and versatile software development kit.
A Single Api for Accelerated Video Processing
Intel Media SDK optimized media libraries for Intel® CPU, Intel® HD Graphics, Intel® Processor Graphics, and future Intel® multi-core architectures give developers easy access through a single API to maximum performing hardware acceleration for video codecs. Having originated as Intel® Performance Primitives (IPP), the software video encoder and software decoder algorithms that form the software libraries of the Intel Media SDK have been road tested, refined, and improved (and often embedded in partner software applications) to a high degree of efficiency. In addition, the versatile API supports third-party software codecs, and – new in 2.0 – custom video preprocessing (VPP) filters.
The Media Library Dispatcher Layer
The Intel® Media SDK 2.0 programming interface is exposed to applications by the Media Library Dispatcher layer, which also opens entry points for the various functions provided by the Intel Media SDK. In addition, the Dispatcher layer makes decisions on whether the CPU or graphics processor is best suited to perform a given function. If no suitable hardware acceleration exists, the Dispatcher redirects the function calls to the CPU library supplied by the SDK.
The CPU Library
Also referred to as the “software fallback,” the CPU library is used if the application requests a task that can’t be performed on the available graphics hardware. The CPU library is highly threaded and optimized using Intel® Streaming Instruction Set (SSE4) to ensure smooth playback or quick encoding. For developers whose audience is still using legacy hardware, the Intel Media SDK includes x86 legacy support, providing surprisingly efficient software encoding and decoding capabilities for systems that lack specific graphics hardware-acceleration features.
Developers who’ve worked with the Microsoft DirectX* Video Acceleration (DXVA) specifications know that it can be a difficult and laborious process, requiring many iterations to achieve the desired result. The Intel Media SDK provides integral support for DXVA, mitigating the complexities and streamlining the DXVA coding workflow.
Intel® Media SDK 2.0 Key Features
|Video Encoders||• H.264, MPEG-2|
|Video Decoders||• H.264, MPEG-2, VC-1|
|Preprocessing Filters||• Brightnessa|
• Color conversion
• Deinterlacing/Inverse Telecine
• Detail (sharpening) filtera
• Frame rate conversion
• Saturation controla
• Scene change detection
|Supported OS||• Microsoft Windows* 7 (32- and 64-bit)|
• Microsoft Windows Vista* (32- and 64-bit)
|Sample Source Code||• Console, DirectShow,* Media Foundation, Transcoding Application|
Intel Media SDK 2.0 feature enhancements
Intel Media SDK Building Blocks for Optimization
Working with coded bit streams and raw video frames, four building blocks provide essential functionality:
- Decode – this class of functions convert compressed bit streams
to raw, display-ready video frames.
- VPP – applies various preprocessing such as scaling, color conversion,
deinterlacing, cropping, procamp filters, and sharpening to raw
- Encode – functions that turn raw video frames into bit streams
(constant and variable bit-rate encoding are supported).
- User – functions for use in creating custom video filters.
Different usage scenarios are accomplished by arranging the building blocks accordingly. For example, simple transcoding is achieved by routing a video bit stream to a decoder and then applying preprocessing before sending it on to an encoder. Of course, much more complex functionality is possible.
And because the Intel Media SDK uses asynchronous functions to perform encode, decode, and VPP functions, various operations are returned without having to wait for results. This enables features that involve multiple data flows and delayed synchronizations prioritized to meet special usage requirements, such as background batch encoding and foreground DVD burning while transcoding takes place in the background. In addition, CPU and graphics load balancing lets you establish priorities for CPU and graphics tasks in parallel, further enabling innovative usage models and parallel workflows.
Better still, the Intel Media SDK automatically optimizes video functions for each of the supported target platforms, so developers only have to write code once. Performance is built into the process. In addition, the Intel Media SDK updates itself automatically when new graphics drivers are installed. This means that any application written with the Intel Media SDK will automatically take advantage of new graphics hardware without having to rewrite your code.
“The Media SDK simplifies that,” Hurst continued. “It is a simple, easyto-use interface for encode and decode, and you get the benefit of the full CPU optimizations as a software fallback. You will get that same benefit of hardware acceleration in the upcoming Intel® architectures without having to do additional work.”
“The efficiencies that are gained from a developer perspective,” Hurst said, “are considerable. From a development perspective, now I can focus on my engineering teams. I can have more cycles for them to add new features that I have always wanted to add or develop a new product that we didn’t have time to develop before. This helps the ISV to be more competitive in the marketplace, focusing their resources on areas in which they can differentiate their products, rather than focusing on performance tuning on each successive platform generation.”
Enhancing Media Player Classic Home Cinema
Eric Sardella, a senior software engineer in the Intel® Software and Services Group, has been collaborating on a project to enhance Media Player Classic Home Cinema (MPC-HC). MPC-HC is an open-source, smallfootprint media player for Microsoft Windows* that resembles Windows Media* Player v6.4, but includes a lot more up-to-date features. Intel Media SDK helped to alleviate the complexities of working with Microsoft DirectX* Video Acceleration (DXVA). Using the Intel Media SDK enabled Sardella to quickly optimize decode for hardware acceleration that takes full advantage of Intel platforms. This was important to the MPC-HC team, because according to Mercury Research, Intel has the largest market share of PC graphics worldwide.
As Sardella put it, “DXVA is hard, and sometimes it takes a lot of trial and error putting video in a player. Microsoft’s documentation is very difficult to decipher, and there’s precious little sample code available. To make matters even more difficult, Intel hardware has specific implementation rules that are not very well known.”
Sardella has experienced the benefits of Intel Media SDK’s evolution first hand. When he first started working on the Media Player Classic project, Intel Media SDK was in its infancy. Over time, Sardella has used each successive version of Intel Media SDK to add native hardwareacceleration support for various codecs, including H.264, VC-1, and MPEG-2.
“With 1.0 and 1.5, developers had to download the SDK and a vendor would have to package the DLLs manually for any new drivers that were required to run a new piece of hardware. That wasn’t ideal for forward compatibility,” Sardella said. “With 2.0, Intel Media SDK updates itself. When the graphics driver is installed it updates the hardware DLLs automatically.
“So if a brand new platform comes to market after you’ve written your code, the platform will automatically have the DLL packaged with it. Any application written with the Intel Media SDK will automatically take advantage of new hardware,” he continued. “Or if the ISV releases a patch for bug fixes, they can just tell the user to update their graphics driver and the Media SDK DLL will come along with it.”
For the open source Media Player Classic project, this capability overcame a stumbling block; previously, an application couldn’t check for a DLL on SourceForge.net, the open source applications and software directory. Sardella elaborated, “Now my application can check and see if the DLL is present on the system. It can tell if the graphics driver installed it. So if it’s there, my application can use it.”
The open source nature of the Media Player Classic project presented unique challenges, but it has given Sardella ample means to put the Intel Media SDK through its paces. “People work on this when they’re finished with their day job, so some new features are implemented really quickly, while others take a very long time to get approved,” Sardella explained. “But the simple interface of the Intel Media SDK lets me get things done in a week that I would’ve struggled with for much longer.”
Fine-tuning Top-selling Consumer
Video Products: ArcSoft
ArcSoft is one of the leading software developers that has been using the Intel Media SDK to optimize its video playback and editing applications, helping it develop easy-to-use, highly scalable products that run on a range of processors, including the Intel® Atom™ processor, the Intel® Core™ i7 processor, and the upcoming 2nd generation Intel® Core™ processor family (code-named Sandy Bridge).
TotalMedia* Showbiz and MediaConverter* are two of the top-selling consumer video products that ArcSoft is using the Intel Media SDK to fine-tune. Although the two applications are targeted at different classes of users – video enthusiasts and quick-and-go users, respectively – each is optimized to take advantage of Intel® multi-core, multithreaded processor technology.
“Developing transcoding and encoding used to be a very complicated process. Thanks to the simplicity of the APIs in the Intel Media SDK for both VPP (video preprocessing) and encoding, our development time was shortened,” said George Tang, ArcSoft vice president and general manager of Video Home Entertainment. “Intel’s engineering team has done a fantastic job on providing guidance on how to further optimize our transcoding engine’s low-level code. They have also helped us to benchmark our application performance to ensure that we are using the full capacity of Intel’s hardware.
“Intel’s multi-core, multi-threaded processor technology significantly reduces the conversion time in our most recent version of MediaConverter,” Tang continued. “The user can now convert four or more files concurrently while still leaving the processor free for other tasks.” The 2nd generation Intel Core processor family combines CPU and media processing functionality with deep parallelism in a single 32nm chip. Because the Intel Media SDK supports 2nd Generation Intel Core processors, ArcSoft applications can readily take advantage of the processor’s capabilities.
“With the latest version of TotalMedia ShowBiz software, Intel’s new Sandy Bridge architecture allows us to composite (layer) multiple video tracks to create more complex video experiences,” Tang said. “In addition, Intel Media SDK provides us with a great way to parallelize the decode/encode sessions that are essential to video-editing applications. Our products, optimized for Intel’s Sandy Bridge architecture, will enhance our users’ video-editing experience.”
“Using Intel’s Media SDK was an enormous help. Not only did the encoding process greatly improve, but it actually transformed the way a traditional transcoding application behaves. With the excellent performance introduced by Intel Media SDK and the 2nd generation Intel Core processor architecture, the user doesn’t need to wait any longer. When converting media files, there’s plenty of processing power to work on other tasks.”
Maximizing Performance: Avid
User-friendly consumer video-editing applications such as Avid’s Pinnacle Studio* family of products make it easy for anyone to make great-looking HD movies, complete with Hollywood-style transitions, effects, sound, and animation, at home using off-the-shelf PCs equipped with Intel processors.
“We develop specifically for Intel multi-core processors, so we turned to the Intel Media SDK to help us get improved performance and increased encoding quality,” product manager of Avid’s Pinnacle Studio Markus Duerr said. “For example, we’re using the Intel Media SDK to scale performance across multiple cores and threads. Having a single API that helps us handle low-level coding functionality so we can focus our engineering efforts on developing new features is one of the benefits of the Intel Media SDK.”
Avid’s Pinnacle Studio product line consists of Studio HD, Studio Ultimate, and Studio Ultimate Collection. Building on Studio HD as a foundation, Studio Ultimate and Ultimate Collection provide additional functionality through plug-ins by Red Giant. These plug-ins enable users to create advanced effects typically associated with Hollywood productions, such as video with a cartoon look, lens flares, special film looks that recreate popular visual styles, and more. Because Avid engineers have been collaborating closely with Intel to tune and optimize their code to take advantage of all that Intel processors have to offer, Studio products are more responsive than ever, whether they’re run on an Intel® Core™2 Duo processor or on an Intel Core i7 processor
Looking to the future, Duerr said, “Our engineers are working closely with Intel engineers to get the most out of the Intel Media SDK and learn how to best leverage 2nd generation Intel Core processor family hardware acceleration.”
Invisible Complexity and Sophistication: Corel
Corel VideoStudio* Pro enables enthusiasts to create professionallooking productions in a simplified and streamlined environment that combines video editing, media authoring, and real-time effects, as well as DVD and Blu-ray* burning.
“One of the challenges,” says Jan Piros, who is the product manager for Corel VideoStudio Pro, “is making the complexity and sophistication of what’s going on under the hood invisible to users. Modern camcorders, point-and-shoot still cameras, and DSLRs give people the ability to shoot very high-quality HD video, but many consumers don’t understand the nuances of all the different formats. Our goal is to streamline the workflow, so people can start being creative immediately.”
“ Since we can now access the chip directly, the upcoming 2nd generation Intel Core processor family cuts out the middle man and delivers pure speed.” – Jan Piros, Product Manager, Corel VideoStudio Pro
VideoStudio Pro leverages all that Intel multi-core processors, including Intel Core i7 processors and the 2nd generation Intel Core processor family have to offer to deliver a significant performance boost and fastest- in-class rendering times. According to Corel, performance is what really matters to users: They want to mix and match formats freely on the timeline and still be able to play back the project smoothly. Enhanced Smart Proxy editing takes advantage of both CPU and GPU acceleration, so HD content can be edited as easily and smoothly as standard-definition footage – even on midrange PCs – by background transcoding project footage to a lower-resolution proxy file.
“Decoding or encoding high-definition formats like AVCHD is computationally intensive,” Piros said. “We’re using the Intel Media SDK to help us optimize our code for multi-threaded Intel CPUs. This lets us handle encode/decode operations in the background, so users aren’t aware of them. What they experience is smooth, stable performance.” With the 2nd generation Intel Core processor family, performance will be boosted even further, thanks to its integrated graphics capabilities, increased parallelism, and greater throughput. “Our application architecture leverages the component layer for everything from encoding, decoding, and transcoding, to color,” Piros commented. “With Sandy Bridge, we’re seeing component layer acceleration that is just amazing. We’re able to work with multiple streams of high-def AVCHD. The performance boost is just huge.”
Supporting Future Graphics Architectures
Keeping abreast of the latest technology advances in the graphicsprocessing arena is no trivial matter, and this area is particularly a key focus for any company producing solutions that generate or play back high-definition video. Future-proofing software products is not only possible, but extremely easy through the use of the Intel Media SDK. The SDK libraries and sample code include support for upcoming graphics platforms from Intel that are scheduled for release. Of particular interest to the development community, the 2nd generation Intel Core processor family microarchitecture is accessible through the API of the Intel Media SDK. This means that as soon as products begin reaching the market with the new microarchitecture inside, software applications will be able to identify and take advantage of the parallelism and unique feature capabilities offered by 2nd generation Intel Core processors, including hardware acceleration.
Craig Hurst explained this concept further. “Here is a way to think about it. Imagine that you’re a developer writing a simple transcoder that is going to take a movie and convert it for use on an iPod*. I can use the Intel Media SDK API for building those functions within my application. If there is a hardware-acceleration capability for encode or decode on a platform, then my application will use that automatically. I get the full performance benefits of that component for the encode and decode operations.”
“If hardware-acceleration components are not present,” Hurst continued, “the application will fall back to a software-only implementation for encode and decode that will be fully threaded and fully optimized to take advantage of the processor. The benefit to me is I don’t have to worry about whether it is a fixed-function piece of silicon that runs my encoder or decoder or if it is software. It just works.”
Access to numerous resources and tools is available through the Intel® Visual Adrenaline developer program. For more details, visit: www.intel.com/software/partner/visualcomputing/
Intel is committed to helping partners plan, develop, and market software products in the rapidly growing visual computing sector and provides a wide range of information, technology roadmaps, development tools, forums, blogs, and online catalogs to ensure that the development community can capitalize on the latest platform components from Intel.
Extending the Possibilities with the Intel Media SDK 2.0
By equipping developers with a powerful tool set that taps into both present and future hardware acceleration, the Intel Media SDK extends the possibilities of application development, freeing development teams to focus on product enhancements and market differentiators with each new video-related application they produce. The productivity benefits also extend into the future, as the Intel Media SDK builds applications that are able to achieve performance gains on future architectures, including the Intel Sandy Bridge microarchitecture. Well positioned to serve as a standardized API that provides optimized support for Intel processor and graphics processor environments. The productivity, performance, and efficiency benefits of the Intel Media SDK have already attracted many software companies creating trend-setting
products and are sure to attract more as its features and capabilities become more familiar to the development community.
Obtain the latest product information and download details for the Intel Media SDK: www.intel.com/software/mediasdk
Learn more about the open-source Media Player Classic Hozme Cinema: http://mpc-hc.sourceforge.net
Get more information on ArcSoft products: www.arcsoft.com/en-us/index.asp
For more information on Avid Pinnacle Studio products: www.pinnaclesys.com/studio
Discover more about Corel products: www.corel.com