Multimedia Framework API

1.0 References

[1] Software Architecture Document, Version 1.1 Revision 1.4, LiMo Foundation, 11 September 2007

2.0 Overview

The Multimedia Framework is designed to provide easy to use services for developing multimedia applications based on an embedded Linux platform.
The Multimedia Framework API reference has defined APIs to support the functions below:
  • Playback of multimedia contents.
  • Controlling volume information.
  • Controlling direction of sound input/output.
  • Recording audio/video stream.
  • Capturing still image.
  • Manipulating image codec.
The manipulated images are not currently part of the multimedia framework.

2.1 Architecture Design

The multimedia framework can be divided into three parts:
  • Libraries that provide high-level APIs to multimedia applications
  • Common library for manipulating attributes of each multimedia instance.
  • Media-engine
The figure below is a high-level diagram of the multimedia framework.

MMArchitecture.png

Multimedia Architecture

2.2 Multimedia Framework Libraries

The Multimedia Framework consists of five libraries to provide high-level APIs for multimedia applications. The Attributes library is a common module for manipulating the attributes of each library. The others provide functions for doing multimedia tasks.

2.2.1 Player Library

The Player Library provides various functions for a multimedia application to implement playback of media contents. An application can play different formats of audio and audio/video files. File formats could include MP3, OGG, MP4, 3GP and others. The Player Library provides a consistent interface for playing different formats by hiding the creation/construction of various streaming elements from the application. This should make the application coding simpler and faster.
The Player Library provides functions that play the media, stop play-back, and pause/resume playback. An application can seek to any desired location of the media by time, video-frame or percentage of file. At the same time, an application can request the current position which can be utilized to implement elapsed time display. Application can utilize trick-mode APIs to implement fast-forward & fast-rewind. The Player Library also facilitates extraction of metadata information like album name, author, track name etc. Controlling the volume is also possible using the Player Library.

2.2.2 Camcorder Library

The Camcorder Library provides various functions for multimedia applications to implement recording of various media contents. An application can record audio and audio/video into local files. File formats could include MP3, AAC, MP4 etc. An application can also capture images into files in JPEG or other image formats.
This library provides a similar interface for audio recording, audio/video recording and image capture. The type of encoding to be used for audio, video or image, and the file format can be selected by the application. An application can also implement preview while simultaneously recording or capturing the image. Image capture could be still-shot or multi-shot where the interval between captures could be defined by the application. The library provides all the standard camcorder controls such as white-balance, color-tone, optical-filters, day/night mode, flip etc.
An application can query the display's orientation, get/set scale parameters, and the width/height of the display. Similarly, an application can also determine the sound device's capabilities such as sample rate, number of audio channels etc.

2.2.3 Sound Library

The Sound Library provides functions to manage the sound system. An application can increase/decrease volume and, mute/unmute using this library. An application can control audio input devices like microphones, hands-free devices, etc. It can also control audio output devices such as ear-phones, speaker-phones, hands-free devices, etc.

2.2.4 Image Library

The Image Library provides functions to convert images into various formats. An application can read files encoded in JPEG, PNG, etc. and convert them into alternate image formats. An application can also obtain the decoded raw image data using this library.

2.2.5 Attributes Library

The Attributes Library provides functions for manipulating the attributes of multimedia handles such as player, camcorder, image, etc. Attributes can be of various types such as integer, float, string or data (void*). An application can use get/set functions to set values for various attributes. For an attribute of type integer, the application could also set the range of values that this attribute can assume. For example, a player could have a sampling-rate attribute and the application could define specific valid values such as 22050, 44100, etc. for this attribute.

2.3 How to Use the Multimedia Framework

Each multimedia function can be manipulated with the handle. The handle is obtained by calling the appropriate create function such as MMPlayerCreate for player, MMCamcorderCreate for Camcorder, MMImageCreateFromFile for image. These functions each create an instance and return a handle. The handle is passed to other APIs to achieve the desired functionality.

For a player, the sequence of function calls is indicated below:
  • MMPlayerCreate is called to create the handle.
  • MMPlayerRealize realizes the pipeline.
  • MMPlayerStart starts playing the file set by the user.
  • MMPlayerStop stops the playback.
  • MMPlayerGet/SetVolume sets or returns the current volume level during playback.
  • MMPlayerUnrealize destroys the pipeline.
  • MMPlayerDestroy is called to destroy the handle and clear the memory associated with this handle.
For camcorder, the following sequence of function calls is used:
  • MMCamcorderCreate is called to create the handle.
  • Attributes need to be set for the camcorder depending on the mode in which it to be used. For example, when used in audio recording mode, the audio codec will need to be set.
  • MMCamcorderRealize realizes the specific pipeline depending upon the mode of the camcorder: image, audio, or video. In the case of image or video mode, a preview pipeline is also created.
  • MMCamcorderStart starts the preview.
  • MMCamcorderCaptureStart is used to capture the image in the case of image mode; MMCamcorderRecord is used instead to record audio or video.
  • MMCamcorderUnrealize destroys the pipeline.
  • MMCamcorderDestroy is called to destroy the handle and clear the memory associated with this handle.
For an image, the following API calls can be used:
  • MMImageCreateFromFile decodes the given image file.
  • MMImageEncodeToFile encodes the image to a specified file format. The encoder properties are set by the user or default settings will be used.
  • MMImageGetData will provide the image data and size of the image.
  • MMImageSetAttrs is used to set image attributes
  • MMImageGetAttrs is used to determine current image attribute settings
  • MMImageDestroy can destroy the image.
The volume API functions are:
  • MMSoundSetVolume is used to set the volume of the audio device
  • MMSoundGetVolume is used to retrieve the current volume level.
  • MMSoundSetMute mutes the device.
Attributes are used to communicate between the framework and the application. For example, passing a filename and location from an application to the framework. Attributes are used to set and get properties. Attributes can be string, data, integer, or float types.
Attributes that are necessary for the initialization of the player/camcorder instances are set before the instance initialization. Changing these attributes after initialization does not have any effect. For example, the codec setting to be used for encoding cannot be changed after initialization. The attributes that determine the controls or properties of the instance can be changed after initialization. For example, the white-balance control of the camcorder.

3.0 Industry Standard

None

4.0 External API Documents

None

5.0 Plug-in Extension Point Interface

None

6.0 Other Interfaces

None

7.0 Other Notes

None

Warning:
APIs that are in the "Draft" state are subject to evolve with new releases.
Backward compatibility is not guaranteed until moved to the "Final" state.
Developers should use APIs in the "draft" state at their own risk.

Generated on Mon Mar 31 01:01:00 2008 by  doxygen 1.5.4