MPEG4IP Project
MPEG4IP Project
The MPEG4IP project provides a standards-based system for encoding,
streaming, and playing encoded audio and video. To achieve
this we've integrated a number of existing open source packages, and
also created some original code to fill in the gaps.
Please note this project is intended for developers who are interested
in standards based audio and video, and Internet streaming. It is not intended
for end-users. Please read all the legal information in the file COPYING!
Also note that the primary development focus of this project is the
Linux platform. If you're going to use the package on other platforms,
especially non-UNIX platforms, you'll probably have some work to do.
Please use the SourceForge site to report problems, suggest enhancements,
ask questions, etc. The URL is http://www.sourceforge.net/projects/mpeg4ip
There is also a project web site at http://www.mpeg4ip.net/ that has some general information on MPEG4IP.
We also have a guide to MPEG4IP donated by everwicked. See
doc/MPEG4IP_Guide.pdf.
Overview
There are two ways to use MPEG4IP to create content:
The older method assumes that you've somehow managed to capture raw audio
and/or video into a file. That's the starting point from which you can use
the MPEG4IP encoding tools to create an MP4 file. The simplest method being
to use the 'mp4encode' script. Detailed instructions for this script and the
individual tools that it uses are in
doc/encoding/encoding.htm
Note that this might be a bit out of date; we're no longer including some of
the packages like xvid and faac - they are seperate downloads, so the command
lines may change. See those packages for specifics on command line.
The newer method is an integrated live encoding tool called mp4live. This
program is designed to make it easy to create MP4 files or transmit live
audio/video streams over the network. It can even do both things
simultaneously! The key requirement to use this tool is to have a video
capture device and a Video for Linux (v4l) or Video for Linux2 (v4l2) driver for it. So far we've
tested with the bttv driver for Brooktree based video capture cards, and
the qce driver for Logitech QuickCam Express USB webcams. Both of these
solutions can be acquired for about $50 US! Please see the file
server/mp4live/README.html for more information about mp4live.
Once an MP4 file is prepared, it can be placed in the content
directory of a streaming server. We typically use Apple's Darwin Streaming
Server, but any server that understands MP4 files (or hinted Quicktime
files) can be used.
When the content is encoded and available on the server, you can run
the player. Start 'gmp4player' and then enter the RTSP URL to the server
and the content, (command line works too). E.g.
Although we're focused on streaming, the player will also playback from
a local file. E.g
That's not all! The player is not limited to local playback of MP4 files.
It can also read AVI, CMP, DIVX, AAC, MP3, MPEG 1/2 and WAV. This is useful for
debugging since the encoded data can be check independently of the MP4
file container, and known good content, such as your favorite MP3, can be
used to verify that the player is working correctly with your hardware. E.g.
And that's still not all... You can set up your player to run a playlist
by creating a simple text file with the extension .mp4plist or .mxu, with
each item (file or stream) on it's own line.
Note: If you prefer a no UI version of the player, or your system does not
include GTK or GLIB, just the bare video window, 'mp4player' is available
to fit that need.
Legal:
Please see the file COPYING.
Building and Installing
We use make dist for our tarball. To build, you should only have to issue the following:
However, several systems have differing requirements. Please see
the section on Operating Systems Supported before
continuing.
Mp4live will only build on Linux systems. We no longer include lame or faac
in our distribution; however, mp4live requires lame version 3.92 or later and
faac as of version 1.20.1. This may change at some point, but for now, it is
required.
If you wish to make without mp4live, you can give the
You can download the sources from here
(lame) and here (faac).
Or you
can download the RPM for lame like this:
We also support ffmpeg as an optional mpeg4 encoder for mp4live. To do this,
enter the argument
In addition, if you have the xvid library installed, we will use that for the
player plugin and for mp4live. The configure scripts will automatically detect it and use the installed library. We've tested with xvidcore version 0.9.2.
If you are using xvidcore, mp4live will not display an encoded preview at
this point, and will inject VOL headers in the transmit stream. You can get
xvid here. We do not recommend using their
daily snapshots - the API has changed. When it is official, we will try to
update mpeg4ip.
If you don't have root privileges, but still wish to install the
distribution to a directory to which you do have write permission,
then here is an example of how to do that:
If this process works for you, you can skip down to the next section on
Configure Script Options. If this does not work, or you have downloaded
from CVS, the below applies.
We've built the distribution using GNU autoconf, automake and libtool.
We have attempted to follow the GNU conventions for open source packages.
This is complicated by the fact that we build on many other packages. Where
the package was already using the GNU tools, we left things alone. Where
the package was using it's own Makefile, we left things alone if it was a
complicated Makefile. If it was straightforward we replaced it with an
equivalent automake file.
If you have to install any of these tools (for example, automake is
not included with Mac OS X), find out where the others are installed
(which autoconf). If the path does not start with /usr/local, use the
For example, if autoconf is in /usr/bin, use the
Note: libtool is sometimes not installed on Linux systems (autoconf,
and automake generally are). You can download libtool from a GNU mirror
site.
In order to compile mpeg4ip from CVS, we require the following tools:
If you don't have these tools and are trying to compile from CVS, don't
complain - update to these tools, then let us know. There may be problems or
warnings with later tools - we don't keep up with the latest and greatest
tools most of the time. Please try to fix these yourself before
continuing.
Be sure to read the OS specific section later before continuing here.
In general, the code should be portable, but as someone once said
"There's no such thing as portable code, just code that has been ported."
When you find problems please be sure to use the SourceForge site to tell
us what you encountered, and hopefully how you fixed it.
To build:
The bootstrap script will pass any arguments to the configure scripts.
For the curious, the bootstrap script invokes the configure scripts of
the included packages that have them, and then our own top level configure
script is generated and run. At the end of this process all the Makefile's
are ready, and setup in the correct hierarchy. If this doesn't work for you,
you're free to hack as needed ;-)
If you are using cvs_bootstrap and are running into problems where your
GTK configurations are not found, you will need to find the aclocal files
associated with gtk and glib (in particular glib-2.0.m4, glib.m4, gtk-2.0.m4,
gtk.m4). These are usually in a share/aclocal directory somewhere on your
system.
We currently search /usr/local, /usr/contrib, /opt/gnome /usr /opt/local. If these
do not work for you, find the above files (find . -name glib-2.0.m4 -print), and add the path (without the
share/aclocal) to cvs_bootstrap.
Configure Script Options
Two options of potential interest are
If you are building on a system with an Intel x86 CPU clone, you may
need to specify the configuration option
To build with IPv6 support, use the
Darwin Quicktime Streaming Server
Please note that the Apple Darwin Quicktime Streaming Server is NOT distributed
with mpeg4ip. It can be downloaded from Apple, as either
source or pre-built binaries. For those who choose the source option, please see the documentation that accompanies the server on how to build and configure it
for your environment.
Note the default content directory is /usr/local/movies and that you can
configure the darwin server by pointing your browser at port 1220. (ie: localhost:1220)
There are some sample mp4 files included with the Darwin Streaming server.
Envivio, also has some sample mp4
files. We suggest first downloading one of these samples and try opening
the file with gmp4player. If that works, then try copying the file to the
streaming server's content directory (e.g. /usr/local/movies), and enter
the appropriate RTSP URL in gmp4player. You may have to "hint" the files
before they will stream.
If you should have problems with DSS, please feel free to contact the
developers over at Apple - they're a friendly bunch. They have 2 mailing lists
- one for
users and the other for developers. Give them a try.
These are described in more detail in doc/encoding/encoding.htm
A few debugging tools are also included:
For playback, you get:
Executable Notes
FAAD
We do not have
the complete version of FAAD, so the faad standalone decoder will not
build. FAAD as of faad2-2.0-rc3 contains a gmp4player plugin - we
recommend download and using that.
An even better idea would be to obtain the complete faac/faad package
from http://www.audiocoding.com.
Directory Structure
If you're going to start hacking, a map of the territory may prove useful:
mpeg4ip - top level project directory
config - autoconf files
doc - the minimal doc we've written so far
include - project wide includes
common - shared code
lib - project wide libraries
player - player specific code
server - server specific code
util - generally useful utilities
Standards Compliance
We're not only supporters of open source, we're supporters of open standards!
We've attempted to use the publically defined standards as much as possible.
Here's what we believe we are following. If you find something non-compliant,
please let us know. We certainly will want to fix it.
Here are the citations:
ISO/IEC 14496-1:2001 MPEG-4 Systems (includes MP4 file format)
The ISO/IEC documents must be purchased from either ISO (www.iso.ch) or one of the national bodies.
In the US, ANSI is the representative body, and provides an online store under www.ansi.org
The IETF RFCs can be found in mpeg4ip/doc/ietf
Note the MP4 file format is derived from Apple's QuickTime file format.
That specification is Apple Computer QuickTime File Format, June 28 2000
and is available through the
QT developer site.
We're also involved in the Internet Streaming Media Alliance (ISMA)
which seeks to standardize the protocols and formats used for streaming.
We believe MPEG4IP is interoperable with that organization's 1.0 Technical
Specification. For more information see http://www.isma.tv
Latency in mpeg4ip
There have been enough questions to merit a place in the README about
the latency in mpeg4ip.
First of all, mpeg4ip is not designed for video conferencing or real
time display of data. Most streaming products are not (look at Real
or Quicktime's buffering schemes - QT buffers 3.0 seconds of data,
while Real can buffer up to 30-40 seconds).
When you need to look at latency in a streaming environment, you need
to look at each potential piece. In the simplest case (mp4live to
mp4player), there are 5 potential places where latency can take place -
at mp4live, in the kernel IP transmit stack, in the network, in the
kernel IP receive stack and in the player.
Mp4live has very little latency. There is no latency from when the video frame
is encoded. In audio, it tends to be 3-4 audio frames (if using AAC, frames
tend to be 1024 samples. MP3 tends to be 1152 samples, but can change based on
sampling frequency and bit rate. Samples can be converted to seconds by
dividing by the sample rate).
The kernel IP stacks can have some latency built in, as well. Probably
not too much, but you should be aware that it can exist (look at a sniffer
trace of DSS output and you'll see what I mean).
Network latency can occur, as well. Collisions, etc, can happen, especially
if server and player are not on the same network.
Finally, the player has latency as well. For streaming, we tend to buffer
2 seconds for each stream. This is changeable by setting the
RtpBufferTimeMsec variable in the .gmp4player_rc file (for Windows users,
you'll have to change the registry to do so). The value is in milliseconds,
and you need to set it to a non-zero value (0 indicates the default value).
Recently, I've noticed that the player seems to increase latency over time -
going from a couple of seconds until 30 or more seconds after a couple of
hours. This is most likely due to a couple of different reasons. I believe
that it is due to the output frequency of the sound card not equaling the
requested sampling frequency, and rounding errors in converting the sampling
frequency to milliseconds. This does not affect the synchronization, which
seems to work. However, after a day or 2, this may lead to the player
crashing.
Player Information
See the new README.html in player/src.
Known Problems
This is due to an error in the thread scheduling code that causes a problem
with a delay of less than the thread scheduler quantum (200 msec in
some versions, 20 in others). Since the average delay used is 9
to 10 msec (less as we get closer to the video rendering time), this
can have a great effect on video playback.
You can get around this error by rebuilding your libc, after changing the
However, as the Windows run-time library only provides up to 1 millisecond
resolution (0.001 sec), such measurement on Windows machines on a LAN may
be very inaccurate.
A typical RRT between two hosts on a LAN is around 0.5 millisecond.
Measurements between Linux/Unix boxes have no such a problem.
Neither QT or Real appear to understand mpeg1/2 or mp3 content in a .mp4
container file for local playback.
Contributing Back
We welcome all contributions back to mpeg4ip. Email is the best way to
accomplish this.
Please do a
Please understand that we are trying to keep this a multi-platform project, so
compiling on multiple platforms is very much welcome.
You can use the forums or mailing list before you start to work out how
to do a certain project.
=== END OF README ===
Overview
Legal
Building and Installing
Configure Script Options
Darwin Quicktime Streaming Server
OS Supported
Executables
Executables Notes
Directory Structure
Standards Compliance
Latency in mpeg4ip
Player Information
Known Problems
$ gmp4player rtsp://myserver.mydomain.com/mycontent.mp4
$ gmp4player mycontent.mp4
$ gmp4player mymusic.mp3
./bootstrap <params for configure>
make
make install (needs root privileges).
--disable-mp4live or --disable-server arguments.
Neither the bootstrap or cvs_bootstrap scripts will allow you to build without
lame and faac installed or one of the above arguments on a Linux system. Other systems
will automatically detect that you are not building on Linux and will not
build mp4live.
lame-3.93.1-fr2.i386.rpm
and lame-devel-3.93.1-fr2.i386.rpm.
su
rpm -ivh lame*--enable-ffmpeg=<directory> to the bootstrap or cvs_bootstrap. <directory> is required and must be the absolute path to where the ffmpeg code has been compiled - do not use ~. We've tested with ffmpeg version 0.4.7 and 0.4.8. You can get ffmpeg from
here. CVS versions are not
recommended.
mkdir -p $HOME/local/bin $HOME/local/lib
./bootstrap --prefix=$HOME/local
make
make install
configure --prefix=<path before /bin on other packages>
configure --prefix=/usr
command when installing automake or libtools.
gcc 3.2 or greater
libtool 1.4.3 or 1.5
autoconf 2.53 or 2.57
automake 1.6.3 or 1.7.6
./cvs_bootstrap <arguments to pass to configure scripts>
make
make install (optional, typically need root privileges)
--disable-server and --disable-player
which disable the building of the server and player respectively. By
default both server and player are built.
--enable-mmx=no. The
configuration script automatically detects an x86 target CPU and
enables MMX assembly code in the build (if the NASM assembler is available).
If your CPU doesn't support MMX instructions you will want to disable
this feature. The configure script will also check for the minimum
version of nasm supported; we require 0.98.19 or greater.
--enable-ipv6 command option.
mp4live
Integrated, live encoding to file or network - Linux only
avi2raw
Extracts raw audio/video tracks from an AVI file
lboxcrop
Vertically crops raw video to a new aspect ratio
faac
Encodes raw audio into MPEG-4 AAC encoded audio
mp4venc
Encodes raw video into MPEG-4 encoded video using ISO codec
mp4creator
Creates and hints audio/video tracks to an mp4 file. Can also be used to extract tracks and reconstruct AAC headers
xvidenc
Encodes raw video into MPEG-4 encoded video using the Xvid codec
mp4encode
Front-end script to the above encoding tools
mp4extract
Utility to extract tracks from an MP4 file (deprecated)
mp4dump
Utility to dump MP4 file meta-information in text form
mp4trackdump
Utility to dump MP4 file track information in text form
mp4info
Utility to display MP4 file summary
avidump
Utility to display AVI file summary
yuvdump
Utility to display a raw video file on the screen
gmp4player
Simple graphical interface player
mp4player
Bare video window with sync'ed audio
|-> encoding - how to encode content
|-> ietf - copies of the relevant RFC's
|-> mcast - how to multicast
|-> mp4v2 - man pages for mp4v2 library
|-> programs - man pages for core programs
|-> video
|-->
libmpeg32 - mpeg1/2 encoder/decoder
|-->
mpeg4-2000 - ISO MPEG-4 video encoder/decoder
|-> SDL - Simple DirectMedia Layer
|-> avi - AVI file format
|-> bitstream - MPEG style low level bitstream utlity
|-> config_file - Configuration file utility
|-> gnu
|-->
getopt - gnu getopt routines
|-> mp4 - MP4 (aka MOV/Quicktime) file format library
|-> mp4v2 - new MP4 library written from scratch
|-->
test - contains some test programs
|-->
util - contains new mp4dump and mp4extract utilities
|-> mpeg2t - mpeg2 transport stream utilities
|-> msg_queue - SDL based Inter-thread messages utlity
|-> rtp - UCL RTP
|-> sdp - Our own SDP
|-> win32 - libary files need for MS Windows
|-> xvid - xvid video encoder/decoder
|-> lib - libraries specifically for the player
|-->
audio
|--->
faad - FAAD AAC decoder
|--->
mp3 - MP3 decoder
|-->
libhttp - Our own http client
|-->
rtsp - Our own RTSP client
|-> plugin - home of player plugins
|-->
audio - audio plugins
|--->
raw - raw audio plugin
|-->
rtp - rtp bytestream plugins
|--->
h261 - h261 rtp plugin
|--->
isma_audio - isma audio format plugin
|-->
video - video plugins
|--->
raw - raw video plugin
|--->
mpeg3 - mpeg1/2 video plugin
|--->
xvid - xvid video plugin
|--->
h261 - h261 decoder
|-> src - the player executable
|-->
codec
|--->
aa - aac plugin
|--->
mp3 - mp3 plugin
|--->
mpeg4 - mpeg4 ISO decoder plugin
|--->
wav - wav plugin
|-->
osx - Mac OS X UI
|-->
win_common - common windows code
|-->
win_client - windows client
|-->
win_gui - windows gui code.
|-> audio
|-->
faac - AAC encoder program
|-> mp4creator - create and hint A/V tracks to an mp4 file
|-> mp4live - mp4 live encoding interface
|-->
gui - gtk gui for mp4 live.
|-> util
|-->
avi2raw - extract raw A/V data from AVI files
|-->
avidump - dump AVI meta information.
|-->
lboxcrop - vertically crop raw video
|-->
mp4encode - front-end script to simplify encoding process
|-->
rgb2yuv - rgb to yuv converter
|-->
xvidenc - command line interface to Xvid MPEG-4 encoder
|-> video
|-->
H26L - ITU H.26L TML 9.4 reference video encoder (EXPERIMENTAL)
|-> iptv - read Cisco IP/TV programs from a content manager
|-> yuv - simple tools for examining raw video
ISO/IEC 14496-2:2000 MPEG-4 Video
ISO/IEC 14496-3:1999 MPEG-4 Audio (includes AAC)
ISO/IEC 11172-3:1993 MPEG-1 Audio (includes MP3)
ISO/IEC 13818-3:1998 MPEG-2 Audio (includes extensions to MP3)
ISO/IEC 13818-7:1997 MPEG-2 AAC
IETF RFC 1889 & 1890
RTP
IETF RFC 2326
RTSP
IETF RFC 2327
SDP
IETF RFC 2250
RTP Payload for MPEG-1/2
Note: we're just using the audio part for MP3 (and video reception for the player).
IETF RFC 3119
A More Loss-Tolerant RTP Payload Format for MP3 Audio
IETF RFC 3016
RTP Payload for MPEG-4 Audio/Visual
Note: we're implementing the MPEG-4 video part
IETF draft-ietf-avt-mpeg4-simple-06.txt
work in progress
Note: we're implementing the AAC audio part, and for the next
release will finalize implementation.
%ld should not be the correct format. You can try to fix
these by removing the l when you get the warning.
--aac-old-file-format parameter to mp4creator.
If you need to generate new aac files, you can use the mp4creator --aac-old-file-format, then extract the track with the mp4creator
--extract command.
RtpIpPortMin=<port> and
RtpIpPortMax=<port>
in the .gmp4player_rc file created in your home
directory. The IETF recommends a range of 6970 to 6999.
#define OLD_SURFACE in player/src/video.cpp.
THREAD_SCHED_USECS to 20000 (or lower) from 200000 in thread_private.h.
LimitAudioSdlBuffer config
variable to 1 in the .gmp4player_rc file. This seems to occur on a
Soundblaster Live, Red Hat 7.3 machine.
RRT = Time_RR - Time_SR - Time_delay_at_client.
-timescale=90000 when using mp4creator with
mp3 files. If you're using mp4live to stream audio, use the
rtpUseMp4RtpPayload14 config file option.
diff -c -w on each file you've changed against
a fresh tarball that you've worked on, or better yet, CVS. Please refrain from
changing any generated file (such as configure, ltmain.sh, any Makefile or
Makefile.in) - instead, please help figure out how to change the configure.in
scripts.