October, 2003
Dave Mackie
Bill May
MP4LIVE
Changes in Version 1.0
Changes in Version 1.0RC1
Changes in Version 0.9.9
Changes in Version 0.9.8
Changes in Version 0.9.7
Changes in Version 0.9.5
Hardware Requirements
Software Requirements
Warnings
Tips
Building and Installing
Using mp4live
Use with QT 6.0 and Real One
Network Transmission
Use with Darwin Streaming Server
Sharing Capture Cards
Command Line Options
Known Issues
Unknowns
Links
Configuration Variables
MP4LIVE
MP4LIVE is a Linux audio/video capture utility that can capture and encode
audio and video in real-time. The results can be written to either an .mp4
file, transmitted onto the network via either unicast or multicast, or both
simultaneously! The audio is encoded with MP3 or AAC, and the video with
MPEG-4 Simple Profile.
Please use the MPEG4IP SourceForge site to report problems, suggest
enhancements, ask questions, etc. The URL is
http://www.sourceforge.net/projects/mpeg4ip
Do not contact us via email
Changes in Version 1.0
Generate correct Video Profile based on bitrate and frame rate
Changes in Version 1.0RC1
Support for V4L2 drivers (recommended)
Changes in Version 0.9.9
Added H.261 video encoding (no recording).
Changes in Version 0.9.8
Major synchronization changes.
Changes in Version 0.9.7
Split out encoder specific information from the base architecture
(rtp transmitter, file recorder). This enables easy addition of
codecs.
Reworked configuration variables.
Changes in Version 0.9.5
Once again, fixes for A/V sync and adapting to encoding speed.
Added UI to easily load and save configuration files
SDP now reflects media bandwidth
Experimental file sources is removed.
Hardware Requirements
Pentium III class machine of at least 500 MHz
(Pentium IV class machine at 2 GHz is very nice.)
Note systems vary quite a bit in their video capture abilities. For instance,
I have a name brand 750 MHz PIII that drops frames when pushed to CIF sizes
at greater than 15 fps, but I have a no-name clone with a 800 MHz PIII that
can encode CIF @ 24 fps no problem.
RAM is not typically an issue. I develop on machines with 128 MB, but I believe much smaller configurations would work fine. The real issue is CPU speed.
We recommend at least 256Mb.
A sound card with an OSS compatible driver and capture ability, preferrably
at least 3.8.2 that have the SNDCTL_DSP_GETERROR define, and that support
the SNDCTRL_DSP_GETISPACE ioctl accurately.
A video device with a video4linux (v4l) compatible driver and memory mapped
capture ability. We also support (and recommend) the video4linux2 (v4l2) driver.
Known to work are:
Video capture cards based on the Brooktree 8x8 chip
with an analog A/V source - VCR, DVD, Camcorder, Settop box, Tivo, etc.
Typically a video capture card will support composite video inputs.
Some will also have S-Video input.
Others will also have a TV Tuner input.
Logitech Quickcam Express Webcam
Note on multi-processor machines (SMP): mp4live is multithreaded at a coarse
level. Specifically the video encoder, audio encoder, file recorder, network
transmitter, and user interface each have their own thread. Unfortunately for
owners of multiprocessor machines, the video encoder thread dominates the
computational requirements so one CPU will be very busy, and the others will
be lightly to moderately loaded. For those looking for a project, a multi-
threaded video encoder would certainly provide an interesting challenge.
Software Requirements
Linux with a 2.4 or later kernel.
Drivers for sound and video devices
bttv 0.7 or later video capture driver
(Included with RedHat 7.1 and later)
(0.9 required for v4l2)
WARNINGS!
Please see the MPEG4IP README regarding legal issues, and the list of
open source packages that are redistributed with this code.
This is a LINUX program! Do not even think about trying to get this to
run on Windows! Even moving it to other UNIX systems would require some
re-programming since the sound and video capture interfaces are Linux
specific.
By far the easiest route is to use a Linux distribution that already has a
2.4 kernel, the bttv driver, and the associated i2c module built into it.
However, this will not get you good long term audio and video lip sync. See
the instructions on
how to build your own kernel with V4L2 included.
I've had many headaches with sound cards under Linux. Before you start using
mp4live, please make sure you're able to playback and record with your sound
card!
You should definitely increase the number of capture buffers for the bttv
driver. This reduces the chance of dropping video frames due to transient
delays in the system. By default bttv uses 2 buffers. You can increase this
by editting /etc/modules.conf and adding the line
"add options bttv gbuffers=32"
at the end of the file. The value 32 is my recommendation but you can
experiment with other values if you are so inclined.
Tips
I suggest you disable any fancy, computationally intensive screensavers
when using mp4live to capture long programs. Along the same lines, don't
run any programs that make large resource demands (CPU, bus, disk, network)
while mp4live is running.
If you're capturing large video image sizes, then you may be able to boost
the encoded video frame rate by disabling video preview. In general, once
you've got the system working, disabling preview is a good idea.
The AAC audio encoder is somewhat slower than the MP3 audio encoder so
you may see lower video frame rates and greater sensitivity to CPU load
if you are using AAC.
Linux supports the POSIX soft real-time extensions and mp4live will attempt
to use these to give it priority over non-real time processes. Typically
these calls can only be made by processes with root privileges, so you may
want to run mp4live as root for this reason.
If you have the latest version of OSS, you have a chance of detecting
audio overruns. That, in combination with the latest version and
working on a fast machine will give fairly good lip syncronization up to
about the hour mark running 90% of the CPU with V4L. With V4L2, we've
had good audio and video sync out beyond 500 hours in our lab.
Building and Installing
See the MPEG4IP README for general notes about the build environment.
Assuming you've already done a build at the top level of mpeg4ip, and
you're on a Linux system then mp4live should be built and waiting for you
in this directory. If you've done a top level 'make install', the mp4live
will be installed into '/usr/local/bin'. Of course, you can also issue
'make' and 'make install' from this directory as well.
Using mp4live
Typically, there is no need for command line options to mp4live. You can
just type 'mp4live' and you'll be up and running.
If you change an mp4live configuration setting, then that change will be
saved in your home directory in ~/.mp4live_rc. This file is read when
mp4live is started, and the configuration settings adjusted accordingly.
The default settings for mp4live are to record 1 minute of audio and video
to an mp4 file, ./capture.mp4 The first time you use the program, it's
a good idea to just hit the Start button, and see what happens. If all
goes well, 1 minute later you have a playable/streamable mp4 file. If you
don't get this, then it's time to review this README, and it that doesn't
help, then fire off a message on the MPEG4IP SourceForge discussion group.
Assuming things are working you can now use the Settings buttons to adjust
things like the video size and frame rate, the audio sampling rate, the
encoded bitrates, etc. The UI is hopefully self-explanatory. If not, let us
know what's confusing and we'll look at fixing that. (I'm a big believer that
if you need to read a document to use a UI, then the UI is broken and should
be fixed. Of course, as I've re-learned many times, what is obvious and natural
to me, isn't always to other people.)
If you're capturing video that uses "widescreen" or "letterbox" format,
it's a big win to change the "Aspect Ratio" in the Video Settings. This
will cause the video to be automatically cropped so you don't waste precious
CPU time encoding the empty black bars at the top and bottom of the screen.
The capture cards will always try to capture frame rates based on what
the setting of the video card is (either NTSC or PAL).
The default is to assume that the video driver is going to capture close
to the correct frame rate of 29.97 for NTSC, 25 for PAL. If you don't
think that this is working quite right, try the "videoTimestampCache=0"
to the .mp4live_rc before you start. (This may be the case with usb
web cams, but most likely not regular capture cards). Note: this is
for v4l users only.
Use with QT 6.0 and Real One
It is possible to create content for QT 6.0 and Real One with the
Envivio plug in. You must create with the audio encoding set to
AAC for both of these.
If you want to stream (using the instructions below), you must have
Envivio version 1.2. Download Real One, then download Envivio TV
version 1.2 afterwards. The Envivio plugin downloaded with Real One
will not play multiple AAC frames in a RTP packet, so your sound will
appear to stutter.
Network Transmission
To use mp4live to transmit live audio/video to the network, follow these
directions:
Select 'Enabled' in the Transmission section of the main window.
By default, mp4live will multicast the media to IP address 224.1.2.3
If you wish to change this, select the 'Settings...' button in the
Transmission section of the main window. A dialog will appear that allows
selection of the destination IP address (either unicast or multicast),
the UDP port numbers used, the TTL used if multicasting, and the name of
the .sdp file to generate (more about that in a minute). When you're done
making changes, select 'OK'.
When you press mp4live's 'Start' button, media transmission to the network
will begin. Also a small text file with extension .sdp will be created that
describes the media transmission for the player. The player needs the .sdp
file to be able to tune into the media streams. (Note the .sdp file can also
be generated from the Transmission Settings dialog)
The most convenient way to distribute the .sdp file is to have mp4live write
it to a directory that is accessible from a web server (httpd) that is running
on the same machine as mp4live. This allows the client to be started with
the HTTP URL of the sdp file, and it will download the .sdp file via http
and then use the information in the .sdp file to tune into the network
transmission. E.g.: gmp4player http://myserver/myprogram.sdp
For Real One, use the Open command with http://myserver/myprogram.sdp. For
QT6.0, use Open URL.
You can of course distribute the .sdp file in a number of other ways, say
ftp, or email. You would then start the player with the local file name
of the sdp file, e.g.:
gmp4player myprogram.sdp
Use with Darwin Streaming Server
If you would like to use mp4live in conjunction with the Darwin Streaming
Server (DSS), that is easy to do. You can have mp4live both record and
transmit live media streams. When you record the .mp4 files, just ensure
that they are written to the media directory that is accessible via the Darwin
Streaming Server, typically /usr/local/movies. Once the recording is complete,
it will be available for on-demand playback.
For example:
gmp4player rtsp://DSS/mymovie.mp4
The Darwin Streaming Server can also be configured to act as a relay agent
for the mp4live media streams. Copy the .sdp file generated by mp4live to
the media directory of the Darwin Streaming Server (e.g. /usr/local/movies)
Players can now request the .sdp file from DSS which will cause DSS to act
as a relay between mp4live and the player.
For example:
gmp4player rtsp://DSS/mymovie.sdp
If you're having problems where gmp4player is stopping after 2 minutes when
relaying through a Darwin Streaming Server, add the line
Sharing Capture Cards
If you have another program that wants to simultaneously process the raw
audio and/or video from the capture cards, there is typically a problem
in that many drivers only support one reader at a time. To address this
issue, mp4live can be configure to write the raw audio and/or video that
it reads from the capture cards to a named pipe (fifo). A named pipe looks
like a file, but the data only exists in memory and never goes to disk.
This is an efficient way to have the two applications share the media data.
To configure this feaure, add the following to ~/.mp4live_rc (or whatever
configuration file you want to use), changing "/dir" to some directory
where you want the named pipes to exist:
rawEnable=1
The audio format is 16 bit PCM, the video format is YUV12 (planar 4:2:0 YUV).
Command Line Options
There are currently four command line options to mp4live:
Known Issues
Using a system with a PCI instead of an AGP video display card can cause
video "tearing" with CIF or larger size images. I.e. the PCI bus quickly
gets swamped moving raw video from the video capture card to the CPU, and
then from the CPU to the video display card. Having the AGP bus for the
CPU to video display card transfer alleviates this problem. If someone is
interested one could experiment with the video overlay capabilities of the
Bt8x8 to bypass this problem, but it would require some rework of our code
with respect to the video preview function.
It took me awhile to figure this out so perhaps I can save some of you some
time. If you use the Hauppage WinTV Go card you need to connect the mini-jack
on the card to the line-in input on your sound card in order to get the
audio signal from the TV tuner.
Note there is currently no support for DV/mini-DV camcorders via FireWire.
You can of course still use these via their composite or S-Video outputs.
More recent versions of mp4live add streaming hint tracks as a post-processing
step (i.e. after the recording is finished). For long duration recording
(1 hour or greater), this step can take a minute or two. I'm hoping to
enhance the UI to provide user feedback while this is taking place, but
for now the application gets very unresponsive during this period. If this
is a big problem for you, there is a configuration option to disable the
hinting process, "recordMp4HintTracks=0". The mp4 file can always be hinted
later with the mp4creator utility.
The audio and video should be in sync if you're using the latest tools
(V4L2 and the latest OSS driver). If you're not, you will have problems
in long term (usually an hour or so).
The current audio/video synchronization algorithm in the MP4 File Recorder
starts by dropping video frames until an audio frame is loaded. It then
drops subsequent video frames until the next I video frame is loaded.
This I frame is stretched to the beginning of the first audio frame to
synchronize the video and audio.
Because of this, the first video frame is displayed for a longer duration
before the video starts rolling. This duration is usually of the order of
4 or 5 video frame times and generally unnoticable.
Sometimes, you may experience a crash while changing the video parameters such
as height/width or aspect ratio. If this occurs, change the
parameter in your .mp4live_rc file, and restart
mp4live.
Fixed long term audio/video sync problems in transmission and file recording
Updated faac encoder to use packaged faac
Updated to enable ffmpeg encoder
Updated to allow use of packaged XVID encoder
Fixes to audio driver from rca.
Added flag to remove b=RR:0 for playing through DSS with gmp4player
We recommend a kernel with V4L2 built in. See the instructions on
building it yourself
qce webcam driver
Note: with v4l2, we're not sure if this is required any more.
rtpNoBRR0=1
to your .mp4live_rc. Darwin is expecting RTCP messages from gmp4player, and the b=RR:0 statement
in the SDP stops gmp4player from sending them.
rawAudioFile=/dir/audio_pipe
rawAudioUseFifo=1
rawVideoFile=/dir/video_pipe
rawVideoUseFifo=1
--file <config-file>
--automatic
--headless
--sdp
--file <config-file>
allows specification of the mp4live configuration file
to something other than ~/.mp4live_rc. Perhaps you have a several frequently
used configurations. You can save the configuration settings to different
files, and then use this option to choose among them.
--automatic
causes mp4live to act as if the Start button was pressed
immediately upon startup. The program will do whatever the current
configuration instructs it to do. This option can be used in conjunction
with the 'cron' utlity to do scheduled recording and/or transmission.
--headless
causes mp4live to behave in the --automatic
mode AND not display any user interface.
--sdp
causes mp4live to just generate the sdp file based on its configuration file and then exit.
We've done our best to try to start the audio first, but since the video
is already running in the preview (if turned on), the file recorder receives
a bunch of video frames before the first audio frame propagates to the
file recorder. Given this, the above algorithm seems to be a resonable
solution.
MPEG4IP | http://www.mpeg4ip.net/ |
bttv driver | http://bytesex.org/bttv/ |
qce driver | http://www.sourceforge.net/projects/qce-ga |
xawtv | http://bytesex.org/xawtv/ |
Xvid | http://www.xvid.org/ |
LAME | http://www.sourceforge.net/projects/lame |
FAAC | http://www.audiocoding.com |
Application Level | ||||
---|---|---|---|---|
Name | Type | Default | Does | |
useRealTimeScheduler | bool | true | attempts to use real time features of the OS Probably only suceeds as root | |
duration | int | 1 | duration in durationUnits | |
durationUnits | int | 60 | Number of seconds per duration unit (1, 60, 3600, 86400) | |
debug | bool | false | Enable debug output | |
signalHalt | string | sighup | Signal used in no gui mode to stop | |
Audio Options | ||||
audioEnable | bool | true | True if audio is enabled | |
audioSourceType | string | OSS | Audio Source Type (only OSS for now) | |
audioDevice | string | /dev/dsp | Audio Device to use | |
audioMixer | string | /dev/mixer | Audio Mixer to use | |
audioInput | string | mix | Audio Input Type to use | |
audioChannels | int | 2 | Number of Encoded Audio Channels (1 or 2) | |
audioSampleRate | int | 44100 | Audio Frequency Sample Rate | |
audioBitRate | int | 128 | Encoded Audio Bit Rate (in 1000 Hz) | |
audioBitRate | int | 128 | Encoded Audio Bit Rate (in 1000 Hz) (unused) | |
audioBitRateBps | int | 128000 | Encoded Audio Bit Rate | |
audioEncoding | string | MP3 | Audio Encoding to use | |
audioEncoder | string | LAME | Audio Encoder to use | |
audioOssUseSmallFrags | bool | true | Enable small fragments size in OSS | |
audioOssFragments | int | 128 | Number of fragments | |
audioOssFragSize | int | 8 | Size of fragments | |
Video Options | ||||
videoEnable | bool | true | Enable Video | |
videoSourceType | string | V4L | Video Source to use (V4L is for both V4L and V4L2) | |
videoDevice | string | /dev/video0 | Video Device to use | |
videoInput | int | 1 | Video Input to use (index from V4L) | |
videoSignal | int | 1 | PAL-0, NTSC-1, SECAM-2 | |
videoChannelListIndex | int | 0 | Which channel list to use for tuner see video_util_tv.cpp | |
videoChannelIndex | int | 1 | 0 based index for channel in above list | |
videoPreview | bool | true | Show Video Preview in Gui | |
videoRawPreview | bool | false | Show Raw Video Preview in Gui | |
videoEncodedPreview | bool | true | Show Encoded Video Preview in Gui | |
videoEncoding | string | MPEG4 | Video Encoding Type | |
videoRawWidth | int | 320 | Width of output frame in pixels | |
videoRawHeight | int | 240 | Height of output frame in pixels | |
videoAspectRatio | float | 1.33 | Aspect ratio | |
videoFrameRate | float | 29.97 | Frame Rate | |
videoKeyFrameInterval | float | 2.0 | Number of Seconds between Key Frames | |
videoBitRate | int | 500 | Encoded Video Bit Rate in 1000 bits per second | |
videoForceProfileId | bool | false | True to force MPEG4 Video Profile to videoProfileId | |
videoProfileId | int | 3 (SP@L3) | MPEG4 Video Profile when forcing | |
videoBrightness | int | 50 | Brightness level (0 to 100) | |
videoHue | int | 50 | Hue level (0 to 100) | |
videoColor | int | 50 | Color level (0 to 100) | |
videoContrast | int | 50 | Contrast level (0 to 100) | |
videoTimestampCache | bool | true | Calculate timestamps, rather than read with timestamp (V4L only) | |
videoH261Quality | int | 10 | Starting H.261 Video Quality | |
videoH261QualityAdjFrames | int | 8 | Number of frames to adjust H.261 Quality over | |
videoCaptureBuffersCount | int | 16 | Number of capture buffers to request (V4L2 only) | |
Recording Options | ||||
recordEnable | bool | true | True to record | |
recordRawAudio | bool | false | True to record raw audio (PCM at encode frequency) | |
recordRawVideo | bool | false | True to record raw video (YUV at height/width) | |
recordEncodedAudio | bool | true | True to record encoded audio | |
recordEncodedVideo | bool | true | True to record encoded video | |
recordMp4File | string | capture.mp4 | Filename to create | |
recordMp4HintTracks | bool | true | Record hint tracks when recording completed | |
recordMp4Optimize | bool | false | Optimize mp4 file when recording completed | |
Transmission (RTP) Options | ||||
rtpEnable | bool | false | True to transmit over the network | |
rtpDestAddress | string | 224.1.2.3 | Video Stream destination address | |
audioRtpDestAddress | string | 224.1.2.3 | Audio Stream destination address | |
rtpAudioDestPort | int | 20002 | Audio Stream destination port | |
rtpVideoDestPort | int | 20000 | Video Stream destination port | |
rtpPayloadSize | int | 1460 | max bytes of audio or video per packet | |
rtpMulticastTtl | int | 15 | Multicast TTl | |
rtpDisableTimestampOffset | bool | false | If true, start RTP timestamps at 0 if false, start at random offset | |
rtpUseSingleSourceMulticast | bool | false | Use SSM multicast | |
sdpFile | string | capture.sdp | Where to store sdp file describing session | |
rtpUseMp4RtpPayload14 | bool | false | if true, use RTP payload 14 and 90000 timescale if false, use dynamic payload and frequency timescale | |
rtpNoBRR0 | bool | false | If true, do not include b=RR:0 in SDP |
Dave Mackie
Bill May
Cisco Systems, Inc.