MP4LIVE README

October, 2003
Dave Mackie
Bill May

MP4LIVE
Changes in Version 1.0
Changes in Version 1.0RC1
Changes in Version 0.9.9
Changes in Version 0.9.8
Changes in Version 0.9.7
Changes in Version 0.9.5
Hardware Requirements
Software Requirements
Warnings
Tips
Building and Installing
Using mp4live
Use with QT 6.0 and Real One
Network Transmission
Use with Darwin Streaming Server
Sharing Capture Cards
Command Line Options
Known Issues
Unknowns
Links
Configuration Variables

MP4LIVE

MP4LIVE is a Linux audio/video capture utility that can capture and encode audio and video in real-time. The results can be written to either an .mp4 file, transmitted onto the network via either unicast or multicast, or both simultaneously! The audio is encoded with MP3 or AAC, and the video with MPEG-4 Simple Profile.

Please use the MPEG4IP SourceForge site to report problems, suggest enhancements, ask questions, etc. The URL is http://www.sourceforge.net/projects/mpeg4ip

Do not contact us via email

Back to top

Changes in Version 1.0

Generate correct Video Profile based on bitrate and frame rate

Changes in Version 1.0RC1

Support for V4L2 drivers (recommended)
Fixed long term audio/video sync problems in transmission and file recording
Updated faac encoder to use packaged faac
Updated to enable ffmpeg encoder
Updated to allow use of packaged XVID encoder

Changes in Version 0.9.9

Added H.261 video encoding (no recording).
Fixes to audio driver from rca.
Added flag to remove b=RR:0 for playing through DSS with gmp4player

Changes in Version 0.9.8

Major synchronization changes.

Back to top

Changes in Version 0.9.7

Split out encoder specific information from the base architecture (rtp transmitter, file recorder). This enables easy addition of codecs.

Reworked configuration variables.

Back to top

Changes in Version 0.9.5

Once again, fixes for A/V sync and adapting to encoding speed.

Added UI to easily load and save configuration files

SDP now reflects media bandwidth

Experimental file sources is removed.

Back to top

Hardware Requirements

Pentium III class machine of at least 500 MHz (Pentium IV class machine at 2 GHz is very nice.)

Note systems vary quite a bit in their video capture abilities. For instance, I have a name brand 750 MHz PIII that drops frames when pushed to CIF sizes at greater than 15 fps, but I have a no-name clone with a 800 MHz PIII that can encode CIF @ 24 fps no problem.

RAM is not typically an issue. I develop on machines with 128 MB, but I believe much smaller configurations would work fine. The real issue is CPU speed. We recommend at least 256Mb.

A sound card with an OSS compatible driver and capture ability, preferrably at least 3.8.2 that have the SNDCTL_DSP_GETERROR define, and that support the SNDCTRL_DSP_GETISPACE ioctl accurately.

A video device with a video4linux (v4l) compatible driver and memory mapped capture ability. We also support (and recommend) the video4linux2 (v4l2) driver.

Known to work are: Video capture cards based on the Brooktree 8x8 chip with an analog A/V source - VCR, DVD, Camcorder, Settop box, Tivo, etc. Typically a video capture card will support composite video inputs. Some will also have S-Video input. Others will also have a TV Tuner input.

Logitech Quickcam Express Webcam

Note on multi-processor machines (SMP): mp4live is multithreaded at a coarse level. Specifically the video encoder, audio encoder, file recorder, network transmitter, and user interface each have their own thread. Unfortunately for owners of multiprocessor machines, the video encoder thread dominates the computational requirements so one CPU will be very busy, and the others will be lightly to moderately loaded. For those looking for a project, a multi- threaded video encoder would certainly provide an interesting challenge.

Back to top

Software Requirements

Linux with a 2.4 or later kernel.
We recommend a kernel with V4L2 built in. See the instructions on
building it yourself

Drivers for sound and video devices

bttv 0.7 or later video capture driver (Included with RedHat 7.1 and later) (0.9 required for v4l2)
qce webcam driver

Back to top

WARNINGS!

Please see the MPEG4IP README regarding legal issues, and the list of open source packages that are redistributed with this code.

This is a LINUX program! Do not even think about trying to get this to run on Windows! Even moving it to other UNIX systems would require some re-programming since the sound and video capture interfaces are Linux specific.

By far the easiest route is to use a Linux distribution that already has a 2.4 kernel, the bttv driver, and the associated i2c module built into it. However, this will not get you good long term audio and video lip sync. See the instructions on how to build your own kernel with V4L2 included.

I've had many headaches with sound cards under Linux. Before you start using mp4live, please make sure you're able to playback and record with your sound card!

You should definitely increase the number of capture buffers for the bttv driver. This reduces the chance of dropping video frames due to transient delays in the system. By default bttv uses 2 buffers. You can increase this by editting /etc/modules.conf and adding the line "add options bttv gbuffers=32" at the end of the file. The value 32 is my recommendation but you can experiment with other values if you are so inclined.
Note: with v4l2, we're not sure if this is required any more.

Back to top

Tips

I suggest you disable any fancy, computationally intensive screensavers when using mp4live to capture long programs. Along the same lines, don't run any programs that make large resource demands (CPU, bus, disk, network) while mp4live is running.

If you're capturing large video image sizes, then you may be able to boost the encoded video frame rate by disabling video preview. In general, once you've got the system working, disabling preview is a good idea.

The AAC audio encoder is somewhat slower than the MP3 audio encoder so you may see lower video frame rates and greater sensitivity to CPU load if you are using AAC.

Linux supports the POSIX soft real-time extensions and mp4live will attempt to use these to give it priority over non-real time processes. Typically these calls can only be made by processes with root privileges, so you may want to run mp4live as root for this reason.

If you have the latest version of OSS, you have a chance of detecting audio overruns. That, in combination with the latest version and working on a fast machine will give fairly good lip syncronization up to about the hour mark running 90% of the CPU with V4L. With V4L2, we've had good audio and video sync out beyond 500 hours in our lab.

Back to top

Building and Installing

See the MPEG4IP README for general notes about the build environment.

Assuming you've already done a build at the top level of mpeg4ip, and you're on a Linux system then mp4live should be built and waiting for you in this directory. If you've done a top level 'make install', the mp4live will be installed into '/usr/local/bin'. Of course, you can also issue 'make' and 'make install' from this directory as well.

Back to top

Using mp4live

Typically, there is no need for command line options to mp4live. You can just type 'mp4live' and you'll be up and running.

If you change an mp4live configuration setting, then that change will be saved in your home directory in ~/.mp4live_rc. This file is read when mp4live is started, and the configuration settings adjusted accordingly.

The default settings for mp4live are to record 1 minute of audio and video to an mp4 file, ./capture.mp4 The first time you use the program, it's a good idea to just hit the Start button, and see what happens. If all goes well, 1 minute later you have a playable/streamable mp4 file. If you don't get this, then it's time to review this README, and it that doesn't help, then fire off a message on the MPEG4IP SourceForge discussion group.

Assuming things are working you can now use the Settings buttons to adjust things like the video size and frame rate, the audio sampling rate, the encoded bitrates, etc. The UI is hopefully self-explanatory. If not, let us know what's confusing and we'll look at fixing that. (I'm a big believer that if you need to read a document to use a UI, then the UI is broken and should be fixed. Of course, as I've re-learned many times, what is obvious and natural to me, isn't always to other people.)

If you're capturing video that uses "widescreen" or "letterbox" format, it's a big win to change the "Aspect Ratio" in the Video Settings. This will cause the video to be automatically cropped so you don't waste precious CPU time encoding the empty black bars at the top and bottom of the screen.

The capture cards will always try to capture frame rates based on what the setting of the video card is (either NTSC or PAL).

The default is to assume that the video driver is going to capture close to the correct frame rate of 29.97 for NTSC, 25 for PAL. If you don't think that this is working quite right, try the "videoTimestampCache=0" to the .mp4live_rc before you start. (This may be the case with usb web cams, but most likely not regular capture cards). Note: this is for v4l users only.

Back to top

Use with QT 6.0 and Real One

It is possible to create content for QT 6.0 and Real One with the Envivio plug in. You must create with the audio encoding set to AAC for both of these.

If you want to stream (using the instructions below), you must have Envivio version 1.2. Download Real One, then download Envivio TV version 1.2 afterwards. The Envivio plugin downloaded with Real One will not play multiple AAC frames in a RTP packet, so your sound will appear to stutter.

Back to top

Network Transmission

To use mp4live to transmit live audio/video to the network, follow these directions:

Select 'Enabled' in the Transmission section of the main window.

By default, mp4live will multicast the media to IP address 224.1.2.3

If you wish to change this, select the 'Settings...' button in the Transmission section of the main window. A dialog will appear that allows selection of the destination IP address (either unicast or multicast), the UDP port numbers used, the TTL used if multicasting, and the name of the .sdp file to generate (more about that in a minute). When you're done making changes, select 'OK'.

When you press mp4live's 'Start' button, media transmission to the network will begin. Also a small text file with extension .sdp will be created that describes the media transmission for the player. The player needs the .sdp file to be able to tune into the media streams. (Note the .sdp file can also be generated from the Transmission Settings dialog)

The most convenient way to distribute the .sdp file is to have mp4live write it to a directory that is accessible from a web server (httpd) that is running on the same machine as mp4live. This allows the client to be started with the HTTP URL of the sdp file, and it will download the .sdp file via http and then use the information in the .sdp file to tune into the network transmission. E.g.:

gmp4player http://myserver/myprogram.sdp

For Real One, use the Open command with http://myserver/myprogram.sdp. For QT6.0, use Open URL.

You can of course distribute the .sdp file in a number of other ways, say ftp, or email. You would then start the player with the local file name of the sdp file, e.g.:

gmp4player myprogram.sdp

Back to top

Use with Darwin Streaming Server

If you would like to use mp4live in conjunction with the Darwin Streaming Server (DSS), that is easy to do. You can have mp4live both record and transmit live media streams. When you record the .mp4 files, just ensure that they are written to the media directory that is accessible via the Darwin Streaming Server, typically /usr/local/movies. Once the recording is complete, it will be available for on-demand playback.

For example:

gmp4player rtsp://DSS/mymovie.mp4

The Darwin Streaming Server can also be configured to act as a relay agent for the mp4live media streams. Copy the .sdp file generated by mp4live to the media directory of the Darwin Streaming Server (e.g. /usr/local/movies) Players can now request the .sdp file from DSS which will cause DSS to act as a relay between mp4live and the player.

For example:

gmp4player rtsp://DSS/mymovie.sdp

If you're having problems where gmp4player is stopping after 2 minutes when relaying through a Darwin Streaming Server, add the line rtpNoBRR0=1 to your .mp4live_rc. Darwin is expecting RTCP messages from gmp4player, and the b=RR:0 statement in the SDP stops gmp4player from sending them.

Back to top

Sharing Capture Cards

If you have another program that wants to simultaneously process the raw audio and/or video from the capture cards, there is typically a problem in that many drivers only support one reader at a time. To address this issue, mp4live can be configure to write the raw audio and/or video that it reads from the capture cards to a named pipe (fifo). A named pipe looks like a file, but the data only exists in memory and never goes to disk. This is an efficient way to have the two applications share the media data.

To configure this feaure, add the following to ~/.mp4live_rc (or whatever configuration file you want to use), changing "/dir" to some directory where you want the named pipes to exist:

rawEnable=1
rawAudioFile=/dir/audio_pipe
rawAudioUseFifo=1
rawVideoFile=/dir/video_pipe
rawVideoUseFifo=1

The audio format is 16 bit PCM, the video format is YUV12 (planar 4:2:0 YUV).

Back to top

Command Line Options

There are currently four command line options to mp4live:
--file <config-file>
--automatic
--headless
--sdp

--file <config-file> allows specification of the mp4live configuration file to something other than ~/.mp4live_rc. Perhaps you have a several frequently used configurations. You can save the configuration settings to different files, and then use this option to choose among them.

--automatic causes mp4live to act as if the Start button was pressed immediately upon startup. The program will do whatever the current configuration instructs it to do. This option can be used in conjunction with the 'cron' utlity to do scheduled recording and/or transmission.

--headless causes mp4live to behave in the --automatic mode AND not display any user interface.

--sdp causes mp4live to just generate the sdp file based on its configuration file and then exit.

Back to top

Known Issues

Using a system with a PCI instead of an AGP video display card can cause video "tearing" with CIF or larger size images. I.e. the PCI bus quickly gets swamped moving raw video from the video capture card to the CPU, and then from the CPU to the video display card. Having the AGP bus for the CPU to video display card transfer alleviates this problem. If someone is interested one could experiment with the video overlay capabilities of the Bt8x8 to bypass this problem, but it would require some rework of our code with respect to the video preview function.

It took me awhile to figure this out so perhaps I can save some of you some time. If you use the Hauppage WinTV Go card you need to connect the mini-jack on the card to the line-in input on your sound card in order to get the audio signal from the TV tuner.

Note there is currently no support for DV/mini-DV camcorders via FireWire. You can of course still use these via their composite or S-Video outputs.

More recent versions of mp4live add streaming hint tracks as a post-processing step (i.e. after the recording is finished). For long duration recording (1 hour or greater), this step can take a minute or two. I'm hoping to enhance the UI to provide user feedback while this is taking place, but for now the application gets very unresponsive during this period. If this is a big problem for you, there is a configuration option to disable the hinting process, "recordMp4HintTracks=0". The mp4 file can always be hinted later with the mp4creator utility.

The audio and video should be in sync if you're using the latest tools (V4L2 and the latest OSS driver). If you're not, you will have problems in long term (usually an hour or so).

The current audio/video synchronization algorithm in the MP4 File Recorder starts by dropping video frames until an audio frame is loaded. It then drops subsequent video frames until the next I video frame is loaded. This I frame is stretched to the beginning of the first audio frame to synchronize the video and audio. Because of this, the first video frame is displayed for a longer duration before the video starts rolling. This duration is usually of the order of 4 or 5 video frame times and generally unnoticable.
We've done our best to try to start the audio first, but since the video is already running in the preview (if turned on), the file recorder receives a bunch of video frames before the first audio frame propagates to the file recorder. Given this, the above algorithm seems to be a resonable solution.

Sometimes, you may experience a crash while changing the video parameters such as height/width or aspect ratio. If this occurs, change the parameter in your .mp4live_rc file, and restart mp4live.

Back to top

Unknowns

I've only used mp4live with two video capture cards the Viewcast Osprey 100 and the Hauppage WinTV Go. There are many other Bt8x8 based capture cards listed in the bttv driver documentation. Odds are you're using one of these ;-) Reports from initial users suggest though that the bttv driver handles the wide variety of cards gracefully, and mp4live doesn't have card specific issues.

If you do have problems with mp4live, my first suggestion is that you download the latest version of the xawtv package, and try it with your capture card. If it works and mp4live doesn't then I'd be glad to hear from you. If xawtv doesn't work with your capture card, then I can't help you. Something is wrong with your capture card/system/bttv driver/kernel combination. I don't have the capability or inclination to debug that for you!

Back to top

Links

MPEG4IP http://www.mpeg4ip.net/
bttv driver http://bytesex.org/bttv/
qce driver http://www.sourceforge.net/projects/qce-ga
xawtv http://bytesex.org/xawtv/
Xvid http://www.xvid.org/
LAME http://www.sourceforge.net/projects/lame
FAAC http://www.audiocoding.com

Back to top

Configuration Variables

Application Level
NameTypeDefaultDoes
useRealTimeSchedulerbooltrueattempts to use real time features of the OS
Probably only suceeds as root
durationint1duration in durationUnits
durationUnitsint60Number of seconds per duration unit (1, 60, 3600, 86400)
debugboolfalseEnable debug output
signalHaltstringsighupSignal used in no gui mode to stop
Audio Options
audioEnablebooltrueTrue if audio is enabled
audioSourceTypestringOSSAudio Source Type (only OSS for now)
audioDevicestring/dev/dspAudio Device to use
audioMixerstring/dev/mixerAudio Mixer to use
audioInputstringmixAudio Input Type to use
audioChannelsint2Number of Encoded Audio Channels (1 or 2)
audioSampleRateint44100Audio Frequency Sample Rate
audioBitRateint128Encoded Audio Bit Rate (in 1000 Hz)
audioBitRateint128Encoded Audio Bit Rate (in 1000 Hz) (unused)
audioBitRateBpsint128000Encoded Audio Bit Rate
audioEncodingstringMP3Audio Encoding to use
audioEncoderstringLAMEAudio Encoder to use
audioOssUseSmallFragsbooltrueEnable small fragments size in OSS
audioOssFragmentsint128Number of fragments
audioOssFragSizeint8Size of fragments
Video Options
videoEnablebooltrueEnable Video
videoSourceTypestringV4LVideo Source to use
(V4L is for both V4L and V4L2)
videoDevicestring/dev/video0Video Device to use
videoInputint1Video Input to use (index from V4L)
videoSignalint1PAL-0, NTSC-1, SECAM-2
videoChannelListIndexint0Which channel list to use for tuner
see video_util_tv.cpp
videoChannelIndexint10 based index for channel in above list
videoPreviewbooltrueShow Video Preview in Gui
videoRawPreviewboolfalseShow Raw Video Preview in Gui
videoEncodedPreviewbooltrueShow Encoded Video Preview in Gui
videoEncodingstringMPEG4Video Encoding Type
videoRawWidthint320Width of output frame in pixels
videoRawHeightint240Height of output frame in pixels
videoAspectRatiofloat1.33Aspect ratio
videoFrameRatefloat29.97Frame Rate
videoKeyFrameIntervalfloat2.0Number of Seconds between Key Frames
videoBitRateint500Encoded Video Bit Rate in 1000 bits per second
videoForceProfileIdboolfalseTrue to force MPEG4 Video Profile to videoProfileId
videoProfileIdint3 (SP@L3)MPEG4 Video Profile when forcing
videoBrightnessint50Brightness level (0 to 100)
videoHueint50Hue level (0 to 100)
videoColorint50Color level (0 to 100)
videoContrastint50Contrast level (0 to 100)
videoTimestampCachebooltrueCalculate timestamps, rather than read with timestamp
(V4L only)
videoH261Qualityint10Starting H.261 Video Quality
videoH261QualityAdjFramesint8Number of frames to adjust H.261 Quality over
videoCaptureBuffersCountint16Number of capture buffers to request
(V4L2 only)
Recording Options
recordEnablebooltrueTrue to record
recordRawAudioboolfalseTrue to record raw audio
(PCM at encode frequency)
recordRawVideoboolfalseTrue to record raw video
(YUV at height/width)
recordEncodedAudiobooltrueTrue to record encoded audio
recordEncodedVideobooltrueTrue to record encoded video
recordMp4Filestringcapture.mp4Filename to create
recordMp4HintTracksbooltrueRecord hint tracks when recording completed
recordMp4OptimizeboolfalseOptimize mp4 file when recording completed
Transmission (RTP) Options
rtpEnableboolfalseTrue to transmit over the network
rtpDestAddressstring224.1.2.3Video Stream destination address
audioRtpDestAddressstring224.1.2.3Audio Stream destination address
rtpAudioDestPortint20002Audio Stream destination port
rtpVideoDestPortint20000Video Stream destination port
rtpPayloadSizeint1460max bytes of audio or video per packet
rtpMulticastTtlint15Multicast TTl
rtpDisableTimestampOffsetboolfalseIf true, start RTP timestamps at 0
if false, start at random offset
rtpUseSingleSourceMulticastboolfalseUse SSM multicast
sdpFilestringcapture.sdpWhere to store sdp file describing session
rtpUseMp4RtpPayload14boolfalseif true, use RTP payload 14 and 90000 timescale
if false, use dynamic payload and frequency timescale
rtpNoBRR0boolfalseIf true, do not include b=RR:0 in SDP

Back to top

Dave Mackie
Bill May
Cisco Systems, Inc.