Video 1.x notes

This page lists some notes concerning version 1.x of the video library. It is no longer maintained, so these notes are kept as an archive.

Listing Capture Devices

The Capture.list() method operates differently on 2.0 than previous releases. In gstreamer, there's a concept of actual supported resolutions (and frame rates) for capture, so Capture.list() has been expanded to use all of that information.

OS X, however, thinks different, and allows any capture dimensions to be valid. To keep things in line with Windows and Linux, we're forced to pre-roll each of the cameras with no/default settings, which seems to give us the native resolution of the camera. Then that size is cut in half a few times over (until sizes get below 100 pixels) to produce a few synthetic sizes that can be reported via Capture.list().

The name-based selection of devices is half-supported in gstreamer, and looks to be going away, so we'll need our own code to keep this feature.

Some notes from Andrés extracted from a bug report

As a reference, here is the link to the bugzilla entry about the qtkitvideosrc element, which is used by the video library to do capture under OSX: https://bugzilla.gnome.org/show_bug.cgi?id=661348

Hopefully they will include my patches into the main line, and won't remove the property probe interface (needed to get the camera names) or, in the case they do, will replace with an analogous API.

I just got the confirmation from the gstreamer developers that the property probe interface has been removed from the upcoming 1.0 release, although they say there will be a replacement at some later point.

Until that replacement comes in, we might need to call the functions in QTKit and DirectShow that gives you the list of currently available devices directly from the video library. This is, actually, very simple code. for QTKit:

NSArray *devices = [QTCaptureDevice inputDevicesWithMediaType:QTMediaTypeVideo];

and for Direct Show:

HDEVINFO devinfo = SetupDiGetClassDevsW (&KSCATEGORY_VIDEO, NULL, NULL, DIGCF_PRESENT | DIGCF_DEVICEINTERFACE);

The advantage of the property probe interface in gstreamer is that we got it for free through the gstreamer-java bindings. Now we will need to add our own binding JNA code to call them directly from Processing.

Another thing is that the property probe for camera names never worked on Linux, but I found that on Linux appears to be a mechanism to retrieve names:

http://www.freedesktop.org/software/systemd/gudev/

http://gudev.sourceforge.net/ (seems like an outdated link)

so it would be a matter to write the appropriate wrapper for it as well:

http://www.freedesktop.org/software/systemd/gudev/GUdevEnumerator.html

Media Formats

We haven't tried all of these media formats so we're not sure if the decoders actually work, but this is an edited list exported from GStreamer by @codeanticode.

• XviD video
• Wavpack audio
• Vorbis audio
• Theora video
• mpeg1 and mpeg2 video
• mad mp3
• FLX video
• FLAC audio
• PCM signed 20-bit little-endian planar
• RealAudio 2.0 (28.8K)
• RealAudio 1.0 (14.4K)
• QDesign Music Codec 2
• MP3 (MPEG audio layer 3)
• MP3 (MPEG audio layer 3)
• MP2 (MPEG audio layer 2)
• MP1 (MPEG audio layer 1)
• MLP (Meridian Lossless Packing)
• FLAC (Free Lossless Audio Codec)
• DCA (DTS Coherent Acoustics)
• MPEG-4 Audio Lossless Coding (ALS)
• ALAC (Apple Lossless Audio Codec)
• ATSC A/52A (AC-3)
• AAC LATM (Advanced Audio Codec LATM syntax)
• Windows Media Video 9
• Windows Media Video 8
• Windows Media Video 7
• Sorenson Vector Quantizer 3 / Sorenson Video 3 / SVQ3
• Sorenson Vector Quantizer 1 / Sorenson Video 1 / SVQ1
• RealVideo 4.0
• RealVideo 3.0
• RealVideo 2.0
• RealVideo 1.0
• QuickTime video (RPZA)
• QuickTime Animation (RLE) video
• PPM (Portable PixelMap) image
• PNG image
• Microsoft Video 1
• Microsoft RLE
• MPEG-4 part 2 Microsoft variant version 3
• MPEG-4 part 2 Microsoft variant version 2
• MPEG-4 part 2 Microsoft variant version 1
• MPEG-1 video
• MPEG-4 part 2
• MPEG-2 video
• Apple MJPEG-B
• MJPEG (Motion JPEG)
• JPEG-LS
• H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
• Intel H.263
• H.263 / H.263-1996, H.263+ / H.263-1998 / H.263 version 2
• H.261
• Flash Video (FLV) / Sorenson Spark / Sorenson H.263
• Autodesk Animator Flic video
• Flash Screen Video v1
• DV (Digital Video)
• AAC audio
• DV video
• JPEG
• CMML stream
• ADPCM
• ATSC A/52 audio