Video editing with open source tools

Valentina Messeri & Eleonora Oreggia, May 2009

Video editing is the process of cutting and pasting video sequences, in order to create a fluid, often narrative, combination of images and sound that develops in time following a certain structure or story board. While analogue video editing requires both a substantial investment in equipment and an aesthetic project, for example a detailed edit list and a well structured script, digital video editing on the other hand has become much more accessible. It is an easy and experimental way to create combinations of any type of footage. Huge databases of video images are readily available and the costs for setting up a home made video editing station are progressively decreasing. Software applications are more reliable, stable and easy to find, machines are faster, video cameras cheaper and hard drives larger. Digital video editing, a non-linear or non destructive method (in comparison to the physical cutting of the negative in film), can also be viewed as the audio/video equivalent of word-processing.

The video-image, unstable sister of the film-image, has become the low-key protagonist of late 20th Century audio-visual imagery. Since the beginning of the 21st century video images are proliferating on the web. Video is nowadays a common medium of expression, and has even become a substitute for socialisation. For example, the widespread use of webcams, and the integration of those in VoIP (Voice over IP) systems, modified the phone call into an audiovisual experience (teleconferencing). The possibility to upload and share fragments of any kind of video on socialisation platforms such as YouTube also creates the possibility to use video as an instrument to network with people, using it to present oneself, and to express personal opinions. Video, in this sense, becomes the mediator between internal, intimate space and external, public context, materialising relations and the interaction between Self and Other.

Digital Video Editing

Before starting any video editing project, it is very important to make sure that the computer has enough RAM (Random-Access Memory) [1], sufficient hard disc space [2], and possibly that an external FireWire drive is at hand. USB and USB2 are too slow for this purpose and frames could drop during capturing (this means some frames could be skipped while importing the video, leading to an incomplete file). It is a good habit to clean your computer before beginning a long term video project, in order to avoid congestion.

If you are running an open source operating system like Linux, it is very important that everything is well installed and configured, that the software is recognising and using all the hardware and its capabilities (for video editing FireWire, external or extended monitor and in some cases 3D acceleration are important). For real time video editing, the X environment (the graphical interface) has to be properly configured [3]. A number of video friendly Linux distributions, such as Ubuntu Studio [4] and pure:dyne [5] can make installation and configuration really easy. When your computer is ready to go, you need to choose a software to do the editing with.

What a video editing application is supposed to offer is four basic functions: import, edit, export, and the possibility to manipulate audio and video.

Importing and capturing media

A video sequence is made of sound, video, still images and graphics. To use digital media (sound, video, etc.) inside a video editing software it is necessary to import all the material required into the project. When the material already consists of digital files, the import process simply creates a symbolic link between a folder inside the source view of the program, and the physical location on the hard-drive. To capture from any external device, such as a video camera, a mini-disk or any other type of player, a digital file on the disk will be created as new. The best quality for importing from a digital camera is obtained by using FireWire. To use it you need to have a FireWire port on your computer and a FireWire output on the video-camera. From photo-cameras it is normal to import using USB. S-video and composite ports provide a lower video quality and can be used if the computer has an analogue port. Audio is normally captured from the audio input.

Editing

Video editing applications are normally organised in the following way: one screen, usually on the left side, displays the source material; the display on the right side shows the sequence as it is structured on the timeline. The timeline can be expanded in different layers that allow superimposition and the combination of different clips, images and sound. On the timeline the source material is mixed, merged and combined to create the new edited sequence.

Manipulating video

Every editing software offers a set of effects and transitions. The traditional ones are inherited from analog video editing, others belong to the digital world. Transitions are used to attach one clip to the next without any visual incongruence or noise that can break the cinematic illusion or distract the eyes. Experienced video editors tend to use a very small amount of those, because in fact, a masterly cut does not need any transition, and the sequence is fluid by itself. Effects can be used to modify colours, lights, image size and basic motion characteristics, or to create psychedelic and unnatural compositions. Although the newbie video editor might experiment with effects, a clean and well organised video piece does not need a lot of manipulation. Individual style and aesthetics can obviously contradict these guidelines. Once set the desired transitions and effects, the process of 'Rendering' will construct a model of the sequence which can be played and displayed. Rendering helps to make small corrections and to calibrate the sequence on the timeline before exporting the video clip.

Exporting (Formats and Codecs)

Understanding digital video means getting accustomed to formats and codecs. Whereas in analogue systems video can be defined as an electronic signal stored on a certain carrier, digital video is a numeric description of this electronic signal. A codec is an algorithm used to optimise, in terms of file size, the description of this data. If a codec is a method used to store video information in a compressed form, a video format can be imagined as a box which contains the data compressed. For example, if a video file has .mov as its extension, its format is QuickTime, while if a file presents a .ogg extension, the container is Ogg, the free, open standard container format maintained by the Xiph.Org Foundation, which can contain an audio file compressed with a Vorbis codec, or a media file whose video codec is generally Theora. Mp4 can be confusing because it is both an extension (thus a format: .mp4) and a codec which can be used to compress video to be contained in any other format. The combination of format and codec structures the compatibility of a file with different video players ; for example the free player VideoLAN [6] and MPlayer [7] can play almost any type of video file, proprietary players are often more restrictive. The selection of a certain codec rather than another, is more significant and consequential than one might think:using free open source codecs means, amongst other things, not binding your content to a commercial company.

Once a video sequence or a project is ready, or requires to be saved and stored, it is time to export the data as a separate video file independent from any project or application. Different encoding settings and formats are required for web, DVD or archive use. For example a video made to be visible online in live streaming must be heavily compressed, to make the file small enough to be transferable over an internet connection. But when storing media for archiving, quality is more important than the file size. Even though most video editing software has internal possibilities to compress the video sequence in a certain variety of formats and codecs, it is best to create a high quality master video for archiving, and re-compressing it to different settings using other applications specifically developed for this purpose, such as MEncoder, which will be discussed below.

FLOSS showcase

The simplest and most effective tool to capture from digital input is, in GNU/Linux systems, dvgrab, which is part of the Kino [8] project and has no GUI (Graphical User Interface) [9].

Kino, a very simple video editing tool, can import video files or capture [10] from a camera; it can perform simple editing tasks, basic effects and useful features like for example titling. Kino can export to many formats, using FFmpeg and MEncoder engines and is really well documented by its main developer, Dan Dennedy.

^{Screenshot of Kino}

A more advanced video editing application is Cinelerra, which can be used to build bigger projects that require an extensive level of audio and video compositing. Cinelerra does not support capturing, and it is necessary to use Dvgrab or Kino to capture video clips that will be then imported in the project as data files. Cinelerra is probably the best open source video editor currently existing: totally free and independent, it is supported by a huge community of users. This project is forked in two: heroine Warrior[11] on one side, and, on the other, the community version[12].

Cinelerra supports DV (Digital Video) and HDV (High Definition Video), and can import footage in many format. It is resolution and frame independent, thus it can work with video of any size and speed. A great feature is the possibility to import .VOB (Video Object File) directly from DVD. Cinelerra is timeline based and multi-tracking. The compositing window allows masking, panning, and various basic operations. This application is perfectly able to work with key-framing and a considerable amount of realtime video effects (some of them, as for example TimeFront, are really special and unusual). Export and encoding are possible thanks to a wide range of render options, from high quality resolution like mpeg2 (DVD) to low resolution web formats like Ogg [13] or Mpeg4. Batch rendering as well as XML (Extensible Markup Language) and EDL (Edit Decision List) import and export are all implemented. EDL is a textual representation of a film or video edit. It contains an ordered list of reel and time-code data (in and out points of the video clips) forming the project and the final edited sequence.

^{Screenshot of Cinelerra}

Although nowadays Cinelerra is quite stable, it is still possible to load the recovered backup data after a crash, selecting from the menu File>LoadBackup.

Even if not as sophisticated and complex as Cinelerra, Kdenlive[14] can do a wide range of amazing operations. Its interface will be familiar to all those who used any video editor on Windows or Mac: at the top the clips and imported video are shown, then the effects list, video preview and project history; along the bottom is the multitrack timeline. Kdenlive will allow you to import material from DV camera, and any video file that FFmpeg can read. You can create a project that combines FLV (Flash Video), H.264, 3G and VOB (DVD MPEG2) video files with raw material [15]. You can import slideshows and add titles and colour filters to the final project. Kdenlive features some nice effects. To use the audio effects make sure you've got the SWH plugins package installed, then you can have a vinyl crackle, if you wish... just right-click the track in the timeline.

^{Screenshot of Kdenlive}

You can export to any format supported by FFmpeg, that is the encoding engine. You will find a wide range of pre-configured formats for exporting, and you can select video size, quality and different audio formats. You can also export to tape, recording on your DV camera using the FireWire output. In addition, if you've got dvdauthor installed (see section below dedicated to this software), you can export to DVD, creating simple menus.

Kdenlive version 0.7 supports HDV, and features superimposing titles or text on video. At this stage (this article was written early 2009) it is still a bit unstable and complicated to install. If you are a Ubuntu user, there are great instructions on the Ubuntu website about this[16].

Two other simple video editing applications are Avidemux and Open Movi Editor.

Avidemux[17] is a small non-linear editor, with encoding and filtering capabilities. Special features are the possibility to embed subtitles in a video stream or split the video output in 6 channels to create an animated DVD menu. A nice tool, called Calculator, can calculate compression options by a given size.

^{Screenshot of Avidemux}

Avidemux can't capture from FireWire port, but can use Ffv1rec[18] to capture from analogue input and import .VOB from any DVD project. It is multi-platform and available in most Linux distributions, but it is best is to build Avidemux from the latest snapshot[19].

Open Movie Editor[20] is an interesting and new project with a good developing potential. Despite its simplicity, it has different tracks for audio and video and offers the possibility to select a source directory to be displayed on the top left of the main window. This way adding titles or graphics becomes as easy as drag and drop. Filtering is really simple, and exporting is just as easy: go to the "Project" menu, name your file, select the a/v codecs you prefer, and... there you go. More encoding presets will be available in the future. As with other applications, it is worth the effort to compile it from the source code. A simple way to solve basic problems when compiling is to paste any error you get in Google: reading about other people's experience can help solve many little problems.

^{Screenshot of Open Movie Editor}

Applying one or more video filters or effect onto a video clip is easy, but before you start you need to install freiOR (effects plugins)[21].

MEncoder[22], a simple movie encoder designed to encode MPlayer-playable [7] movies to other MPlayer-playable formats, can be used to compress audio/video files using command line instructions. MEncoder and MPlayer are real jewels of the open source panorama, and there is no equivalent in the commercial world. The only player that can be compared to MPlayer is videoLAN[6], which is a multi platform open source software.

FFmpeg[23] and ffmpeg2theora[24] are other command line tools composed of a collection of FLOSS libraries. They can be used to compress, for example, a lossless quality, high resolution audio/video file (suitable for DVD authoring)[25].

DVD authoring

DVD is a physical support for storing video in good quality, although data is still quite compressed. The main advantage is the possibility to play it from any home player, without the need of a computer. Authoring a DVD means combining video and audio sequences with titles, menus and other features (software interface). The usual FLOSS tool able to achieve such work is dvdauthor[26]. 'Q' dvdauthor[27] is intended to be more of a frontend, in fact it collects several tools needed to achieve the most complete DVD authoring environment possible. It imports only .VOB format files, although an encoder engine will be released soon (statement accurate at time of writing, early 2009).

^{Screenshot of 'Q' dvdauthor}

While rendering, 'Q' dvdauthor displays a funny and useful terminal window which reports about errors and other possible complications of creating a menu. This application is continuously growing into a stable and reliable application. It also permits scripting for graphic animations and other advanced menu features.

Subtitling

The open source panorama offers various simple possibilities for subtitling. The following tools work perfectly, and can perform almost all the facilities other common subtitlers can do. MPlayer has to be properly configured and off course it is always a good idea to check all the general options (in Preferences) before starting. For example, have a look at 'video output': if xv doesn't work, try X11[28].

Subtitle Editor[29] is really easy to use and presents the following features: in the main window an integrated video player can play video preview (using MPlayer). It has timing and can generate and display waveforms. The subtitling is previewed over the video. It is possible to edit the script header (authors, translators, timers, video information, etc.), check the spelling, convert frame-rate and scale. It supports the following formats: SSA, ASSA, SubRip, MicroDVD, MPsub, Adobe DVD and plain text. The internal format is Advanced SubStation Alpha[30].

KSubtile[31] and Gnome Subtitles[32] are the subtitlers for KDE and GNOME (the two main desktop environment systems for GNU/Linux) so choose them according to what window manager you use. They work in a very similar way and are very easy to install in any Linux distribution.

^{Screenshot of Ksubtile}

^{Screenshot of Gnome Subtitles}

Perspective on Real-time and Procedural video

Pure Data[33], and other systems, provide the best environment for live video making and interactivity. Pure Data[34] is an infinite source of applications for real-time audio,video and interactivity. Pd-extended[35] offers an easy to install system integrating most of the up-to-date libraries and extensions.

^{Screenshot of Pure Data}

Veejay[36] is a visual instrument and real-time video sampler allowing you to play video 'like you would play a piano'. It features on-the-fly video sampling, a wide range of possibilities for connecting to other applications and devices and a veejaypuredata object[37]. It has support for (multiple) live camera, and streaming capabilities. A very promising application.

^{Screenshot of Veejay}

Installing is possible only by building from source, but there is a package for the dynebolic project[38], and Ubuntu Hardy[39].

Conclusion

A video editing software is a very big and complex application, and its development is not fully complete within the Floss panorama. Nowadays it is still a challenge to make video using only FLOSS, but development is progressing and each user is supported by a great community of other users and by the developers. Because Free Software has many forms, from big and complete packages such as Cinelerra to smaller applications like dvgrab, which can often be combined and used in all kinds of different configurations, FLOSS can propose a workflow which is not organised from above, but reticular, "bottom-up", and highly customisable to the individual needs of the artist.

A user/creator approaching the FLOSS video editing world for the first time is better off using one of the video friendly distributions, such as pure:dyne and Ubuntu Studio, bypassing manual installation processes, quickly getting the hands on the digital project. Once inside this brave new world, great fun will start: working (and wandering around) inside a system whose structures are open implies a number of important advantages, like, for example, the possibility to get actively involved in the projects one uses, and being able to suggest future directions of development. In fact the margin between users and creator is, in FLOSS projects, soft and blurred, and nobody is a customer.

While complete and complex video editing applications, like for example Cinelerra, become more and more usable and reliable, a great number of small video applications, often very functional and well performing, as well as innovative and experimental, are available within the FLOSS panorama. These smaller applications for creative and experimental video making allow the user/artist to create a different workflow, segment the working process in smaller steps and glue together several different applications (piping[40]), rather than structuring an entire project inside one single software. In this way both the audiovisual material and the working method can be radically manipulated.

Notes

[1] This is the operative memory of the computer, used on the fly by the system. When working with video you need a substantial amount of RAM. The exact amount depends on the hardware within your computer.

[2] Consider, by example, that 30 minutes of raw DV (the usual capturing setting) are approximately 7 Gigabytes. For exporting, on the other side, 1.49 minutes of Raw uncompressed material is 2.13 Gigabytes.

[3] For example, when using compiz or beryl effects and transitions, the video plays back a bit flaky. It is possible to fix this problem by changing window manager, for instance by choosing a non-composite one.

[4] http://ubuntustudio.org/

[5] http://puredyne.goto10.org/

[6] http://videolan.org

[7] MPlayer is a free and open source media player. The program is available for all major operating systems, including Linux and other Unix-like systems. MPlayer supports a wide variety of media formats. A companion program, MEncoder, can take an input stream or file and transcode it into several different output formats.

[8] http://www.kinodv.org/

[9] For the poetical implication of the command line method see $(echo echo) echo $(echo): Command Line Poetics in this handbook

[10] Capturing from video camera or audio device means digitising analogue material from FireWire, analogue video or audio port. When the footage required is already a digital file, it can be easily imported inside a project or application.

[11] http://www.heroinewarrior.com/

[12] http://cinelerra.org/

[13] Normally the format Ogg contains audio compressed with Vorbis codec, and video compressed with Theora codec. For Theora Video compression see: http://theora.org/Theora is a free and open video compression format from the Xiph.org Foundation. Like all our multimedia technology it can be used to distribute film and video online and on disc without the licensing and royalty fees or vendor lock-in associated with other formats. To know more about Vorbis please visit: http://www.vorbis.com/

[14] http://www.kdenlive.org/

[15] For a detailed and critical overview on codecs see this research (now outdated) produced by the NIMK in Amsterdam: http://xname.cc/text/video-streaming-on-wan.pdf

Use Wikipedia to gather information on FLV and other codecs not included here.

[16] https://help.ubuntu.com/community/KdenliveSVN

[17] http://www.avidemux.org/admWiki/index.php?title=Main_Page

[18] http://www.avidemux.org/admWiki/index.php?title=Ffv1rec

[19] http://www.avidemux.org/admWiki/index.php?title=Compiling_Avidemux#Choos...

[20] http://www.openmovieeditor.org/

[21] http://www.piksel.org/frei0r

[22] http://www.mplayerhq.hu/design7/news.html

[23] http://www.ffmpeg.org/

[24] http://v2v.cc/~j/ffmpeg2theora

[25] A variety of encoding settings can be found in the research mentioned above: http://xname.cc/text/video-streaming-on-wan.pdf

Some more examples here:

ffmpeg -i Test.dv -target dvd -aspect 16:9 -sameq my_dvd_video.mpg

Here is an example of how to encode an audiovisual file in Ogg Theora (open source codec for good quality on the web). This gives best results encoding video from PhotoJPEG :

ffmpeg2theora test.mov --audiobitrate 128 --samplerate 44 -a 8 --videobitrate 600 --videoquality 8 -o test.ogg

[26] http://dvdauthor.sourceforge.net/

[27] http://qdvdauthor.sourceforge.net/

[28] The X Window System (commonly X or X11) is a computer software system and network protocol that provides a graphical user interface (GUI) for networked computers. It implements the X display protocol and provides windowing on raster graphics (bitmap) computer displays and manages keyboard and pointing device control functions.

[29] http://subtitleeditor.sourceforge.net/

[30] http://en.wikipedia.org/wiki/SubStation_Alpha

[31] http://ksubtile.sourceforge.net/

[32] http://gnome-subtitles.sourceforge.net/

[33] See article in this handbook - Pure Dataflow - Diving into Pd

[34] http://puredata.info/

[35] http://puredata.info/downloads

[36] http://www.veejayhq.net

[37] http://www.veejayhq.net/2007/05/sendvims-for-pure-data-a-veejay-client/a...

[38] http://sourceforge.net/project/showfiles.php?group_id=47564&package_id=2...

[39] http://www.veejayhq.net/download-2/

[40] In Unix-like computer operating systems, a pipeline is an original software pipeline: a set of processes chained by their standard streams, so that the output of each process (stdout) feeds directly as input (stdin) of the next one.

Images

All images courtesy of the authors