FFmpeg: The ultimate Video and Audio Manipulation Tool

February 24, 2012 by . 15 comments

What is FFmpeg?

Chances are you’ve probably heard of FFmpeg already. It’s a set of tools dedicated to decoding, encoding and transcoding video and audio. FFmpeg is based on the popular libavcodec and libavformat libraries that can be found in many other video conversion applications, like Handbrake.

So why would you need FFmpeg? Got a video in an obscure format that every other player couldn’t recognize? Transcode it with FFmpeg. Want to automate cutting video segments out of movies? Write a short batch script that uses FFmpeg. Where I work, I constantly have to encode and process video material, and I’d never want to go back to using a GUI tool for doing this.

This post is a follow-up on Video Conversion done right: Codecs and Software, where I discussed the various codecs and containers that you can find these days. For a quick overview, I’d suggest to read this one as well, because it covers some important basics.

Now, let’s dive into the more practical aspects.

Installation

FFmpeg is free and open source, and it’s cross-platform. You’ll be able to install it on your Windows PC as well as your Linux server. The only thing you need to be comfortable with is using your command line. As it’s actively developed, you should try to update your version from time to time. Sometimes, new features are added, and bugs are fixed. You won’t believe how many times updating FFmpeg solved encoding problems I had.

Windows

Compiling FFmpeg on Windows is not too easy. That’s why there are (semi-) automated builds of the latest version available online, most prominently the ones from Zeranoe.com. Download the latest build labeled as “static” for your 32 or 64 Bit system. Those builds work fairly well, but they might not include all possible codecs. For the commands where I used libfaac below, with the Zeranoe builds you have to choose another AAC codec instead, like libvo_aacenc, or aac, with the additional options -strict experimental.

The Zeranoe download will include ffmpeg.exe, which is the main tool we are going to use. And you’re done. That was easy, was it?

OS X

On OS X, you have the tools for building programs from source. It’s very easy to install FFmpeg if you use a package manager like Homebrew or MacPorts. Install one of them if you haven’t already, and then run either

brew install ffmpeg --with-ffplay 
or
sudo port install ffmpeg-devel 
This takes a while, because dependencies have to be built too. When you’re done, you should be able to call FFmpeg by running ffmpeg from the command line. You should see a copyright header and some build details.

If you’re lazy, you can also download a static build, which just works out of the box. You only need to extract the archive and you’ll find an ffmpeg binary.

Note that there’s also ffmpegX, but its bundled version always is a bit behind, and I wouldn’t recommend using it unless you have a good reason for doing so. The .app package contains a compiled version of the ffmpeg command line tool already, which you could theoretically use.

Linux

On many Linux distributions, FFmpeg comes in the default packages (e.g. on Ubuntu, where Libav – an FFmpeg fork – is bundled). However, mostly this is a very old release version. Do yourself a favor and compile from source – it sounds harder than it actually is. FFmpeg has a few dependencies, so therefore you’ll want to install encoders such as LAME for MP3 and x264 for videos as well. To compile FFmpeg, look at the FFmpeg wiki, which has a few compilation guides for various Linux distributions.

Similar to Windows, static builds for FFmpeg are also available. You can download these if you don’t want to compile yourself or install the outdate version from your package manager.


Our first encoded Video

Now that we have FFmpeg working, we can start with our video or audio conversion. Note that this is always a very resource intensive task, especially with video. Expect some conversions to take a little while, for example when re-encoding whole movies. A quad-core CPU like the i7 can easily be saturated with a good encoder and performs very fast then.

Before you handle videos, you should know what the difference between “codec” and “container” (also, “format”) is, as I will be using these terms without explaining them any further. I know these can be easily mixed up and aren’t as self-explaining as you’d like them to be. To read more about this, see this Super User answer: What is a Codec (e.g. DivX?), and how does it differ from a File Format (e.g. MPG)?

Basic Syntax

Okay, now let’s start. The most basic form of an FFmpeg command goes like this – note the absence of an output option like -o:

ffmpeg -i input output 
And you’re done. For example, you could convert an MP3 into a WAV file, or an MKV video into MP4 (but don’t do that just yet, please). You could also just extract audio from a video file.
ffmpeg -i audio.mp3 audio.wav
ffmpeg -i video.mp4 video.mkv
ffmpeg -i video.mp4 audio.wav
FFmpeg will guess which codecs you want to use depending on the format specifier (e.g. “.wav” container obviously needs a WAV codec inside, and MKV video will use x264 video and AC3 audio). But that’s not always what we want. Let’s say you have an MP4 container – which video codec should it include? There are a couple of possible answers. So, we’re better off specifying these codecs ourselves, since some codecs are better than others or have particularities that you might need.

Specifying Video and Audio codecs

In most cases, you want a specific output codec, like h.264, or AAC audio. Today, there’s almost no reason not to encode to h.264, as it offers incredible quality at small file sizes. But which codecs do really work in FFmpeg? Luckily, it has a ton of built-in codecs and formats, which you can get a list of. Check the output: The D and E stand for “decoding” and/or “encoding” capabilities, respectively.

ffmpeg -codecs
ffmpeg -formats 
Now, we can use any of those to perform our transcoding. Use the -c option to specify a codec, with a :a or :v for audio and video codecs respectively. These are called “streams” in FFmpeg – normally there is one video and one audio stream, but you could of course use more. Let’s start by encoding an audio file with just one audio stream, and then a video file with a video stream and an audio stream.
ffmpeg -i input.wav -c:a libfaac output.mp4
ffmpeg -i input.avi -c:v libx264 -c:a libfaac output.mkv 
The possible combinations are countless, but obviously restricted by the format/container. Especially those pesky old AVI containers don’t like all codecs, so you’re better off encoding to MP4 or MKV containers these days, because they accept more codecs and can be easily handled by any modern player.

When to copy, when to encode?

You might not have thought about this before, but sometimes, you want to just copy the contents of a video and not re-encode. This is actually very critical when just cutting out portions from a file, or only changing containers (and not the codecs inside). In the example I gave above, we wanted to change MP4 to MKV.

The following command is wrong: It will re-encode your video. It will take forever, and the result will (probably) look bad.

ffmpeg -i input.mp4 output.mkv
What FFmpeg did here was encoding the file yet another time, because you didn’t tell it to copy. This command however does it right:
ffmpeg -i input.mp4 -c:v copy -c:a copy output.mkv 
If you were to re-encode your videos all the time when you cut them, instead of just copying the codec contents, you’d eventually end up with something like this (exaggerated, but still highly entertaining) example of generation loss. Looks scary, does it?

If changing the container without encoding does not work for you, you can try the mkvmerge tool that mkvtoolnix offers. This will for example create an MKV file.

mkvmerge input.avi -o output.mkv
Similarly, MP4Box can create MP4 files from existing video.


Advanced Options

If you’ve used some test videos or audio files for the above sequences, you might have seen that the quality wasn’t good enough to be useful. This is completely normal. Also, you will probably have constraints about bit rate, file size or target devices, such as when encoding a video for the PlayStation 3 or your Android phone.

Quality Settings

Quality comes first. What is “quality”, even? Generally, the more bits you can spend on a file, the better it will look or sound. This means, that for the same codec, larger file size (or larger bit rate) will equal in better quality. In most cases, that is.

Most people associate quality with “bit rate”. This is a relict from the good old days where everyone would encode MP3 files from their CDs. Of course, this is a relatively simple approach, and you can get a good feeling for the bit rates you might need for certain videos. If you want to set the bit rate, you can do that with the -b option, again specifying the audio or video stream (just to resolve any ambiguities). The bit rate can be specified with suffixes like “K” for kBit/s or “M” for mBit/s.

ffmpeg -i input.wav -b:a 192K out.mp3
ffmpeg -i input.avi -c:v libx264 -b:v 500K -c:a copy out.mp4 
But often, bit rate is not enough. You only need to restrict bit rate when you have a very specific target file size to reach. In all other cases, use a better concept, called “constant quality”. The reason is that sometimes you don’t want to spend the same amount of bits to a segment of a file. That’s when you should use variable bit rate. Actually, this concept is very well known for MP3 audio, where VBR rips are commonly found.

In video, this means setting a certain value called Constant Rate Factor. For x264 (the h.264 encoder you should use), this is very easy:

ffmpeg -i source.mp4 -c:v libx264 -crf 23 out.mp4 
The CRF can be anything within 0 and 51, with the reasonable range being 17 to 26. The lower, the better the quality, the higher the file size. The default value is 23 here. For MPEG-4 video (like XviD), a similar concept exists, called “qscale”.
ffmpeg -i source.mp4 -c:v mpeg4 -qscale:v 3 out.mp4 
Here, the qscale:v can range from 1 to 31. The lower, the higher the quality, with values of 3 to 5 giving a good enough result in most cases.

In general, the best bet is to just try and see for yourself what looks good. Take into account the result file size you want and how much quality you can trade in for smaller file sizes. It’s all up to you. As a last note, don’t ever fall for the sameq option. Sameq does not mean same quality and you should never use it.

Cutting Video

Often, you want to just cut out a portion from a file. FFmpeg supports basic cutting with the -t and -ss options. The first one will specify the duration of the output, and the second one will specify the start point. For example, to get the first five seconds, starting from one minute and 30 seconds:

ffmpeg -ss 00:01:30 -i input.mov -c:v copy -c:a copy -t 5 output.mov 
The time values can either be seconds or in the form of HH:MM:SS.ms. So, you could also cut one minute, ten seconds and 500 milliseconds:
ffmpeg -ss 00:01:30 -i input.mov -c:v copy -c:a copy -t 00:01:30.500 output.mov 
Note that we’ve again just copied the contents instead of re-encoding because we used the copy codec. Also, see how the -ss option is before the actual input? This will first seek to the point in the file, and then start to encode and cut. This is especially useful when you’re actually not just copying content, but encoding it, because it’ll speed up your conversion (although it’s less accurate). If you want to be more accurate, you’ll have to tell FFmpeg to encode the whole video and just discard the output until the position indicated by -ss:
ffmpeg -i input.mov -ss 00:01:30 -c:v copy -c:a copy -t 5 output.mov

Resizing

Got a High Definition 1080p video that you want to watch on your mobile phone? Scale it down first! FFmpeg supports software-based scaling when encoding with the -s option. The following example will do that:

ffmpeg -i hd-movie.mkv -c:v libx264 -s:v 854x480 -c:a copy out.mp4 
Phones have some restrictions on video playback, which can be tackled by using so-called “profiles” when encoding. More about this can be found in the resources at the bottom.

Other useful Examples

On Super User, I’m currently very active in the FFmpeg tag. You could browse the questions there, or see this list of questions and answers I’ve compiled.

General issues:

Specific conversion problems:

Of course, if you have any specific problem, feel free to ask it on the main site and we Super Users can help you out! However, make sure you always post the command you’re using and the full, uncut output FFmpeg gives you. Also, always try to use the latest version if possible. Ideally, we’d like to see a sample video as well.

15 Comments

Subscribe to comments with RSS.

  • barlop[] says:

    This is a really good post about ffmpeg. I sympathise with this questioner http://superuser.com/questions/373018/resources-to-use-ffmpeg-effectively/373024#373024 and check out the answer he got to this one and his bemused comment http://superuser.com/questions/372432/ffmpeg-converting-from-m4v-to-mp4-does-not-stop-processing

  • ffmpegNewbie says:

    Hi, thanks for the informative post on FFMPEG. Could you provide the “best” commands for taking an unknown input video and converting them to:

    • OGG
    • MP4
    • WEBM

    The goal is to allow users to upload their videos to a website, for that website to convert the uploaded videos so that it becomes available to the widest possible audience, put it in http://mediaelementjs.com/, and let people watch videos.

    FFMPEG has too many options for the casual developer to understand and use efficiently.

    • mvark says:

      True, there are too many options

      For casual users, there are GUI tools available for FFMPEG, like WinFF – http://winff.org/

    • lakhan says:

      here is your solution

      ffmpeg -i STREAM.MP4 -acodec libvorbis -ac 2 -ab 96k -ar 44100 -b 345k -s 640×360 output.ogv

      ffmpeg -i STREAM.MP4 -acodec libvorbis -ac 2 -ab 96k -ar 44100 -b 345k -s 640×360 output.webm

      ffmpeg -i STREAM.MP4 -acodec libfaac -ab 96k -vcodec libx264 -vpre slower -vpre main -level 21 -refs 2 -b 345k -bt 345k -threads 0 -s 640×360 output.mp4

  • Tarje says:

    Great guide!

    I do have some problems though, every time I try to either reencode or copy a .mp4 to a new container it only takes the audio, and there is no video in the new file.. :/

    • slhck says:

      I’m sure we can help you if you post a question on the main site, superuser.com! Make sure to include the command and the full, uncut output from the console too.

  • reggie says:

    Great info on ffmpeg, which there is a bit of confusion about out there.

  • Vaibhav says:

    Hi ,

    I have question regarding the scaling down or resizing of video resolution.

    I am trying to resize in resolution with width : 2 and height : 2 but getting this error

    [swscaler @ 02edc040] 352×288 -> 2×2 is invalid scaling dimension [swscaler @ 02edc040] 352×144 -> 2×1 is invalid scaling dimension Last message repeated 1 times [scaler for output stream 0:0 @ 02f36380] Failed to configure output pad on scaler for output stream 0:0 Error opening filters!

    Thus , does it mean that libswscaler can’t scale down video to any arbitray size ?

    Please help me in this case as with some GUI video tool scaling down to any arbitrary size is possible

    Thanks in advance.~

    • slhck says:

      Not every size is possible (i.e. dimensions might have to be in multiples of 8), but it’s hard to trace back the error if we don’t have additional information. Please post a question on the main site, and include the command you use as well as the complete, uncut console output you get, including the error messages. If you could supply a sample of the video, that would be great as well.

  • SKC says:

    Hi, i was exploring all the ways of using FFMpeg functionalities.

    1. Tried running FFMpeg commands using Android activity as a command line terminal, but was unable to do so on a unrooted device[Working directory comes as NULL everytime].

    2. Tried exploring the JNI way of using different FFMpeg library functions and still looking into it.

    Now my query, can i find the function definitions(or sample code) for the FFMpeg commands that we pass through commandline. Please let me know. For a command like ffmpeg -f image2 -r 10 -i %04d.png -vcodec copy output2.avi [to convert a series of images into a video], how can i implement the same using JNI calls from an android activity.Is is possible to call upon any direct function from the FFMpeg library to execute the above mentioned command.Please let me know your views on the same.

    • slhck says:

      Hi! Using the FFmpeg API is not the same as using the command line options. I’ve never used FFmpeg and the JNI, all I can say is that it’s going to be way more difficult than just executing one shell command. So, you have two options: a) Compile a command-line version of FFmpeg for Android or b) learn the FFmpeg API with the JNI. In any case, you’re going to find more info on that on Stack Overflow.

  • Dose says:

    I am trying to convert mkv to mp4. I used following command to convert it.

    ffmpeg.exe -i input.mv -c:v copy -c:a copy output.mp4

    It converted very fast, but when I played there is not audio. Video quality is excellent. Original file has audio properly playing. Can you point me to fix this issue.

  • Vishal Parkash says:

    This is really helpful post.. actually making aware about the software. Good Job man.. very well done..

  • pushpa says:

    I am successfully able to convert a sequence of images into a video. But now my requirement is to put some transition effects like left to right ,top to bottom to be shown in video with the change of every image. Please provide me suitable ffmpeg command to have transition effects in video.

  • Hi, really good post so far I want few questions 1) How to concatenate multiple videos to form a single video 2) How to loop the video until the audio is not stop

    Note: I have a different audio and video file

  • Leave a comment

    Log in
    with Stack Exchange
    or