Slimming Down FFmpeg for a Web App: Compiling a Custom Version

Optimizing video and audio workflows for your users

Offical WebAssembly logo with FFmpeg written on it

The capability to compile an application into web assembly was always intreaging to me. It means we can use tried and tested tools on the web. Now that even mobile devices are more powerful than most servers and we can use that capability for free, there is no reason not to do it.

So lets turn it to 11 with Audio/Video processing in the browser.

FFMPEG.WASM

The best open source tool to do that is undoubtedly FFmpeg. It’s command line interface is capable yet simple. And there is already a project called ffmpeg.wasm that does the packaging and provides ready made binaries in the form of the ffmpeg.wasm-core.

There are 2 problems however:

1. Performance. Don’t expect full hd video processing.
2. Size. The final wasm binary is 24.5 mb. Even with gzip its 8.5 mb wich might take some time time to download and increase hosting cost.

Simulated download of 8.5 mb with simulated ADSL 16000

It is probably not reasonable to download such a binary just to compress e.g. an mp3 ahead of an upload for example.

So how can we improve on that?

Compile your own ffmpeg.wasm

The instructions in the readme are actually quiet good.

• Checkout ffmpeg.wasm-core
• init the git submodules
• But instead of running the build-with-docker.sh script, we are going to modify the project a bit.

In wasm/build-scripts/configure-ffmpeg.sh you find some of the parameters FFmpeg compiles with. By default it looks something like this:

#!/bin/bash

set -euo pipefail
source $(dirname $0)/var.sh

FLAGS=(
  "${FFMPEG_CONFIG_FLAGS_BASE[@]}"
  --enable-gpl            # required by x264
  --enable-nonfree        # required by fdk-aac
  --enable-zlib           # enable zlib
  --enable-libx264        # enable x264
  --enable-libx265        # enable x265
  --enable-libvpx         # enable libvpx / webm
  --enable-libwavpack     # enable libwavpack
  --enable-libmp3lame     # enable libmp3lame
  --enable-libfdk-aac     # enable libfdk-aac
  --enable-libtheora      # enable libtheora
  --enable-libvorbis      # enable libvorbis
  --enable-libfreetype    # enable freetype
  --enable-libopus        # enable opus
  --enable-libwebp        # enable libwebp
  --enable-libass         # enable libass
  --enable-libfribidi     # enable libfribidi
  # --enable-libaom         # enable libaom
)
echo "FFMPEG_CONFIG_FLAGS=${FLAGS[@]}"
emconfigure ./configure "${FLAGS[@]}"

As you can see, the default configuration just enables external libraries but not much else.

Run ./configure --help within the project to get a list of options you can use. You are mostly interested in the Individual component options:

  --disable-everything     disable all components listed below
  --enable-encoder=NAME    enable encoder NAME
  --enable-decoder=NAME    enable decoder NAME
  --enable-muxer=NAME      enable muxer NAME
  --enable-demuxer=NAME    enable demuxer NAME
  --enable-parser=NAME     enable parser NAME
  --enable-filter=NAME     enable filter NAME

So what you want to do now is figure out what components you actually need, so lets first look at what they are:

• Encoders and decoders: These components handle the heavy lifting of reading and writing video and audio information. There are a total of 224 encoders and 535 decoders available in FFmpeg. Some of the most commonly used encoders include libx264 (for H.264 video) and libfdk_aac (for AAC audio), while popular decoders include h264 (for H.264 video) and aac (for AAC audio).
• Muxers and demuxers: These components are responsible for packing and unpacking container formats like MP4, MKV, and AVI. There are a total of 165 muxers and 310 demuxers available in FFmpeg.
• Parsers: These components read the output of encoders in order to correctly split it into container formats. For example, if you’re using the libx264 encoder to create H.264 video, you’ll need to include the h264 parser in your build. There are a total of 47 parsers available in FFmpeg.
• Filters: These components allow you to edit the video or audio stream in a variety of ways. Some common filters include scale (for resizing the video or audio stream) and fps (for changing the frame rate). There are a total of 461 filters available in FFmpeg. These might include filters for deinterlacing video, adding text and vector graphics, or creating a color palette for GIFs.

So, let’s say you only transcode h264 video and aac audio, how would that look?:

FLAGS=(
  "${FFMPEG_CONFIG_FLAGS_BASE[@]}"

  # make sure all components are enabled explicitly
  --disable-all

  # external libaries
  --enable-gpl            # required by x264
  --enable-nonfree        # required by fdk-aac
  --enable-zlib           # enable zlib
  --enable-libx264        # enable x264
  --enable-libfdk-aac     # enable libfdk-aac

  # basic requirements to process video
  --enable-protocol=file
  --enable-avcodec
  --enable-avformat
  --enable-avfilter
  --enable-swresample
  --enable-swscale

  # all components we explicitly need
  --enable-demuxer=mov # also mp4,m4a,3gp,3g2,mj2
  --enable-decoder=h264,libfdk_aac
  --enable-encoder=libx264,libfdk_aac
  --enable-parser=h264,aac
  --enable-muxer=mp4

  # filters, that ffmpeg might add automatically
  # insert_trim https://github.com/FFmpeg/FFmpeg/blob/45ab5307a6e8c04b4ea91b1e1ccf71ba38195f7c/fftools/ffmpeg_filter.c#L355
  --enable-filter=trim,atrim
  # configure_output_video_filter https://github.com/FFmpeg/FFmpeg/blob/45ab5307a6e8c04b4ea91b1e1ccf71ba38195f7c/fftools/ffmpeg_filter.c#L428
  --enable-filter=buffersink,scale,format,fps
  # configure_output_audio_filter https://github.com/FFmpeg/FFmpeg/blob/45ab5307a6e8c04b4ea91b1e1ccf71ba38195f7c/fftools/ffmpeg_filter.c#L522
  --enable-filter=abuffersink,aformat
  # configure_input_video_filter https://github.com/FFmpeg/FFmpeg/blob/45ab5307a6e8c04b4ea91b1e1ccf71ba38195f7c/fftools/ffmpeg_filter.c#L710
  --enable-filter=transpose,hflip,vflip
  # configure_input_audio_filter https://github.com/FFmpeg/FFmpeg/blob/45ab5307a6e8c04b4ea91b1e1ccf71ba38195f7c/fftools/ffmpeg_filter.c#L835
  --enable-filter=abuffer
  # negotiate_audio https://github.com/FFmpeg/FFmpeg/blob/41a558fea06cc0a23b8d2d0dfb03ef6a25cf5100/libavfilter/formats.c#L336
  --enable-filter=amix,aresample
)
echo "FFMPEG_CONFIG_FLAGS=${FLAGS[@]}"
emconfigure ./configure "${FLAGS[@]}"

The important part are the --enable-filter statements at the bottom, it took me quiet a bit of research to find all filters that FFmpeg might use implicitly so you don’t have too. There are also the null and anull filters, that you need if you don’t transcode but just remux files.

If you now run ./build-with-docker.sh (wich takes quiet a while) you get a a wasm file that can only do what you specified.

I created a few examples (I haven’t tested all):

• the h264 version from above: 5.2 mb (1.8 mb gzipped)
• an mp3 only version: 1.2 mb (0.5 mb gzipped)
• a version for clip.marco.zone, which can read many common formats but encodes only h264 and gif’s: 8.4 mb (2.8 mb gzipped)

As you can see, throwing away most encoders and filters will drastically reduce the size of the final package. The version I use for clip downloads in 1/4th of the original time.

Download of 2.8 mb gzip version with simulated ADSL 16000

How to use the new files?

After you have compiled ffmpeg, you’ll get 3 files in wasm/packages/core/dist:

ffmpeg-core.js
ffmpeg-core.wasm
ffmpeg-core.worker.js

You’ll have to store those files in your project and then specify the location of the js file in createFFmpeg.

createFFmpeg({
  corePath: new URL(`ffmpeg/custom/ffmpeg-core.js`, document.location as any).href,
});

Are there more reasons for a custom build?

There might be patent issues with libfdk_aac, so you might also choose compiling FFmpeg without it and without --enable-nonfree. FFmpeg has a a free aac encoder that is just called aac which you can use instead which is mostly fine but does not support some of the more advanced profiles.

I hope I could help you figure out how to build a custom ffmpeg.wasm.

Feel free to share projects in the comments, if you want. I’d like to know if this information is useful to someone.

Also check out https://clip.marco.zone/, which is a tool I made to quickly cut and compress videos within the browser. I’m kind of proud of it.

More content at PlainEnglish.io.

Sign up for our free weekly newsletter. Follow us on Twitter, LinkedIn, YouTube, and Discord.

Build awareness and adoption for your tech startup with Circuit.