Speeding up MKVToolNix compilation speed with zapcc

Compiling MKVToolNix can take quite a bit of time. It’s a C++ application, it uses a lot of template code, and it doesn’t make use of the pimpl idiom as much as it could. For years I’ve been using the usual several techniques trying to keep the time down: parallel compilation and pre-compiled headers. However, it still takes quite a lot of time, and that’s a bother during development.[1]

That’s why I was instantly stoked when reading about an announcement earlier this weak: zapcc, a clang-based C/C++ compiler heavily tuned towards performance, is being open-sourced. Having more Open Source options in the compiler world is great, having someone working on speed is even better.

zapcc’s web site has some outrageous numbers, toting 40x speedup during re-compilation. That sure sounds like marketing numbers. So how much better is it for my use case, compiling MKVToolNix? Well, look at this:

chart of compilation time of different compilers with different options

This sure looks nice! Here are the actual numbers:

Compiler drake -j1 drake -j5
gcc 8.1.1, no precompiled headers 39:49 15:21
gcc 8.1.1, with precompiled headers 24:34 09:23
clang 6.0.0, no precompiled headers 29:11 12:27
clang 6.0.0, with precompiled headers 18:27 07:05
zapcc revision c2b11ba7 07:16 03:05

Now those numbers explore the whole range between “no help at all” (no pre-compiled headers, no parallelism during compilation, slowest compiler) and “bells and whistles” (maximum parallelism, fastest compiler). A realistic comparison is between my usual setup and the fastest zapcc variant. Those two numbers are the bold ones above: clang using pre-compiled headers with five running compilers in parallel vs. zapcc with five running compilers in parallel.[2] And this is quite remarkable: from seven minutes down to three, down to 43% of the original time. Yes, this does make quite a difference during development.

So if you’re looking for a way to speed up your C++ compilation time, take a look at zapcc. I’m just glad people focus on different aspects of compilers, and us users can profit from them thanks to the compilers being Open Source. A big thanks to all compiler developers!

All tests were done on my Arch Linux installation:

  • MKVToolNix revision 7008661ed951e79c9cc6b7dc167137e84bed8805
  • CFLAGS=-fno-omit-frame-pointer CXXFLAGS=-fno-omit-frame-pointer ./configure --enable-debug --enable-optimization
  • gcc & clang from Arch’s repositories
  • zapcc compiled from git
  • Intel i5-4690K (four real cores, no hyperthreading)
  • 32 GB RAM DDR-3 1600 MHz
  • Samsung EVO 850 SSD
  • no swap space
  • all compiled binaries pass my test suite
  1. [1]I’m also caching compilation results using ccache so that re-compiling is super fast, but that doesn’t help with initial compilation times or if something in one of the central header files that’s included all over the place changes.
  2. [2]zapcc doesn’t support pre-compiled headers, hence only one line for zapcc

One thought on “Speeding up MKVToolNix compilation speed with zapcc

  1. OldskoolOrion

    Thanks for pointing me towards zapcc !
    I clocked 2.3x faster compile time on my current project compared to the clang compiler and even more compared to the Intel compiler.So while developing it’s most definitely a keeper :-)
    Releases for now I will however still compile with the Intel compiler suite.. the output and reports are superior and I have a strong suspicion timing the performance of the executable that Intel delivers far more optimized executables. (didn’t really benchmark and compared to full extend, and I need to put more time in tweaking and fiddling around with everything – so it’s not set in stone or an indicator of what will be in the future hehe).

    I’m hella excited about slashing waiting time to half during dev-work tho :-) that’s huuuuuuggeeee… and I’m instantly not bitching anymore about my old workstation anymore.. my dual 2680v1 setup feels.. like… snappy again lol :-)
    (I’ll give it two days and start bitching again tho.. I want new shit ! Ï needs the precious ! :P)

    Thanks for the great tip because I totally missed zapcc somehow :-)

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *