Author Archives: mosu

MKVToolNix v27.0.0 released

Hey,

I’ve just released MKVToolNix v27.0.0. This release is smallish but important due to two bug fixes. The first is an annoying crash easily triggered on Windows simply by using the program for a while. The second is indubitably more important: mkvmerge was corrupting E-AC-3 frames when the “dialog normalization gain removal” functionality was used.

You can download the source code or one of the binaries. The Windows and macOS binaries as well as the Linux AppImage are available already. The other Linux binaries are stil being built and will be available of the course of the next couple of hours.

Here are the NEWS since the previous release:

New features and enhancements

  • mkvmerge: chapters: the timestamps of chapters read from containers or from
    chapter files can be adjusted (multiplication and addition) with the new
    --chapter-sync option or using the special track ID -2 for the existing
    --sync option. Part of the implementation of #2358.
  • MKVToolNix GUI: multiplexer: adjusted & added controls for mkvmerge’s new
    feature of being able to adjust chapter timestamps. Part of the
    implementation of #2358.
  • MKVToolNix GUI: multiplexer: the GUI can now ask for confirmation when the
    user is about to create a file that won’t contain audio tracks. It does this
    by default if at least one source file contains an audio track. Implements
    #2380.

Bug fixes

  • mkvmerge: AC-3: dialog normalization gain removal was corrupting E-AC-3
    frames irreversibly by writing checksums in places where they didn’t
    belong. Additionally only the first E-AC-3 frame in a Matroska was processed
    but not additional dependent frames in the same block. Fixes #2386.
  • MKVToolNix GUI: fixed a leak of Windows font resources leading to a general
    slowdown and subsequent crash. Fixes #2372.

Have fun :)

MKVToolNix v26.0.0 released

Hey,

time for another release of MKVToolNix, v26.0.0. It’s on the smaller side regarding the number of changes.

You can download the source code or one of the binaries. The Windows and macOS binaries as well as the Linux AppImage are available already. The other Linux binaries are stil being built and will be available of the course of the next couple of hours.

Here are the NEWS since the previous release:

New features and enhancements

  • mkvmerge: chapter generation: if the name template given by
    --generate-chapters-name-template is empty, no names (ChapterDisplay
    master elements with ChapterString/ChapterLanguage children) will be
    generated for the chapter atoms. Part of the implementation of #2275.
  • mkvmerge: chapters: chapter names generated from MPLS files will now use the
    name template if one is set via --generate-chapters-name-template. Part of
    the implementation of #2275.
  • mkvmerge: mkvmerge will no longer abort with an error message if no audio,
    video and subtitle tracks should be multiplexed. This allows copying of
    chapters from non-chapter source files (e.g. Matroska or MP4 files).
  • MKVToolNix GUI: the font size in the tool selector on the left will scale
    with the font size the user selects in the preferences.
  • MKVToolNix GUI: the GUI will no longer automatically resize the columns in
    tree and list views to match the content size. Instead it remembers and
    restores the widths set by the user. Implements #2353.
  • MKVToolNix GUI: multiplexer: the chapter name template will now be set
    automatically to the name template in the preferences’ "chapter editor"
    section. Additionally the option --generate-chapters-name-template … will
    be passed to mkvmerge in situations when mkvmerge will generate chapters
    (either because automatic generation is enabled or if chapters are generated
    for MPLS playlists). Part of the implementation of #2275.
  • MKVToolNix GUI: chapter editor: if the chapter name template is empty,
    chapters will be generated without names. Part of the implementation of
    #2275.
  • MKVToolNix GUI: chapter editor: added an option to remove all chapter names
    to the "additional modifications" dialog. Part of the implementation of
    #2275.

Bug fixes

  • mkvmerge: Matroska reader: fixed wrong timestamps when appending Matroska
    files where the second Matroska file’s first timestamp is bigger than 0.
    Fixes #2345.
  • mkvmerge: MP4 reader: fixed division by zero errors during file
    identification if the timescale is 0 in the MVHD atom.
  • mkvmerge: Windows Television DVR files are now recognized as an unsupported
    file type. This prevents mis-detection as MPEG-2 with an accompanying flood
    of error messages. Fixes #2347.
  • MKVToolNix GUI: info tool: under certain circumstances "cues" were shown at
    the wrong level (inside the previous master element instead of on level
    1). Fixes #2361.
  • MKVToolNix GUI: job queue: fixed invalid memory handling and consequent
    crashes when using the "edit in corresponding tool & remove from job queue"
    option if one of the files in that job contained attached files. Fixes
    #2368.

Build system changes

  • An AppStream metadata file will be installed in $prefix/share/metainfo.

Have fun :)

MKVToolNix v25.0.0

Hello everyone,

July’s release of MKVToolNix is here: v25.0.0. After two smaller ones, this one packs quite a number of bug fixes and enhancements, including fixing a regression in the GUI’s header editor preventing elements from being removed that was introduced in v24.0.0. Sorry about that.

AV1 support hasn’t changed. While the bitstream format’s been finalized in the meantime, the mapping to Matroska & WebM hasn’t. It’s currently being discussed on the CELLAR mailing list.

You can download the source code or one of the binaries. The Windows and macOS binaries as well as the Linux AppImage are available already. The other Linux binaries are stil being built and will be available of the course of the next couple of hours.

Here are the NEWS since the previous release:

New features and enhancements

  • mkvmerge: SRT/ASS/SSA text subtitles: for files for which no encoding has
    been specified, mkvmerge will try UTF-8 first before falling back to the
    system’s default encoding. Part of the implementation of #2246.
  • mkvmerge: SRT/ASS/SSA/WebVTT text subtitles: a warning is now emitted if
    invalid 8-bit characters are encountered outside valid multi-byte UTF-8
    sequences. Part of the implementation of #2246.
  • mkvmerge: Matroska & MPEG transport stream readers: the encoding of text
    subtitles read from Matroska files can now be changed with the
    --sub-charset parameter.
  • Linux: starting with release 25 an AppImage will be provided which should
    run on any Linux distribution released around the time of CentOS 7/Ubuntu
    14.04 or later.
  • macOS: translations: updated the build.sh script to build libiconv and a
    complete gettext. Together with an additional fix to how translation files
    are located, MKVToolNix can now use all interface languages on macOS,
    too. Fixes #2110, #2307, #2323.

Bug fixes

  • mkvmerge: AVC/h.264: fixed file identification failing for certain
    elementary streams due to internal buffers not being cleared properly. Fixes
    #2325.
  • mkvmerge: HEVC/h.265: fixed file identification failing for certain
    elementary streams due to internal buffers not being cleared properly. This
    is the HEVC analog to what was fixed for AVC in #2325.
  • mkvmerge: MLP code: fixed various issues preventing MLP from being parsed
    correctly. Fixes #2326.
  • mkvmerge: TrueHD/MLP packetizer; dialog volume normalization removal isn’t
    attempted if the track is an MLP track as the operation is only supported
    for TrueHD, not MLP.
  • mkvmerge: MPEG TS reader: when reading MPLS mkvmerge will now compare the
    MPLS’s start and end timestamps against the transport stream’s PTS instead
    of its DTS. Otherwise the first key frame of a video track might be dropped
    if it isn’t the first in presentation order. Fixes #2321.
  • mkvmerge: JSON identification: mkvmerge will ensure that all strings passed
    to the JSON output modules are valid UTF-8 encoded strings by replacing
    invalid bytes with placeholder characters. This avoids the JSON library
    throwing an exception and mkvmerge aborting on such data. Fixes #2327.
  • mkvmerge: audio packetizers: mkvmerge will now keep discard padding values
    if they’re present for packets read from Matroska files. Fixes #2296.
  • mkvmerge: Ogg Opus reader: packet timestamps aren’t calculated by summing up
    the duration of all packets starting with timestamp 0 anymore. Instead the
    algorithm is based on the Ogg page’s granule position and which packet
    number is currently timestamped (special handling for the first and last
    packets in the stream).

    • This fixes the first timestamp if the first Ogg packet’s granule position
      is larger than the number of samples in the first packet (= if the first
      sample’s timestamp is bigger than 0). mkvmerge will keep those offsets now
      and inserts "discard padding" only where it’s actually needed.
    • It also improves handling of invalid files where the first Ogg packet’s
      granule position is smaller than the number of samples in the first packet
      (= the first sample’s timestamp is smaller than 0). mkvmerge will now
      shift all timestamps up to 0 in such a case instead of inserting "discard
      padding" elements all over the place.
    • mkvmerge will no longer insert "discard padding" elements if the
      difference between a) the calculated number of samples in the packet
      according to the granule position and b) the actual number of samples as
      calculated from the bitstream is one sample or less and if the packet
      isn’t the last one in the stream. This circumvents certain rounding
      errors.
    • The timestamp of the first packet after a gap in the middle of the stream
      is now calculated based on the Ogg page the packet belongs to, and not
      based on the timestamps before the gap.
      Fixes #2280.
  • mkvmerge: complete rewrite of the progress handling. It’s now based upon the
    total size of all source files and the current position within them instead
    of the number of frames/blocks to be processed. This simplifies calculation
    when appending files and fixes rare cases of when progress report was
    obvious wrong (e.g. stuck at 0% right until the end). Fixes #2150 and #2330.
  • MKVToolNix GUI: header editor: non-mandatory elements couldn’t be removed
    anymore due to a regression while fixing #2320. They can now be removed
    again. Fixes #2322.

Have fun :)

Speeding up MKVToolNix compilation speed with zapcc

Compiling MKVToolNix can take quite a bit of time. It’s a C++ application, it uses a lot of template code, and it doesn’t make use of the pimpl idiom as much as it could. For years I’ve been using the usual several techniques trying to keep the time down: parallel compilation and pre-compiled headers. However, it still takes quite a lot of time, and that’s a bother during development.[1]

That’s why I was instantly stoked when reading about an announcement earlier this weak: zapcc, a clang-based C/C++ compiler heavily tuned towards performance, is being open-sourced. Having more Open Source options in the compiler world is great, having someone working on speed is even better.

zapcc’s web site has some outrageous numbers, toting 40x speedup during re-compilation. That sure sounds like marketing numbers. So how much better is it for my use case, compiling MKVToolNix? Well, look at this:

chart of compilation time of different compilers with different options

This sure looks nice! Here are the actual numbers:

Compiler drake -j1 drake -j5
gcc 8.1.1, no precompiled headers 39:49 15:21
gcc 8.1.1, with precompiled headers 24:34 09:23
clang 6.0.0, no precompiled headers 29:11 12:27
clang 6.0.0, with precompiled headers 18:27 07:05
zapcc revision c2b11ba7 07:16 03:05

Now those numbers explore the whole range between “no help at all” (no pre-compiled headers, no parallelism during compilation, slowest compiler) and “bells and whistles” (maximum parallelism, fastest compiler). A realistic comparison is between my usual setup and the fastest zapcc variant. Those two numbers are the bold ones above: clang using pre-compiled headers with five running compilers in parallel vs. zapcc with five running compilers in parallel.[2] And this is quite remarkable: from seven minutes down to three, down to 43% of the original time. Yes, this does make quite a difference during development.

So if you’re looking for a way to speed up your C++ compilation time, take a look at zapcc. I’m just glad people focus on different aspects of compilers, and us users can profit from them thanks to the compilers being Open Source. A big thanks to all compiler developers!

All tests were done on my Arch Linux installation:

  • MKVToolNix revision 7008661ed951e79c9cc6b7dc167137e84bed8805
  • CFLAGS=-fno-omit-frame-pointer CXXFLAGS=-fno-omit-frame-pointer ./configure --enable-debug --enable-optimization
  • gcc & clang from Arch’s repositories
  • zapcc compiled from git
  • Intel i5-4690K (four real cores, no hyperthreading)
  • 32 GB RAM DDR-3 1600 MHz
  • Samsung EVO 850 SSD
  • no swap space
  • all compiled binaries pass my test suite
  1. [1]I’m also caching compilation results using ccache so that re-compiling is super fast, but that doesn’t help with initial compilation times or if something in one of the central header files that’s included all over the place changes.
  2. [2]zapcc doesn’t support pre-compiled headers, hence only one line for zapcc