GNU Linux Packaging Deploying Qt on Linux

Packaging for GNU/Linux

Qt5 (as of 5.11) gives the developer options for deploying Qt5 programs into compiled binaries for Mac and Windows, but not GNU/Linux platforms. This page will detail the approach used to circumvent this and why that approach was taken.

Linuxdeployqt

A brief google search for a qt deployment tool for GNU/Linux will reveal the linuxdeployqt repository. This tool is perfect for what CaPTk needs as it produces an AppImage as the result. From the description of linuxdeployqt:

"This Linux Deployment Tool, linuxdeployqt, takes an application as input and makes it self-contained by copying in the resources that the application uses (like libraries, graphics, and plugins) into a bundle. The resulting bundle can be distributed as an AppDir or as an AppImage to users, or can be put into cross-distribution packages. It can be used as part of the build process to deploy applications written in C, C++, and other compiled languages with systems like CMake, qmake, and make. When used on Qt-based applications, it can bundle a specific minimal subset of Qt required to run the application."

CaPTk in particular benefits greatly from being distributed in the AppImage format. AppImages are basically secluded sandboxes that ship their own software versions and libraries directly into a single binary, meaning that you are guaranteed to have a binary that can not only work across distributions of GNU/Linux, but also different kernel versions, systems with out of date versions, and systems that don't even have Qt or a C++11 compatible compiler.

Setting up linuxdeployqt

Linuxdeployqt requires a strange directory structure to build an AppImage from, as it's different from what Windows and Mac do. The layout looks like:

└── usr
    ├── bin
    │   └── your_app
    ├── lib
    └── share
        ├── applications
        │   └── your_app.desktop
        └── icons
            └── <theme>
                └── <resolution> 
                    └── apps 
                        └── your_app.png

Cmake upon a normal make install will send all of the files to ./install, which is not what we want. We use the command cmake -DCMAKE_INSTALL_PREFIX="./install/appdir/usr" .. to establish where cmake will put our bin and lib folders. The share folder is created as a part of our packaging script, captk-pkg. The script, before compiling an AppImage, will populate the share folder with the .desktop file and CaPTk icons.

However, even if the AppImage was buildable at this point, we'd run into a bunch of other issues.

Fixing documentation, LIBRA, and the WebEngine

Before an AppImage can be produced, this directory structure creates some issues. Once the AppImage is built, CaPTk can't dynamically link to the docs or the Matlab runtime for LIBRA as those have not been added as a part of our install step. In addition, several files are intended to be used on GNU/Linux but were made on a Windows machine, meaning that every time you check them out, their permissions are broken which results in a useless CaPTk AppImage. In addition, if we packaged the AppImage now, Qt's WebEngine won't signal to linuxdeployqt that the NSS libraries need to be pulled in (since the WebEngine is just a chromium instance) meaning that the splash screen, and documentation rendering ceases to function (The docs are even installed anyway!).

Once again, our packaging script handles these cases as a part of the FIX_CMD procedure. This procedure:

1. Makes LIBRA usable
2. Manually add the user's NSS libraries to the lib folder
3. Marks files as executable
4. Runs dos2unix on select files (LIBRA and Confetti)
5. Moves the docs folder into the install tree

Using linuxdeployqt & Ignore Glob

Our packager after issuing the various fixes will then:

1. Pull down the latest version of linuxdeployqt
2. Mark it as executable
3. Run $ ./linuxdeployqt-5-x86_64.AppImage ./appdir/usr/share/applications/*.desktop -appimage

But this actually doesn't work. See, when we pulled in the Matlab runtime to our install tree, linuxdeployqt says that we need to pull in Matlab libs that don't exist. After all, an AppImage needs to contain all of the relevant libraries to be completely frictionless, and interprets Matlab as wanting to pull in libraries it actually doesn't need, since it already comes with everything it wants. Linuxdeployqt can't tell the difference but this was foreseen as a special flag was added for situations like this. The solution is to use the -ignore-glob flag of linuxdeployqt, however we need a regex to exclude both a subdir of lib folder (snap contains all of it's necessary libs but suffers from non-existent libraries as well) AND all of Matlab's libraries. So our finished command to deploy an AppImage is:

$./linuxdeployqt-5-x86_64.AppImage ./appdir/usr/share/applications/*.desktop -ignore-glob=usr/{libexec/MCR/**,lib/snap-3.6.0-rc1/**} -appimage

Our AppImage is now complete, but now we have another problem: How will users agree to the combined licenses?

The Installer

CaPTk does not natively make users agree to the plethora of licenses it is using from it's dependencies on the initial startup. Now we need to further wrap our AppImage.

Makeself is a tool to make self-extracting zip files that are accessible through a shell script. It was chosen because a zip file should reduce the size of our resulting installer and because using a shell script to agree to a license has the least friction as /bin/bash is pretty much guaranteed to be present. Considering how our packager is already written in bash, this means any future maintainers will not need to split focus across multiple languages.

So now captk-pkg, after jumping through all the previous hoops, will then download makeself, create a directory including the CaPTk AppImage, the licenses, and the makeself script that will be ran when the user goes to install the software (linux-makeself) and then run $ ./makeself-2.4.0/makeself.sh --gzip pkg/ CaPTk_${ver}_Installer.bin "CaPTk Linux Installer" ./linux-installer to package an installer.

linux-makeself, when run, will display the licenses and prompt for an explicit yes/no answer. It will then pedantically check to make sure that there's enough space on the drive and that no conflicting filenames exist, and self-extract the AppImage and *.cwl files to a directory. Once done, it creates a symlink to CaPTk.bin (our AppImage), and it's done.

Summary

Here's a quick rundown of how this is deployed:

We use makeself to create a self-extracting archive accessible through a shell script that asks users to agree to the licenses which then extracts the archive which contains an AppImage made via linuxdeployqt that contains every binary file, documentation file, and library needed by the program (but explicitly doesn't pull in whatever matlab and snap want because they already have their necessary libraries), as well as the numerous fixes that need to be present to get a complete piece of software on GNU/Linux platforms.

It is complicated, but we have tools to automate it. captk-pkg does all of this automatically, and contains relevant comments and documentations to help future maintainers/collaborators through the process.