Wilhelm Meier <wilhelm.meier@hs-kl.de>

Table of Contents

• 1. Overview
• 2. Prerequisites
  • 2.1. Compiler
  • 2.2. Libraries
  • 2.3. Tools
• 3. Installation
  • 3.1. Build
    • Linux
    • 3.1.1. Windows
      • 3.1.1.1. Windows 10
• 4. Usage

Extractor: a source code extractor

1. Overview

Extractor is a tool for extracting information from C++ or Java source files.

If you write technical documents like program documentation, how-to’s (like this) or other mostly software oriented documents, it is a very common requirement to show some source code. To limit the chances of documentation errors, these source code examples should be real world examples. This means, that they should be extracted from real code examples or production code and they should not be cut-and-pasted into the document or originally be written in the document. This leads to untested and unchecked examples with a high risk of being wrong (or outdated).

AsciiDoctor is capable of including source code files into the document.

But real world examples are mostly too long to be fully included. It would be much more readable if you could focus on the important parts of the source code and leave unrelevant parts out. This is what extractor is for:

• extracting parts (snippets) from real world source code.

• omitting lines of the source code

• generating auto-callouts from comments of the source code

• highlighting parts of the source code

2. Prerequisites

2.1. Compiler

You need a C++ compiler that supports at least C++17 (both GCC and CLang will do).

2.2. Libraries

The following system libraries are required:

• from Boost: Boost-Regex (header-only library)

• AStyle

• Magic Number Recognition Library

2.3. Tools

Not strictly needed, but helpful: GNU Source Highlight.

And of course: AsciiDoctor

3. Installation

After cloning (or downloading) the repository you can build the tool by utilizing make.

3.1. Build

Linux

First navigate into the root folder of extractor:

Command:1.

---
$ cd /to/your/location/of/extractor
---

Then simply type:

Command:2.

$ make all

Optional you can also add the "installation_home/extract" to your PATH environment variable so that you may start extractor from any directory.

3.1.1. Windows

3.1.1.1. Windows 10

The most simple way of building the extractor on Windows 10 is using the Windows Subsystem for Linux. Therefore, after installing the subsystem, one can follow the installation instructions of the {linux_build} section.

4. Usage

The following example will show the basic idea behind extractor.

Suppose you’ll document the following very simple C++ example:

Zeilen aus der Datei test.cc

#include <iostream>
#include <iomanip>
int main() {
    return EXIT_SUCCESS;
}

It could be that you want to describe the portion of the souce file where the include directives are in a special way. Therefore it would be nice to extract that part of the file like:

Zeilen aus der Datei test.cc [Snippet: include]

#include <iostream>
#include <iomanip>

In a later section of your documentation you want to emphasize the int main() function:

Zeilen aus der Datei test.cc [Snippet: main]

int main() {
    return EXIT_SUCCESS;
}

For this to be possible without copying manually anything from your sources you have to mark these parts directly in your sources. These parts are called snippets.

A simple source snippet definition

//[<name> (1)
...
//] (2)

1. Begin of snippet name

2. End of snippet name

Caution

Source snippets resemble the AsciiDoctor feature of include tags, but they have to be strictly nested. They must not overlap!

With this you can annotate your real source code with the neccessary snippet definitions:

The file test.cc with the source snippets include and main

//[include
#include <iostream>
#include <iomanip>
//]
//[main
int main()
{
    return EXIT_SUCCESS;
}
//]

Then you run the extractor for your file test.cc:

Command:3.

$ extractor test.cc

The outcome from this is the file test.extractor with the contents:

The Content of the snippet database file

Snippet [ all [ ( 0 , 6 ) ] exclude [  ] ]
Snippet [ include [ ( 0 , 2 ) ] exclude [  ] ]
Snippet [ main [ ( 2 , 6 ) ] exclude [  ] ]

At the moment this file isn’t very useful (except if you want to automate building the whole documentation, this file will become very handy), but if you look carefully into the directory of the file test.cc you’ll find a newly created directory named .extractor. The files within this directory contain the snippets, e.g. the file test.cc.main obviously contains the snippet main of file test.cc. The file test.cc.all contains the full file test.cc but without the snippet definitions.

In your AsciiDoctor documentation files you can include these snippet files. Especially useful are definitions of some attributes like srcbase, srcdir and extractdir:

include::{srcbase}/{srcdir}/{extractordir}/test.cc.main[]

The content of a snippet file is asciidoc:

.Zeilen aus der Datei link:{srcbase}/{srcdir}/test.cc.html[`test.cc`,window="newwindows"]
[Snippet: main]
[source,cpp,indent=0]
----
int main() {
    return EXIT_SUCCESS;
}
----

Tip

Please note that this file contains asciidoc syntax, e.g. this simple case generates a caption with a link to the file containing the displayed snippet. If you click on the link, you should get a new browser window with the original code highlighted via source-highlight as a html-file (s.a. [make]). If the snippet contains auto-callouts these will also be collected into the snippet file (s.a. [callouts]).

If you use the above include macro in your documentation you’ll get the following result:

Zeilen aus der Datei test.cc [Snippet: main]

int main() {
    return EXIT_SUCCESS;
}

For further more detailed information about extractor and his various features read the documentation.