Skip to content

Latest commit

 

History

History
145 lines (107 loc) · 5.86 KB

doc_jobs.rst

File metadata and controls

145 lines (107 loc) · 5.86 KB
Raw

doc jobs

A doc job is used for generating API documentation as well as extracting information from the package manifests.

Different types of documentation

Each doc build files can have one of four specific documentation types:

  • the rosdoc_lite type operates on doc repositories. For each doc build file with that type a separate Jenkins view is created. Each doc job operates on a specific branch of a single doc repository and is triggered based on a timer (once every few days). After a partial build step (more about this see below) it uses rosdoc_lite to run various tools for the generation step.
  • the released_manifest type operates on the released packages. For each doc build file with that type a separate Jenkins job is created. It operates on all released packages for which no doc entry is present in the ROS distribution file. The information from the package manifest is extracted to be used in the ROS wiki.
  • the make_target type operates on independent repositories. For each doc build file with that type a separate Jenkins job is created. For each repository a simple make doc command is executed which must honor an environment variable identifying the destination of the generated documentation.
  • the docker_build type also operates on independent repositories. For each doc build file with that type a Jenkins job is created. Each repository must provide its own Dockerfile below docker/image/ (Example: [ros-infrastructure/rosindex](https://github.com/ros-infrastructure/rosindex/tree/ros2/docker/image)), taking and enforcing user and uid arguments appropriately. Provided containers are expected to generate and commit content to the upload git repository mounted at $SITE when run with no arguments, potentially using their own repository content mounted at $REPO.

There is no specific diagram showing the correlation between the various scripts and templates but they follow the same naming scheme as the release and devel jobs.

Entry points

The following scripts are the entry points for doc jobs. The scripts operate on a specific doc build file in the ROS build farm configuration:

  • generate_doc_maintenance_jobs.py generates a set of jobs on the farm which will perform maintenance tasks.
    • The reconfigure-jobs job will (re-)configure the doc jobs with the type rosdoc_lite for each repository on a regular basis (e.g. once every day).
    • The trigger-jobs job is triggered manually to trigger doc jobs with the type rosdoc_lite selected by their current build status.
  • related to the rosdoc_lite type:
    • generate_doc_jobs.py invokes generate_doc_job.py for every doc repository matching the criteria from the doc build file.
    • generate_doc_job.py generates a doc jobs for a specific doc repository for the platform and architecture listed in the doc build file.
    • generate_doc_script.py generates a shell script which will run the same tasks as the doc job for a specific doc repository on a local machine.
  • related to the released_manifest type:
    • generate_doc_metadata_job.py generates a job which extracts the information from all releases packages matching the criteria from the doc build file.
  • related to the make_target and docker_build types:
    • generate_doc_independent_job.py generates a job which either invokes a custom doc target on all repositories listed in the doc build file OR runs source-side containers as provided by each repository listed in the same file, respectively.

The build process in detail

In order to generate API documentation for certain languages the source code for ROS messages etc. must be generated. In order to avoid performing a full build the CMakeLists.txt file provided by each package is replaced with a mock. The mock CMake file will search for ROS message definitions in the recommended locations and perform the generation step if the package depends on the necessary generators. The actual build is performed within a Docker container in order to only make the declared dependencies available.

Known limitations

Several limitations result from the naive approach to simplify the build step:

  • Packages which perform custom logic in the CMakeLists.txt file to e.g. fetch additional content at configure / build time can't be documented as expected since the mock CMake file is not replicating the custom logic.
  • Message files in non-standard locations will not be recognised and therefore no code will be generated for them.
  • Message files in standard location will be recognised even if they are not being used in the original CMake file of the package.

Run the doc job locally

In order to use ros_buildfarm locally you need to setup your environment with the necessary Python packages.

The entry point generate_doc_script.py can be used to generate a shell script which will perform the same tasks as the build farm. It requires certain tools to be available on the local machine (e.g. the Python packages catkin_pkg, rosdistro).

When the generated script is being invoked in runs the generate documentation task in a Docker container.

Example invocation

The following commands run the doc job for the roscpp_core repository from ROS Indigo for Ubuntu Trusty amd64:

mkdir /tmp/doc_job
generate_doc_script.py https://raw.githubusercontent.com/ros-infrastructure/ros_buildfarm_config/production/index.yaml indigo default roscpp_core ubuntu trusty amd64 > /tmp/doc_job/doc_job_indigo_roscpp_core.sh
cd /tmp/doc_job
sh doc_job_indigo_roscpp_core.sh