Skip to content

Developing Features with Git

Jump to bottom Edit New page
Mael Rouxel-Labbé edited this page Mar 28, 2024 · 1 revision

Table of Contents

Any change to the existing code of CGAL, such as the introduction of a new function or functor in an existing file, the fixing of a bug in a package, or the refactoring of some code in various packages, is classified either as a feature or as a bug fix. Features must be reviewed by the Editorial Board. The development of features or bug-fixes should follow a certain procedure described in this page.

A new feature must be developed in a dedicated branch called the feature branch. This branch is cloned from the master branch. The feature branch, just as the master branch, supports the branch-build, which allows the building of CGAL from the sources in their package structure. This is advantageous especially in cases where new features alter the generated library objects. Once the development of a feature is complete, the branch with all the modifications is merged back into master. The release manager clones master to create the branch for upcoming public release a short period before the release. This way new features end up in the upcoming release.

The following assumes that

  1. you are familiar with CGAL's SCM-layout and which branches exists, as described in the guidelines,
  2. you have cloned the remote Git repository, and
  3. took a crash course on git commands (our Quickstart), or for example: this one (from Section 2.4 onwards), as this documentation is not a general introduction to GIT. It will only list the most important use-cases for developing code in CGAL using Git as SCM., and
  4. your version of Git is at least 2.23 (which came up with a new command git switch).

This page discusses the case of adding a package. The same steps similarly apply for adding a demo/example or non-trivial bug-fixes. At the beginning Jenny enters her local Git repository.

> cd /path/to/my/cgal_repository

Opening a feature branch

Assume that Jenny would like to develop a new package named Kinetic_arrangement_2.

First, following the quick start Jenny creates her own feature branch (based on the master branch). Following the naming recommendations, Jenny calls the branch Kinetic_arrangement_2-jenny.

> git switch --create Kinetic_arrangement_2-jenny cgal/master

This is a local branch, and not visible to anyone besides Jenny until the first call to the git push command (see below).

Note: if you already have made a "pull request", your local repository has a remote branch named mine/master. That branch is useless. It is a leftover of the fork. Unless you have regularly merged cgal/master into it, mine/master branch is probably obsolete.

If your branch is a bug-fix branch, and you know the bug was already in a released version of CGAL, you may want to open the feature branch from a release branch, instead of master. For example:

> git switch --create AABB_tree-bug_fix-jenny cgal/5.3.x-branch

Coding

Now, Jenny can implement the package. She is iterating the following commands until the package is implemented.

> mkdir Kinetic_arrangement_2
# [hack boom bang]
# Rebuild CGAL with branch-build to include the package in the build system
# Use `git add Kinetic_arrangement_2/path/to/file.cpp` to add new files to the source code management.
> git add <all-new-relevant-files>
> git commit <files>

This opens an editor to enter the commit message. Please use the established format for Git repositories.

If she wants to share the new feature with others, or have a back-up, she can push the new commits to the remote repository:

 > git push -u cgal-public-dev Kinetic_arrangement_2-jenny

Now her branch is visible to everyone with access to the remote cgal-public-dev. Once the first push of the branch has been done, the following ones no longer need the -u cgal-public-dev option, and are done simply with

 > git push

Otherwise, she could postpone this step until major progress took place. Note: There are techniques (not explained here) to combine several small commits into one bigger.

To deprecate something, she should make modifications in the code and in the documentation.

  • In the .h code-file that she wishes to deprecate, she should add the following :
#define CGAL_DEPRECATED_HEADER "<CGAL/old_header_file.h>"
#define CGAL_REPLACEMENT_HEADER "<CGAL/new_header_file.h>"
#define CGAL_DEPRECATED_MESSAGE_DETAILS "Additional information about the depreciation"
#include <CGAL/Installation/internal/deprecation_warning.h>

The inclusion of the header deprecation_warning.h will output a warning (or an error if the macro CGAL_NO_DEPRECATED_CODE is enabled) during compilation. Note that all three macros are optional; if they are defined, their values will be printed along with the warning/error.

  • In the .h file that contains the documentation, in the description of the class she should add :
\deprecated This class is deprecated since N.M. The class `New_class` should be used instead.

Once the package is ready, she submits the documentation of the package to the editorial board. She gets some feedback and improves the package according to the review.

Synchronizing the feature branch

The CGAL project has an integration branch that is used to integrate new features to the recent development that took place on master and by the nightly testsuite. Thus, Jenny delays the integration until her feature has been accepted.

To remember: In order to avoid unnecessary dependencies and to keep the shape of history simple, Jenny follows the procedure of section "My branch is really old. I would like to update it to get latest update from master." of our Git FAQ to update her feature branch.

The merge of cgal/master should stay an exceptional procedure, and only done when that is really necessary.

Integrating the feature branch and adding it to the testsuite

Once Jenny got the feature accepted by CGAL's Editorial Board, she gains the right and obligation to test her feature on various platforms and to ensure the interoperability with other parts of the library. The Testing chapter explains how to test code.

Then Jenny can ask her branch to be merged into master by opening a Pull Request. Once the pull-request passes the continuous integration tests (see below), the release management team will eventually select the pull-request to be tested in the testsuite, by merging it in the integration branch from the cgal-dev repository.

Continuous Integration

GitHub uses a continuous integration process to swiftly notice errors in a pull request without having to wait for the nightly batch. It relies on GitHub Actions. Config files are available here, one file per action.

Integration Branch

Notice that a snapshot of the integration branch is taken every day exactly at 21:00 CEST (local time in France: UTC/GMT + 1 hour + Daylight saving time when applicable) and the integration branch is reset to reflect the master branch immediately after.

Finally, Jenny waits for the results reported on the test results web page on the following day (assuming that she completed the steps above before the snapshot was taken). She can click on the y/w/n/r symbols in the table to see for which commit the test suite was run. If the testsuite is green, her feature can be merged into the master branch upon approval of the release manager (see below).

If not green, note that integration gets thrown away and is reset to the content of master each day, during the daily internal release creation. As a consequence, if one day Jenny merges her branch into integration so that it is tested during the night, and if the tests were not fully successful, Jenny will have to remerge again the branch into integration the day after (i.e. redoing the steps above). Anyway, if the tests were not fully successful (e.g. the feature did not work correctly on all platforms), she has to modify something in her code and remerge into integration. This strategy shows Jenny that she is responsible to get her feature properly tested and if successful to quickly merge into in the stable master branch.

In order to check previous runs of the testsuite, the Git repository will keep a sort of "cyclic backup" of all branches that were tested during the last week, as branches in the cgal-dev repository:

  • testsuite-Monday
  • testsuite-Tuesday
  • and so on (seven such branches)

Merging the feature branch to master

Once a pull-request has been successfully tested in the testsuite, the release management team will merge it in master to integrate the new feature or bug-fix.

Closing the feature branch

At the end of the reintegration Jenny removes her feature branch, which has reached the end of its life-time with the merge into master.

# change to the master-branch
> git checkout master
# delete the local branch
> git branch -d Kinetic_arrangement_2-jenny
# delete the branch on cgal-public-dev
> git push cgal-public-dev --delete Kinetic_arrangement_2-jenny

Remarks:

  • Removing the local branch is actually not a requirement. What happens in your own repository is your responsibility.
  • Once a branch is merged into another, even if all the references to the branch has been removed, the branch itself still exist in the history of the branch where it has been merged. Thus, it is possible to revive old feature branches, using for example the procedure described on the Kitware wiki.

Problem with end-of-line encoding in a feature branch

The CGAL repository used to include a few files with crlf, and crlfcr end-of-line encoding. These have been fixed in master by updating the .gitattributes (6b43b44e642a560cf63f6dac3438298cbd3f4de4) and problematic files (1adf441b18227f2e064abe59173a6fb7a6e48d65 and 7d20531b1def82fe7f8bcc1bb557c3a350b23270). In your feature branch created before those fixes, it might happen that you need to modify the re-encoded files. In order to avoid a problem while merging your branch to master, it is advised to do the following:

# stash current work
> git stash
# update the .gitattributes
> git cherry-pick -x 6b43b44e642a560cf63f6dac3438298cbd3f4de4
# resolve the conflict by accepting the new version
# -x write the picked hash to the commit message, -ff tries a fast-forward
> git add .gitattributes
> git commit
# force re-encoding of eol
> rm .git/index
> git reset
> git commit -a -m "Convert all CRLF files to LF"
# then pop the stash to retrieve the modifications you have stashed
> git stash pop

(An alternative to be checked could be to set git config merge.renormalize true as written in this thread 5

How to develop several features in parallel

How to work on a different branch?

Assume Jenny is developing several features in different branches. With Git it is very easy to deal with different (local) branches. Before she checks out another branch, she ensures that her workspace has no modifications, that is, git status reports no alterations. If there are, she either commits them to her local repository (see above) or stash them onto a stack (not explained here).

To checkout her feature branch Convex_hull_2-make_it_faster-jenny she uses the following command:

 > git checkout Convex_hull_2-make_it_faster-jenny

NOTE: In contrast to SVN, Jenny is not required to be in the root of the branch to switch them. Git-commands likes status or checkout can be called in any subdirectory and still work on the whole branch.

Future commits from that working copy will be made to the branch Convex_hull_2-make_it_faster-jenny (until the next use of git checkout command).

Then, she can continue developing the new feature and commit changes into the right branch. If she adds a new package, a new branch-build may be required to include the new package in CGAL.

Which branch Jenny is currently working on?

When Jenny develops more than one feature in several feature branches she may frequently checkout various branches. If at any point Jenny is uncertain which feature branch is currently checked out she can issue the following command:

 > git branch Kinetic_arrangement_2-jenny

At this point, the repository's workspace contains a full checkout of CGAL, with all the packages, representing the current state of a branch.

Some shells can be extended to give the branch name in the prompt. See, e.g. 6 (untested). Ask a search engine if you want to have this gimmick. There is also tab-completion for branch names available.

Building after switching/updating

For now, it is compulsory to rebuild CGAL each time a working copy either is switched from one branch to another or if the branch is simply updated.

Remarks:

  • In fact, rebuilding CGAL is only needed if a cpp file has been added, removed or altered or include dirs have been added or deleted during the switch/update
  • As CGAL mainly consists of header files, actual rebuilds are often not needed.
  • For the same reason, it is actually not a big harm to rebuild, as this is done within less than a minute.
  • There is a chance that in the future a warning/error is reported if build and working copy do not match.

Comment: Jenny can also maintain several working directories in parallel from one repository. This is recommended if switching branches happens quite often. On the other hand, Git encourages to often commit (to the local repository, while pushes to the central repository can be less often), and such a single checkout should usually not be dirty (and even if: git stash offers a nice way to temporarily clean up - and to get it dirty again.

How to use another feature in own feature branch

At some point Jenny notices that she needs, in her branch Convex_hull_2-make_it_faster-jenny, a feature that has been implemented by Adam in the STL_extension-new_iterator-adam branch. It is required that Adam's feature is already integrated. Then, Jenny has to revive his feature branch and merge it into her branch; she never merges the branches integration, master or maint into her feature branch! We follow roughly the ideas in 7 with details from 8 and 9 (note that their next is our integration branch):

 # update the local copies of the remote branches cgal-public-dev/* &gt; git fetch cgal-public-dev

Use

 > git log --first-parent cgal/master

to find the merge-commit for Adam's feature. The commit message gives you the name of Adam's branch. The second parent of the commit (see the message) is the end of Adam's feature branch, say 0a398e5. Next Jenny creates a branch from this commit

 > git branch STL_extension-new_iterator-adam 0a398e5

Now Jenny can merge that branch into her feature branch

> git checkout Convex_hull_2-make_it_faster-jenny
> git merge STL_extension-new_iterator-adam
# resolved conflicts!
# and delete Adam's revived branch
> git branch -d STL_extension-new_iterator-adam

This way, only Adam's feature is merged into Jenny's branch, while all other new features on integration are excluded. Thus Jenny's feature has no dependencies to those features. Furthermore there is a crisp commit message for the merge "Merge branch 'STL_extension-new_iterator-adam' into 'Mesh_3-jenny'" which is much more appealing than the merge for the whole integration branch.

Downside: Jenny's feature is now dependent on Adam's.

Developing and committing to several orthogonal feature branches

There are scenarios where Jenny wants to compile programs with sources from several branches, like

  • preparing some experiments for a paper
  • providing a demo or a web service with experimental code (e.g. for reviewers of papers)

There are two ways:

  • Checkout several branches and tweak CMakeLists.txt of your executable to collect the right includes from several checkouts.
  • Create a new local branch Combined-jenny and merge all relevant feature branches into it: git merge branchA branchB [... branchG]

The former feels more dirty and less gitty. The latter has the problems that commits are expected on the individual feature branches and in contrast to Combined-jenny. In the following Jenny decides for the second way. Her rationale is to submit the features independently! But she is aware of the fact that this is only recommended if her commits are rather orthogonal across the branches (e.g. adding GPU support to an algebraic kernel in one branch, and implementing a new point location in arrangements in another branch). As soon as a commit in one branch requires a commit in the other (e.g. a change in the interface) the following method is not recommended.

Combining orthogonal feature branches

Sharing your branches with the outside

If Jenny wants to share a branch with a person who does not have access to a private remote, she has two solutions:

  • The branch is not sensitive and can be made public: she can push it into her own cgal fork (mine). The procedure is the same as when she prepares a "pull request".
  • The branch is sensitive and cannot be made public yet: If not already done, create a fork of the repository private-fork by visiting https://www.github.com/CGAL/private-fork and clicking on the fork button. she can now visit https://www.github.com/jenny/private-fork, click on Settings and add Collaborators to which she wants to give access to the private repository. She has to update her git config to add this new repository as a remote:
> git remote add private-mine git@github.com:jenny/private-fork.git  #assuming your GitHub ID is jenny
> git fetch private-mine

Then pushing a branch Pkg-modif-jenny can be done using:

> git checkout Pkg-modif-jenny # switch to the branch from cgal-dev you want to share
> git push mine Pkg-modif-jenny # public option
> git push private-mine Pkg-modif-jenny # private option

Note that if Adam or Jenny did a modification that (s)he pushed into the private repository, when ready the other remotes can be updated by simply pushing the branch into them.

Home > Developer Resources

Home

General Information

Developer Resources

Maintainer Resources

Clone this wiki locally