Modernize (and fix) CircleCI scripts

[vissarion]

Following this PR #1153 I have a couple of questions I want to ask regarding CircleCI.

1. Some tests are missing from the script e.g. algorithms/closest_points, algorithms/convex_hull. Supposedly, this is not intentional, this has to do with the manual assigning of tests to ci jobs. Should we add a job (that requires all other jobs to finish) that contain all tests e.g. a job algorithms that executes all tests on algorithms after finishing jobs like algorithms_area so all missing (an possible newly added) algorithms are tested? Otherwise, we have to manually add the missing tests and update the script every time we create a new test directory. Since, I am not a yaml expert there could be probably a more delicate solution to this...
2. Should we update the deprecated Docker convenience image from circleci/ruby:2.3 to cimg/ruby:2.3.0 or even to some base ubuntu image (do we use Ruby at all?). Currently, these are some tests failing due to some debian stretch repositories missing. This could be fixed in current image but why not upgrading to a newer one? Finally, if we move to a newer image we should probably still test our suite with gcc-6 see this discussion.

I am willing to implement any changes/fixes but I need your feedback @awulkiew @barendgehrels @mloskot

[mloskot]

(...) this has to do with the manual assigning of tests to ci jobs.

Yes, tests are listed and arranged manually in .circleci/config.yml and that is to fine-tune order and dependency of tests, and that is to optimise CI build times.

Should we add a job (...)

A 'closing' job making up the fan-in workflow seems like a good idea, assuming it would be no-op for already built tests and will only do any work for tests which are not explicitly listed indeed.

Since, I am not a yaml expert there could be probably a more delicate solution to this...

I'm quite sure there is any such solution to be expressed in YAML and a proper scripting would be required.
An alternative to the idea of a concluding job is to have a script that runs between these two steps
https://github.com/boostorg/geometry/blob/c479ca194e8b491de9ebbe0e9d6afd3e1968826e/.circleci/config.yml#L52C17-L53
and collects names of all tests from Jamfile-s or .cpp files and compares against tests listed in the .circleci/config.yaml.

The config.yaml can be scanned with the powerful yq which can output it grep-friendly or convert to JSON and then a tool called gron could be used - mikefarah/yq#931

do we use Ruby at all?

I can't recall why the Ruby image is used. It's been there since 2018, so possibly it was a hacky way to get an image with C++ toolset.

I don't see why those images shouldn't be upgraded.

[barendgehrels]

Is circleci still used? I think it's all github actions now?
If not, can we close it?

[vissarion]

Yes circleCI is still used. In particular, circleCI runs the full test suite on ubuntu with gcc. Then github actions are testing the minimal test suite on linux/mac/win with gcc/clang. Actions also test headers, documentation.

[vissarion]

There is also the cmake action which is actually equivalent to circleCI but with cmake instead of b2.