KTH Master's thesis in Systems, Control and Robotics titled Feature-Feature Matching for Object Retrieval in Point Clouds.
Please see the final report for details about the project.
The code for this project uses ROS Indigo.
You may need to install some additional packages to get things working.
Documentation is generated by Doxygen. The test framework uses the Google C++ Testing Framework. You can install both of these with
sudo apt-get install doxygen libgtest-dev
As of writing (03/15) The libgtest-dev package on Ubuntu 14.04 does not
install libraries, only the headers. To create the library files you need to
compile them manually (code from
here
and
here
for some pointers).
sudo apt-get install cmake # install cmake
cd /usr/src/gtest
sudo cmake CMakeLists.txt
sudo make
# copy or symlink libgtest.a and libgtest_main.a to your /usr/lib folder
sudo cp *.a /usr/lib
Some of the packages used by the STRANDS project require qt_build and
mongodb_store. On Ubuntu systems set up according to the
tutorial, this can be done
with
sudo apt-get install ros-indigo-qt-build
sudo apt-get install ros-indigo-mongodb-store
You will also require the OpenCV nonfree library. There doesn't seem to be a
package for this in Ubuntu (although libopencv-nonfree-dev existed at some
point). To install, download OpenCV and run the following in the top level of
the extracted zip:
cmake CMakeLists.txt
make
This will take a while to compile everything. It may be possible to just compile
the nonfree library, but I don't know how. Find where other libraries from
OpenCV sit on your system with locate --regexp 'libopencv.*.so'. For me, they
sit in /usr/lib/x86_64-linux-gnu/. Copy the library files to that location.
sudo cp lib/libopencv_nonfree* /usr/lib/x86-64-linux-gnu
And things should compile when catkin_make is run in the workspace which
contains the project. To set up a workspace, see the
ROS tutorial.
In the code/test directory there are some files which you can use to check if
things are properly configured. To compile use
catkin_make run_tests
This will also run the tests and give information about which ones passed or failed (all should pass).
The data used comes from an internal dataset at KTH which consists of multiple frames of rooms taken from a single position within the room. The frames are registered to form a single large cloud of the whole room. Both the full cloud and individual frames are available. Some information about transforms relating the frames to the full cloud are found in an XML file.
All clouds start off in their own frame of reference, so the origin is at the
position at which the camera was when the frames were taken. The complete cloud
can be transformed into the map frame using the transform of the first
intermediate cloud from room.xml. The rooms are in a corridor, and there are
several rooms for which there are scans available, so doing this puts the cloud
in the correct position in the corridor. To transform the individual frames (e.g.
intermediate_cloud0001.pcd) into the map frame, it is necessary to first apply
the transform from the original position to the registered position, and then
apply the same transform as was applied to the complete cloud.
In addition, some annotated clouds of objects present in the rooms are also
available. Each cloud (e.g. rgb_0013_label_0.pcd) corresponds to an object
extracted from one of the intermediate clouds, in this case the 13th. Each cloud
has an associated label found in a text file with a single line (e.g.
rgb_0013_label_0.txt).
Some global parameters can be configured in the
params file. In
particular, the raw_data_dir and processed_data_dir should be set to the
location of the raw data, and where the processed data should be output to.
These values are used in certain parts of the system to facilitate the output of
files to the same subdirectories in the processed directory as the raw
directory. So, for example, if a file is in raw/data/data1/data2, then we
output corresponding processed data to processed/data/data1/data2.
The preprocessing node will reduce the size of the cloud by trimming the floors and ceilings, and removing large planes. It is intended to be run on raw data from scans of a room from a single position. There are numerous parameters which can be specified in the launch file. The default is to carry out all preprocessing steps: downsampling, plane extraction, trimming, annotation rotation and normal extraction.
The result of the processing is output to a processed directory. If given data
in /pcddata/raw/annotated/rares/20140901/patrol_run_31/room_2/, for example,
the resulting processed data will be placed in
/pcddata/processed/annotated/rares/20140901/patrol_run_31/room_2/.
This command will do preprocessing steps on the complete cloud specified, and also process the associated annotations.
roslaunch preprocess preprocess.launch cloud:=/home/michal/Downloads/pcddata/raw/annotated/rares/20140901/patrol_run_31/room_2/complete_cloud.pcd
This command will do preprocessing steps for all complete clouds in subdirectories of the specified directory.
roslaunch preprocess preprocess.launch cloud:=/home/michal/Downloads/pcddata/raw/annotated/rares/
This command will do preprocessing steps for all clouds in subdirectories of the specified directory that contain the string "intermediate". You can also provide regex input in the same position and it should work (untested).
roslaunch preprocess preprocess.launch cloud:=/home/michal/Downloads/pcddata/raw/annotated/rares/ match:=intermediate
The intended use of the preprocessing on intermediate clouds is to generate clouds which can be used to check the efficacy of the system. We extract features from the complete cloud and an intermediate cloud, and then check the correspondences between the extracted features.
roslaunch preprocess preprocess.launch
cloud:=/home/michal/Downloads/pcddata/raw/annotated/rares/20140901/patrol_run_31/room_2/intermediate_cloud0014.pcd
The following will process only annotations, and output them to the specified directory.
roslaunch preprocess preprocess.launch cloud:=/media/michal/Pauli/masterdata/raw/annotated/rares/ output:=/home/michal/Downloads/pcddata/processed/testing/dsannot/0,01 downsample_leafsize:=0.01 downsample:=false planes:=false trim:=false normals:=false annotations:=true
This will compute shot features using uniform interest point selection on all
files that match nonPlanes.pcd in the given cloud directory. The data will be
output to a directory according to the setting of the global parameter
obj_search/processed_data_dir. If this is set to
/home/michal/Downloads/pcddata/processed/, then an example output feature
cloud would be
/home/michal/Downloads/pcddata/processed/paramtest/dt0_02/annotated/rares/20140909/patrol_run_59/room_3/features/nonPlanes<shotcolor_uniform_2015-05-26_11-12-20.pcd,
where the input cloud was
/home/michal/Downloads/pcddata/processed/paramtest/dt0_02/annotated/rares/20140909/patrol_run_59/room_3/nonPlanes.pcd.
roslaunch feature_extraction feature_extraction.launch cloud:=/home/michal/Downloads/pcddata/processed/paramtest/dt0_02 feature:=shot feature_selection:=uniform
One can also specify an output directory using the output parameter. With the following command, the cloud above would be output to /home/michal/dt0_02/annotated/rares/20140909/patrol_run_59/room_3/nonPlanes.pcd.
roslaunch feature_extraction feature_extraction.launch cloud:=/home/michal/Downloads/pcddata/processed/paramtest/dt0_02 feature:=shot feature_selection:=uniform output:=/home/michal
If the input path does not match either the global raw data directory
(obj_search/raw_data_dir) or the processed directory, then the subpath of that
directory is not extracted, and data will be output directly - if one is
processing multiple clouds then this is not ideal because all the results will
be placed in a single location instead of being separated by directories.
Comparing SHOT features extracted from the 14th intermediate cloud to the features extracted from the complete cloud.
roslaunch object_query object_query.launch query:=/home/michal/Downloads/pcddata/processed/annotated/rares/20140901/patrol_run_31/room_2/features/0014_nonPlanes_shot.pcd target:=/home/michal/Downloads/pcddata/processed/annotated/rares/20140901/patrol_run_31/room_2/features/nonPlanes_shot.pcd
Compare shot uniform features from the chair 1 object to the same features for
the room cloud, with some modified parameters. Results will be output to the
directory /home/michal/Downloads/pcddata/processed/query/iter500. A
subdirectory for chair1 will be created there.
roslaunch object_query object_query.launch query:=/home/michal/Downloads/pcddata/processed/testing/querytest/features/rgb_0015_label_chair1\<shot_uniform_2015-05-19_13-24-47.pcd target:=/media/michal/Pauli/masterdata/processed/paramtest/iter500/annotated/rares/20140901/patrol_run_33/room_1/features/nonPlanes\<shot_uniform_2015-05-19_00-51-50.pcd n_max_points:=200 cluster_tolerance:=0.2 K:=5 results_out:=/home/michal/Downloads/pcddata/processed/query/iter500
Run on a target directory, on all feature clouds which match the string nonPlanes<pfhrgb_sift.
roslaunch object_query object_query.launch query:=/home/michal/Downloads/pcddata/processed/query/queryobjects/0,01/top_couch_jacket2/features/rgb_0003_label_top_couch_jacket2<pfhrgb_sift_2015-05-26_15-12-21.pcd target:=/media/michal/Pauli/masterdata/processed/annotated/rares n_max_points:=200 match:=nonPlanes<pfhrgb_sift cluster_tolerance:=0.2 K:=5 results_out:=/home/michal/Downloads/pcddata/processed/query/rares
Having computed various types of features using different interest point selection methods, one can use featureprocess.py to aggregate the data into a usable form. The following command creates files in . which contain the data from all the files in . which have the extension .txt, grouped by which feature was used.
python scripts/features/featureprocess.py -af . `find . -regex .*.txt | xargs echo`
This command is similar to the above, but instead groups by interest point selection method
python scripts/features/featureprocess.py -ai . `find . -regex .*.txt | xargs echo`
Finally, this command can be used to create files which contain only information about the timings for interest point selection.
python scripts/features/featureprocess.py -al . `find . -regex .*.txt | xargs echo`
For a single directory containing query results, with directories for different object types, can use the following to output results to the current directory. Data is aggregated according to the feature type used.
python3 scripts/query/process_results.py -m . `find . -regex .*[0-9].txt`
The switch -mi will aggregate according to the interest point type used.
-m[i]k will also produce results, but will skip SHOT, PFH, FPFH, SIFT and
SUSAN data.