Skip to content

Repository for RGB-D image Segmentation using the JCSA_RM method.

Notifications You must be signed in to change notification settings

mhasnat/JCSA_RM_Image_Seg

Repository files navigation

JCSA-RM RGBD Image Segmentation and Analysis Method [1,2,3]

Repository for the source code of MATLAB implementation of the "RGB-D image Segmentation using the Joint Color-Spatial-Axial clustering and Region Merging (JCSA-RM)" method.

  • The JCSA-RM method is an RGB-D (joint color+depth) image segmentation method. This repo provides demos with/without a GUI with MATLAB code to perform the following tasks:
    a. load RGB-D image data from a mat file (contains RGB, Depth and Image Normals in an structure) and display them.
    b. Generate segmented image and display it.

How to use demo (tested in Matlab2017b):

Run the MATLAB file name: RGBD_Seg_JCSA_RM.m for the GUI version and demo_NO_GUI.m otherwise
Load data/samples files name: rgbd_info_1.mat, rgbd_info_2.mat, rgbd_info_1_better_normals.mat and rgbd_info_2_better_normals.mat.

  • Select _better_normals in order to experiment with unambiguas surface normals.
  • Choose different methods for testing, among: (a) JCSA, (b) JCSD, (c) JCSA-RM and (d) JCSD-RM.

Application:

This segmentation method has been used to segment/analyze RGB-D images captured by the Microsoft Kinect camera. For details and other possible applications please see the references.

Results to compare:

JCSD_RM_Results.zip file contains the results of applying JCSA-RM [1,2,3] method on the NYU depth database (NYUD2) [4]. Each result file consists of segmentation - labels of pixels and final scores – VoI, BDE, PRI and GTRC for the 1449 NYUD2 [4] images in half scaled (down) image.

Code running issues:

It runs on Matlab2017b. If you encounter error with - computeTraceTerm then go to the directory called 'rgbd' and compile mex file as:
mex computeTraceTerm.cpp.

Extensions and scopes:

  • You can extend the JCSA method for clustering heterogeneous data. However, for now the method is limited to the 3 Dimensional data with the directional distributions [5,6]. You can also extend it to work with higher dimensional data by extending [5] or [6].

  • You can use the RM method independently to perform segmentation. It requires the clustering labels, the color image and image normals as input.

References:

[1] Abul Hasnat, Olivier Alata and Alain Trmeau, Joint Color-Spatial-Directional clustering and Region Merging (JCSD-RM) for unsupervised RGB-D image segmentation, In IEEE Trans. on Pattern Analysis and Machine Intelligence (TPAMI), Vol 38, Issue 11, pages 2255 - 2268, 2015. pdf download

[2] M Hasnat, O Alata, A Trémeau, Joint Color-Spatial-Directional clustering and Region Merging (JCSD-RM) for unsupervised RGB-D image segmentation, in arXiv preprint arXiv:1509.01788, 2015. pdf download

[3] M. A. Hasnat, O. Alata, and A. Tremeau, “Unsupervised RGB-D image segmentation using joint clustering and region merging,” in British Machine Vision Conference (BMVC). BMVA Press, 2014. pdf download

[4] N. Silberman, D. Hoiem, P. Kohli, and R. Fergus, “Indoor segmentation and support inference from RGBD images,” in ECCV 2012. Springer, 2012.

[5] Abul Hasnat, Olivier Alata and Alain Trmeau, Model-Based Hierarchical Clustering with Bregman Divergence and Fisher Mixture Model: Application to Depth Image Analysis, In Statistics and Computing (STCO), Vol 26, Issue 4, pp 861880, 2015. pdf download

[6] Abul Hasnat, Olivier Alata and Alain Tremeau, Unsupervised Clustering of Depth Images using Watson Mixture Model, In Int. Conference on Pattern Recognition (ICPR) 2014, Stockholm, Sweden. pdf download

About

Repository for RGB-D image Segmentation using the JCSA_RM method.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published