3D mesh optimization strategy

I developed a 3D mesh optimization strategy for my Bachelor's thesis in 2022. It is based on the pipeline proposed by [CV14]. You can find the abstract of the thesis at the very end of this README (below sources). A zebrafish (Danio rerio) embryo model and a more complex mouse mammary gland model were used. This pipeline consists of a couple of steps:

0. Data acquisition (done by specialists)
1. Image alignment
2. Point cloud extraction (including creating an oriented point cloud)
3. 3D reconstruction (Screened Poisson, Alpha Shapes and Ball Pivoting)

Step 1 is done using the OpenCV library [Bra00]. Step 2 and 3 are done using the Open3D library [ZPK18]. Step 3 is partly done using PyMeshlab [MC21].

Extra information about step 1 and 2: For finding the contours and shifting the contours, the tutorial by [R21] and the OpenCV documentation [Ope22] were used. For extracting the point cloud, the Open3D documentation [ZPK18] was used.

Extra information about step 3: For the 3D reconstruction code, the Open3D [ZPK18] and PyMeshLab [MC21] documentations were used.

1. Screened Poisson Method proposed by [KH13]. This was done using the function in PyMeshlab. No extra pre or post processing was done except for the generation of an oriented point cloud.

2. Alpha Shapes Method proposed by [EM94]. This was done using the function in PyMeshlab. This step required extra post processing in Meshlab, namely using the Taubin Smooth [Tau95] function.

3. Ball Pivoting Method proposed by [BMR+99]. This was done using the function in the Open3D library. This step required extra post processing in PyMeshlab, namely closing the holes using the Close Holes function, and surface smoothing using the Taubin Smooth function.

Sources

This section contains the sources from the libraries and papers that developed/proposed the used reconstruction algorithms.

[BMR+99] F. Bernardini, J. Mittleman, H. Rushmeier, C. Silva, and G. Taubin. The ball-pivoting algorithm for surface reconstruction. IEEE Transactions on Visualization and Computer Graphics, 5(4):349–359, 1999

[Bra00] G. Bradski. The OpenCV Library. Dr. Dobb’s Journal of Software Tools, 2000.

[CV14] Lu Cao and Fons J. Verbeek. Nature inspired phenotype analysis with 3d model representation optimization. In Ajith Abraham, Pavel Krömer, and Václav Snášel, editors, Innovations in Bio-inspired Computing and Applications, pages 165–174, Cham, 2014. Springer International Publishing.

[EM94] Herbert Edelsbrunner and Ernst P. M¨ucke. Three-dimensional alpha shapes. ACM Trans. Graph., 13(1):43–72, jan 1994

[KH13] Michael Kazhdan and Hugues Hoppe. Screened poisson surface reconstruction. ACM Transactions on Graphics (TOG), 32(3):29, 2013.

[MC21] Alessandro Muntoni and Paolo Cignoni. PyMeshLab, January 2021.

[Ope22] OpenCV. The OpenCV Reference Manual, 4.6.0-dev edition, June 2022.

[R21] Rosebrock, A. (2021, April 17). OpenCV center of contour. PyImageSearch. Retrieved June 20, 2022, from https://pyimagesearch.com/2016/02/01/opencv-center-of-contour/

[Tau95] Gabriel Taubin. A signal processing approach to fair surface design, 1995.

[ZPK18] Qian-Yi Zhou, Jaesik Park, and Vladlen Koltun. Open3D: A modern library for 3D data processing. arXiv:1801.09847, 2018.

Abstract of my Bachelor's thesis

The analysis of biological structures on a macroscopic and microscopic scale can lead to new insights in the biology research field. By gathering bioimages from microscope systems, extracting the point cloud and converting it to a surface representation, computerized three-dimensional (3D) models can be generated for measurement and visualisation. In this paper, a pipeline is proposed to improve the quality of the 3D mesh derived from 3D bioimages. This will in turn increase the accuracy of the 3D representation. For this research, a zebrafish embryo model and a mouse mammary gland model are used. The contour stack derived from the raw image data is first aligned and then converted to an oriented point cloud. Three surface reconstruction algorithms (Screened Poisson, Alpha Shapes and Ball Pivoting) are then used to generate the 3D models. The results are compared to determine which algorithm can be best used for the pipeline. Looking at the zebrafish embryo it can be concluded that the model generated by the Screened Poisson method adheres to the evaluation criteria the most. Aside from creating a smooth watertight surface, the model contains the most preserved details. Due to inconsistent orientation of the point cloud normals, the mammary gland model could not be constructed optimally for two of the three algorithms. This is a point of improvement that can be worked on in future development of the pipeline.