This project calibrates multiple cameras using a planar calibration checkerboard. The pipeline is comprised of four main steps: (1) detect checkerboard corners, (2) remove outlier corners from the previous step (using VAE, variational auto-encoder), (3) estimate initial camera parameters and world (checkerboard) points, (4) refine the initial estimates (using bundle adjustment), then optionally (5) analyze the calibration result.
- This calibration method has been used for "Capturing Detailed Deformations of Moving Human Bodies". Chen He, Hyojoon Park, Kutay Macit, and Ladislav Kavan. SIGGRAPH 2021.
- This project is focused on being simple and scalable, applicable to different calibration checkerboards and different number of cameras.
- The steps (1)-(3)/(5) are implemented using python 3 and the step (4) using C/C++.
- On the example dataset provided (./example_data/), this project achieves mean reprojection error 0.08 pixels with standard deviation 0.05.
The code execution follows the pipeline below:
(Input) (0). Multi-view images: The input is a set of images capturing a freely moving checkerboard.
(1). Corner detection: Checkerboard corners are detected and localized with sub-pixel precision using OpenCV.
(2). VAE outlier detector: Outlier corners are identified using VAE (variational auto-encoder), and corresponding images are discarded. As expained here (Section 3), open-source corner detectors (e.g., OpenCV, MATLAB, etc.) are known to give incorrect or inaccurate (i.e., outlier) corners which may deteriorate the accuracy of camera caibration. Therefore, we identify and remove such outliers by tranining a VAE outlier detector. Its effectiveness is explored more here (Section 4).
(3). Initial camera calibration: Initial camera parameters and frame-wise checkerboard poses are first estimated from a subset of images using OpenCV.
(4). Bundle adjustment: The estimated camera parameters and frame-wise checkerboard poses are further refined using bundle adjustment.
(Optional) (5). Analyze the calibration result: Computes reprojection errors and render corresponding histogram, reprojected images, etc. for analysis.
(1) Download the project. In a command prompt (Windows) or terminal (Linux):
git clone https://github.com/hjoonpark/MultiCamCalib.git
(2) Install Ceres
(3) Build the bundle adjustment project (C/C++)
- Click here
(4) Create Anaconda environment for Python
-
Windows
Assuming Anaconda is already installed, navigate to ./MultiCamCalib/ where environment.yml exists. Then in a Anaconda Powershell Prompt:
conda env create --file environment.yml -
Linux: Great thanks to Hongsuk Benjamin Choi for the Linux version: https://github.com/hongsukchoi/MultiCamCalib
(5) Run the codes → Follow this tutorial!
Follow this quick start tutorial (a step-by-step example)
Click here
For questions, please contact Hyojoon Park: hjoonpark.us@gmail.com