Skip to content

Jupyter notebooks for end-to-end differentiable blind tip reconstruction

License

Notifications You must be signed in to change notification settings

matsunagalab/differentiable_BTR

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

87 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

End-to-end differentiable blind tip reconstruction

Deconvolution example

This repository contains Jupyter notebooks for the end-to-end differentiable blind tip reconstruction (BTR) proposed in Matsunaga et al (2023).

Originally, this method was implemented in the Julia language, but currently, we are implementing the method in PyTorch that can be executed on Google Colab.

For explanations on the method in Japanese, please refer to the following presentation video.

All notebooks are written in Julia programming language. You need to install julia before using the notebooks. Also, the notebooks depend on several packages. The packages can be installed as follows:

$ julia
julia> 
# enter the package mode by pressing ]
pkg> add IJulia Flux Plots cuDNN Statistics BSON Revise HTTP ArgParse MLBase
pkg> add https://github.com/matsunagalab/MDToolbox.jl.git
# return to the REPL mode by pressing BACKSPACE or DELETE
julia> using IJulia
julia> exit()

Descriptions on files

The files are organized as follows. Note that all scripts assume Angstrom as the unit.

Standalone scripts

Standalone scripts written in Julia are available in script/ directory for the end-to-end differentiable blind tip reconstruction dblindtip.jl, cross validation dblindtip_cv.jl, erosion erosion.jl, dilation dilation.jl, visualization of AFM data (afm2png.jl and afm2gif.jl), visualization of tip (tip2png.jl), and RANSAC ransac.jl (for correcting tilt in AFM images). All the scripts read ASD file or CSV files and write CSV files. For each usage, please see the ouptus of --help option. Note that all scripts assume nm as the unit.

A typical work flow using the scripts would be follows. You can try them using test data in script/data/ (a double-tip case).

0. check the usages and options of scripts

$ cd script/
$ julia dblindtip.jl --help # check usage and options

1. visualize AFM data

# make PNG of CSV files
$ julia afm2png.jl data/
# make GIF of CSV files
$ julia afm2gif.jl --output original.gif data/

Original AFM

2. perform cross validaton (LOOCV) and select an appropriate lambda value

# Perform LOOCV and choose an lambda value according to the one standard error rule
$ julia dblindtip_cv.jl --output cv.png --lambda_start 1.0e-5 --lambda_stop 0.01 --lambda_length 4 data/

Following the one standard error rule, it is recommended to select the largest lambda whose mean (indicate by the solid line in the figure below) falls within or around the one standard deviation of the minimum (indicated by the shaded area). In this case, lambda = 1.0e-4 would be recommended.

Cross validation

3. perform the end-to-end differentiable blind tip reconstruction

# perform the end-to-end differentiable blind tip reconstruction
$ julia dblindtip.jl --output tip.csv --lambda 1.0e-4 data/
# visualize the reconstructed tip
$ julia tip2png.jl tip.csv

Reconstructed tip

4. perform erosion (deconvolution) with the reconstructed tip shape

# perform erosion (deconvolution)
$ julia erosion.jl --tip tip.csv data/
# visualize eroded (deconvoluted) molecular surfaces
$ julia afm2png.jl --ext csv_erosion data/
$ julia afm2gif.jl --output erosion.gif --ext csv_erosion data/

Original AFM

Citation and Acknowledgement

Original blind tip reconstruction

The original BTR in the notebooks is based on the algorithm and code provided by Villarrubia, J. Res. Natl. Inst. Stand. Technol. 102, 425 (1997). If you use the end-to-end differentiable BTR or the original BTR, please cite this paper.

J. S. Villarrubia, 
Algorithms for Scanned Probe Microscope Image Simulation, Surface Reconstruction, and Tip Estimation, 
J. Res. Natl. Inst. Stand. Technol. 102, 425 (1997).

Regularized blind tip reconstruction

The original BTR with an improved regularization scheme is based on F. Tian, X. Qian, and J. S. Villarrubia, Ultramicroscopy 109, 44 (2008), and G. Jóźwiak, A. Henrykowski, A. Masalska, and T. Gotszalk, Ultramicroscopy 118, 1 (2012).

F. Tian, X. Qian, and J. S. Villarrubia, 
Blind Estimation of General Tip Shape in AFM Imaging, 
Ultramicroscopy 109, 44 (2008).
G. Jóźwiak, A. Henrykowski, A. Masalska, and T. Gotszalk, 
Regularization Mechanism in Blind Tip Reconstruction Procedure, 
Ultramicroscopy 118, 1 (2012).

End-to-end differentiable blind tip reconstruction

Citation information on the end-to-end differentiable blind tip reconstruction:

Y. Matsunaga, S. Fuchigami, T. Ogane, and S. Takada. 
End-to-end differentiable blind tip reconstruction for noisy atomic force microscopy images. 
Sci. Rep. 13, 129 (2023). 
https://doi.org/10.1038/s41598-022-27057-2

License

This repository is licensed under the terms of MIT license.

Quaternion data contained in quaternion/ directory were taken from the repository of the BioEM program written by Cossio et al. https://github.com/bio-phys/BioEM. These are separately licensed under the terms of the GNU General Public License. Please check the license file quaternion/LICENSE.

Contact

If you have any questions or troubles, feel free to create GitHub issues, or send email to us.

Yasuhiro Matsunaga

ymatsunaga@mail.saitama-u.ac.jp

About

Jupyter notebooks for end-to-end differentiable blind tip reconstruction

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published