Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can you provide a sample evaluation metric file? #3

Open
peiran88 opened this issue Jul 28, 2022 · 3 comments
Open

Can you provide a sample evaluation metric file? #3

peiran88 opened this issue Jul 28, 2022 · 3 comments

Comments

@peiran88
Copy link

peiran88 commented Jul 28, 2022

I tried to get PSNR, SSIM and LPIPS using the D-NeRF Quantitatively evaluation file: metrics.ipynb but the result seems not correct. My result for the cutting_tissues_twice dataset is {'mse': 0.02320987731218338, 'psnr': 16.343271255493164, 'ssim': 0.7986437082290649, 'lpips': 0.36871644854545593} which seems not correct compared with the paper. Did I make a mistake? I used the ground truth images(with tools) and the estimated RGB images to calculate these metrics, is that correct?

If possible, can you provide a sample evaluation metric file like metrics.ipynb of D-NeRF ?

@yuehaowang
Copy link
Collaborator

yuehaowang commented Jul 28, 2022

I didn't use the 'metrics.ipynb' script in D-NeRF to evaluate our method as I need to follow the evaluation procedure in E-DSSR for a fair comparison. I uploaded the evaluation script (eval_rgb.py) and provided a brief instruction in readme. Please check it out and let me know if it works.

@chensjtu
Copy link

I have read the eval_rgb.py, is this eval file used for the final metric reported in the manuscript? All the frames except the first one are used for train and all the images are used for eval. If so, I will follow this setting.

@yuehaowang
Copy link
Collaborator

To be precise, all frames are used for training and all images except the first one are used for evaluation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants