Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MME_score.py cannot work #14

Open
dggeed opened this issue Nov 4, 2024 · 0 comments
Open

MME_score.py cannot work #14

dggeed opened this issue Nov 4, 2024 · 0 comments

Comments

@dggeed
Copy link

dggeed commented Nov 4, 2024

Hi! The script MME_score.py cannot work properly and there is a bug in the code. For example, in the judgment

HALC/eval/MME_score.py

Lines 124 to 127 in fc32840

if gt_question.lower() in pred_caption:
pred_ans = True
else:
pred_ans = False

the pred_caption is a single word (i.e. Yes or No), whereas gt_question.lower() is a sentence. As a result, pred_ans = False almost always holds true. BTW, Can you add comments to the key parts of the code? Thanks a lot! :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant