Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Evaluation issues encountered while fine-tuning blip2 #720

Open
fjt12138 opened this issue Jul 9, 2024 · 0 comments
Open

Evaluation issues encountered while fine-tuning blip2 #720

fjt12138 opened this issue Jul 9, 2024 · 0 comments

Comments

@fjt12138
Copy link

fjt12138 commented Jul 9, 2024

When I use the fine-tuning BLIP_restrivel, I set the path of the file in the running configuration file, but in the evaluation process after training, there are two evaluation processes, which seem to be the evaluation of the original coco dataset, I want to know why there is this result, my configuration is as follows:
_#Copyright (c) 2022, salesforce.com, inc.
#All rights reserved.
#SPDX-License-Identifier: BSD-3-Clause
#For full license text, see the LICENSE file in the repo root or https://opensource.org/licenses/BSD-3-Clause

model:
arch: blip2
model_type: coco
load_pretrained: True
load_finetuned: False
freeze_vit: False

#finetune blip2 with clip-vit-large
use_grad_checkpoint: False
pretrained: "https://storage.googleapis.com/sfr-vision-language-research/LAVIS/models/BLIP2/blip2_pretrained_vitL.pth"
vit_model: clip_L

datasets:
coco_retrieval: # name of the dataset builder
vis_processor:
train:
name: "blip2_image_train"
image_size: 364
eval:
name: "blip_image_eval"
image_size: 364
text_processor:
train:
name: "blip_caption"
eval:
name: "blip_caption"
build_info:
annotations:
train:
url: /data4/coco_train.json
# md5: aa31ac474cf6250ebb81d18348a07ed8
storage: /data4/coco_train.json
val:
url: /data4/coco_val.json
# md5: b273847456ef5580e33713b1f7de52a0
storage: /data4/coco_val.json
test:
url: /data4/coco_test.json
# md5: 3ff34b0ef2db02d01c37399f6a2a6cd1
storage: /data4/coco_test.json
images:
#storage: '/data1/lavis/coco/images/'
storage: ''

run:
task: retrieval
#optimizer
lr_sched: "linear_warmup_cosine_lr"
init_lr: 1e-5
min_lr: 0
warmup_lr: 1e-8
warmup_steps: 1000
weight_decay: 0.05
max_epoch: 5
batch_size_train: 14
batch_size_eval: 16
lr_layer_decay: 0.95 # layer-wise learning rate decay for the ViT
num_workers: 4
accum_grad_iters: 1

seed: 42
output_dir: "output/BLIP2/Retrieval_coco"

amp: True
resume_ckpt_path: null

evaluate: False
train_splits: ["train"]
valid_splits: ["test"]
#test_splits: ["test"]
k_test: 128

device: "cuda"
world_size: 1
dist_url: "env://"
distributed: False
use_dist_eval_sampler: False

1720497558175

Looking forward to your help!

@fjt12138 fjt12138 closed this as completed Jul 9, 2024
@fjt12138 fjt12138 reopened this Jul 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant