Code for Alignment with Perceived Ambiguity (APA)
Run sh scripts/main.sh
.
stage_0.sh
: select ambgiuous queries and build train data.train.sh
: train model.stage_1.sh
: evaluate trained model.
Change configs/main.yaml
model.name
: backbonemodel.offload_path
: model offload pathmodel.cache_path
: huggingface cache pathpath.data
: path to load datasetpath.output
: output path (logs, weights, ...)dataset.name
: test dataset namepipeline.stage_index
: set from 0 or 1explicit.template_id
: explicit inference QA templateexplicit.evaluation_method
: 'rouge' as defaultexplicit.correct_threshold
: generations with score above the threshold is evaluated as correct.implicit.method_id
: how to measure INFOGAIN (default 0)implicit.disambiguation_template_id
: template id for self-disambiguationimplicit.generation_template_id
implicit.threshold
: threshold value to filter ambiguous queriesimplicit.aggregate_method
explanation.template_id
: template to generate explanationsgeneration.num_generations_per_prompt
: generation configsgeneration.num_single_generation
: generation configsgeneration.max_new_tokens
: generation configsgeneration.temperature
: generation configsablation_methods
: data selection methodstrain.num_train_epochs
: train configs (number of training epochs)train.per_device_train_batch_size
: train configs (train batch size)train.gradient_accumulation_steps
: train configs (gradient accumulation steps)train.learning_rate
: train configs (learning rate)