vantuan5644/ucit-llava-1.5-7b-baselines

UCIT continual instruction tuning — baselines — LLaVA-1.5-7B

LoRA adapters trained sequentially on the 6-task UCIT benchmark, in this order:

ImageNet-R → ArxivQA → VizWiz-Caption → IconQA → CLEVR-Math → Flickr30k

Each <run>/Task<N>_<Name>/ holds the adapter after training task N, so the full 6x6 forward-transfer matrix evaluates checkpoint N on every task K<=N.

The CLIP vision tower is frozen in every run here; non_lora_trainables.bin carries the mm_projector only (21.0M parameters, no vision_tower.* keys).

Contents

run	size	tasks
clmoe_ucit_llava	5.3G	6
disco_ucit_llava	3.0G	6
disco_dko_ucit_llava	3.0G	6
ewc_ucit_llava	3.0G	6
hide_ucit_llava	3.0G	6
moelora_ucit_llava	3.0G	6
olora_ucit_llava	3.0G	6
seqft_ucit_llava	3.0G	6
same_ucit_llava	4.7G	6
smolora_ucit_llava	3.0G	6
sefe_ucit_llava	14G	6
joint_ucit_llava	498M	1

joint_ucit_llava

The multi-task upper bound, not a continual run. One adapter is trained for one epoch over the union of the six UCIT training sets (213,857 examples) with the same hyperparameters as seqft_ucit_llava. It is stored as Task6_Joint because it is scored only as the final row of the matrix, on all six test sets.

sefe_ucit_llava

The RegLoRA half of SEFE (arXiv 2505.02486), without the answer-style diversification data. Each task trains a fresh LoRA on top of the earlier task adapters, which are first merged into the base weights, so adapter_model.bin alone is not the model after task N. Task<N>_<Name>/sefe_chain.json lists the earlier task dirs: merge those into the base in that order, each at scaling alpha/r, then apply adapter N.

key_elements.pth is the RegLoRA mask: flat indices of the top 2% of |BA| entries in each linear layer, where the next task's adapter is penalised (weight 2500). The file in task N's dir accumulates tasks 1..N, so the task-6 mask contains every earlier one. It is needed only to continue training or to check which positions the penalty covered.