Making specialist language accessible
This project fine-tunes instruction-tuned FLAN-T5 models on the BioLaySumm 2025 dataset to translate expert biomedical reports into summaries that non-specialists can understand.
I compared three adaptation strategies:
- Full fine-tuning, updating every model parameter.
- Low-Rank Adaptation, training approximately 2–3% of model parameters.
- Evolution Strategies, using gradient-free population-based optimisation.
Findings
Models were evaluated with ROUGE-1, ROUGE-2, ROUGE-L, and ROUGE-Lsum. The experiments achieved more than 70% ROUGE performance across held-out evaluation metrics.
FLAN-T5 Base with LoRA provided the strongest balance between output quality and compute efficiency, matching or slightly exceeding full fine-tuning while updating only a small fraction of the model.
Experiments were implemented with PyTorch and Hugging Face Transformers and run with mixed precision on NVIDIA A100 GPUs.