Data-efficient model training for sustainable artificial intelligence

Artificial intelligence (AI) has become integral to modern life, with applications ranging from healthcare, medical diagnostics to cultural heritage preservation. Recent progress in foundation models, notably vision-language models (VLMs) and multimodal large language models, has demonstrated remarkable capabilities across diverse tasks. Yet, their training and deployment demand enormous computational resources, creating significant environmental costs and limiting accessibility. Current research emphasises scaling models and datasets to boost performance, but comparatively little attention is given to the usage efficiency and quality of the underlying data. A key question arises: must we always rely on all available data, or can similar performance be achieved with substantially less data, thereby reducing costs and environmental impact?

This proposal, DeTAI, addresses this challenge by improving data efficiency in VLM training. The project targets a 50% reduction in data volume while maintaining performance within 2% of standard benchmarks. DeTAI introduces a unified framework for analysing data redundancy through insights into VLM learning dynamics, enabling principled selection and generation of high-quality data. Redundancy quantification will inform the design of multimodal dataset distillation methods that exploit model learning patterns. The project will conduct comprehensive benchmarking of data-efficient training strategies, including evaluations of generalisation, robustness and fairness in low-data regimes such as medical image analysis.

By advancing state of the art in data-efficient VLM training, DeTAI promotes sustainable AI development and reliable deployment. It democratises access to foundation model research, particularly for under-resourced groups. Beyond its scientific contributions, DeTAI will strengthen the fellow’s expertise in AI and project leadership, consolidating the position as an emerging leader in efficient and responsible AI.