Fast-adaptive Foundation Models - CSRIO's Data61 scholarship to Tianchi Sha

Start year: 2024

Summary: To train domain-specific foundation models usually takes two ways: one is to train from scratch which requires a large-scale dataset and large investment in AI engineering, and another solution is to adapt an pre-trained foundation model to a new domain with a relatively small-scale dataset. However, how to balance general knowledge and domain-specific knowledge is an open challenge of model adaptation. This project aims to develop various adaptation strategies on foundation models. We will collaborate with Associate Professor Jing Jiang and Distinguished Professor Chengqi Zhang who are the leading researcher in AI. Firstly, we need to choose a proper architecture of the foundation model that can easily be tuned to general knowledge and domain-specific knowledge respectively. Secondly, a new learning strategy will be designed to enable the foundation model to be fastly adapted to a new domain. The project’s outcome can be applied and evaluated on various tasks, such as science tasks in different domains. A benchmark setting and standard evaluation criteria will be developed for this fast-adaptive foundation model.

Acknowledgement of Country

UTS acknowledges the Gadigal people of the Eora Nation, the Boorooberongal people of the Dharug Nation, the Bidiagal people and the Gamaygal people, upon whose ancestral lands our university stands. We would also like to pay respect to the Elders both past and present, acknowledging them as the traditional custodians of knowledge for these lands.