Ayat AL-Waisi
About
My research project
Efficient Adaptation of Foundation ModelsMy PhD research looks at how medical vision-language models (VLMs) can learn a new clinical imaging task from a handful of examples shown at inference time, rather than from costly retraining. Clinical settings change constantly - new imaging modalities, new question types, new hospitals - and collecting large labelled datasets for each of them is rarely practical. In-context learning offers an appealing alternative: give the model a few image-question-answer examples and let it adapt on the spot.
In practice this works far less reliably on medical images than on natural ones. Models are sensitive to the order in which examples are presented, are easily distracted by irrelevant or noisy examples, and will often produce a plausible-sounding answer without properly reading the query image. My project asks why this happens and what can be done about it, combining attention-level analysis of how visual evidence actually flows through these models with the design of mechanisms that compress a set of demonstrations into a compact memory used to guide how the image itself is read.
The work is carried out with open medical vision-language models and established benchmarks for medical visual question answering, using controlled, pre-registered comparisons between architectural alternatives so that claims about robustness and grounding can be tested rather than assumed.
Supervisors
My PhD research looks at how medical vision-language models (VLMs) can learn a new clinical imaging task from a handful of examples shown at inference time, rather than from costly retraining. Clinical settings change constantly - new imaging modalities, new question types, new hospitals - and collecting large labelled datasets for each of them is rarely practical. In-context learning offers an appealing alternative: give the model a few image-question-answer examples and let it adapt on the spot.
In practice this works far less reliably on medical images than on natural ones. Models are sensitive to the order in which examples are presented, are easily distracted by irrelevant or noisy examples, and will often produce a plausible-sounding answer without properly reading the query image. My project asks why this happens and what can be done about it, combining attention-level analysis of how visual evidence actually flows through these models with the design of mechanisms that compress a set of demonstrations into a compact memory used to guide how the image itself is read.
The work is carried out with open medical vision-language models and established benchmarks for medical visual question answering, using controlled, pre-registered comparisons between architectural alternatives so that claims about robustness and grounding can be tested rather than assumed.
ResearchResearch interests
My research sits at the intersection of multimodal machine learning and clinical imaging, with a focus on how large vision-language models adapt to new tasks and whether they genuinely read the images they are asked about. My interests include:
- Multimodal in-context learning and few-shot adaptation of vision-language models
- Medical visual question answering and clinical imaging applications
- Visual grounding and interpretability in medical multimodal LLMs, including attention analysis and activation patching
- Robustness of demonstration-based learning to example ordering, irrelevant examples and context overload
- Efficient adaptation of large multimodal models without full retraining
Research interests
My research sits at the intersection of multimodal machine learning and clinical imaging, with a focus on how large vision-language models adapt to new tasks and whether they genuinely read the images they are asked about. My interests include:
- Multimodal in-context learning and few-shot adaptation of vision-language models
- Medical visual question answering and clinical imaging applications
- Visual grounding and interpretability in medical multimodal LLMs, including attention analysis and activation patching
- Robustness of demonstration-based learning to example ordering, irrelevant examples and context overload
- Efficient adaptation of large multimodal models without full retraining