Deep learning for increasing human sketch expressivity
The goal of this PhD work is to propose new deep learning based algorithms and design a set of data collection and perceptual studies aimed at making sketching a more powerful visual communication language accessible to everyone.
Start date1 July 2022
Funding sourceSurrey Institute for People-Centred AI
A stipend of £15,609 for 2021/22, which will increase each year in line with the UK Research and Innovation (UKRI) rate, plus Home rate fee allowance of £4,500 (with automatic increase to UKRI rate each year). For exceptional international candidates, there is the possibility of obtaining a scholarship to cover overseas fees
Sketching as a powerful visual communication tool has been one of the core research topics in diverse scientific fields, ranging from Humanities to Natural Sciences. In recent years, there has been a resurgence of research interest in sketching thanks to the appearance of large-scale datasets and advances in machine learning. The PhD work will contribute to a vision in which sketching becomes a common visual communication tool accessible to everyone, increasing human communication ability as well as the ability to understand the world around us.
The primary goal of this work would be to study the diversity of abstraction levels and visual depictions of human sketches, contributing a new dataset, sketch recognition or generative models and their role in increasing the expressive power of human sketches.
Visual depictions are biased by long-term cultural habits of depicting certain objects. For instance, a majority of people represent a house as a square with a triangular roof and a chimney. Similarly, AI agents learning from existing datasets are biased towards the most common visual depictions in these datasets. In our recent work, we demonstrated how sketch recognition agent can be used to guide human sketching strategies but observed that participants were converging to a similar sketching strategy representing the most common strategy in the training set.
In summary, the PhD work will focus on:
- Designing novel data collection tasks for sketching that can increase the diversity of visual depictions
- Proposing a new sketch recognition agent that takes the diversity of visual depictions and temporal strokes salience into account, advancing deep learning
- Proposing a few alternative human-AI interaction scenarios and deep learning based algorithmic solutions enabling support for diverse level of guidance, studying human preferences and the impact on the depictions’ diversity and quality.
Related linksSurrey Institute for People-Centred Artificial Intelligence Pixelor: a competitive sketching AI agent (PDF) Graphical convention formation during visual communication (PDF)
All applicants should have (or expect to obtain) a first-class degree in a numerate discipline (mathematics, science or engineering) or MSc with Distinction (or 70% average) and a strong interest in pursuing research in this field. Passion for sketching and human perception is advantageous.
English language requirements
IELTS Academic 6.5 or above (or equivalent) with 6.0 in each individual category.
How to apply
Applications should be submitted via the Centre for Vision, Speech and Signal Processing programme page. Please clearly state the studentship title and supervisor on your application.