Product design sketch understanding: unsupervised learning, neural rendering, and novel 3D shape representations

Ideation and concept sketching is used by product designers to externalize a shape from their imagination. At the ideation stage designers create multiple quick sketches. At the concept sketching stage, to achieve an accurate shape depiction, designers use dedicated sketching techniques in which earlier strokes create anchors for future strokes and carefully depict shape proportions in perspective. Concept sketching is the central element of the product design pipeline, usually followed by prototyping 3D shapes as digital or physical models. To facilitate a quick understanding of the 3D shape, the PhD work will address the problem of lifting 2D sketches to 3D in an unsupervised fashion.

The goal of this PhD proposal is to exploit and advance AI and deep learning to facilitate the creative design process. The primary objective is to exploit multi-view sketching and to propose new 3D shape representations that are easy for designers to work with. Leveraging state-of-art in differentiable and neural rendering the successful applicant will develop unsupervised methods aiming at 3D reconstruction. The student will have the opportunity to collaborate with industrial designers to collect an extended dataset of concept and/or ideation sketches to demonstrate the effectiveness of the proposed approach on representative product design sketches. The synthetic data and existing sketches from the OpenSketch dataset will be used for prototyping deep models.

In summary, the PhD work is expected to:

• Introduce multi-view shape encoders that leverage perspective distortions and different levels of details
• Introduce new 3D shape representations in the context of deep learning, and network architectures enabling such representations
• Contribute a new dataset of multi-view concept sketches.

All applicants should have (or expect to obtain) a first-class degree in a numerate discipline (mathematics, science or engineering) or MSc with Distinction (or 70% average) and a strong interest in pursuing research in this field. Passion for sketching and product design is advantageous.

English language requirements 

IELTS Academic 6.5 or above (or equivalent) with 6.0 in each individual category.

Our research 

Surrey Institute for People-Centred Artificial Intelligence