Dr Yulia Gryaditskaya
Dr Yulia Gryaditskaya is a Lecturer in Artificial Intelligence at Surrey Institute for People-Centred AI and CVSSP, UK. Prior to that, she was a Senior Research Fellow (2020-2022) in Computer Vision and Machine Learning at the Centre for Vision Speech and Signal Processing (CVSSP), in the SketchX group, led by Prof. Yi-Zhe Song.
Before joining CVSSP, she was a postdoctoral researcher (2017-2020) at Inria, GraphDeco, under the guidance of Dr Adrien Bousseau. She had the opportunity to visit MIT and collaborate with Professor Fredo Durand, MIT, CSAIL, and Professor Alla Sheffer, UBC, British Columbia. She received her PhD (2012-2016) from MPI Informatik, Germany, advised by Professor Karol Myszkowski and Professor Hans-Peter Seidel.
While working on her PhD (2014), she did a research internship in the Color and HDR group in Technicolor R&D, Rennes, France, under the guidance of Dr Erik Reinhard. She received a degree (2007-2012) in Applied Mathematics and Computer Science with a specialisation in Operation Research and System Analysis from Lomonosov Moscow State University, Russia.
Dr. Gryaditskaya's research focuses on the study of AI in conjunction with sketching for creation and creativity. She considers the ability to create new content to be one of the most impressive human abilities, motivating her research. Content creation is ubiquitous in the entertainment and media industries. In her research, she aims to explore how AI can be used to assist people through the content creation process, increasing people's productivity. Creativity, imagination, and skills allow us to effectively translate our experience and vision into something unique. Therefore, Dr. Gryaditskaya is interested in exploring how AI can help boost and train human creativity. In particular, her research focuses on sketching, a ubiquitous content creation medium and powerful visual communication language.
We present the first algorithm capable of automatically lifting real-world, vector-format, industrial design sketches into 3D. Targeting real-world sketches raises numerous challenges due to inaccuracies, use of overdrawn strokes, and construction lines. In particular, while construction lines convey important 3D information, they add significant clutter and introduce multiple accidental 2D intersections. Our algorithm exploits the geometric cues provided by the construction lines and lifts them to 3D by computing their intended 3D intersections and depths. Once lifted to 3D, these lines provide valuable geometric constraints that we leverage to infer the 3D shape of other artist drawn strokes. The core challenge we address is inferring the 3D connectivity of construction and other lines from their 2D projections by separating 2D intersections into 3D intersections and accidental occlusions. We efficiently address this complex combinatorial problem using a dedicated search algorithm that leverages observations about designer drawing preferences , and uses those to explore only the most likely solutions of the 3D intersection detection problem. We demonstrate that our separator outputs are of comparable quality to human annotations, and that the 3D structures we recover enable a range of design editing and visualization applications, including novel view synthesis and 3D-aware scaling of the depicted shape.
We present the first competitive drawing agent Pixelor that exhibits human-level performance at a Pictionary-like sketching game, where the participant whose sketch is recognized first is a winner. Our AI agent can autonomously sketch a given visual concept, and achieve a recognizable rendition as quickly or faster than a human competitor. The key to victory for the agent’s goal is to learn the optimal stroke sequencing strategies that generate the most recognizable and distinguishable strokes first. Training Pixelor is done in two steps. First, we infer the stroke order that maximizes early recognizability of human training sketches. Second, this order is used to supervise the training of a sequence-to-sequence stroke generator. Our key technical contributions are a tractable search of the exponential space of orderings using neural sorting; and an improved Seq2Seq Wasserstein (S2S-WAE) generator that uses an optimal-transport loss to accommodate the multi-modal nature of the optimal stroke distribution. Our analysis shows that Pixelor is better than the human players of the Quick, Draw! game, under both AI and human judging of early recognition. To analyze the impact of human competitors’ strategies, we conducted a further human study with participants being given unlimited thinking time and training in early recognizability by feedback from an AI judge. The study shows that humans do gradually improve their strategies with training, but overall Pixelor still matches human performance. The code and the dataset are available at http://sketchx.ai/pixelor.
In this paper, for the first time, we investigate the problem of generating 3D shapes from professional 2D sketches via deep learning. We target sketches done by professional artists, as these sketches are likely to contain more details than the ones produced by novices, and thus the reconstruction from such sketches poses a higher demand on the level of detail in the reconstructed models. This is importantly different to previous work, where the training and testing was conducted on either synthetic sketches or sketches done by novices. Novices sketches often depict shapes that are physically unrealistic, while models trained with synthetic sketches could not cope with the level of abstraction and style found in real sketches. To address this problem, we collected the first large-scale dataset of professional sketches, where each sketch is paired with a reference 3D shape, with a total of 1,500 professional sketches collected across 500 3D shapes. The dataset is available at http://sketchx.ai/downloads/. We introduce two bespoke designs within a deep adversarial network to tackle the imprecision of human sketches and the unique figure/ground ambiguity problem inherent to sketch-based reconstruction. We show that existing 3D shapes generation methods designed for images fail to be naively applied to our problem, and demonstrate the effectiveness of our method both qualitatively and quantitatively.
Deep image-based modeling received lots of attention in recent years, yet the parallel problem of sketch-based modeling has only been brieﬂy studied, often as a potential application. In this work, for the ﬁrst time, we identify the main differences between sketch and image inputs: (i) style variance, (ii) imprecise perspective, and (iii) sparsity. We discuss why each of these differences can pose a challenge, and even make a certain class of image-based methods inapplicable. We study alternative solutions to address each of the difference. By doing so, we drive out a few important insights: (i) sparsity commonly results in an incorrect prediction of foreground versus background, (ii) diversity of human styles, if not taken into account, can lead to very poor generalization properties, and ﬁnally (iii) unless a dedicated sketching interface is used, one can not expect sketches to match a perspective of a ﬁxed viewpoint. Finally, we compare a set of representative deep single-image modeling solutions and show how their performance can be improved to tackle sketch input by taking into consideration the identiﬁed critical differences.