Dr Nicolas Pugeault

Research Staff

Qualifications: Dipl. Ing., MSc., PhD.

Email:
Phone: Work: 01483 68 6032
Room no: 7A AB 05

Further information

Biography

My core research interests are Cognitive and vision and machine learning. More specifically, I am interested in how internal representation of visual information can emerge from interaction with an agent's environment. This involves aspects such as feature extraction & selection, perceptual grouping of features, accumulation and disambiguation of visual information over time, discovering objects' shape from experience, and more. Moreover, I have a keen interest in biologically plausible functional models of human vision.

Further details can be found on my personal web page.

A full list of publications, conference presentations, and patents can be found here.

Research Interests

My core research interests are cognitive vision and machine learning. More specifically, I am interested in how internal representations of visual information can emerge from interaction with an agent's environment. This involves aspects such as feature extraction and selection, perceptual grouping of features, accumulation and disambiguation of visual information over time, discovering objects' shape from experience, and more. Moreover, I have a keen interest in biologically plausible functional models of human vision.

Publications

Highlights

  • Pugeault N, Woergoetter F, Krueger N. (2010) 'VISUAL PRIMITIVES: LOCAL, CONDENSED, SEMANTICALLY RICH VISUAL DESCRIPTORS AND THEIR APPLICATIONS IN ROBOTICS'. WORLD SCIENTIFIC PUBL CO PTE LTD INTERNATIONAL JOURNAL OF HUMANOID ROBOTICS, 7 (3), pp. 379-405.
    doi: 10.1142/S0219843610002209
  • Pugeault N, Bowden R. (2010) 'Learning pre-attentive driving behaviour from holistic visual features'. ECCV 2010, Part VI, LNCS 6316, , pp. 154-167.
  • Kraft D, Detry R, Pugeault N, Başeski E, Guerin F, Piater JH, Krüger N. (2010) 'Development of object and grasping knowledge by robot exploration'. IEEE Transactions on Autonomous Mental Development, 2 (4), pp. 368-383.
    doi: 10.1109/TAMD.2010.2069098
  • Detry R, Pugeault N, Piater JH. (2009) 'A Probabilistic Framework for 3D Visual Object Representation'. IEEE COMPUTER SOC IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 31 (10), pp. 1790-1803.
    doi: 10.1109/TPAMI.2009.64
  • Kraft D, Pugeault N, cseski EB, Popović M, Kragic D, Kalkan S, Wörgötter F, Krüger N. (2009) 'Birth of the Object: Detection of Objectness and Extraction of Object Shape through Object Action Complexes'. World Scientific Publishing International Journal of Humanoid Robotics, 5 (2), pp. 247-265.
    doi: 10.1142/S021984360800139X

Journal articles

  • Cooper H, Ong E-J, Pugeault N, Bowden R. (2012) 'Sign language recognition using sub-units'. Journal of Machine Learning Research, 13, pp. 2205-2231.
  • Klette R, Kruger N, Vaudrey T, Pauwels K, van Hulle M, Morales S, Kandil FI, Haeusler R, Pugeault N, Rabe C, Lappe M. (2011) 'Performance of Correspondence Algorithms in Vision-Based Driver Assistance Using an Online Image Sequence Database'. IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, 60 (5), pp. 2012-2026.
    doi: 10.1109/TVT.2011.2148134
  • Pugeault N, Krüger N. (2011) 'Temporal accumulation of oriented visual features'. Elsevier Journal of Visual Communication and Image Representation, 22 (2), pp. 153-163.
    doi: 10.1016/j.jvcir.2010.12.001
    Full text is available at: http://epubs.surrey.ac.uk/39623/
  • Pugeault N, Wörgötter F, Krüger N. (2010) 'Disambiguating multi-modal scene representations using perceptual grouping constraints.'. Public Library of Science PloS one, 5 (6) Article number e10663
    doi: 10.1371/journal.pone.0010663
    Full text is available at: http://epubs.surrey.ac.uk/39620/
  • Sabatini SP, Gastaldi G, Solari F, Pauwels K, Van Hulle MM, Diaz J, Ros E, Pugeault N, Krüger N. (2010) 'A compact harmonic code for early vision based on anisotropic frequency channels'. Computer Vision and Image Understanding, 114 (6), pp. 681-699.
    doi: 10.1016/j.cviu.2010.03.008
  • Sabatini SP, Gastaldi G, Solari F, Pauwels K, Van Hulle MM, Diaz J, Ros E, Pugeault N, Kruger N. (2010) 'A compact harmonic code for early vision based on anisotropic frequency channels'. ACADEMIC PRESS INC ELSEVIER SCIENCE COMPUTER VISION AND IMAGE UNDERSTANDING, 114 (6), pp. 681-699.
    doi: 10.1016/j.cviu.2010.03.008
  • Popovic M, Kraft D, Bodenhagen L, Baseski E, Pugeault N, Kragic D, Asfour T, Kruger N. (2010) 'A strategy for grasping unknown objects based on co-planarity and colour information'. ELSEVIER SCIENCE BV ROBOTICS AND AUTONOMOUS SYSTEMS, 58 (5), pp. 551-565.
    doi: 10.1016/j.robot.2010.01.003
    Full text is available at: http://epubs.surrey.ac.uk/39617/
  • Baseski E, Pugeault N, Kalkan S, Bodenhagen L, Piater JH, Krüger N. (2010) 'Using multi-modal 3D contours and their relations for vision and robotics'. Journal of Visual Communication and Image Representation, 21 (8), pp. 850-864.
    doi: 10.1016/j.jvcir.2010.06.006
  • Kraft D, Detry R, Pugeault N, Başeski E, Guerin F, Piater JH, Krüger N. (2010) 'Development of object and grasping knowledge by robot exploration'. IEEE Transactions on Autonomous Mental Development, 2 (4), pp. 368-383.
    doi: 10.1109/TAMD.2010.2069098
  • Pugeault N, Woergoetter F, Krueger N. (2010) 'VISUAL PRIMITIVES: LOCAL, CONDENSED, SEMANTICALLY RICH VISUAL DESCRIPTORS AND THEIR APPLICATIONS IN ROBOTICS'. WORLD SCIENTIFIC PUBL CO PTE LTD INTERNATIONAL JOURNAL OF HUMANOID ROBOTICS, 7 (3), pp. 379-405.
    doi: 10.1142/S0219843610002209
  • Detry R, Pugeault N, Piater JH. (2009) 'A Probabilistic Framework for 3D Visual Object Representation'. IEEE COMPUTER SOC IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 31 (10), pp. 1790-1803.
    doi: 10.1109/TPAMI.2009.64
  • Kraft D, Pugeault N, cseski EB, Popović M, Kragic D, Kalkan S, Wörgötter F, Krüger N. (2009) 'Birth of the Object: Detection of Objectness and Extraction of Object Shape through Object Action Complexes'. World Scientific Publishing International Journal of Humanoid Robotics, 5 (2), pp. 247-265.
    doi: 10.1142/S021984360800139X
  • Pugeault N, Worgotter F, Krueger N. (2005) 'Extraction of rich and reliable scene representations making use of perceptual grouping and motion'. PION LTD PERCEPTION, 34, pp. 115-115.
  • Wörgötter F, Krüger N, Pugeault N, Calow D, Lappe M, Pauwels K, Hulle MV, Tan S, Johnston A. (2004) 'Early Cognitive Vision: Using Gestalt–Laws for Task–Dependent, Active Image–Processing'. Natural Computing, 3 Article number 3 , pp. 293-321-293-321.

Conference papers

  • Ong EJ, Cooper H, Pugeault N, Bowden R. (2012) 'Sign Language Recognition using Sequential Pattern Trees'. Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, , pp. 2200-2207.
  • Pugeault N, Bowden R. (2011) 'Driving me Around the Bend: Learning to Drive from Visual Gist'. IEEE 2011 IEEE International Conference on Computer Vision, Barcelona, Spain: ICCV 2011: 1st IEEE Workshop on Challenges and Opportunities in Robotic Perception, pp. 1022-1029.
    doi: 10.1109/ICCVW.2011.6130363
    Full text is available at: http://epubs.surrey.ac.uk/531438/

    Abstract

    This article proposes an approach to learning steering and road following behaviour from a human driver using holistic visual features. We use a random forest (RF) to regress a mapping between these features and the driver's actions, and propose an alternative to classical random forest regression based on the Medoid (RF-Medoid), that reduces the underestimation of extreme control values. We compare prediction performance using different holistic visual descriptors: GIST, Channel-GIST (C-GIST) and Pyramidal-HOG (P-HOG). The proposed methods are evaluated on two different datasets: predicting human behaviour on countryside roads and also for autonomous control of a robot on an indoor track. We show that 1) C-GIST leads to the best predictions on both sequences, and 2) RF-Medoid leads to a better estimation of extreme values, where a classical RF tends to under-steer. We use around 10% of the data for training and show excellent generalization over a dataset of thousands of images. Importantly, we do not engineer the solution but instead use machine learning to automatically identify the relationship between visual features and behaviour, providing an efficient, generic solution to autonomous control.

  • Pugeault N, Bowden R. (2011) 'Spelling It Out: Real–Time ASL Fingerspelling Recognition'. IEEE 2011 IEEE International Conference on Computer Vision Workshops, Barcelona, Spain: ICCV 2011: 1st IEEE Workshop on Consumer Depth Cameras for Computer Vision, pp. 1114-1119.
    doi: 10.1109/ICCVW.2011.6130290
    Full text is available at: http://epubs.surrey.ac.uk/531435/

    Abstract

    This article presents an interactive hand shape recognition user interface for American Sign Language (ASL) finger-spelling. The system makes use of a Microsoft Kinect device to collect appearance and depth images, and of the OpenNI+NITE framework for hand detection and tracking. Hand-shapes corresponding to letters of the alphabet are characterized using appearance and depth images and classified using random forests. We compare classification using appearance and depth images, and show a combination of both lead to best results, and validate on a dataset of four different users. This hand shape detection works in real-time and is integrated in an interactive user interface allowing the signer to select between ambiguous detections and integrated with an English dictionary for efficient writing.

  • Cooper HM, Pugeault N, Bowden R. (2011) 'Reading the Signs: A Video Based Sign Dictionary'. IEEE 2011 International Conference on Computer Vision: 2nd IEEE Workshop on Analysis and Retrieval of Tracked Events and Motion in Imagery Streams (ARTEMIS 2011), Barcelona, Spain: ICCV 2011, pp. 914-919.
    doi: 10.1109/ICCVW.2011.6130349
    Full text is available at: http://epubs.surrey.ac.uk/531442/

    Abstract

    This article presents a dictionary for Sign Language using visual sign recognition based on linguistic subcomponents. We demonstrate a system where the user makes a query, receiving in response a ranked selection of similar results. The approach uses concepts from linguistics to provide sign sub-unit features and classifiers based on motion, sign-location and handshape. These sub-units are combined using Markov Models for sign level recognition. Results are shown for a video dataset of 984 isolated signs performed by a native signer. Recognition rates reach 71.4% for the first candidate and 85.9% for retrieval within the top 10 ranked signs.

  • Pugeault N, Pilz F, Pauwels K, van Hulle M, Krüger N. (2010) 'A three–level architecture for model-free detection and tracking of independently moving objects'. INSTICC Press Proceedings of the Fifth International Conference on Computer Vision Theory and Applications, Angers, France: VISAPP'10 1, pp. 237-244.
    Full text is available at: http://epubs.surrey.ac.uk/39615/

    Abstract

    We present a three–level architecture for detection and tracking of independently moving objects (IMOs) in sequences recorded from a moving vehicle. At the first stage, image pixels with an optical flow that is not entirely induced by the car’s motion are detected by combining dense optical flow, egomotion extracted from this optical flow, and dense stereo. These pixels are segmented and an attention mechanism is used to process them at finer resolution at the second level making use of sparse 2D and 3D edge descriptors. Based on the rich and precise information on the second level, the full rigid motion for the environment and for each IMO is computed. This motion information is then used for tracking, filtering and the building of a 3D model of the street structure as well as the IMO. This multi-level architecture allows us to combine the strength of both dense and sparse processing methods in terms of precision and computational complexity, and to dedicate more processing capacity to the important parts of the scene (the IMOs).

  • Pugeault N, Bowden R. (2010) 'Learning pre-attentive driving behaviour from holistic visual features'. ECCV 2010, Part VI, LNCS 6316, , pp. 154-167.
  • Baseski E, Bodenhagen L, Pugeault N, Kalkan S, Piater J, Krueger N. (2009) 'Using 3D Contours and Their Relations for Cognitive Vision and Robotics'. IEEE 2009 24TH INTERNATIONAL SYMPOSIUM ON COMPUTER AND INFORMATION SCIENCES, Guzelyurt, CYPRUS: 24th International Symposium on Computer and Information Sciences, pp. 408-413.
  • Boesman B, Jensen LBW, Baseski E, Pugeault N, Kruger N. (2009) 'Bayesian Reasoning Using 3D Relations for Lane Marker Detection'. Proceedings of VMV 2009,
  • Kraft D, Detry R, Pugeault N, Baseski E, Piater J, Kruger N, Fritz M, Schiele B, Piater JH. (2009) 'Learning Objects and Grasp Affordances through Autonomous Exploration'. SPRINGER-VERLAG BERLIN COMPUTER VISION SYSTEMS, PROCEEDINGS, Liege, BELGIUM: 7th International Conference on Computer Vision Systems 5815, pp. 235-244.
  • Pilz F, Pugeault N, Kruger N, Cremers D, Rosenhahn B, Yuille AL, Schmidt FR. (2009) 'Comparison of Point and Line Features and Their Combination for Rigid Body Motion Estimation'. SPRINGER-VERLAG BERLIN STATISTICAL AND GEOMETRICAL APPROACHES TO VISUAL MOTION ANALYSIS, Dagstuhl, GERMANY: International Dagstuhl Seminar 2008 5604, pp. 280-304.
  • Baseski E, Jensen LBW, Pugeault N, Pilz F, Pauwels K, Van Hulle MM, Woergoetter F, Kruger N, Ranchordas A, Araujo H. (2009) 'ROAD INTERPRETATION FOR DRIVER ASSISTANCE BASED ON AN EARLY COGNITIVE VISION SYSTEM'. INSTICC-INST SYST TECHNOLOGIES INFORMATION CONTROL & COMMUNICATION VISAPP 2009: PROCEEDINGS OF THE FOURTH INTERNATIONAL CONFERENCE ON COMPUTER VISION THEORY AND APPLICATIONS, VOL 1, Lisbon, PORTUGAL: 4th International Conference on Computer Vision Theory and Applications, pp. 496-505.
  • Pugeault N, Kalkan S, Baseski E, Wörgötter F, Krüger N. (2008) 'Reconstruction uncertainty and 3D relations'. Proceedings of Int. Conf. on Computer Vision Theory and Applications (VISAPP’08),
  • Jensen LBW, Baseski E, Kalkan S, Pugeault N, Woergoetter F, Krueger N, Caputo B, Vincze M. (2008) 'Semantic Reasoning for Scene Interpretation'. SPRINGER-VERLAG BERLIN COGNITIVE VISION, Santorini, GREECE: 4th International Cognitive Vision Workshop (ICVW 2008) 5329, pp. 121-134.
  • Pugeault N, Kalkan S, Woergoetter F, Baseski E, Krueger N, Ranchordas AN, Araujo HJ. (2008) 'Relations between reconstructed 3D entities'. INSTICC-INST SYST TECHNOLOGIES INFORMATION CONTROL & COMMUNICATION VISAPP 2008: PROCEEDINGS OF THE THIRD INTERNATIONAL CONFERENCE ON COMPUTER VISION THEORY AND APPLICATIONS, VOL 2, Funchal, PORTUGAL: 3rd International Conference on Computer Vision Theory and Applications, pp. 186-193.
  • Aarno D, Sommerfeld J, Kragic D, Pugeault N, Kalkan S, Woergoetter F, Kruger N, Lee S, Suh IH, Kim MS. (2008) 'Early reactive grasping with second order 3D feature relations'. SPRINGER-VERLAG BERLIN RECENT PROGRESS IN ROBOTICS: VIABLE ROBOTIC SERVICE TO HUMAN, Jeju Isl, SOUTH KOREA: 13th International Conference on Advanced Robotics 370, pp. 91-105.
  • Pugeault N, Wörgötter F, Krüger N. (2008) 'Accumulated Visual Representation for Cognitive Vision'. Proceedings of the British Machine Vision Conference (BMVC’08),
  • Detry R, Pugeault N, Piater J, Caputo B, Vincze M. (2008) 'Probabilistic Pose Recovery Using Learned Hierarchical Object Models'. SPRINGER-VERLAG BERLIN COGNITIVE VISION, Santorini, GREECE: 4th International Cognitive Vision Workshop (ICVW 2008) 5329, pp. 107-120.
  • Petrick R, Kraft D, ao KM, Geib C, Pugeault N, Krüger N, Steedman M. (2008) 'Representation and Integration: Combining Robot Control, High-Level Planning, and Action Learning'. Proceedings of the 6th International Cognitive Robotics Workshop (CogRob 2008), Patras, Greece, July 21-22,
  • Kraft D, Baseski E, Popović M, Batog AM, Kjær-Nielsen A, Krüger N, Petrick R, Geib C, Pugeault N, Steedman M, Asfour T, Dillmann R, Kalkan S, Wörgötter F, Hommel B, Detry R, Piater J. (2008) 'Exploration and Planning in a three Level Cognitive Architecture'. International Conference on Cognitive Systems (CogSys),
  • Pugeault N, Baseski E, Kraft D, Worgotter F, Kruger N, Sorrenti DG. (2007) 'Extraction of multi-modal object representations in a robot vision system'. INSTICC-INST SYST TECHNOLOGIES INFORMATION CONTROL & COMMUNICATION Robot Vision, Barcelona, SPAIN: 1st International Workshop on Robot Vision in Conjunction with VISAPP 2007, pp. 126-135.
  • Sabatini SP, Gastaldi G, Solari F, Diaz J, Ros E, Pauwels K, Hulle KMMV, Pugeault N, Krüger N. (2007) 'Compact and accurate early vision processing in the harmonic space.'. Proceedings of Int. Conf. on Computer Vision Theory and Applications (VISAPP’07),
  • Baseski E, Pugeault N, Kalkan S, Kraft D, Woergoetter F, Krueger N. (2007) 'A scene representation based on multi-modal 2D and 3D features'. IEEE 2007 IEEE 11TH INTERNATIONAL CONFERENCE ON COMPUTER VISION, VOLS 1-6, Rio de Janeiro, BRAZIL: 11th IEEE International Conference on Computer Vision, pp. 63-69.
  • Pilz F, Shi Y, Grest D, Pugeault N, Kalkan S, Krueger N, Bebis G, Boyle R, Parvin B, Koracin D, Paragios N, Tanveer SM, Ju T, Liu Z, Coquillart S, CruzNeira C, Muller T, Malzbender T. (2007) 'Utilizing semantic interpretation of junctions for 3D-2D pose estimation'. SPRINGER-VERLAG BERLIN ADVANCES IN VISUAL COMPUTING, PT I, Lake Tahoe, NV: 3rd International Symposium on Visual Computing 4841, pp. 692-701.
  • Pilz F, Shi Y, Grest D, Pugeault N, Kalkan S, Krüger N. (2007) 'Utilizing Semantic Interpretation of Junctions for 3D-2D Pose Estimation'. Proceedings of the ISVC 2007, , pp. 271-280.
  • Aarno D, Sommerfeld J, Kragic D, Pugeault N, Kalkan S, Wörgötter F, Kraft D, Krüger N. (2007) 'Model-independent grasping initializing object-model learning in a cognitive architecture'. IEEE International Conference on Robotics and Automation (ICRA07) Workshop: “From features to actions — Unifying perspectives in computational and robot vision”,
  • Pugeault N, Wörgötter F, Krüger N. (2006) 'Rigid Body Motion in an Early Cognitive Vision Framework'. Proceedings of the IEEE Systems, Man and Cybernetics Society Conference on Advances in Cybernetic Systems,
  • Geib C, Mourao K, Petrick R, Pugeault N, Steedman M, Krüger N, Wörgötter F. (2006) 'Object Action Complexes as an Interface for Planning and Robot Control'. Workshop ’Toward Cognitive Humanoid Robots’ at IEEE-RAS International Conference on Humanoid Robots (Humanoids 2006),
  • Pugeault N, Wörgötter F, Krüger N. (2004) 'Grouping over stereo for visual cues disambiguation'. Proceedings of the Conference Brain Inspired Cognitive Systems,
  • Pugeault N, Wörgötter F, Krüger N. (2004) 'A non-local Stereo Similarity based on Collinear Groups'. Proceedings of the Fourth International ICSC Symposium on ENGINEERING OF INTELLIGENT SYSTEMS,
  • Pugeault N, Krüger N. (2003) 'Multi–modal Matching applied to Stereo'. Proceedings of the BMVC 2003, , pp. 271-280.

Books

  • Pugeault N. (2008) Early Cognitive Vision: Feedback Mechanisms for the Disambiguation of Early Visual Representation. Verlag Dr. Muller

Book chapters

  • Pugeault N, Wörgötter F, Krüger N. (2004) 'Stereo matching with a contextual confidence based on collinear groups'. in Ilg UJ, Bülthoff HH, Mallot HA (eds.) Dynamic Perception Infix Verlag, St. Augustin , pp. 195-200.

Theses and dissertations

  • Pugeault N. (2008) Early Cognitive Vision: Feedback Mechanisms for the Disambiguation of Early Visual Representation.

Stay Connected

Contact Us

Accessibility | Contacting the University | Cookies | Disclaimer | Freedom of Information

© University of Surrey, Guildford, Surrey, GU2 7XH, United Kingdom. +44 (0)1483 300800

Page Owner: np0008
Page Created: Monday 28 November 2011 13:18:50 by lb0014
Last Modified: Monday 28 November 2011 13:26:39 by lb0014
Expiry Date: Thursday 28 February 2013 13:14:41
Assembly date: Tue Mar 26 22:42:40 GMT 2013
Content ID: 69130
Revision: 1
Community: 1379