J. Starck
ABSTRACT
In this paper we introduce a video-based representation for free viewpoint visualization and motion control of 3D character models created from multiple view video sequences of real people. Previous approaches to video-based rendering provide no control of scene dynamics to manipulate, retarget, and create new 3D content from captured scenes. Here we contribute a new approach, combining image based reconstruction and video-based animation to allow controlled animation of people from captured multiple view video sequences. We represent a character as a motion graph of free viewpoint video motions for animation control. We introduce the use of geometry videos to represent reconstructed scenes of people for free viewpoint video rendering. We describe a novel spherical matching algorithm to derive global surface to surface correspondence in spherical geometry images for motion blending and the construction of seamless transitions between motion sequences. Finally, we demonstrate interactive video-based character animation with real-time rendering and free viewpoint visualization. This approach synthesizes highly realistic character animations with dynamic surface shape and appearance captured from multiple view video of people.
- {AFO03} Arikan O., Forsyth D. A., O'Brien J. F.: Motion synthesis from annotations. Proceedings ACM SIGGRAPH (2003), 402--408.]] Google Scholar
Digital Library
- {BA83} Burt P., Adelson E.: A multiresolution spline with application to image mosaics. ACM Transactions on Graphics 2, 4 (1983), 217--236.]] Google ScholarDigital Library
- {BCS97} Bregler C., Covell M., Slaney M.: Video rewrite: Driving visual speech with audio. Proceedings ACM SIGGRAPH (1997), 1--8.]] Google ScholarDigital Library
- {Bou03} Bouguet J.-Y.: Camera Calibration Toolbox for Matlab: www.vision.caltech.edu/bouguetj/calib-doc. Tech. rep., MRL-INTEL, 2003.]]Google Scholar
- {BSM*03} Briceño H., Sander P., McMillan L., Gortler S., Hoppe H.: Geometry videos: A new representation for 3d animations. ACM Symposium on Computer Animation (2003), 136--146.]] Google ScholarDigital Library
- {CG00} Cosatto E., Graf P.: Photo-realistic talking heads from image samples. IEEE Transaction on Multimedia 2, 3 (2000), 152--163.]] Google ScholarDigital Library
- {CMS99} Culbertson W. B., Malzbender T., Slabaugh G.: Generalized voxel coloring. Proceedings of the International Workshop on Vision Algorithms, Lecture Notes in Computer Science 1883 (1999), 100--115.]] Google ScholarDigital Library
- {CR03} Chui H., Rangarajan A.: A new point matching algorithm for non-rigid registration. Computer Vision and Image Understanding 89 (2003), 114--141.]] Google ScholarDigital Library
- {CTMS03} Carranza J., Theobalt C., Magnor M., Seidel H.-P.: Free-viewpoint video of human actors. Proceedings ACM SIGGRAPH 22, 3 (2003), 569--577.]] Google ScholarDigital Library
- {DYB98} Debevec P., Yu Y., Borshukov G.: Efficient view-dependent image-based rendering with projective texture-mapping. 9th Eurographics Rendering Workshop (1998), 105--116.]]Google ScholarCross Ref
- {EGP02} Ezzat T., Geiger G., Poggio T.: Trainable videorealistic speech animation. Proceedings ACM SIGGRAPH (2002), 388--398.]] Google ScholarDigital Library
- {GGSC96} Gortler S., Grzeszczuk R., Szeliski R., Cohen M.: The lumigraph. Proceedings ACM SIGGRAPH 30 (1996), 43--54.]] Google ScholarDigital Library
- {GM04} Goldluecke B., Marcus M.: Space-time isosurface evolution for temporally coherent 3d reconstruction. IEEE International Conference on Computer Vision and Pattern Recognition I (2004), 350--355.]]Google ScholarCross Ref
- {Hop96} Hoppe H.: Progressive meshes. Proceedings ACM SIGGRAPH (1996), 99--108.]] Google ScholarDigital Library
- {KGP02} Kovar L., Gleicher M., Pighin F.: Motion graphs. Proceedings ACM SIGGRAPH (2002), 473--482.]] Google ScholarDigital Library
- {KRN97} Kanade T., Rander P., Narayanan P.: Virtualized reality: Constructing virtual worlds from real scenes. IEEE Multimedia 4, 1 (1997), 34--47.]] Google ScholarDigital Library
- {KSE*03} Kwatra V., Schödl A., Essa I., Turk G., Bobick A.: Graphcut textures: Image and video synthesis using graph cuts. Proceedings ACM SIGGRAPH (2003), 277--286.]] Google ScholarDigital Library
- {LAS02} Li Y., Ang T., Shum H.-Y.: Motion texture: A two-level statistical model for character motion synthesis. Proceedings ACM SIGGRAPH (2002), 465--471.]] Google ScholarDigital Library
- {LC87} Lorenson W., Cline H.: Marching cubes: A high resolution 3d surface construction algorithm. Computer Graphics 21, 4 (1987), 163--169.]] Google ScholarDigital Library
- {LCR*02} Lee J., Chai J., Reitsma P. S., Hodgins J. K., Pollard N. S.: Interactive control avatars animated with human motion data. Proceedings ACM SIGGRAPH (2002), 491--500.]] Google ScholarDigital Library
- {LH96} Levoy M., Hanrahan P.: Light field rendering. Proceedings ACM SIGGRAPH 30 (1996), 31--42.]] Google ScholarDigital Library
- {MBR*00} Matusik W., Buehler C., Raskar R., Gortler S., McMillan L.: Image-based visual hulls. Proceedings of ACM SIGGRAPH (2000), 369--374.]] Google ScholarDigital Library
- {MTG97} Moezzi S., Tai L., Gerard P.: Virtual view generation for 3d digital video. IEEE Multimedia 4, 1 (1997), 18--25.]] Google ScholarDigital Library
- {PF01} Plankers R., Fua P.: Articulated soft objects for video-based body modeling. IEEE International Conference on Computer Vision (2001), 394--401.]]Google ScholarCross Ref
- {PH03} Praun E., Hoppe H.: Spherical parameterization and remeshing. Proceedings ACM SIGGRAPH (2003), 340--349.]] Google ScholarDigital Library
- {SE02} Schödl A., Essa I.: Controlled animation of video sprites. ACM Symposium on Computer Animation (2002), 121--127.]] Google ScholarDigital Library
- {SH03} Starck J., Hilton A.: Model-based multiple view reconstruction of people. IEEE International Conference on Computer Vision (2003), 915--922.]] Google ScholarDigital Library
- {SH05} Starck J., Hilton A.: Virtual view synthesis of people from multiple view video sequences. Graphical Models (to appear 2005).]] Google ScholarDigital Library
- {SSSE00} Schödl A., Szeliski R., Salesin D., Essa I.: Video textures. ACM Symposium on Computer Animation (2000), 489--498.]] Google ScholarDigital Library
- {VBK02} Vedula S., Baker S., Kanade T.: Spatiotemporal view interpolation. Eurographics Workshop on Rendering (2002), 1--11.]] Google ScholarDigital Library
- {VMR} www.ee.surrey.ac.uk/research/vssp/vmrg/vcphome.htm.]]Google Scholar
- {ZBS04} Zhou K., Bao H., Shi J.: 3d surface filtering using spherical harmonics. Computer-Aided Design, Elsevier 36 (2004), 363--375.]]Google Scholar
- {ZKU*04} Zitnick C., Kang S. B., Uyttendaele M., Winder S. A. J., Szeliski R.: High-quality video view interpolation using a layered representation. Proceedings ACM SIGGRAPH 23 (2004), 600--608.]] Google ScholarDigital Library
Index Terms
- Video-based character animation
Recommendations
Layered acting for character animation
SIGGRAPH '03: ACM SIGGRAPH 2003 PapersWe introduce an acting-based animation system for creating and editing character animation at interactive speeds. Our system requires minimal training, typically under an hour, and is well suited for rapidly prototyping and creating expressive motion. A ...
Physics-based character animation with cascadeur
SIGGRAPH '19: ACM SIGGRAPH 2019 StudioIn this workshop we will create a realistic acrobatic 3D fighting animation using the animation software Cascadeur. We will learn the key features of physics-based character animation and will immediately apply the learned knowledge by creating an ...
Layered acting for character animation
We introduce an acting-based animation system for creating and editing character animation at interactive speeds. Our system requires minimal training, typically under an hour, and is well suited for rapidly prototyping and creating expressive motion. A ...
Comments