Accurate 3D modelling of real world objects is essential in many applications such as digital film production and cultural heritage preservation. However, current modelling techniques rely on assumptions to constrain the problem, effectively limiting the categories of scenes that can be reconstructed. A common assumption is that the scene’s surface reflectance is Lambertian or known a priori. These constraints rarely hold true in practice and result in inaccurate reconstructions. Helmholtz Stereopsis (HS) addresses this limitation by introducing a reflectance agnostic modelling constraint, but prior work in this area has been predominantly limited to 2.5D reconstruction, providing only a partial model of the scene. In contrast, this paper introduces the first Markov Random Field (MRF) optimisation framework for full 3D HS. First, an initial reconstruction is obtained by performing 2.5D MRF optimisation with visibility constraints from multiple viewpoints and fusing the different outputs. Then, a refined 3D model is obtained through volumetric MRF optimisation using a tailored Iterative Conditional Modes (ICM) algorithm. The proposed approach is evaluated with both synthetic and real data. Results show that the proposed full 3D optimisation significantly increases both geometric and normal accuracy, being able to achieve sub-millimetre precision. Furthermore, the approach is shown to be robust to occlusions and noise.
The Dataset contains 2 synthetic scenes: `bunny' and `armadillo'; rendered to be used during Helmholtz Stereopsis and 6 real scenes: `bee', `fox', `corgi', `duck', `llama' and `giraffe'; captured at Centre for Vision, Speech and Signal Processing at the University of Surrey.All scenes include the ordered input reciprocal pairs views, a silhouette or trimap for each view and their calibration. The real scenes include an extra image per view, where the object is lit directly by a flash positioned on top of the camera for calibration purposes.All views are calibrated and the data is contained in .txt files for the synthetic scenes, in the form of the projection matrix, and as .xml files for the real scenes, in the form of opencv compliant calibration data.The models used to render the synthetic scenes are the Stanford Bunny and Armadillo from:Zippered Polygon Meshes from Range ImagesGreg Turk and Marc LevoyComputer Graphics (SIGGRAPH 1996 Proceedings)andFitting Smooth Surfaces to Dense Polygon MeshesVenkat Krishnamurthy and Marc LevoyComputer Graphics (SIGGRAPH 1996 Proceedings)available at: http://graphics.stanford.edu/data/3Dscanrep/