Keywords: 3-D, geometric modeling, photorealism, videography, room, virtual reality
Videography is currently perhaps the most effective media for 3-D data collection of complete indoor facilities. The im-ages contain inherently all the necessary geometrics for the 3-D modeling procedure. They contain the radiometric data as well for the photorealistic part of modeling. Each video sequence contains 25 frames in a second and there are practi-cally no limits for the recording as it regards scaling or viewing. It should be pointed out, that it is not expected the imag-ing geometry to be known or precalibrated. The exact geometry of the network and its images can be processed during the modeling.
In 1993-1996 we participated in a research project on Advanced computerization of the building information system where our main task was to develop a videogrammetry based 3-D modeling procedure for documenting of rooms. These 3-D models would have been used for creating the necessary geometric reference base for a facilities management system, which was developed at the Tampere University of Technology. The project was part of a national technology pro-gramme called Machine Vision 1992-1996 and funded by Technology Development Centre TEKES.
The approach to solve the modeling procedure by the traditional way of measuring first - then modeling did not suc-ceed. As it regarded the initially set outlines, the procedure became far too complicated and we did never accomplished it in the way that it would have been acceptable by the end users. Thereafter we started to develop the proper solution via an alternative approach which we called reverse modeling.
The initial idea of using videography only for 3-D indoor modeling has now been verified and the procedure will be pre-sented in our paper. The project Fotorealism in 3-D video digitizing has been funded by TEKES as well. Part of our basic research on the same topic is currently carried out within a third project called Dynamic strategy for building 3-D photorealistic models and funded by the Finnish Academy and Helsinki University of Technology.
The first step in interpreting an image is to orient it relative to the room model. This can be done graphically or computationally. A straightforward way is to overlay the wireframe on the image and change its viewpoint until it is coincident with the background image (Fig. 1). After this orientation the room model and each of the objects in it like walls, floor, ceiling or any other primitive chairs, tables etc. will match its correspondent in the image. All those which do not match, are added to the 3-D model or relocated depending on whether they already exist or not in the model. As the primary geometric model for the objects of the functional model we use a rectangular prism which has three width parameters: the breadth, the depth and the height. The more frames are read the more complete becomes the model.
Figure 1. The graphical relative orientation of an image and the room model. The change of a viewpoint is shown on the left and the results on the right: a) a video image presented as a background and the movable room model on it; b) the viewpoint of the model changed so that the scale corresponds the one of the image; c) the viewpoint of the model changed until the location in X and Z is the right one; d) the orientation specified by the rotations around X- and Y-axes.
In order to define a suitable coordinate system for the editing procedure we define a Cartesian room coordinate system having its origin in one of the room corners and its axes parallel to the principal room directions. All objects will have their own coordinate frame according to their width parameters. These frames are kept parallel to the room coordinate system and they are organized hierarchically. The transformation parameters of these coordinate frames are the three shifts to the upper level. Thus, the geometry of the functional model of the room and all objects inside is nothing more than a hierarchical set of reserved cubic volumes - or coordinate frames - in a common coordinate framework.
The locating and sizing of each individual object is performed during the image interpretation phase. We know that any object in the room is facing each other and connected by their surfaces. In similar way the geometric models of the objects may be connected as well. We further know that there always exists one object - or the room itself - which has been already defined by its location and size. We suppose that the orientation of the image relative to the geometric room model has been determined. Now, the location of the new object can be defined projectively along the common 2-D plane between these two objects. The size of the object can be defined similarly and the coordinate frame will be drawn along each coordinate axis (Fig. 2).
Figure 2. The correct view angle (on the left) and a new door object created to the model (on the right).
Both the object locations and the frame sizes are so far preliminary. The main inaccuracies are due to the interpretation procedure as it is based on a single image so far. The camera orientations may be inaccurate and the assumed object planes are not planes. However, the first defined object geometries may be edited closer as soon as they are seen on any of the following images. This new interpretation will enhance the object geometry as the locating and sizing of it is now conditioned by space intersection, not by plane projection anymore. The geometry of the whole collection of object models within the room model becomes the more rigid the more redundant intersections are made.
After a sufficient number of images have been interpreted the functional room model becomes complete (Fig. 3). The objects are identified and defined by their functions and mutual relations, and the coordinate frames define the object geometries according to their locations and sizes. The room model can be further connected to adjacent rooms and further to the whole building. By scaling the model becomes also metric. It can be then transformed geographically to any coordinate system and used for visualizing purposes. In spite of being more or less generalized, the functional model and its geometry fits well for various quantitative tasks as well. In facilities management applications it can be used even for area and volume calculations.
Figure 3. The functional room model as completed.
Figure 4. The points identified for block adjustment.
Figure 5. Image coordinates measured using the wireframe model.
The bundle block adjustment is performed as a free network adjustment including necessary additional parameters for nonlinear image distortions. The exact geometry of the room is now defined by the residuals of its corner points. Similarly, all the images and the objects within the room become now exactly referenced to the common room coordinate system (Table 1). If necessary, coordinate frames of single objects can be also shifted and resized in the functional model closer to their actual appearance within the room model.
Table 1. Model coordinates (X, Y, Z) and their accuracies (s_x, s_y and s_z) in centimeters.
In the case the room model and the objects should be measured in more details, this can be now done by using all images of the adjusted block and all the coordinate frames as primary reference control. Any new image or digitized object model can be oriented and added to the common network using these controls.
Figure 6. Texture maps projected onto the object surfaces.
Although the object points can be exactly located in the image planes the photorealistic texture mapping is a largely ambiguous task. The colors may be photorealistic but not real and therefore the values are relative only and depend on the viewing direction of each image. The texture maps contain perspective distortions as the object surfaces are more or less curved unlike the coordinate frames which contain only planar surfaces. In the case the texture maps are collected under changing light conditions the shadows will appear unrealistic. The texture maps for transparent or mirrorlike surfaces are misleading unless the objects seen therein are included to the complete model.
The examples ambiguity listed here do not indicate that these problems could not be solved. On the contrary, as the research on the photorealism still is in its very infancy, these examples rather indicate the prominent research tasks within photogrammetric modeling.
Figure 7. The photorealistic room model with some activity added like opening the door and moving the box.
The development of the reverse modeling procedure presented here was part of the masters thesis of Saara Mattila at Helsinki University of Technology. The photogrammetric block adjustment based on projective transformations was developed by Ilkka Niini and will be part of his doctors thesis. The graphical interface for both functional and geometric modeling was realized on basis of the WorldToolKit 3-D graphics library of Sense8 Corporation.
