In the 3DVIDEO-project, we developed basic measuring techniques for 3-D video digitizing of objects and space extending the videogrammetric measuring from pointwise to profilewise and surfacewise measurements. The techniques have been realized using either a movable stereo head and a feature projector, or a laser-camera profilometer scanning the object surface stepwise (see Section 3 for the description of the measuring devices).
In the stereo case, we developed techniques for the automation of disparity measurement [ ]. The disparity measurements are recorded by projecting light rays with the feature projector onto the object surface. In order to avoid the correspondence problem, the number of projected lines is increased as a power of two (1,2,4,8,16,...) and the correspondence solved linewise by detecting the gradient variations along the image signals in the images rectified to the normal case of stereography. This initial digitizing is continued until the resulting disparity map is dense enough for interpolating the rest of disparities. The initial digitizing is necessary for the automatic control of a subsequent full digitizing that is performed by sliding small image windows along the image rows. The correspondence between the two images is refined by matching the image windows in 1-D. The full digitizing is continued until a required precision of the disparities is reached and a final disparity map obtained. The precision control is of passive nature and just for providing precision estimates for the subsequent geometric analysis stage.
The idea of projecting light rays on the object surface led us to develop techniques for another way of digitizing using a single camera and laser called a laser-camera profilometer [ ]. In this case, the laser with a cylindrical lens illuminates a stripe on the object surface which is then recorded by the camera. The whole object surface is scanned by moving the object in the direction not parallel to the plane defined by the laser sheet. The key scientific idea lies in the observation that the recorded profiles are projectively distorted in the image plane of the camera but when they are rectified by a projective transformation to a plane parallel to the one defined by the laser sheet, they have the correct shape and size. The parameters of the projective transformation are determined by measuring at least four known corresponding points from the image and object coordinate systems using a calibration object with known profile geometry. The measuring is performed by detecting for each row in the image of the camera the brightest column and rectifying the observed profiles to the object coordinate system. There is no correspondence problem like in the stereo case since only one stripe is projected on the object surface at a time. Outliers may, however, occur if there is a pixel whose gray level exceeds a treshold set for stripe detection but no stripe is actually visible in the row.
During the 3DVIDEO2-project, we developed the measuring with
the laser-camera profilometer further ahead. Especially, the problem of
outliers was solved by developing techniques for outlier detection.
The basic idea here is to compare the local average of the detected column
indices in two neighborhoods of a pixel. The first neighborhood is the
usual 
local neighborhood while the second one is a punctuated
neighborhood. The pixel is classified as an outlier, if the local
averages differ from each other more than a given treshold.
The main scientific outcome of the 3DVIDEO2-project was achieved in developing an area based method for the registration of multiple 3-D data sets acquired from different viewpoints. The first part of the research dealt with methods to solve the relative orientation of two disparity maps measured by a movable stereo head [ ] while in the second part, we considered the case of matching multiple profile maps acquired by the laser-camera profilometer in a computer-controlled driveway [ ].
The registration algorithm is intended for applications where the aim is to build a complete model of an object by measuring it from all directions. The rigid body transformations between different views are determined using only information available in the data, no extra targetted points are used and there are thus no exact or known correspondences between the data sets. Instead, the registration proceeds iteratively so that pointwise correspondences are established at each iteration according to the current orientation estimates and then the relative orientations are updated using the established correspondences. These two steps are iterated until convergence.
One of the key ideas in our registration algorithm is to represent the 3-D data sets as parametric surfaces. These surfaces can then be processed in a very similar way as conventional gray level images. In the stereo case, the natural parameterization is given by the disparity map while in the laser-camera profilometer case, we have proposed a new representation called a 3-D profile map. As preprocessing steps for the registration of multiple maps, we perform outlier removal, surface normal estimation, and edge detection. Initial registration is estimated by a feature based method where the features include modeled points and lines such as normals of planes, vertices of cones, and axes of cones and cylinders. Surface matching techniques are then used to iteratively update the pairwise correspondences between the maps established in the parametric domains. Our algorithm minimizes the mean sum of the squares of weighted distances between all the overlapping maps and solves the minimization task simultaneously for multiple maps. In order to discard wrong matches caused by occlusion, we use a weighting image that adaptively rejects matches that have a large discrepancy in the direction of the surface normal or in the distance between corresponding points with respect to the statistical distribution of the normal directions or the distances. Edge detection is used to compensate the interpolation error caused by the resampling of the parametric domains. The registration algorithm has been formulated using the notation of an image algebra allowing a parallel software implementation of such image processing tasks as outlier and edge detection, segmentation of convex areas, and bilinear interpolation.