Ms. Katri Oksanen, M.Sc. (Eng.)
Researcher
The Institute of Photogrammetry and Remote Sensing
Helsinki University of Technology
Finland

THE DESIGN AND SIMULATION OF VIDEO DIGITIZING BY USING THREE-DIMENSIONAL CAD-MODELS


Keywords: Design, Simulation, Visualization, Three-dimensional CAD model, Measurement Model, Close_range

Abstract

1. Introduction

2. The design of video digitizing 3. The measurement model design tool, (MMD) 4. Simulation of precision
5. Conclusions

References
Figures: 1, 2, 3, 4, 5,

Obs!The figures of this www-version are not the same as in the printed version of this article.


Commission V, Working Group 3

ABSTRACT

This paper describes the use of three-dimensional CAD-models in the design of a measurement model for video digitizing. The paper also describes the simulation of the measurements defined by this measurement model. A tool made for this purpose, the Measurement Model Design tool (MMD) is presented. With the tool, the measurement can be designed using three-dimensional CAD-models of the object and measuring environments. The user can select from various camera models. The intersection precision is calculated using limiting error propagation. Calculated precision values are visualized in 3D object space in AutoCAD. Also simulated images are shown. After its creation, the measurement model can be used to control or assist the actual measurement. The MMD tool and its simulation part form an experts' tool for planning video digitizing.

Keywords: Design, Simulation, Visualization, Three-dimensional CAD model, Measurement Model, Close_range

1. INTRODUCTION

For the planning of traditional photogrammetric measurements like aerial photogrammetry, there is plenty of theory and knowledge available. The planning of video digitizing in large scale applications, such as the quality control or reverse-engineering tasks, differs from these traditional measuring situations. It is typical to use convergent images. The shape of the object can complicate the camera placement. To cover the whole object, it is needed to be surrounded by cameras. The cameras have to see overlapping areas. Especially in industry the measuring environment can be really restrictive. On the other hand, the accuracy requirements are usually very high. The placement of the cameras demands various and often almost opposite restrictions to be taken into account. The three-dimensional measurement demands three-dimensional planning.

1.1 The measurement situations

In quality control measurements there is usually a CAD-model of the object available. The idea of the measurements is to check how well the real object meets the design. In reverse-engineering the purpose of the measurements is to produce a CAD-model. However, in this case an approximate model can be used in the planning of the measurement.

1.2. The measurement model

The concept of the measurement model includes the sensors and the measuring points (or features). It includes sensor exterior orientation (the projection center coordinates and rotations) and interior orientation information. Interior orientation can be taken either from calibration data or from user-defined 'ideal' values.

The measurement model can also produce command files for the sensor control, or its information can be used as starting values for the measuring system (for example, sensor orientation, approximated image coordinates and so on).

Some general aspects of designing video digitizing are introduced in chapter two. The measurement model design tool (MMD) and its use are described in chapter three. The simulation of the measurements designed with this way are explained in more detail in chapter four. At the end some conclusions are made.

2. THE DESIGN OF VIDEO DIGITIZING

The design of photogrammetric network in close-range applications is a complicated task. There is not one right design or one best design available. From many possible designs the suitable one for every situation have to be selected. Many requirements and constraints are opposites and the stability between them has to be found. One way to find, if requirements and constraints are full-filled, is simulation.

2.1. General aspects on design of close-range photogrammetric measurement

The photogrammetric system bases on the collinearity condition. This condition defines that the image point, the projection center of a camera and the object point lie on the same line. The object coordinate measurement is based on this condition. The system is not ideal, for example, lens distortions have to be taken into account.

The design has mostly based on experience of photogrammetrists. In last years there have been efforts to build automated design systems or simulation systems for designing of close-range photogrammetric applications (Luhmann, 1994; Mason, 1994; Cowan, 1988). The main purpose in our work has been to buildt an experts' tool. By simulation, it is possible to get information about achievable precision and reliability of the measurements.

In real measurement situations there are many restrictions. For example, the best possible imaging geometry is often hard to achieve. Therefore it is very important to know how accurate measurements can be made and how the measurement accuracy can be improved. The selection of the measuring points is also important. Selected points should represent the object and its interesting properties. The accuracy of unknown parameters can be calculated by simulation.

In the design of photogrammetric measurement or network you have to consider several things that effect to the measurements. For example, Fraser (1989) has listed following questions to be answered:

There are many more questions available. Many answers depend on each other - so the designing of the measurement is not a simple task. Not only an object to be measured does effect to the design. In design you have to take into account the measurement environment, too. The use of CAD-system is a great help if you have some kind of CAD-models about the object and the environment available.

In the design of the measurement, different requirements have to be full-filled despite the restrictions. The goal of the measurement design is to achieve accurate and reliable results with an economical way. Because of the opposite requirements, the simulation of measurement is the best way to inspect that all design requirements are achieved. It is also a good way to see how the changes of different parameters (for example, sensor locations, sensor parameters, etc.) effect on the measurement results.

When planning a photogrammetric measurement, the cameras should be placed so that each point is seen at least from two cameras. The reliability of the measurement increases when the number of image rays increases. In special applications the convergent imagery is usually used instead of normal situation (stereo). Mason (1994) has given the following basic constraints, which should be satisfied in camera placement:

There exists also a number of objectives that should be considered in camera station placement (Mason, 1994).

The influence of these constraints and objectives to camera station placement can be visualized geometrically. An example of this in 2D is given by Mason (1994). Because measurements are done in 3D the influence of these constraints and objectives have to be considered in 3D. This is one reason, why it is so important to design the measurements three-dimensionally.

2.2. The use of 3D-models in measurement design

The purpose of the measurements can be either to produce a model of an object or to compare how well the real object meets the design. The reverse-engineering tasks produce models. In the quality control and deformation measurements the real object is compared to an available model. In later cases the model can either be a design or it can be a model based on the previous measurements. In most cases, three-dimensional CAD-models are used.

The planning of measurements have traditionally been done based on two-dimensional maps or drawings. In addition, the planning has been done by experts, who in most cases have a long experience of similar measurements. The measurement planning of complicated 3D-objects can be difficult using 2D-documents, like maps and drawings. The sensor placement demands several, often conflicting, constraints and objectives to be taken into account, which is done easier in three-dimensional planning. The visibility of measuring points and the finding of common points for separate camera groups can be done in CAD-environment using three-dimensional models.

The accuracy of the models available for the measurement planning can vary. The more exact model of the object and the workspace is available, the more exact the design of measurement model and its simulation are going to be. The most exact 3D-models are available in quality control and deformation measurements. For example, the design of deformation measurements using 3D-model can assure that the most relevant information to detect the deformation is going to be measured. The camera placement can be greatly assisted by the use of 3D-models. Already a rough 3D-model can be very useful, when the camera placement around an object (needing several camera groups) has to be designed. If no other model is available, the model of measurement environment can be done from building drawing by extruding. When the constraints in working space can be taken into account from the beginning of camera placement design, there is no need to change camera placement later due to difference in reality and optimal planning (done in 2D without knowledge about these difficulties).

In AutoCAD, there are different ways to model three-dimensional objects. It can be done by using wireframe, surface, or solid modeling. The MMD tool uses solid models of the object. The measurement environment can be modeled using wireframe or surface modeling.

3. THE MEASUREMENT MODEL DESIGN TOOL, (MMD)

The measurement model design tool (MMD) uses three-dimensional CAD-models of the object and measuring environment. The user of the tool can define the camera placements and orientations in AutoCAD. The user also selects the camera parameters. If the real calibration data of the camera is available, it can be used in simulation. The tool visualizes the part of the object seen by each camera. The definition of the measuring points is interactive. The tool checks the visibility of the points for each camera. The point has to be seen from at least two cameras. Because the visibility checks are made three-dimensionally, possible occlusion points can easily be observed. The designed geometry is used for simulation of precision values. Calculated precision measures are visualized in AutoCAD with MMD tool.

3.1. Technical information about the tool

The measurement model design tool works in AutoCAD environment. The tool is written by AutoLISP-command language. It utilizes AutoCAD's Advanced Modeling Extension (AME) package. The tool enables the design of measurements in 3D. The restriction of the tool is, so far, that the object has to be modeled as solid model. The tool has been developed in 3D2000 project. The base of the tool is written at VTT, Technical Research Centre of Finland, in Oulu by Hannu Kallio-Kokko and Petri Pajulampi. The further development of the tool's photogrammetric part is done by the author of this paper in Helsinki University of Technology. The design part (AutoLISP-functions) works in AutoCAD both in MS-DOS- and UNIX-environments.

The simulation part of the tool has been written in C-language and it works for the present in UNIX. Simulation calculation part use functions originally made for the Mapvision System. These have been connected to the design tool in AutoCAD. The simulation part uses the information created by the MMD tool. The calibration files of the Mapvision System can also be utilized in simulation. The results of the simulation (precision measures) are visualized in AutoCAD by the MMD tool.

3.2. The use of the tool

The MMD tool was originally made for CAD-based graphic measurement planning for rangefinder-based 3D coordinate measurements (Ailisto et al., 1995). The tool can, for example produce command files to control the rangefinder. Later the planning of photogrammetric measurement is added to the tool. The simulation part is so far done only for the photogrammetric measurements. The simulation part is possible to expand for other measurement types, for example the rangefinder measurements, too. In this paper only the photogrammetric functions of the tool are described.

3.3 The definition of measuring situation

The tool utilizes AutoCAD's graphic representation properties. The basic data used in design are the solid model of the object and the three-dimensional CAD-model of measurement environment, which is optional. The measurement environment can be modeled by photogrammetry, laser-profiling, or theodolites. Also extruding of building plans can be a useful way to get the rough model of workspace. At present, the workspace model is only a visual help for the designer. It is planned to be taken into account in the visibility calculations.

The model of the object can be, for example, a design of the object (quality control tasks), or an approximated model (reverse-engineering tasks), or an old model (based on earlier measurements). Various types of camera models can be used. If camera calibration files are available, they can be used. Another case is to use an old calibration file of the similar camera type. The last possibility is to use ideal pinhole camera for the planning of measurements. The user of the tool defines which kind of camera model is used.

3.4. The sensor placement process

The designer chooses the number of cameras. The camera sensors are placed around the object. The target point for each camera is defined. The camera parameters (for example, camera constant, image width and height) can be altered during the design. Camera places and target points can also be changed as long as suitable camera geometry is founded. The MMD tool does not include any artificial intelligent for the sensor location. This means that the designer of the network decides, for example, how many cameras are used, which kind of cameras are used and how they are located. In practice, the designer puts each camera into image and shows the target point for each camera. After locating cameras, designer can visualize how different images cover the object. Camera placements, orientations and parameters can be changed, if wanted. In the present version of the tool, the maximum of four cameras can be placed at time.

3.5. The object points location process

Object points can be selected after fixing the camera locations. There are various strategies to select object points to be measured. Points can be selected one by one, or points can be selected using profile or raster definitions. With profile and raster definitions, the points are selected automatically from selected line or area defined by the cursor. The distance between the points is given by the designer. The visibility check is made before an object point is accepted. If the point is not visible for at least two cameras, it is rejected. After deciding the locations of measuring points, a simulation stage can be made. The figure 1 is an example of three-dimensional measurement plan.

Figure 1. An example of measurement model. The camera placements are shown with big crosses and object points with smaller crosses.

MMD tool saves the plan as an AutoCAD drawing file (DWG). It also produces input files for the simulation part. These files include the camera positions and orientations and object point coordinates. In photogrammetric measurements the simulated image coordinates and the sensor orientation information can be used as approximated starting values for the measurement system.

3.6. The calculation of simulated images and precision values

After locating the cameras and measuring points with the MMD tool, the following information can be used in simulation:

The simulated image coordinates are calculated from this measurement model information (camera parameters, possible camera calibration data, camera orientation, the coordinates of object points).

The simulation is made by the least-square method. The precision values (standard errors of intersected 3D-coordinates, error ellipses and error ellipsoids) are calculated from the variance-covariance matrix of the unknown parameters. The accuracy values are visualized in 3D object space and geometrically weak areas are easily located. The simulation is explained in more detail in chapter four.

3.7. Visualization of precision measures and simulated images

The MMD tool enables the visualization of calculated precision measures and images in AutoCAD. The simulation program produces AutoLISP-files needed by the MMD tool's visualization functions.

The simulated images are visualized. Point distribution on the images can be evaluated visually. An example of the simulated images are shown in figure 2. There are two frames in every image. The outer frame shows the image area and the inner frame shows the area, where the targets are measurable.

Precision measures like standard errors of the unknown coordinates, error ellipses in various planes (XY, XZ, YZ), error ellipsoids or mean radial spherical error (MRSE) can be drawn for every point. This kind of visualization is very illustrative compared to precision values expressed only as numbers. However, for the visualization purposes, the precision measures have to be scaled and this should always be kept in mind. Examples of visualization are given in figures 3, 4, and 5.

Figure 2. Example of simulated images. The outer frame visualize the image area and the inner frame shows the area, where the target is able to be measured.

Figure 3. Example of visualization of standard errors for the unknown coordinates of the object points. See chapter 4.2. for more details.

Figure 4. Example of visualized mean radial spherical errors (MRSE). See chapter 4.2. for more details.

Figure 5. Example of standard error ellipses in XY-, XZ- and YZ- planes.

4. SIMULATION OF PRECISION

The simulation is made by the least-square method. The linear functional and stochastic model of the least squares adjustment is (see for example, Fraser, 1989) is following:

(1)

, where l is the vector of observations, v is the vector of residuals, x is the vector of unknown parametrs, A is the design matrix, Cl is the covariance matrix of observations, and P is the weight matrix. The observations are assumed to be normally distributed random variables.

If design matrix A is of full rank, the least-square estimate for x is:

(2)

and the corresponding covariance matrix is:

(3)

It is possible to evaluate the measurement accuracy before actual measurements. The design matrix A is totally defined by the geometry of the network and by the information of the observations between the points. The real observations do not effect to the structure of the design matrix A. The weight matrix P is made based on the accuracies of the observations. In photogrammetry, the weight matrix is usually . (Fraser,1989). That means that image observations are equally weighted. The cofactor matrix Qx (in equation 3) is the function of the network geometry and the observation accuracy only. All information about the precision of unknown parameters is therefore included in the variance-covariance matrix Cx (3), which is get by multiplying the cofactor matrix Qx by the variance of the unit weight.

In this way, the precision of the network can be evaluated before the actual measurements. To evaluate the precision of the points in network, only their approximate values has to be known. These approximation values are needed for the linearization of the condition equations.

At first the simulated image coordinates are calculated from the measurement model information (camera parameters, camera calibration data, measuring points coordinates). After that the cofactor matrix and the variance-covariance-matrix are created. Various precision measures can be calculated from the variance-covariance matrix of the unknown parameters. These are, for example, the variances of intersected 3D-coordinates, error ellipses, and error ellipsoids. These values are visualized in 3D object space. The shape of the error figures depends on the network structure (geometry). The variance of the unit weight is only a scaler. The variance of the unit weight could be improved, for example, by taking multiple exposures. The selection of a priori can based on earlier measurements or experiments.

4.1 Limiting error propagation

So far, the calculation of precision values is based on limiting error propagation (Brown, 1980; Fraser, 1989). Only the precision of the image coordinate measurement is taken into account in calculating accuracy estimates for the XYZ coordinates. The assumption is made, that projective and additional parameters are precisely known. This means that possible errors of the calibration are not taken into account. The simulation model could be expanded so, that the effect of these is also taken into account when calculating precision values.

Using limiting error propagation, the variance-covariance-matrix of unknown object coordinates can be obtained as a simple inverse of a 3 x 3 matrix (Fraser, 1989):

(4)

The error propagation could be expanded to total error propagation, when also the effect of calibration errors are taken into account.

4.2. The variances of the unknown parameters

The variances of unknown object coordinates are get from the diagonal elements of the variance-covariance matrix Cx . The standards errors of the unknown parameters are square roots of these diagonal elements. In this way the standard errors (sX, sY, and sZ) for unknown object coordinates are calculated. However, the errors are not always biggest in the direction of coordinate axes. The example of visualization of mean errors was already shown in figure 3.

Instead of the separate mean errors for X, Y, and Z, one precision measure can be used. One possible measure is the mean radial spherical error (MRSE) (Mikhail, 1976, p. 34). In three-dimensional case this measure is visualized with a sphere, which radius is equal to the MRSE value. It gives an approximation about the precision of the point. An example of the visualization of MRSE values was given in figure 4. More exact way to express the precision of the point is to use error ellipsoids.

4.3. The error ellipses and the error ellipsoids

The variance-covariance matrix includes the variances and the covariances of the unknown parameters. The error ellipses and the error ellipsoids are defined by calculating the eigenvalues and eigenvectors from the variance-covariance matrix or its submatrix. The probability of the point to lie inside the standard error ellipse (k=1) is 0.394. In case of the standard error ellipsoid the probability is 0.199. (Cooper, 1987).

The standard error ellipses in three different planes (XY, XZ, YZ) are visualized with the MMD tool. Standard error ellipsoids are also calculated and visualized.

5. CONCLUSIONS

The design of measurements using three-dimensional CAD-models enables the more detailed measurement plan. Using network simulation, it is possible to consider the reachable measurement accuracies in advance.

The MMD tool can produce the measurement model. The measurement model defines the steps of the measurement process. Simulated image coordinates or camera orientation values can be used as approximate values for the measurement system. The measurement situation is also visualized using the tool.

The tool can also be used in the design of calibration measurements. After deciding the volume in the object space to be calibrated, the visualization properties of the tool can be used to check that the calibration points are well-selected. By well-selected is meant, that calibration points are spread out everywhere in the object volume, and also that calibration point distribution in every image enables the reliable calculation of image distortions.

The measurement model design (MMD) tool and its simulation part form an experts' tool for planning video digitizing. The tool uses 3D-models of the objects and the measuring environments. The calibration data of the cameras can be taken into account in simulation, or the tool can be used for the planning of the calibration stage itself. The simulated precision values give information about the achievable accuracy of measuring arrangement. This is very important information for the designer of the measurements. The visualization and simulation properties of the tool are possible to expand and the work is continuing with this.

REFERENCES:

Ailisto, H., Mitikka, R., Moring, I., Kaisto, I., 1995. Optical three-dimensional coordinate meter with vision guidance for submillimeter pointing to tags,. Optical Engineering, 34(9) , pp. 2603-2610.

Brown, D.C., 1980. Application of Close-Range Photogrammetry to Measurements of Structures in Orbit. GSI Tehcnical Report No. 80-012, Geodetic Services Inc., Melbourne, Florida, Vol. 1, 131 p.

Cooper, M.A.R., 1987. Control Surveys in Civil Engineering. Collins Professional and Technical Books, Williams Collins Sons & Ltd, London. 381 p.

Cowan, G.K., et al., 1988. Automatic sensor placement from vision task requirements. IEEE Transactions on Pattern Analysis and Machine Intelligence. 10(3), pp.407-416.

Fraser, C.S. 1989. Optimization of Networks in Non-Topographic Photogrammetry. In: Non-Topographic Photogrammetry, Second Edition 1989. H.M. Karara(Editor-in-chief), American Society for Photogrammetry and Remote Sensing, pp. 95-105.

Luhmann, T., 1994. Rechnergestützte Planung und Berechnung von Aufnahmekonfigurationen für CCD-Bildsensoren. Zeitschrift für Photogrammetrie und Fernenkundung 3/1994. pp. 103-110.

Mason, S.O., 1994. Expert System-Based Design of Photogrammetric Networks. Institute of Geodesy and Photogrammetry, ETH. 187 p.

Mikhail, E.M., 1976. Observations and least squares. University Press of America, New York. 497 p.


Katri Elina Oksanen <Katri.Ok anen@hut.fi>

WWW-version of this article created: July 19, 1996