Henrik Haggrén
Helsinki University of Technology
Institute of Photogrammetry and Remote Sensing
FIN-02150 Espoo, Finland
E-mail Henrik.Haggren@hut.fi
"The Role of Models in Automated Scene Analysis", Joint Workshop of ISPRS WG III/2 and IC WG II/III, August 30 - September 1, 1995, Stockholm, Sweden, Photogrammetric Reports, No. 63, Editors: K. Torlegĺrd, E. Gülch, 16 p., Stockholm 1995.
Implementation Issues for Orientation Algorithms
Coordinate Systems
- Image. The image coordinate system is the one in which the image will be recorded. It is two-dimensional and presumed to be a plane. As it regards digitized images, the datum relates to the CCD array of the imager chip, to the frame grabber clock, or to the film digitizer geometry.
- Camera. The camera coordinate system is three-dimensional with its origin in the perspective Centre. The datum relates to the image plane parallel to two of the coordinate axes and is determined by coordinates of the fiducial marks and the camera constant.
- Stereo. The stereo model is a result of a relative orientation between two images or between two camera coordinate systems. The coordinate system is three-dimensional. Its origin is usually first defined to be one of the perspective centers. The primary coordinate axis is aligned by the second perspective Centre, whereas the secondary axes coincide with the principal horizontal and principal vertical.
- Object. The object model is a result of several images, camera coordinate systems, or stereo models transformed to a common coordinate system. Unless the object coordinates are transformed to a local world coordinate system, a principal datum can be defined by any other 3-D coordinate system, e.g. one of the stereo model coordinate systems.
- Design. The design model is either analog or digital model of an object. It may be e.g. a clay model or a CAD model. In such cases the datum is fictitious and relates to geometric entities, like planes, rotation axes, their intersections, etc. The coordinate systems are three-dimensional and can vary upon functional definition from traditional rectangular systems to parametric ones.
- World. The world coordinate system is any local or global system to be finally referred by the above mentioned other coordinate systems. The world coordinate system is usually monumented by control points for local referencing.
Datums
The datum is the basis of any coordinate system.
For transformations between different coordinate systems we distinguish the internal and external datums.
Each internal datum is defined for the purpose of the process use only, whereas an external datum is defined for the purpose of external use.
For example, the image, camera, and stereo model coordinates relate primarily to internal datums, and the object model, design model and world coordinates relate to external datums.
- External Datum. The external datum is by default uniform.
It is a 3-D coordinate system that is both orthogonal and of metric unity.
The orthogonality relates to a rectangular coordinate system and ensures the geometric conformity as it regards the rotated geometries.
The metric unity ensures that the unit vector is referred to the metric system.
The local world coordinate system is usually orthogonal and metric.
Similarly, the coordinate system of any design model is by intention orthogonal and metric.
An external 3-D datum is usually definite by a minimum of seven coordinate values.
These can be any coordinate combinations produced by at least three control points.
- Internal Datum. Opposite to the external datum the default for any internal coordinate system is that they are neither orthogonal nor metric unless they have been externally calibrated.
However, this kind of projective 3-D datum is well defined with respect to that internal control function it is used for.
For example, in a 2-D machine vision application the internal datum is very likely the original image coordinate system and all actions are controlled relative to that scene.
Similarly, we are capable of controlling us more precisely in 3-D relative to our actual perception than by knowing an orthogonal datum.
The internal datum can be either 2-D or 3-D.
A 2-D projective datum becomes orthogonally definite by at least four XY-control points or by five control distances, and a 3-D projective datum by at least five control points or by nine control distances.
In the case the datum is orthogonal, the minimum for a 2-D datum definition is two control points, and for a 3-D datum the seven coordinate values or the three control points.
- Orientation to Datum. In the case the object geometry defined in one datum should be transformed to any higher datum the transformation is solved by determining the orientation of said geometry.
The orientation parameters usually include six parameters: three rotations around the coordinate axes and three shifts along them.
This assumes both coordinate systems to be both orthogonal and of same scale.
If that is not the case, the scale will increase the number of parameters by one, the orthogonality by eight.
In the case the orthogonality is solved separately, this should be done first, as the three rotation parameters assume orthogonality of both coordinate systems.
- Orthogonality. The local coordinate systems are expectably both orthogonal and metric.
A camera coordinate system is orthogonal by default only, if the transformation from image to camera coordinates is based on calibration certificate.
The coordinate system according to which the fiducial coordinates and the camera constant have been calibrated should be de facto orthogonal.
In the case the camera calibration will be performed on-the-job, the orthogonality can be solved under certain conditions.
These may be:
- A minimum of three images with the same interior orientation are used for reconstruction.
- A minimum of eight distances of a common scale reference will recover the orthogonality.
For external scaling, at least one external scale reference should be included to reconstruction.
The ill-conditioned cases are the more likely, the closer the three cameras locate along one straight line.
Similarly, the eight distances should be directed randomly and not parallel to each other.
Orientations
The orientation is the procedure where the transformation parameters from one coordinate system to a second coordinate system are determined.
A 2-D orientation includes the shift and rotation of an object along a plane, and a 3-D orientation includes the determination of the position and attitude of an object in both coordinate systems.
In photogrammetry, the orientations are described as interior or exterior orientations, or as relative and absolute orientations.
The division to interior and exterior orientation relates to the camera body, whereas the relative and absolute orientations were introduced aside exterior orientations for the purpose of operating analog stereo plotters.
In following both the traditional definitions of orientations are described and the recent definitions outlined with respect to procedural aspects.
Interior orientation. The interior orientation is the determining of the interior perspective of the image as it is or was at the instant of recording.
The datum for the orientation is most likely a calibrated camera coordinate system.
The transformation results in a 3-D bundle of imaging rays.
The collinearity condition will be fulfilled by compensating the non-linear imaging distortions which are given as functions of the camera coordinates.
The ordinary four parameters for the interior orientation are the 2-D coordinates of the location of the principal point and the camera constant and the rotation around optical axis.
In the case of affine transformation two further parameters are used, namely the aspect ratio and the angle between the coordinate axes.
The number of non-linear distortion parameters vary largely upon purpose.
In the case of analog images the interior orientation is an image related variable.
As it regards the measuring procedure, the interior orientation is a preceding operation before the actual object measurements.
It includes the observation of fiducial marks in the coordinate system of the measuring instrument first, and then an adjustment for solving the parameter values for interior orientation.
The same values are then used for transforming the actual object measurements to the camera coordinate datum.
The transformation will be different for all subsequent images and therefore the procedure has to be repeated separately.
In the case of digital cameras the interior orientation is a camera related variable.
In the case of imaging system, which is a combination of a video camera, a digitizer and cables in-between, perhaps a feature projector, the interior orientation is a system related variable.
As it regards the measuring procedure, this simplifies the entire interior orientation largely, as the transformation parameters have to be determined only once for each camera or system.
Exterior orientation. The exterior orientation is the determining of the exterior perspective of the image as it is or was at the instant of recording, i.e. by determining the position of the camera station and the attitude of the camera at that instant.
By exterior orientation the bundles are oriented in such way that the collinearity condition will be fulfilled all along to the object space.
After exterior orientation of at least two images, new object points and features can be coordinated in 3-D by intersection.
The datum is external and usually a local one or an object model coordinate system.
The ordinary six parameters for the exterior orientation are the 3-D coordinates of the projection Centre and the three rotations around the coordinate axes.
The exterior orientation is determined directly by resection or indirectly by block adjustment.
For resection, the 3-D object geometry should be known by at least three control points, but in the case of digital images more likely by a set of geometric object entities or features.
In block adjustment, the exterior orientations are determined for several images simultaneously relative to a given external datum.
In the case the object geometry is still largely unknown, the block adjustment gives a more precise determination of exterior orientations than the resections using control points.
In the case the interior orientation is unknown, the five parameters of the interior orientation are included to the exterior orientation.
The transformation is called 11-parameter transformation or DLT, Direct Linear Transformation.
Then at least six XYZ control have to be used for determining of the parameters.
The 11-parameter transformations are not used for block adjustments as they cannot utilize any image-to-image related observations but only the image-to-object related ones.
As it regards the usual measuring procedure, the exterior orientation is an image related variable.
However, in fixed video camera configurations like in real-time photogrammetric stations, both the exterior and interior orientations are camera related variables.
They are all determined by block adjustment during the set-up calibration of the station and thereafter considered as fixed variables unless there is no need for updates.
In some applications the exterior orientations, or some parameters of it, are determined by direct measurements.
Examples of these can be found in aerial triangulation, in GIS data collection, and in close range video profiling.
The exterior orientations are observations of type of relative and kinematic GPS and inertial surveying.
In video profiling the position of the camera system is externally controlled in one coordinate direction and that reduces the number of the parameters by one.
Relative Orientation. Traditionally the relative orientation is defined as the determining of the position and attitude of one of a pair of overlapping images with respect to another image. The practical procedure of relative orientation includes the observation of homologue image points in both images. These produce correspondingly homologue bundles which are then transformed for mutual intersection. During the relative orientation, the datum is usually defined to be one of the images or its camera model. After relative orientation, new object points and features can be coordinated in 3-D space by the said intersections. This 3-D coordinate system is called here the stereo model.
The relative orientation of two images is a transformation of five parameters. According to datum definition, these parameters are chosen usually in two alternative ways. In the case the projection centers define the datum, five rotations are used. In the case one of the cameras define the datum, two shifts and three rotations of the second camera are used. These both assume that the orthogonality is included by camera calibration.
In the case the interior orientation is unknown, the five parameters of the interior orientation of the second image are included to the relative orientation.
The number of parameters for relative orientation becomes seven and they are e.g. the 2-D epipole coordinates, the three rotations, the aspect ratio and the angle between the image coordinate axes.
The resulting stereo model conforms to the image coordinate system of the first image.
The further relative orientation of such stereo model to an external datum is similarly a seven parameter transformation up to scale. After scaling it includes 15 parameters like the 3-D projective transformation.
This was under the assumption, that the images were taken with different interior orientations. (Note by the author: This chapter is not proof!)
During recent times, the relative orientation is used also for definition of determining the position and attitude of one of a pair of overlapping stereo models with respect to another stereo model.
The common 3-D coordinate system is called then as an object model.
This kind of relative orientation is a 3-D conformal coordinate transformation of seven parameters, like three shifts, three rotations and scaling.
The relative orientation is thus more likely the procedure of collecting separate object models into a common datum in order to form a entire surface model of spatial objects.
The need of this kind of relative orientation comes from general 3-D modeling applications like reverse engineering or factory design.
Absolute Orientation. The absolute orientation is used for determining the final coordinate transformation of the model coordinates to the external datum like the one of the local coordinate system or of the design model.
It includes seven parameters like three shifts, three rotations and one scaling.
In the case of projectively transformed 3-D model the number of parameters is 15.
These can be e.g. three shifts, three scales, and 3 x 3 rotations.