Building a Stereo Camera. Scientific and Technical Aspects - Literature review Example

Building a stereo camera Literature review Introduction Photography was started in the 19thcentury. Eadweard Muybridge a British-born used 12 similarly spaced cameras to capture pictures of a galloping horse. The pictures were then used in a rotating drum which had small viewing openings that when spun made the captured pictures in the drum seem to be on motion. Eadweard found a way which known as the Zoopraxiscope to project the still images of motion on a screen (Pupilli & Calway, 2005). One of the most common problem in 3D camera filming results from the use of cameras with the converged rather than the parallel lens axis. This absolutely causes vertical parallax and horizontal parallax and contributes greatly to eyestrain. It has been proved that the closer the converged item or object gets to the camera, the more eyestrain and fusion is impossible. It is generally considered that parallel cameras usually give better and perfect overlays (Pupilli & Calway, 2005). However, even the parallel cameras with the perfectly matched lenses tend to give huge distortions and thus spherical lenses should be used. If there is the need to converge, one can do it without the distortions by horizontally shifting the lens rather than changing the entire camera or its imaging chip. A large literature on the stereo camera image rectification deals with trying to correct the converged into parallel cameras. According to (Chen, Khatibi & Kulesza, 2007), a system of stereo vision was introduced to generate 3D models of the real objects automatically. Stereo camera calibration was introduced. This helped in rectifying the range images and stereo images of an item from multiple viewpoints. Two major approaches were achieved; the first one was based on processing the photographic images using the volumetric reconstruction technique and the second one was based on merging of the multi-view range images into one complete 3D model. The generation of the 3D model is based on merging of the multi-view range images that were obtained from the stereo camera. The images obtained are then registered automatically to a common coordinate system then they are integrated into the 3D mesh model. Camera Calibration Calibration is a key property of 3D camera construction. Generally the internal parameters of a camera are accurately known before the construction and the overall environment is usually highly controlled, or a calibration item in the scene is used in calibration of the camera. However, in most situations the source of the images is unknown which leads to the conclusion that the camera’s parameters are also unknown or it is necessary to adjust the camera midway during an image application. Thus, it leads to the conclusion that the internal parameters of the camera are and can only be determined from the images themselves. Classical Calibration Methods This classical calibration method makes the use of a calibration pattern of a determined known size inside the view of a camera. Sometimes this is usually a flat plate with some regular pattern marked on it or a scene that contains some control points and with known coordinates. A major disadvantage of these methods is the impossibility to calibrate the camera while it is involved in the imaging task (Fusiello, Trucco, & Verri, 2000). If any change occurs in the camera’s settings, a correction is not easily possible without the interruption of the task. The change of the settings of the camera may be a change in the focal length, or small thermal or mechanical changes affecting the camera. Estimating the Perspective Projection Matrix By reducing the image error, the perspective projection matrix is estimated for n 3D points Mi corresponding to image point mi. This error of the image is taken as the distance between actual image points and projection of the world points onto image plane using P. Making use of the equation [sm˜ = PM˜], with m˜ = [u, v, 1]T and M˜ = [X, Y, Z, 1]T , we can obtain three equations, but if we divide by the third one, we get two equations in the 12 unknown parameters of P ; u = P11X + P12Y + P13Z + P14 P31X + P32Y + P33Z + P34 v = P21X + P22Y + P23Z + P24 P31X + P32Y + P33Z + P34 A function that needs to be minimized defined as the squared geometric distance separating the projected image points and the actual image points:  +  ] The error function above is non-linear and may be minimized using the Levenberg-Marquardt Minimization algorithm. Between iterations, the matrix P is usually scaled (||P|| = 1) or one parameter of P can be fixed (P34 = 1). To find an initial estimate, the equations of (6.1) are rearranged, so that the algebraic distance Ea is minimized instead of minimizing geometric distance Eg, Ea = P31X + P32Y + P33Z + P34) - (P11X + P12Y + P13Z + P14 + (vi (P31X + P32Y + P33Z + P34) − (P21X + P22Y + P23Z + P24) This function of the error is linear in the unknown parameters of P and can also be rearranged into the following; min || Zp|, p Subject to ||p| = 1. Extracting the Camera Calibration Matrix When the perspective projection matrix P is estimated, it is decomposed into the form of equation sm˜ = K [ R t ] M˜. The following matrix of P maybe expressed as: [P11 P12 P13] [P21 P22 P23] = K R, [P31 P32 P33] where K is the camera calibration matrix that is upper triangular and R is orthogonal. Other methods have been used to calibrate cameras. For instance a calibration grid that has markers is also used in the calibration of cameras. The markers’ 3D coordinates are known and hence techniques such as Direct Linear Transformation and Bundle Adjustment are then used to estimate accurately the internal parameters of the camera (Fusiello, Trucco, & Verri, 2000). Mostly it is more than one image of the same calibration item with various orientations is used. Depth of Field The depth of field usually denoted by DOF is the portion of the scene that usually appears sharp on the sensor in the image. Though a lens can correctly focus at one distance only, the decrease in the sharpness of the image is gradual on either side of the focused distance, as such within the DOF, the image as seen as in focus under the conditions of normal viewing. In calculating the depth of field at specific distance there is the need of knowing th characteristics of the lens, that is; the circle of confusion of the sensor and the hyper focal distance of the lens. On circle of Confusion, using any lens a correct precise focus is usually possible only at the focus distance. Here, a point object produces a point image. On any other distance, point object is normally defocused and thus produces a blur spot shaped circular in shape. When the blur spot is significantly small, it is not distinguishable from the point and appears to be in focus. The diameter of the blur spot gradually increases with the distance from the focus point and the largest diameter blur spot that is not distinguishable from a point is known as circle of confusion. The area of the scene in the depth of field can appear sharp and the areas beyond and in front of the DOF appear blurred. Thus Zeiss formula can be used in calculating the circle of confusion C, where d is the sensor’s diagonal. C = d/1730 Hyper focal distance refers to the nearest focus distance where the DOF extends to infinity. The focusing of the camera at this distance usually results in the biggest possible depth for the field of a given number. Focusing beyond this hyper focal distance do not increase the far DOF but decreases near DOF in front of the object, thus generally this decreases the total DOF. A new algorithm of solving the N-Camera stereo correspondence problem by turning it into maximum flow problem have been described. The results have shown improved depth estimation and the better handling of the depth discontinuities. A wide angle lenses have been used to enhance the depth of the images. It has also been noted that if stereo cameras have higher resolution, it alleviates the distortions (Falcou, Serot & F. Jurie, 2005). If a stereo camera resolution can be increased and enhanced by an order of magnitude, the stereoscopic camera system may reach the stereoscopic depth threshold of a human. Wide inter-view point distances would thus not be necessarily in increasing the depth resolution and this in turn would lead to a reduction in the distortion to the resolution ratio. Object recognition (Soon-Yong & Murali, 2005) proposed a 3D reconstruction method of the small scale scenes improved by new image acquisition method. They used the stereovision system in acquiring the images so as to obtain several shots of the object. They also used the Scale Invariant Feature Transform (SIFT) method since the points of interest are invariant to the image rotation and scaling and partially invariant to camera viewpoints. (6) Introduced a new technique for image feature generation that was known as the Scale Invariant Feature Transform (SIFT). This approach is associated in transforming an image into large collection of the local feature vectors that are invariant to image scaling, rotation and translation and partially invariant to 3D projection. The scientists achieved effective or maximum robustness by being able to detect many feature types and then relying on the clustering and indexing to select the ones that are most useful in a certain or particular image. References Chen, Khatibi, & Kulesza, 2007. “Planning of a multi stereo visual sensor system – depth accuracy and variable baseline approach,” Proceedings of IEEE Computer Society 3DTV Con, the True Vision Capture, Transmission and Display of 3D Video. Falcou, Serot & Jurie, 2005 “A Parallel Implementation of a 3D Reconstruction Algorithm for RealTime Vision. Parallel Computing Fusiello, Trucco, & Verri, 2000. A compact algorithm for rectification of stereo pairs. Machine Vision and Applications. Pupilli, Calway, 2005. Real-time Camera Tracking Using a Particle Filter. In Proceedings of British Machine Vision Conference, Oxford. Soon-Yong & Murali, 2005. Multiview 3D modeling system based on stereo vision techniques, Machine Vision and Applications. Read More