1. Introduction

The most common feature of modern optical measurements obtained with stereoscopic and tomographic systems, is the  viewing angle of registries. Examples are measurement systems using stereoscopic and tomographic PIV and other correlation methods [1, 2]. Optical measurement systems in which the object plane is not perpendicular to the optical system axis, to obtain high-quality images require appropriate correction, which is carried out in accordance with the Scheimpflug principle [3]. Due to the proliferation of multichannel measuring systems, the Scheimpflug correction application is increased. The  related works often suggest this correction to significantly extend the depth of field [4-6], while the work [6] clarifies that the term depth of field does not mean the full depth of field of the optical system, but only the depth of field of the scanning laser, i.e. part of the measuring system. In [7] it is reported that in accordance with the Scheimpflug principle, the studied flow section focuses on the CCD matrix in the best way, but it’s not specified what is meant. Taking into account that in modern optical measuring systems one of the advantages is the possibility of using a non diffraction-limited optics, it is important to study the influence of Scheimpflug correction on the depth of field in real optical systems.

In case of obtaining an oblique object image  with focus on the center of the object plane, the depth of field is obtained for the peripheral part of the frame.. The use of Scheimpflug correction allows to expand the sharpness area in comparison with the images obtained without correction. It is obvious that within the sharpness area of the frame, the spatial resolution is higher than outside it. Despite the fact that it is difficult to take into account all the measurements error components performed by correlation methods, it can be assumed that within the image area with a higher resolution, the measurements error will be less.

In this paper we propose a method for visualizing the sharpness area in order to localize the measurement area with potentially higher accuracy. The correlation methods often employ limiting the region of interest (ROI). This makes it possible to significantly increase the processing speed, as well as to avoid the appearance of erroneous vectors that appear outside the zone of the studied flow when processing experimental images. Visualizing the sharpness area allows you to determine the size and position of the ROI if it can be limited to the sharpest image area.

 

2. Visualization of background images sharpness area by the contrast distribution

2.1. The background images registration scheme

To estimate the dependence of the depth of field on the matrix tilt angle, a series of experiments was conducted. The scheme of the experimental setup is shown in figure 1. The background 1, mounted on the rotation platform 2, is illuminated by incoherent radiation. A black background with white circles evenly distributed in a staggered order was used. Its format is A3, the circle diameter is 4 mm and the distance between the circles centers is 10 mm. Background image is formed by the lens 3 on the matrix of the digital video camera 5. Rotation of the sensor plane relative to the optical axis was carried out using the Scheimpflug adapter 4. Measurements were carried out for the object distance of 1500 mm, which corresponds to thirty focal lengths. In this case it is possible to obtain background images with a noticeable difference between the sharp center of the frame and the unsharp periphery. The value of the viewing angle was changing from 0° to 40° with increments 5°. The matrix tilt angle was changing from 0° to 10° with increments of 2°. The experiment used a video camera Videoscan 285 USB with monochrome CCD-matrix Sony ICX285AL [8], the image from the camera via USB interface is transmitted to a personal computer 6. To ensure rotation the camera and not the lens by rotating the adapter screw, the adapter was fixed on an unmovable base 7, which like a rotation platform 2, mounted on an optical rail 8. This ensures the intersection of the background rotation axis and the camera matrix axis with the optical system axis. In this research a lens with a fixed focal length Nikon AF Nikkor 50 mm 1:1.8 D and the adapter LaVision Scheimpflug Mount version 3 was used.

 

Fig. 1 Scheme of background images registration

 

2.2.  Background images processing algorithm and visualization results

The modulation transfer coefficient (contrast) was selected as a criterion of sharpness change. The background central part is displayed by the optical system with a higher quality, i.e. more sharply. The periphery sharpness is significantly worse at the background, which is expressed not only in spots blurring but also an intensity decreasing. The variation between maximum and minimum intensity values is decreased. The contrast can be expressed as the ratio of the difference between the maximum and minimum signal intensities to their sum. In present case, the contrast will not be constant across the field. Figure 2 shows an example of a background image and its fragments: the central and peripheral parts of the frame.

 

Fig. 2 An example of a background image: α = 10°, q = 40°

 

The analyzed image is divides into rectangular windows of size 58 × 52 pixels. For a chip size of 1392 × 1040 effective pixels we get 24 × 20 = 480 windows without overlap. Further, the windows are processed in order from left to right, from top to bottom, starting from the top left. Within each window the maximum and minimum values of the signal are determined and their arithmetic mean is calculated. The mean is a threshold, the signal values above which will be written to the maximum values vector, and below – to the minimum values vector. The average value of the light and dark pixels vectors is computed. Then the image contrast within the window is calculated by the standard formula

 

.

 

When the contrast was determined for each window, it is possible to visualize the distribution of contrast across the frame. Figure 3 shows a block diagram of the contrast distribution visualization algorithm. In this research the implementation of the algorithm was performed in Mathcad.

 

Fig. 3 The contrast distribution visualization algorithm

 

Figure 4 shows examples distributions of contrast for a background at viewing angle of 10º and 40º obtained at the matrix tilt angle of 10º.

 

Fig. 4 The distributions of the contrast

 

The boundaries of the sharpness area are parallel to the camera matrix rotation axis, in this case – almost vertically. With the background rotation angle increase sharpness area width is reduced. By setting the contrast threshold value for the resulting distributions sharpness area size can be estimated. Figure 5 shows the dynamics of the sharpness area width changing with increasing matrix plane tilting for different values of the object plane tilting angle.

 

Fig. 5 Dynamics of sharpness area changes when the matrix plane tilting at the angle range from 0° to 10°

 

As the sharpness area width is narrowed, both the matrix tilting and at large the viewing angles are increased. The uneven distribution of contrast on the frames is obtained at the normal position of the screen (q = 0°). This is due to the uneven illumination of the background. Due to much illumination of the upper right corner of the image, the contrast is reduced, which is well confirmed by the visualization results. It can also be seen that when the background is tilted at 40°, the contrast decreases across the field of view. However, the tendency to narrow the sharpness area as the matrix tilting plane increases is also observed for these images. The border of the background screen is also visualized on the right on these contrast distributions. The area with low contrast at the bottom of the frame corresponds to the image of the metal ruler used for the calibration scale.

Thus, at this stage of the research the following disadvantages of the proposed methodology for estimation the size and position of the sharpness area were identified: sensitivity to the light distribution on the frame and to the overall illumination change during measurements. Also, the type of adapter used in the research does not provide the camera positioning with the required accuracy: about 0.4° for every 10° tilt background. To estimate the dependence of the sharpness area size on the relative position of the object plane and the image one without taking into account the negative influence of these factors, a computer simulation of the stereoscopic system measuring arm was performed.

3. Visualization of field sharpness on synthesized images

3.1.  Model parameters

A lens with a focal length of 52 mm was chosen for optical part of the simulation of the stereoscopic system  measurement. The aperture diaphragm is located between the second and third lenses. The simulation was performed for three wavelengths: 450 nm, 550 nm and 650 nm. Primary wavelength is 550 nm. Figure 6 schematically shows the rays’ path. Distributions were obtained for the object distance A = 500 mm and aperture f ¢/8.

 

Fig. 6 The rays’ path in the optical system with the image plane tilt: q = 40°, a = 5°

 

Range of viewing angles was 0° - 40° with increments of 5°. For each value of the viewing angle q according to the formula [5]

 

 

the corresponding angle of the image plane tilt a was calculated, which provides the maximum degree of image correction for this configuration. The calculation results are shown in table 1.

 

Table 1 Image plane tilting angles for relative object distance A/f ' = 9.5.

 

3.2. Simulation results

The Image Simulation was used to synthesize background images. At first three series images was synthesized for the same background: a) the viewing angle q changes from 0° to 40° with increments of 5° without Scheimpflug correction (α = 0°); b) the background was located normal to the optical axis, the sensor tilting angle a changes from 0° to 10° in 1° increments; c) the viewing angle q changes from 0° to 40° with increments of 5° with Scheimpflug correction (table. 1). To obtain the contrast distribution, the synthesized images were processed by the method described above. Examples of synthesized images and visualization results are shown in figure 7.

 

Fig. 7 Examples of synthesized images (left) and corresponding contrast distributions (right)

a) q = 40°, a = 0°; b) q = 0°, a = 8°; c) q= 40°, a = 5°

 

The synthesized images were obtained taking into account the lens residual aberrations and the detector settings corresponding to the chip ICX285AL parameters: size of square pixel is 6.45 µm; the number of pixels is 1392 × 1040. It is visually difficult to distinguish the images shown at fig. 7 left, however, further processing allows you to visualize areas of the frame that are displayed sharply. The sharpness area, as in the case of real background images, is oriented parallel to the background rotation axis and, accordingly, the image rotation axis. The boundaries of this area can be recognized by a rapid transition from a higher level of contrast to a lower one. For example, in Fig. 7b there is a sharp transition from yellow to blue. It was suggested that the local contrast level decrease in the frame center occurs due to the background structure.

The contrast distribution examples are a good illustrated the following:

-  the sharpness area is decreased for both the background tilting and the image plane tilting;

-  the size of sharpness area for the same viewing angle is increased if the Scheimpflug correction is carried out;

-  however, it was not possible to significantly expand the sharpness area of the image at the same angles in accordance with the Scheimpflug principle, despite theory predicts [3].

To clarify the degree of influence of the background structure, two series of the dot background images (dot pattern) were synthesized for the same optical system model: with and without Scheimpflug correction. Background images were processed in the same way. The dynamics of changes the background images and contrasts distributions are shown in fig. 8. The contrast decreases with increasing observation angle. At low angles, the sharpness area is not rectangular, but has an elliptical shape, a large half-axis oriented parallel to the rotation axis. It was suggested that this is due to an image quality loss from the center field of view to the periphery due to lens aberrations.

 

Fig. 8 Dynamics of changing the background images (top) and contrast distribution (bottom) with increasing viewing angle for the dot pattern with Scheimpflug correction (left) and without one (right)

 

It should be taken into account that the oblique objects image taking images cease to be symmetrical with respect to the optical axis and differs for the two halves of the pupil, symmetrical with respect to the object plane rotation axis. In Fig. 9 the field of view of the simulated optical system is shown. The round field of view given by the lens is limited by the rectangular aperture of the detector. The rotation axis is vertical and passes through of the field of view center.

 

Fig. 9 Fields of view for which the point spread function (PSF) changes are shown further (Footprint Diagram)

 

Fig. 10 shows the displacement dynamics of the traces intersection of the rays with the image plane for increasing viewing angles. Fig. 11 and fig. 12 shows the PSF dynamics for the edge (± 17°) and the half (± 8.5°) of the field of view in a direction perpendicular to the background and the image plane rotation axis. PSF were obtained for primary wavelength 550 nm. For other wavelengths of the visible range, a similar pattern is observed with small changes due to chromatic aberrations. In the integral light spots, respectively, are larger, and their structure is more difficult.

 

Fig. 10 Dynamics of the Footprint Diagram changes with viewing angle increasing

 

For the half of the field of view (fig. 11) it is demonstrated that during viewing angle increasing the Scheimpflug correction provides almost unchanged as well image structure as the Strehl ratio. Without correction, the point image structure is almost unchanged, but the image is shifted in accordance with the direction of background rotation.

 

Fig. 11 PSF changes dynamics for the field of view of ±8.5°: with Scheimpflug correction (top) and without correction (bottom)

 

For the field of view edge (fig. 12) PSF becomes significantly different for field points symmetric about the rotation axis. As the viewing angle increases, the spots pattern changes greatly and the Scheimpflug correction does not allow refining the image quality. All the above images clearly show aberrations coma and distortions. For comparison, fig. 13 shows PSF for the field of view center, it is a classical diffraction image with a central core and several concentric rings of low intensity. The Strehl ratio 0.856 also corresponds to the diffraction quality. For the vertical axis, the image quality will not change as the background is rotated. Since contrast distributions were obtained for the whole field of view, including at extreme field angles, it was not possible to obtain a large sharpness area even in the case of Scheimpflug correction.

 

Fig. 12 PSF changes dynamics for the field of view of ±17°: with Scheimpflug correction (top) and without correction (bottom)

 

4. Conclusion

Numerical and physical modeling of the measuring arm of a stereoscopic optical system was performed. The algorithm of the sharpness area visualization on contrast distribution across the background image is offered.

The method of the sharpness area size estimation on contrast distribution has a few disadvantages: sensitivity to the level of illumination, as well as to the choice of the contrast threshold below which the image is considered to be blurred. In [9], the authors proposed a method for the sharpness area size estimation on changing the images columns standard deviation from the first one in the direction perpendicular to the matrix rotation axis. Combining this two methods can improve the accuracy of the sharpness area estimation.

The results obtained for the same background on computer and physical models are qualitatively the same. The structure of the background influences on the distribution of contrast. The use of a small-scale irregular image as a background revealed another factor that has a significant influence on the contrast distribution in the presence of significant optical aberrations in the system, and thus characterizes the depth of field, namely – the image quality given by the lens. The results of the physical experiment and computer simulation can be treated as the approximation of the real optical system taking into account the distribution of aberrations on the field of view.

References

1.  Raffel M. et al. Particle Image Velocimetry. A Practical Guide / Raffel M., Willert C.E., Scarano F., Kähler C., Wereley S.T., Kompenhans J. Springer International Publishing, 2018. – 669 ð. DOI 10.1007/978-3-319-68852-7.

2.  Sutton M. A., Orteu J. J., Schreier H. Image Correlation for Shape, Motion and Deformation Measurements. Basic Concepts, Theory and Applications. Springer US, 2009. – 322 ð. DOI 10.1007/978-0-387-78747-3.

3.  Merklinger H. M. Focusing the View Camera, Published by the author: Harold M. Merklinger P. O. Box 494 Dartmouth, Nova Scotia Canada, B2Y 3Y8. Internet Edition (v. 1.6.1) 8 Jan 2007. 146 p. https://www.cs.cmu.edu/~ILIM/courses/vision-sensors/readings/FVC16.pdf.

4.  Depth of field for the tilted lens plane © 2008 Leonard Evens. http://www.math.northwestern.edu/~len/photos/pages/tilt_dof.pdf.

5.  Cong S., Haibo L., Mengna J., Shengyi C. Review of Calibration Methods for Scheimpflug Camera // Journal of Sensors, Volume 2018, Article ID 3901431, 15 p. 

6.  Li J. et al. Large Depth-of-view portable three-dimensional laser scanner and its segmental calibration for robot vision / Li J., Guo Y., Zhu J., Lin X., Xin Y., Duan K., Tang Q. // Optics and Laser in Engineering, 2007, vol. 45, pp. 1077-1087.

7.  Particle Image Velocimetry. User’s manual "ActualFlow". Novosibirsk: LTD SigmaPRO. 2016. http://polis-instruments.ru.

8.  Sony ICX285AL CCD Image Sensor datasheet. http://datasheet.elcodis.com/pdf2/107/40/1074000/icx285al.pdf.

9.  Pechinskaya O. V., Sangadzhieva E. D., Skornyakova N. M. Investigation of the influence lens axis inclination to the camera sensor axis on depth of field // Measurement Techniques ¹1, 2019, pp. 31-33 DOI: 10.32446/0368-1025it.2019-1-31-33