WO2013087184A1

WO2013087184A1 - Measuring arrangement and corresponding image processing method

Info

Publication number: WO2013087184A1
Application number: PCT/EP2012/005088
Authority: WO
Inventors: Jan-Friso Evers-Senne; Martin Stratmann; Matthias Schmieder; Jörg Lange
Original assignee: Testo Ag
Priority date: 2011-12-16
Filing date: 2012-12-10
Publication date: 2013-06-20
Also published as: CN204334775U; DE102011121334A1; DE102011121334B4

Abstract

For a measuring arrangement (1) having a detector (3) that records in the nonvisible spectral domain on a camera (2), it is proposed that an electrically and/or mechanically operable modality (8) be used between the recording of images (6, 7) to execute a relative shift in a depicted scene (10, 11) in relation to a two-dimensional recording field (4) of the detector (3) in order to obtain images (6, 7) of an essentially unaltered scene (15) which are able to be combined to form an SR image with increased output image resolution (Figure 4).

Description

Measuring arrangement and corresponding image processing method

The invention relates to a measuring arrangement comprising a camera accommodating in a non-visible spectral range, having a detector for recording in the non-visible spectral range, providing a recording field with a recording image resolution, and further comprising an image processing unit which is used to generate an SR. Image having an output image resolution by combining at least two images of a scene taken with the detector at the captured image resolution, the output image resolution being greater than the captured image resolution.

The invention further relates to an image processing method for improving the image resolution, comprising the steps Abbil- to a scene on a recordable image resolution defining recording field of a detector of a receiving in a non-sichtba ^¬ ren spectral camera, containing a quantity of images with the captured image resolution and generating a processed SR image by a combination of the captured images, wherein the output image resolution is greater than the recording image resolution includes.

It is known that the resolution of images is often limited by the imaging resolution of the receiving detector. This is especially true if the detector is to record in a non-visible spectral range.

There are suggestions as to how this resolution limit can be broken. This will be several pictures of the same

CONFIRMATION COPY Scene taken with the detector, and the captured images are combined in an optimization process to an SR image with a higher output image resolution. However, the proposals developed so far fail when the camera or the detector is to be set up stationary.

The invention therefore has the task of achieving an improved resolution for these cases, which exceeds the recording image resolution of the detector.

According to the invention, a mechanically and / or electrically operable modality is provided in a measuring arrangement of the type mentioned above with which a position of the imaged scene in the recording field of the detector can be changed. A modality here means a functional unit of the measuring arrangement and in particular the camera. The invention offers the advantage that a subpixel-accurate displacement of the scene imaged on the detector in relation to the recording image resolution of the detector can be carried out between the exposures without the need to move the measuring arrangement or the scene. This can be used particularly advantageously when the measuring arrangement and in particular the camera is stationary and when static scenes are viewed or measured.

The recording field defines a two-dimensional pixel arrangement, by which the recording image resolution is given.

To generate the image shift can be provided that the modality is set up to generate or excite mechanical vibrations. The results from the vibrations The movement of the imaged scene in the recording field of the detector is particularly suitable for generating the image shifts required for the calculation of an SR image with increased output image resolution.

For example, these mechanical vibrations can be generated by electrical, in particular piezoelectric, means or inductive means, in particular with at least one plunger coil, or by mechanical means, for example via imbalances or crank gears on a motor.

The use of mechanical vibrations offers the further advantage that the space requirement within the measuring arrangement is low, since the amplitudes of the oscillations can be kept small.

In an embodiment of the invention, it may be provided that the modality has an element which acts optically in the non-visible spectral range and whose optical properties are mechanically and / or electrically changeable in the non-visible spectral range. This can for example be achieved in that the optically acting element is deformed ^'in order to achieve altered optical properties, such as focal length and the like.

It can also be provided that the modality has an optically acting element in the non-visible spectral region whose position in a beam path of the camera, in particular with respect to a housing part of the camera, is mechanically and / or electrically changeable. Thus, the size and / or the image position, ie generally the position of the scene in the recording field of the detector can be changed in a simple manner, for example by focusing and / or zooming. factor and / or another property of the optical system of the camera is changed.

It can also be provided that the modality is set up to move at least part of the camera. This part may for example be the detector itself. In this way, it can be easily achieved that the position of the imaged scene in the recording image of the detector changes since the detector is moved relative to the scene.

It is particularly favorable if the modality is set up to carry out an irregular sequence of movements. The sequence of movements can thus be set up and executed in a random and / or uncontrolled manner according to type, direction, direction, size and / or displacement. The advantage here is that a simple method is feasible, with which random, subpixel accurate shifts of the imaged scene with respect to the recording field of the detector can be generated.

The invention has the advantage that the movement on the type, direction, sense of direction, size and / or adjustment must not be known. Thus, complex electronic or computational measures for monitoring a controlled shift or other movement are dispensable. As a result, the energy efficiency of the measuring arrangement according to the invention and of the method according to the invention can be further increased. In order to be able to assemble the recorded images into an SR image, it can be provided that the image processing unit has a coarse registration unit which is used to perform a coarse registration of the images from the set of recorded images is set up. Coarse registration here means a method in which mutually corresponding image pixels of at least two images are assigned to one another on the basis of the analyzed image contents in such a way that the images can at least approximately coincide in content with this assignment.

It can also be provided that the image processing unit is set up to carry out an optimization method, wherein a subpixel-precise displacement of the images relative to one another can be used as a variation parameter of the optimization method. For example, the optimization method can use an energy function as an optimization function in accordance with J. Flusser et al. : A unified approach to super resolution and multi-channel blind deconvolution, IEEE transactions on image processing, vol. 16, no. 9, September 2007, pp. 2322-2332. The image processing unit is preferably set up to carry out an optimization process which is downstream of the coarse registration. In this case, the variation parameter can describe a subpixel-accurate displacement of the already roughly registered images relative to each other.

The two-stage distribution of the generation of the SR image into a coarse registration and a downstream optimization method has the advantage that the SR images can be generated with reasonable computational effort and reasonable computing time.

To achieve the stated object in an image processing method described above, the invention provides that during and / or between the individual shots of the images with a mechanically and / or electrically operated modality a position of the imaged scene in the recording field is changed. The user thus has an opportunity to to selectively generate an image shift between the images of the images, which is required for the generation of an SR image with increased output image resolution, even if the receiving camera is arranged stationary.

For example, the change in position of the imaged scene can be achieved by generating or exciting a mechanical oscillation with the modality. In this case, it is particularly favorable if an irregular, in particular a random and / or uncontrolled, movement is carried out with the modality, since in this case interference effects can be avoided particularly well.

In order to carry out the change in position, it is possible to use the modality to mechanically and / or electrically change at least one optical property in the non-visible spectral range of the camera of an optically active element in this non-visible spectral range. This can be, for example, a magnification factor or a focal length or the position of an image plane into which images are sharpened.

Alternatively or additionally, it may be provided that the modality at least one position in a beam path of the camera of an optically acting in the non-visible spectral range element is changed mechanically and / or electrically operated.

It can also be provided that with the modality at least a part of the camera is moved. It is particularly advantageous if the movement takes place relative to a beam path of the camera. For example, the moving part of the camera may be the detector. The advantage here is that when detained Conditions in the optical beam path in a simple manner, a bearing change of the imaged scene in the recording field is achievable, since the detector is moved under the imaged scene.

In one embodiment of the invention, it can be provided that the images of the set of images in a coarse registration unit are automatically brought into a coarse registration. As a result, an assignment rule between the images to be combined can thus be provided, which assigns individual image pixels of the different images to one another in such a way that a content match results. This content match can be determined, for example, with the aid of an analysis of the image content, for example by a feature analysis and / or by evaluation of a correlation function.

It can also be provided that the recorded images are processed for combination in an optimization method, wherein a sub-pixel-precise displacement of the images relative to one another is used as a variation parameter of the optimization method. The optimization method is preferably downstream of a coarse registration, wherein the coarsely registered images are shifted relative to one another with subpixel precision. Here it is particularly favorable if the already mentioned energy function is optimized in the optimization process.

For example, it can be provided that in the optimization method for each of the images from the set of images, a convolution of an SR image, which is produced on a trial basis from the images, is calculated with a point spread function characterizing the variation parameter. The advantage here is that point spreading functions represent particularly easy to handle means. len to model subpixel accurate shifts of the images relative to each other. The quality of the sampled SR image can be easily calculated by determining the deviation of the result of the respective convolution to the individual images. The optimization method may then determine, for example, that SR image for which the deviations are minimal for all images of the set of images.

The invention will now be described in more detail with reference to embodiments, but is not limited to these embodiments. Further embodiments result from combining one or more features of the protection claims with each other and / or with one or more features of the embodiments.

It shows:

1 shows a measuring arrangement according to the invention in a three-dimensional oblique view from the front, FIG. 2 shows the measuring arrangement according to FIG. 1 in a three-dimensional oblique view from behind, FIG.

Fig. 3 the detector of a measuring arrangement according to the invention in a schematic representation to explain the image processing method according to the invention,

4 shows the detector according to FIG. 3, wherein the position of the imaged scene in the recording field of the detector changes

Fig. 5 is a schematic diagram of another invention

Measuring arrangement, Fig. FIG. 6 shows the measuring arrangement according to FIG. 5 with the optical property of an optically active element in the non-visible spectral range modified according to the invention and FIG

FIG. 7 shows the measuring arrangement according to FIG. 5, wherein the position of an element optically acting in the non-visible spectral range in a beam path is changed according to the invention.

A measuring arrangement designated as a whole by 1 in FIGS. 1 and 2 has, in a manner known per se, a camera 2 which is set up for recording in a non-visible spectral range.

In the illustrated embodiment, the camera 2 is a thermal imaging camera. In further exemplary embodiments, the camera 2 is designed as a UV camera, as a THz camera, as a μτη wave camera or as a general camera for recording non-VIS images in a non-visible spectral range.

The camera 2 has in its interior in a manner known per se a detector 3 which is sensitive in the non-visible spectral range and with which images can be recorded in this non-visible spectral range.

By way of example, FIG. 3 shows such a detector 3. The detector 3 provides a two-dimensional image field 4 which defines a recording image resolution.

This recording image resolution is in a detector with a two-dimensional array of sensor elements by the number and size of the sensor elements in the array and their distance from each other. In a detector in scanner technology, the recording image resolution is given by the size of the sensor element and the screening of the scanner mechanism.

The measuring arrangement 1 according to FIGS. 1 and 2 further has an image processing unit 5 in its interior.

In this case, the image processing unit 5 is set up as a data processing device such that a predefined set of images, which were recorded with the detector 3 with the acquisition image resolution, can be combined to form an SR image having an output image resolution which is higher than the acquisition image resolution.

In order to obtain useful results in the form of an SR image of appealing quality in the image processing unit 5, the image processing unit 5 needs as input variables images 6 and 7 which are of a scene which has remained substantially unchanged between the images.

In order to be able to obtain information from the images 6, 7 which belong to image distances below the smallest pixel spacing of the detector 3, a modality 8 is provided in the measuring arrangement 1.

The modulator 8 has electrically operated actuators (not shown), to which the detector 3 can be mechanically displaced in at least two spatial directions in a manner known per se. These actuators are electrically controlled by electronic components 9. By this shift, the position of the imaged scene 10, 11 in the recording field 4 of the detector 3 is changed.

As a result, the imaged scene 10 appears in the image 6 at a different image position than the imaged scene 11 in the image 7.

Since the imaged scene 10, 11 remains essentially unchanged between the shots, this apparent image shift between the images 6 and 7, which results from the relative movement of the recording field 4 relative to the scene 10, 11 projected or imaged on the capturing field 4, is easily recognizable ,

The actuators of the modality 8 thus cause a mechanical movement of the detector 3 supporting part 12 of the camera 2 with respect to a further part 13 of the camera 2, which may for example be firmly connected to the housing 14 of the camera 2. In the exemplary embodiment, the part 13 is a circuit board, which carries the components 9 or other electronic components. Furthermore, the board carries the detector 13, which is arranged movably on the board.

To clarify the principle of the invention, the parts 12, 13 are shown only schematically and greatly simplified. In the embodiments of the invention, the parts 12, 13 have more complex shapes.

The actuators of the modality 8 are electrically controlled so that the part 12 with the detector 3 performs a mechanical vibration, which has an irregular sequence of movements. The detector 3 thus leads to an uncontrolled and random movement relative to the part 13, the board.

FIGS. 5 to 7 show another possibility according to the invention for changing the position of an imaged scene in the recording field 4 of a detector 3 between the individual recordings or during the recording of an image sequence.

In the highly simplified representations according to FIGS. 5 to 7, a scene 15 containing an object A is shown. The object A radiates in a non-visible spectral range electromagnetic radiation, which is detected with an optically active in this non-visible spectral range optics assembly 17 and on a sensitive in this non-visible spectral range detector 3 of the measuring device 1 is led.

For this purpose, the optical arrangement 17 has optically acting elements 18, 19, 20, which guide the electromagnetic radiation 16 in the non-visible spectral range in a beam path onto the recording field 4 of the detector 3 of the measuring arrangement 1.

In the exemplary embodiments according to FIGS. 5 to 7, an infrared radiation is detected as electromagnetic radiation 16, and the camera 2 of the measuring arrangement 1 is designed as a thermal imaging camera. In other embodiments, however, other spectral ranges can be measured.

The optical assembly 17 is arranged so that the imaged scene 10 is projected onto the recording field 4 at a desired position.

In order to capture further information for the above-mentioned generation of an SR image, the location of the imaged scene should be determined 10 in the recording field 4 with respect to the position shown in Fig. 5 are changed.

This is achieved in the variant shown in FIG. 6 in that at least one optical property, for example a refractive index or an optical activity, of at least one optically active element 19 of the optical assembly 17 is varied. For this purpose, the element 19 can also be constructed from a plurality of individual elements which can be adjusted relative to one another.

As a result, the beam path of the optical assembly 17 changes, and the electromagnetic radiation 16 is directed in an altered manner to the detector 3, so that the imaged scene 11 is projected onto the recording field 4 at a changed image position.

This results in a change in the position of the imaged scene 11 in the image recorded with the recording field 4. In a further variant, the position of the optically active element 19 in the beam path of the radiation 16 is changed to change the position, so that the position of the imaged scene

11 in the recording field 4 is also changed. The change in the optical properties or the position of the element 19 takes place here uncontrolled and random, so that there is an irregular sequence of movements that is not monitored or regulated, but only excited or triggered.

The variants according to FIG. 6 and FIG. 7 can also be combined with one another, and it is provided in further exemplary embodiments that a part of the optically acting elements 18, 19, 20 of the optical assembly 17 are equipped with a modality 8 according to the principle of Fig. 6, while another part of the optically active elements 18, 19, 20 are equipped with a modality 8, which operates on the principle of Fig. 7.

In the illustrated exemplary embodiments according to FIGS. 3 to 7, the respective modality 8 between the individual recorded images 6, 7 is thus actuated in order to achieve a positional shift of the respectively recorded scenes 10, 11 in the recording field 4.

The recorded images 6, 7 are subsequently fed to a coarse registration unit 21, which is integrated in the image processing unit 5.

In the rough registration unit 21, the images 6, 7 are rough-registered, so that image pixels of the images 6, 7 are mutually assigned to image areas corresponding to each other.

For this purpose, the coarse registration unit can evaluate a control signal or a measurement signal of the modality 8 in order to reduce the computation time during coarse registration. The coarse-registered images 6, 7 then carry out an optimization process in which an energy function is minimized. An energy function according to J. Flusser et al. : A unified approach to super resolution and multi-channel blind deconvolution, IEEE transactions on image processing, vol. 16, no. 9, September 2007, pp. 2322-2332.

In this case, the recorded images 6, 7 are recorded with a dot spread function ion, wherein the point spread function contains a variation parameter which describes a subpixel-accurate displacement. In the optimization method, the variation parameters are varied accordingly until the energy function takes an optimal value.

The SR image thus calculated can then be displayed on a display unit 21 or output via a data interface to an output unit 23.

In the exemplary embodiments according to FIGS. 1 to 7, the components of the measuring arrangement 1 are integrated into the housing 14 of the camera 2.

In other embodiments, individual components of the measuring arrangement 1 may also be formed separately. For example, the image processing unit 5 can be embodied in a PC which is connected to the camera 2 comprising the detector 3 for image processing.

In the measuring arrangement 1 with a detector 3 of a camera 2 receiving in the non-visible spectral range, it is proposed, with an electrically and / or mechanically operable modality 8 between the recording of images 6, 7, to shift a scene 10, 11 shown in FIG Referring to a two-dimensional image field 4 of the detector 3 to obtain images 6, 7 of a substantially unchanged scene 15, which can be combined into an SR image with increased output image resolution.

Claims

Expectations

Measuring arrangement (1), comprising a camera recording in a non-visible spectral range

(2), with a detector (3) set up for recording in the non-visible spectral range and providing a recording field (4) with a recording image resolution, and further comprising an image processing unit (5) which is used to generate an SR image with an output image resolution by combination of at least two with the detector

(3) images (6, 7) of a scene (15) recorded with the recording image resolution are set up, the output image resolution being greater than the recording image resolution, characterized in that a mechanically and/or electrically operable modality (8) is provided, with which a position of the depicted scene (10, 11) in the recording field

(4) of the detector (3) can be changed.

Measuring arrangement (1) according to claim 1, characterized in that the modality (8) is set up to generate or excite mechanical vibrations.

Measuring arrangement (1) according to claim 1 or 2, characterized in that the modality (8) has an element (19) which acts optically in the non-visible spectral range, whose optical properties are in the non-visible spectral range and/or its position in one Beam path of the camera (2) can be changed mechanically and/or electrically.

Measuring arrangement (1) according to one of claims 1 to 3, characterized in that the modality (8) is for movement at least one part (12) of the camera (2), in particular the detector (3), is set up and/or that the modality (8) is set up to carry out an irregular, in particular random and/or uncontrolled, sequence of movements.

5. Measuring arrangement (1) according to one of claims 1 to 4, characterized in that the image processing unit (5) has a coarse registration unit (21) which is used to carry out a coarse registration of the images (6, 7) from the set of recorded images (6 , 7) is set up.

6. Measuring arrangement (1) according to one of claims 1 to 5, characterized in that the image processing unit (5) is set up to carry out an optimization method which is preferably downstream of a coarse registration, with a sub-pixel-precise shift of the preferably coarsely registered images (6, 7) can be used relative to each other.

7. Image processing method for improving the image resolution, which includes the steps of imaging a scene (15) onto a recording field (4) of a detector (3) of a camera (2) recording in a non-visible spectral range, recording a set of images ( 6, 7) with the recorded image resolution and generating a processed SR image by a combination of the recorded images (6, 7), the output image resolution being greater than the recorded image resolution, characterized in that during and/or between the individual recordings of the pictures (6, 7) with a A position of the imaged scene (10, 11) in the recording field (4) is changed using a mechanically and/or electrically operated modality (8).

8. Image processing method according to claim 7, characterized in that a mechanical vibration is generated or excited with the modality (8) and / or that an irregular, in particular a random and / or uncontrolled, movement is carried out with the modality (8).

9. Image processing method according to claim 7 or 8, characterized in that with the modality (8) at least one optical property in the non-visible spectral range and / or a position in a beam path of the camera (2) in the non-visible spectral range optically acting element (19) is/are changed mechanically and/or electrically operated.

10. Image processing method according to one of claims 7 to

9, characterized in that with the modality (8) at least a part (12) of the camera (2), in particular the detector (3), is moved, in particular relative to a beam path.

11. Image processing method according to one of claims 7 to

10, characterized in that the images (6, 7) of the set of images are automatically brought into a coarse registration in a coarse registration unit (21) and/or that the recorded images (6, 7) are combined in an optimization process, preferably downstream of a coarse registration are processed, being used as a variation parameter of the optimization process a sub-pixel-precise displacement of the preferably coarsely registered images (6, 7) relative to one another is used.

Image processing method according to one of claims 7 to 11, characterized in that in the optimization method for each of the images (6, 7) from the set of images (6, 7) a convolution of an SR generated on a trial basis from the images (6, 7) Image is calculated with a point spread function characterizing the variation parameter.