WO2013143820A1

WO2013143820A1 - Method and device for tracking the position of an object

Info

Publication number: WO2013143820A1
Application number: PCT/EP2013/054576
Authority: WO
Inventors: Hubert Burggraf; Axel REHDER; Bernhard Hampel-Vogedes
Original assignee: Rheinmetall Defence Electronics Gmbh
Priority date: 2012-03-27
Filing date: 2013-03-07
Publication date: 2013-10-03
Also published as: EP2831618A1; DE102012204855A1

Abstract

The invention relates to a method for tracking the position of an object (1) comprising the steps of projecting a two-dimensional image sequence by means of a projector (2), detecting in each case a light signal sequence from the image sequence by means of at least four sensors (S1, S2, S3, S4) arranged in a known configuration on the object (1), and determining the position of the object (1) relative to the projector (2) on the basis of the output signals of the sensors (S1, S2, S3, S4) and the two-dimensional image sequence, wherein the position of the object (1) relative to the projector (2) is determined at least every second.

Description

DESCRIPTION

Method and device for tracking the position of an object The present invention relates to a method and a device for tracking the position of an object. This is also called tracking. The term position encompasses the location of the object in up to three translational dimensions and / or the orientation of the object in up to three rotational dimensions. Possible applications of the tracking are, in particular, the head tracking, in which the position of a head is tracked, for example to produce an image corresponding to the head position, or the tracking of objects such as participants of an exercise and / or their weapons in a battlefield simulation, in particular also in closed rooms.

Various approaches for tracking the position of an object are known. One variant is to attach acceleration sensors to the object and to conclude from their output signals to a change in position. This is quite inaccurate. Another variant uses a GPS signal. However, this can not determine the orientation of the object and the tracking is not possible within buildings. Document WO 2005/015904 A1 discloses a method and apparatus for determining the correspondence between points on a projection surface of arbitrary shape and pixels in the image of a projector. For this purpose, a two-dimensional image sequence from Gray Codes projected by the projector and detected by individual light sensors in the projection surface. The output signals of the sensors are used to determine the correspondence between the sensor positions and pixels of the projected image.

It is an achievement of the present invention to transfer the principle of document WO 2005/015904 A1, which relates to the adaptation of a projected image to the uniquely determined relationship between the projector and the projection surface, to the tracking of the position of an object.

The present invention relates to a method for tracking the position of an object with the steps of projecting a two-dimensional image sequence by means of a projector, respectively detecting a light signal sequence from the image sequence with at least four sensors arranged in a known configuration on the object and determining the position of the object relative to the projector based on the output signals of the sensors and the two-dimensional image sequence, wherein the position of the object relative to the projector is determined at least every second.

The two-dimensional image sequence emitted by the projector generates corresponding light signal sequences in different partial spatial regions of the illuminated spatial region. When a light signal sequence is detected by a sensor, it is thus possible to determine in which partial space region the sensor is located. A partial space area is usually conical or in particular pyramidal with the apex at the location of the projector. The longer the image sequence is, the more images the image sequence contains, the more light signal sequences and correspondingly more partial spatial regions can be distinguished. The sensors, which are light sensors and in particular photodiodes, are arranged in a known configuration on the object. This means that the relative positions between the sensors and the relative positions of the sensors to the object are known. Thus, from the positions of the sensors relative to the projector, the position of the object relative to the projector can be calculated. In particular, the orientation of the sensors is disregarded and only their location relative to the projector is considered. In particular, sensors may be used which are already arranged on the object for other purposes, for example sensors for detecting hits in a battlefield simulation.

The position of the object is determined according to the invention at least once a second. Preferably, the frequency is even higher, for example exactly or at least 2, 5, 10, 20, 25, 50, 100, 200 or 250 times per second. Due to the repeated, preferably regular, determination of the position tracking of the object is possible.

In one embodiment of the invention, the image sequence is a binary image sequence. This means that only one binary information, that is a bit sequence, is contained in a light signal sequence. Thus only the states light on and light off are used. Thus, a simple and robust determination of the partial space area becomes possible. In another embodiment, the image sequence is monochrome. This means that the image sequence contains only light of a single wavelength or a narrow wavelength range. Thus, narrow-band filters can be arranged in front of the sensors which filter out light of a different wavelength and thus suppress the interfering influence of extraneous light. However, it is also within the scope of this invention to use a polychromatic image sequence and / or an image sequence with more than two brightness levels. This can be distinguished with the same length of the image sequence more partial space areas. However, this comes at the expense of more complex light sensors and evaluation as well as less robust detection.

Preferably, the image sequence is a bar code such as a two-dimensional Gray code. This consists of two subsequences. In the first picture of each sequence, one half of the picture is dark, the other bright. In the next image, each half of the image is divided into two halves and one of the halves inverted. Thus, for example, there is an alternating stripe pattern of alternating light and dark stripes, whereby the stripe width is halved from image to image. In one of the subsequences the stripes run horizontally, in the other vertically. Each of the sequences is therefore a one-dimensional Gray code. The strips of the two Gray codes are preferably perpendicular to each other. The image sequence is preferably in the infrared or ultraviolet frequency spectrum. This means that the light emitted by the projector is in the infrared or ultraviolet spectrum and preferably contains no light in the visible spectrum. Thus, for example, people who are in the area illuminated by the projector, not disturbed, such as dazzled. Using an ultraviolet spectrum alone will not hinder the use of night vision equipment.

In one embodiment of the invention, a digital projector is used to project the image sequence. One pixel or several adjacent pixels then illuminate a partial spatial area into which the same light signal sequence is emitted. For each subspace, a middle straight is defined. In a partial space area in the form of a straight pyramid, in particular with a rectangular base, the middle straight is then the height of the pyramid. From the light signal sequences of the sensors, the middle straight lines are identified on which the sensors are located. In other words, it is assumed that a sensor located in a partial space area lies on the middle straight of this partial space area in order to simplify the subsequent calculations. Then, the locations of the sensors are virtually shifted along the respective middle line until the virtual relative locations of the sensors coincide with the actual configuration of the sensors. In an alternative, the position of the entirety of the sensors in their known arrangement is virtually changed until virtually every sensor is located on the middle straight assigned to it.

Taking into account only one sensor and the associated middle echelon, the object can only lie on a known straight line. About the third translational component (the distance from the projector) and the three rotational components (orientation) of the position of the object, no statement can be made yet. Four degrees of freedom of the object are thus still undetermined. If three additional sensors are considered, the remaining degrees of freedom can also be calculated. Thus, the position and orientation of the object are uniquely determined.

Optionally, additional information is taken into account when determining the position of the object relative to the projector from the output signals of the sensors and the image sequence. An example of such additional information is the distance of a sensor from the projector. The present invention further relates to a device for tracking the position of an object with a projector, which is adapted to project a two-dimensional image sequence, with at least four sensors, which are arranged in a known configuration on the object and are adapted to each one Capture real-time signal sequence from the image sequence, and with a computing unit which is adapted to determine the position of the object relative to the projector based on the output signals of the sensors and the two-dimensional image sequence at least every second.

The advantages are analogous to those described in the context of the method. In particular, the arithmetic unit is adapted to carry out the method described above in one or more embodiments. This means that the arithmetic unit either itself carries out the method steps or instructs another component of the device to execute a method step.

In one embodiment of the invention, four sensors are arranged at the tips of an imaginary square. This means that the distances between each two sensors are the same, so the sensors are arranged symmetrically. This simplifies in particular the calculation of the position of the object from the light signal sequences of the sensors.

In one embodiment, the device has two projectors that are each configured to project a two-dimensional image sequence. The projectors are preferably synchronized, so that the transmitted image sequences are synchronized. By using multiple projectors, the availability of position tracking of the object can be improved and / or the space area for tracking can be increased. If only one projector is used, it may be that one of the sensors is shaded, Thus, no light signal sequence detected by the projector By using two or more projectors, the probability of the occurrence of this case is significantly reduced, especially if the projectors are located in different locations.

In one embodiment, the projectors project image sequences in different wavelength ranges. In this case, the projectors can project simultaneously without interfering with each other at the sensors. However, this assumes that the sensors can differentiate between the wavelength ranges. Alternatively, the projectors project image sequences in the same wavelength range, and in particular monochromatically at the same wavelength. In this case, a collision of the projections on the sensors must be prevented, for example by the fact that the projectors project in time division multiplex.

The present invention further relates to the use of a device having a projector that is configured to project a two-dimensional image sequence, having at least four sensors, which are arranged in a known configuration on an object and are adapted to each a light signal sequence from the image sequence and a computing unit configured to determine the position of the object relative to the projector from the output signals of the sensors and the two-dimensional image sequence for tracking the position of the object. The use is in particular in the context of a battlefield simulation or to determine the position of a head (head tracking). In a battlefield simulation, the object to be tracked is, for example, a vehicle, a participant or a weapon. A battlefield simulation serves to train local and house battles in combat training centers. When used in a head tracker, the viewing direction of the observed person can be determined. It can then be used to control the appearance of a head-mounted display (HMD), to target weapon systems, to determine the line of sight of individuals, or to determine the firing direction of simulated weapons. It is also possible to track and control sensor platforms or autonomous vehicles, for example in factories or warehouses. The present invention makes it possible to locate and track objects in confined spaces and, with proper choice of the light spectrum of the projector, it prevents the interference of persons and devices located in the area illuminated by the projector. The device and method are not affected by magnetic, electrical or acoustic sources of interference. An application is possible in both stationary and moving environments. A location-dependent calibration of the device, for example in a cockpit, is not required. It is within the scope of the present invention to combine individual features of embodiments or Ausgestaitungsformen or optional features together or omit features not essential to the invention of an embodiment or embodiment. The present invention will be explained in more detail with reference to an embodiment. Showing:

Figure 1 is a schematic representation of a

device according to the invention,

FIG. 2 is a flowchart of the invention

Procedure and

FIG. 3 shows an exemplary image sequence. 1 shows schematically a device according to the invention for determining the position of an object 1. The device comprises a projector 2, a control unit 3 and four sensors S1, S2, S3 and S4. The projector is, for example, a Light Commander from Texas Instruments. It contains a DLP 0.55 XGA chipset with a 0.55 inch Digital Micromirror Device (DMD), a DLPA200 (DMD Micromirror Driver) and a DLPC200 (Controller) to display 1024x768 pixels. The control unit 3 controls the projector 2 in such a way that it emits an image sequence with binary two-dimensional Gray codes. The transmission of a single image is designated in FIG. 2 by S01.

The sensors S1 to S4 are mounted in a known arrangement, in particular in pairs with the same distance, on the object 1 and connected to the control unit 3 so that they can receive and evaluate the output signals of the sensors. The sensors are adapted to receive the light emitted by the projector 2 and in particular to convert it into an electrical signal. This is designated in FIG. 2 as method step S02.

An exemplary image sequence is shown in FIG. It consists of two partial image sequences. The first partial image sequence, which is indicated in the upper half of FIG. 3, has 10 individual images with narrowing vertical stripes. In the first picture, the left half of the picture is dark and the right half is bright. In the second picture, the right half of each half of the picture is inverted, so the first and fourth picture quarters are dark and the second and third picture areas are bright. Generally speaking, a homogeneous (same color and / or equal to brighter divided) image area at the center and inverted for the next picture one half of the image area. ^"includes Analogously, in the lower half of Figure 3 indicated second part-image sequence from one image to narrowing horizontal stripes. The projector 2 illuminates a region of space, which is indicated by dashed lines in FIG. 1, with the image sequence. Due to the image sequence, the illuminated room area is subdivided into partial room areas, wherein a characteristic light signal sequence is emitted in each partial room area. The size of a partial room area depends on the number of pictures in the picture signal sequence and / or the resolution of the projector. The projection geometry of the projector 2 is known. This means that the relationship between an area in the image and the partial space area is known. This depends inter alia on the optical system of the projector 2. Optionally, a 2D mapping to compensate for the distortions by the optics of the projector. 2

The transmission of the image sequence is shown in FIG. 2 as a loop of steps S01 and S02. Each time the loop is looped, another image is projected from the sequence (another part of the code). On the basis of the output signal of a sensor and the known image sequence, the control unit 3 can determine in which subspace region the sensor is located. To simplify the position determination, it is assumed that the sensor lies on the middle straight of the partial spatial area. The middle straight is the geometric center axis of the partial space area. The origin of each middle straight is the position PO of the projector 2. In the present example, it is determined that the sensor S1 on the straight line L1, the sensor S2 on the straight line L2, the sensor S3 on the straight line L3 and the sensor S4 on the straight line L4 lies.

In method step S03, the sensor positions are calculated by virtually displacing the four sensors S1 to S4 along the corresponding middle straight lines L1 to L4 until the virtual locations of the sensors result in an arrangement of the sensors corresponding to the actual arrangement of the sensors. In the present case with pairwise equidistant arrangement of the sensors, the virtual pairwise distances of the Be identical sensors. Thus, the three location coordinates X, Y and Z of the sensors are known. The index i in FIG. 2 identifies the respective sensor. The location of the object 1 is, for example, the center of gravity of the square formed by the four sensors, but may also be generally related to the arrangement of the sensors.

In method step S04, the orientation of the object 1 is calculated from the location coordinates of the sensors. The spatial relationship between the object 1 and the sensors S1 to S4 is known. Thus, all six degrees of freedom (DOF) of the position of the object 1 are known. The three rotational degrees of freedom relate in particular to heading, pitch and roll of the object 1 and are given as angles Θ, Φ and Ψ, in particular with reference to an earth-fixed reference system.

In method step S05, the position of the object 1 is output, for example to a control computer of a simulator such as a flight simulator or a battlefield simulator. Optionally, the position of the object 1, which was determined relative to the projector 2, is converted by means of a coordinate transformation on the basis of the known position of the projector 2 into another reference system.

In this example, each sub-image sequence consists of 10 images, so the image sequence consists of 20 images. The position determination takes place 250 times per second. For this purpose, the projector 2 sends out the image sequence 250 times per second. This means that the projector 2 transmits a total of 5000 images per second. However, a larger or smaller frequency is possible as long as the continuous or quasi-continuous tracking of the object 1 is possible. The image sequence is either stored in the control unit 3 (or generated there) and is transmitted to the projector 2 for display or the image sequence is stored in the projector 2 and is clocked by the control unit 3. The control unit 3 can also be integrated in the projector 2.

The calculation of the sensor positions in method step S03 does not have to be done by moving the positions along the middle straight line. Rather, any calculation method that determines the locations of the sensors and thus the position of the object 1 relative to the projector 2 from the middle straight line and the known arrangement of the sensors can be used.

Claims

Rheinmetall Defense Electronics GmbH, Bremen PATENT APPLICATIONS

1.

Method for tracking the position of an object (1) with the steps of projecting a two-dimensional image sequence by means of a projector (2), respectively detecting a light-signal sequence from the image sequence with at least four sensors (S1, S2) arranged on the object (1) in a known configuration , S3, S4) and in the position of the object (1) relative to the projector (2) from the output signals of the sensors (S1, S2, S3, S4) and the two-dimensional image sequence, the position of the object (1) relative to Projector (2) is determined at least every second.

Second

The method of claim 1, wherein the image sequence is a binary image sequence.

Third

The method of claim 2, wherein the image sequence is a two-dimensional Gray code.

4th

Method according to one of claims 1 to 3, wherein the image sequence is in the infrared or ultraviolet frequency spectrum.

5th

Method according to one of claims 1 to 4, wherein a digital projector (2) is used for projection of the image sequence, one pixel or several adjacent pixels each illuminate a region in which the same light signal sequence is emitted, a middle straight (L1, L2 , L3, L4), from the light signal sequences of the sensors (S1, S2, S3, S4) the middle straight lines (L1, L2, L3, L4) are identified, on which the sensors (S1, S2, S3, S4) are located , and the positions of the sensors (S1, S2, S3, S4) are virtually displaced along the respective center lines (L1, L2, L3, L4) until the virtual relative position of the sensors (S1, S2, S3, S4) with the actual configuration of the sensors (S1, S2, S3, S4) matches.

6th

Device for tracking the position of an object (1) with a projector (2), which is set up to project a two-dimensional image sequence, at least four sensors (S1, S2, S3, S4), which in a known configuration on the object ( 1) and are arranged to each detect a light signal sequence from the image sequence, and a computing unit (3) which is adapted to the position of the object (1) relative to the projector (2) based on the output signals of the sensors ( S1, S2, S3, S4) and the two-dimensional image sequence at least every second.

7th

Apparatus according to claim 6, characterized in that the arithmetic unit (3) is adapted to carry out the method according to one of claims 2 to 5.

8th.

Device according to one of claims 6 or 7, characterized in that four sensors (S1, S2, S3, S4) are arranged at the tips of an imaginary square.

9th

Device according to one of claims 6 to 8, characterized by two projectors which are adapted to each project a two-dimensional Büdsequenz.

10th

Apparatus according to claim 9, characterized in that the two projectors project image sequences in different wavelength ranges.

1 1.

Use of a device with a projector (2), which is adapted to project a two-dimensional image sequence, with at least four sensors (S1, S2, S3, S4), which are arranged in a known configuration on an object (1) and to are each arranged to detect a light signal sequence from the image sequence, and with a computing unit (3) which is adapted to the position of the object (1) relative to the projector (2) based on the output signals of the sensors (S1, S2, S3 , S4) and the two-dimensional image sequence, for tracking the position of the object (1).

12th

Use according to claim 1 1 in the context of a battlefield simulation or for determining the position of a head.