WO2009033708A2

WO2009033708A2 - Method for aligning an object

Info

Publication number: WO2009033708A2
Application number: PCT/EP2008/007565
Authority: WO
Inventors: Ulrich Ehrenfried; Rüdiger Opper; Falk Thürmer
Original assignee: Pepperl + Fuchs Gmbh
Priority date: 2007-09-12
Filing date: 2008-09-12
Publication date: 2009-03-19
Also published as: WO2009033708A3; DE102007043497A1

Abstract

The invention relates a method for aligning an object by rotating the same about at least one rotation axis, wherein a marker is detected by way of a camera and is imaged as a marker image, and either the camera takes part in the rotation and the marker is fixed in a direction, or vice versa, such that a relative rotation occurs between the marker and the camera upon the rotation of the object. The actual orientation, which is present between the camera and the marker before the rotation, is derived by means of image processing either from the orientation having the marker image in the field of view of the camera prior to the rotation, or from the linear extension the marker image has in the field of view of the camera prior to the rotation, and the result is used in order to determine a target rotation angle, about which the object is to be rotated about the rotation axis for the alignment thereof in a certain target direction. Subsequently, the object is rotated by the target rotation angle about the rotation axis, and thus aligned in the target direction.

Description

Method for aligning an object

Technical area:

The invention relates to a method for aligning an object, such as robot, arm of a robot or laser, by rotating the object about at least one axis of rotation.

State of the art:

In the prior art it is known to rotate objects which are rotatable about a rotation axis, e.g. Robot to align with the help of mechanical angle encoder in a desired direction. Such an angle encoder usually consists of an immovable part, which e.g. is connected to a wall or the floor, and a movable part, which is connected to the robot. The axis of rotation passes through the angle encoder. The movable and the immovable part are arranged on a pivot bearing to each other. If the robot is rotated relative to its surroundings about the axis of rotation, the movable relative to the immovable part of the angle encoder is also rotated about the axis of rotation. This relative rotation is detected by a built-in encoder in the encoder and converted into a signal containing information about the angular, related to the rotation axis orientation of the object relative to its environment or about the rotation angle of rotation.

The sensor may e.g. operate in the manner of a potentiometer whose tapping point changes during rotation. Another type of sensor detects the rotation angle optically using a coded angle plate. Yet another type of sensor detects the angle magnetically.

A disadvantage of the alignment of objects with such encoders is that a high precision of the alignment can be achieved only with great effort; High-resolution angle encoders are therefore expensive. Another disadvantage of the alignment of objects using angle encoders of the types mentioned is the sensitivity to contamination or Aging. In addition, such angle encoders must be arranged on the axis of rotation, which can be problematic, for example, in confined spaces.

If an object is to be oriented in one direction without a rotation axis mechanically existing (eg aligning a robot freely wheeled on wheels by rotating it in place or aligning an object floating on a magnetic pad), the use of such angle encoders is not, or at least not easy, possible. For the same reason, with such angle encoders also it is not readily possible to use the robot e.g. To move transversely to the axis of rotation to another location and align it there by turning on the spot, for this purpose would first have mechanically decoupled the axis of rotation of the environment and mechanically coupled at the new location to the environment, i. The axis of rotation would have to be relocated together with the robot.

A further disadvantage of using angular encoders for aligning objects is that three angular encoders are needed to align an object in any spatial direction by rotating the object about three mutually perpendicular axes of rotation, one per axis of rotation.

Technical task:

The invention is therefore based on the object to provide a method for aligning an object, which does not have these disadvantages and manages without the use of a Winkelcodierers.

[A1] This object is achieved by a method for aligning an object, such as robot, arm of a robot or laser, by rotating the object about at least one axis of rotation, characterized in that a marker is detected with a camera and imaged as a marker image, wherein either the camera participates in the rotation and the marker is directionally fixed or vice versa, so when turning the object relative rotation between the marker and the camera takes place about the axis of rotation, the actual orientation which exists between the camera and the marker before the rotation, either from the orientation which the marker image has before the rotation in the field of view of the camera, or from the linear

Extension, which has the marker image before rotation in the field of view of the camera, derived by electronic image processing and the result is used to determine a target rotation angle by which the object to achieve a predetermined target orientation between the camera and marker and thus to Aligning the object is to be rotated in a certain direction about the rotation axis, and then the object is rotated about the target rotation angle about the rotation axis and thus aligned in the target direction.

[A2] According to a variant of the method, the following steps are carried out: a) the marker is fixed in the surroundings of the object, b) the camera is arranged on the object so that it participates in the rotation of the object, its optical axis parallel or runs obliquely to the axis of rotation and the marker is in the detection range of the camera and is thus imaged by the same as lying within their image field marker image, c) by means of electronic image processing of the orientation of the marker image in the image field of the camera, the actual orientation of the camera relative to the Derived from the marker and compared with the target orientation of the camera relative to the marker, d) the result of step c) is used to determine the SoII rotation angle, e) the object is rotated about the desired rotation angle about the axis of rotation, so that after the step e) the predetermined target orientation between camera and marker is present and thus de The object is oriented in the target direction. The camera is preferably arranged rigidly on the object in step b).

The inventive method is characterized in particular by the fact that it can be dispensed with the use of an angle encoder with all its disadvantages.

[A3] If the orientation between the object and the camera is known, the orientation between the object and the marker can be determined immediately from the orientation between the camera and the marker. The orientation between the object and the camera can be known in advance. According to a variant, the orientation between the object and the camera is determined, e.g. by measurement.

If the orientation of the marker relative to the outside world or environment of the object is known or measured, the orientation of the object relative to the marker immediately follows the absolute orientation of the object with respect to the outside world or surroundings. For example, the orientation of the marker relative to the north-south direction can be measured. A12 According to a variant of the method, therefore, the orientation of the marker relative to the outside world is determined and the absolute orientation of the object with respect to the outside world determined from the orientation of the object relative to the marker and the orientation of the marker relative to the outside world or to the environment of the object.

According to another variant of the method, the orientation of the marker relative to the surroundings of the object is determined and the absolute orientation of the object with respect to the environment of the object is determined from the orientation of the object relative to the marker and the orientation of the marker to the outside world or to the environment of the object. [A4] According to a variant, the following steps are carried out: a ') the marker is placed on the object so that the marker participates in the rotation of the object, b ¹ ) the camera is placed in the environment of the object directionally so that its optical axis is parallel or oblique to the axis of rotation and the marker is within the detection range of the camera and thus is imaged by the same as lying within their image field marker image, c ') by means of electronic image processing is the orientation of the marker image in the image field of the camera, the actual orientation of Markers relative to the camera derived and compared with the target orientation of the marker relative to the camera, d ¹ ) the result of step c ¹ ) is used to determine the desired rotation angle, e ¹ ) the object is the target Rotation angle rotated about the axis of rotation, so that after step e ¹ ) the predetermined target orientation between marker and camera before and thus the object is aligned in the target direction.

With the orientation between camera and marker, i. with the actual orientation of the camera relative to the marker or vice versa, that orientation is meant, which has the projection of the camera on a plane perpendicular to the axis of rotation relative to the projection of the marker on this plane.

[A5] Once the orientation between the object and the marker is known, the orientation between the marker and the camera can be used to instantly determine the orientation between the object and the camera. The orientation between the object and the marker can be known in advance. According to a variant, the orientation between the object and the marker is determined, e.g. by measurement.

The orientation between the object and the marker means that orientation which the projection of the object on to the axis of rotation vertical plane relative to the projection of the marker on this plane. Preferably, the camera in step b) or in step b ¹ ) is arranged so that its optical axis of the camera is parallel to the axis of rotation or against these by an angle of at most 85 °, preferably at an angle of at most 60 °, particularly preferred is inclined at an angle of at most 30 ° or at an angle of at most 15 °.

[A6] According to a further variant of the method according to the invention, the following steps are carried out: A) the marker is placed on the article such that the marker participates in the rotation of the article,

B) the camera is directionally fixed in the vicinity of the object arranged so that its optical axis is perpendicular or oblique to the axis of rotation and the marker is in the detection range of the camera and thus from the same as lying within their image field

Marker image is displayed,

C) by means of electronic image processing, the actual orientation of the marker relative to the camera is derived from the linear extent, which has in the image field of the camera, the marker image perpendicular to the direction of the image of the axis of rotation, and with the desired orientation of the marker relative to the camera compared,

D) the result of step C) is used to determine the desired angle of rotation,

E) the object is rotated about the desired rotation angle about the axis of rotation, so that after step E) the predetermined desired orientation between marker and camera is present and thus the object is aligned in the target direction.

Since the axis of rotation is not a visible, physical body, but an imaginary line, it goes without saying that the image of the axis of rotation is not a real, visible image in the image field of the camera, but also an imaginary line. The direction of the image of the axis of rotation means the direction of this imaginary line in the field of view of the camera. Preferably, the camera is arranged in step B) so that its optical axis of the camera is perpendicular to the axis of rotation or against this by an angle of at least 5 °, preferably by an angle of at least 30 °, more preferably by an angle of at least 60th ° or inclined at an angle of at least 85 °.

[A7] According to a variant of the method, the object is movable along a path or on or parallel to a tread, wherein in step a) a plurality or multiplicity of markers are arranged along the path or on the tread, the object is successively moved to different locations which are selected such that at each of the locations at least one of the markers is detected and imaged by the camera, and steps b) to e) are performed separately for each of the locations.

That is, instead of only one marker several markers are used in succession according to this variant, wherein the object is aligned first with the aid of the first marker, then at another location by means of the second marker, etc .. [A8] Here, each marker of All other markers different and individually identifiable by means of the camera and image processing.

[A9] As a marker, e.g. a code carrier or character or string or pattern or dot pattern is used, the marker carrying information read by the camera and image processing. This information may e.g. serve for the individual identifiability of the marker and / or specify the location of the marker in question.

[A10] As the code carrier, e.g. a one-dimensional barcode can be used. According to another variant, a two-dimensional bar code or a data matrix code is used as the code carrier.

[A11] Each marker preferably carries information about its own absolute position along the route or about the absolute position of it Projection on the tread each with respect to a predetermined reference point, this information is read out by means of the camera and image processing. In this way, the markers or the information carrying the markers can be used to locate the object along the path or tread by image processing.

[A12] According to another variant of the method, the orientation of the marker (M) with respect to the outside world or to the environment of the object (R) is determined, wherein the absolute orientation of the object (R) with respect to the outside world or with respect to the environment (R) the object (R) is determined from the orientation of the object (R) relative to the marker (M) and the orientation of the marker (M) relative to the outside world or to the environment of the object (R).

Brief description of the drawing, in which schematically show by means of preferred embodiments of the invention:

Figure 1 is a plan view of a robot shown in a transparent, which is on a tread and is rotatable about a vertical axis of rotation, wherein on the underside of the robot, a camera is also shown transparent and arranged under the

Robot is a marker, which is glued to the tread and is detected and imaged by the camera, Figure 2 shows the field of view of the camera in the situation of Figure 1,

FIG. 3 shows the robot of FIG. 1, once again shown in a transparent manner, in plan view after rotation about one according to a first embodiment

4 shows the image field of the camera in the situation of FIG. 3,

FIG. 5 shows a further top view of the robot of FIG. 1, wherein the camera not shown here is arranged above the robot and the marker is glued to the top side of the robot and is detected and imaged by the camera,

FIG. 6 shows the image field of the camera in the situation of FIG. 5, FIG. 7 shows the robot of FIG. 5 in plan view after the rotation about one determined according to a second variant of the invention variant rotational angle,

FIG. 8 shows the image field of the camera in the situation of FIG. 7,

FIG. 9 shows a further plan view of the robot of FIG. 1, wherein the camera likewise not shown here is arranged above the robot and the marker is glued to the upper side of the arm of the robot and is detected and imaged by the camera, FIG 11 shows the image field of the camera after pivoting of the robot arm about a pivot axis upward about a desired rotation angle determined according to a third variant of the invention, FIG.

FIG. 12 shows the robot of FIG. 9 in a plan view after turning the robot arm about its longitudinal axis to the left about a target rotational angle likewise determined according to the third variant of the invention FIG. 13 shows the image field of the camera in the situation of FIG. 12, and FIG a situation in which the arm of the robot is both upwardly pivoted and counterclockwise with respect to the situation of FIG.

Figures 1, 3, 5, 7, 9 and 12 each show a robot R, which is rotatable about a rotational axis V, namely about the vertical axis V of the robot R by means of a drive, not shown. The robot R can on a tread F The axis of rotation V runs in the present example perpendicular to the tread F.

The robot R has an arm A, which is arranged on the robot via an angle W. The angle W is pivotable together with the arm A about a pivot axis S, whereby the robot can raise and lower the arm A. The arm A is further rotatable about a rotation axis RA, which coincides with the longitudinal axis RA of the arm A, whereby the robot R can rotate the arm A about its longitudinal axis RA.

On the arm, a laser L is arranged in its end portion facing away from the angle W, by means of which the robot R can make spot welds on workpieces, for example. The longitudinal axis of the laser L is perpendicular to the axis of rotation RA. Thus, when the arm A is rotated about its longitudinal axis RA, the orientation of the laser L changes. By rotating the robot R about the vertical axis V, by pivoting the arm about the pivot axis S and rotating the arm about its longitudinal axis RA The laser L can thus be aligned in any spatial direction.

Four variants of the method according to the invention will be explained below with reference to the figures. In the following explanations of the first and the second variant of the method according to the invention is generally - with the orientation of an object (eg marker M) relative to another object (eg camera K ₁ robot R), or, which is the same, with the orientation of the Object with respect to the other object, or, which is also the same, with the orientation between these two objects, in each case the orientation meant, which has the projection of the object on a plane perpendicular to the vertical axis V relative to the projection of the other object on this plane.

Figures 1 to 4 illustrate a first variant of a method according to the invention. This variant is used to align the robot R by rotating it around the vertical axis V and comprises the following steps:

Step a): The marker M is in the vicinity of the robot R ₁ namely in the present example on the tread F under the robot R, arranged stationary and directionally fixed, eg glued. The marker M in the present example is a one-dimensional barcode M, on which for better illustration of the method according to the invention a marker coordinate system Mx ₁ My with the parallel to the tread F aligned axes Mx ₁ My based, the Mx-axis parallel and the My -Axis perpendicular to the bars of the bar code M runs. The information content of the bar code M does not matter here. In the situation of Figure 1, the robot R is aligned on the tread F so that the pivot axis S is in an initial direction Z1 and at an angle φ1 to the Mx axis. The angle φ1 is therefore a measure of the actual orientation of the robot R with respect to the marker M, which is present in the situation of FIG.

Step b): On the robot R, a CCD camera K also shown transparent in Figure 1 is arranged so that the camera K takes part in each rotational movement of the robot R, looking from top to bottom on the tread F, their optical axis parallel to The axis of rotation V runs and the marker M lies in the detection range E of the camera K and is thus imaged by the same as within its image field BF lying marker image M ¹ (Figure 2).

The camera K is referenced to a camera coordinate system Kx ₁ Ky whose mutually perpendicular axes Kx ₁ Ky each extend parallel to the running surface F and in each case parallel to a boundary line of the image field BF of the camera K. The axes Kx ₁ Ky span the plane of the image field BF.

The Kx axis intersects the pivot axis S at an angle β, or the projection of the Kx axis onto a plane perpendicular to the vertical axis V intersects the projection of the pivot axis S onto this plane at the angle β (FIG. 1). The angle β is thus a measure of the orientation between camera K and robot R. Since the camera K takes part in each rotation of the robot, the angle β does not change during rotation of the robot R, but remains constant.

In the situation of Figure 1, the robot R is aligned on the tread F so that the Kx axis is at an angle α1 to the Mx axis. The angle α1 is thus a measure of the actual orientation of the camera K with respect to the marker M, which is present in the situation of FIG. We have φ1 = α1 + β.

FIG. 2 shows the image field BF which supplies the camera K in the situation of FIG. The marker M is used by the camera K in the image field BF as marker image M ¹ displayed. The lying in the detection range E of the camera K part of the tread F is shown as a tread image F ¹ . The axis Mx is represented in the image field BF as an axis image Mx '. Likewise, the axis My in the image field BF is depicted as an axis image My ¹ . The pivot axis S is imaged in the image field BF as an axis image S '. Of course, all the axis images Mx'.My'.S 'are not real, visible images in the image field BF, but just like axes Mx ₁ My ₁ S only imaginary lines and in Figure 2 (also in Figure 4, on soft below still referred to ) only for better illustration of the invention.

The camera K provides a right-angled and upright image, wherein the magnification in the Kx direction, in the Ky direction and in all intermediate directions is the same, ie the image of the camera K is undistorted and thus angle-faithful. Therefore, in FIG. 2, the bars of the marker image M ¹ in the image field BF of the camera K likewise run at an angle α1 to the Kx axis, ie the angle α1 is also a measure of how the marker image M ¹ in the image field BF is in the situation of FIG is oriented.

Step c): First of all, in this step, the orientation which the marker image M ^{1 has} in the image field BF of the camera K is determined by means of electronic image processing. This can be achieved, in particular, by determining the direction of the bars of the marker image M ¹ in the image field BF by an image processing program which is able to recognize these bars in the image field BF, and from this the angle α1 is determined.

Thereafter, the actual orientation of the camera K relative to the marker M is derived in step c). Since the camera K in the present example provides a true-to-the-page, upright and undistorted image, the angle at which the bars of the marker image M ¹ in the BF field are tilted against the Kx axis is the same as the angle about which the bars of the Markers M are inclined even against the Kx axis. This angle is given by α1, ie the actual orientation of the camera K relative to the marker M is determined by the angle α1. Now, the thus found actual orientation of the camera K relative to the marker M is compared with a predetermined desired orientation between camera K and marker M, ie the deviation between the target and actual orientation between camera K and marker M is detected. The desired orientation of the camera K relative to the marker M is defined in the present example by an angle α2 (FIG. 3). This completes step c).

Step d): Now, from the deviation found in step c), a target rotational angle φ (FIG. 3) is determined, by which the robot R is to bring about the predetermined target orientation between camera K and marker M and thus for aligning the robot R in a certain, specifiable target direction to rotate about the axis of rotation V is.

The inventive method is completed when the predetermined SoII orientation between camera K and marker M is reached, because then the robot R is aligned in the target direction. The target rotational angle φ is thus predetermined by the deviation of the actual orientation between camera K and marker M from the desired orientation between camera K and marker M and is determined according to the invention by image processing.

Step e): Now, the robot R is rotated about the rotation angle V by the target rotation angle φ determined in step d). FIG. 3 shows the robot R of FIG. 1 after the rotation about the desired rotation angle φ. FIG. 4 shows the image field BF which supplies the camera K in the situation of FIG. This rotation is indicated in Figure 1 by curved arrows. As a result of this rotation, the direction of the pivot axis S changes from the initial direction Z1 to the target direction Z2, as a result of which the robot R is aligned in a specific direction according to the invention.

The bars of the marker M extend after rotation at an angle α2 to the Kx axis; the pivot axis S now extends at an angle φ2 to the bars of the marker M, where φ = α1-α2 = φ1-φ2 and φ2 = α2 + β. Thus, after step e), the robot is oriented in a specific, predetermined direction relative to the marker M, namely in the present example such that the pivot axis S extends at an angle φ2 to the bars of the position and direction marker M.

The target direction Z2 does not necessarily have to be predetermined. If the orientation β between robot R and camera K is known, the orientation of the robot R follows directly from the orientation of the camera K. Then follows from the actual orientation α1 between camera K and marker M, the initial direction Z1 and from the target orientation α2 between camera K and marker M, the target direction Z2 as a predetermined target target direction.

Reference is now made to Figures 5 to 8, which illustrate a second variant of the method according to the invention. This second variant also serves to align the robot R by rotating it about the vertical axis V and comprises the following steps:

FIG. 5 again shows the robot R of FIG. 1, whereby it is aligned in FIG. 5 as well as in FIG. 1, namely in such a way that the pivot axis S runs in the initial direction Z1.

The variant of the method explained with reference to FIGS. 5 to 8 serves to align the robot R relative to the camera K and, instead of the above steps a) to e), comprises the following steps a ¹ ) to e ¹ ):

Step a ¹ ): Unlike in the first variant explained above, the marker M already shown in FIG. 1 is not glued to the running surface F but to the top side of the robot R, so that the marker M on each rotation of the robot R participates. On the marker M, the marker coordinate system Mx ₁ My is related to the parallel to the tread F aligned axes Mx ₁ My.

The Mx axis intersects the pivot axis S or its projection on the plane spanned by the Mx axis and the My axis at an angle ε. Of the Angle ε is thus a measure of the orientation between marker M and robot R. Since the marker participates in every rotation of the robot R, the angle ε does not change during rotation of the robot R, but remains constant.

Step b ¹ ): The camera K, not shown in FIGS. 5 and 7, unlike in the first variant explained above, is not arranged on the robot R, but arranged above it in such a way that the camera is directed from top to bottom onto the robot R and the marker M views, their optical axis is parallel to the axis of rotation V and the marker M is in the detection range E of the camera and thus of the same as within their image field BF lying marker image M ^{1 is} shown (Figure 6). The camera is arranged directionally fixed so that it does not participate in rotational movements of the robot R.

The camera is based on the camera coordinate system Kx ₁ Ky, whose mutually perpendicular axes Kx ₁ Ky extend parallel to the running surface F and parallel to a boundary line of the image field BF of the camera K, respectively. The axes Kx ₁ Ky span the plane of the image field BF.

In the situation of FIG. 5, the robot R is oriented on the running surface F such that the Kx-axis extends at an angle δ1 to the Mx-axis and at an angle φ1 to the pivoting axis S (or the projection of the Kx-axis onto a to the vertical axis V vertical plane at an angle δizur projection of the Mx axis to this plane and at an angle φ1 for the projection of the pivot axis S on this plane extends). The angle 61 is a measure of the actual orientation of the marker M in the situation of FIG. 5 with respect to the camera. Similarly, the angle φ1 is a measure of the present in the situation of Figure 1 actual orientation of the robot R with respect to the camera. Here, φ1 = δ1 + ε.

FIG. 6 shows the image field BF which supplies the camera K in the situation of FIG. The marker M is imaged by the camera in the image field BF as marker image M ¹ . The part of the upper side of the robot R lying in the detection area E of the camera K is imaged as a robot image R ¹ . Since the camera K delivers a correct, upright and undistorted image, in FIG. 6 the bars of the marker image M 'in the image field BF also run at an angle 61 to the Kx axis, ie the angle δ1 is also a measure of this, as in the situation of FIG 6, the marker image M ¹ is oriented in the image field BF.

Step c ¹ ): This step differs from the step c) explained above only in that the angle δ1, by which the actual orientation of the marker M relative to the camera K is given, now occurs at the location of the angle α1. Analogously to step c), the deviation of the actual orientation between camera and marker M determined by a predetermined desired orientation is also determined in step c ¹ ), which in the present example is defined by a δ 2 angle (FIG. 7).

Step d ¹ ): Now, in complete analogy to step d) explained above, a desired rotation angle φ (FIG. 7) is determined from the deviation found in step c ¹ ), about which the robot R determines the desired target orientation between the camera and marker M, and thus to align the robot R so as to rotate about the rotation axis V, that the pivot axis S is in the direction Z3.

Step e ¹ ): Finally, the robot R is rotated about the rotational axis V by the target rotational angle φ determined in step d ¹ ). Figure 7 shows the robot R of Figure 1 after this rotation. FIG. 8 shows the image field BF which the camera supplies in the situation of FIG. This rotation is indicated in Figure 5 by curved arrows. As a result of this rotation, the direction of the pivot axis S changes from the initial direction Z1 to the target direction Z3, as a result of which the robot R is aligned in a specific direction according to the invention. The bars of the marker M run after rotation at an angle δ2 to the Kx axis; the pivot axis S now extends at an angle φ2 to the Kx axis, where φ = δ1-δ2 = φ1-φ2 and φ2 = δ2 + ε. Thus, after step e ¹ ), the robot R is oriented in a specific, predetermined direction relative to the camera, namely in the present example such that the pivot axis S extends at an angle φ2 to the Kx axis.

Reference is now made to Figures 9 to 11, which illustrate a third variant of the method according to the invention.

Since in the third variant it is not about a rotation about the vertical axis V, but about a rotation about the pivot axis S, in the following explanations to the third variant of the inventive method is generally with the orientation of an object (eg marker M) relative to a another object (eg camera), or what is the same, with the orientation of the object with respect to the other object, or, which is also the same, with the orientation between these two objects, in each case that orientation meant, which the projection of the object on having a plane perpendicular to the pivot axis S relative to the projection of the other object on this plane.

The third variant of the method can be carried out additionally or simultaneously to the first variant of the method and serves not to align the robot R itself, but to align arm A of the robot R relative to the camera K by rotating or pivoting the arm A about the pivot axis S, and comprises the following steps A) to E):

Step A): FIG. 9 shows the robot R of FIG. 1 again. The marker M already shown in FIG. 1 is not glued to the robot R itself but to the top of the arm A of the robot R according to step A), so that the marker M participates in each rotation and pivoting of the arm A.

Step B): The camera K (not shown in FIG. 9) is arranged so that the camera looks at the arm A and the marker M from top to bottom optical axis perpendicular to the pivot axis S and the marker M in the detection range E of the camera is located and thus from the same as within their image field BF lying marker image M ^{1 is} shown (Figure 12). The camera is arranged directionally fixed so that it does not participate in rotational movements of the arm A.

A possible arrangement of the robot R, its arm A and the marker M after carrying out steps A) and B) is shown in FIG. 9 as an example. In the situation of Fig. 9, the robot R is aligned on the tread F so that the Kx axis is parallel and the longitudinal axis RA of the arm A is perpendicular to the pivot axis S, with the arm A of the robot R slightly raised against the horizontal. However, such an orientation is not a prerequisite for the feasibility of the method according to the invention.

FIG. 10 shows the image field BF ₁ which supplies the camera K in the situation of FIG. The marker M is imaged by the camera in the image field BF as a marker image M '. The lying in the detection range E of the camera K part of the arm A is shown as Armbild A ¹ . The angle W is shown as an angle piece image W.

Step C): The image S ^{1 of} the pivot axis S extends in Figure 10 from right to left. Of course, the axis image S ^{1 is} not a real, visible line in the image field BF ₁ but just like the pivot axis S itself only an imaginary line and in Figures 2, 4, 6, 8, 10, 11, 13 and 14 only for a better illustration of Invention each shown as a dotted line (the same applies to the image RA ^{1 of} the rotation axis RA). Perpendicular to the direction of the axis image S ¹ , the marker image M ¹ in the image field BF has a certain linear extent y1 (FIG. 10). This expansion changes upon rotation of the arm A about the pivot axis S due to the changing perspective distortion. The extension yi is therefore a measure of the angle at which the optical axis of the camera intersects the longitudinal axis RA of the arm A, or below the projection of the optical axis of the camera on a plane perpendicular to the pivot axis S projection of the longitudinal axis RA this plane intersects. In other words, the extent y1 is a measure of the actual orientation of the arm A (and thus of the marker M) present in the situation of FIG. 10 relative to the camera.

In step C), the actual orientation of the marker M relative to the camera K is now derived by means of electronic image processing from the linear expansion y1 and compared with a predetermined desired orientation of the marker M relative to the camera K.

If the marker M is in the desired orientation relative to the camera K, the linear extent which has the marker image M ¹ perpendicular to the direction of the axis image S ¹ in the image field BF of the camera K has the value y 2 in the present example (Figure 11). In step C), therefore, the value y1 is simply determined by image processing and compared with the predetermined value y2.

Step D): Now, from the deviation y1-y2 found in step C), a desired rotational angle is determined by which the arm A is to be rotated about the pivot axis S in order to bring about the desired target orientation between camera and marker M.

Step E): Finally, the arm A is rotated about the predetermined in step D) rotational angle about the pivot axis S. FIG. 11 shows the image field BF of the camera after this rotation. The arm A is now more strongly raised against the horizontal than in the situation of Figure 9. By this rotation, the longitudinal direction of the arm A is in a certain direction, whereby the arm A is inventively aligned in this direction. Thus, the arm A after the step E) in a certain predetermined direction is aligned relative to the camera, namely in the present example so that the linear extent, which has the marker image M ¹ in the image field BF of the camera perpendicular to the direction axis image S ¹ , which assumes value y2.

In principle, the linear extent which the marker image M ^{1 has} in the image field BF of the camera perpendicular to the direction of the axis image S ¹ can also be brought from the value y1 to the value y2 by the arm A not being raised against the horizontal, but lowered. Because even in this case, there is an orientation of the arm A relative to the camera, in which this linear expansion takes the value y2. Similarly, there are two orientations of the arm A relative to the camera, in which the linear extent which the marker image M ^{1 has} in the image field BF of the camera perpendicular to the direction of the axis image S ¹ assumes the value y1. The same applies to every other value of the linear extent which the marker image M ^{1 has} in the image field BF of the camera perpendicular to the direction of the axis image S ¹ .

According to a preferred variant of the invention, this ambiguity is eliminated by suitable measures. For example, this can be distinguished during the rotation of the arm A about the pivot axis S between forward and backward rotation, so that it is monitored whether, for example, the value y1 or another specific value of the linear extent which the marker image M ¹ in the image field BF of the camera perpendicular to the direction of axis image S ¹ has been achieved by forward or reverse rotation of the arm A about the pivot axis S.

Another possibility for avoiding ambiguity is that the coarse alignment of the arm is determined by another method, for example by using a probe, a light barrier, an angle encoder, a compass or a spirit level, and by means of the method according to the invention. Alignment is made. The reading of the compass or spirit level can also be done automatically by means of the camera and image processing. Numerous other methods for eliminating the mentioned ambiguity are possible.

These remarks on the ambiguity and its elimination are analogously also true for the orientations which result from the values x1 and x2 explained below.

Referring again to Figure 10 and additionally to Figures 12 and 13, which illustrate a fourth variant of the method according to the invention.

Since in the fourth variant it is not about a rotation about the vertical axis A or about the pivot axis S, but about a rotation about the axis of rotation RA, in the following explanations to the fourth variant of the method according to the invention is generally with the orientation of an object (eg M, arm A) relative to another object (eg camera), or what is the same, with the orientation of the object with respect to the other object, or, which is also the same, with the orientation between these two objects, respectively that orientation meaning, which has the projection of the object on a plane perpendicular to the axis of rotation RA relative to the projection of the other object on this plane.

The fourth variant of the method can be carried out additionally or simultaneously to the first and the third variant of the method and serves to align the laser L relative to the camera K by rotating the arm A about the axis of rotation RA, and comprises the following steps A ¹ ) to E '). These steps differ from the steps A) to E) of the third variant of the method only in that the rotation takes place about the axis of rotation RA and not about the pivot axis S, and therefore to the position of the linear expansions y1 and y2 the linear expansions x1 and x2 explained below to step. Steps A ¹ ) and B ¹ ): These steps are identical to the above steps A) and B). FIG. 9 shows as an example a possible arrangement of the robot R _{1 of} its arm A with the laser L and the marker M after execution of steps A ¹ ) and B ¹ ). FIG. 10 shows the image field BF which supplies the camera in the situation of FIG. In step A ¹ ), the marker M is indirectly, namely arranged on the arm A, the laser L, so that the marker M takes part in each rotation of the laser L.

Step C): The image RA ^{1 of} the rotation axis R extends from top to bottom in FIG.

Perpendicular to the direction of the axis image RA ¹ , the marker image M ¹ in the image field BF has a specific linear extent x ¹ (FIG. 10). This expansion changes as the arm A rotates about the axis of rotation RA due to the changing perspective distortion. The extent x1 is therefore a measure of the orientation of the arm A with respect to its own longitudinal axis RA. In other words, the extent x1 is a measure of the actual orientation of the arm A (and thus of the marker M) present in the situation of FIG. 10 relative to the camera.

In step C), the actual orientation of the marker M relative to the camera K is derived by means of image processing from the linear extent x1 and compared with a predetermined desired orientation of the marker M relative to the camera K.

If the marker M is in the desired orientation relative to the camera K, has the linear extent, which has the marker image M ¹ in the image field BF perpendicular to the direction of the axis image RA ¹ , in the present example, the value x2 (Figure 13). In step C), therefore, the value x1 is simply determined by image processing and compared with the predetermined value x2.

Step D ¹ ): Now, from the deviation x1-x2 found in step D), a desired angle of rotation is determined by which the arm R is to produce the desired Target orientation between camera and marker M to rotate about its longitudinal axis RA.

Step E ¹ ): Finally, the arm A is rotated around the rotation axis S by the SoII rotation angle determined in step D '). Figure 12 shows the robot R with the arm A ₁ the laser L and the marker M after this rotation. FIG. 13 shows the image field BF ₁ which supplies the camera in the situation of FIG.

Due to this rotation, the arm A in FIG. 12 is turned to the left with respect to FIG. As a result of this rotation, the orientation of the laser L changes into a specific target direction, as a result of which the laser L is aligned in this direction in accordance with the invention.

Thus, the laser L is aligned relative to the camera after the step E ¹⁾ in a certain predetermined direction, namely in the present example so that the linear expansion which the marker image ^M1 in the image field BF of the camera perpendicular to the direction of axis of the image RA ¹ has the value x2.

FIG. 14 shows the field of view of the camera in a situation which results from the situation of FIG. 9 in that first steps A) to E) and then steps A ¹ ) to E ') are carried out. Figure 14 illustrates that the laser L can be brought by the superposition of rotations about the axes of rotation V, S and RA in an arbitrary predetermined spatial orientation with the inventive method, the required target rotation angle all by using one and the same marker M and one and the same camera K can be determined automatically by electronic image processing.

The bar code M carries information. This information may for example contain the location of the marker M on the tread F and be read in during the execution of the method. The use of a barcode as a marker M is therefore very advantageous. The rotation axis does not need to run through the marker. Therefore, when attaching the marker on the tread F (step a) or the robot R (step a ¹ ) or the arm R (steps A or A ¹ ), care is not taken to use the marker M with high precision to place in a specific location, which saves effort and costs; also, therefore, the location of the marker can be adjusted within wide limits to the space available in each case.

The achievable with the inventive method accuracy of alignment in the target direction is limited only by the resolution of the camera K. The method therefore allows even when using a low-cost CCD camera as the camera K a very precise orientation in any direction in space. The method according to the invention thus has considerable advantages over the prior art.

Industrial Applicability:

The invention is industrially applicable e.g. in the field of logistics, the

Warehousing, industrial manufacturing and robotics.

List of reference numbers:

A arm of R

A 'Image of the part of A lying within E

BF image field of K V vertical axis of R

E detection range of K

F movement area

F ¹ Image of the part of F within E

K CCD camera Kx, Ky x-axis, y-axis of the camera coordinate system

Kx ¹ , Ky ¹ Pictures of Kx, Ky

L laser

L ¹ Picture of L M barcode

Mx, My x-axis, y-axis of the marker coordinate system

Mx ¹ , My ¹ pictures by Mx, My

M ¹ image of MR robot

R 'image of the part of R inside E

RA rotation axis of A

RA ¹ image from RA

S swivel axis S ¹ Picture of S

W elbow of R

W image of the part of W x1 lying within E, x2 linear expansions in BF perpendicular to RA ¹ y1, y2 linear expansions in BF perpendicular to S ¹ Z1 initial direction

Z2.Z3 Aiming directions α1 Orientation between K and M before rotation by φ α2 Orientation between K and M after rotation by φ ß Orientation between S and K with directionally fixed marker φ Target rotation angle with directionally fixed marker φ1 Orientation between S and M before Rotation by φ φ2 Orientation between S and M after rotation by φ

61 Orientation between M and K before rotation by φ

62 Orientation between M and K after rotation by φ φ Target rotation angle with directionally fixed camera φ1 Orientation between S and K before rotation by φ φ2 Orientation between S and K after rotation by φ ε Orientation between S and M with directional camera

Claims

claims:

A method of aligning an object (R ₁ A ₁ L) such as a robot (R), arm (A) of a robot (R) or laser (L) by rotating the object (R ₁ A ₁ L) around at least one Axis of rotation (V ₁ S ₁ RA), characterized in that a marker (M) is detected with a camera (K) and imaged as a marker image (M '), whereby either the camera (K) takes part in the rotation and the marker ( M) is fixed in direction or vice versa, so that upon rotation of the object (R ₁ A ₁ L) about the axis of rotation (V ₁ S ₁ RA) takes place a relative rotation between the marker (M) and the camera (K), the actual orientation (α1, δ1), which exists between camera (K) and marker (M) before the rotation, either from the orientation, which the marker image (M ¹ ) before the rotation in the image field (BF) of the camera (K), or from the linear extension (y1, x1) representing the marker image

(M ¹ ) before rotation in the image field (BF) of the camera (K), derived by means of electronic image processing and the result is used to determine a desired rotation angle (φ, φ), around which the object (R, A , L) for bringing about a predetermined target orientation (α2, δ2) between camera (K) and marker (M) and thus for aligning the object (R ₁ A ₁ L) in a specific target direction (Z2.Z3) about the axis of rotation (V ₁ S ₁ RA) is to be rotated, and then the object (R ₁ A ₁ L) by the target rotational angle (φ, φ) about the axis of rotation (V ₁ S ₁ RA) is rotated and thus in the target direction (Z2 .Z3) is aligned.

2. The method according to claim 1, characterized in that the following steps are carried out: a) the marker (M) is arranged directionally fixed in the environment of the article (R), b) the camera (K) is at the object (R) so arranged to participate in the rotation of the object (R), its optical axis extends parallel or obliquely to the axis of rotation (V) and the marker (M) lies in the detection area (E) of the camera (K) and thus is imaged by the same as a marker image (M ') within its image field (BF), c ) by means of electronic image processing, the orientation (α1) of the marker image (M ¹ ) in the image field (BF) of the camera (K), the actual orientation (α1) of the camera (K) relative to the marker (M) and derived with the Target orientation (α2) of the camera (K) relative to the marker (M) compared, d) the result of step c) is used to determine the SoII rotation angle (φ), e) the object (R) is rotated by the desired rotational angle (φ) about the axis of rotation (V), so that after step e) the predetermined target orientation (α2) between the camera (K) and marker (M) is present and thus the object (R) in the target direction (Z2) is aligned.

3. The method according to claim 2, characterized in that the orientation (ß) between the object (R) and the camera (K) is known or determined.

4. The method according to claim 1, characterized in that the following steps are carried out: a ') the marker (M) is arranged on the object (R) so that it participates in the rotation thereof, b ¹ ) the camera (K) is in the vicinity of the object (R) directionally arranged so that their optical axis parallel or oblique to the axis of rotation (V) - and the marker (M) in the detection range (E) of the camera (K) and thus of the same as within its image field (BF) lying marker image (M ¹ ) is mapped, c ¹ ) by means of electronic image processing is from the orientation (61) of Marker image (M ¹ ) in the image field (BF) of the camera (K) the actual orientation (δ1) of the marker (M) relative to the camera (K) derived and with the desired orientation (δ2) of the marker (M) relative to the camera (K), d ') the result of the step c') is used to determine the SoII rotation angle (φ), e ¹ ) the object (R) is changed by the desired rotation angle (φ) rotated the rotation axis (V), so that after the step e ¹ ) the predetermined target orientation (δ2) between

Marker (M) and camera (K) is present and thus the object (R) in the target direction (Z3) is aligned.

5. The method according to claim 4, characterized in that the orientation (ε) between the object (R) and the marker (M) is known or determined by measurement.

A method according to any one of claims 1 to 5, characterized in that the following steps are carried out: A) the marker (M) is placed on the object (A, L) so as to participate in its rotation,

B) the camera (K) is in the vicinity of the object (A ₁ L) fixed direction arranged so that its optical axis perpendicular or oblique to the axis of rotation (S, RA) and the marker (M) in the detection range (E) of the camera (K) is located and thus imaged by the same as within its image field (BF) lying marker image (M '),

C) by means of electronic image processing is from the linear extent (y1, x1), which in the image field (BF) of the camera (K) the marker image (M ¹ ) perpendicular to the direction of the image (S ¹ , RA ¹ ) of the rotation axis (S ₁ RA), the actual orientation of the marker (M) relative to the camera (K) derived and with the desired orientation of the marker (M) relative to the camera (K) compared,

D) the result of step C) is used to determine the desired angle of rotation,

E) the object (A ₁ L) is rotated by the desired rotational angle about the axis of rotation (S ₁ RA), so that after step E) the predetermined target orientation between marker (M) and camera (K) is present and thus the object (A ₁ L) is aligned in the target direction.

7. The method according to claim 2, characterized in that the article (R) along a path or on or parallel to a

Running surface (F) is movable, in step a) a plurality or plurality of markers along the path or on the tread (F) are arranged, the object (R) is moved sequentially to different locations, which are selected so that at each the locations of at least one of the markers are detected and imaged by the camera (K), and steps b) to e) are performed separately for each of the locations.

8. The method according to claim 7, characterized in that each marker is different from all other markers and individually identifiable by means of the camera (K) and image processing.

9. The method according to any one of the preceding claims, characterized in that as a marker (M) either a code carrier (M) or a character or a string or a pattern or dot pattern is used, wherein the marker (M) carries information that by means of the camera (K) and image processing is read out.

10. The method according to claim 9, characterized in that a one-dimensional bar code (M) is used as a code carrier, or as a code carrier, a two-dimensional bar code or a data matrix code is used.

11. The method of claim 7 or 8 and claim 9 or 10, characterized in that each marker (M) information about its own absolute position along the track or on the absolute position of its projection on the tread (F) in each case with respect to a predetermined reference point carries, and this information by means of the camera (K) and image processing is read out.

12. The method claim 2 or 3, characterized in that the orientation of the marker (M) relative to the outside world or the environment of the object (R) determined and the absolute orientation of the object (R) with respect to the outside world or with respect to the environment (R ) of the object (R) from - the orientation of the object (R) relative to the marker (M) and the orientation of the marker (M) relative to the outside world or relative to the environment of the object (R) is determined.