WO2010046759A2

WO2010046759A2 - 3d object motion tracking and locating system by means of synchronised light emitters with a stereoscopic vision system

Info

Publication number: WO2010046759A2
Application number: PCT/IB2009/007186
Authority: WO
Inventors: Paulo José CERQUEIRA GOMES DA COSTA; Paulo Sergio Lima Malheiros; António Paulo GOMES MENDES MOREIRA
Original assignee: Universidade Do Porto
Priority date: 2008-10-21
Filing date: 2009-10-21
Publication date: 2010-04-29
Also published as: PT104224A; WO2010046759A3; PT104224B; WO2010046759A4

Abstract

The present invention describes a system for tracking (6) motion (7), making it easier to detect objects (9, 10) in space by means of artificial stereoscopic vision, using high-intensity, visible light emitters (8) synchronised with the stereoscopic vision system, at least two cameras (3, 4, 5) connected to a computer. A synchronisation module (2) controls the activation time of the light emitters (8), ensuring that all cameras (3, 4, 5) acquire at this time simultaneously. The use of high-intensity light emitters (8) synchronised with the cameras differentiates the light emitters from the surroundings, rendering the system robust, and since the light emitters need be turned on only for short periods of time, the apparent intensity is very low and they cause no interference with the users (9) of the system. The use of light emitters with various colours of the visible spectrum makes detection easy, preventing uncertainties and enhancing reliability.

Description

Title of Invention: 3D MOTION CAPTURE AND OBJECT LOCATION SYSTEM USING SYNCHRONIZED LIGHT

STEREOSCOPIC VISION

Technical field of the invention

[1] The present invention relates to a stereoscopic vision-based motion capture system utilizing two or more cameras to detect and position objects in space with pulsed light emitters.

Summary of the invention

[2] The present invention describes a stereoscopic vision system for capturing motion and locating objects comprising colored and pulsed light emitters (8) as position markers.

[3] A preferred embodiment of the present invention has the feature of further comprising a synchronization module (2) between the light emitters (8) and the stereoscopic vision system (1,3,4,5).

[4] A more preferred embodiment of the present invention has the feature that the light emitters (8) are high intensity color LEDs.

[5] An even more preferred embodiment of the present invention is that the pulsating LEDs (8) last in the order of 10E-6 seconds, more specifically from 10E-2 to 10E-7 seconds.

[6] Another preferred embodiment of the present invention has the feature that the synchronization system between LEDs (8) and cameras (3,4,5) is realized via a data connection (13), in particular a wireless connection. .

[7] A still further preferred embodiment of the present invention has the feature of comparing two successively captured images in which one pulse was emitted and in the other not, thus identifying the emitter (8) and minimizing interference from other light sources.

Background of the invention

[8] US20080075324, hereinafter referred to as D1, uses an array of LEDs to calibrate the camera system while the present invention utilizes the object of study itself (moving tool) to perform such calibration. This invention does not need a specific calibration tool: you can use the markers placed on the tool to establish the calibration, you only need to know the distance between markers, allowing for easier and faster calibration of this system. [9] US20080075324 uses single-color LEDs where it is important to turn them on or off, while the present invention provides LEDs of different colors in the visible spectrum where it is only necessary to study the state in which they are active. This allows the present invention with only one image to find the positioning of the object, whereas in document D1 they require multiple images, as they need to check the on / off state of the various LEDs.

[10] JP2003035515 uses multiple frames to encode the identification of each led, while the present invention requires only one frame. This enables faster object detections, essential for real-time applications.

[11] US6608688 discloses the use of infrared LEDs while the present invention discloses the use of colored LEDs. This allows the use of standard vision equipment reducing the installation cost. This document discloses a system that requires the use of a time slot per led, while the present invention discloses the use of all leds at the same sampling time. This allows detection of all markers by time interval, thereby increasing the temporal resolution of this invention, also achieving a shorter delay in obtaining the measurement.

[12] JP2004045321 discloses a system that requires two or more

frames for detecting each marker, while the present invention provides a system that requires only one frame for detecting the markers. This allows this invention to have higher temporal resolution and a shorter delay in obtaining the measurement.

General Description of the Invention

[13] The object of the invention relates to a stereoscopic vision-based motion capture system. This system uses two or more cameras to detect and position objects in space using the same principle as human vision.

[14] Capturing images of markers representing points in space by two or more cameras placed in different positions and directions enables stereoscopic vision techniques to determine the location in space of those markers, with camera locations known or, alternatively, determining the location of the known cameras the location of the markers. However, current techniques require very careful lighting and / or the use of special (infrared) light sources and appropriate markers. This results in generally expensive systems with some complexity to put into operation and usable only in relatively restricted environments. [15] Using a small light source with high luminous power and the ability to switch on and off very quickly, it is possible for an electronic circuit to be sent via radio, wire or infrared a sync signal to the cameras and to the light source power circuit such that the short time the light is being emitted coincides with the time the camera opens. The light source that fits the presented characteristics are high intensity LEDs (Light Emitting Diode). Since the intensity of the source is very high, the camera's sensitivity can be adjusted so that only that point is visible in the captured image. Thus the system is extremely robust against disturbances resulting from ambient lighting and does not need any special lighting. In very difficult situations, for example with direct sunlight, you can only pulse the pulse in some of the images taken. By comparing two successively captured images in which one pulse was emitted and the other not, it is possible to further minimize interference from other light sources.

[16] Being pulsed also allows light intensity to appear to be low, does not disturb or interfere with people present, and reduces system power consumption.

[17] To distinguish between different markers, light-emitting

different colors and / or modulate the pulse of each marker with different frequencies. For example, a marker emits a pulse at half the frequency of image acquisition successively resulting in a pulse image followed by a pulseless image, while another marker emits two pulses in two successive images followed by a pulseless image. The use of several different markers also allows the determination not only of the position of the object to be measured, but also the orientation of that same object.

[18] The invention has as its main advantage that it allows the detection of several points in space at an immediate and very low cost. Most motion capture systems are expensive in both software and equipment, requiring specialized operators, which make these systems prohibitive for small businesses. The use of different color LEDs ensures high robustness and reliability opening new application environments.

Description of the Figures

[19] For an easier understanding of the invention, attached are figures which represent preferred embodiments of the invention which, however, are not intended to limit the scope of the present invention.

[20] In Figure 1, a possible application of this system to the control of an industrial manipulator is illustrated. Two or more cameras record the movement of a tool equipped with light emitters, controlled by an operator. This movement is then used to command and / or program the industrial robot movement.

[21] In the figure, (1) represents the host computer of the system;

(2) represents a synchronization equipment;

(3) represents camcorder 1;

(4) represents camcorder 2;

(5) represents camcorder N;

(6) represents the trajectory of the robot;

(7) represents the human trajectory;

(8) represents the pulsed light emitters;

(9) represents the human operator; and

(10) represents the robot.

[22] Figure 2 shows a mobile robot guidance application. The robot

It observes the markers distributed on the ceiling and comparing it with a map on a computer can position itself correctly in the world.

[23] In the figure, (1) represents the host computer of the system;

(2) represents a synchronization equipment;

(3) represents camcorder 1;

(4) represents camcorder 2;

(8) represents the pulsed light emitters;

(10) represents the robot, mobile in this case;

(11) represents a light emitting control unit;

(12) represents the ceiling; and

(13) represents a synchronization radio signal.

Realization Example

[24] A preferred embodiment is described below. In this preferred embodiment, the markers used must be high intensity LEDs, e.g. OSRAM Golden Dragon LEDs are LEDs that support up to 400mA of current and have about 6000mcd of light intensity.

[25] The cameras used were Sync Imaging Source. These had a resolution of 640x480 (higher resolution can also be used, thus improving the obtained measurement) and can be connected by Firewire or USB to a common laptop.

[26] The sync signal only serves to command the LEDs to light in

simultaneously with the purchase of the cameras. There is no special protocol for this timing, just an on / off signal.

[27] Typically LEDs are lit (simultaneously) long enough to be visible. by the chambers. Since these LEDs are of high intensity, it takes just a few microseconds of light to make them clearly visible.

[28] The sync system is wired to cameras, while for led markers this connection can be wired or wireless (infrared or radio frequency communication).

[29] This sync signal typically has a frequency of 60 to 5Hz. With

More sophisticated cameras will obviously increase the system's acquisition frequency.

Claims

Stereoscopic vision system for capturing motion and locating objects comprising colored and pulsed light emitters (8) as position markers. System according to the preceding claim, characterized in that it further comprises a synchronization module (2) between the light emitters (8) and a vision system.

stereoscopic (1,3,4,5).

System according to the preceding claim, characterized in that the light emitters (8) are high intensity colored LEDs. System according to the preceding claim, characterized in that the LED pulses (8) have a duration of about 10E-6 seconds in the range 10E-2 to 10E-7 seconds.

System according to the preceding claim, characterized in that the synchronization system between LEDs (8) and cameras (3,4,5) is realized via a data link (13), in particular a wireless link.

System according to any one of the preceding claims, characterized in that it compares two successively captured images in which in one of them the pulse has been emitted and in the other not, thus identifying the emitter (8) and minimizing interference from other light sources.