WO2001035208A1

WO2001035208A1 - Method for acquisition of motion capture data

Info

Publication number: WO2001035208A1
Application number: PCT/KR2000/001158
Authority: WO
Inventors: Byoung Ick Hwang
Original assignee: Byoung Ick Hwang
Priority date: 1999-11-11
Filing date: 2000-10-17
Publication date: 2001-05-17
Also published as: DE10083785T1; KR20000017755A; KR100361462B1; CN1340170A; AU7967200A; JP2003514298A

Abstract

A method for acquisition of motion capture data which is capable of reducing the amount of data indicative of motions, readily expressing natural motions in image production fields such as game character production, digital animation production, etc. and enabling a data provider to act and move with no limitation in sphere because of the absence of cable connections to his body. A plurality of color pointers with different color values are attached respectively to main joints and terminal portions of the body of the data provider. Three digital cameras (100, 200 and 300) or more are used to shoot actions and motions of the data provider. Acomputer is used to convert images picked-up by the digital cameras into X, Y and Z coordinate values. The motions of the body are extracted as three-dimensional position coordinate values based on motions of the color pointers.

Description

METHOD FOR ACQUISITION OF MOTION CAPTURE DATA

Technical Field

The present invention relates in general to the acquisition of motion capture data, and more particularly to a method for acquisition of motion capture data, wherein motions of a person or animal are captured using color markers and three-dimensional motion data is produced on the basis of the captured motions.

Background Art

With the development of motion picture and game industries, there have been introduced three-dimensional modeling, rendering and animation technologies taking a concentrated interest in a technique capable of producing natural motions.

In order to meet such an interest, there has been proposed a motion capture technique which is capable of directly capturing motions of a person, numerically expressing the captured motions, entering the resulting motion data in a computer and moving a character, etc. on the basis of the entered motion data.

In this motion capture technique, it is most important to rapidly acquire data regarding natural body motions.

Most conventional motion capture systems are adapted to attach markers or sensors to the human body, analyze output data from the sensors or marker images picked-up by cameras and measure the positions and orientations of respective joints of the human body in accordance with the analyzed results.

Such conventional motion capture systems may be classified into four types, an acoustic type, mechanical type, magnetic type and optical type, according to the operational types of markers or sensors used therein. Among them, the systems of the magnetic type and optical type are most widely used now.

The above conventional motion capture systems of the various types have the following characteristics and problems. The motion capture system of the acoustic type comprises a plurality of acoustic sensors and three acoustic receivers. The acoustic sensors are attached to respective joints of an actor to sequentially generate acoustic waves, and the acoustic receivers are disposed apart from the acoustic sensors to receive the acoustic waves generated by the sensors. This motion capture system is adapted to calculate distances from the acoustic sensors to the acoustic receivers using time periods required for the receivers to receive the acoustic waves from the sensors. The position of each of the sensors in a three-dimensional space is obtained on the basis of a triangular surveying principle using values calculated respectively by the three receivers. This system of the acoustic type is disadvantageous in that the sampling frequency is low and the number of acoustic generators available at the same time is small. Also, motions of the actor may become unnatural because the sensors are large in size. Moreover, the acoustic-type motion capture system is greatly influenced by acoustic reflex because of the properties of its acoustic arrangement.

The motion capture system of the mechanical type comprises a combination of a potentiometer and a slider for measuring motions of joints of an actor. Because this motion capture system has no receiver, it provides an absolute measurement arrangement which does not suffer any environmental interference or effects. Accordingly, as compared with the system of the acoustic type, the system of the mechanical type scarcely requires an initial calibration and need not have high-cost studio equipment. Further, as compared with other motion capture systems, the mechanical-type system is low in cost and very high in sampling frequency. However, the motion capture system of the mechanical type has a disadvantage in that the actor may unnaturally move because of mechanical units attached thereto. Moreover, the measurement accuracy of the mechanical-type system depends on how accurately the mechanical units are positioned on the respective joints of the actor.

The motion capture system of the magnetic type comprises a plurality of sensors attached to respective joints of an actor for sensing a magnetic field. When the actor moves around a magnetic field generator, the sensors sense magnetic field variations, which are then transformed into spatial values used for motion measurement. The sensors, magnetic field generator and system body are interconnected via cables or, recently, wirelessly. The motion capture system of the magnetic type is advantageous in that it is relatively low in cost and excellent in cost-to-performance ratio and can perform real-time processing. However, in the case where the sensors are of a wired type, the actor must move within a limited sphere due to cable connections from the sensors to his or her body, thereby making it difficult for the actor to naturally express his or her complex and rapid motions. For a wireless system employing a wireless transmitter instead of the cables, the actor cannot naturally move, either, because of the large size of the transmitter attached to his or her body. Further, the actor must move within a limited sphere because he or she must remain within the range of the magnetic field.

The motion capture system of the optical type comprises a plurality of reflex markers attached respectively to main joints of an actor for reflecting infrared rays, and a plurality of cameras, each having three to sixteen infrared filters mounted thereto. The cameras are adapted to create two-dimensional coordinates of reflected marker images. A dedicated program is used to analyze the two-dimensional coordinates captured by the cameras and transform them into three-dimensional spatial coordinates as a result of the analysis. This conventional optical-type motion capture system is disadvantageous in that it lays emphasis on the realistic expression of motions of a character, resulting in the need for an extraordinary large amount of data and a high-performance computer to process it.

Disclosure of the Invention

Therefore, the present invention has been made in view of the above problems, and it is an object of the present invention to provide a method for acquisition of motion capture data which is capable of processing a reduced amount of data by extracting and processing only coordinate values varying with motions.

It is another object of the present invention to provide a method for acquisition of motion capture data which is capable of very simply and conveniently performing an initial calibration for motion capture. It is a further object of the present invention to provide a method for acquisition of motion capture data wherein the amount of data to be processed is small and a low-cost personal computer, camcorder and USB camera are used with no need for complex or high-cost peripheral equipment, thereby implementing a more cost-effective system. It is another object of the present invention to provide a method for acquisition of motion capture data wherein no cable is connected between an actor and a motion data processing system, thereby making it possible to conveniently extract motions of the actor, including a violent action.

It is yet another object of the present invention to provide a method for acquisition of motion capture data which is capable of accurately measuring motions of an actor irrespective of ambient environments such as an ambient illumination, radio interference, etc.

In accordance with the present invention, the above and other objects can be accomplished by a provision of a method for acquisition of motion capture data in a motion capture system which is capable of recording motions of a person or object in a computer-processable form. This method is based on a variety of technical requirements, or a color extraction algorithm, a color position tracking algorithm, a technique for tracking motions of an actor using a plurality of color markers with different color values, an image processing technique for perceiving images of 24 to 30 frames per second by a USB camera, and a technique for transforming extracted two-dimensional data into three-dimensional data.

This motion capture technique has a basic interface function of acquiring data regarding motions of a character prior to realistically expressing the motions of the character. In view of the fact that the motion capture technique is a kind of interface method, the present invention proposes a method capable of readily acquiring data about motions of an actor.

Brief Description of the Drawings

The above and other objects, features and other advantages of the present invention will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings, in which:

Fig. 1 is a view showing the construction of a system for execution of a method for acquisition of motion capture data in accordance with the present invention;

Fig. 2 is a view showing an arrangement of cameras for acquisition of three-dimensional data in accordance with the present invention;

Fig. 3 is a schematic view illustrating the detection of X and Z coordinate values by a front camera in Fig. 2;

Fig. 4 is a schematic view illustrating the detection of X, Y and Z coordinate values by a left camera in Fig. 2; Fig. 5 is a schematic view illustrating the detection of X, Y and Z coordinate values by a right camera in Fig. 2;

Fig. 6 is a view showing a virtual motion area of an object whose motions are to be captured; and

Figs. 7 to 9 are views showing respective dead zones of the front, left and right cameras in Fig. 2 and illustrating the detection of data in the dead zones.

Best Mode for Carrying Out the Invention

As seen from Figs. 1 and 2, a plurality of color pointers with different color values are attached respectively to main joint portions and terminal portions of the body of a data provider. Three or more cameras are equiangularly arranged while being spaced apart from one another at regular intervals, to shoot actions and motions of the data provider. A personal computer is used to convert image data picked-up by the cameras into three-dimensional coordinate values (X, Y and Z), detect position variations of the color pointers on the respective body portions of the data provider from the converted three-dimensional coordinate values and store the detected position variations as motion capture data.

The computer is adapted to receive image signals from the cameras through its buffer and extract only position coordinate values about motions of the color pointers on the respective body portions of the data provider from the received image signals. Because the computer extracts and stores not all the image signals, but only position coordinate values varying with motions of the color pointers on the respective body portions, the amount of data to be processed is reduced, resulting in an increase in processing speed.

Now, a description will be given of a procedure of capturing motions and extracting three-dimensional motion data from captured images in accordance with the present invention.

The following Table 1 shows an example of the assignment of 13 primary colors among 256 colors to main body portions of an actor.

[TABLE 1]

Herein, the resolution is set to 160 pixels x 120 pixels, the color depth is set to 16 bits, the frame rate is set to 30 frames/sec, and N represents the number of color pointers. Under the above conditions, a camera extracts three primary color elements, or red, green and blue elements, from input images and transfers them to a personal computer (PC). At this time, the amount of data being transferred from the camera to the PC is 160 pixels x 120 pixels x 16 bits x 30 frames/sec = 9216000 bits/sec because it can be expressed by a multiplication of the resolution, color depth and frame rate. Calculating this value in terms of bytes, the result is 1152000 bytes/sec. Furthermore, calculating it in terms of Kbytes, the result is 1152 Kbytes/sec. In conclusion, data of 1152 Kbytes per second is transferred from the camera to the computer.

Because the amount of data calculated in the above manner is concerned with each of the red, green and blue elements, the amount of data captured for all the elements is three times the calculated data amount, or 1152000 x 3 = 3456000 bytes/sec. Calculating it in terms of Mbytes, the result is 3.456 Mbytes/sec. Finally, the total amount of data to be processed in the computer is obtained by multiplying the amount of data captured for all the red, green and blue elements by the number of color pointers.

Namely, the total amount of data to be processed in the computer is 3.456 Mbytes/sec if N = 1 and 34.56 Mbytes/sec if N = 10. Coordinate systems shown in Figs. 1 to 9 are absolute coordinate systems available for computer three-dimensional (3D) graphic and animation technologies. Taking a front view of the coordinate systems shown in Figs. 1 to 9, the Z axis is placed in the upward and downward directions, the X axis is placed in the left and right directions and the Y axis is placed in the forward and backward directions.

Hereinafter, a detailed description will be given of a method for acquisition of motion capture data in accordance with the present invention in conjunction with the accompanying drawings.

First, three or more cameras shoot actions and motions of a data provider and transfer the resulting images respectively to associated computers. The computers extract position variations of a plurality of color pointers, attached respectively to main joint portions and terminal portions of the body of the data provider, from the images transferred from the associated cameras and obtain two- dimensional motion data coordinate values from the extracted position variations. Subsequently, the computers transfer the obtained two-dimensional motion data coordinate values over an internal network to a server computer, which then transforms the transferred coordinate values into three-dimensional motion capture data according to a synthesis program based on a three-dimensional synthesis function engine.

Fig. 1 is a view showing the construction of a system for execution of a method for acquisition of motion capture data in accordance with the present invention. As shown in this drawing, three or more cameras are provided to shoot motions of a data provider and transfer the resulting images respectively to associated computers. The computers are adapted to process the images transferred from the associated cameras in the above calculation manner and transfer the processed results to a server computer over an internal network. The server computer is adapted to obtain three-dimensional motion capture data on the basis of data transferred from the above computers.

Fig. 2 is a view showing an arrangement of cameras for acquisition of three-dimensional data in accordance with the present invention. As shown in this drawing, three cameras, or a front camera 100, left camera 200 and right camera 300 are disposed around an object, or a data provider.

The three cameras are equiangularly arranged in such a manner that they are spaced apart from one another at regular angles of 120 degrees. A virtual area is set to a predetermined range around a cross point of views of the three cameras, and it is recognized as a coordinate system by computers.

In the case where the three cameras are disposed as shown in Fig. 1, they acquire Z coordinate values on all sides of an object. X and Y coordmate values of the object are obtained by performing an appropriate calculation algorithm for X coordinate value variations acquired by the front camera 100 and +X, -Y, +Y and - Y data acquired by the left and right cameras 200 and 300. Motion capture data on all sides of the object can be acquired in this manner.

Fig. 3 is a schematic view illustrating the detection of X and Z coordinate values by the front camera 100 in Fig. 2. All X and Z position coordinate values of pointers on the front side of an object are detected through the front camera 100 and then provided as a reference for comparison with data detected through the left and right cameras 200 and 300. The X and Z coordinate values detected by the front camera 100 are also provided as a reference for correction of data detected by the other cameras.

Fig. 4 is a schematic view illustrating the detection of X (30°), Y (60°) and Z coordinate values by the left camera 200 in Fig. 2. Z coordinate values and X and Y coordinate values of pointers on the left and rear sides of an object are detected through the left camera 200. The detected X and Y coordinate values are rotated on the basis of the detected Z coordinate values and then provided as a reference for comparison with data detected by the front camera 100 and right camera 300. Fig. 5 is a schematic view illustrating the detection of Y (60°), X (30°) and Z coordinate values by the right camera 300 in Fig. 2. Z coordinate values and X and Y coordinate values of pointers on the right and rear sides of an object are detected through the right camera 300. The detected X and Y coordinate values are rotated on the basis of the detected Z coordinate values and then provided as a reference for comparison with data detected by the front camera 100 and left camera 200.

Fig. 6 is a view showing a virtual motion area (an X and Y coordinate system) of an object whose motions are to be captured. The virtual area is set to a predetermined range around a cross point of views of the three cameras 100, 200 and 300, and all motions of the object are limited to within the virtual area. A dead zone of each of the cameras is detected through the crossing views of the other two cameras and accurate coordinate values thereof are obtained through verification and correction operations.

The dead zones of the three cameras are shown respectively in Figs. 7 to 9.

Coordinate values of pointers positioned in the dead zone of the front camera 100 in Fig. 7 are determined by comparing data detected by the left and right cameras 200 and 300 with each other and verifying and correcting data detected by the front camera 100 on the basis of the compared result. Coordinate values of pointers positioned in the dead zone of the left camera 200 in Fig. 8 are determined by comparing data detected by the front and right cameras 100 and 300 with each other and verifying and correcting data detected by the left camera 200 on the basis of the compared result.

Coordinate values of pointers positioned in the dead zone of the right camera 300 in Fig. 9 are determined by comparing data detected by the front and left cameras 100 and 200 with each other and verifying and correcting data detected by the right camera 300 on the basis of the compared result.

Industrial Applicability

As apparent from the above description, the present invention provides a method for acquisition of motion capture data which is capable of extracting and processing only color pointer data representing motions of an actor. Therefore, the amount of data to be processed is significantly reduced, resulting in an increase in processing speed as compared with the processing of the entire image. Further, a low-cost personal computer, camcorder and USB camera are used with no need for complex or high-cost peripheral equipment, thereby implementing a more cost- effective system capable of very simply and conveniently performing an initial calibration for motion capture at any place.

Further, there are no wired and wireless connections made between an actor and a motion data processing system, so that the actor can move with no limitation in sphere, thereby making it possible to conveniently extract motions of the actor, including a violent action.

Moreover, the entire performance of the present system can be simpler improved by merely upgrading software, as compared with the conventional performance improvement based on hardware.

Furthermore, owing to the above features, the present invention is widely applicable to an entertainment field, a virtual reality field and other image production fields.

Although the preferred embodiments of the present invention have been disclosed for illustrative purposes, those skilled in the art will appreciate that various modifications, additions and substitutions are possible, without departing from the scope and spirit of the invention as disclosed in the accompanying claims.

Claims

Claims:

1. A method for acquisition of motion capture data, comprising the steps of: a) attaching a plurality of color pointers with different color values respectively to main body portions of an actor; b) shooting motions of said actor by camera means; and c) converting image data picked-up by said camera means into three- dimensional coordinate values (X, Y and Z), detecting position variations of said color pointers on the respective body portions of said actor from the converted three-dimensional coordinate values and extracting three-dimensional position coordinate values based on motions of said color pointers from the detected position variations.

2. A method for acquisition of motion capture data, as set forth in Claim

1, wherein said camera means includes at least three cameras equiangularly arranged while being spaced apart from one another at regular intervals, said three cameras being a front camera, a left camera and a right camera disposed around said actor; and wherein said step c) includes the steps of: acquiring vertical motion (Z coordinate) data from all of said cameras, acquiring horizontal motion (X coordinate) data from said front camera and acquiring forward and backward motion data by performing a predetermined calculation algorithm for the acquired horizontal motion data and data acquired from said left and right cameras; and acquiring data regarding respective dead zones of said cameras by performing a parallel calculation algorithm for complementary data from said cameras, and generating three-dimensional data on the basis of the acquired dead zone data.