WO2012165882A2

WO2012165882A2 - Apparatus for rehabilitation exercise, wearable communication apparatus, and application system and method for applying same

Info

Publication number: WO2012165882A2
Application number: PCT/KR2012/004313
Authority: WO
Inventors: 최용근
Original assignee: 주식회사 네오펙트
Priority date: 2011-05-31
Filing date: 2012-05-31
Publication date: 2012-12-06
Also published as: WO2012165882A9; WO2012165882A3

Abstract

Provided is an apparatus for finger rehabilitation exercise, comprising: a rehabilitation exercise glove, which is made in a glove shape, comprising a wireless communication module, a sensor for sensing hand movement information in accordance with at least one of a finger movement, and a change in hand shape, a change in hand position, and a control portion for controlling so that the hand movement information of a user that is sensed by the sensor portion, which is transmitted to a remotely located communication terminal, through the wireless communication module; and the communication terminal comprising a communication module, an object generation portion for generating a virtual hand-shaped object which corresponds to the hand movement information of the user which is received, when the hand movement information is received from the rehabilitation exercise glove by means of the communication module, and a control portion for controlling so that the hand-shaped object that is generated by the object generation portion is displayed through a display screen.

Description

Rehabilitation exercise device, wearable communication device and application system and method using the same

The present invention relates to a rehabilitation exercise device, a wearable communication device that can be used for a variety of purposes such as exercise (training), game / entertainment, data input, and remote cooperative rehabilitation training, remote control of electronic devices using such devices This relates to a possible application system and a training schedule optimization method applicable to the above apparatus or application system.

In case of stroke or Parkinson's disease, various physical changes occur depending on the condition. In particular, in the case of such a disease, the hand is paralyzed and the fingers are associated with the phenomenon of fading. If the numbness of the hand and finger gradation are left unattended, the muscles or joints become stiff and gradually feel pain when moving, and even nerve recovery may interfere with normal activities. In addition to the case of the above-mentioned specific disease, there are many cases where the movement of the finger is impaired by an accident. Therefore, in such a case, it is very important that the paralyzed or handicapped hand is continuously rehabilitated to maintain movement ability by promoting blood circulation and neural communication.

Various assistive measures such as hand movement assisting robots have been disclosed to induce continuous movement of the hand of neurological disorders such as stroke and Parkinson's disease, but most of them are difficult to carry due to their weight and volume and are expensive. It is used only in institutions and is not widely distributed.

In addition, the finger assist force required for rehabilitation of the hand function of a stroke patient does not need sufficient force to perform normal hand function like the general person, and it is more effective to assist the minimum force for performing specific task training. Finger assist forces need not be constant and need to be constantly changed to appropriate values depending on other factors such as rehabilitation status and motivation.

On the other hand, the data input device according to the related art may be used by attaching a device such as a keyboard, a mouse, or a joystick when implementing the external type of the wireless terminal device, but it is inconvenient to carry. In the case of the embedded type, it is necessary to make efforts to miniaturize the contact surface in consideration of the trend of miniaturization of the terminal device. Therefore, the built-in keyboard, key buttons, direction keys, etc. are inconvenient for the user to operate due to the small size. In other words, since the size of the terminal device is small, the size of the keypad and keys is also small, so data input is not easy. It is inconvenient to use a small input device compared to the user's hand, and a person with a thick finger may make a mistake of wrongly pressing a key.

In addition, the wearable game control device according to the prior art, as disclosed in Korean Patent Laid-Open No. 2011-0012334, is provided with a separate key button, direction selection button, and the like for game operation. Accordingly, there is a difficulty in implementing various hand gestures, postures, position variations, and the like of the user, and thus, there are limitations in various game operations, and there are aspects that are difficult to be utilized in various application fields other than games.

In addition, recent research in athletic science consistently supports that when training exercise tasks, the level of long-term learning after training depends on the training schedule. It has been found that distributed training schedules are more effective than dense training schedules, variable training schedules are more effective than single training schedules, and random training schedules are more effective than sequential training schedules. However, it is not yet known how to find a training schedule that generally optimizes long-term learning levels in various situations.

A first object of the present invention is to provide a rehabilitation exercise apparatus and a method for assisting finger rehabilitation of a patient with a finger disorder or a person in need of rehabilitation of a finger.

It is a second object of the present invention to provide a rehabilitation training system capable of finger rehabilitation training through cooperation between respective users of two or more communication terminals remotely.

A third object of the present invention is to use a wearable communication device capable of sensing various body motions of a user and quickly and easily perform operations of an electronic device, an operation of an application running on the electronic device, and data input through communication interworking therewith. The present invention provides a remote control system using a wearable communication device, which can be controlled and enables more intuitive data input and motion control.

A fourth object of the present invention is to provide a training schedule optimization method using a brain science exercise learning model and an optimal control theory when training a single or a plurality of exercise tasks.

According to a first aspect of the present invention, there is provided a device for finger rehabilitation exercise,

A wireless communication module; A detector configured to detect hand motion information according to at least one of a finger movement, a change in posture of a hand, and a change in position of a hand; A rehabilitation exercise glove including a control unit for controlling the user's hand motion information detected by the detection unit to be transmitted to a communication terminal located remotely through the wireless communication module, and manufactured in the form of a glove;

Communication module; An object generator configured to generate a virtual hand-shaped object corresponding to the received hand gesture information of the user when the hand gesture information is received from the rehabilitation exercise glove by the communication module; Provided is a finger rehabilitation exercise apparatus including a communication terminal including a control unit for controlling the hand-shaped object generated by the object generating unit to be displayed on a display screen.

According to a second aspect of the present invention, there is provided a device comprising: a first finger rehabilitation exercise device including a first rehabilitation exercise glove and a first communication terminal, and used for a finger rehabilitation exercise of a first user; And a communication server configured to perform data communication with the first communication terminal and to relay data communication between a second communication terminal by a second user located at a distance with the first communication terminal and cooperating with a rehabilitation exercise of the first user. Provided is a remote cooperative rehabilitation training system comprising a.

Here, the first rehabilitation exercise gloves,

A first wireless communication module; A first detector configured to detect hand motion information according to at least one of a movement of a finger of the first user, a change in posture of a hand, and a change in position of a hand; And a control unit which transmits the hand gesture information detected by the first sensing unit to the first communication terminal through the first wireless communication module.

Here, the first communication terminal,

A first communication module; Create a virtual first hand-shaped object corresponding to the hand motion information of the first user received from the first rehabilitation exercise gloves, and rehabilitation of the first user from the second communication terminal by relaying the communication server A first object generation unit generating a virtual cooperation object corresponding to the received rehabilitation exercise cooperation information when the cooperation information regarding the exercise is received; And a controller configured to display, on a display screen, a first rehabilitation exercise assistance image in which the first hand-shaped object and the virtual cooperative object are generated by the first object generator.

The apparatus may further include a second finger rehabilitation exercise device used for the finger rehabilitation exercise of the second user.

Here, the second finger rehabilitation exercise device,

A second wireless communication module; A second sensor configured to detect hand motion information according to at least one of a movement of a finger of the second user, a change in posture of a hand, and a change in position of a hand; And a second rehabilitation exercise glove including a control unit for transmitting the hand gesture information detected by the second detection unit to the second communication terminal through the second wireless communication module, and the second communication terminal.

Here, the second communication terminal,

A second communication module; A second object generator configured to generate a virtual second hand-shaped object corresponding to the hand gesture information of the second user received from the second rehabilitation exercise glove; And a control unit for transmitting the received hand gesture information of the second user to the communication server through the second communication module and displaying the generated second hand object on a display screen.

According to a third aspect of the present invention, there is provided a remote control system using a wearable communication device, the wearable communication device which is worn on a user's body and transmits motion sensing information generated at a worn portion to a remote electronic device; And an electronic device that is driven by the electronic device, performs network connection with the wearable communication device, obtains user motion information from the motion sensing information received from the wearable communication device, and corresponds to the obtained user motion information. There is provided a remote control system comprising a device interworking agent, which controls the operation of any one of the applications running on the electronic device.

According to the fourth aspect of the present invention, a method of optimizing a training schedule when training a single or a plurality of exercise tasks, the method comprising: measuring an initial learning level of an individual exercise task, and brain science exercise learning for the individual exercise task; Predicting the parameters of the model, optimizing the long-term learning based on the learning level of the individual exercise task, the brain science exercise learning model, the parameters of the brain science exercise learning model corresponding to the individual exercise task, and the optimal control theory There is provided a training schedule optimization method comprising the step of finding a training schedule.

According to the first aspect of the present invention, the effect of the rehabilitation exercise can be enhanced by assisting the finger rehabilitation therapist to make the rehabilitation exercise more enjoyable and visually stimulating the rehabilitation exercise.

According to the second aspect of the present invention, there is an effect that a cooperative rehabilitation training among a plurality of rehabilitation patients, and guidance rehabilitation training between a rehabilitation patient and a rehabilitation therapist or a doctor are possible. In addition, according to the remote cooperative rehabilitation training system according to the embodiment of the present invention, the rehabilitation training by the cooperation of two or more persons in the remote, there is an effect that can improve the sociality of rehabilitation patients.

According to a third aspect of the present invention, by using a wearable communication device that can sense a variety of body movements of the user and through the communication interworking with the operation of the electronic device, the operation of the application running on the electronic device, data input, etc. Quick and easy control, more intuitive data input, motion control is possible. In addition, when the present invention is applied as a game control system, there is an effect that can maximize the user's interest through the reproduction of more various application operations, more intuitive game control.

According to the fourth aspect of the present invention, in training a single or a plurality of exercise tasks, there is an effect that can optimize the training schedule using the brain science exercise learning model and the optimal control theory. In addition, it is widely applied to athlete training, music equipment training, and rehabilitation training to which exercise learning is applied, thereby maximizing the training effect.

1 is a view for explaining the overall finger rehabilitation exercise device according to the first aspect of the present invention.

2 is a view for explaining the rehabilitation exercise gloves according to the first aspect of the present invention.

3 is a block diagram of a communication terminal according to the first aspect of the present invention performing communication with a rehabilitation exercise glove and performing a finger rehabilitation exercise assistance method.

4 is a view for explaining a finger rehabilitation exercise assistance method according to a first aspect of the present invention.

5 is a view showing an application example of the finger rehabilitation exercise assistance method according to the first aspect of the present invention.

6 is a view for explaining the overall cooperative rehabilitation training system according to a second aspect of the present invention.

7 is a diagram showing an application example of the remote cooperative rehabilitation training method according to the second aspect of the present invention.

8 is a view for explaining a rehabilitation training method according to a second aspect of the present invention by a remote cooperative rehabilitation training system.

9 is an overview of a remote control system using the wearable communication device according to the third aspect of the present invention.

10 is a block diagram of a device interlocking agent driven on a wearable communication device and an electronic device according to a third aspect of the present invention.

FIG. 11 is a flowchart schematically illustrating a remote control method according to interworking between a wearable communication device and a device interworking agent according to a third aspect of the present invention. FIG.

12 is a diagram illustrating a training schedule optimization method using a brain science exercise learning model and an optimal control theory according to a fourth aspect of the present invention.

The present invention may be variously modified and have various embodiments, and specific embodiments will be illustrated in the drawings and described in detail with reference to the accompanying drawings. However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention.

In describing the present invention, when it is determined that the detailed description of the related known technology may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted. In addition, numerals (eg, first, second, etc.) used in the description process of the present specification are merely identification symbols for distinguishing one component from another component.

In addition, in the present specification, when one component is referred to as "connected" or "connected" with another component, the one component may be directly connected or directly connected to the other component, but in particular It is to be understood that, unless there is an opposite substrate, it may be connected or connected via another component in the middle.

In addition, the terms "~ unit", "~ group", "~ ruler", "~ module" as used herein refers to a unit for processing at least one function or operation, which means hardware or software or hardware and It can be implemented in a combination of software.

In addition, it is intended to clarify that the division of the components in the present specification is only divided by the main functions in charge of each component. That is, two or more components to be described below may be combined into one component, or one component may be provided divided into two or more for each function. Each of the components to be described below may additionally perform some or all of the functions of other components in addition to the main functions of the components, and some of the main functions of each of the components are different. Of course, it may be carried out exclusively by.

Hereinafter, embodiments of the present invention will be described in detail in turn.

In the present specification, the rehabilitation patient will be described in the case of using the rehabilitation exercise (training) through the finger movement, the rehabilitation exercise apparatus according to the present invention, in addition to the purpose of rehabilitation, to increase the muscle strength of the finger or to practice the fine hand movement Of course, the required person may use it for a specific purpose. For example, the finger rehabilitation exercise device of the present invention is useful to those who have a specific purpose of increasing the strength of the finger (for example, a person who needs to greatly increase the grip of the hand such as a wrestler, a gymnast, etc.). Make it clear first that it can be used.

[손가락 재활 운동 장치 및 재활 운동 보조 방법] [Finger Rehabilitation Exercise Device and Rehabilitation Exercise Aid Method]

According to a first aspect of the present invention, an apparatus and a rehabilitation exercise assisting method for a finger rehabilitation exercise in connection with a finger rehabilitation exercise are disclosed.

Hereinafter, an apparatus and a rehabilitation exercise assisting method for a finger rehabilitation exercise according to a first aspect of the present invention will be described with reference to FIGS. 1 to 5 of the accompanying drawings.

1 is a view for explaining the overall finger rehabilitation exercise device.

Referring to FIG. 1, the finger rehabilitation exercise device may include a rehabilitation exercise glove 100b and a communication terminal 200b connected thereto.

The rehabilitation exercise glove 100b may be manufactured in the form of a glove that can be worn on the hand to assist the rehabilitation exercise (training) of the finger rehabilitation patient (see FIG. 2), as in the block diagram of FIG. 1, The wireless communication module 14 may include a detector 12 and a controller 16. However, FIG. 1 illustrates only some of the main components, and in addition, a configuration for performing basic functions (for example, the battery of FIG. 2) or a configuration required for adding various functions (for example, EMG It is obvious that the addition of a measurement function may further include an EMG sensor.

In the present invention, the rehabilitation exercise glove 100b basically transmits various hand motion information (ie, finger movement, hand posture change, hand position change (movement), etc.) of a user wearing the glove by a wireless communication method. It serves to transmit to the external communication terminal (200b). Detailed configuration description for this will be described later with reference to FIG.

The communication terminal 200b receives the hand motion information transmitted from the rehabilitation exercise glove 100b, and a virtual hand object corresponding to the received hand motion information of the user (hereinafter, referred to as a user hand object). Name) and display it on the display screen. Accordingly, the finger rehabilitation patient can immediately visually check his or her hand movement in the process of the finger rehabilitation exercise, thereby being able to be motivated to the rehabilitation exercise and become interested in the rehabilitation exercise. . This will be more clearly understood from the following description, and the specific configuration of the communication terminal 200b for this will be described later with reference to FIG. 3.

2 is a view for explaining the rehabilitation exercise gloves. As described above, the rehabilitation exercise glove 100b may include a wireless communication module 14, a sensor 12, and a controller 16. Hereinafter, each component is demonstrated in order.

First, the detection unit 12 serves to detect various hand motion information of a rehabilitation patient wearing the rehabilitation exercise glove 100b. Here, the hand motion information that may be sensed by the sensor 12 may include at least one of a finger movement, a change in posture of the hand, and a change (movement) of the hand. To this end, the sensing unit 12 may include the following detailed components.

For example, when detecting information about a user's finger movement, the detector 12 may include a finger angle sensor as indicated in FIG. 2. In this case, the finger angle sensor may be installed at a portion corresponding to the position where each finger is inserted when the user wears gloves. For example, the finger angle sensor may be installed to extend in the longitudinal direction of the finger on the glove portion corresponding to the back surface or the bottom surface of the finger. Alternatively, a finger angle sensor may be separately installed at each portion corresponding to each node of the finger. That is, the installation site and the number of installation of the finger angle sensor may be variously modified according to needs and designs.

According to such a finger angle sensor, it is possible to quantitatively check data regarding the degree of bending / unfolding of a finger as the rehabilitation patient takes an action of bending or extending a finger in a rehabilitation exercise process.

In addition, the sensing unit 12 may further include a sensor (eg, a gyro sensor, a gravity sensor, an acceleration sensor, etc.) for detecting a change in posture of a user's hand and / or a change in position of the hand. In such a configuration, the sensing unit 12 can confirm a change in posture, such as hand inversion, rotation, and the like, and can also confirm a position change of the hand indicating a two-dimensional or three-dimensional movement trajectory by the hand.

In addition, the sensing unit 12 may further include a pressure sensor, a force sensor, and the like, if necessary, so that the rehabilitation patient may sense a force (grip force, etc.) input during the finger rehabilitation exercise process.

When the hand motion information (that is, motion information according to the degree of bending / unfolding of the finger, hand posture information, hand position information, etc.) is detected by the detector 12 as described above, the controller 16 may determine that the information is included. Control to be transmitted to the communication terminal 200b remotely located through the wireless communication module 14.

In this case, the controller 16 may transmit the hand gesture information detected by the sensor 12 to the communication terminal 200b as it is, and collect or / or predefine the sensing data output from each sensor. After processing to transmit to the communication terminal 200b.

In another embodiment, the rehabilitation exercise glove 100b and the communication terminal 200b may be connected in a wired manner. However, in the case of a wired connection, when the rehabilitation patient performs a finger rehabilitation exercise, there may be inconvenience in hand operation. In one embodiment of the present invention, the wireless communication module 14 is mounted on the rehabilitation exercise glove 100b. Therefore, a description will be given mainly on the case of transmitting the corresponding hand gesture information to the communication terminal 200b through a wireless communication method.

In this case, as a wireless communication method in which the user's hand operation information is transmitted to the communication terminal 200b, a Bluetooth communication method may be used as shown in FIG. 2. In this case, since the communication terminal 200b also includes a Bluetooth communication module, the communication terminal 200b may be connected to the rehabilitation glove 100b and peer-to-peer to directly receive the hand gesture information. However, in the present invention, a communication method for transmitting and receiving hand gesture information is not limited to the above-described Bluetooth method, and various short-range wireless communication methods such as WIFI Direct technology may be used. In the present invention, the communication method between the rehabilitation exercise glove 100b and the communication terminal 200b is not limited to the short range wireless communication method. For example, the communication terminal 200b may receive the user's hand motion information transmitted from the rehabilitation exercise glove 100b through a wireless relay device such as an access point (AP).

In the above, the rehabilitation exercise glove 100b has been described. Hereinafter, the configuration of the communication terminal 200b for assisting the finger rehabilitation exercise of the rehabilitation patient will be described with reference to FIG. 3.

3 is a block diagram of a communication terminal communicating with a rehabilitation exercise glove and performing a finger rehabilitation exercise assistance method.

First, referring to FIG. 3, the communication terminal 200b may include a communication module 210, a camera module 220, a user input unit 230, a storage unit 240, an image output device 250, and an audio output device ( 255, an object generator 260, an image processor 270, a controller 280, and a power source 290. Here, the components shown in FIG. 3 are components provided according to an example of the communication terminal 200b, and may be implemented as a communication terminal having fewer components or a communication terminal having more components. .

In addition, in the present invention, the communication terminal 200b for performing a function to assist the finger rehabilitation exercise in accordance with the communication connection with the rehabilitation exercise gloves 100b, such as a mobile phone, a smartphone, a tablet PC (smart pad), a notebook, etc. It may have a form of a terminal (or mobile terminal) having portability. However, this is because, in the case of having a portable terminal, the rehabilitation patient has the advantage of being able to perform the rehabilitation exercise anytime and anywhere without a specific spatial limitation, and the communication terminal 200b is in the form of a fixed terminal such as a desktop PC or a smart TV. You may have. For example, when a rehabilitation patient is uncomfortable in the behavior itself, and the rehabilitation exercise is performed only in a fixed space, it is not necessary to have a form of a portable terminal. That is, in the present invention, the communication terminal 200b includes a communication module 210 capable of receiving hand motion information from the rehabilitation exercise glove 100b, and virtualizes the hand motion information on the screen. As long as it has the minimum configuration possible, the terminal type is not particularly limited.

However, hereinafter, for convenience of description, it is assumed that the communication terminal 200b is a portable terminal used by the rehabilitation patient. 3, the camera module 220, the user input unit 230, the storage unit 240, the image output device 250, the audio output device 255, the image processor 270, the power supply unit 290, and the like are illustrated. Since the basic configuration of a mobile terminal such as a smartphone, a detailed description thereof will be omitted.

Hereinafter, the core components related to the method for assisting the finger rehabilitation exercise of the present invention will be described with reference to the object generator 260 and the controller 280, with reference to FIGS. 4 and 5.

4 is a view for explaining a finger rehabilitation exercise assistance method, Figure 5 is a view showing an application example of the finger rehabilitation exercise assistance method.

Description of Steps S100, S102, S104

In step S100 of FIG. 4, a communication connection is made between the rehabilitation glove 100b and the communication terminal 200b. As an example, assuming a case of the Bluetooth communication method, when the communication terminal 200b is booted with the power turned on, the boot loader is automatically set to search for a nearby Bluetooth communication device by the boot loader. The rehabilitation gloves 100b in the vicinity may be identified and a communication connection may be made thereto. In this case, the control unit 280 of the communication terminal 200b, as the communication connection is made between the rehabilitation gloves 100b and the communication terminal 200b, the rehabilitation exercise program 282 installed in the communication terminal 200b. It can be executed automatically (see step S102). When the rehabilitation exercise program 282 is executed in this way, the rehabilitation exercise assistance screen (see FIG. 5) may be displayed on the display screen (see step S104).

In this case, as the rehabilitation exercise program 282 is executed, the rehabilitation exercise auxiliary screen initially displayed on the screen may be a predetermined default screen. Such a default screen may have various screen configurations depending on how the rehabilitation exercise program 282 is implemented. In some cases, a virtual user hand object, which will be described later, may be set in a default screen to be displayed at a predetermined position and a finger posture. In this case, as the user's hand motion information is received from the rehabilitation exercise glove 100b, the corresponding position and posture may be changed accordingly. In a similar sense, a virtual demonstration hand object and / or a virtual rehabilitation tool, which will be described later, may also be displayed in advance in the default screen.

However, the communication connection step, the rehabilitation program execution step, and the like described through the steps S100, S102, and S104 may be different from those in FIG. 4. That is, in FIG. 4, the case in which the communication terminal 200b communicates with the rehabilitation exercise gloves 100b by searching for a nearby device capable of short-range communication in the booting process of the communication terminal 200b has been described. have. That is, the communication connection between the rehabilitation exercise glove 100b and the communication terminal 200b may be made after step S106 of FIG. 4. For example, only when the hand motion information of the user is detected in the rehabilitation exercise glove 100b, before performing step S108 (that is, transmitting the hand motion information to the communication terminal 200b) of FIG. 4, The rehabilitation exercise glove 100b may attempt to connect with the corresponding communication terminal 200b by searching for a surrounding communication device.

In addition, although FIG. 4 illustrates the case where the execution step S102 of the rehabilitation exercise program is made after the communication connection step S100, the order may be reversed. For example, after the rehabilitation patient or another person (guardian, therapist, etc.) executes the rehabilitation exercise program 282 installed in the communication terminal 200b, the communication connection between the devices may be made.

In addition, although FIG. 4 illustrates a case in which the rehabilitation exercise program 282 is set to be executed automatically when a communication connection is established, on the contrary, after the hand motion information is received from the rehabilitation exercise glove 100b in step S108 of FIG. 4. In addition, the rehabilitation exercise program 280 may be set to be automatically executed.

Here, the rehabilitation exercise program 282 may be manufactured in the form of a software application and installed in the communication terminal 200b, or may be downloaded and installed from an external application providing server or the like.

In this case, the rehabilitation exercise program 282 includes information on a rehabilitation sequence that defines the order of the finger rehabilitation exercise, information on each rehabilitation operation required for the finger rehabilitation exercise, and an operation guide for guiding the rehabilitation operation. At least one of information and information on a rehabilitation tool to be used in the finger rehabilitation exercise may be included.

Here, the operation guide information for guiding the rehabilitation operation, may be composed of any one of text, voice, image, video or a combination of two or more thereof. For example, a rehabilitation operation of holding and releasing a hand may be used to guide the rehabilitation patient by text or voice, or an image or video demonstrating such an operation may correspond to the operation guide information.

The information on the rehabilitation tool is information on actual aids and objects that can be used for the finger rehabilitation exercise, which is then generated by the object generator 260 as a virtual object and displayed on the screen. This is to give the rehabilitation patient the experience of training with a real tool or object. For example, the cylindrical object of FIG. 5 corresponds to this.

Of course, in addition to the various information that can be used in the finger rehabilitation exercise and it is obvious that the algorithm may be further included. The rehabilitation exercise program 282 may be updated in real time or periodically. For example, a rehabilitation exercise is performed by the communication terminal 200b receiving or periodically downloading training schedules, training guidance data, and the like, which are given by doctors, therapists, and the like who are in charge of rehabilitation of the rehabilitation patient from the server of the medical institution. Program 282 may be updated continuously.

Description of Steps S106, S108, S110, S112

When the user's hand motion information is detected by the sensing unit 12 of the rehabilitation exercise glove 100b in step S106 of FIG. 4, the hand motion information detected by the rehabilitation exercise glove 100b according to step S108 is transmitted to the communication terminal 200b. Is sent). In this case, the transmission of the hand gesture information may be periodically performed at a predetermined time interval (for example, 0.2 seconds). Alternatively, only when a change occurs in the hand gesture information, the hand gesture information may be transmitted.

As described above, when the user's hand motion information is received from the rehabilitation exercise glove 100b, in step S110, the object generation unit 260 of the communication terminal 200b virtually corresponds to the received user's hand motion information. Creates a hand-shaped object. Accordingly, the controller 280 of the communication terminal 200b displays the generated user hand object on the display screen (that is, the rehabilitation exercise assistance screen) in step S112.

An example of this is shown in FIG. 5. Referring to FIG. 5, it can be seen that a virtual hand-shaped object that is performing the same hand gesture as the hand gesture actually taken by the user is displayed together on the rehabilitation exercise assistance screen.

In the above, with reference to FIG. 4, when the hand gesture information of the user is received from the rehabilitation exercise glove 100b at one point in time by the communication terminal 200b, a virtual user corresponding to the hand gesture information received at the point in time. Only one process of creating a hand-shaped object and displaying it on the rehabilitation exercise assistance screen has been described.

However, the process of the above steps S110 and S112 may be repeated continuously whenever the user's hand motion information is received from the rehabilitation gloves 100b. That is, the object generator 260 continuously updates the virtual user hand object whenever the user's hand motion information is received, and the controller 280 updates the hand object when the hand object is updated. The image of the hand-shaped object may be reflected in real time through the rehabilitation exercise assistance screen.

In FIG. 5, the case where the rehabilitation exercise assistance screen is output through the display device of the communication terminal 200b is illustrated, but it is not necessarily the same. For example, the communication terminal 200b may transmit the image related to the rehabilitation exercise assistance screen generated by the communication terminal 200b to another medium having a large display screen so that the corresponding screen is displayed through the screen of the other medium. For example, in the case of a smartphone or a smart pad, the cable may be connected to another large screen medium (for example, a TV, etc.) provided in a corresponding space through an HDMI (High Definition Multimedia Interface) terminal provided in the terminal. In addition, the rehabilitation exercise assistance screen may be displayed on a large screen medium. Of course, even in such a case, the screen may be transmitted to another medium through a wireless communication method or a near field communication method, rather than a wired connection method such as a cable.

In the above, the finger rehabilitation exercise apparatus and the method for assisting the rehabilitation exercise according to an embodiment of the present invention have been described. Hereinafter, various methods for implementing the same will be described. However, in the following description, only various examples may be provided in the various implementation manners.

Use of virtual rehabilitation tool objects

As described above, the rehabilitation exercise program 282 may include information on various rehabilitation tools that can be used for finger rehabilitation exercises. Accordingly, when a rehabilitation tool is determined according to the rehabilitation exercise program 282 that can be usefully used to train the rehabilitation according to a specific rehabilitation operation, the object generator 260 may be configured to the rehabilitation tool in addition to the user's hand-shaped object. You can also create more virtual objects.

In the case of FIG. 5, a cylindrical virtual object is illustrated as the virtual rehabilitation tool object. Accordingly, the rehabilitation patient may perform the rehabilitation exercise by changing his hand motion in accordance with a specific rehabilitation motion while viewing this on the screen. Examples are described below.

As an example, assume that the rehabilitation operation moves the virtual cylinder of FIG. 5 to a specific position.

In this case, the controller 280 of the communication terminal 200b may overlap the area where the user hand-shaped object is displayed on the screen with the area where the rehabilitation tool object is displayed by a predetermined ratio or more, and receive the hand gesture of the received user. Based on the information, when it is determined that the user's finger is bent at a predetermined angle or more, it may be determined that the user has taken a virtual operation of grasping (holding) the virtual rehabilitation object.

The controller 280 determines that a change has occurred in the position of the user's hand according to the user's hand motion information which is subsequently received while the virtual operation of holding the rehabilitation tool object by hand is performed. The tool object may be moved and displayed on the screen in response to a change in the position of the hand object.

As another example, it is assumed that the rehabilitation operation is the bending and releasing of the hands, and that the virtual rehabilitation tool has the shape of a flexible ball, for example.

In this case, the control unit of the communication terminal 200b may receive the user's hand motion information successively received in a state where an area where a user's hand object is displayed on the screen and a portion overlapping with the area where the rehabilitation tool object is displayed are present. According to the present invention, when a movement change occurs in the user's finger, the shape of the rehabilitation tool object displayed on the screen may be modified as much as the user's hand object and the rehabilitation tool object overlap.

In addition to the examples described above, there can be more various examples of rehabilitation training using the virtual rehabilitation tool object.

Use of demonstration hand objects

In another example, a demonstration hand object corresponding to a rehabilitation operation to be performed by a rehabilitation patient may be further displayed on the rehabilitation exercise assistance screen. To this end, the object generator 260 may further generate a demonstration hand-shaped object for demonstrating the corresponding rehabilitation operation.

In this case, the demonstration hand object may be displayed on a screen area that does not overlap with the user's hand object on the rehabilitation exercise assistance screen. For example, a small window may be displayed at the bottom of the screen. As another example, a demonstration hand object may be displayed on the left side of the screen, and a user hand object may be displayed on the right side.

In a different implementation manner from the above, the demonstration hand object may overlap the user's hand object, but may be displayed in a state that can be distinguished from each other. For example, a demonstration hand object that demonstrates rehabilitation behavior is displayed in a translucent state, and the user's hand object is initially displayed in a position overlaid with the user so that the user has the same hand as the demonstration hand object. It may be displayed so that it is easy to check whether or not the coincidence between the hand gestures of both when taking an action.

Use of Rehabilitation Exercise Assistance Screen with Augmented Reality

The background screen may be configured as an augmented reality screen of the above-described rehabilitation exercise assistance screen. That is, the communication terminal 200b sets the preview image (ie, the camera front image) obtained through the camera module 220 as a background screen, and then displays the finger-shaped object and the rehabilitation tool object described above, thereby rehabilitation of the rehabilitation. It may also implement an augmented reality environment for exercise.

Adoption of game progression form

In addition, rehabilitation exercises performed through the rehabilitation exercise assistance screen may be configured as a type of game progression. In this case, the rehabilitation exercise program 282 may also include game progress algorithms and data therefor.

For example, the rehabilitation exercise may be implemented in a game form such as building blocks, blowing balloons, etc., depending on the implementation of the rehabilitation exercise program 282. Here, assuming that the case of a block stacking game type, a rehabilitation patient can obtain a virtual experience such as moving a real block by performing a virtual action such as stacking a block, which is a virtual rehabilitation tool, at a specific position.

And assuming that the game is in the form of a balloon blowing game, the rehabilitation patient repeats the action of squeezing and releasing the hand, and then a virtual action such as blowing a balloon through a virtual action such as repeatedly pressing an actual air injector. You will be able to experience. In this case, an air injector may be displayed as a virtual rehabilitation tool object on the rehabilitation exercise assistance screen.

As described above, in the case of the game progression form, it is possible to reduce the boredom felt by the rehabilitation patient during the rehabilitation exercise and to give interest and motivation to the rehabilitation exercise.

[원격 협동 재활 훈련 시스템] [Remote Cooperative Rehabilitation Training System]

According to a second aspect of the present invention, a rehabilitation training system is disclosed that enables finger rehabilitation training through cooperation between respective users of two or more communication terminals remotely.

Hereinafter, a remote cooperative rehabilitation training system according to a second aspect of the present invention will be described with reference to FIGS. 6 to 8 of the accompanying drawings.

6 is a view for explaining the overall remote cooperative rehabilitation training system. 7 is a diagram illustrating an application example of the remote cooperative rehabilitation training method.

6 and 15, a remote cooperative rehabilitation training system is a device for assisting a finger rehabilitation exercise of a first user (eg, a rehabilitation patient) who wants to perform finger rehabilitation training (hereinafter, referred to as a first finger). A rehabilitation exercise device) and a communication server 300b.

In addition, the remote cooperative rehabilitation training system is a second user side device (hereinafter referred to as a second finger rehabilitation exercise device) for cooperating with the first user to assist or perform rehabilitation training together with the finger of the first user. ) May be included.

Here, as shown in FIG. 6, the first finger rehabilitation exercise device may include a rehabilitation exercise glove 1 (100c) and a communication terminal 1 (200c).

The second finger rehabilitation exercise device may also include a rehabilitation exercise glove 1 (100d) and a communication terminal 2 (200d), as shown in FIG.

In addition, the communication server 300b performs data communication with the communication terminal 1 200c and the communication terminal 2200d, thereby rehabilitating the first user by remotely communicating with the communication terminal 1 200c. It serves to relay data communication between the communication terminal 2 (200d) by the second user cooperating with.

In addition, although both the first user and the second user may be rehabilitation patients requiring rehabilitation training, either one of the first user and the second user may serve only as a collaborator who cooperates in another finger rehabilitation training process. It can also be done. Here, a collaborator who cooperates in the finger rehabilitation training process may include a rehabilitation therapist, a doctor, a caregiver, and the like.

In addition, although the first finger rehabilitation exercise device and the second finger rehabilitation exercise device may be configured differently according to their respective roles in the rehabilitation training process, both may have the same configuration.

For example, when the first user who uses the first finger rehabilitation exercise device is a rehabilitation patient, and the second user who uses the second finger rehabilitation exercise device is a therapist who helps rehabilitation treatment, the second finger rehabilitation exercise device may be used. 1, the rehabilitation gloves 2 (100d) may be omitted.

That is, although the therapist may assist the rehabilitation training of the first user while directly wearing the rehabilitation exercise gloves 2 (100d), since the rehabilitation training may be protected / cooperated using only the communication terminal 2 (200d). to be. For example, the first therapist, while using the rehabilitation exercise gloves 2 (100d), while watching the rehabilitation exercise auxiliary image to be described later screen sharing through his terminal (that is, communication terminal 2 (200d)), the first It can assist / cooperate the user's rehabilitation training.

More specifically, the therapist may drag and drop a rehabilitation tool object (refer to reference numeral 208 of FIG. 7) to be described later on a screen displaying a rehabilitation exercise assistance image of a first user shared together through his terminal. By performing a pointer manipulation action as described above, rehabilitation training of a first user who shares this image can be assisted / cooperated. As another example, when the therapist's virtual hand object is displayed at a specific position on the rehabilitation exercise assisted image, the therapist uses the mouse, touch manipulation, key input, and the like to position and shape the virtual hand object. If implemented to operate the back, the therapist will be able to fully assist / cooperate with the rehabilitation training of the first user even if not necessarily using the rehabilitation exercise gloves 2 (100d).

Rehabilitation gloves 1 (100c) can be made in the form of a glove that can be worn on the hand to assist the rehabilitation exercise (training) of the finger rehabilitation patient (see FIG. 7), as shown in the block diagram of FIG. The wireless communication module 12 may include a detector 14 and a controller 16. However, FIG. 6 illustrates only some of the main components, and additionally, a configuration for performing basic functions (for example, a battery) or a configuration required for adding various functions (for example, adding an EMG measurement function). In this case, the EMG sensor may be further included.

Rehabilitation exercise glove 1 (100c) is basically a variety of hand motion information (ie, finger movements, hand posture change, hand position change (movement), etc.) of the first user wearing the glove by using a wireless communication method It serves to transmit to the communication terminal 1 (200c). This is the same as the detailed content of the rehabilitation exercise gloves 100b described above with reference to FIGS. 1 and 2, and thus redundant description will be omitted.

The communication terminal 1 (200c) receives the hand gesture information of the first user transmitted from the rehabilitation exercise glove 1 (100c), the virtual hand-shaped object corresponding to the received hand gesture information of the first user (hereinafter, This is called 'first hand object' and displays it on the display screen. Accordingly, the finger rehabilitation patient can immediately visually check his or her hand movement in the process of the finger rehabilitation exercise, thereby being able to be motivated to the rehabilitation exercise and become interested in the rehabilitation exercise. . The specific configuration of the communication terminal 1 (200c) for this purpose may be the same as that of the communication terminal 200b of FIG. 3 described above, the description thereof will be omitted.

However, the contents related to the configuration and function of the rehabilitation exercise glove 1 (100c) may be applied to the rehabilitation exercise glove 2 (100d) as it is, and the contents related to the configuration and the function of the communication terminal 1 (200c) are also the communication terminal 2 (200d). Edo can be applied as it is by using only different subjects.

In addition, the rehabilitation exercise assistance process performed between the rehabilitation exercise glove 1 (100c) and the communication terminal 1 (200c) or / and between the rehabilitation exercise glove 2 (100d) and the communication terminal 2 (200d) is the same as described above in / It can be processed by a similar process, a detailed description thereof will be omitted. In addition, the contents related to the "configuration method of the rehabilitation exercise assistance screen" described through the description of the finger rehabilitation exercise apparatus and the rehabilitation exercise assistance method according to the first aspect of the present invention may be equally applied thereto.

Therefore, hereinafter, as shown in Figs. 6 and 15, both the first user and the second user use a finger rehabilitation exercise device (i.e., the first and second finger rehabilitation exercise devices) having the same configuration, The case of cooperation in liver rehabilitation training will be described.

8 is a view for explaining a rehabilitation training method according to an embodiment of the present invention by a remote cooperative rehabilitation training system. In FIG. 8, it is assumed that two rehabilitation patients perform cooperative rehabilitation training remotely for convenience of explanation.

In step S210, as the communication terminal 1 (200c) constituting the first finger rehabilitation exercise device enters the cooperative rehabilitation training mode, in step S220, the communication terminal 1 (200c) and the communication terminal 2 (200d) is a communication server 300b. Communication connection through the relay of data communication.

Here, the entry into the cooperative rehabilitation training mode is, for example, after the first user executes a rehabilitation exercise program of his or her communication terminal 1 (200c), the upper part of the rehabilitation exercise assistance screen that is driven according to the corresponding program and displayed on the screen. It may be entered by selecting a particular button for entering the cooperative rehabilitation training mode. In this case, the counterpart to cooperate in the cooperative rehabilitation training mode may be predetermined, but may be directly selected by the user. That is, the entry into the cooperative rehabilitation training mode is a communication through the relay of the communication server 300b with the requested counterpart terminal in response to a request by any one of two or more persons to be subjected to the cooperative rehabilitation training. By connection.

On the other hand, in another example, two or more patients who are to perform the cooperative rehabilitation training may execute a rehabilitation exercise program installed in the corresponding terminal in their respective terminals, thereby connecting to the communication server 300b, respectively, thereby establishing a communication connection with each other. In this case, an IP address of the communication server 300b may be stored in advance in a rehabilitation exercise program installed in each terminal. In this case, the communication server 300b may have a phone number and network connection information (for example, an IP address or a MAC address) of each terminal owned by patients, doctors and therapists capable of cooperative rehabilitation training.

In addition, when entering the cooperative rehabilitation training mode, for example, the rehabilitation exercise program (for example, can be implemented in the form of a rehabilitation training application) installed in each terminal is connected to each other, so that the rehabilitation exercise program in each terminal Accordingly, a state in which mutual rehabilitation exercise information and cooperation information for the rehabilitation exercise (training) can be exchanged with each other may be achieved.

In this state, in step S230, when the hand motion information of the rehabilitation patient 1 is received from the rehabilitation exercise glove 1 100c of the first user (hereinafter referred to as rehabilitation patient 1) to the communication terminal 1 200c. As described above, the communication terminal 1 200c generates the virtual hand object of the rehabilitation patient 1 in correspondence with the received hand motion information. The communication terminal 1 (200c) constructs a rehabilitation exercise assistance screen including the generated virtual hand object of the rehabilitation patient 1 and displays it on the display screen (see step S240 and reference numeral 206 of FIG. 7). The rehabilitation exercise assistance screen displayed up to this step is, for example, a hand-shaped object 207 (which is a hand-shaped object that virtually expresses the hand motion of the second user, ie, the rehabilitation patient 2) on the screen 205 of FIG. 7. The excluded screen may be configured.

In addition, in the cooperative rehabilitation training mode as described above, the communication terminal 1 (200c), as in step S235, the hand operation information of the rehabilitation patient 1 received from the rehabilitation exercise gloves 1 (100c) data of the communication server 300b. It transmits to the communication terminal 2 (200d) through a communication relay process. In the case of Fig. 8, step S235 is shown as preceded by step S240, but the order may be reversed and may be processed in parallel.

As described above, when the hand operation information of the rehabilitation patient 1 by the operation of the rehabilitation exercise glove 1 (100c) is transmitted from the communication terminal 1 (200c) to the communication terminal 2 (200d) by the relay of the communication server 300b. In operation S245, the communication terminal 2 200d may also generate a virtual hand object corresponding to the received hand motion information of the rehabilitation patient 1 according to the method described above, and configure the rehabilitation exercise assistance screen.

For convenience of explanation, hereinafter, the rehabilitation exercise assistance screen displayed through the display screen of the communication terminal 1 200c is referred to as the rehabilitation exercise assistance screen 1 and the rehabilitation exercise assistance displayed through the display screen of the communication terminal 2 200c. The screen will be referred to as rehabilitation exercise assistance screen 2. In the case of FIG. 7, the case where the rehabilitation exercise assistance screen 1 and the rehabilitation exercise assistance screen 2 have the same screen configuration is illustrated. In addition, the case where the hand-shaped object of the rehabilitation patient 1 and the hand-shaped object of the rehabilitation patient 2 correspond to the position and shape in each corresponding screen is illustrated. However, although the position and shape of the hand object of each patient are displayed in the same manner in both screens, the background screen may be differently displayed. In addition, the hand shape is expressed the same on each screen, the position where the hand object is displayed may be displayed differently on both screens. In addition, various modifications are possible depending on the implementation method.

Subsequently, in step S2205, when the hand motion information of the rehabilitation patient 2 is received from the rehabilitation exercise glove 2 (100d) of the second user (that is, the rehabilitation patient 2) to the communication terminal 2 (200d), the communication terminal 2 (200d) ) May transmit the hand motion information of the rehabilitation patient 2 to the communication terminal 1 200c through the relay of the communication server 300b as in step S255.

Accordingly, the communication terminal 1 (200c) further generates a hand-shaped object corresponding to the hand motion information of the rehabilitation patient 2 transmitted from the communication terminal 2 (200d) through the object generating unit 160, and thus generated The first rehabilitation exercise assistance screen is reconstructed so that the hand object of the rehabilitation patient 2 can be reflected and displayed on its first rehabilitation exercise assistance screen (see step S260).

In addition, the communication terminal 2 (200c) also generates a hand object (hereinafter referred to as a second hand object) for the rehabilitation patient 2 corresponding to the hand motion information received from the rehabilitation exercise glove 2 (100d) on its side. Then, the second rehabilitation exercise assistance screen can be reconstructed so as to be reflected in its second rehabilitation exercise assistance screen (see step S265).

A screen when the processes of steps S260 and S265 have been performed are shown in FIG. 5, where reference numeral 207 of FIG. 7 denotes the second hand object. Referring to FIG. 7, a case in which the first rehabilitation exercise assistance screen displayed on the communication terminal 1 200c and the second rehabilitation exercise assistance screen displayed on the communication terminal 2 200d have exactly the same screen is illustrated.

As described above, the first rehabilitation exercise assistance screen and the second rehabilitation exercise assistance screen may have different screen configurations, but when they have the same screen configuration, they may be more effective in cooperative rehabilitation training of each rehabilitation patient. Therefore, hereinafter, methods of synchronizing both screens will be described.

First, the synchronization of the background screen constituting each rehabilitation exercise auxiliary screen may be used as follows. For example, when the corresponding screen is configured as an augmented reality environment, the other terminal receives information regarding the augmented reality environment (for example, information about a preview image acquired through a camera from one terminal). Can be used. In addition, as another example of the method of synchronizing the background screen, the screen reconstruction information such as identification information on the background screen selected by either terminal in the rehabilitation exercise program installed in each terminal may be synchronized to each other by transmitting to the other terminal. have. Alternatively, both terminals may synchronize both by connecting to a cooperative rehabilitation training channel configured by a specific background screen in a rehabilitation exercise program.

Next, the position and shape synchronization with respect to the hand-shaped object of each rehabilitation patient to be displayed on the background screen and the virtual rehabilitation tool object used for the cooperative rehabilitation exercise are, for example, the display position with respect to the rehabilitation tool object. That is, the coordinate) and the object selection (identification) information and the coordinate information about the display position of the hand-shaped object of each rehabilitation patient can be shared with each other through the relay of the communication server 300b to synchronize. There will be.

According to this, as shown in the example of Figure 7, the rehabilitation exercise auxiliary screen displayed on both terminals can be shared the same, and each of the rehabilitation patients on the screen by taking a specific hand gesture, as if progressing a kind of game Rehabilitation training is available.

For example, in the rehabilitation exercise assistance screen of the same screen configuration, the hand-shaped object of the rehabilitation patient 1 and the hand-shaped object of the rehabilitation patient 2 are displayed overlapping each other with a specific rehabilitation tool object or more than a predetermined ratio, and the hand of the rehabilitation patient 1 When the shape object and the hand object of the rehabilitation patient 2 each have a hand shape such that a finger is bent over a predetermined angle, the hand of the rehabilitation patient 1 and the hand of the rehabilitation patient 2 cooperate to grab the rehabilitation tool object. Cooperative rehabilitation training may be possible, such as taking a fictitious action and then taking a cooperative action such as moving the rehabilitation tool object to a specific location.

In FIG. 8, a description is given of a case in which cooperative rehabilitation training is performed by exchanging hand motion information of each rehabilitation patient in both terminals, but the cooperative information shared for giving and receiving cooperative rehabilitation training may vary. Of course. For example, when the rehabilitation patient and the doctor share the above rehabilitation exercise assistance screen between the respective terminals, the doctor watches the rehabilitation training process of the rehabilitation patient through his terminal, and the rehabilitation timely necessary in the rehabilitation training process The tool may be recommended, and in this case, the communication terminal of the rehabilitation patient may generate and reconfigure the object so that the virtual object related to the rehabilitation tool is displayed on the screen.

[착용형 통신 장치를 이용한 원격 제어 시스템] [Remote control system using wearable communication device]

According to a third aspect of the invention, a remote control system using a wearable communication device is disclosed. Hereinafter, a remote control system using a wearable communication device according to a third aspect of the present invention will be described in detail with reference to FIGS. 9 to 11 of the accompanying drawings.

9 is an overview of a remote control system using a wearable communication device, and FIG. 10 is a block diagram of a device interworking agent driven on the wearable communication device and the electronic device.

Referring to FIG. 9, a remote control system according to an exemplary embodiment of the present invention may include a wearable communication device 100e and an electronic device 200e interoperating therewith.

Here, the electronic device 200e is a device including a communication module (see 1130 in FIG. 10) and a display (see 1110 in FIG. 10) for communicating with the wearable communication device 100e. If the device interworking agent 1200 is to be mounted or installed, this may be the case without any limitation. Of course, as a general configuration, as shown in FIG. 2, the power supply unit 1120, the memory 1140, and the electronic device controller 1150 may be further included. However, in the present specification, for convenience and concentration of the description, the electronic device 200e will be described with reference to a smart TV as shown in FIG. 1.

The wearable communication device 100e is worn on a user's body to obtain motion sensing information generated at a corresponding wearing portion according to a user's motion, and the predetermined communication device (in this embodiment, a short range wireless communication module). If the device can transmit the acquired information to the remote electronic device 200e through (refer to 50e of FIG. 10), of course, there is no particular limitation. That is, the wearable communication device 100e will be described with reference to the communication device in the form of a glove worn on the user's hand as shown in FIG. 9, but the wearing portion is not necessarily limited to the hand. In addition, the wearable communication device 100e does not need to be worn only on any one body part of the user, and a plurality of wearable communication devices 100e are configured as one set, and are worn on the user's hands, both feet, and the like, respectively. It may work. This may be more clearly understood by the following description.

Hereinafter, the component included in the wearable communication device 100e and the component included in the device interworking agent 100e mounted or installed in the electronic device 200e will be described in more detail based on the embodiment of FIG. 10. Let's explain.

Referring to FIG. 10, the wearable communication device 100e may include

motion detection units

20e and 30e, a controller 40e, a short range wireless communication module 50e, and a battery 60e. However, the present invention is not limited only to these components, and additional components may be further included according to needs and designs. Exemplary configurations and functions of the wearable communication device 100e applied to the remote control system according to the third aspect of the present invention will be described in detail several times with reference to the drawings of FIGS. 7, 8, and 2. In the following description, redundant description will be omitted.

The device interworking agent 1200 may include a network manager 1210, an information interpreter 1220, a device interworking manager 1230, and a lookup table 1240. Of course, it may further include an additional configuration in addition to the need or design, the lookup table 1240 may be included in the memory 1140 of the electronic device (200e).

Here, the device interworking agent 1200 may be pre-loaded or installed in the electronic device 200e and driven on the electronic device 200e, but may be implemented in the form of firmware or software (application) and be a user of the electronic device 200e. You can also download and install it. For example, although not installed in advance in a smart TV, the user purchases the wearable communication device 100e according to the embodiment of the present invention separately and uses the wearable communication device 100e to use the electronic device 200e. ) And a device interworking agent 1200 suitable for the electronic device 200e to remotely control the operation of a specific application (for example, a specific game application to be described later, etc.) driven by the electronic device 200e. It can also be installed.

Accordingly, the device interworking agent 1200 is driven on the electronic device 200e to perform device interworking and control between the electronic device 200e and the wearable communication device 100e. same.

When the wearable communication device 100e is identified through the communication module 1130 mounted on the electronic device 200e, the network manager 1210 performs a network connection for exchanging data with the wearable communication device 100e. .

The information interpreter 1220 may, from the motion sensing information received from the wearable communication device 100e, user motion information to be used for controlling any one of an operation of the electronic device 100e and an application running on the electronic device 100e. Acquire. Here, the user motion information is interpreted to be meaningful information for performing the remote control according to the present invention among the motion sensing information received from the wearable communication device 100e (that is, the intention of the user among the various actions taken by the user Operation information which is confirmed to be an operation for remote control).

That is, even when the user wears a glove-shaped communication device 100e as shown in FIG. 9, there may be an unintentional movement. The information interpreter 1220 filters the meaningful information with a remote control intention. It is responsible for extracting only actions. To this end, the information interpreter 1220 may refer to a specific table (hereinafter, referred to as a first lookup table) stored in the lookup table 1240. The first lookup table may exist as, for example, the following table.

Table 1-First Lookup Table

Therefore, referring to the first lookup table according to the example of [Table 1], the information interpreter 1220 ignores the operations except the hand gestures defined in the table, and remotely controls only the hand gestures defined in the table. By determining that the user's intentional motion for the user can obtain only the user's motion information corresponding to the operation. Here, the user operation information to be transmitted to the device interworking manager 1230, which will be described later, may be operation identification information (ie, # 1) of the table.

However, the first lookup table is merely an example, and more various operations may be defined as meaningful user motion information. As will be described below, if a game application such as "tennis game" or "table tennis game" is being executed on the smart TV screen, a virtual player displayed on the game execution screen and identified with a user who enjoys the game. The user behavior corresponding to the (virtual player) swinging the racket can be defined through the lookup table above. Or, in the case of a first-person shooter, a hand motion (ie, bending the index finger or bending the index finger at the same time, moving the thumb, etc.) corresponding to the user's triggering of the gun is significant. It may be defined as information. In addition, there may be a wide variety of hand gesture combinations, such as two or more finger combinations, predetermined hand gesture inputs, and a variety of application examples based on the object to be controlled, the characteristics of the execution application, and the properties of the virtual object to be displayed in the execution application. Will be readily understood by those skilled in the art. That is, the first lookup table may be defined differently according to the control target, the type and characteristic of the application, the virtual object, and the like.

As described above, when user operation information, which is a significant operation for remote control, is acquired through the information analyzer 1220, the device interworking manager 1230 may perform an operation of the electronic device and an application running on the electronic device in response to the user operation information. It serves to control the operation and so on.

Here, the device interworking manager 1230 may refer to a specific table stored in the lookup table 1240 (hereinafter referred to as a second lookup table) for the same control as described above. An example second lookup table corresponding to the previous example may be as follows.

Table 2-Second Lookup Table

However, it is obvious that the second lookup table of the above example may be defined differently according to the control target, the type and characteristic of the application, the virtual object, and the like. In other words, the second lookup table is merely an example in which user motion information and application motion control are mapped one-to-one assuming a first-person shooting game. Therefore, if it is utilized to control the motion of the electronic device itself, user motion information # 7 (move right hand) is channel up, # 8 (hand move left) is channel down, # 9 (move hand down) Volume down, # 10 (move up) may be mapped, such as volume up. In addition, when the execution application is a document writing application (for example, notepad, WORD, etc.), the above user action information may correspond to each "Character". In the case of a CALL application, each "number" May be the equivalent of ".

That is, the device interworking manager 1230 may perform different control operations according to the object of device control. For example, in the case of controlling the operation of the electronic device, the function of each device (audio volume, channel selection, screen switching, etc.) of the electronic device may be controlled based on the user operation information and the second lookup table. In the case of controlling the execution operation of the letter / number input application, the letter / number input may be performed through a virtual keyboard displayed on the screen. As another example, a user action may be performed to select icons displayed on the home screen. The pointer displayed in the application screen running on the electronic device may be controlled to follow the information. In addition, as will be described below, an operation corresponding to the user's motion information may be controlled to be reflected in the virtual object (see "Hand-shaped virtual object" in FIG. 5 described above) displayed in the running application screen.

11 is a flowchart briefly illustrating a remote control method according to interworking between a wearable communication device and a device interworking agent.

Referring to FIG. 11, when the wearable communication device 100e is turned on [step S12], when a connection request message is transmitted from the wearable communication device 100e [step S14], the network of the device interworking agent 1200 is connected. The manager 1210 may identify the wearable communication device 100e (step S16). If so identified, the network manager 1210 performs a network connection with the wearable communication device 100e (step S18).

As such, when the wearable communication device 100e is identified, the device interworking manager 1220 may transfer control authority to at least one of an operation of the electronic device and an application running on the electronic device to the wearable communication device 100e. Switch (step S20), and instruct (request) the device list or the application list controllable by the wearable communication device 100e to be displayed on the screen of the electronic device 200e to the GUI (Graphic User Interface) [step] S22]. Such an instruction (request) is transmitted to the controller 1150 of the electronic device 200e, so that the electronic device controller 1150 generates a corresponding GUI and expresses it on the display screen.

Accordingly, the user of the wearable communication device 100e may visually check the objects that can be controlled through the screen of the electronic device 200e. Subsequently, when motion sensing information for selecting a control target is transmitted from the wearable communication device 100e according to the user selection (that is, the user takes an action of instructing such selection) [step S24], the device interworking agent 1200 ) May obtain user motion information related thereto through the information analyzer 1220. Based on the acquired information, the device interlocking manager 1230 performs setting for device control as intended by the user and completes the setting (step S26).

As described above, after the setting process for controlling the device is completed, if the motion sensing information for controlling the selection target is transmitted from the wearable communication device 100e (step S28), the device interworking manager 1230 is an information analyzer. The control unit 1150 of the electronic device 200e is instructed (requested) to perform a corresponding control operation according to the user operation information acquired by the 1220 (step S32).

Of course, in the above, when the wearable communication device 100e is identified, a process for selecting a control target to be remotely controlled by this (that is, receiving motion sensing information regarding a GUI display process in step S22 and a selection operation in step S24). ) Are described as prior, but of course, this step may be omitted as needed or according to the implementation of the invention. For example, if the wearable communication device 100e is set to have authority only for operation control of a specific application, the above step may be omitted. In this case, the device interworking manager 1230 may perform the setting for device control by automatically executing the specific application related to the wearable communication device 100e.

Here, the setting process for controlling the device through the device interworking manager 1230 according to step S26 may further include the following process. For example, if it is assumed that a specific application is running in the electronic device 200e, a specific control element (see "Hand virtual object" in FIG. 5 described above) among the elements displayed in the application execution screen is described. The process of converting the control right to the wearable communication device 100e may be further included.

In addition, the GUI presentation process according to the above-described step S22 may further include the following process according to the situation. For example, assuming that there are a plurality of identified wearable communication devices, it may be necessary to identify in which manner the plurality of wearable communication devices will be used by whom. Accordingly, in the above case, the device interworking manager 1230, for each of the identified wearable communication devices, may include a first selection item for selecting a user who will use the device and a second part for selecting a wearing part of the device. The GUI including at least one of the selection items may be displayed on the screen of the electronic device.

According to the above-described process, if the plurality of wearable communication devices are assigned to one user with different wearing parts, the device interworking manager 1230 may display the virtual displayed in the application screen running on the electronic device. The control authority for each control element of the object may correspond to the plurality of wearable communication devices one by one. In this case, the device interworking manager 1230 distinguishes the user operation information obtained by the information analyzing unit 1220 based on the identification information of the plurality of wearable communication devices, thereby allowing the virtual object to be classified by the control element. It can also be controlled independently.

This can be applied to the following cases. If the user wishes to remotely control the virtual player displayed through the "tennis game" screen using the wearable communication device 100e, the user wears one wearable communication device 100e on the right hand to perform operation / control. Although it is possible to do this, one may want to operate by wearing a plurality of wearable communication devices. That is, in addition to one wearable communication device that controls the racket held by the right hand of the virtual player, the wearer may further wear the wearable communication device on both ankles, thereby controlling the left / right movement of the virtual player. In this case, motion sensing information received from each of the set of wearable communication devices needs to be independently controlled in correspondence with each control element (i.e. racket, player's left / right movement) regarding the virtual player. For these cases, the above control can be useful.

In addition, other control schemes may be utilized. This is described as follows. If the plurality of wearable communication devices are assigned to a single user by different wearing parts, two or more wearable communication devices may be provided in one control element due to a property of a virtual object displayed in a running application screen. There may be times when groups need to be set up. For example, a handle as a virtual object in a "car racing" game may correspond to this control element. In this case, the device interworking manager 1230 may control the control element corresponding to the combined operation of the user motion information of each of the grouped wearable communication devices (for example, devices worn on the right hand and the left hand, respectively). In the example, the handle) can be controlled. That is, in this case, the user may remotely control the manipulation of the virtual handle displayed on the game execution screen by performing an operation such as operating a steering wheel using a communication device worn on both hands.

As another example, a plurality of wearable communication devices may be used differently for each user. That is, two or more users (i.e. game players) each want to remotely control a virtual object (e.g. a car race or a boxing match) assigned to them with their own communication device. can do. In this case, the device interworking manager 1230 corresponds to the control authority for each virtual object expressed in the running application screen for each user, and the user action information acquired by the information analyzer 1220 is used for the plurality of wearables. By distinguishing based on each identification of the communication device, the virtual object can be controlled independently for each user.

[훈련 일정 최적화 방법][How to optimize training schedule]

According to a fourth aspect of the present invention, as an invention related to exercise science and brain science technology, a training schedule optimization method using a brain science exercise learning model and an optimal control theory when training a single or multiple exercise tasks is disclosed. The training schedule optimization method to be described below may be programmed by a user terminal (not shown) such as a portable terminal, a PC, or a training schedule management server (not shown) connected to the user terminal and implemented by one or more processors. have.

Hereinafter, a training schedule optimization method according to a fourth aspect of the present invention will be described in detail with reference to FIG. 12, but prior to this, the brain science exercise learning model and the optimal control theory will be briefly introduced. Shall be.

Computational neuroscientific model of learning

Recently, motor learning has been achieved by learning modules with multiple different processing speeds inside the brain, and it has been found that the externally observable learning level is the sum of the learning levels of the modules (Smith et al., 2006 PLoS Biol, Kording et al., 2007 Nat Neuro).

In addition, when training an exercise task, the long-term learning level of the exercise task (measured after 24 hours after completing the training) has no correlation with the learning level observed during the exercise task training or the learning level of the fast learning module. Instead, it has been found that there is a significant positive correlation with the learning level of the slow learning module (Joiner and Smith, 2008 J Neuro Physiol).

In particular, when training multiple motor tasks, a neuroscience motor learning model has been proposed in which a single fast learning module and a plurality of slow learning processes learn simultaneously (Lee and Schweighofer, 2009, J Neurosci). Verified by other groups of clinical trials (Pekny et al., 2010 SfN).

Optimal control theory

The optimal control theory is a theory that finds an array of manipulated variables that optimizes a specific evaluation function on a target system expressed as a function of status and control. Dynamic programming and the Fontryagin's maximum principle are two ways to find an array of manipulated values that optimize the evaluation function (Kirk 1970, Todorov 2007).

Hereinafter, the general content of the present invention will be described.

The present invention provides a method of finding a learning schedule that maximizes long-term learning levels when training single or multiple athletic tasks.

Long-term learning level refers to the learning level of an exercise task measured after a certain period of time (typically 24 hours or more) after completing the exercise task training.

When training multiple exercise tasks, maximizing long-term learning means optimizing a specific assessment function expressed as a function of the learning levels of the individual exercise tasks.

A feature of the present invention is to propose a method for finding a theoretically proven optimal learning schedule that maximizes long-term learning by applying a brain science exercise learning model and an optimal control theory to exercise training.

In order to apply the brain science exercise learning model, the present invention measures and normalizes the initial learning level of individual exercise tasks (see S400 of FIG. 12). Initial learning level measurements are typically performed before training, but can be included in the training process.

When measuring the initial learning level before training, the individual exercise task is repeated a certain number of times before training, and then the average of the obtained values is averaged.

When including the initial learning level measurement in the training process, the measured value is inserted by randomly inserting the learning level measurement schedule in the middle of the training schedule.

In all cases, the initial learning level measurement does not give the trainer feedback on the learning level to prevent learning from occurring during the measurement and inaccurate measurements.

The initial learning level value measured through the measurement process is normalized to the maximum possible learning level of the individual exercise task.

In addition, the present invention predicts the parameters of the brain science exercise learning model for the individual exercise task in order to apply the brain science exercise learning model (see S420 of FIG. 12).

Motor learning model parameters for individual exercise tasks are typically predicted using training data from other trainer groups, but may be predicted in real time from training data of individual trainers.

When predicting the exercise learning model parameters using training data of other trainer groups, the exercise learning model parameters that best reproduce the training data of other trainer groups of individual exercise tasks may be the maximum likelihood method or the like. Obtained by regression analysis such as least square method.

In real-time prediction of motor learning model parameters from individual trainer's training data, real-time prediction methods such as non-linear Kalman filter are used to estimate the maximum posterior probability (from the trainee's learning level accumulated at each moment during training). Obtain the motor learning model parameters that guarantee the maximum posterior probability.

The present invention expresses an object system using an exercise learning model, an exercise learning model parameter of an individual exercise task, a trainee's learning level for an individual exercise task, and a training schedule to apply an optimal control theory (see S440 of FIG. 12). ).

The training level of the trainer for the individual exercise task is a state quantity, and the initial learning level value measured for the individual exercise task is used as an initial value.

The training schedule uses a value obtained by converting an exercise task presented at each time point as a manipulated variable into a mathematical vector.

The exercise learning model is an object system formula, and the parameter is a value predicted in the parameter prediction process of the exercise learning model.

The evaluation function uses the criteria that the trainer wants to achieve as a function of the state quantity and the manipulated quantity. For example, an evaluation function may be used to minimize the long-term mean-square error of multiple exercise tasks or to maximize the long-term mean-square performance.

The present invention uses an optimal control theory method, such as dynamic programming and the maximum principle of Fontriyagin, to find a training schedule that optimizes the evaluation function (see S460 in FIG. 12).

Depending on the parameter prediction method of the motor learning model, the optimal control theory can be used to find the optimal training schedule before training or in real time during training.

In case of predicting the motor learning model parameters using the training data of other trainer groups, the optimal training schedule is found using the optimal control theory before the training using the predicted parameters.

When predicting an exercise learning model parameter in real time from individual trainer's training data, the parameter value can be changed. Therefore, it is necessary to find the optimal training schedule using the optimal control theory at each time point using the predicted parameter.

In current exercise learning, it is common to select a training schedule based on the training level observed by the trainer or the trainer himself. However, recent neuroscience studies have shown that the level of training observed during training has no correlation with long-term learning (Joiner and Smith 2008, J Neuro Physiol). Accordingly, the present invention converts the problem of selecting a training schedule in exercise learning into a formula for applying control theory using a brain science exercise learning model, and then theoretically optimizes long-term learning level by applying an optimal control theory. Show how to find a training schedule.

Hereinafter, the following examples will be described in order to help a concrete understanding of the present invention. It should be noted, however, that the present invention is not limited to the detailed methods, models, numerical values, and procedures mentioned in the following detailed examples.

In one embodiment, the present invention can be used to search for an optimal training schedule in tennis training. To learn tennis, the trainer must learn two separate exercise tasks: the forehand stroke and the backhand stroke. In the present embodiment, a training schedule for optimizing the learning level of two individual exercise tasks after training by optimally combining the forehand stroke and the backhand stroke in tennis training. Training is assumed to be performed in regular tennis code. The present invention measures the trainee's learning level on an individual exercise task before training. To determine the level of training, the trainer performs a random forehand stroke and backhand stroke five times each. Make sure to close your eyes after the stroke so that the trainer does not receive visual feedback on the level of learning during the measurement. The learning level for each stroke is expressed as a function of the geometric distance from the designated target point on the opposite side of the court to the ball's falling point after the stroke:

Learning level = (maximum geometric distance-geometric distance) / maximum geometric distance

The maximum geometry is the geometry between the learner and the target. By expressing the learning level as the function, the effective range of the learning level is 0 to 1, so that no normalization is required.

As a neuroscience exercise model, a parallel single fast multiple 1-fast N-slow model can be used. The model equation is:

xf (t + 1) = Af (t) * xf (t) + Bf (t) * e (t)

xs (t + 1) = As (t) * xs (t) + Bs (t) * e (t) * c (t)

y (t + 1) = xf (t + 1) + xs (t + 1) '* c (t + 1)

e (t + 1) = f (t + 1)-y (t + 1)

xf (t) is the state amount of the fast learning process at time t, Af is the forgetting speed parameter of the fast learning process, and Bf is the learning speed parameter of the fast learning process. xs (t) is a state vector of the slow learning process state value at time t, and is a mathematical vector of the slow learning process state values of the forehand stroke and the backhand stroke.

xs (t) = (xs_1 (t), xs_2 (t)) '

As is the forgetting speed parameter of the slow learning process, Bs is the learning speed parameter of the slow learning process, e (t) is the error learning level at time t, and c (t) is the manipulated value at time t. Use the value converted from the exercise task to the equation vector. C (t) = c_1 = (1, 0) 'for a forehand stroke and c (t) = c_2 = (0, 1)' for a backhand stroke. y (t) is the learning level observed at time t, and f (t) is the ideal learning level for the motor task presented at time t.

In the present embodiment, the training learning model parameters Af_1, Bf_1, As_1, Bs_1, Af_2, Bf_2, As_2, and Bs_2 are predicted using the training data of another trainer group. Two other groups of trainers train the forehand stroke and the backhand stroke, respectively, and then use least squares regression to find the motor learning model parameters that best represent the training data for each group. In other words,

e = sigma_ {i = 1} ^ {N} (y (i)-y_hat (i)) ^ 2

Find the parameter that minimizes the value. y (i) is the actual training data at time i, y_hat (i) is the prediction obtained from the model equation at time i, and N is the total number of training.

In this embodiment, after obtaining the parameters of the exercise learning model, in order to find the optimal training schedule, the maximum principle of the fontriagin, which is one of the optimal control theory methods, may be used. The expression function uses the following function to minimize long-term mean square error.

J = (f_1-xs_1 (N)) ^ 2 + (f_2-xs_2 (N)) ^ 2

f_1 is the abnormal learning level for the forehand stroke, f_2 is the abnormal learning level for the backhand stroke, xs_1 (N) is the slow learning level for the forehand stroke at time N, and xs_2 (N) is the backhand stroke at time N. For slow learning levels. N is the total number of training. In the present embodiment, using the maximum principle of the fontriyagin, the following optimal training schedule selection criteria is shown.

(t)>

(t) if c (t) = c_1, i.e. forehand stroke

(t)>

(t) if c (t) = c_2, i.e. backhand stroke

(t) ==

(t) then c (t) is randomly determined between c_1 and c_2

In the above formula, H_1, H_2 and related variables are defined as follows.

That is, at every time t

and

By comparing the values of

If it's large, we train forehand strokes,

If it's large, we train the backhand stroke

and

When is equal, training one of the two randomly will result in tennis training that minimizes the long-term mean square error, that is, optimizes the long-term learning level of the forehand and backhand strokes.

Although the above has been described with reference to embodiments of the present invention, those skilled in the art may variously modify the present invention without departing from the spirit and scope of the present invention as set forth in the claims below. And can be changed easily.

Claims

A device for finger rehabilitation exercises,

A wireless communication module; A detector configured to detect hand motion information according to at least one of a finger movement, a change in posture of a hand, and a change in position of a hand; A rehabilitation exercise glove including a control unit for controlling the user's hand motion information detected by the detection unit to be transmitted to a communication terminal located remotely through the wireless communication module, and manufactured in the form of a glove;

Communication module; An object generator configured to generate a virtual hand-shaped object corresponding to the received hand gesture information of the user when the hand gesture information is received from the rehabilitation exercise glove by the communication module; Finger rehabilitation exercise apparatus comprising a communication terminal including a control unit for controlling to display the hand-shaped object generated by the object generating unit via a display screen.
The method of claim 1,

The rehabilitation exercise program is installed in the communication terminal,

The rehabilitation exercise program,

Information on a rehabilitation exercise sequence defining the order of the finger rehabilitation exercise, information on each rehabilitation operation required for the finger rehabilitation exercise, motion guide information for guiding the rehabilitation operation, and a rehabilitation tool to be used for the finger rehabilitation exercise Finger rehabilitation exercise device comprising at least one of the information relating to.
The method of claim 2,

The object generator further generates a virtual demonstration hand-shaped object corresponding to the rehabilitation operation,

The control unit, the finger rehabilitation exercise apparatus for displaying the demonstration hand-shaped object generated by the object generating unit via the display screen.
The method of claim 2,

When a rehabilitation tool to be used for the finger rehabilitation exercise is determined according to the rehabilitation exercise program,

The object generating unit generates a virtual rehabilitation tool object corresponding to the rehabilitation tool,

The control unit, finger rehabilitation exercise apparatus for displaying the rehabilitation tool object generated by the object generating unit via the display screen.
The method of claim 4, wherein

The object generation unit continuously generates and updates the hand-shaped object whenever the user's hand motion information is received from the rehabilitation exercise gloves,

The controller, in response to the hand object is updated, the finger rehabilitation exercise device to reflect the image of the updated hand object on the display screen in real time.
The method of claim 5,

The control unit,

An area where the hand object is displayed on the display screen and an area where the rehabilitation tool object is displayed overlap with each other by a predetermined ratio or more, and the user's finger is a predetermined angle based on the received user's hand motion information. If it is determined that the bent above, it is determined that the user has taken a virtual motion of holding the virtual rehabilitation tool object by hand,

If it is determined that a change has occurred in the position of the user's hand according to the user's hand motion information which is subsequently received while the virtual operation of holding the rehabilitation tool object by hand is performed, the rehabilitation tool object is displayed on the display screen. To move the position corresponding to the change in the position of the hand-shaped object

Finger rehabilitation exercise device, characterized in that.
The method of claim 5,

The control unit,

In a state where an area overlapping the area where the hand-shaped object is displayed on the display screen and an area where the rehabilitation tool object is displayed, a movement change occurs in the finger of the user according to the hand motion information of the user that is subsequently received. Occation,

The shape of the rehabilitation tool object is deformed on the display screen in accordance with the degree of overlap of the hand-shaped object and the rehabilitation tool object.

Finger rehabilitation exercise device, characterized in that.
The method of claim 5,

The communication terminal further includes a camera module,

The controller may be configured to display the hand-shaped object and the rehabilitation tool object by using the preview image acquired by the camera module as a background image, thereby implementing an augmented reality environment related to the rehabilitation exercise.

Finger rehabilitation exercise device, characterized in that.
In the finger rehabilitation exercise assisting method using a communication terminal in communication with the rehabilitation exercise glove including a sensing means for detecting the hand gesture information of the user,

(a) receiving the hand motion information of the user from the rehabilitation exercise glove, wherein the hand motion information includes at least one of finger movement, hand posture change, and hand position change;

(b) generating a virtual hand object corresponding to the received hand gesture information;

(c) displaying the generated virtual hand object on a display screen; And

(d) repeating steps (b) and (c) each time the hand motion information of the user is received from the rehabilitation exercise gloves;

Finger rehabilitation exercise assistance method comprising a.
The method of claim 9,

Creating a virtual rehabilitation tool object to be used for the finger rehabilitation exercise;

Displaying the generated rehabilitation tool object on the display screen;

An area where the hand-shaped object is displayed on the display screen and an area where the rehabilitation tool object is displayed are overlapped by a predetermined ratio or more, and the user's finger is a predetermined angle based on the received user's hand motion information. Determining that the user has taken a virtual motion of grasping the virtual rehabilitation tool object by hand when the user is bent; And

In the state where a virtual motion of holding the rehabilitation tool object by hand is performed, when the position of the user's hand is changed according to the user's hand motion information which is subsequently received, the rehabilitation tool object is shaped like the hand on the display screen. And moving the position corresponding to the change in the position of the object.
The method of claim 9,

Generating a virtual demonstration hand object corresponding to a rehabilitation operation defined according to a rehabilitation exercise program; And

The method may further include displaying the generated demonstration hand object on the display screen.

The demonstration hand object is displayed on a screen area that does not overlap with the user's hand object on the display screen, or overlaps with the user's hand object and is displayed in a state that can be distinguished from each other.

Finger rehabilitation exercise assistance method, characterized in that.
A recording medium on which a program relating to the method for assisting finger rehabilitation exercise of any of claims 9 to 11 is recorded and readable by a computer.
As a remote cooperative rehabilitation training system,

A first finger rehabilitation exercise device including a first rehabilitation exercise glove and a first communication terminal and used for a finger rehabilitation exercise of a first user; And

A communication server for performing data communication with the first communication terminal and relaying data communication between a second communication terminal by a second user located remotely from the first communication terminal and cooperating with a rehabilitation exercise of the first user; Including,

The first rehabilitation exercise gloves,

A first wireless communication module; A first detector configured to detect hand motion information according to at least one of a movement of a finger of the first user, a change in posture of a hand, and a change in position of a hand; It includes a control unit for transmitting the hand gesture information detected by the first detection unit to the first communication terminal through the first wireless communication module,

The first communication terminal,

A first communication module; Create a virtual first hand-shaped object corresponding to the hand motion information of the first user received from the first rehabilitation exercise gloves, and rehabilitation of the first user from the second communication terminal by relaying the communication server A first object generation unit generating a virtual cooperation object corresponding to the received rehabilitation exercise cooperation information when the cooperation information regarding the exercise is received; And a controller configured to display, on a display screen, a first rehabilitation exercise assistance image reflecting the first hand-shaped object and the virtual cooperative object generated by the first object generator.
The method of claim 13,

Further comprising a second finger rehabilitation exercise device used for the finger rehabilitation exercise of the second user, wherein the second finger rehabilitation exercise device,

A second wireless communication module; A second sensor configured to detect hand motion information according to at least one of a movement of a finger of the second user, a change in posture of a hand, and a change in position of a hand; A second rehabilitation exercise glove including a control unit for transmitting the hand gesture information detected by the second detection unit to the second communication terminal through the second wireless communication module, and the second communication terminal,

The second communication terminal,

A second communication module; A second object generator configured to generate a virtual second hand-shaped object corresponding to the hand gesture information of the second user received from the second rehabilitation exercise glove; And a control unit for transmitting the received hand gesture information of the second user to the communication server through the second communication module and displaying the generated second hand object on a display screen. .
The method of claim 14,

The communication server relays communication so that hand gesture information of the second user received from the second communication terminal is transmitted to the first communication terminal,

When the first communication terminal receives the hand motion information of the second user transmitted from the second communication terminal from the communication server, the first object generation unit is a virtual corresponding to the hand motion information of the second user. A remote cooperative rehabilitation training system, characterized by generating a hand-shaped object as the virtual cooperative object.
The method of claim 14,

The first communication terminal transmits the received hand gesture information of the first user to the second communication terminal through the relay of the communication server,

When the second communication terminal receives the hand gesture information of the first user transmitted from the first communication terminal from the communication server, the second communication terminal is connected to the first user through the second object generation unit. A virtual hand object corresponding to the hand gesture information is generated, and a second rehabilitation exercise auxiliary image reflecting the generated hand object of the first user and the second hand object of the second user is displayed on the display screen. Remote cooperative rehabilitation training system, characterized in that the display.
The method of claim 16,

And a first rehabilitation exercise assistance image generated by the first communication terminal and a second rehabilitation exercise assistance image generated by the second communication terminal have the same screen configuration.
The method of claim 17,

The first communication terminal and the second communication terminal so that the rehabilitation exercise auxiliary image displayed on the screen in each terminal may have the same screen configuration,

At least one of the first communication terminal and the second communication terminal, the screen configuration confirmation information associated with the rehabilitation exercise auxiliary image displayed on the screen of its own terminal, to the other terminal through the relay of the communication server Remote cooperative rehabilitation training system, characterized in that for transmitting.
The method of claim 18,

In the first rehabilitation exercise assistance image and the second rehabilitation exercise assistance image, each of the display positions of the hand-shaped objects related to the hand motion information of the first user coincides with each other, and the hand is associated with the hand motion information of the second user. To ensure that each of the shape objects' display positions match each other,

The first communication terminal transmits the coordinate information where the first hand-shaped object is located in the first rehabilitation exercise assistance image to the second communication terminal through the communication server,

And the second communication terminal transmits coordinate information on which the second hand-shaped object is located in the second rehabilitation exercise assistance image to the first communication terminal through the communication server.
The method of claim 19,

The screen configuration confirmation information, including the position and the tool identification information for the virtual rehabilitation tool object displayed on the rehabilitation exercise auxiliary image,

The first communication terminal,

In the first rehabilitation exercise assistance image, based on the display position and hand shape of each of the first hand-shaped object and the hand-shaped object with respect to the second user, and the display position of the rehabilitation tool object,

The first hand-shaped object and the hand-shaped object of the second user are overlapped and displayed with the rehabilitation tool object, respectively, by a predetermined ratio or more, and the hand-shaped object of the first hand-shaped object and the second user is displayed. When each of the hand has a hand shape such as bent fingers over a predetermined angle, it is determined that the hand of the first user and the hand of the second user for the virtual operation of holding the rehabilitation tool object in cooperation Remote cooperative rehabilitation training system.
The method of claim 13,

The first communication terminal further includes a camera module,

And a control unit of the first communication terminal implements an augmented reality environment in the rehabilitation exercise assistance image by using the preview image acquired by the camera module as a background image.
The method of claim 21,

The first communication terminal transmits information about the augmented reality environment to the second communication terminal through the communication server so that the augmented reality environment can be shared through a second rehabilitation exercise assistance image of the second communication terminal. Remote cooperative rehabilitation training system, characterized in that for transmitting.
A remote control system using a wearable communication device,

A wearable communication device which is worn on a user's body and transmits motion sensing information generated at a wearing part to a remote electronic device;

The electronic device is driven by the electronic device, performs a network connection with the wearable communication device, obtains user motion information from the motion sensing information received from the wearable communication device, and corresponds to the obtained user motion information. A remote control system using a wearable communication device comprising a device interlocking agent, controlling any one of an operation and an application running on the electronic device.
The method of claim 23, wherein

The wearable communication device,

A near field communication module;

A battery for supplying power;

A motion detector including at least one sensor for detecting the motion sensing information about at least one of a movement, a posture change, and a position change of the worn portion;

And a control unit which collects the motion sensing information output from the motion detection unit and transmits the motion sensing information to the networked device agent through the short range wireless communication module.
The method of claim 23, wherein

The device interworking agent,

A network manager for performing a network connection for exchanging data with the wearable communication device when the wearable communication device is identified through the communication module mounted on the electronic device;

Based on the motion sensing information received from the wearable communication device, obtaining the user motion information that is interpreted as a meaningful motion to be used for control of any one of the operation of the electronic device and an application running on the electronic device. Information analyzer; And

A device interworking manager for instructing control of any one of an operation of the electronic device and an application running in the electronic device according to the user motion information obtained by the information analysis unit; Control system.
The method of claim 25,

The device interworking manager,

Based on the user operation information obtained by the information analysis unit,

Instruct the control operation mapped to the user operation information to be executed in the electronic device or the application running on the electronic device, or to input data mapped to the user operation information into a data input window in the application running on the electronic device. A virtual object instructed to move the pointer displayed in the application screen running on the electronic device in response to the user motion information or in the application screen running on the electronic device in response to the user motion information. To be reflected in,

Remote control system using wearable communication device.
The method of claim 25,

The device interworking manager,

When the wearable communication device is identified, the control authority for at least one of the operation of the electronic device and the application running on the electronic device is switched to the wearable communication device, the device controllable by the wearable communication device A remote control system using a wearable communication device for causing a list or an application list to be displayed on a GUI (Graphic User Interface) through a screen of the electronic device.
The method of claim 25,

The device interworking manager,

When the wearable communication device is identified, a designated application is automatically executed in the electronic device, and the control authority of a predetermined virtual object to be displayed on the execution screen of the designated application is switched to the wearable communication device.

Remote control system using wearable communication device.
The method of claim 25,

If there are a plurality of identified wearable communication devices, the device interworking manager,

For each of the identified wearable communication devices, a GUI including at least one of a first selection item for selecting a user who will use the device and a second selection item for selecting a wearing part of the device may be included. Remote control system using a wearable communication device to be displayed on the screen.
The method of claim 29,

When the plurality of wearable communication devices are allocated to one user by different wearing parts, the device interworking manager may include:

The control authority for each control element of the virtual object displayed in the application screen running in the electronic device corresponds to the plurality of wearable communication devices one by one, and the user operation information acquired by the information analysis unit is used in the plurality of wearable items. By distinguishing based on each identification of the communication device, so that the virtual object can be controlled independently for each control element, a remote control system using a wearable communication device.
The method of claim 29,

The plurality of wearable communication devices are allocated to a single user by different wearing parts, and two or more wearable communication devices are provided to one control element on the property of a virtual object displayed in an application screen running on the electronic device. If the device should be grouped,

The device interworking manager is a remote control system using the wearable communication device to enable the virtual object to be controlled in accordance with the combined operation of the user motion information of each of the grouped wearable communication devices.
The method of claim 29,

When the plurality of wearable communication devices are divided and allocated to two or more users, the device interworking manager may include:

The control authority for each virtual object displayed in the application screen running in the electronic device is corresponded to each user one by one, and the user operation information obtained by the information analysis unit is based on each identification information of the plurality of wearable communication devices. By distinguishing, so that the virtual object can be controlled independently for each user, a remote control system using a wearable communication device.
A method of optimizing a training schedule when training a single or multiple exercise tasks, executed by one or more processors mounted in an electronic device or server,

Obtaining an initial level of learning of an individual exercise task,

Predicting parameters of the neuroscience motor learning model for individual motor tasks,

And a training schedule for optimizing the long-term learning level based on an optimal control theory based on the learning level of the individual exercise task, the brain science exercise learning model, the parameters of the brain science exercise learning model corresponding to the individual exercise task, and the training. How to optimize your schedule.
The method of claim 33, wherein

The initial learning level is,

The training schedule optimization method, characterized in that the training level normalized to the average value of the learning level value measured in a state in which the individual exercise task is repeated a predetermined number of times before the actual training.
The method of claim 33, wherein

The initial learning level is a training schedule optimization method, characterized in that during the training schedule, the learning level normalized the average value of the learning level value measured by randomly inserting the learning level measurement schedule.
The method of claim 33, wherein

Predicting the parameter,

A training schedule optimization method comprising predicting the parameters of the exercise learning model by applying a minimum likelihood method regression method or a least square method regression method to training data of another trainee group.
The method of claim 33, wherein

Predicting the parameter,

By applying real-time prediction by a non-linear kalman filter to the trainer's training data, the mediator ensures the maximum posterior probability from the accumulated learning level in real time during the trainer's training. A training schedule optimization method comprising predicting a variable.
The method of claim 33, wherein

Finding a training schedule for optimizing the long-term learning level,

Deriving an evaluation function expressing a training criterion to be achieved by the trainer as a function of a state quantity and an manipulated quantity on a target system expressed using the exercise learning model, the predicted parameter, the initial learning level, and the training schedule. Including, training schedule optimization method.
The method of claim 38,

The step of finding a training schedule for optimizing the long-term learning level, the training schedule optimization method of finding a training schedule in which a long-term mean-square error is minimized in the evaluation function.