WO2008094067A2

WO2008094067A2 - Method for inputting information from a touch panel using a virtual stylus

Info

Publication number: WO2008094067A2
Application number: PCT/RU2007/000515
Authority: WO
Inventors: Alexander Viliamovich Kazakevich
Original assignee: Kazakevich Alexander Viliamovi
Priority date: 2006-09-28
Filing date: 2007-09-24
Publication date: 2008-08-07
Also published as: WO2008094067A3; RU2006134368A

Abstract

The inventive method for inputting information from a touch panel consists in inputting into a computer the coordinates of a specified area of the panel pointed by a finger and in displaying on the panel near the fingertip, which touches the panel or approaches it, the image of a virtual stylus in the form of a graphic symbol pointing at the specified panel area.

Description

METHOD FOR ENTERING INFORMATION FROM A TOUCH PANEL USING A VIRTUAL STYLUS

FIELD OF TECHNOLOGY

The method is intended for inputting symbolic and graphic information from the touch panel of the device without the use of an additional device - a stylus, electronic pen or mouse.

BACKGROUND OF THE INVENTION

The use of information input devices - a touch panel, a mini-keyboard, a miniature control panel - is limited by the physical capabilities of a person. In particular, the 'non-pointedness' of the finger does not allow them to confidently point to a graphic object that is inferior in size to the 'cross-section' of the finger. Moreover, paradoxically - the finger just blocks the object that it points to. To overcome this problem, as an artificial extension of the finger, an ancient tool such as a stylus is used - a pointed wand, which is driven not on the wax plate, but on the touch panel. The use of a stylus, however, means a return from the individual use of each finger (as is the case when typing from the keyboard) to the 'collective finger ¹ - several fingers holding one stylus. Of course, this entails a decrease in the rate of manual input of information.

In some cases, information is input from a compact touch panel without using a stylus - as is done, for example, in the LG KG 800 mobile phone of LG company (Korea) equipped with a touch panel with a keyboard image displayed on it. Using such a device, especially when working with one hand and interacting with the keyboard with your thumb, is associated with an increased probability of Incorrect input the matter is aggravated by the fact that the key located under the finger is not visible, and the key relief on the surface of the touch panel, of course, is absent.

Entering information without using a stylus is also provided in the virtual touch panel of the IBIZ Techlog Sogr company (USA). This device forms a touch panel on an arbitrary surface using a laser projector and optical scanning means of this surface in order to determine the position of the fingers. In this device, the keyboard characters displayed on the virtual panel, for the reason mentioned above, also cannot be significantly inferior to fingers in size.

A similar solution, but for entering handwritten text, was proposed by Sumsuпg (Japan). The device, made in the form of a mobile phone or smartphone, forms by means of a projection of the image a virtual touch field on which the user writes with ¹ stylus. The disadvantage of this product, despite the use of advanced technologies such as laser scanning and image recognition, is still the use of the same stylus.

The compact computer 'HW6515' of the company Helt-Raskard (USA) used a mechanical keyboard with miniature keys and sufficiently large gaps between them to confidently press the keys with your fingers. However, the comfort of the keyboard is bought by increasing the dimensions of the device.

It is interesting to use the touch panel in tablet computers. This type of device includes, for example, the computer 'Stуlistiс ST-5022 ^{1 of the} joint company Fujitsu-Siemens and the computer' RoverWook Ragtepeg T210 ^{1 of the} company RoverWook (RF). A product resembling a thin folder does not require any action (other than turning it on) to prepare it for work, for example, folding the lid. Thanks to this, the tablet computer is convenient for use in various situations, for example, in the classroom or in transport. The touch panel in this device provides both output and input of information from a virtual keyboards, which requires efficient use of panel resources. However, in these models, the problems mentioned above also did not find their satisfactory resolution.

Also interesting is the method of entering information used in the smartphone “iRhop” of Arrl company (USA). This product is designed to work without a stylus. When a finger touches a virtual keyboard key (significant in size), the image of the key is lengthened, remaining, due to this, visible and telling this user which key 'pressed ¹ . Such a mechanism allows you to control the result of 'pressing' virtual keys, but does not increase the reliability of the touch itself. In general, it seems that the information about the result of pressing the key appears on the panel too late.

A more compact virtual keyboard is used in the communicator ¹ HTC Toucfϊ of the company Nigh Tech Computers (Taiwan). You can work with the keyboard without a stylus by pressing the keys with your fingernail. The accuracy of pressing is not high enough, due to the roundness of the edge of the nail and the need for the last phalanx of the finger to be pressed with the nail vertically, which interferes with the perception of information on the touch panel. Pressing virtual keys with a fingertip does not give a clear result, since it is carried out blindly. The mentioned features prevent the simultaneous operation of several fingers.

The closest analogue is the information input method developed by ONE-XT (Taiwan). This method is based on the use of an individual finger stylus - a mechanical device, like a thimble fixed on the last phalanx of a finger and equipped with a spike. Each finger, equipped with an individual stylus, interacts with the touch panel independently of other fingers and allows you to accurately point them to a given section of the panel, however, the need to attach this device to each finger and remove it after use is not quite convenient SUMMARY OF THE INVENTION

The objective of the invention is to develop a method for fast, confident input of symbolic and graphic information from compact devices without the use of an additional device - a stylus or mouse.

The technical result that allows to solve this problem is to increase the accuracy of finger pointing to a given area of the touch panel;

The technical result is achieved by the fact that when entering information from the touch panel, including entering into the computing device the coordinates of a given section of the panel indicated by the finger, in accordance with the formula of the present invention, next to the end of the finger touching the panel or close to it, an image is output to the panel virtual stylus' - a graphic sign pointing to a given section of the panel.

Signs that distinguish from the closest analogue are:

- near the end of the finger touching the panel or close to it, the panel displays an image of a graphic sign - 'virtual stylus'; - The virtual stylus points to a given section of the panel.

DESCRIPTION OF THE DRAWINGS

Further, the invention is illustrated by a description of examples of specific performance and sketches, on which:

- in FIG. 1 shows a device with a touch panel and a user's hand located at the bottom of the touch panel - as is the case when using a handheld computer, with the fingers located on the touch panel, on the device’s body, and on the support next to it; - in FIG. Figure 2 shows a device with a touch panel held by a hand located on the side of the panel, as it is done when using a mobile phone with one hand. The device is made in the housing 1 (Fig. 1), equipped with a touch panel 2 with the image displayed on it, for example, a keyboard (not shown). It is assumed that for the user’s work, under item 2, an “extended” panel is available — a touch panel and an adjacent zone — a device body 1 and a support under it (not shown). In the housing 1 of the device according to paragraphs. 11, 12 three laser emitters 3 of the means for determining the position of the fingers are integrated. The fingers are resting on the “extended” touch panel or close to it, with the index 4, middle 5 and ring finger 6 located at the touch panel 2, little finger 7 above the body 1, and thumb 8 above the support. At the end of each finger, more precisely, at a certain distance from it, according to p. 1, panel 2 displays an image of a virtual stylus (hereinafter referred to as the stylus) in the form of a certain graphic symbol acting as a graphic extension of the finger. In FIG. 1, the function of graphical continuation of the index finger 4 is performed by the crosshair 9; thumb 8, middle finger 5 and little finger 7 - arrows 10-12; nameless 6 - graphic sign 13. The position (relative to the end of the finger) and the azimuthal orientation of the stylus according to claim 5 are associated with the azimuthal orientation and inclination of the last phalanx of the finger to obtain the effect that the stylus is an extension of the thumb.

Note that according to claim 12, the touch panel, like the stylus, can be virtual, formed by projecting the image onto a certain surface.

The device shown in FIG. 2, is designed to be used with one hand, while thumb 8 rests on the touch panel 2 or is close to its surface. In the housing 1 of the device according to paragraphs. 11, 12 an integrated laser emitter 3 means for determining the position of the fingers. The shape of the stylus 14 of the thumb 8 is given the appearance of a pointed finger, and the position of the stylus 14 is associated with the azimuthal orientation and inclination of the last phalanx of the thumb 8 to obtain the effect that the stylus 14 is a graphic extension of the finger. For a better perception the stylus can be rotated by the user (pos. 15) around the end of thumb 8 to the top of panel 2.

DETAILED DESCRIPTION OF THE EMBODIMENTS OF THE INVENTION

Information is entered from the touch and 'extended ¹ panels in the' non-contact 'or' contact 'mode, discussed below.

Bring your finger (Fig. 1, 2) to the surface of the 'expanded' panel ('non-contact' mode) or hold your finger on the touch panel 2 ('pin ¹ mode). The computing device, in accordance with the distinguishing feature of paragraph 1 of the claims, displays on the touch panel 2 near the end of the finger (more precisely, at a certain distance from it) the image of the stylus. Next, move your finger over the 'expanded' panel ('non-contact' mode) or slide your finger across the touch panel 2 ('contact ¹ mode) to the point where the end of the stylus points to a given section of the touch panel 2. Next, when performing one of the following actions, read (enter into the computing device) the coordinates of the end of the stylus. When using the 'non-contact' mode, the coordinates of the end of the stylus are read in accordance with item 2 when touching the 'extended' panel, i.e. the touch panel 2, or the housing 1 or the support under it. So, in the example shown in FIG. 1, index 4, middle 5 and ring finger 6 fingers touch the touch panel 2; the little finger 7 touches the housing 1, and the thumb 8 - the supports on which the housing 1 rests. For a more accurate determination of the position of the fingers relative to the touch panel 2 and outside, as well as for a more accurate registration of touches, the emitters 3 can be located on the periphery of the housing 1 - as shown in FIG. 2. When using the 'contact' mode, reading the coordinates of the end of the stylus is performed under one of the following conditions:

1. When the finger is removed from the touch panel 2 - according to paragraph 2. The finger is moved along the touch panel 2, without lifting it, until it reaches the end stylus on a given area, after which they tear off a finger from the touch panel 2.

2. In case of a short-term detachment of the finger from the touch panel 2 - according to p. 2. The finger is moved along the touch panel 2 until the end of the stylus hits a predetermined area, after which the finger is removed from the touch panel 2 for a short time and again touch the panel in the same place (without a significant deviation of the place of repeated contact from the place of separation of the finger).

3. When touching the touch panel 2 with increased force, according to paragraph 3. 4. When touching the touch panel 2 with a second finger, according to paragraph 3. In this case, it is convenient to use the middle finger 5 as the “pointing ¹ (first) finger” (Fig. .1), sliding on the touch panel 2, and as a second finger - the index finger 4 or ring finger 6, depending on which part of the touch panel 2 (respectively, right or left) is currently 'pointing '' middle finger 5.

5. With a short touch of the extended panel - according to p. 3.

6. When pressed with the other finger (of the same or second hand) on the device button provided for this.

Using the 'non-contact' mode is the most effective from the point of view of convenience and productivity, but it requires equipping the device with a means for determining the position of the fingers.

In the case of the use of the 'contact' mode, the change in the contact force can be determined according to claim 13 using the means of measuring the contact force integrated in the device, as well as by the change in the value of the contact spot. The device may include a 'learning' procedure for tapping.

The aforementioned means of measuring the touch force may be in accordance with claim 14 in the form of a load cell or a system of several, for example, three, load cells (not shown) located on the periphery of a loosely mounted touch panel, between it and the body of the device, with the summation of signals from sensors, so that the resulting signal reflects the amount of pressure on the touch panel, regardless of where tiya. According to clause 15, load cells can be supplemented with an acceleration sensor (not shown) inserted into the device to eliminate the effect of false touch of the touch panel when the device is accidentally shaken, and touch screen touch registration is performed if there is a signal from the load cells and there is no signal from the acceleration sensor.

Obviously, the means of measuring the touch force can also be made in the form of a group of load cells distributed over the surface of the touch panel.

For effective work, an initial adjustment procedure is advisable, including setting the stylus' shape '(for example, the stylus'length' - the distance between the end of the stylus and the end of the finger) and its' behavior ^* . In addition, according to clause 4, the image of the virtual stylus can be displayed voluminously on the touch panel made by a stereoscopic one, which allows to achieve greater realism of the image of the virtual stylus. According to clauses 5, 7, the stylus is drawn (shape, brightness, color), the 'length' of the stylus, its position and orientation relative to the end of the finger, as well as the functions of the stylus can change with a change in the position, nature of the movement of the finger, as well as the effort of touching the touch panel, thereby informing the user. For example, the image of the stylus can become brighter with the approximation of a finger to the panel, with increased pressure on the panel, with accelerated or rapid movement of the finger. To indicate objects located at the borders of the panel, the stylus can independently rotate (shift) around the end of the finger in the direction of the nearest border (as if attracted to it). In accordance with this rule, the stylus 11 of the middle finger 5 (Fig. 1), located in the middle of the panel 2, is not turned in any direction and its direction coincides with the direction of the last phalanx of finger 5; the stylus 9 of the index finger 4, close to the left side of the panel 2, is shifted to the left; the stylus 13 of the ring finger 6 and the stylus 12 of the little finger 7, close to the right side of the panel 2, are rotated to the right. A graphic object itself (for example, a virtual keyboard key) can also be used as a virtual stylus if it is highlighted (for example, by underlining or highlighting) when a 'pointing' finger approaches it. It seems effective combination of two methods - moving the stylus and highlighting the corresponding object - as it is customary to do when working with the mouse.

According to p. 8, pressing a virtual key in uppercase mode can be done by touching the 'extended' panel with the vertically located end of the finger. When touching the touch panel according to claim 9, it is possible to recognize such a position of the finger, or rather, its last phalanx, by the ratio of the force of touch and the size of the touch spot.

When using the 'non-contact' mode, it is possible to move the cursor without using an additional touch pad - 'track zone ¹ : touching the' extended 'panel and detaching the finger in the same place (i.e., without sliding the finger) is interpreted as pointing a virtual stylus to an object on the panel, and successively touching the 'extended' panel, sliding and tearing off a finger according to claim 10, are interpreted as the operation of moving the cursor.

Using the middle finger as the “pointing” finger, the functions corresponding to pressing the left and right mouse buttons are performed by additional (in addition to the middle finger) touching the “extended” panel, respectively, with the index and ring fingers. Further, the scrolling function - scrolling the image on the panel - is performed by simultaneously sliding two or three fingers in the same direction. And finally, the image can be rotated by sliding two fingers in opposite directions. In general, the management of graphic objects on the touch panel (reading the coordinates of the object, its displacement, rotation, and other operations) is carried out by a combination of touching and sliding one or more fingers - with or without taking into account the force of pressing the touch panel.

Efficiency and comfort of the operations considered above makes it advisable to replace mechanical and mechanical components in portable and mini-computers. keyboard on the touch panel as a flexible, customizable keyboard. The second touchpad, if necessary, can also be used as a second screen. In general, it is necessary to apply the proposed method in these non-compact products, as well as in tablet computers close to them in size, taking into account the fact that the user is forced to place his fingers on the keyboard, and not keep them on weight. To do this, according to claim 3, the device must respond to a short touch of the extended panel or touch with a second finger or to the touch of the touch panel with increased force.

Since the proposed technology makes it possible to work with graphic information displayed on a small scale, to facilitate the perception of the image, the virtual stylus according to claim 6 can be given the auxiliary function of the virtual magnifier 16 (Fig. 1). A magnifying glass can automatically appear when the end of the virtual stylus is transferred to a specific area of panel 2, for example, to the keyboard, and disappear with the end of the virtual stylus transferred to the text area. There are other possible modes of functioning of this tool, for example, a magnifying glass can appear when the finger moves (stops) or “pulsate” (appear and disappear).

The input into a small-sized device of a large amount of information is conveniently performed by placing the fingers on the support in front of the device (in Fig. 1, thumb 8 is located). In this case, it is advisable according to claim 5 to vary the length of the stylus of each finger in such a way as to maintain an increased scale of the movement of the finger along the support in comparison with the movement of the end of the stylus. Since the thumb has greater mobility in the direction along the body of the device, while the remaining fingers in the opposite direction, for each of the two directions of each finger can be set its own scale of movement. This approach allows us to reduce the psychophysiological stress when entering information and the probability of incorrect 'pressing ¹ . Thus, thanks to virtual stylus technology:

- the finger is endowed with the accuracy of technical devices such as a 'real' stylus or mouse, because it allows you to point with a virtual stylus to a graphic object that is repeatedly inferior to a finger in size; - expanded the space of the touch panel due to the adjacent zone;

- if the touch panel is stereoscopic, the image of the virtual stylus can be voluminous;

- high-speed input of information from a compact touch keyboard (including 'extended') by several fingers of one or both hands is possible with slight movement of fingers and practically motionless palms;

- the cursor is controlled without using the 'track zone': the cursor is moved by sliding your finger across the 'extended' panel, and 'click' on the virtual keys, touching the 'extended' panel and lifting your fingers away from it without sliding;

- the process of entering graphic information is simplified: moving your finger across the 'advanced' panel, enter handwritten text or your own signature, draw a curve, draw, that is, use your finger as a fountain pen or pencil;

- a reliable (with a small number of errors) input of information is ensured and the psychophysical load on the user is reduced, due to the fact that the input is not performed “blindly”: fingers do not block the graphic objects that are pointed to.

INDUSTRIAL APPLICABILITY

The proposed method can find application in products for various purposes - mobile phones; handheld, laptop, and tablet computers; in control panels for household appliances and other devices.

Claims

CLAIM

1. A method of inputting information from a touch panel, including entering into a computing device the coordinates of a given section of a panel indicated by a finger, characterized in that near the end of a finger touching or close to the panel, an image of a 'virtual stylus' is displayed on the panel - graphic a sign indicating a given section of the panel.

2. The method according to p. 1, characterized in that the input to the computing device the coordinates of a given section of the touch panel indicated by the virtual stylus, is carried out with a (short-term) detachment of the finger from the touch panel or when the finger touches the "extended" panel - the touch panel or some surface next to it, for example, the housing of the device or support under it.

3. The method according to claim 2, characterized in that the coordinates of a given section of the touch panel indicated by the virtual stylus are input into the computing device when the touch panel is touched with a second finger or when the touch panel is touched with increased force or when the 'extended' panel is briefly touched.

4. The method according to p. 1, characterized in that the image of the virtual stylus is displayed volumetric on the touch panel made stereoscopic.

5. The method according to p. 1, 4, characterized in that the image and (or) the functions of the virtual stylus depend on the position or orientation of the finger relative to the 'expanded' panel or the content of the information indicated by the virtual stylus or the nature of the movement of the finger or force tapping the touchpad or a combination of the above conditions.

6. The method according to p. 5, characterized in that the virtual stylus performs the function of a virtual magnifier.

7. The method according to p. 5, characterized in that with accelerated and (or) rapid movement of the virtual stylus, its image is displayed more noticeable, for example, “bold”, enlarged, with increased brightness.

8. The method according to claim 5, characterized in that the touch of the 'expanded' panel with the vertically located end of the finger is recorded as a keystroke with an upper case.

9. The method according to p. 8, characterized in that the touch of the touch panel with the vertically located end of the finger is recognized by the ratio of the force of touch and the size of the touch spot.

10. The method according to p. 5, characterized in that the successively touching the 'expanded' panel, sliding and detaching the finger are interpreted as the operation of moving the cursor.

11. The device according to p. 1, characterized in that it is equipped with means for determining the position of the fingers close to the touch panel.

12. The device according to p. 11, characterized in that the means for determining the position of the fingers is made in the form of at least one emitter, moreover, the touch panel can be formed by projecting the image onto a certain surface.

13. The device according to p. 1, characterized in that it is equipped with a means of measuring the touch force of the touch panel.

14. The device according to p. 13, characterized in that the means of measuring the touch force of the touch panel is made in the form of at least one strain gauge located between the housing and the non-rigidly mounted touch panel.

15. The device according to p. 13, characterized in that the device is equipped with an acceleration sensor designed to block the signal from the means of measuring the touch force of the touch panel during case shaking.