WO1995024681A1

WO1995024681A1 - Speech-controlled input device with logic standard interface

Info

Publication number: WO1995024681A1
Application number: PCT/SE1995/000234
Authority: WO
Inventors: Hassan Toosi
Original assignee: Hassan Toosi
Priority date: 1994-03-08
Filing date: 1995-03-06
Publication date: 1995-09-14
Also published as: AU1965095A; SE502013C2; JPH09510307A; SE9400768L; SE9400768D0; EP0749596A1

Abstract

The invention concerns a device with logic standard interface (11), for speech-controlled input of data into a computer, etc. (9). The device completely replaces a conventional input device, e.g., a keyboard and/or a control unit. The invention has been designed along the lines of speech-recognition technology and can either be a physically independent unit (5) or a logic unit (8), that is, a part of another device. A physically independent unit can consist of a physical interface for speech input (12), an audio card (12), a system program (3), a microcomputer (4), a physical interface for output (13), also a communications medium (6, 7) between the device and equipment for speech input (1) as well as the device and equipment that is to be operated (9). The invention is chiefly used for speech-control and hands-free use of equipment that has any kind of standard input device and/or control unit, e.g., a computer or a computer system.

Description

SPEECH-CONTROLLED INPUT DEVICE WITH LOGIC STANDARD INTERFACE

This invention appertains to a device, hereinafter called SID, that speech-controls the input of data, to an operative unit, for example, a computer. Made up of hardware and software, the SID has been designed on speech-recognition technology principles and can be speech-dependent or speech-independent.

The invention is chiefly intended for the simple and easy operation of a computer without the need to use hands.

The SID is a device for speech-control of indata along with a logic standard interface, i.e., a device that generates standard codes or signals. It completely does away with the use of a keyboard and/or a control unit. In other words, the SID is a system driver in order to function as a standard interface between applications and external units, for example, a microphone. That is, all outdata to the world outside, such as other applications, is a code according to a standard code system.

This can be compared to a computer keyboard and its drive routines.

They make use of standard codes, e.g., codes according to the American National Standards

Institute (ANSI), for their outdata to the world outside and via a general, standardized code system, they are able to write texts in all standard applications and also control these applications, such as, e.g., a menu-controlled application, with the help of keys and key combinations.

The keyboard has both character keys, e.g., a, b; control and special keys, e.g., ESC, TAB,

INSERT, HOME; function keys, e.g., Fl, F2 and key combinations with ALT-, CTRL- or

SHIFT-keys.

These always generate the same code, according to the standard code system, irrespective of the application one is working with. Of course, this code may have entirely different functions in different applications. This is something determined by the application designer.

Since the keyboard's drive routines, Device Drivers, is the standard interface for communication between keyboard and the outside world (applications), then a keyboard is sufficient in order to input text and control all computer programs including menu-controlled programs in the computer environment without the need of having to adapt either the keyboard, drive routines or the applications. This is precisely what the SID is, a general input device. And it is for this reason that the SID uses standard codes for its outdata. Where the SID uses, for example, the code system according to ANSI standards, then, from a function point of view, the SID operates exactly like a keyboard and its drive routines, but instead of depressing keys in a physical keyboard, we speak into a microphone.

No such speech-controlled device so far exists (refer to new patents reviews, Dnr P 914/43, issued by the Swedish Patent and Registration Office).

All presently known speech-recognition products that can generate texts have one big disadvantage, they do not have a standard interface, so they function only with one or several applications. They are based on word recognition. This in its turn means that the product requires considerable memory, has a limited vocabulary, has a large register, is relatively slow, has little reliability, is speaker-dependent, is a single-user system, has little tolerance for dialectical variations, is sensitive to background noise and requires a great deal of time to train the computer to "understand" the user's words and commands, is difficult to learn and use, also it is relatively expensive.

Examples of such products are Voice type and Dictation system from IBM and Dragon Dictate from Dragon Systems.

Another application that is based on the same principle and uses speech-recognition technology is "Text editor for speech input", with patent number US, 4914704, hereinafter called USP.

Basically, in purpose and results, the SID differs radically from the above-mentioned USP.

In other words, the SID can only be compared to a keyboard or mouse and their drive routines, but not with an application program, for example, the word processor "Microsoft

WORD for Windows".

In the same way, one can compare the word processor "Text editor for speech input" according to USP with other word processors such as Word Perfect, Microsoft WORD for

Windows, Dragon Dictate, and so on, but not with a keyboard's drive routines. So, exactly in the same way one cannot compare the word processor Microsoft WORD for Windows with the calculator program EXCEL or the fax program WINFAX, and so on, so neither can one compare SID with a word processor such as, e.g., "Text editor for speech input".

USP is a "Text editor for speech input" word processor, that is, a separate application program that employs speech recognition technology to input text into its own application,

"Text editor".

The text that is fed through speech input into the "Text editor" is not any standard code, but a so-called "token and string format" (USP: fig. 3, fig. 4), which the "Text editor" designer has defined just for this particular application.

Therefore, this word processor has no other interface for other applications than its "Text editor", so it cannot input text or operate any other application than itself.

The SID, On the other hand, is a drive routine and uses standard codes that follow a standard code system in order to be able to replace a keyboard completely. If, for example, according to the ANSI standards, code "97" is in-coded instead of "a", and code "00;59" instead of "Fl" as outdata in order that SID can be a standard interface for all standard applications in the computer environment.

USP writes only those writeable characters for the "Text editor" that are included as part of the codes listed in the ASCII-table.

The SID, on the other hand, sends as its outdata to an operating unit, not only the standard code for the writeable characters but also the standard code for one or a combination of several of all the keyboard keys, according to those to be found in a standard code system. For example, these can be:

Control and special keys, e.g.: ESC, TAB, HOME, INSERT; CTRL + character key, e.g. : CTRL+A, CTRL+B; ALT + character key, e.g.: ALT+A, ALT+B; SHIFT + key, e.g.: SHIFT+TAB; Function keys: Fl, F2; also one or a combination of several of these keys, e.g.: CTRL+F1, ALT+F2, and so on.

It is just these unusual features of the SID, which are forthcoming in the characteristic parts of the patent claims, that will make it entirely unique with many unique properties:

A. Since the SID uses the code system according to ANSI standards, it will be entirely on a par with and operate like a keyboard and its drive routines. This means that:

A.1: In contrast to the USP, which inputs text only to its own word processor, the SID is a general input device for controlling of and text input to other applications.

A.2: SID activates menus in menu-controlled programs in order to roll up the "blinds" and operate the different commands in the sub-menus and conversational displays by sending out standard codes for the ALT+character key combinations. For example, ALT+A combination has standard code "00;30", which activates, e.g., the "Archive" menu in the menu row in any standard application whatsoever that has an underlined A, "A" in its menu row. This is exactly the same as depressing the ALT-key and the A-key simultaneously on an ordinary computer keyboard.

A.3: With the help of CTRL+characters or other key, the SID briefly activates the command, that is, it activates the command without needing to roll up the blinds, and it also executes the defined functions for these codes in different programs.

A.4: The SID also sends the code for the control and special keys such as ESC, HOME, Function keys and the code for the other key combinations according to the standardized code system.

A.5: The SID directs the cursor on the display by using Escape sequences according to ANSI standards.

The USP can do none of these whatsoever, apart from the ability to write only writeable characters, i.e., some of the ASCII characters, via its own defined code format. B: The SID uses the same standardized code or signal systems when it is applied to other equipment that has a keyboard and uses a standard code or signal system. This enables new products to be created that are controlled by the voice instead of by a keyboard.

C: Because of its design, the SID is, moreover, a device that, among other things: is easy and quick to install and start operating, is easy to learn and not difficult to use, can be very simply translated into other languages, has "on-board" unlimited vocabulary, and so on.

Where, e.g., the ASCII-code is used as standard coding, the SID not only can generate the ASCII-code for digits and letters, but also generate the code for other characters, for example, the control characters that are used for control and communication purposes, graphics, foreign and mathematical symbols. This feature automatically provides the system with an on-board "unlimited" vocabulary. Moreover, due to the fact that a set-up of the expanded ASCII-code (Extended ASCII Code), also contains character codes for foreign characters, the SID has an on-board unlimited "multi-language" vocabulary. The invention is a simple, easy-to-use, flexible product from the viewpoint of the user, since it eliminates or reduces the above-mentioned disadvantages and the problems with word-based speech-recognition products. This invention also has a number of properties non-existent in other systems, properties that pave the way for new possibilities, areas of use and product ideas.

The means necessary to achieve such a device are components required to introduce voice signals, e.g., a microphone, an audio card, system driver, i.e., SID's program, a microcomputer for the physically independent input device and a communications medium.

The attached drawings schematically show the execution form of the invention. Figure 1 shows a general layout.

Figures 2 shows a physically independent, speech-controlled input device and 3 show a logic such. Figure 4 shows where the application program and the SID program are situated in a data system. SID's program, which is a system driver, is a standard interface between the application program and external unit, e.g., a microphone.

The SID can be designed either with speech-dependent or speech-independent recognition, depending on the application in which the SID is to be used. If user identification and unit limitation use is desired, a speech-dependent recognition SID should be used, but if, e.g., a multi-user system is desired, then a speech-independent recognition unit is to be preferred.

As shown in Figs. 2 and 3, the SID is made up of hardware and software, chiefly in the following order: a device for introducing voice signals in the system, e.g., a microphone 1, a audio card 2, SID's program 3, a microcomputer 4, an arrangement for a physically independent unit 5, and also communications medium 6, 7, partly between the arrangement for the introduction of voice signals 1, and the unit 5/8, and partly between the unit and the device(s) 9, to be controlled.

In its simplest form, Fig. 1, the SID should be regarded as a "black box" 10, into which the voice signal, which represents, for example, a letter, a digit, or other keys and key combinations, is inputted and there compared to signals already stored in the "box", from which comes a standard code according to whichever standard system is used, e.g., ANSI, ASCII, ISO, EBEDIC, teletext, digit tones, etc. In other words, when we speak in "a", "b", etc., the unit generates, if ASCII-code is used, 97, 98, etc., see Fig. 3. In this way, we create a logic standard interface 11, in order to be able to communicate simply with other equipment 9, which uses the same code system.

The way in which this is done can vary to a certain extent depending on whether the speech recognition is speech-dependent or speech-independent.

In the speech-dependent speech recognition version, the user starts, for example, by "training the computer" for the "vocabulary" intended for the computer program. That is to say, an installation- /set up program requests the user to speak into the microphone as per instructions. The user inputs the name of several or all the keys and their standard combinations, one after the other, by speaking them into the microphone. The uttered characters are digitized by a audio card.

The SID program stores these vocal expressions of these names in a register, in which each item is related to the standard code for each of these keys or standard key combinations. After the training session has been completed, the SID is ready for use. Now, when the user says, e.g., a character into the computer, this character is digitized and this is then compared with the characters already stored in the SID-program's register. When the character is recognized, the standard code for the corresponding character is fetched and sent further for processing of other programs or units.

With speech-independent speech recognition, the user does not need to train the computer with regard to the "vocabulary" in question, since the language sounds of these characters, the phoneme, are already stored in the SID-program's register. Here a brief training with a reference text is sufficient.

Because the character sounds are pre-programmed, the program functions irrespective who the user is, as long as he/she speaks the same (human) language for which the program is pre-programmed.

The SID-program does not need to be seen on the display, just as in the same way the keyboard's drive routines are not shown on it.

Any incorrect messages can be indicated as text, voice or signals.

The fact that the SID can be variously configured for different purposes means that it can be applied to a computer or other apparatuses that have a keyboard and/or use a standard code system, in order to replace completely or be a complement to the keyboard. This, therefore, makes it possible to operate, for example, the computer and all its programs with only the speech.

Since the SID uses, among other things, the alphabet in a language, it has, therefore, an

"unlimited" vocabulary. The SID can be used for writeable characters, that is, letters, digits and other characters, also for non-writeable characters, that is, control and special characters.

The SID can be used in order to replace function keys, replace buttons on a control unit, replace commands or special command sequences in a program. The SID can be used to provide alias for certain characters for special occasions, e.g., in connection with computer games, and also for calling up files or for its own defined commands.

The SID can be applied to a printer or an electronic typewriter in order to create a speech-controlled typewriter.

The SID can be applied to a fax-machine in order to create a speech-controlled fax-machine that vocally faxes messages without the use of paper.

It can be applied to a mobile telephone in order to create a speech-controlled, paperless, mobile fax-machine that vocally faxes texts to, e.g., Minicall texts or other mobile unit.

The SID can be applied to a code-lock or authorized control system with personal code.

It can be applied to a remote control for speech-controlling, for example, domestic electronics with the voice.

The SID can be applied to a telephone or to a code-lock system for telephone numbering by generating, for example, digit tones instead of ASCII-code.

The SID can also be used in applications in connection with interactive telecommunication and data communication, speech-controlled toys, computer games, and so on.

It should also be possible to use the SID to replace completely or be a complement to a control unit. Here, e.g., the cursor is steered on a display by indicating the position you wish it to have. For example, an XY- co-ordinate or X- and Y- co-ordinates, can be used to move the cursor to the desired position.

The chief function of the SID is to replace a keyboard and/or a control unit. This makes it an ideal handicap aid.

Physically the SID can be designed as an independent unit 5, with physical interfaces 12, 13. Its connect-up 13 could then be plugged into the standard input socket 9 on the equipment to be controlled or used.

The communication medium between the SID and other equipment 7, also between the speech input and the equipment for speech-input 6, may vary, and will therefore require different kinds of cables or other medium, such as lighting, modulated radio, etc. The SID can also be a logic unit 8. In other words, it can be a part of, for example, a computer system and make use of its hardware to operate and control the system, the programs 14, or other equipment 9 connected to the system.

There is room for the use of this invention within a number of technical fields, branches and user groups.

It can be configured and applied for speech-operating a number of apparatuses and equipment, or in connection with the production of new products. Examples of these are: Speech-controlled computers; speech-controlled typewriters; speech-controlled, paperless fax machines; speech-controlled, paperless mobile fax machines; speech-controlled code-lock system or authorized control with personal code; speech-controlled remote control; speech-controlled telephone, applications within telecommunication and data communication; speech-controlled toys; speech-controlled computer games, etc.

The SID can have many different areas of use, such as a handicap aid that makes it easy to control and operate a computer, different programs and the equipment that is connected to a computer. With the SID a handicapped person will be able to work or communicate with the world outside. It can be used to improve the working environment where it could prevent, e.g., working ailments that arise from overuse of a computer mouse and keyboard. The SID can be used to improve data security, that is, to reduce the risks of incorrect input in connection with, for example, registration or word processing. Productivity can also be increased since it is quicker to speak than to write; greater flexibility is achieved through the elimination of the need for a keyboard and hands since these will be replaced by the voice and, e.g., the microphone for such things as stock handling and mobile units can be more compact, take up less space because the need for a keyboard, etc., has been eliminated.

Naturally the SID is not limited just to the units, applications, areas of use described above. It also has varied potential within the framework for the basic idea behind it.

Claims

PATENT REQUIREMENTS

1. Device for speech-controlled input of data in the form of codes to an operative unit, e.g., a computer, a computer program, a code-lock system, an electronic typewriter, etc., designed along the lines of speech-recognition technology, which may be either speech-dependent or speech-independent. The device is made up of hardware and software.

Its distinctive feature is that it has a logic standard interface for outdata (11) and is so designed that the incoming voice signals, that represent one or a combination of some of all the keys on the keyboard according to those to be found in a standard code system, are compared and converted in the speech-controlled input device to outdata in the form of a standard code or signal according to the standardized code system, and this code is then transferred to an operative unit (9, 14) with an indata logic standard interface.

2. Device according to requirement 1, with the characteristic feature that, functionwise, it is equivalent to a computer keyboard. In other words, a device that is designed to operate menu-controlled computer programs and to input text into standard applications without prior application adaptation by being able to use standard codes for one or a combination of several of all the keyboard's keys.

3. Device according to the foregoing requirements, with the characteristic feature that, functionwise, it is equivalent to function keys and is designed to be used in applications to execute those functions that are specific to the application in question, without prior application adaptation by being able to use the standard code for the function keys as well as combinations of these keys with other keys.

4. Device according to one or several of the foregoing requirements, with the characteristic feature that the device is designed to steer the cursor on a display by vocally indicating the desired cursor position, for example, the XY- co-ordinate or X- co-ordinate and Y- co-ordinate respectively according to a standard code system.

5. Device according to one, several or all of the foregoing requirements, with the characteristic feature that the device physically can either be an independent unit (5), with a physical standard interface (12, 13), that is, it is directly connected to the input of the equipment to be operated, or a logic unit (8), that is, a part of another equipment or computer system for operating these.

6. Device according to one, several or all of the foregoing requirements, with the characteristic feature that the device can be applied to a fax-machine to create a speech-controlled, paperless, automatic fax-machine.

7. Device according to one, several or all of the foregoing requirements, with the characteristic feature that the device can be applied to a mobile telephone to create a speech-controlled, paperless, mobile fax-machine that transmits text messages to, e.g., Minicall text or another mobile unit.

8. Device according to one, several or all of the foregoing requirements, with the characteristic feature that the device can be applied to a code-lock or authorized control system with personal codes.

9. Device according to one, several or all of the foregoing requirements, with the characteristic feature that the device can be applied to a telephone or a code-lock system for ringing telephone numbers by generating, e.g., digit tones instead of codes according to another standard.

10. Device according to one, several or all of the foregoing requirements, with the characteristic feature that the device has applications within telecommunication and data communication, speech-control of toys, computer games, remote control for domestic electronics, etc.