US8340958B2 - Text and speech recognition system using navigation information - Google Patents

Text and speech recognition system using navigation information Download PDF

Info

Publication number
US8340958B2
US8340958B2 US12/693,392 US69339210A US8340958B2 US 8340958 B2 US8340958 B2 US 8340958B2 US 69339210 A US69339210 A US 69339210A US 8340958 B2 US8340958 B2 US 8340958B2
Authority
US
United States
Prior art keywords
entries
list
input
navigation information
language model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US12/693,392
Other versions
US20100191520A1 (en
Inventor
Rainer Gruhn
Andreas Marcel Riechert
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Harman Becker Automotive Systems GmbH
Original Assignee
Harman Becker Automotive Systems GmbH
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Harman Becker Automotive Systems GmbH filed Critical Harman Becker Automotive Systems GmbH
Assigned to HARMAN BECKER AUTOMOTIVE SYSTEMS GMBH reassignment HARMAN BECKER AUTOMOTIVE SYSTEMS GMBH ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: GRUHN, RAINER, RIECHERT, ANDREAS
Assigned to JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT reassignment JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT SECURITY AGREEMENT Assignors: HARMAN BECKER AUTOMOTIVE SYSTEMS GMBH
Publication of US20100191520A1 publication Critical patent/US20100191520A1/en
Assigned to HARMAN INTERNATIONAL INDUSTRIES, INCORPORATED, HARMAN BECKER AUTOMOTIVE SYSTEMS GMBH reassignment HARMAN INTERNATIONAL INDUSTRIES, INCORPORATED RELEASE Assignors: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
Assigned to JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT reassignment JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT SECURITY AGREEMENT Assignors: HARMAN BECKER AUTOMOTIVE SYSTEMS GMBH, HARMAN INTERNATIONAL INDUSTRIES, INCORPORATED
Assigned to HARMAN INTERNATIONAL INDUSTRIES, INCORPORATED, HARMAN BECKER AUTOMOTIVE SYSTEMS GMBH reassignment HARMAN INTERNATIONAL INDUSTRIES, INCORPORATED RELEASE Assignors: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
Application granted granted Critical
Publication of US8340958B2 publication Critical patent/US8340958B2/en
Active legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/02Input arrangements using manually operated switches, e.g. using keyboards or dials
    • G06F3/023Arrangements for converting discrete items of information into a coded form, e.g. arrangements for interpreting keyboard generated codes as alphanumeric codes, operand codes or instruction codes
    • G06F3/0233Character input methods
    • G06F3/0237Character input methods using prediction or retrieval techniques
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00Speech recognition
    • G10L15/22Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G10L2015/226Procedures used during a speech recognition process, e.g. man-machine dialogue using non-speech characteristics
    • G10L2015/228Procedures used during a speech recognition process, e.g. man-machine dialogue using non-speech characteristics of application context

Definitions

  • This invention relates to a method for recognizing a user's speech input, as well as a method for supporting text input utilizing navigation information.
  • the invention also relates to the systems support such methods.
  • the invention finds especially, but not exclusively, application in vehicles; however, it can also be used in connection with portable navigation units.
  • Speech-driven electronic systems are known that can be controlled by a user uttering speech commands.
  • Speech commands are recognized by a speech recognition unit that converts the spoken words to a string of characters.
  • the recognition is based on language model statistics including a database of known words and including the likelihood of an occurrence of the different words in the database taking into account the grammar of the language in which the speech command was uttered.
  • a method for recognizing a user speech input including the steps of detecting the user's speech input. Furthermore, the user's speech input is recognized by comparing the speech input to a list of entries using language model statistics in order to determine the most likely entry best matching the user's speech input. In an additional step navigation information of a navigation unit is detected, where the most likely entry as recognized is determined by modifying the language model statistics taking into account the navigation information.
  • the speech recognition unit is used in connection with a navigation unit, the likelihood that navigation-related information of an already determined trip to a predetermined destination is present in the speech input increases. Accordingly, this information is provided to the speech recognition unit recognizing the speech input allowing the system to take into account environment information that is available from the navigation unit.
  • An example of a user speech input while being on the way to a predetermined destination may be a user dictating a message to a speech recognition unit that can be transmitted to a recipient via a telecommunications network.
  • a speech recognition unit that can be transmitted to a recipient via a telecommunications network.
  • a method for supporting a text input in an input unit is also provided in which a first part of the text input is detected.
  • the text input is recognized by comparing the first part of the text input to a list of entries based on language model statistics to suggest a most likely candidate of a completed first input.
  • navigation information relating to a trip to a predetermined destination is detected and the most likely candidate of the completed text input is determined by modifying the language model statistics taking into account the navigation information.
  • the most likely candidate or the most likely candidates of a completed text input are suggested to a user of the input unit.
  • the navigation information may contain information such as the starting location of the trip, the destination location of the trip, via points of the trip, the current position, the overall travel time of the trip or the travel time until now and the expected arrival time.
  • the probability that the message contains the pieces of information mentioned above is quite high.
  • a speech recognition system for recognizing the user's speech input, the system includes a speech detector for detecting the user speech input and a speech recognition unit recognizing the user speech input by comparing the speech input to a list of entries using language model statistics to determine the most likely entry which best matches the user's speech input. Furthermore, a navigation unit is provided in which navigation information is present that can be utilized by the recognition unit for determining the most likely entry by modifying the language model statistics taking into account said navigation information.
  • Such a speech recognition system has an improved speech recognition rate as the user's environment information can be better considered than in speech recognition systems which recognize the speech input based on language model statistics that only take into account some general text database generated by considering the likelihood of a word isolated (unigram) or considering the context and input history (n-gram).
  • a system for supporting a text input or for automatically suggesting completions for words given by the first letters including a text recognition unit for recognizing the text input by comparing a first part of the text input to the list of entries using said language model statistics that is used to suggest the most likely candidate of the completed text input.
  • the navigation unit of the system provides navigation information and the text recognition unit determines the most likely candidate by modifying the language model statistics taking into account the navigation information.
  • the text recognition unit furthermore suggests the most likely candidate or most likely candidates of completed text inputs to the user of the system.
  • FIG. 1 illustrates a schematic view of one example of a speech recognition system that takes into account navigation information.
  • FIG. 2 illustrates one example of an input system that automatically suggests completed text input by taking into account navigation information.
  • FIG. 3 illustrates a flowchart showing example steps for recognizing speech input by taking into account navigation information.
  • FIG. 4 illustrates a flowchart showing example primary steps for automatically suggesting a completed text input by taking into account navigation information.
  • FIG. 1 one example of a speech recognition system 100 is shown that utilizes current navigation information to increase speech recognition performance.
  • the system 100 may be, for example, incorporated into a vehicle. It is, however, also possible that the system 100 be utilized in connection with a mobile stand-alone unit that may be used by a pedestrian, a bicycle user or in a vehicle.
  • the speech recognition system 100 includes a navigation unit 102 providing guiding instructions to the user to a predetermined destination. The selected user destination can be input via an input unit 104 or may be otherwise provided to the navigation unit 102 .
  • the navigation unit 102 may be a conventional navigation unit 102 that includes a position detecting unit (not shown) and a database with map data 106 on the basis of which a route can be calculated from the present position of the navigation unit 102 to a desired destination location.
  • the navigation unit 102 calculates, based on different criterions, such as shortest or fastest route, a route to said destination location.
  • the user can furthermore specify destination information via points that are located the route to the destination location. Such points may be points of interest, such as interesting sightseeing monuments, a hotel, a gas station, etc.
  • the user of the speech recognition system 100 may desire to dictate a message, such as a SMS or an e-mail to be sent via wireless communication networks, to a recipient or may want to write a memo for future use.
  • a message such as a SMS or an e-mail to be sent via wireless communication networks
  • the probability that the information sent in this message contains information of the trip to the destination is quite high.
  • the user of a system may want to inform the recipient of the letter of his/her arrival time or may inform the recipient that he/she is on his/her way to an arranged meeting point etc.
  • the user of the speech recognition system 100 may dictate the letter using speech input into a speech recognition unit 108 .
  • the speech recognition unit 108 detects the speech input and converts the speech signal to a sequence of words by first recognizing a phoneme string and by comparing the phoneme string to a list of entries using a language model.
  • the language model contains statistical, grammatical and/or syntax rules of a language and recognizes the speech input.
  • model statistics are used, such as the likelihood that a word occurs in isolated form (unigram) in a context and input history (n-gram).
  • the language model together with the language model statistics may be stored in a language model database 110 .
  • the speech recognizer 108 is then able to use the language model database 110 for correctly interpreting the speech input.
  • each word has a predetermined likelihood of occurrence, be it alone, be it as a word pair in the form of a bigram or be it in view of the context in the case of an n-gram.
  • the starting location, the current position, the destination location, via points, the starting time, the current travel time, the expected arrival time are known by the navigation unit 102 .
  • Such navigation information can be provided to the language model database 110 to change the language model statistics in the language model database 110 that are utilized to recognize the speech input.
  • the entries in the list of entries, i.e. the database of known words, can be emphasized by increasing the probability of occurrence of the entries also contained in the navigation information.
  • the navigation information such as “Hamburg”, “Jungfernsteg”, “3:15” can be fed to the language model database, the likelihood of these list entries being increased.
  • the probability that the speech recognition unit 108 correctly interprets the speech input is remarkably increased.
  • the language model statistics is not only calculated on the basis of some general text before the assembly of the products as it is known in the art, but the language model statistics is updated taking into account any given environmental information, such as the navigation information.
  • FIG. 2 another example of an implementation of the system 200 is shown in which the navigation information provided by a navigation unit 206 can increase the functionality of the man-machine-interface.
  • an input unit 202 is provided for the entry of text via, for example, a keyboard.
  • the input unit 202 may also be a unit capable of recognizing a hand-written character input.
  • the input unit 202 may contain hard keys, such as a keyboard, or may contain a display showing soft keys that can be selected by touching the display.
  • the system 200 may help increase the speed of entry of text into the input unit 202 . To speed up the input process, an automatic completion is utilized as soon as the first part of the input text is input.
  • the input unit 202 may suggest completed words. For suggesting a completed text input, based on the first part of the text input, the input unit 202 makes reference to the language model database 204 , which may correspond to the language model database 110 described in connection with FIG. 1 .
  • the language model database 204 the different expressions known to the system are stored as list of entries, the database language model database 204 furthermore storing the likelihood of occurrence of the different words.
  • the text input into the input unit 202 may be a message, memo, note or letter, as discussed in connection with FIG. 1 . However, in connection with FIG. 2 , the text is manually input and not dictated.
  • navigation information provided by navigation unit 206 can help to improve and to speed up the text input.
  • the navigation unit 206 may have a separate input unit 202 .
  • the input unit 202 may also be the same as input unit 208 .
  • the map data contained in the map database 210 may assist in calculating the optimum route to a predetermined destination.
  • the navigation information given by the navigation unit 206 is fed to the language model database 204 , where the weight of list entries also contained in the navigation information may be increased.
  • the system 200 may further increase the speed of text as the system 200 can suggest faster and more reliable the correct completed text input.
  • the input unit may propose “Hamburg” as the most likely candidate of a completed text instead of other expressions beginning with the same two letters.
  • the input unit 202 may be an input unit of a cellular phone where the text input is not unambiguous as, by pressing an actuating element, different characters may be selected.
  • the system 200 may be improved by providing the navigation information to the language model database 204 where the most likely candidate of the input text is suggested based on the actuated actuating element and based on the received navigation information.
  • the system 200 may assist in a method in which the text input is a phonetic representation of the text as it is common in many Asian languages.
  • the text input is a phonetic representation of the text as it is common in many Asian languages.
  • a homophone one of the different possible meanings of the phonetic representation has to be selected. This selection or suggestion of the most likely candidate of the different possible meanings, i.e. of the completed text input may similar be improved by using the navigation information.
  • the text is input using a handwritten form, a handwriting recognition unit being provided as part of the input unit to recognize the handwritten text input.
  • a handwriting recognition unit being provided as part of the input unit to recognize the handwritten text input.
  • the provided navigation information may also be helpful.
  • a flow diagram 300 is illustrated that provides example steps to increase the speech recognition rate in a system, such as the speech recognition system 100 , in FIG. 1 , utilizing current information, such as information provided by a trip to a predetermined destination.
  • the process starts in step 302 .
  • speech input is detected.
  • navigation information can be detected.
  • the current navigation information may be fed to the speech recognition unit 108 , so that the latter is always updated with the current navigation information.
  • the list entries contained in the language model database 110 and contained in the navigation information can be weighted more heavily—meaning that the weight is increased compared to a normal likelihood of a word given for a predetermined language.
  • the speech recognition step may utilize statistical models, such as a hidden Markov models (HMMs) to recognize the speech input.
  • the speech input as it was recognized by the speech recognition unit, can be output to the user, as shown in step 312 .
  • the most likely word or sequence of words is output.
  • the recognized speech output presented to the user can be confirmed or corrected by the user.
  • the most likely entries of the recognized word may be presented to the user on a display allowing the user to select the desired expression.
  • the method ends in step 314 .
  • FIG. 4 a flow diagram is provided that illustrate example primary steps for text input using the system 200 of FIG. 2 .
  • the process starts at step 402 .
  • the first characters are input into the input unit 202 .
  • the navigation information is detected in step 406 . It should be understood that the navigation information could also have been input before the user starts to input the text into the input unit 202 .
  • the list entries that are also contained in the navigation information are weighted by increasing their weight compared to the weight before the navigation information was input.
  • step 410 the first characters input in step 404 are compared to the weighted list of entries and in step 408 and the most likely candidate or the most likely candidates are displayed to the user, at step 412 , who can either confirm the proposed most likely candidate or select one of the most likely candidates or discard the suggestions.
  • step 412 who can either confirm the proposed most likely candidate or select one of the most likely candidates or discard the suggestions. The method ends in step 414 .
  • the proposed systems and methods help to increase the comfort and speech recognition accuracy and are especially useful in connection with languages where words are input with some phonetical transcription, such as Asian languages, and then converted into characters by selection of a user.
  • This conversion process requires statistical models which can profit from the proposed consideration of information from the navigation unit.
  • infotainment unit connected to the systems 100 and 200 shown in FIGS. 1 and 2 .
  • This infotainment unit can provide the current infotainment status, such as the piece of music currently played or the film that is currently streamed or the TV station that is currently received.
  • the given environmental information helps to increase the recognition rate in the case of a speech recognition or in the case a completed text input should be suggested.
  • the invention may also be applied in case of a text input that is not unambiguous.
  • the navigation information can also help to recognize the first part of the text input or the complete text input using the navigation information.
  • the navigation information of a combined system having a cellular phone and a navigation module can help to render the ambiguous text input unambiguous.
  • the user may input the text in a phonetic representation of the text.
  • different expressions having the input phonetic representation are displayed to the user for further selection, or the most likely expression may be automatically selected.
  • the most likely candidate of the text input i.e. one expression among different expressions having the same phonetic representation can be selected taking into account the navigation information.
  • the list entries relating to said current infotainment status can be determined and the likelihood of said entertainment related list entries can also be increased.
  • the invention can furthermore be applied in connection with hand-writing recognition, where the text is input in hand-written form into an input unit, the input unit recognizing the hand-writing.
  • the input unit recognizing the hand-writing.
  • it can also be helpful to use the available navigation information such as destination location, arrival time, etc. This information can help to increase the recognition rate of the hand-writing recognition, when the user writes down the message to be transmitted or a memo for future use.
  • the proposed systems and methods have special relevance for Asian languages, where input methods with statistical support are much more common than in Latin character-based languages. In those languages words are often input with some phonetic transcription such as Kana or Pinyin and then converted to Chinese characters. In general, the invention is especially helpful when used in connection with languages having lots of homophones, i.e. expressions having the same or similar articulation, but are written differently.
  • one or more processes, sub-processes, or process steps described in connection with FIGS. 1-4 may be performed by hardware and/or software. If the process is performed by software, the software may reside in software memory (not shown) in a suitable electronic processing component or system such as, one or more of the functional components or modules schematically depicted in FIGS. 1-4 .
  • the software in software memory may include an ordered listing of executable instructions for implementing logical functions (that is, “logic” that may be implemented either in digital form such as digital circuitry or source code or in analog form such as analog circuitry or an analog source such an analog electrical, sound or video signal), and may selectively be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that may selectively fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions.
  • a “computer-readable medium” is any means that may contain, store or communicate the program for use by or in connection with the instruction execution system, apparatus, or device.
  • the computer readable medium may selectively be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device. More specific examples, but nonetheless a non-exhaustive list, of computer-readable media would include the following: a portable computer diskette (magnetic), a RAM (electronic), a read-only memory “ROM” (electronic), an erasable programmable read-only memory (EPROM or Flash memory) (electronic) and a portable compact disc read-only memory “CDROM” (optical).
  • the computer-readable medium may even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.

Abstract

A system and method are provided for recognizing a user's speech input. The method includes the steps for detecting the user's speech input, recognizing the user's speech input by comparing the speech input to a list of entries using language model statistics to determine the most likely entry matching the user's speech input, and detecting navigation information of a trip to a predetermined destination, where the most likely entry is determined by modifying the language model statistics taking into account the navigation information. A system and method is further provided that takes into account navigation trip information to determine the most likely entry using language model statistics for recognizing text input.

Description

RELATED APPLICATIONS
This application claims priority of European Patent Application Serial Number 09 000 983.8, filed on Jan. 23, 2009, titled IMPROVED TEXT AND SPEECH INPUT USING NAVIGATION INFORMATION, which application is incorporated in its entirety by reference in this application.
BACKGROUND OF THE INVENTION
1. Field of the Invention
This invention relates to a method for recognizing a user's speech input, as well as a method for supporting text input utilizing navigation information. The invention also relates to the systems support such methods. The invention finds especially, but not exclusively, application in vehicles; however, it can also be used in connection with portable navigation units.
2. Related Art
To input a SMS via a keyboard, a user can type the full words via a numerical keyboard. However, this procedure is very time-consuming. To speed up the process, automatic completion systems have been built that suggest completions of words given the first part of the word, i.e. given the first letters. These systems are based on dictionaries with word frequency statistics. Those statistics may be updated if the user types the same word frequently, but they do not take into account environment information that is available to a navigation unit.
Furthermore, speech-driven electronic systems are known that can be controlled by a user uttering speech commands. Speech commands are recognized by a speech recognition unit that converts the spoken words to a string of characters. The recognition is based on language model statistics including a database of known words and including the likelihood of an occurrence of the different words in the database taking into account the grammar of the language in which the speech command was uttered.
If, while a navigation unit is guiding a user to a predetermined destination, the user of the speech recognition unit or a user of a text input unit utters a speech command or enters a text into a system while being on his/her way to the predetermined destination, the likelihood that the uttered speech command or the input text contains information about the trip to a predetermined destination is quite high. Accordingly, a need exists to improve a speech recognition unit or to improve an automatic completion during a text input when the user is on the way to a predetermined destination.
SUMMARY
A method for recognizing a user speech input is provided including the steps of detecting the user's speech input. Furthermore, the user's speech input is recognized by comparing the speech input to a list of entries using language model statistics in order to determine the most likely entry best matching the user's speech input. In an additional step navigation information of a navigation unit is detected, where the most likely entry as recognized is determined by modifying the language model statistics taking into account the navigation information. When the speech recognition unit is used in connection with a navigation unit, the likelihood that navigation-related information of an already determined trip to a predetermined destination is present in the speech input increases. Accordingly, this information is provided to the speech recognition unit recognizing the speech input allowing the system to take into account environment information that is available from the navigation unit. An example of a user speech input while being on the way to a predetermined destination may be a user dictating a message to a speech recognition unit that can be transmitted to a recipient via a telecommunications network. By taking into account the navigation information the speech recognition accuracy can be improved.
A method for supporting a text input in an input unit is also provided in which a first part of the text input is detected. The text input is recognized by comparing the first part of the text input to a list of entries based on language model statistics to suggest a most likely candidate of a completed first input. Additionally, navigation information relating to a trip to a predetermined destination is detected and the most likely candidate of the completed text input is determined by modifying the language model statistics taking into account the navigation information. The most likely candidate or the most likely candidates of a completed text input are suggested to a user of the input unit.
By way of example, the navigation information may contain information such as the starting location of the trip, the destination location of the trip, via points of the trip, the current position, the overall travel time of the trip or the travel time until now and the expected arrival time. In case the user of the speech recognition system or of the text input system sends an e-mail to another person either using speech commands or using a keyboard-based text input, the probability that the message contains the pieces of information mentioned above is quite high. By increasing the likelihood of a list entry contained in the list of entries and also contained in a navigation information, the speech recognition rate or success probability of the proposal for the most likely candidate in the case of an automatic completion of words is increased.
A speech recognition system is also provided for recognizing the user's speech input, the system includes a speech detector for detecting the user speech input and a speech recognition unit recognizing the user speech input by comparing the speech input to a list of entries using language model statistics to determine the most likely entry which best matches the user's speech input. Furthermore, a navigation unit is provided in which navigation information is present that can be utilized by the recognition unit for determining the most likely entry by modifying the language model statistics taking into account said navigation information. Such a speech recognition system has an improved speech recognition rate as the user's environment information can be better considered than in speech recognition systems which recognize the speech input based on language model statistics that only take into account some general text database generated by considering the likelihood of a word isolated (unigram) or considering the context and input history (n-gram).
A system is also provided for supporting a text input or for automatically suggesting completions for words given by the first letters including a text recognition unit for recognizing the text input by comparing a first part of the text input to the list of entries using said language model statistics that is used to suggest the most likely candidate of the completed text input. The navigation unit of the system provides navigation information and the text recognition unit determines the most likely candidate by modifying the language model statistics taking into account the navigation information. The text recognition unit furthermore suggests the most likely candidate or most likely candidates of completed text inputs to the user of the system.
Other devices, apparatus, systems, methods, features and advantages of the invention will be or will become apparent to one with skill in the art upon examination of the following figures and detailed description. It is intended that all such additional systems, methods, features and advantages be included within this description, be within the scope of the invention, and be protected by the accompanying claims.
BRIEF DESCRIPTION OF THE FIGURES
The invention may be better understood by referring to the following figures. The components in the figures are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the invention. In the figures, like reference numerals designate corresponding parts throughout the different views.
FIG. 1 illustrates a schematic view of one example of a speech recognition system that takes into account navigation information.
FIG. 2 illustrates one example of an input system that automatically suggests completed text input by taking into account navigation information.
FIG. 3 illustrates a flowchart showing example steps for recognizing speech input by taking into account navigation information.
FIG. 4 illustrates a flowchart showing example primary steps for automatically suggesting a completed text input by taking into account navigation information.
DETAILED DESCRIPTION
In FIG. 1, one example of a speech recognition system 100 is shown that utilizes current navigation information to increase speech recognition performance. The system 100 may be, for example, incorporated into a vehicle. It is, however, also possible that the system 100 be utilized in connection with a mobile stand-alone unit that may be used by a pedestrian, a bicycle user or in a vehicle. The speech recognition system 100 includes a navigation unit 102 providing guiding instructions to the user to a predetermined destination. The selected user destination can be input via an input unit 104 or may be otherwise provided to the navigation unit 102. The navigation unit 102 may be a conventional navigation unit 102 that includes a position detecting unit (not shown) and a database with map data 106 on the basis of which a route can be calculated from the present position of the navigation unit 102 to a desired destination location. The navigation unit 102 calculates, based on different criterions, such as shortest or fastest route, a route to said destination location. The user can furthermore specify destination information via points that are located the route to the destination location. Such points may be points of interest, such as interesting sightseeing monuments, a hotel, a gas station, etc.
On the way to the destination, the user of the speech recognition system 100 may desire to dictate a message, such as a SMS or an e-mail to be sent via wireless communication networks, to a recipient or may want to write a memo for future use. In such a situation, the probability that the information sent in this message contains information of the trip to the destination is quite high. By way of example, the user of a system may want to inform the recipient of the letter of his/her arrival time or may inform the recipient that he/she is on his/her way to an arranged meeting point etc.
The user of the speech recognition system 100 may dictate the letter using speech input into a speech recognition unit 108. The speech recognition unit 108 detects the speech input and converts the speech signal to a sequence of words by first recognizing a phoneme string and by comparing the phoneme string to a list of entries using a language model. The language model contains statistical, grammatical and/or syntax rules of a language and recognizes the speech input. For interpreting the speech input language, model statistics are used, such as the likelihood that a word occurs in isolated form (unigram) in a context and input history (n-gram). The language model together with the language model statistics may be stored in a language model database 110. The speech recognizer 108 is then able to use the language model database 110 for correctly interpreting the speech input. In the language model statistics, each word has a predetermined likelihood of occurrence, be it alone, be it as a word pair in the form of a bigram or be it in view of the context in the case of an n-gram. In the present example, the starting location, the current position, the destination location, via points, the starting time, the current travel time, the expected arrival time are known by the navigation unit 102. Such navigation information can be provided to the language model database 110 to change the language model statistics in the language model database 110 that are utilized to recognize the speech input. The entries in the list of entries, i.e. the database of known words, can be emphasized by increasing the probability of occurrence of the entries also contained in the navigation information. By way of example, in case the user is traveling to Hamburg and it is known from the navigation unit 120 that the user will drive to a certain address, such as “Jungfernsteg” and will arrive there at approximately 3:15 pm, the navigation information such as “Hamburg”, “Jungfernsteg”, “3:15” can be fed to the language model database, the likelihood of these list entries being increased. When the user of the speech recognition unit 108 utters one of the words contained in the navigation information, the probability that the speech recognition unit 108 correctly interprets the speech input is remarkably increased. In the present invention, the language model statistics is not only calculated on the basis of some general text before the assembly of the products as it is known in the art, but the language model statistics is updated taking into account any given environmental information, such as the navigation information.
In FIG. 2, another example of an implementation of the system 200 is shown in which the navigation information provided by a navigation unit 206 can increase the functionality of the man-machine-interface. In FIG. 2, an input unit 202 is provided for the entry of text via, for example, a keyboard. The input unit 202 may also be a unit capable of recognizing a hand-written character input. The input unit 202 may contain hard keys, such as a keyboard, or may contain a display showing soft keys that can be selected by touching the display. The system 200 may help increase the speed of entry of text into the input unit 202. To speed up the input process, an automatic completion is utilized as soon as the first part of the input text is input. By way of example, after the first two or three letters are input by user, the input unit 202 may suggest completed words. For suggesting a completed text input, based on the first part of the text input, the input unit 202 makes reference to the language model database 204, which may correspond to the language model database 110 described in connection with FIG. 1. In the language model database 204, the different expressions known to the system are stored as list of entries, the database language model database 204 furthermore storing the likelihood of occurrence of the different words. The text input into the input unit 202 may be a message, memo, note or letter, as discussed in connection with FIG. 1. However, in connection with FIG. 2, the text is manually input and not dictated. In both situations, navigation information provided by navigation unit 206 can help to improve and to speed up the text input. The navigation unit 206 may have a separate input unit 202. However, the input unit 202 may also be the same as input unit 208. As discussed in connection with FIG. 1, the map data contained in the map database 210 may assist in calculating the optimum route to a predetermined destination. The navigation information given by the navigation unit 206 is fed to the language model database 204, where the weight of list entries also contained in the navigation information may be increased. The system 200 may further increase the speed of text as the system 200 can suggest faster and more reliable the correct completed text input. In the above-referenced example, in case the user wants to input the location “Hamburg” and may have already input the first two letters “Ha”, the input unit may propose “Hamburg” as the most likely candidate of a completed text instead of other expressions beginning with the same two letters.
The input unit 202 may be an input unit of a cellular phone where the text input is not unambiguous as, by pressing an actuating element, different characters may be selected. For supporting the text input using systems such as T9, the system 200 may be improved by providing the navigation information to the language model database 204 where the most likely candidate of the input text is suggested based on the actuated actuating element and based on the received navigation information.
In another example of an implementation, the system 200 may assist in a method in which the text input is a phonetic representation of the text as it is common in many Asian languages. In case of a homophone, one of the different possible meanings of the phonetic representation has to be selected. This selection or suggestion of the most likely candidate of the different possible meanings, i.e. of the completed text input may similar be improved by using the navigation information.
In yet another example of an implementation, the text is input using a handwritten form, a handwriting recognition unit being provided as part of the input unit to recognize the handwritten text input. To recognize the different characters input in handwritten form, the provided navigation information may also be helpful. By way of example, it is difficult to unambiguously differentiate between the number “1”, the large character “I” and the small character “l”. Accordingly, the recognition rate of the handwritten input can also be improved by taking into account the navigation information provided by the navigation unit.
In both implementations illustrated in FIG. 1 and FIG. 2, it may happen that information provided in the navigation unit 102 and 206, such as an address or a destination location, such as a small village, is not contained in the list of entries. In this situation, this information may be provided to the language model database 110 and 204 as a new list entry, and this new list entry may be weighted more heavily than other list entries of the same type.
In FIG. 3, a flow diagram 300 is illustrated that provides example steps to increase the speech recognition rate in a system, such as the speech recognition system 100, in FIG. 1, utilizing current information, such as information provided by a trip to a predetermined destination. The process starts in step 302. In step 304, speech input is detected. In an additional step 306, which may occur before the detection of the speech input, navigation information can be detected. By way of example, every time the navigation unit 102 is utilized, the current navigation information may be fed to the speech recognition unit 108, so that the latter is always updated with the current navigation information. In step 308, the list entries contained in the language model database 110 and contained in the navigation information, such as the name of the destination location or the name of via points or any figures, such as the arrival time, can be weighted more heavily—meaning that the weight is increased compared to a normal likelihood of a word given for a predetermined language. In step 310, the speech recognition is carried out, the speech recognition step may utilize statistical models, such as a hidden Markov models (HMMs) to recognize the speech input. The speech input, as it was recognized by the speech recognition unit, can be output to the user, as shown in step 312. Usually, the most likely word or sequence of words is output. The recognized speech output presented to the user can be confirmed or corrected by the user. Furthermore, it is possible that in the case of a single word, the most likely entries of the recognized word may be presented to the user on a display allowing the user to select the desired expression. The method ends in step 314.
In FIG. 4, a flow diagram is provided that illustrate example primary steps for text input using the system 200 of FIG. 2. The process starts at step 402. In step 402, the first characters are input into the input unit 202. Depending on the sequence of characters, it may be enough to input two or three characters that form a basis for the input unit 202 to suggest a completed input. The navigation information is detected in step 406. It should be understood that the navigation information could also have been input before the user starts to input the text into the input unit 202. In step 408, the list entries that are also contained in the navigation information are weighted by increasing their weight compared to the weight before the navigation information was input. In step 410, the first characters input in step 404 are compared to the weighted list of entries and in step 408 and the most likely candidate or the most likely candidates are displayed to the user, at step 412, who can either confirm the proposed most likely candidate or select one of the most likely candidates or discard the suggestions. The method ends in step 414.
Summarizing, the proposed systems and methods help to increase the comfort and speech recognition accuracy and are especially useful in connection with languages where words are input with some phonetical transcription, such as Asian languages, and then converted into characters by selection of a user. This conversion process requires statistical models which can profit from the proposed consideration of information from the navigation unit.
In addition to the navigation unit additional information may be provided by the infotainment unit connected to the systems 100 and 200 shown in FIGS. 1 and 2. This infotainment unit can provide the current infotainment status, such as the piece of music currently played or the film that is currently streamed or the TV station that is currently received. The given environmental information helps to increase the recognition rate in the case of a speech recognition or in the case a completed text input should be suggested.
The invention may also be applied in case of a text input that is not unambiguous. By way of example, when the text is input using an input module in which each key represents more than one character, e.g. in case of a cellular phone, the navigation information can also help to recognize the first part of the text input or the complete text input using the navigation information. By way of example, in a cellular phone in case the user presses the buttons “2” and “3” of the cellular phone, this could be the character sequence “AD”, “AF” or “BE”. For dete fining which text input is meant, the navigation information of a combined system having a cellular phone and a navigation module can help to render the ambiguous text input unambiguous.
The invention may also be applied in the following context. In some languages such as Japanese, Chinese or Korean, the user may input the text in a phonetic representation of the text. In case of homophones, different expressions having the input phonetic representation are displayed to the user for further selection, or the most likely expression may be automatically selected. In this example, not only the first part but the complete phonetic representation of the text input is detected and the most likely candidate of the text input, i.e. one expression among different expressions having the same phonetic representation can be selected taking into account the navigation information.
In another implementation, it is furthermore possible to additionally consider information of other sources, e.g. in a vehicle application the current infotainment status meaning that it is determined which CD or DVD is currently played or which radio or TV station is currently received. The list entries relating to said current infotainment status can be determined and the likelihood of said entertainment related list entries can also be increased.
The invention can furthermore be applied in connection with hand-writing recognition, where the text is input in hand-written form into an input unit, the input unit recognizing the hand-writing. In this context it can also be helpful to use the available navigation information such as destination location, arrival time, etc. This information can help to increase the recognition rate of the hand-writing recognition, when the user writes down the message to be transmitted or a memo for future use.
The proposed systems and methods have special relevance for Asian languages, where input methods with statistical support are much more common than in Latin character-based languages. In those languages words are often input with some phonetic transcription such as Kana or Pinyin and then converted to Chinese characters. In general, the invention is especially helpful when used in connection with languages having lots of homophones, i.e. expressions having the same or similar articulation, but are written differently.
It will be understood, and is appreciated by persons skilled in the art, that one or more processes, sub-processes, or process steps described in connection with FIGS. 1-4 may be performed by hardware and/or software. If the process is performed by software, the software may reside in software memory (not shown) in a suitable electronic processing component or system such as, one or more of the functional components or modules schematically depicted in FIGS. 1-4. The software in software memory may include an ordered listing of executable instructions for implementing logical functions (that is, “logic” that may be implemented either in digital form such as digital circuitry or source code or in analog form such as analog circuitry or an analog source such an analog electrical, sound or video signal), and may selectively be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that may selectively fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. In the context of this disclosure, a “computer-readable medium” is any means that may contain, store or communicate the program for use by or in connection with the instruction execution system, apparatus, or device. The computer readable medium may selectively be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device. More specific examples, but nonetheless a non-exhaustive list, of computer-readable media would include the following: a portable computer diskette (magnetic), a RAM (electronic), a read-only memory “ROM” (electronic), an erasable programmable read-only memory (EPROM or Flash memory) (electronic) and a portable compact disc read-only memory “CDROM” (optical). Note that the computer-readable medium may even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
The foregoing description of implementations has been presented for purposes of illustration and description. It is not exhaustive and does not limit the claimed inventions to the precise form disclosed. Modifications and variations are possible in light of the above description or may be acquired from practicing the invention. The claims and their equivalents define the scope of the invention.

Claims (13)

1. A method for recognizing a user's speech input comprising the following steps:
utilizing a navigation unit for:
detecting navigation information of a trip to a predetermined destination, and
modifying language model statistics in a language model database containing a list of entries for list entries that are in the language model database and that are contained in the navigation information;
detecting whether words in the navigation information are contained in the list of entries,
where any words in the navigation information that are not contained in the list of entries are added to the list of entries as new entries, and where the likelihood of the new entries is set higher than an average likelihood of the entries in the list of entries;
detecting the user's speech input; and
recognizing the user's speech input by comparing the user's speech input to the list of entries using the modified language model statistics to determine the most likely entry best matching the user's speech input,
where the navigation information contains at least one of the following information: an overall travel time of the trip, the current travel time of the trip, and an expected arrival time.
2. The method of claim 1, where the language model statistics indicate a likelihood of the different entries in the list of entries, where the likelihood of a list entry contained in the navigation information is increased in the step of modifying language model statistics.
3. The method of claim 1, where the navigation information further contains at least one of the following information:
a starting location of the trip, a destination location of the trip, via points of the trip, and the current position.
4. The method of claim 1, further comprising the steps of:
detecting a current infotainment status;
determining list entries relating to said current infotainment status; and
increasing the likelihood of said infotainment-related list entries.
5. A method for supporting a text input in an input unit, the method comprising the steps of:
utilizing a navigation unit for:
detecting navigation information of a trip to a predetermined destination, and
modifying language model statistics in a language model database containing a list of entries for list entries that are in the language model database and that are contained in the navigation information;
detecting whether words in the navigation information are contained in the list of entries; where any words in the navigation information that are not contained in the list of entries are added to the list of entries as new entries, and where the likelihood of the new entries is set higher than an average likelihood of the entries in the list of entries;
receiving text input at the input unit of an application that is not the navigation unit;
detecting at least a first part of the text input;
recognizing the text input by comparing said at least first part of the text input to a list of entries in the language model database based on the modified language model statistics in order to suggest a most likely candidate of a completed text input; and
suggesting said most likely candidate of the completed text input to a user of the input unit,
where the navigation information contains at least one of the following information: an overall travel time of the trip, the current travel time of the trip, and an expected arrival time.
6. The method of claim 5, where the language model statistics indicate a likelihood of the different entries in the list of entries, where the likelihood of a list entry contained in the navigation information is increased in the step of modifying language model statistics.
7. The method of claim 5, where the navigation information further contains at least one of the following information:
a starting location of the trip, a destination location of the trip, via points of the trip, and the current position.
8. The method of claim 5, further comprising the steps of:
detecting a current infotainment status;
determining list entries relating to said current infotainment status; and
increasing the likelihood of said infotainment-related list entries.
9. The method of claim 5, further comprising the step of recognizing said first part of the text input using the navigation information based on the modified language model statistics when the first part of the text input is not unambiguous.
10. The method of claim 5, where the text input is input into the input unit in a hand-written form, where the text input is recognized taking into account the navigation information based on the modified language model statistics.
11. The method of claim 5, where the text is input in a phonetic representation and the complete text input is detected, where the phonetic representation is converted to different expressions having substantially the same phonetic representation, the most likely expression being selected taking into account the navigation information based on the modified language model statistics.
12. A speech recognition system recognizing a user's speech input, comprising:
A navigation unit configured to generate navigation information of a trip to a predetermined destination;
a speech detector configured to detect the user's speech input;
a language model database comprising a list of entries and language model statistics for the list of entries; and
a speech recognition unit configured to receive words in the navigation information from the navigation unit, and to adjust the language model statistics for entries in the list of entries that are contained in the navigation information, the speech recognition unit being configured to recognize the user's speech input by comparing the speech input to the list of entries in the language model database using the modified language model statistics in order to determine the most likely entry best matching the user's speech input,
where the speech recognition unit detects whether words in the navigation information are contained in the list of entries,
where any words in the navigation information that are not contained in the list of entries are added to the list of entries as new entries, and where the likelihood of the new entries is set higher than an average likelihood of the entries in the list of entries, and
where the navigation information contains at least one of the following information: an overall travel time of the trip, the current travel time of the trip, and an expected arrival time.
13. A system for supporting a text input in an input unit, comprising:
a navigation unit configured to generate navigation information of a trip to a predetermined destination;
an input unit for inputting the text input;
a language model database comprising a list of entries and language model statistics for the list of entries; and
a text recognition unit configured to receive words in the navigation information from the navigation unit, and to adjust the language model statistics for entries in the list of entries that are contained in the navigation information, the text input recognition unit being configured to recognize the text input by comparing at least a first part of the text input to the list of entries in the language model database based on the modified language model statistics in order to suggest a most likely candidate of a completed text input,
where the text recognition unit detects whether words in the navigation information are contained in the list of entries,
where any words in the navigation information that are not contained in the list of entries are added to the list of entries as new entries, and where the likelihood of the new entries is set higher than an average likelihood of the entries in the list of entries, and
where the navigation information contains at least one of the following information: an overall travel time of the trip, the current travel time of the trip, and an expected arrival time.
US12/693,392 2009-01-23 2010-01-25 Text and speech recognition system using navigation information Active 2030-12-19 US8340958B2 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
EP09000983.8A EP2211336B1 (en) 2009-01-23 2009-01-23 Improved speech input using navigation information
EPEP09000983.8 2009-01-23
EP09000983 2009-01-23

Publications (2)

Publication Number Publication Date
US20100191520A1 US20100191520A1 (en) 2010-07-29
US8340958B2 true US8340958B2 (en) 2012-12-25

Family

ID=40524882

Family Applications (1)

Application Number Title Priority Date Filing Date
US12/693,392 Active 2030-12-19 US8340958B2 (en) 2009-01-23 2010-01-25 Text and speech recognition system using navigation information

Country Status (2)

Country Link
US (1) US8340958B2 (en)
EP (1) EP2211336B1 (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130046539A1 (en) * 2011-08-16 2013-02-21 International Business Machines Corporation Automatic Speech and Concept Recognition
US20130332147A1 (en) * 2012-06-11 2013-12-12 Nvoq Incorporated Apparatus and Methods to Update a Language Model in a Speech Recognition System
US9318102B2 (en) * 2012-12-07 2016-04-19 Postech Academy—Industry Foundation Method and apparatus for correcting speech recognition error
DE102017216571A1 (en) 2017-09-19 2019-03-21 Volkswagen Aktiengesellschaft motor vehicle

Families Citing this family (153)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8677377B2 (en) 2005-09-08 2014-03-18 Apple Inc. Method and apparatus for building an intelligent automated assistant
US9318108B2 (en) 2010-01-18 2016-04-19 Apple Inc. Intelligent automated assistant
US10002189B2 (en) 2007-12-20 2018-06-19 Apple Inc. Method and apparatus for searching using an active ontology
US9330720B2 (en) 2008-01-03 2016-05-03 Apple Inc. Methods and apparatus for altering audio output signals
US8996376B2 (en) 2008-04-05 2015-03-31 Apple Inc. Intelligent text-to-speech conversion
US20100030549A1 (en) 2008-07-31 2010-02-04 Lee Michael M Mobile device having human language translation capability with positional feedback
US8676904B2 (en) 2008-10-02 2014-03-18 Apple Inc. Electronic devices with voice command and contextual data processing capabilities
US20120311585A1 (en) 2011-06-03 2012-12-06 Apple Inc. Organizing task items that represent tasks to perform
US10241644B2 (en) 2011-06-03 2019-03-26 Apple Inc. Actionable reminder entries
US10241752B2 (en) 2011-09-30 2019-03-26 Apple Inc. Interface for a virtual digital assistant
US9431006B2 (en) 2009-07-02 2016-08-30 Apple Inc. Methods and apparatuses for automatic speech recognition
US10276170B2 (en) 2010-01-18 2019-04-30 Apple Inc. Intelligent automated assistant
US8682667B2 (en) 2010-02-25 2014-03-25 Apple Inc. User profiling for selecting user specific voice input processing information
US8538760B2 (en) * 2010-07-13 2013-09-17 Qualcomm Incorporated Methods and apparatuses for identifying audible samples for use in a speech recognition capability of a mobile device
US9262612B2 (en) 2011-03-21 2016-02-16 Apple Inc. Device access using voice authentication
US10057736B2 (en) 2011-06-03 2018-08-21 Apple Inc. Active transport based notifications
US10134385B2 (en) 2012-03-02 2018-11-20 Apple Inc. Systems and methods for name pronunciation
US9620111B1 (en) * 2012-05-01 2017-04-11 Amazon Technologies, Inc. Generation and maintenance of language model
US9280610B2 (en) 2012-05-14 2016-03-08 Apple Inc. Crowd sourcing information to fulfill user requests
US10417037B2 (en) 2012-05-15 2019-09-17 Apple Inc. Systems and methods for integrating third party services with a digital assistant
US9721563B2 (en) 2012-06-08 2017-08-01 Apple Inc. Name recognition system
US10354650B2 (en) 2012-06-26 2019-07-16 Google Llc Recognizing speech with mixed speech recognition models to generate transcriptions
US9547647B2 (en) 2012-09-19 2017-01-17 Apple Inc. Voice-based media searching
US9570076B2 (en) * 2012-10-30 2017-02-14 Google Technology Holdings LLC Method and system for voice recognition employing multiple voice-recognition techniques
US9704486B2 (en) 2012-12-11 2017-07-11 Amazon Technologies, Inc. Speech recognition power management
KR20230137475A (en) 2013-02-07 2023-10-04 애플 인크. Voice trigger for a digital assistant
US9542947B2 (en) 2013-03-12 2017-01-10 Google Technology Holdings LLC Method and apparatus including parallell processes for voice recognition
US10652394B2 (en) 2013-03-14 2020-05-12 Apple Inc. System and method for processing voicemail
US10748529B1 (en) 2013-03-15 2020-08-18 Apple Inc. Voice activated device for use with a voice-based digital assistant
DE102013007964B4 (en) 2013-05-10 2022-08-18 Audi Ag Automotive input device with character recognition
US9058805B2 (en) 2013-05-13 2015-06-16 Google Inc. Multiple recognizer speech recognition
WO2014197334A2 (en) 2013-06-07 2014-12-11 Apple Inc. System and method for user-specified pronunciation of words for speech synthesis and recognition
WO2014197335A1 (en) 2013-06-08 2014-12-11 Apple Inc. Interpreting and acting upon commands that involve sharing information with remote devices
EP3937002A1 (en) 2013-06-09 2022-01-12 Apple Inc. Device, method, and graphical user interface for enabling conversation persistence across two or more instances of a digital assistant
US10176167B2 (en) 2013-06-09 2019-01-08 Apple Inc. System and method for inferring user intent from speech inputs
US20150134244A1 (en) * 2013-11-12 2015-05-14 Mitsubishi Electric Research Laboratories, Inc. Method for Predicting Travel Destinations Based on Historical Data
US10296160B2 (en) 2013-12-06 2019-05-21 Apple Inc. Method for extracting salient dialog usage from live data
US9311930B2 (en) * 2014-01-28 2016-04-12 Qualcomm Technologies International, Ltd. Audio based system and method for in-vehicle context classification
JP6427755B2 (en) * 2014-02-24 2018-11-28 パナソニックIpマネジメント株式会社 Data input device, data input method, program, and in-vehicle device
CN104978015B (en) 2014-04-14 2018-09-18 博世汽车部件(苏州)有限公司 Navigation system and its control method with languages self application function
US9430463B2 (en) 2014-05-30 2016-08-30 Apple Inc. Exemplar-based natural language processing
US9785630B2 (en) * 2014-05-30 2017-10-10 Apple Inc. Text prediction using combined word N-gram and unigram language models
US9633004B2 (en) 2014-05-30 2017-04-25 Apple Inc. Better resolution when referencing to concepts
US10170123B2 (en) 2014-05-30 2019-01-01 Apple Inc. Intelligent assistant for home automation
US9715875B2 (en) 2014-05-30 2017-07-25 Apple Inc. Reducing the need for manual start/end-pointing and trigger phrases
EP3480811A1 (en) 2014-05-30 2019-05-08 Apple Inc. Multi-command single utterance input method
US9338493B2 (en) 2014-06-30 2016-05-10 Apple Inc. Intelligent automated assistant for TV user interactions
US9818400B2 (en) 2014-09-11 2017-11-14 Apple Inc. Method and apparatus for discovering trending terms in speech requests
US10074360B2 (en) 2014-09-30 2018-09-11 Apple Inc. Providing an indication of the suitability of speech recognition
US9668121B2 (en) 2014-09-30 2017-05-30 Apple Inc. Social reminders
US10127911B2 (en) 2014-09-30 2018-11-13 Apple Inc. Speaker identification and unsupervised speaker adaptation techniques
US10152299B2 (en) 2015-03-06 2018-12-11 Apple Inc. Reducing response latency of intelligent automated assistants
US9721566B2 (en) 2015-03-08 2017-08-01 Apple Inc. Competing devices responding to voice triggers
US9886953B2 (en) 2015-03-08 2018-02-06 Apple Inc. Virtual assistant activation
US10567477B2 (en) 2015-03-08 2020-02-18 Apple Inc. Virtual assistant continuity
CN107532914A (en) * 2015-05-05 2018-01-02 纽昂斯通讯公司 Vehicle-mounted voice destination inputs(VDE)Automaticdata switching method in navigation solution
US10460227B2 (en) 2015-05-15 2019-10-29 Apple Inc. Virtual assistant in a communication session
US10083688B2 (en) 2015-05-27 2018-09-25 Apple Inc. Device voice control for selecting a displayed affordance
US9578173B2 (en) 2015-06-05 2017-02-21 Apple Inc. Virtual assistant aided communication with 3rd party service in a communication session
US11025565B2 (en) 2015-06-07 2021-06-01 Apple Inc. Personalized prediction of responses for instant messaging
US20160378747A1 (en) 2015-06-29 2016-12-29 Apple Inc. Virtual assistant for media playback
US10671428B2 (en) 2015-09-08 2020-06-02 Apple Inc. Distributed personal assistant
US10747498B2 (en) 2015-09-08 2020-08-18 Apple Inc. Zero latency digital assistant
US11010550B2 (en) 2015-09-29 2021-05-18 Apple Inc. Unified language modeling framework for word prediction, auto-completion and auto-correction
US10366158B2 (en) 2015-09-29 2019-07-30 Apple Inc. Efficient word encoding for recurrent neural network language models
US11587559B2 (en) 2015-09-30 2023-02-21 Apple Inc. Intelligent device identification
US10691473B2 (en) 2015-11-06 2020-06-23 Apple Inc. Intelligent automated assistant in a messaging environment
US10049668B2 (en) 2015-12-02 2018-08-14 Apple Inc. Applying neural network language models to weighted finite state transducers for automatic speech recognition
US10223066B2 (en) 2015-12-23 2019-03-05 Apple Inc. Proactive assistance based on dialog communication between devices
US10446143B2 (en) 2016-03-14 2019-10-15 Apple Inc. Identification of voice inputs providing credentials
US9934775B2 (en) 2016-05-26 2018-04-03 Apple Inc. Unit-selection text-to-speech synthesis based on predicted concatenation parameters
US9972304B2 (en) 2016-06-03 2018-05-15 Apple Inc. Privacy preserving distributed evaluation framework for embedded personalized systems
US11227589B2 (en) 2016-06-06 2022-01-18 Apple Inc. Intelligent list reading
US10249300B2 (en) 2016-06-06 2019-04-02 Apple Inc. Intelligent list reading
US10049663B2 (en) 2016-06-08 2018-08-14 Apple, Inc. Intelligent automated assistant for media exploration
DK179309B1 (en) 2016-06-09 2018-04-23 Apple Inc Intelligent automated assistant in a home environment
US10192552B2 (en) 2016-06-10 2019-01-29 Apple Inc. Digital assistant providing whispered speech
US10067938B2 (en) 2016-06-10 2018-09-04 Apple Inc. Multilingual word prediction
US10509862B2 (en) 2016-06-10 2019-12-17 Apple Inc. Dynamic phrase expansion of language input
US10586535B2 (en) 2016-06-10 2020-03-10 Apple Inc. Intelligent digital assistant in a multi-tasking environment
US10490187B2 (en) 2016-06-10 2019-11-26 Apple Inc. Digital assistant providing automated status report
DK179415B1 (en) 2016-06-11 2018-06-14 Apple Inc Intelligent device arbitration and control
DK179343B1 (en) 2016-06-11 2018-05-14 Apple Inc Intelligent task discovery
DK201670540A1 (en) 2016-06-11 2018-01-08 Apple Inc Application integration with a digital assistant
DK179049B1 (en) 2016-06-11 2017-09-18 Apple Inc Data driven natural language event detection and classification
US10474753B2 (en) 2016-09-07 2019-11-12 Apple Inc. Language identification using recurrent neural networks
US10043516B2 (en) 2016-09-23 2018-08-07 Apple Inc. Intelligent automated assistant
US11281993B2 (en) 2016-12-05 2022-03-22 Apple Inc. Model and ensemble compression for metric learning
US10593346B2 (en) 2016-12-22 2020-03-17 Apple Inc. Rank-reduced token representation for automatic speech recognition
US11204787B2 (en) 2017-01-09 2021-12-21 Apple Inc. Application integration with a digital assistant
DK201770383A1 (en) 2017-05-09 2018-12-14 Apple Inc. User interface for correcting recognition errors
US10417266B2 (en) 2017-05-09 2019-09-17 Apple Inc. Context-aware ranking of intelligent response suggestions
US10726832B2 (en) 2017-05-11 2020-07-28 Apple Inc. Maintaining privacy of personal information
DK201770439A1 (en) 2017-05-11 2018-12-13 Apple Inc. Offline personal assistant
US10395654B2 (en) 2017-05-11 2019-08-27 Apple Inc. Text normalization based on a data-driven learning network
US11301477B2 (en) 2017-05-12 2022-04-12 Apple Inc. Feedback analysis of a digital assistant
DK179496B1 (en) 2017-05-12 2019-01-15 Apple Inc. USER-SPECIFIC Acoustic Models
DK179745B1 (en) 2017-05-12 2019-05-01 Apple Inc. SYNCHRONIZATION AND TASK DELEGATION OF A DIGITAL ASSISTANT
DK201770428A1 (en) 2017-05-12 2019-02-18 Apple Inc. Low-latency intelligent automated assistant
DK201770432A1 (en) 2017-05-15 2018-12-21 Apple Inc. Hierarchical belief states for digital assistants
DK201770431A1 (en) 2017-05-15 2018-12-20 Apple Inc. Optimizing dialogue policy decisions for digital assistants using implicit feedback
US20180336275A1 (en) 2017-05-16 2018-11-22 Apple Inc. Intelligent automated assistant for media exploration
US10311144B2 (en) 2017-05-16 2019-06-04 Apple Inc. Emoji word sense disambiguation
US20180336892A1 (en) 2017-05-16 2018-11-22 Apple Inc. Detecting a trigger of a digital assistant
US10403278B2 (en) 2017-05-16 2019-09-03 Apple Inc. Methods and systems for phonetic matching in digital assistant services
DK179560B1 (en) 2017-05-16 2019-02-18 Apple Inc. Far-field extension for digital assistant services
US10657328B2 (en) 2017-06-02 2020-05-19 Apple Inc. Multi-task recurrent neural network architecture for efficient morphology handling in neural language modeling
US20190004821A1 (en) * 2017-06-29 2019-01-03 Microsoft Technology Licensing, Llc Command input using robust input parameters
US10445429B2 (en) 2017-09-21 2019-10-15 Apple Inc. Natural language understanding using vocabularies with compressed serialized tries
US10755051B2 (en) 2017-09-29 2020-08-25 Apple Inc. Rule-based natural language processing
CN107678561A (en) * 2017-09-29 2018-02-09 百度在线网络技术(北京)有限公司 Phonetic entry error correction method and device based on artificial intelligence
US10636424B2 (en) 2017-11-30 2020-04-28 Apple Inc. Multi-turn canned dialog
US10733982B2 (en) 2018-01-08 2020-08-04 Apple Inc. Multi-directional dialog
US10733375B2 (en) 2018-01-31 2020-08-04 Apple Inc. Knowledge-based framework for improving natural language understanding
US10789959B2 (en) 2018-03-02 2020-09-29 Apple Inc. Training speaker recognition models for digital assistants
US10592604B2 (en) 2018-03-12 2020-03-17 Apple Inc. Inverse text normalization for automatic speech recognition
US10818288B2 (en) 2018-03-26 2020-10-27 Apple Inc. Natural assistant interaction
US10909331B2 (en) 2018-03-30 2021-02-02 Apple Inc. Implicit identification of translation payload with neural machine translation
US11145294B2 (en) 2018-05-07 2021-10-12 Apple Inc. Intelligent automated assistant for delivering content from user experiences
US10928918B2 (en) 2018-05-07 2021-02-23 Apple Inc. Raise to speak
US10984780B2 (en) 2018-05-21 2021-04-20 Apple Inc. Global semantic word embeddings using bi-directional recurrent neural networks
DK201870355A1 (en) 2018-06-01 2019-12-16 Apple Inc. Virtual assistant operation in multi-device environments
US10892996B2 (en) 2018-06-01 2021-01-12 Apple Inc. Variable latency device coordination
US11386266B2 (en) 2018-06-01 2022-07-12 Apple Inc. Text correction
DK180639B1 (en) 2018-06-01 2021-11-04 Apple Inc DISABILITY OF ATTENTION-ATTENTIVE VIRTUAL ASSISTANT
DK179822B1 (en) 2018-06-01 2019-07-12 Apple Inc. Voice interaction at a primary device to access call functionality of a companion device
US11076039B2 (en) 2018-06-03 2021-07-27 Apple Inc. Accelerated task performance
US11010561B2 (en) 2018-09-27 2021-05-18 Apple Inc. Sentiment prediction from textual data
US10839159B2 (en) 2018-09-28 2020-11-17 Apple Inc. Named entity normalization in a spoken dialog system
US11170166B2 (en) 2018-09-28 2021-11-09 Apple Inc. Neural typographical error modeling via generative adversarial networks
US11462215B2 (en) 2018-09-28 2022-10-04 Apple Inc. Multi-modal inputs for voice commands
US11475898B2 (en) 2018-10-26 2022-10-18 Apple Inc. Low-latency multi-speaker speech recognition
US10719340B2 (en) 2018-11-06 2020-07-21 Microsoft Technology Licensing, Llc Command bar user interface
DE102018221712B4 (en) 2018-12-13 2022-09-22 Volkswagen Aktiengesellschaft Method for operating an interactive information system for a vehicle, and a vehicle
EP3671120B1 (en) 2018-12-17 2023-07-26 HERE Global B.V. Enhancing the accuracy for device localization
US11638059B2 (en) 2019-01-04 2023-04-25 Apple Inc. Content playback on multiple devices
US11017771B2 (en) * 2019-01-18 2021-05-25 Adobe Inc. Voice command matching during testing of voice-assisted application prototypes for languages with non-phonetic alphabets
US11348573B2 (en) 2019-03-18 2022-05-31 Apple Inc. Multimodality in digital assistant systems
US11217251B2 (en) 2019-05-06 2022-01-04 Apple Inc. Spoken notifications
US11475884B2 (en) 2019-05-06 2022-10-18 Apple Inc. Reducing digital assistant latency when a language is incorrectly determined
US11423908B2 (en) 2019-05-06 2022-08-23 Apple Inc. Interpreting spoken requests
US11307752B2 (en) 2019-05-06 2022-04-19 Apple Inc. User configurable task triggers
US11140099B2 (en) 2019-05-21 2021-10-05 Apple Inc. Providing message response suggestions
US11289073B2 (en) 2019-05-31 2022-03-29 Apple Inc. Device text to speech
US11496600B2 (en) 2019-05-31 2022-11-08 Apple Inc. Remote execution of machine-learned models
DK201970510A1 (en) 2019-05-31 2021-02-11 Apple Inc Voice identification in digital assistant systems
DK180129B1 (en) 2019-05-31 2020-06-02 Apple Inc. User activity shortcut suggestions
US11360641B2 (en) 2019-06-01 2022-06-14 Apple Inc. Increasing the relevance of new available information
WO2021056255A1 (en) 2019-09-25 2021-04-01 Apple Inc. Text detection using global geometry estimators
JP7230793B2 (en) * 2019-12-24 2023-03-01 トヨタ自動車株式会社 Information processing device, information processing method, and system
CN111538890B (en) * 2020-04-02 2023-12-12 中国铁道科学研究院集团有限公司 Indoor guiding method and system based on voice recognition
US11755276B2 (en) 2020-05-12 2023-09-12 Apple Inc. Reducing description length based on confidence
CN112163685A (en) * 2020-09-11 2021-01-01 广州宸祺出行科技有限公司 Intelligent trip matching method and system based on voice AI

Citations (49)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5752232A (en) * 1994-11-14 1998-05-12 Lucent Technologies Inc. Voice activated device and method for providing access to remotely retrieved data
US6108631A (en) * 1997-09-24 2000-08-22 U.S. Philips Corporation Input system for at least location and/or street names
US6334102B1 (en) * 1999-09-13 2001-12-25 International Business Machines Corp. Method of adding vocabulary to a speech recognition system
US20020045463A1 (en) * 2000-10-13 2002-04-18 Zheng Chen Language input system for mobile devices
US20020087309A1 (en) * 2000-12-29 2002-07-04 Lee Victor Wai Leung Computer-implemented speech expectation-based probability method and system
US20020111810A1 (en) * 2001-02-15 2002-08-15 Khan M. Salahuddin Spatially built word list for automatic speech recognition program and method for formation thereof
US20030182131A1 (en) * 2002-03-25 2003-09-25 Arnold James F. Method and apparatus for providing speech-driven routing between spoken language applications
US6646573B1 (en) * 1998-12-04 2003-11-11 America Online, Inc. Reduced keyboard text input system for the Japanese language
US20040139404A1 (en) 2002-11-29 2004-07-15 Takashi Kawashima Text editing assistor
US20040260543A1 (en) * 2001-06-28 2004-12-23 David Horowitz Pattern cross-matching
US6850886B2 (en) * 1999-10-21 2005-02-01 Sony Corporation System and method for speech verification using an efficient confidence measure
WO2005066934A1 (en) 2004-01-07 2005-07-21 Toyota Infotechnology Center Co., Ltd. Method and system for speech recognition using grammar weighted based upon location information
US20060074660A1 (en) * 2004-09-29 2006-04-06 France Telecom Method and apparatus for enhancing speech recognition accuracy by using geographic data to filter a set of words
US20060190258A1 (en) * 2004-12-16 2006-08-24 Jan Verhasselt N-Best list rescoring in speech recognition
US20070162281A1 (en) * 2006-01-10 2007-07-12 Nissan Motor Co., Ltd. Recognition dictionary system and recognition dictionary system updating method
US20070192311A1 (en) * 2006-02-10 2007-08-16 Pun Samuel Y L Method And System Of Identifying An Ideographic Character
US20070208555A1 (en) * 2006-03-06 2007-09-06 International Business Machines Corporation Dynamically adjusting speech grammar weights based on usage
WO2007112543A1 (en) 2006-04-06 2007-10-11 Research In Motion Limited Handheld electronic device and method for learning contextual data during disambiguation of text input
US20080126314A1 (en) 2006-11-27 2008-05-29 Sony Ericsson Mobile Communications Ab Word prediction
US20080133228A1 (en) * 2006-11-30 2008-06-05 Rao Ashwin P Multimodal speech recognition system
US20080154612A1 (en) 2006-12-26 2008-06-26 Voice Signal Technologies, Inc. Local storage and use of search results for voice-enabled mobile communications devices
US20080177541A1 (en) * 2006-09-05 2008-07-24 Honda Motor Co., Ltd. Voice recognition device, voice recognition method, and voice recognition program
US20080221898A1 (en) * 2007-03-07 2008-09-11 Cerra Joseph P Mobile navigation environment speech processing facility
US20080235017A1 (en) * 2007-03-22 2008-09-25 Honda Motor Co., Ltd. Voice interaction device, voice interaction method, and voice interaction program
US7430473B2 (en) * 2004-10-01 2008-09-30 Bose Corporation Vehicle navigation display
US20080270118A1 (en) * 2007-04-26 2008-10-30 Microsoft Corporation Recognition architecture for generating Asian characters
US20080288252A1 (en) * 2007-03-07 2008-11-20 Cerra Joseph P Speech recognition of speech recorded by a mobile communication facility
US20090030698A1 (en) * 2007-03-07 2009-01-29 Cerra Joseph P Using speech recognition results based on an unstructured language model with a music system
US20090171659A1 (en) * 2007-12-31 2009-07-02 Motorola, Inc. Methods and apparatus for implementing distributed multi-modal applications
US20090171669A1 (en) * 2007-12-31 2009-07-02 Motorola, Inc. Methods and Apparatus for Implementing Distributed Multi-Modal Applications
US20090187399A1 (en) * 2008-01-22 2009-07-23 O'dell Robert B Using Homophones and Near-Homophones to Improve Methods of Computer Text Entry for Chinese Characters
US20090228281A1 (en) * 2008-03-07 2009-09-10 Google Inc. Voice Recognition Grammar Selection Based on Context
US20090248415A1 (en) * 2008-03-31 2009-10-01 Yap, Inc. Use of metadata to post process speech recognition output
US20090287681A1 (en) * 2008-05-14 2009-11-19 Microsoft Corporation Multi-modal search wildcards
US7634409B2 (en) * 2005-08-31 2009-12-15 Voicebox Technologies, Inc. Dynamic speech sharpening
US20090313017A1 (en) * 2006-07-07 2009-12-17 Satoshi Nakazawa Language model update device, language Model update method, and language model update program
US20090326927A1 (en) * 2008-06-27 2009-12-31 Microsoft Corporation Adaptive generation of out-of-dictionary personalized long words
US7650348B2 (en) * 2002-07-23 2010-01-19 Research In Motion Limited Systems and methods of building and using custom word lists
US7680661B2 (en) * 2008-05-14 2010-03-16 Nuance Communications, Inc. Method and system for improved speech recognition
US20100131447A1 (en) * 2008-11-26 2010-05-27 Nokia Corporation Method, Apparatus and Computer Program Product for Providing an Adaptive Word Completion Mechanism
US20100185448A1 (en) * 2007-03-07 2010-07-22 Meisel William S Dealing with switch latency in speech recognition
US20110054894A1 (en) * 2007-03-07 2011-03-03 Phillips Michael S Speech recognition through the collection of contact information in mobile dictation application
US7904298B2 (en) * 2006-11-17 2011-03-08 Rao Ashwin P Predictive speech-to-text input
US20110066634A1 (en) * 2007-03-07 2011-03-17 Phillips Michael S Sending a communications header with voice recording to send metadata for use in speech recognition, formatting, and search in mobile search application
US20110093265A1 (en) * 2009-10-16 2011-04-21 Amanda Stent Systems and Methods for Creating and Using Geo-Centric Language Models
US20110106527A1 (en) * 2001-07-03 2011-05-05 Apptera, Inc. Method and Apparatus for Adapting a Voice Extensible Markup Language-enabled Voice System for Natural Speech Recognition and System Response
US7953598B2 (en) * 2007-12-17 2011-05-31 Verizon Patent And Licensing Inc. Grammar weighting voice recognition information
US8005488B2 (en) * 2007-12-14 2011-08-23 Promptu Systems Corporation Automatic service vehicle hailing and dispatch system and method
US8086444B2 (en) * 2008-05-21 2011-12-27 Resolvity, Inc. Method and system for grammar relaxation

Patent Citations (54)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5752232A (en) * 1994-11-14 1998-05-12 Lucent Technologies Inc. Voice activated device and method for providing access to remotely retrieved data
US6108631A (en) * 1997-09-24 2000-08-22 U.S. Philips Corporation Input system for at least location and/or street names
US6646573B1 (en) * 1998-12-04 2003-11-11 America Online, Inc. Reduced keyboard text input system for the Japanese language
US6334102B1 (en) * 1999-09-13 2001-12-25 International Business Machines Corp. Method of adding vocabulary to a speech recognition system
US6850886B2 (en) * 1999-10-21 2005-02-01 Sony Corporation System and method for speech verification using an efficient confidence measure
US20020045463A1 (en) * 2000-10-13 2002-04-18 Zheng Chen Language input system for mobile devices
US20020087309A1 (en) * 2000-12-29 2002-07-04 Lee Victor Wai Leung Computer-implemented speech expectation-based probability method and system
EP1233407A1 (en) 2001-02-15 2002-08-21 Navigation Technologies Corporation Spatially built word list for automatic speech recognition program and method for formation thereof
US20020111810A1 (en) * 2001-02-15 2002-08-15 Khan M. Salahuddin Spatially built word list for automatic speech recognition program and method for formation thereof
US20040260543A1 (en) * 2001-06-28 2004-12-23 David Horowitz Pattern cross-matching
US20110106527A1 (en) * 2001-07-03 2011-05-05 Apptera, Inc. Method and Apparatus for Adapting a Voice Extensible Markup Language-enabled Voice System for Natural Speech Recognition and System Response
US20030182131A1 (en) * 2002-03-25 2003-09-25 Arnold James F. Method and apparatus for providing speech-driven routing between spoken language applications
US7016849B2 (en) * 2002-03-25 2006-03-21 Sri International Method and apparatus for providing speech-driven routing between spoken language applications
US7650348B2 (en) * 2002-07-23 2010-01-19 Research In Motion Limited Systems and methods of building and using custom word lists
US7328155B2 (en) * 2002-09-25 2008-02-05 Toyota Infotechnology Center Co., Ltd. Method and system for speech recognition using grammar weighted based upon location information
US20040139404A1 (en) 2002-11-29 2004-07-15 Takashi Kawashima Text editing assistor
WO2005066934A1 (en) 2004-01-07 2005-07-21 Toyota Infotechnology Center Co., Ltd. Method and system for speech recognition using grammar weighted based upon location information
US20060074660A1 (en) * 2004-09-29 2006-04-06 France Telecom Method and apparatus for enhancing speech recognition accuracy by using geographic data to filter a set of words
US7430473B2 (en) * 2004-10-01 2008-09-30 Bose Corporation Vehicle navigation display
US20060190258A1 (en) * 2004-12-16 2006-08-24 Jan Verhasselt N-Best list rescoring in speech recognition
US7634409B2 (en) * 2005-08-31 2009-12-15 Voicebox Technologies, Inc. Dynamic speech sharpening
US20070162281A1 (en) * 2006-01-10 2007-07-12 Nissan Motor Co., Ltd. Recognition dictionary system and recognition dictionary system updating method
US20070192311A1 (en) * 2006-02-10 2007-08-16 Pun Samuel Y L Method And System Of Identifying An Ideographic Character
US20070208555A1 (en) * 2006-03-06 2007-09-06 International Business Machines Corporation Dynamically adjusting speech grammar weights based on usage
WO2007112543A1 (en) 2006-04-06 2007-10-11 Research In Motion Limited Handheld electronic device and method for learning contextual data during disambiguation of text input
US20090313017A1 (en) * 2006-07-07 2009-12-17 Satoshi Nakazawa Language model update device, language Model update method, and language model update program
US20080177541A1 (en) * 2006-09-05 2008-07-24 Honda Motor Co., Ltd. Voice recognition device, voice recognition method, and voice recognition program
US7904298B2 (en) * 2006-11-17 2011-03-08 Rao Ashwin P Predictive speech-to-text input
US20080126314A1 (en) 2006-11-27 2008-05-29 Sony Ericsson Mobile Communications Ab Word prediction
US20080133228A1 (en) * 2006-11-30 2008-06-05 Rao Ashwin P Multimodal speech recognition system
US20080154612A1 (en) 2006-12-26 2008-06-26 Voice Signal Technologies, Inc. Local storage and use of search results for voice-enabled mobile communications devices
US20080221879A1 (en) * 2007-03-07 2008-09-11 Cerra Joseph P Mobile environment speech processing facility
US20100185448A1 (en) * 2007-03-07 2010-07-22 Meisel William S Dealing with switch latency in speech recognition
US20080221898A1 (en) * 2007-03-07 2008-09-11 Cerra Joseph P Mobile navigation environment speech processing facility
US20110066634A1 (en) * 2007-03-07 2011-03-17 Phillips Michael S Sending a communications header with voice recording to send metadata for use in speech recognition, formatting, and search in mobile search application
US20090030698A1 (en) * 2007-03-07 2009-01-29 Cerra Joseph P Using speech recognition results based on an unstructured language model with a music system
US20110054894A1 (en) * 2007-03-07 2011-03-03 Phillips Michael S Speech recognition through the collection of contact information in mobile dictation application
US20080288252A1 (en) * 2007-03-07 2008-11-20 Cerra Joseph P Speech recognition of speech recorded by a mobile communication facility
US20080235017A1 (en) * 2007-03-22 2008-09-25 Honda Motor Co., Ltd. Voice interaction device, voice interaction method, and voice interaction program
US20080270118A1 (en) * 2007-04-26 2008-10-30 Microsoft Corporation Recognition architecture for generating Asian characters
US8005488B2 (en) * 2007-12-14 2011-08-23 Promptu Systems Corporation Automatic service vehicle hailing and dispatch system and method
US7953598B2 (en) * 2007-12-17 2011-05-31 Verizon Patent And Licensing Inc. Grammar weighting voice recognition information
US20090171669A1 (en) * 2007-12-31 2009-07-02 Motorola, Inc. Methods and Apparatus for Implementing Distributed Multi-Modal Applications
US20090171659A1 (en) * 2007-12-31 2009-07-02 Motorola, Inc. Methods and apparatus for implementing distributed multi-modal applications
US20090187399A1 (en) * 2008-01-22 2009-07-23 O'dell Robert B Using Homophones and Near-Homophones to Improve Methods of Computer Text Entry for Chinese Characters
US20090228281A1 (en) * 2008-03-07 2009-09-10 Google Inc. Voice Recognition Grammar Selection Based on Context
US20090248415A1 (en) * 2008-03-31 2009-10-01 Yap, Inc. Use of metadata to post process speech recognition output
US7680661B2 (en) * 2008-05-14 2010-03-16 Nuance Communications, Inc. Method and system for improved speech recognition
US20090287680A1 (en) * 2008-05-14 2009-11-19 Microsoft Corporation Multi-modal query refinement
US20090287681A1 (en) * 2008-05-14 2009-11-19 Microsoft Corporation Multi-modal search wildcards
US8086444B2 (en) * 2008-05-21 2011-12-27 Resolvity, Inc. Method and system for grammar relaxation
US20090326927A1 (en) * 2008-06-27 2009-12-31 Microsoft Corporation Adaptive generation of out-of-dictionary personalized long words
US20100131447A1 (en) * 2008-11-26 2010-05-27 Nokia Corporation Method, Apparatus and Computer Program Product for Providing an Adaptive Word Completion Mechanism
US20110093265A1 (en) * 2009-10-16 2011-04-21 Amanda Stent Systems and Methods for Creating and Using Geo-Centric Language Models

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130046539A1 (en) * 2011-08-16 2013-02-21 International Business Machines Corporation Automatic Speech and Concept Recognition
US8676580B2 (en) * 2011-08-16 2014-03-18 International Business Machines Corporation Automatic speech and concept recognition
US20130332147A1 (en) * 2012-06-11 2013-12-12 Nvoq Incorporated Apparatus and Methods to Update a Language Model in a Speech Recognition System
US9489940B2 (en) * 2012-06-11 2016-11-08 Nvoq Incorporated Apparatus and methods to update a language model in a speech recognition system
US9318102B2 (en) * 2012-12-07 2016-04-19 Postech Academy—Industry Foundation Method and apparatus for correcting speech recognition error
DE102017216571A1 (en) 2017-09-19 2019-03-21 Volkswagen Aktiengesellschaft motor vehicle
WO2019057580A1 (en) 2017-09-19 2019-03-28 Volkswagen Aktiengesellschaft Motor vehicle
DE102017216571B4 (en) 2017-09-19 2022-10-06 Volkswagen Aktiengesellschaft motor vehicle
US11530930B2 (en) 2017-09-19 2022-12-20 Volkswagen Aktiengesellschaft Transportation vehicle control with phoneme generation

Also Published As

Publication number Publication date
EP2211336A1 (en) 2010-07-28
EP2211336B1 (en) 2014-10-08
US20100191520A1 (en) 2010-07-29

Similar Documents

Publication Publication Date Title
US8340958B2 (en) Text and speech recognition system using navigation information
US9830912B2 (en) Speak and touch auto correction interface
KR102596446B1 (en) Modality learning on mobile devices
JP4666648B2 (en) Voice response system, voice response program
JP4412504B2 (en) Speech recognition apparatus, speech recognition method, and speech recognition program
US8548806B2 (en) Voice recognition device, voice recognition method, and voice recognition program
US8311829B2 (en) Multimodal disambiguation of speech recognition
US20060100871A1 (en) Speech recognition method, apparatus and navigation system
AU2010346493B2 (en) Speech correction for typed input
CN108733655B (en) Word level correction of speech input
US20080177541A1 (en) Voice recognition device, voice recognition method, and voice recognition program
EP2196989A1 (en) Grammar and template-based speech recognition of spoken utterances
EP1751737A2 (en) Multimodal disambiguation of speech recognition
JP2010139826A (en) Voice recognition system
JP4914632B2 (en) Navigation device
JP2008076811A (en) Voice recognition device, voice recognition method and voice recognition program
US10832675B2 (en) Speech recognition system with interactive spelling function
KR100910302B1 (en) Apparatus and method for searching information based on multimodal
JP2005070330A (en) Speech recognition device and program
JP4645708B2 (en) Code recognition device and route search device
JP2007271901A (en) Navigation device and its address input method
JP2008076812A (en) Voice recognition device, voice recognition method and voice recognition program
JP2006184421A (en) Speech recognition device and speech recognition method

Legal Events

Date Code Title Description
AS Assignment

Owner name: HARMAN BECKER AUTOMOTIVE SYSTEMS GMBH, GERMANY

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:GRUHN, RAINER;RIECHERT, ANDREAS;SIGNING DATES FROM 20081020 TO 20081021;REEL/FRAME:024196/0274

AS Assignment

Owner name: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT

Free format text: SECURITY AGREEMENT;ASSIGNOR:HARMAN BECKER AUTOMOTIVE SYSTEMS GMBH;REEL/FRAME:024733/0668

Effective date: 20100702

AS Assignment

Owner name: HARMAN BECKER AUTOMOTIVE SYSTEMS GMBH, CONNECTICUT

Free format text: RELEASE;ASSIGNOR:JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT;REEL/FRAME:025795/0143

Effective date: 20101201

Owner name: HARMAN INTERNATIONAL INDUSTRIES, INCORPORATED, CON

Free format text: RELEASE;ASSIGNOR:JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT;REEL/FRAME:025795/0143

Effective date: 20101201

AS Assignment

Owner name: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT

Free format text: SECURITY AGREEMENT;ASSIGNORS:HARMAN INTERNATIONAL INDUSTRIES, INCORPORATED;HARMAN BECKER AUTOMOTIVE SYSTEMS GMBH;REEL/FRAME:025823/0354

Effective date: 20101201

AS Assignment

Owner name: HARMAN INTERNATIONAL INDUSTRIES, INCORPORATED, CON

Free format text: RELEASE;ASSIGNOR:JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT;REEL/FRAME:029294/0254

Effective date: 20121010

Owner name: HARMAN BECKER AUTOMOTIVE SYSTEMS GMBH, CONNECTICUT

Free format text: RELEASE;ASSIGNOR:JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT;REEL/FRAME:029294/0254

Effective date: 20121010

STCF Information on status: patent grant

Free format text: PATENTED CASE

FPAY Fee payment

Year of fee payment: 4

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 8