US20140244253A1 - Systems and Methods for Continual Speech Recognition and Detection in Mobile Computing Devices - Google Patents
Systems and Methods for Continual Speech Recognition and Detection in Mobile Computing Devices Download PDFInfo
- Publication number
- US20140244253A1 US20140244253A1 US14/347,456 US201214347456A US2014244253A1 US 20140244253 A1 US20140244253 A1 US 20140244253A1 US 201214347456 A US201214347456 A US 201214347456A US 2014244253 A1 US2014244253 A1 US 2014244253A1
- Authority
- US
- United States
- Prior art keywords
- computing device
- action
- trigger word
- taking
- mobile computing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/28—Constructional details of speech recognition systems
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/28—Constructional details of speech recognition systems
- G10L15/285—Memory allocation or algorithm optimisation to reduce hardware requirements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/16—Sound input; Sound output
- G06F3/167—Audio in a user interface, e.g. using voice commands for navigating, audio feedback
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/26—Speech to text systems
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L17/00—Speaker identification or verification
- G10L17/02—Preprocessing operations, e.g. segment selection; Pattern representation or modelling, e.g. based on linear discriminant analysis [LDA] or principal components; Feature selection or extraction
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L17/00—Speaker identification or verification
- G10L17/22—Interactive procedures; Man-machine interfaces
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/06—Transformation of speech into a non-audible representation, e.g. speech visualisation or speech processing for tactile aids
- G10L21/10—Transforming into visible information
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/08—Speech classification or search
- G10L2015/088—Word spotting
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L2015/223—Execution procedure of a spoken command
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L2015/226—Procedures used during a speech recognition process, e.g. man-machine dialogue using non-speech characteristics
- G10L2015/228—Procedures used during a speech recognition process, e.g. man-machine dialogue using non-speech characteristics of application context
Definitions
- activating a voice command interface on a mobile computing device requires some physical activation of the voice command interface by a user.
- a user may be required to press or hold a physical button or switch on the exterior of the mobile computing device, or the user may be required to activate a virtual button or switch on a graphical user interface associated with the mobile computing device.
- Controlling a mobile computing device via voice commands is often useful when a user cannot easily interact with the mobile computing device physically. For example, controlling a mobile computing device via voice commands can be useful when the user is driving a car or when the mobile computing device is some distance from the user (e.g., across the room or otherwise out of the user's immediate reach).
- a voice command interface for controlling a mobile computing device generally requires operating a microphone or other speech input device to receive speech from a user, and running speech recognition and detection algorithms on the mobile computing device to detect and recognize speech received via the microphone.
- speech recognition and detection algorithms on the mobile computing device to detect and recognize speech received via the microphone.
- continually operating a microphone and continually running speech detection and recognition applications on a mobile computing device may be undesirable in many situations because the power required to operate the microphone and continually execute the detection and recognition applications can rapidly deplete the mobile computing device's battery.
- the present application discloses systems and methods that allow a user to enjoy the advantages of controlling a mobile computing device via voice commands without the limitations of having to either press or hold a physical button or switch on the mobile computing device or manually activate a virtual button or switch on a graphical user interface of the mobile computing device as required with existing devices.
- a mobile computing device configured to (i) detect coupling to an external power source, and (ii) in response to the detecting, activate a speech recognition mode.
- Some embodiments include a power monitoring routine for detecting coupling to an external power source.
- a mobile computing device operating in a speech recognition mode may include a trigger word detection subroutine. The combination of detecting the coupling to the external power source and activating the speech recognition mode enables the disclosed embodiments to overcome the limitations of existing devices with negligible impact on the mobile computing device's battery in most situations.
- Some embodiments include determining whether the mobile computing device is receiving operating power either from an external power source or from a battery. If the mobile computing device is receiving power from the external power source, then the mobile computing device activates a speech recognition mode.
- the speech recognition mode includes a trigger word detection subroutine.
- the external power source may be a wall charger for the mobile computing device or a docking station for the mobile computing device, such as a desktop or car-based docking station. Some embodiments may also include exiting from the speech recognition mode (and/or perhaps deactivating trigger word detection subroutine) when the mobile computing device is receiving operating power from the battery.
- the trigger word detection subroutine includes determining whether spoken words received via a microphone include at least one trigger word of a set of one or more trigger words. In response to determining that the spoken words include at least one trigger word, the trigger word detection subroutine launches an application corresponding to the at least one trigger word included in the spoken words. In some embodiments, the trigger word detection subroutine operates continually as long as the mobile computing device continues to receive power from the external power source.
- the launched application corresponding to the detected trigger word (or words) may be a voice command prompt configured to receive a voice command from a user.
- a user may control the mobile computing device via one or more voice commands.
- the voice command after the trigger word (or words) may include an action and an object of the action.
- the trigger word to launch the voice command prompt application may be “Android,” and the voice command may include “call” (action) and “Jane Smith” (the object of the action).
- the voice command could include additional words as described herein.
- the launched application may be a particular communications application, multimedia application, or an application for configuring settings of the mobile computing device.
- the trigger word could even be the name of the application.
- the trigger word to launch the phone dialer may be “call.”
- the mobile computing device may be configured to launch the phone dialer and then either (i) wait to receive the name of the desired called party or (ii) use the words following the “call” trigger word as the name of the desired called party.
- the phone dialer (or other launched application) could be configured to receive additional voice commands after it is launched. The types of commands may vary depending on the particular application as described herein.
- the trigger word detection routine may be launched in response to coupling the device to the external power source and not in response to any other type of input (e.g., touch input, manual button press, etc.).
- the speech recognition mode and/or trigger word detection routine may be activated automatically only when the mobile computing device is receiving power from an external power source.
- the speech recognition mode may be activated only when the mobile computing device is receiving operating power from an external power source, and the speech recognition mode may always be deactivated when the mobile computing device is receiving operating power from a battery.
- Other embodiments may include receiving a manual input from a user to activate (or deactivate) the trigger word detection subroutine regardless of whether the mobile computing device is receiving operating power from the external power source or the battery.
- the mobile computing device may still activate a trigger word detection subroutine even though the mobile computing device may be receiving operating power from the battery.
- the trigger word detection subroutine may still operate continually while the mobile computing device is receiving power from the battery.
- the features and functions of the disclosed methods may be integrated with the operating system of the mobile computing device.
- the features and functions of the disclosed embodiments may be components of an application that is installed and executed by the mobile computing device, such an application downloaded from an online application store.
- any of the methods described herein may be implemented in the form of instructions stored on a non-transitory, tangible computer readable media. When executed by a mobile computing device, the instructions may cause the mobile computing device to perform one or more functions of the disclosed method. Further examples may also include articles of manufacture including tangible, non-transitory computer-readable media that have computer-readable instructions encoded thereon. The instructions may comprise instructions to perform one or more functions of the methods described herein.
- the computer readable media may include non-transitory computer readable media, such as computer-readable media that stores data for short periods of time like register memory, processor cache, and Random Access Memory (RAM).
- the computer readable media may also include non-transitory media, such as secondary or persistent long term storage, like read only memory (ROM), optical or magnetic disks, or compact-disc read only memory (CD-ROM), for example.
- the computer readable media may also be any other volatile or non-volatile storage system.
- the computer readable media may be considered a computer readable storage media, for example, or a tangible storage media.
- circuitry configured to perform logical functions in any of the processes or methods described herein.
- many types of devices may be used or configured to perform logical functions in any of the processes or methods described herein.
- many types of devices and/or components or sub-components of the devices) may be used or configured as means for performing functions of any of the methods described herein (or any portions of the methods described herein).
- FIG. 1 shows an example of a simplified block diagram of a mobile computing device according to some embodiments.
- FIG. 2A shows an example algorithm for a power detection routine according to some embodiments.
- FIG. 2B shows an example algorithm of a trigger word detection subroutine according to some embodiments.
- FIGS. 3A-3B show examples of a mobile computing device 300 launching applications in response to detecting trigger words according to some embodiments.
- FIG. 4 shows an example method according to some embodiments.
- FIG. 5 shows an example method according to some embodiments.
- FIG. 1 shows an example of a simplified block diagram of a mobile computing device 100 according to some embodiments.
- the mobile computing device 100 may be a mobile telephone, cellular telephone, smartphone or similar mobile communications device now known or later developed that may be controlled via voice commands.
- the mobile computing device 100 may be a laptop computer, tablet computer, or other type of mobile computing device now known or later developed that may be controlled via voice commands.
- the mobile computing device 100 includes a plurality of components, such as one or more processors 101 , data storage 102 , power management sub-system 103 , user input/output interfaces 104 , communications interfaces 105 , and sensors 106 .
- the individual components are connected via one or more data busses 115 or similar communications connections that are configured to enable the individual components to send and receive data to and from other components of the mobile computing device 100 .
- the individual components are also connected to the power management sub-system 103 via a power distribution bus 116 or similar connection that is configured to provide operating power to the individual components of the mobile computing device 100 .
- the one or more processors 101 may include any type of general purpose processor (e.g., processors from Intel, Advanced Micro Devices (AMD), etc.) or any type of special purpose processors (e.g., application specific integrated circuits (ASICs), digital signal processors (DSP), Coder/Decoders (CODECs), etc.) now known or later developed.
- general purpose processor e.g., processors from Intel, Advanced Micro Devices (AMD), etc.
- special purpose processors e.g., application specific integrated circuits (ASICs), digital signal processors (DSP), Coder/Decoders (CODECs), etc.
- the data storage 102 is configured to store data as well as computer-readable program instructions 108 .
- the data storage 102 may include any of the types of data storage described herein, including short term volatile memory (e.g., RAM) for storing data and/or program code or instructions during execution of the program code or instructions and/or longer-term non-volatile memory (e.g., ROM) for storing data and/or program code or instructions.
- the data storage may be implemented as solid state and/or flash memory or other similar types of small form factor storage media.
- the power management sub-system 103 includes a battery power interface 112 , an external power interface 113 , and a power source selector 114 .
- the battery power interface 112 is configured to receive operating power from a battery source, such as an internal battery, a removable battery that is electrically coupled to the mobile computing device 100 , or any other sort of battery or similar type of portable power supply now known or later developed.
- the battery that supplies operating power for the mobile computing device 100 via the battery interface 112 may be rechargeable or non-rechargeable.
- the external power interface 113 is configured to receive power from an external power source for the mobile computing device 100 .
- external power sources include electrical wall outlets, desktop docking stations, car docking stations or power cables, alarm clocks or entertainment systems with docking stations, or any other type of connection or device that is configured to provide the mobile computing device 100 with operating power so that the mobile computing device 100 does not need to rely on operating power from a battery via the battery interface 112 .
- the power source selector 114 may be configured to selectively provide operating power from one of the battery interface 112 or the external power interface 113 .
- the power source selector 114 may be configured to both (i) provide operating power to the mobile computing device 100 from the external power supply via the external power interface 113 and (ii) charge the battery with power from the external power supply by providing an electrical connection from the external power interface 113 to the battery interface 112 .
- the user input/output interfaces 104 include a plurality of input and output components, such as sound input/output components 107 , display input/output components 108 , and an optional keyboard 109 .
- the sound components 107 may include a microphone or other similar type of sound detection device now known or later developed that is configured to detect and receive sound waves.
- the sound components 107 may also include one or more speakers or similar sound generation devices now known or later developed that are configured to generate sound waves.
- the display input/output components 108 may include a Liquid Crystal Display (LCD), plasma display, or similar type of display device now known or later developed that is configured to display a graphical user interface and/or other information to a user.
- the display 108 may include a touch-screen interface or other touch-sensitive components to enable a user to enter commands or other information (or otherwise interact with) the mobile computing device 100 .
- the display 108 may be configured to generate and display a virtual keyboard for data entry by a user.
- the mobile computing device 100 may optionally include a physical keyboard 109 . However, a keyboard (virtual or physical) is not required in all embodiments.
- Some embodiments may also include one or more external switches or buttons (not shown) to turn the mobile computing device 100 on or off, to wake the device from a sleep state, to control volume or other similar functions.
- the communications interfaces 105 may include one or more wireless communications interfaces 110 and one or more wired interfaces 111 .
- the wireless interfaces 110 may include any type of wireless communications interface configured to transmit and receive data according to any of a number of different wireless communications protocols, such as Bluetooth, IEEE 802.11 (IEEE 802.11 may refer to IEEE 802.11-2007, IEEE 802.11n-2009, or any other IEEE 802.11 revision) or other Wi-Fi variant, Code-Division Multiple Access (CDMA), Global System for Mobile Communications (GSM), 3rd Generation (3G), 4th Generation (4G), 3rd Generation Partnership Project—Long Term Evolution (3GPP-LTE), Worldwide Interoperability for Microwave Access (WiMAX), Zigbee, General Packet Radio Service (GPRS), Evolution-Data Optimized (EV-DO), High-Speed Downlink Packet Access (HSDPA), or any other type of wireless communications protocol now known or later developed.
- IEEE 802.11 may refer to IEEE 802.11-2007, IEEE 802.11n-2009
- the wired interfaces 111 may include any type of wired communications interface (e.g., electrical, optical, etc.) configured to transmit and receive data according to any of a number of different communications protocols, such as Universal Serial Bus (USB), Ethernet, IEEE 1394, Firewire, Thunderbolt, or any other type of serial or parallel data communications protocol now known or later developed.
- the wired interfaces 110 and wireless interfaces 111 may be configured to communicate with other computing or communications devices via networks, e.g. the Internet, and/or communicate with peripheral devices, such as disk drives, external monitors, mouse or trackball/pad devices, and external keyboards etc., via Bluetooth, USB, or similar connections.
- mobile computing device 100 may use wired interfaces 110 and/or wireless interfaces 111 to communicate with a server that performs speech recognition on behalf of mobile computing device 100 .
- the wired communications interface 111 may also be configured to provide operating power via the external power interface 113 as shown by power connection 117 from the wired interface 111 to the external power interface 113 .
- USB is one type of wired communications interface that can be configured to provide both communications and power.
- the mobile computing device 100 may also include one or more sensors 106 .
- the sensors 106 may include any one or more cameras or other image sensors, accelerometers, Global Positioning System (GPS) sensors, temperature sensors, or any other type of sensor known now or later developed that may be desirable to integrate with the mobile computing device 100 .
- GPS Global Positioning System
- a mobile computing device configured according to the disclosed embodiments may enable a user to enjoy the advantages of controlling the mobile computing device via voice commands without the limitation of having to either press or hold a physical button or switch on the mobile computing device or manually activate a virtual button or switch on a graphical user interface of the mobile computing device.
- the above-described advantages may be achieved via a combination of a power detection routine and a trigger word detection subroutine that are executed by the mobile computing device 100 .
- FIG. 2A shows an example algorithm for a power detection routine 201 according to some embodiments
- FIG. 2B shows an example algorithm for a trigger word detection subroutine 210 according to some embodiments.
- the description of the algorithms of FIGS. 2A and 2B includes references to the example mobile computing device 100 and its corresponding components shown and described herein with reference in FIG. 1 .
- the power detection routine 201 and/or the trigger word detection subroutine 210 may be components of the operating system of the mobile computing device 100 .
- the power detection routine 201 and/or the trigger word detection subroutine 210 may be components of an application or widget installed on the mobile computing device 100 , such as an application or widget downloaded from an online application store.
- the power detection routine 201 may be configured to operate continually (or at least substantially continually) on the mobile computing device 100 . Some embodiments may also allow a user to activate and deactivate the power detection routine 201 . In some embodiments, the power detection routine 201 may be selectively activated or deactivated by a user.
- the trigger word detection subroutine 210 is a “subroutine” of the power detection routine 201 in that the trigger word detection subroutine 210 is activated by the power detection routine 201 under certain conditions, as described herein.
- the trigger word detection capability need not be implemented as a subroutine of a power detection routine in all embodiments.
- the power detection routine starts at block 202 .
- the power detection routine 201 determines whether operating power for the mobile computing device 100 is being received from either (1) an external power source via the external power interface 113 or (2) a battery power source via the battery interface 112 .
- the determination may be made by the power source selector 114 .
- the determination may be made by one or more processors 101 configured to monitor any of the battery interface 112 , the external power interface 113 , and/or the power source selector 114 . If an external power source is not detected (i.e., the mobile computing device 100 is receiving operating power from a battery source), then the power detection routine 201 ends at block 207 .
- power detection routine 201 re-starts at block 202 , either immediately (or after a brief waiting period) so that power detection routine 201 operates continually (or at least substantially continually).
- the power detection routine 201 determines that the mobile computing device 100 is receiving operating power from an external power source via the external power interface 113 , then the power detection routine 201 advances to block 204 .
- the power detection routine 201 activates the trigger word detection subroutine 210 . Once it is activated by the power detection routine 201 at block 204 , the trigger word detection subroutine 210 may be configured to execute continually (or at least substantially continually) until it is deactivated.
- the power detection routine 201 determines whether the communications device 100 is receiving operating power from an external power source via the external power interface 113 . In some embodiments, the determination may be made by the power source selector 114 . In other embodiments, the determination may be made by one or more processors 101 configured to monitor any of the battery interface 112 , the external power interface 113 , and/or the power source selector 114 .
- the power detection routine 201 returns to block 205 . But if the power detection routine 201 determines that the mobile computing device 100 is no longer receiving its operating power from an external power source (i.e., the external power source has been disconnected from the external power interface 113 and/or the mobile computing device 100 has switched to battery power), then the power detection algorithm 201 advances to block 206 . At block 206 , the power detection routine 201 deactivates the trigger word detection subroutine 210 . The power detection routine 201 then ends at block 207 . After ending at block 207 , power detection routine 201 re-starts at block 202 , either immediately (or after a brief waiting period) so that power detection routine 201 operates continually (or at least substantially continually).
- FIG. 2B shows an example algorithm of a trigger word detection subroutine 210 .
- the trigger word detection subroutine 210 is configured to execute continually (or at least substantially continually) until it is deactivated.
- the trigger word detection subroutine 210 may be deactivated by the power detection routine 201 at block 206 as described above.
- the trigger word detection subroutine 210 may be selectively activated and/or deactivated by a user.
- the trigger word detection subroutine 210 starts at block 212 .
- the trigger word detection subroutine receives a sound sample.
- the sound sample may be received via the sound input 107 of the mobile computing device 100 .
- the sound input 107 may include a microphone or other type of sound input or detection device.
- Some embodiments may include a sound detector that triggers the activation of a microphone. Once activated, the microphone captures a sound sample.
- a microphone may be configured to continually (or at least substantially continually) capture sound samples once the trigger word detection subroutine 210 has been activated.
- the trigger word detection subroutine 210 determines whether the sound sample received at block 213 includes speech. Determining whether the sound sample includes speech can be accomplished by any of a number of speech detection algorithms now know or later developed. In some embodiments, the speech detection algorithm may be executed by the one or more processors 101 . In other embodiments, the speech detection algorithm may be executed by one or more components of the sound input 107 .
- the trigger word detection subroutine 210 ends at block 217 and, thereafter, re-starts at block 212 (either immediately or after a brief waiting period) to receive an additional sound sample at block 213 . However, if the sound sample includes speech, then the trigger word detection subroutine 210 advances to block 215 .
- the speech from the sound sample is analyzed to determine whether the speech includes at least one of a set of one or more trigger words. Trigger words are described in more detail with respect to FIGS. 3A-3C .
- the sound sample may be analyzed with any of a number of sound or voice recognition algorithms now known or later developed. For example, speech recognition may be performed on the spoken input to obtain text, and the text may be analyzed to determine whether the text includes one or more trigger words.
- the speech recognition may be performed by the mobile computing device 100 , by a server with which mobile computing device 100 communicates, or by a combination thereof.
- the trigger word detection subroutine ends at block 217 and, thereafter, re-starts at block 212 (either immediately or after a brief waiting period) to receive an additional sound sample at block 213 .
- the trigger word detection subroutine advances to block 216 .
- the trigger word detection subroutine 210 launches an application corresponding to the at least one trigger word determined at block 215 .
- the trigger word detection subroutine 210 ends at block 217 .
- the trigger word detection subroutine 210 re-starts at block 212 (either immediately or after a brief waiting period) to receive a new sound sample at block 213 .
- the trigger word detection subroutine 210 continues in this manner continually (or at least substantially continually) until the trigger word detection subroutine 210 is deactivated by either the power detection routine 201 or a user.
- FIGS. 3A-3B show examples of a mobile computing device 300 launching applications in response to detecting trigger words according to some embodiments.
- the mobile computing device 300 may be similar to the mobile computing device 100 shown and described with respect to FIG. 1 .
- the mobile computing device 300 may be configured to execute the power detection routine 201 and/or the trigger word detection subroutine 210 shown and described with respect to FIGS. 2A-2B .
- the mobile computing device 300 is receiving operating power from an external power source connected to the mobile computing device 300 via cable 301 .
- Power icon 302 on the display 303 of the mobile computing device 300 indicates that the mobile computing device 300 is receiving power from an external power source.
- the power detection routine 201 FIG. 2A
- the trigger word detection subroutine 210 FIG. 2B
- the trigger word detection subroutine 210 is configured to operate continually (or at least substantially continually) until it is deactivated by either the power detection routine 201 or a user.
- the trigger word detection subroutine 210 is receiving sound samples via the microphone 304 of the mobile computing device 300 .
- the mobile computing device 300 determines that a sound sample received via its microphone 304 includes at least one trigger word of a set of one or more trigger words, the mobile computing device 300 is configured to launch an application corresponding to the at least one determined trigger word.
- the mobile computing device 300 may be configured to detect a single trigger word.
- the trigger word may be “Android” (or perhaps some other user-defined trigger word or group of words).
- the mobile computing device 300 may launch a voice command prompt.
- FIG. 3A shows an example of the mobile computing device 300 launching a voice command prompt 305 in response to detecting a trigger word corresponding to the voice command prompt 305 .
- the voice command prompt 305 After the voice command prompt 305 has been launched, a user can then issue a spoken command string to the mobile computing device 300 .
- the spoken command string may be “Call . . . Jane Smith . . . Office.”
- the command includes an action (“call”), an object of the action (“Jane Smith”), and a parameter (“Office”).
- the mobile computing device 300 may (i) launch an application corresponding to the action (and possibly one or more parameters associated with the action), and (ii) execute the application based on the object of the action (and possibly one or more parameters associated with the object of the action).
- FIG. 3B shows the mobile computing device 300 launching a phone application 306 and calling “Jane Smith” at her office number in response to the spoken command string received via the voice command prompt 305 ( FIG. 3A ).
- the mobile computing device 300 may be configured to launch other types of applications in response to particular spoken command strings received via the voice command prompt 305 .
- the mobile computing device 300 may be configured to activate or deactivate a particular application, feature, or service.
- the mobile computing device 300 may be configured to silence or deactivate its ringer in response to the command string such as “Configuration Ringer Silent.”
- the mobile computing device 300 may be configured to launch a multimedia player and play a particular song or video in response to a spoken command string received via the voice command prompt 305 .
- the spoken command string “Play Song All You Need Is Love” may cause the mobile computing device 300 to launch a multimedia player and play the song “All You Need Is Love” by The Beatles.
- the spoken command string may include one or more parameters associated with either an action or an object to ameliorate or otherwise minimize ambiguity.
- the “call” was the action
- “Jane Smith” was the object
- “Office” was the parameter associated with object “Jane Smith.”
- the parameter “Office” specifies which of Jane Smith's numbers to call (i.e., home, office, mobile, etc.).
- the trigger word was “Android” and the command string was “Call Jane Smith Office.”
- the user may need to pause slightly after the trigger word before speaking a command string to allow time for the mobile computing device 300 to launch the voice command prompt 305 .
- the user may need to pause slightly after speaking the “action” for the voice command prompt to allow time for the mobile computing device 300 to launch the application corresponding to the action, e.g., the phone application in this example.
- the mobile computing device 300 may be configured to cache a brief history of spoken words, e.g., up to about 5-30 seconds of spoken words or perhaps a configurable number of words, e.g., up to 10-20 spoken words, depending on the configuration.
- the computing device 300 may be configured to analyze the words in the cache to identify actions, objects, and parameters that may follow a trigger word (or words).
- One advantage of caching a brief history of spoken words is that a user may not need to pause long (or at all) after a trigger word or after an “action” to wait for the mobile computing device 300 to launch the voice command prompt in response to the trigger word or to launch an application in response to the “action” received via the voice command prompt.
- the mobile computing device 300 may be configured to detect multiple trigger words (or groups of words, i.e., a trigger phrase), where individual trigger words (or phrases) correspond to different applications or actions to take in a particular application.
- a single trigger word or perhaps trigger phrase
- the mobile computing device 300 could receive spoken command strings to launch other applications (i.e., a phone application as shown in FIG. 3B ).
- the mobile computing device 300 may be able to launch applications and process commands faster than embodiments that may rely on a single trigger word to first open a voice command prompt 305 before receiving spoken command strings to launch applications.
- the set of trigger words may include a specific trigger word (or words) for a voice command prompt, a specific trigger word (or words) for a phone application, a specific trigger word (or words) for a text messaging application, a specific trigger word (or words) for an email application, and a specific trigger word (or words) for a multimedia player application, etc.
- the mobile computing device 300 could have many different trigger words or phrases corresponding to many different applications and/or functions.
- a user may be able to configure or customize particular trigger words of phrases and associate those customize trigger words or phrases with particular applications or other computing, communications, or entertainment tasks. For example, a user may configure a trigger phrase “Favorite Song” to cause the mobile computing device to launch a multimedia player and play the song “All You Need Is Love” by The Beatles.
- FIG. 3B can also correspond to an example of the mobile computing device 300 launching a telephone application in response to detecting a trigger word corresponding to the telephone application.
- the trigger word corresponding to the phone application may be “call.”
- the mobile computing device 300 may launch the phone application and then listen for further instructions, such as the number or person to call.
- FIG. 3C shows an example of the mobile computing device 300 launching an email application 307 in response to detecting a trigger word corresponding to the email application 307 .
- the trigger word corresponding to the email application 307 may be “mail,” and upon detecting the “mail” trigger word, the mobile computing device 300 may launch the email application 307 and then listen for further instructions, such as the recipient, subject, and body of the email message.
- FIG. 4 shows an example method 400 according to some embodiments.
- the method 400 begins at block 401 , where a mobile computing device determines whether it is receiving operating power from one of (i) an external power source or (ii) one or more batteries configured to power the mobile computing device.
- the mobile computing device may be similar to any of the mobile computing devices described elsewhere herein.
- the mobile device includes a trigger word detection subroutine that is activatable by a user input and automatically in response to determining that the mobile computing device is receiving external power.
- the mobile computing device activates the trigger word detection subroutine in response to determining that it is receiving power from the external power source.
- the trigger word detection subroutine is shown in blocks 402 a - 402 d , which are described in more detail below.
- the mobile computing device may additionally or alternatively activate the trigger word detection subroutine in response to receiving an input from a user to activate the trigger word detection subroutine even when the mobile computing device is receiving power from the one or more batteries.
- the mobile computing device deactivates the trigger word detection subroutine in response to determining that it is receiving operating power from the one or more batteries.
- the mobile computing device may additionally or alternatively deactivate the trigger word detection subroutine in response to receiving an input from a user to deactivate the trigger word detection subroutine even when the mobile computing device is receiving power from an external power source.
- the trigger word detection subroutine is shown in blocks 402 a - 402 d .
- the trigger word detection subroutine may be configured to execute continually (or at least substantially continually) until it is deactivated either at step 403 or by a user.
- the trigger word detection subroutine begins at block 402 a where the mobile computing device receives spoken input via a microphone.
- the mobile computing device obtains text corresponding to the spoken input, based on speech recognition performed on the spoken input.
- the speech recognition could be performed by the mobile computing device.
- the speech recognition could be performed, in whole or in part, by one or more devices external to the mobile computing device.
- the mobile computing device may transmit the spoken input to a server, the server may perform speech recognition on the spoken input to determine a corresponding text, and the mobile computing device may obtain the text from the server.
- the mobile computing device analyzes the text to determine whether the text includes at least one of a set of one or more predefined trigger words.
- the mobile computing device launches an application corresponding to the at least one trigger word included in the text.
- the application corresponding to the at least one trigger word may be a voice command prompt application configured to receive a voice command from a user.
- the voice command may comprise one or more of (i) an action, (ii) an object of the action, and (iii) one or more parameters associated with the action or the object.
- the voice command prompt and the voice commands received via the voice command prompt may be similar to the voice command prompts and voice commands described elsewhere herein.
- the application corresponding to the at least one trigger word may be any of a voice command prompt, a communications application, a multimedia application, or an application for configuring settings of the mobile computing device.
- a speech recognition mode of a mobile computing device can be activated by coupling the mobile computing device to an external power source.
- FIG. 5 shows an example method 500 according to such embodiments. The method 500 begins at block 501 , when a mobile computing device detects coupling of the mobile computing device to an external power source. In response to the detected coupling, the mobile computing device activates a speech recognition mode, at block 502 . Blocks 502 - 505 describe an example of how the speech recognition mode may operate after being activated.
- the mobile computing device receives spoken input via a microphone.
- the mobile computing device determines whether the spoken input includes one or more trigger words. This can involve speech recognition performed by the mobile computing device and/or by a server, as described above.
- the mobile computing device launches an application corresponding to the at least one trigger word, at block 505 .
Abstract
The present application describes systems, articles of manufacture, and methods for continuous speech recognition for mobile computing devices. One embodiment includes determining whether a mobile computing device is receiving operating power from an external power source or a battery power source, and activating a trigger word detection subroutine in response to determining that the mobile computing device is receiving power from the external power source. In some embodiments, the trigger word detection subroutine operates continually while the mobile computing device is receiving power from the external power source. The trigger word detection subroutine includes determining whether a plurality of spoken words received via a microphone includes one or more trigger words, and in response to determining that the plurality of spoken words includes at least one trigger word, launching an application corresponding to the at least one trigger word included in the plurality of spoken words.
Description
- This patent application claims priority to U.S. Provisional Application No. 61/542,074, filed on Sep. 30, 2011, the contents of which are entirely incorporated herein by reference, as if fully set forth in this application.
- Typically, activating a voice command interface on a mobile computing device requires some physical activation of the voice command interface by a user. For example, to activate a voice command interface on a mobile computing device, a user may be required to press or hold a physical button or switch on the exterior of the mobile computing device, or the user may be required to activate a virtual button or switch on a graphical user interface associated with the mobile computing device.
- Controlling a mobile computing device via voice commands is often useful when a user cannot easily interact with the mobile computing device physically. For example, controlling a mobile computing device via voice commands can be useful when the user is driving a car or when the mobile computing device is some distance from the user (e.g., across the room or otherwise out of the user's immediate reach).
- A voice command interface for controlling a mobile computing device generally requires operating a microphone or other speech input device to receive speech from a user, and running speech recognition and detection algorithms on the mobile computing device to detect and recognize speech received via the microphone. However, continually operating a microphone and continually running speech detection and recognition applications on a mobile computing device may be undesirable in many situations because the power required to operate the microphone and continually execute the detection and recognition applications can rapidly deplete the mobile computing device's battery.
- The present application discloses systems and methods that allow a user to enjoy the advantages of controlling a mobile computing device via voice commands without the limitations of having to either press or hold a physical button or switch on the mobile computing device or manually activate a virtual button or switch on a graphical user interface of the mobile computing device as required with existing devices.
- In operation, a mobile computing device according to some embodiments of the disclosed systems and methods is configured to (i) detect coupling to an external power source, and (ii) in response to the detecting, activate a speech recognition mode. Some embodiments include a power monitoring routine for detecting coupling to an external power source. Likewise, in some embodiments, a mobile computing device operating in a speech recognition mode may include a trigger word detection subroutine. The combination of detecting the coupling to the external power source and activating the speech recognition mode enables the disclosed embodiments to overcome the limitations of existing devices with negligible impact on the mobile computing device's battery in most situations.
- Some embodiments include determining whether the mobile computing device is receiving operating power either from an external power source or from a battery. If the mobile computing device is receiving power from the external power source, then the mobile computing device activates a speech recognition mode. In some embodiments, the speech recognition mode includes a trigger word detection subroutine. The external power source may be a wall charger for the mobile computing device or a docking station for the mobile computing device, such as a desktop or car-based docking station. Some embodiments may also include exiting from the speech recognition mode (and/or perhaps deactivating trigger word detection subroutine) when the mobile computing device is receiving operating power from the battery.
- In operation, the trigger word detection subroutine includes determining whether spoken words received via a microphone include at least one trigger word of a set of one or more trigger words. In response to determining that the spoken words include at least one trigger word, the trigger word detection subroutine launches an application corresponding to the at least one trigger word included in the spoken words. In some embodiments, the trigger word detection subroutine operates continually as long as the mobile computing device continues to receive power from the external power source.
- In some embodiments, the launched application corresponding to the detected trigger word (or words) may be a voice command prompt configured to receive a voice command from a user. When the voice command prompt is launched, a user may control the mobile computing device via one or more voice commands. The voice command after the trigger word (or words) may include an action and an object of the action. For example, the trigger word to launch the voice command prompt application may be “Android,” and the voice command may include “call” (action) and “Jane Smith” (the object of the action). The voice command could include additional words as described herein.
- In other embodiments, the launched application may be a particular communications application, multimedia application, or an application for configuring settings of the mobile computing device. In these embodiments, the trigger word could even be the name of the application. For example, the trigger word to launch the phone dialer may be “call.” In response to detecting the trigger word “call,” the mobile computing device may be configured to launch the phone dialer and then either (i) wait to receive the name of the desired called party or (ii) use the words following the “call” trigger word as the name of the desired called party. The phone dialer (or other launched application) could be configured to receive additional voice commands after it is launched. The types of commands may vary depending on the particular application as described herein.
- In some embodiments, the trigger word detection routine may be launched in response to coupling the device to the external power source and not in response to any other type of input (e.g., touch input, manual button press, etc.). In these embodiments, the speech recognition mode and/or trigger word detection routine may be activated automatically only when the mobile computing device is receiving power from an external power source. In some embodiments, the speech recognition mode may be activated only when the mobile computing device is receiving operating power from an external power source, and the speech recognition mode may always be deactivated when the mobile computing device is receiving operating power from a battery.
- Other embodiments may include receiving a manual input from a user to activate (or deactivate) the trigger word detection subroutine regardless of whether the mobile computing device is receiving operating power from the external power source or the battery. In these embodiments, the mobile computing device may still activate a trigger word detection subroutine even though the mobile computing device may be receiving operating power from the battery. When the trigger word detection subroutine is manually activated by the user (e.g., touch input, manual button press, etc.), the trigger word detection subroutine may still operate continually while the mobile computing device is receiving power from the battery.
- In some embodiments, the features and functions of the disclosed methods may be integrated with the operating system of the mobile computing device. In other embodiments, the features and functions of the disclosed embodiments may be components of an application that is installed and executed by the mobile computing device, such an application downloaded from an online application store.
- Any of the methods described herein may be implemented in the form of instructions stored on a non-transitory, tangible computer readable media. When executed by a mobile computing device, the instructions may cause the mobile computing device to perform one or more functions of the disclosed method. Further examples may also include articles of manufacture including tangible, non-transitory computer-readable media that have computer-readable instructions encoded thereon. The instructions may comprise instructions to perform one or more functions of the methods described herein.
- The computer readable media may include non-transitory computer readable media, such as computer-readable media that stores data for short periods of time like register memory, processor cache, and Random Access Memory (RAM). The computer readable media may also include non-transitory media, such as secondary or persistent long term storage, like read only memory (ROM), optical or magnetic disks, or compact-disc read only memory (CD-ROM), for example. The computer readable media may also be any other volatile or non-volatile storage system. In some embodiments, the computer readable media may be considered a computer readable storage media, for example, or a tangible storage media.
- In addition, some aspects of the disclosed methods may be performed by circuitry configured to perform logical functions in any of the processes or methods described herein. In still further examples, many types of devices may be used or configured to perform logical functions in any of the processes or methods described herein. In yet further examples, many types of devices (and/or components or sub-components of the devices) may be used or configured as means for performing functions of any of the methods described herein (or any portions of the methods described herein).
-
FIG. 1 shows an example of a simplified block diagram of a mobile computing device according to some embodiments. -
FIG. 2A shows an example algorithm for a power detection routine according to some embodiments. -
FIG. 2B shows an example algorithm of a trigger word detection subroutine according to some embodiments. -
FIGS. 3A-3B show examples of amobile computing device 300 launching applications in response to detecting trigger words according to some embodiments. -
FIG. 4 shows an example method according to some embodiments. -
FIG. 5 shows an example method according to some embodiments. - The following detailed description includes references to the accompanying figures. In the figures, similar symbols typically identify similar components, unless context dictates otherwise. The example embodiments outlined in the detailed description, figures, and claims are not meant to be limiting. Other embodiments may be utilized, and other changes may be made, without departing from the scope of the subject matter presented herein. It will be readily understood that the aspects of the present disclosure, as generally described herein, and illustrated in the figures, can be arranged, substituted, combined, separated, and designed in a wide variety of different configurations, all of which are contemplated herein.
-
FIG. 1 shows an example of a simplified block diagram of amobile computing device 100 according to some embodiments. In some embodiments, themobile computing device 100 may be a mobile telephone, cellular telephone, smartphone or similar mobile communications device now known or later developed that may be controlled via voice commands. In other embodiments, themobile computing device 100 may be a laptop computer, tablet computer, or other type of mobile computing device now known or later developed that may be controlled via voice commands. - The
mobile computing device 100 includes a plurality of components, such as one ormore processors 101,data storage 102,power management sub-system 103, user input/output interfaces 104, communications interfaces 105, andsensors 106. The individual components are connected via one or more data busses 115 or similar communications connections that are configured to enable the individual components to send and receive data to and from other components of themobile computing device 100. The individual components are also connected to thepower management sub-system 103 via apower distribution bus 116 or similar connection that is configured to provide operating power to the individual components of themobile computing device 100. - The one or
more processors 101 may include any type of general purpose processor (e.g., processors from Intel, Advanced Micro Devices (AMD), etc.) or any type of special purpose processors (e.g., application specific integrated circuits (ASICs), digital signal processors (DSP), Coder/Decoders (CODECs), etc.) now known or later developed. - The
data storage 102 is configured to store data as well as computer-readable program instructions 108. Thedata storage 102 may include any of the types of data storage described herein, including short term volatile memory (e.g., RAM) for storing data and/or program code or instructions during execution of the program code or instructions and/or longer-term non-volatile memory (e.g., ROM) for storing data and/or program code or instructions. In some embodiments, the data storage may be implemented as solid state and/or flash memory or other similar types of small form factor storage media. - The
power management sub-system 103 includes abattery power interface 112, anexternal power interface 113, and apower source selector 114. Thebattery power interface 112 is configured to receive operating power from a battery source, such as an internal battery, a removable battery that is electrically coupled to themobile computing device 100, or any other sort of battery or similar type of portable power supply now known or later developed. The battery that supplies operating power for themobile computing device 100 via thebattery interface 112 may be rechargeable or non-rechargeable. - The
external power interface 113 is configured to receive power from an external power source for themobile computing device 100. Examples of external power sources include electrical wall outlets, desktop docking stations, car docking stations or power cables, alarm clocks or entertainment systems with docking stations, or any other type of connection or device that is configured to provide themobile computing device 100 with operating power so that themobile computing device 100 does not need to rely on operating power from a battery via thebattery interface 112. - In some embodiments, the
power source selector 114 may be configured to selectively provide operating power from one of thebattery interface 112 or theexternal power interface 113. In embodiments with rechargeable batteries, when an external power source is connected via theexternal power interface 113, thepower source selector 114 may be configured to both (i) provide operating power to themobile computing device 100 from the external power supply via theexternal power interface 113 and (ii) charge the battery with power from the external power supply by providing an electrical connection from theexternal power interface 113 to thebattery interface 112. - The user input/output interfaces 104 include a plurality of input and output components, such as sound input/
output components 107, display input/output components 108, and anoptional keyboard 109. Thesound components 107 may include a microphone or other similar type of sound detection device now known or later developed that is configured to detect and receive sound waves. Thesound components 107 may also include one or more speakers or similar sound generation devices now known or later developed that are configured to generate sound waves. - The display input/
output components 108 may include a Liquid Crystal Display (LCD), plasma display, or similar type of display device now known or later developed that is configured to display a graphical user interface and/or other information to a user. In some embodiments, thedisplay 108 may include a touch-screen interface or other touch-sensitive components to enable a user to enter commands or other information (or otherwise interact with) themobile computing device 100. In some embodiments, thedisplay 108 may be configured to generate and display a virtual keyboard for data entry by a user. In other embodiments, themobile computing device 100 may optionally include aphysical keyboard 109. However, a keyboard (virtual or physical) is not required in all embodiments. Some embodiments may also include one or more external switches or buttons (not shown) to turn themobile computing device 100 on or off, to wake the device from a sleep state, to control volume or other similar functions. - The communications interfaces 105 may include one or more
wireless communications interfaces 110 and one or morewired interfaces 111. The wireless interfaces 110 may include any type of wireless communications interface configured to transmit and receive data according to any of a number of different wireless communications protocols, such as Bluetooth, IEEE 802.11 (IEEE 802.11 may refer to IEEE 802.11-2007, IEEE 802.11n-2009, or any other IEEE 802.11 revision) or other Wi-Fi variant, Code-Division Multiple Access (CDMA), Global System for Mobile Communications (GSM), 3rd Generation (3G), 4th Generation (4G), 3rd Generation Partnership Project—Long Term Evolution (3GPP-LTE), Worldwide Interoperability for Microwave Access (WiMAX), Zigbee, General Packet Radio Service (GPRS), Evolution-Data Optimized (EV-DO), High-Speed Downlink Packet Access (HSDPA), or any other type of wireless communications protocol now known or later developed. Thewired interfaces 111 may include any type of wired communications interface (e.g., electrical, optical, etc.) configured to transmit and receive data according to any of a number of different communications protocols, such as Universal Serial Bus (USB), Ethernet, IEEE 1394, Firewire, Thunderbolt, or any other type of serial or parallel data communications protocol now known or later developed. Thewired interfaces 110 andwireless interfaces 111 may be configured to communicate with other computing or communications devices via networks, e.g. the Internet, and/or communicate with peripheral devices, such as disk drives, external monitors, mouse or trackball/pad devices, and external keyboards etc., via Bluetooth, USB, or similar connections. In some examples,mobile computing device 100 may usewired interfaces 110 and/orwireless interfaces 111 to communicate with a server that performs speech recognition on behalf ofmobile computing device 100. - In some embodiments, the
wired communications interface 111 may also be configured to provide operating power via theexternal power interface 113 as shown bypower connection 117 from thewired interface 111 to theexternal power interface 113. For example, USB is one type of wired communications interface that can be configured to provide both communications and power. - Some embodiments of the
mobile computing device 100 may also include one ormore sensors 106. Thesensors 106 may include any one or more cameras or other image sensors, accelerometers, Global Positioning System (GPS) sensors, temperature sensors, or any other type of sensor known now or later developed that may be desirable to integrate with themobile computing device 100. - As described briefly above, a mobile computing device configured according to the disclosed embodiments may enable a user to enjoy the advantages of controlling the mobile computing device via voice commands without the limitation of having to either press or hold a physical button or switch on the mobile computing device or manually activate a virtual button or switch on a graphical user interface of the mobile computing device. In one embodiment, the above-described advantages may be achieved via a combination of a power detection routine and a trigger word detection subroutine that are executed by the
mobile computing device 100. -
FIG. 2A shows an example algorithm for apower detection routine 201 according to some embodiments, andFIG. 2B shows an example algorithm for a triggerword detection subroutine 210 according to some embodiments. The description of the algorithms ofFIGS. 2A and 2B includes references to the examplemobile computing device 100 and its corresponding components shown and described herein with reference inFIG. 1 . In some embodiments, thepower detection routine 201 and/or the triggerword detection subroutine 210 may be components of the operating system of themobile computing device 100. In other embodiments, thepower detection routine 201 and/or the triggerword detection subroutine 210 may be components of an application or widget installed on themobile computing device 100, such as an application or widget downloaded from an online application store. - In some embodiments, the
power detection routine 201 may be configured to operate continually (or at least substantially continually) on themobile computing device 100. Some embodiments may also allow a user to activate and deactivate thepower detection routine 201. In some embodiments, thepower detection routine 201 may be selectively activated or deactivated by a user. - In the embodiment of
FIGS. 2A and 2B , the triggerword detection subroutine 210 is a “subroutine” of thepower detection routine 201 in that the triggerword detection subroutine 210 is activated by thepower detection routine 201 under certain conditions, as described herein. However, the trigger word detection capability need not be implemented as a subroutine of a power detection routine in all embodiments. - The power detection routine starts at
block 202. Atblock 203, thepower detection routine 201 determines whether operating power for themobile computing device 100 is being received from either (1) an external power source via theexternal power interface 113 or (2) a battery power source via thebattery interface 112. In some embodiments, the determination may be made by thepower source selector 114. In other embodiments, the determination may be made by one ormore processors 101 configured to monitor any of thebattery interface 112, theexternal power interface 113, and/or thepower source selector 114. If an external power source is not detected (i.e., themobile computing device 100 is receiving operating power from a battery source), then thepower detection routine 201 ends atblock 207. After ending atblock 207,power detection routine 201 re-starts atblock 202, either immediately (or after a brief waiting period) so thatpower detection routine 201 operates continually (or at least substantially continually). - However, if at
block 203, thepower detection routine 201 determines that themobile computing device 100 is receiving operating power from an external power source via theexternal power interface 113, then thepower detection routine 201 advances to block 204. Atblock 204, thepower detection routine 201 activates the triggerword detection subroutine 210. Once it is activated by thepower detection routine 201 atblock 204, the triggerword detection subroutine 210 may be configured to execute continually (or at least substantially continually) until it is deactivated. - After activating the trigger
word detection subroutine 210 atblock 204, the power detection routine advances to block 205. Atblock 205, thepower detection routine 201 determines whether thecommunications device 100 is receiving operating power from an external power source via theexternal power interface 113. In some embodiments, the determination may be made by thepower source selector 114. In other embodiments, the determination may be made by one ormore processors 101 configured to monitor any of thebattery interface 112, theexternal power interface 113, and/or thepower source selector 114. - If the
mobile computing device 100 is receiving its operating power from an external power source, then thepower detection routine 201 returns to block 205. But if thepower detection routine 201 determines that themobile computing device 100 is no longer receiving its operating power from an external power source (i.e., the external power source has been disconnected from theexternal power interface 113 and/or themobile computing device 100 has switched to battery power), then thepower detection algorithm 201 advances to block 206. Atblock 206, thepower detection routine 201 deactivates the triggerword detection subroutine 210. Thepower detection routine 201 then ends atblock 207. After ending atblock 207,power detection routine 201 re-starts atblock 202, either immediately (or after a brief waiting period) so thatpower detection routine 201 operates continually (or at least substantially continually). -
FIG. 2B shows an example algorithm of a triggerword detection subroutine 210. As described with respect toFIG. 2A , once thepower detection routine 201 activates the triggerword detection subroutine 210 atblock 204, the triggerword detection subroutine 210 is configured to execute continually (or at least substantially continually) until it is deactivated. The triggerword detection subroutine 210 may be deactivated by thepower detection routine 201 atblock 206 as described above. In some embodiments, the triggerword detection subroutine 210 may be selectively activated and/or deactivated by a user. - The trigger
word detection subroutine 210 starts atblock 212. Atblock 213, the trigger word detection subroutine receives a sound sample. The sound sample may be received via thesound input 107 of themobile computing device 100. In some embodiments, thesound input 107 may include a microphone or other type of sound input or detection device. Some embodiments may include a sound detector that triggers the activation of a microphone. Once activated, the microphone captures a sound sample. In other embodiments, a microphone may be configured to continually (or at least substantially continually) capture sound samples once the triggerword detection subroutine 210 has been activated. - At
block 214, the triggerword detection subroutine 210 determines whether the sound sample received atblock 213 includes speech. Determining whether the sound sample includes speech can be accomplished by any of a number of speech detection algorithms now know or later developed. In some embodiments, the speech detection algorithm may be executed by the one ormore processors 101. In other embodiments, the speech detection algorithm may be executed by one or more components of thesound input 107. - If the sound sample does not include speech, then the trigger
word detection subroutine 210 ends atblock 217 and, thereafter, re-starts at block 212 (either immediately or after a brief waiting period) to receive an additional sound sample atblock 213. However, if the sound sample includes speech, then the triggerword detection subroutine 210 advances to block 215. - At
block 215, the speech from the sound sample is analyzed to determine whether the speech includes at least one of a set of one or more trigger words. Trigger words are described in more detail with respect toFIGS. 3A-3C . The sound sample may be analyzed with any of a number of sound or voice recognition algorithms now known or later developed. For example, speech recognition may be performed on the spoken input to obtain text, and the text may be analyzed to determine whether the text includes one or more trigger words. The speech recognition may be performed by themobile computing device 100, by a server with whichmobile computing device 100 communicates, or by a combination thereof. - If the speech does not includes at least one trigger word, then the trigger word detection subroutine ends at
block 217 and, thereafter, re-starts at block 212 (either immediately or after a brief waiting period) to receive an additional sound sample atblock 213. However, if the received speech includes at least one trigger word, then the trigger word detection subroutine advances to block 216. Atblock 216, the triggerword detection subroutine 210 launches an application corresponding to the at least one trigger word determined atblock 215. After launching the application atblock 216, the triggerword detection subroutine 210 ends atblock 217. After ending atblock 217, the triggerword detection subroutine 210 re-starts at block 212 (either immediately or after a brief waiting period) to receive a new sound sample atblock 213. - As described above, the trigger
word detection subroutine 210 continues in this manner continually (or at least substantially continually) until the triggerword detection subroutine 210 is deactivated by either thepower detection routine 201 or a user. -
FIGS. 3A-3B show examples of amobile computing device 300 launching applications in response to detecting trigger words according to some embodiments. Themobile computing device 300 may be similar to themobile computing device 100 shown and described with respect toFIG. 1 . Similarly, themobile computing device 300 may be configured to execute thepower detection routine 201 and/or the triggerword detection subroutine 210 shown and described with respect toFIGS. 2A-2B . - In
FIGS. 3A-3B , themobile computing device 300 is receiving operating power from an external power source connected to themobile computing device 300 viacable 301.Power icon 302 on thedisplay 303 of themobile computing device 300 indicates that themobile computing device 300 is receiving power from an external power source. Because themobile computing device 300 is receiving operating power from an external power source, the power detection routine 201 (FIG. 2A ) will have activated the trigger word detection subroutine 210 (FIG. 2B ). As described herein, once activated, the triggerword detection subroutine 210 is configured to operate continually (or at least substantially continually) until it is deactivated by either thepower detection routine 201 or a user. - While activated, the trigger
word detection subroutine 210 is receiving sound samples via themicrophone 304 of themobile computing device 300. When themobile computing device 300 determines that a sound sample received via itsmicrophone 304 includes at least one trigger word of a set of one or more trigger words, themobile computing device 300 is configured to launch an application corresponding to the at least one determined trigger word. - In some embodiments, the
mobile computing device 300 may be configured to detect a single trigger word. For example, the trigger word may be “Android” (or perhaps some other user-defined trigger word or group of words). In operation, when themobile computing device 300 detects the trigger word (or group of words), the mobile computing device may launch a voice command prompt.FIG. 3A shows an example of themobile computing device 300 launching a voice command prompt 305 in response to detecting a trigger word corresponding to thevoice command prompt 305. - After the
voice command prompt 305 has been launched, a user can then issue a spoken command string to themobile computing device 300. For example, the spoken command string may be “Call . . . Jane Smith . . . Office.” Here, the command includes an action (“call”), an object of the action (“Jane Smith”), and a parameter (“Office”). In response to receiving the spoken command string, themobile computing device 300 may (i) launch an application corresponding to the action (and possibly one or more parameters associated with the action), and (ii) execute the application based on the object of the action (and possibly one or more parameters associated with the object of the action).FIG. 3B shows themobile computing device 300 launching aphone application 306 and calling “Jane Smith” at her office number in response to the spoken command string received via the voice command prompt 305 (FIG. 3A ). - In other embodiments, the
mobile computing device 300 may be configured to launch other types of applications in response to particular spoken command strings received via thevoice command prompt 305. In one example, themobile computing device 300 may be configured to activate or deactivate a particular application, feature, or service. For example, themobile computing device 300 may be configured to silence or deactivate its ringer in response to the command string such as “Configuration Ringer Silent.” In another example, themobile computing device 300 may be configured to launch a multimedia player and play a particular song or video in response to a spoken command string received via thevoice command prompt 305. For example, the spoken command string “Play Song All You Need Is Love” may cause themobile computing device 300 to launch a multimedia player and play the song “All You Need Is Love” by The Beatles. - As described above, the spoken command string may include one or more parameters associated with either an action or an object to ameliorate or otherwise minimize ambiguity. In the example shown in the
FIG. 3B , the “call” was the action, “Jane Smith” was the object, and “Office” was the parameter associated with object “Jane Smith.” The parameter “Office” specifies which of Jane Smith's numbers to call (i.e., home, office, mobile, etc.). - In the foregoing example, the trigger word was “Android” and the command string was “Call Jane Smith Office.” In some embodiments, the user may need to pause slightly after the trigger word before speaking a command string to allow time for the
mobile computing device 300 to launch thevoice command prompt 305. Similarly, the user may need to pause slightly after speaking the “action” for the voice command prompt to allow time for themobile computing device 300 to launch the application corresponding to the action, e.g., the phone application in this example. - However, in other embodiments, the
mobile computing device 300 may be configured to cache a brief history of spoken words, e.g., up to about 5-30 seconds of spoken words or perhaps a configurable number of words, e.g., up to 10-20 spoken words, depending on the configuration. In embodiments that cache a brief history of spoken words, thecomputing device 300 may be configured to analyze the words in the cache to identify actions, objects, and parameters that may follow a trigger word (or words). One advantage of caching a brief history of spoken words is that a user may not need to pause long (or at all) after a trigger word or after an “action” to wait for themobile computing device 300 to launch the voice command prompt in response to the trigger word or to launch an application in response to the “action” received via the voice command prompt. - In some embodiments, the
mobile computing device 300 may be configured to detect multiple trigger words (or groups of words, i.e., a trigger phrase), where individual trigger words (or phrases) correspond to different applications or actions to take in a particular application. In the previous example, a single trigger word (or perhaps trigger phrase), such as “Android” for example, caused themobile computing device 300 to launch a voice command prompt 305 (FIG. 3A ). Once thevoice command prompt 305 was launched, themobile computing device 300 could receive spoken command strings to launch other applications (i.e., a phone application as shown inFIG. 3B ). But in embodiments where themobile computing device 300 is configured to detect multiple trigger words (or sets of trigger words or phrases), themobile computing device 300 may be able to launch applications and process commands faster than embodiments that may rely on a single trigger word to first open avoice command prompt 305 before receiving spoken command strings to launch applications. - In some embodiments, the set of trigger words may include a specific trigger word (or words) for a voice command prompt, a specific trigger word (or words) for a phone application, a specific trigger word (or words) for a text messaging application, a specific trigger word (or words) for an email application, and a specific trigger word (or words) for a multimedia player application, etc. In operation, the
mobile computing device 300 could have many different trigger words or phrases corresponding to many different applications and/or functions. In some embodiments, a user may be able to configure or customize particular trigger words of phrases and associate those customize trigger words or phrases with particular applications or other computing, communications, or entertainment tasks. For example, a user may configure a trigger phrase “Favorite Song” to cause the mobile computing device to launch a multimedia player and play the song “All You Need Is Love” by The Beatles. - Although
FIG. 3B was described above with respect to launching a telephone application in response to a spoken command string received via a voice command prompt,FIG. 3B can also correspond to an example of themobile computing device 300 launching a telephone application in response to detecting a trigger word corresponding to the telephone application. For example, the trigger word corresponding to the phone application may be “call.” When themobile computing device 300 detects the trigger word “call,” the mobile computing device may launch the phone application and then listen for further instructions, such as the number or person to call. -
FIG. 3C shows an example of themobile computing device 300 launching anemail application 307 in response to detecting a trigger word corresponding to theemail application 307. In this example, the trigger word corresponding to theemail application 307 may be “mail,” and upon detecting the “mail” trigger word, themobile computing device 300 may launch theemail application 307 and then listen for further instructions, such as the recipient, subject, and body of the email message. -
FIG. 4 shows anexample method 400 according to some embodiments. Themethod 400 begins atblock 401, where a mobile computing device determines whether it is receiving operating power from one of (i) an external power source or (ii) one or more batteries configured to power the mobile computing device. The mobile computing device may be similar to any of the mobile computing devices described elsewhere herein. In this example, the mobile device includes a trigger word detection subroutine that is activatable by a user input and automatically in response to determining that the mobile computing device is receiving external power. - At
block 402, the mobile computing device activates the trigger word detection subroutine in response to determining that it is receiving power from the external power source. The trigger word detection subroutine is shown inblocks 402 a-402 d, which are described in more detail below. In some embodiments, the mobile computing device may additionally or alternatively activate the trigger word detection subroutine in response to receiving an input from a user to activate the trigger word detection subroutine even when the mobile computing device is receiving power from the one or more batteries. - At
block 403, the mobile computing device deactivates the trigger word detection subroutine in response to determining that it is receiving operating power from the one or more batteries. In some embodiments, the mobile computing device may additionally or alternatively deactivate the trigger word detection subroutine in response to receiving an input from a user to deactivate the trigger word detection subroutine even when the mobile computing device is receiving power from an external power source. - The trigger word detection subroutine is shown in
blocks 402 a-402 d. In operation, once activated, the trigger word detection subroutine may be configured to execute continually (or at least substantially continually) until it is deactivated either atstep 403 or by a user. - The trigger word detection subroutine begins at
block 402 a where the mobile computing device receives spoken input via a microphone. Atblock 402 b, the mobile computing device obtains text corresponding to the spoken input, based on speech recognition performed on the spoken input. The speech recognition could be performed by the mobile computing device. Alternatively, the speech recognition could be performed, in whole or in part, by one or more devices external to the mobile computing device. For example, the mobile computing device may transmit the spoken input to a server, the server may perform speech recognition on the spoken input to determine a corresponding text, and the mobile computing device may obtain the text from the server. - At
block 402 c, the mobile computing device analyzes the text to determine whether the text includes at least one of a set of one or more predefined trigger words. Atblock 402 d, in response to determining that the text includes at least one of the set of one or more predefined trigger words, the mobile computing device launches an application corresponding to the at least one trigger word included in the text. - In some embodiments, the application corresponding to the at least one trigger word may be a voice command prompt application configured to receive a voice command from a user. The voice command may comprise one or more of (i) an action, (ii) an object of the action, and (iii) one or more parameters associated with the action or the object. The voice command prompt and the voice commands received via the voice command prompt may be similar to the voice command prompts and voice commands described elsewhere herein.
- In further embodiments, the application corresponding to the at least one trigger word may be any of a voice command prompt, a communications application, a multimedia application, or an application for configuring settings of the mobile computing device.
- In some embodiments, a speech recognition mode of a mobile computing device can be activated by coupling the mobile computing device to an external power source.
FIG. 5 shows anexample method 500 according to such embodiments. Themethod 500 begins at block 501, when a mobile computing device detects coupling of the mobile computing device to an external power source. In response to the detected coupling, the mobile computing device activates a speech recognition mode, atblock 502. Blocks 502-505 describe an example of how the speech recognition mode may operate after being activated. - At
block 503, the mobile computing device receives spoken input via a microphone. At block 504, the mobile computing device determines whether the spoken input includes one or more trigger words. This can involve speech recognition performed by the mobile computing device and/or by a server, as described above. In response to determining that the spoken input includes at least one trigger word, the mobile computing device launches an application corresponding to the at least one trigger word, atblock 505. - While various aspects and embodiments have been disclosed herein, other aspects and embodiments will be apparent to those skilled in the art. The various aspects and embodiments disclosed herein are for purposes of illustration only and are not intended to be limiting, with the true scope and spirit being indicated by the following claims.
Claims (21)
1-20. (canceled)
21. A method comprising:
determining whether a computing device is receiving operating power from an external power source, wherein the computing device has a trigger word detection subroutine that is activatable by a user input and automatically in response to determining that the computing device is receiving external power; and
in response to determining that the computing device is receiving operating power from the external power source, taking a trigger word detection action, wherein the trigger word detection action includes:
receiving spoken input via a microphone of the computing device,
based on speech recognition performed on the spoken input, obtaining text,
determining whether the text includes one or more trigger words associated with for a voice command prompt action, and
in response to determining that the text includes the one or more trigger words associated with the voice command prompt action, taking the voice command prompt action, wherein the voice command prompt action is configured to receive via the microphone additional spoken input that causes the computing device to take one or more other actions, and wherein taking the voice command prompt action comprises displaying a voice command prompt on the computing device.
22. The method of claim 21 , further comprising: taking the trigger word detection action continually until it is deactivated.
23. The method of claim 21 , further comprising: deactivating the trigger word detection action in response to determining that the computing device is receiving operating power front one or more batteries.
24. The method of claim 21 , further comprising:
receiving the user input; and
taking the trigger word detection action in response to receiving the user input regardless of whether the computing device is receiving operating power from the external power source.
25. The method of claim 21 , wherein the one or more other actions include at least one of a communications action, a multimedia action, and an action for configuring settings of the computing device.
26. The method of claim 21 , wherein the external power source is at least one of a charger associated with the computing device and a docking station for the computing device.
27. The method of claim 21 , wherein one or more of the determining step, the launching step, and the trigger word detection action are components of a computing device operating system.
28. The method of claim 21 , wherein the additional spoken input comprises a corresponding action that corresponds to the additional spoken input, and wherein the method further comprises:
in response to receiving the additional spoken input, taking the corresponding action on the computing device.
29. The method of claim 28 , wherein the additional spoken input further includes one or more parameters, and wherein taking the corresponding action comprises taking the corresponding action based on the one or more parameters.
30. The computing device of claim 28 , wherein the additional spoken input comprises an object of the corresponding action, and wherein taking the corresponding action comprises taking the corresponding action based on the object of the corresponding action.
31. A computing device comprising:
a sound detector;
a display;
a power selector configured to selectively power the computing device from one of either an external power source or one or more batteries; and
one or more processors configured for
continually taking a trigger word detection action while the power selector is powering the computing device from the external power source and stop taking the trigger word detection action when the power selector is powering the computing device from the one or more batteries,
wherein the trigger word detection action comprises determining whether spoken input received via the sound detector includes one or more trigger words associated with a voice command prompt action, and in response to determining that the spoken input includes the one or more trigger words associated with the voice command prompt action, taking the voice command prompt action, wherein the voice command prompt action is configured to receive via the sound detector additional spoken input that causes the computing device to take one or more other actions, and wherein taking the voice command prompt action comprises displaying a voice command prompt on the display.
32. The computing device of claim 31 , further comprising: a user interface configured to receive an input from a user to take the trigger word detection action when the power selector is powering the computing device from the one or more batteries.
33. The computing device of claim 31 , wherein the one or more other applications include at least one of a communications application, a multimedia application, or an application for configuring settings of the computing device.
34. The computing device of claim 31 , wherein the additional spoken input comprises a corresponding action that corresponds to the additional spoken input, and wherein the one or more processors are further configured to:
in response to receiving the additional spoken input, taking the corresponding action.
35. The computing device of claim 34 , wherein the additional spoken input further includes one or more parameters associated with the corresponding action, and wherein taking the corresponding action comprises taking the corresponding action based on the one or more parameters.
36. The computing device of claim 34 , wherein the additional spoken input comprises an object of the corresponding action, and wherein taking the corresponding action comprises taking the corresponding action based on the object of the corresponding action
37. A non-transitory computer readable medium having stored therein executable instructions that, when executed by a computing device having at least one processor, causes the computing device to perform functions comprising:
determining whether the computing device is receiving operating power from either an external power source or an internal batter source; and
in response to determining that the computing device is receiving operating power from the external power source, taking a trigger word detection action, wherein the trigger word detection action includes:
continually determining whether spoken input received via a microphone includes one or more trigger words associated with an action for a voice command prompt and,
in response to determining that the spoken input includes the one or more trigger words associated with the voice command prompt, taking the action for the voice command prompt, wherein the voice command prompt is configured to receive via the microphone additional spoken input that causes the computing device to take one or more other actions, and wherein taking the action for the voice command prompt comprises displaying a voice command prompt on the computing device.
38. The non-transitory computer readable medium of claim 37 , wherein the functions further comprise: deactivating the trigger word detection subroutine in response to determining that the computing device is receiving operating power from the internal battery source.
39. The non-transitory computer readable medium of claim 37 , wherein the functions further comprise:
receiving a user input; and
taking the trigger word detection action in response to receiving the user input regardless of whether the computing device is receiving operating power from the external power source.
40. The non-transitory computer readable medium of claim 37 , wherein the additional spoken input comprises a corresponding action that corresponds to the additional spoken input, and wherein the functions further comprise:
in response to receiving the additional spoken input, taking the corresponding action on the computing device.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US14/347,456 US20140244253A1 (en) | 2011-09-30 | 2012-09-27 | Systems and Methods for Continual Speech Recognition and Detection in Mobile Computing Devices |
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201161542074P | 2011-09-30 | 2011-09-30 | |
US13/621,068 US8452597B2 (en) | 2011-09-30 | 2012-09-15 | Systems and methods for continual speech recognition and detection in mobile computing devices |
US14/347,456 US20140244253A1 (en) | 2011-09-30 | 2012-09-27 | Systems and Methods for Continual Speech Recognition and Detection in Mobile Computing Devices |
PCT/US2012/057576 WO2013049358A1 (en) | 2011-09-30 | 2012-09-27 | Systems and methods for continual speech recognition and detection in mobile computing devices |
Related Parent Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/621,068 Continuation US8452597B2 (en) | 2011-09-30 | 2012-09-15 | Systems and methods for continual speech recognition and detection in mobile computing devices |
PCT/US2012/057576 A-371-Of-International WO2013049358A1 (en) | 2011-09-30 | 2012-09-27 | Systems and methods for continual speech recognition and detection in mobile computing devices |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/045,128 Continuation US20160180851A1 (en) | 2011-09-30 | 2016-02-16 | Systems and Methods for Continual Speech Recognition and Detection in Mobile Computing Devices |
Publications (1)
Publication Number | Publication Date |
---|---|
US20140244253A1 true US20140244253A1 (en) | 2014-08-28 |
Family
ID=47993412
Family Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/621,068 Active US8452597B2 (en) | 2011-09-30 | 2012-09-15 | Systems and methods for continual speech recognition and detection in mobile computing devices |
US14/347,456 Abandoned US20140244253A1 (en) | 2011-09-30 | 2012-09-27 | Systems and Methods for Continual Speech Recognition and Detection in Mobile Computing Devices |
US15/045,128 Abandoned US20160180851A1 (en) | 2011-09-30 | 2016-02-16 | Systems and Methods for Continual Speech Recognition and Detection in Mobile Computing Devices |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/621,068 Active US8452597B2 (en) | 2011-09-30 | 2012-09-15 | Systems and methods for continual speech recognition and detection in mobile computing devices |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/045,128 Abandoned US20160180851A1 (en) | 2011-09-30 | 2016-02-16 | Systems and Methods for Continual Speech Recognition and Detection in Mobile Computing Devices |
Country Status (4)
Country | Link |
---|---|
US (3) | US8452597B2 (en) |
EP (1) | EP2761615A1 (en) |
CN (1) | CN103930945B (en) |
WO (1) | WO2013049358A1 (en) |
Cited By (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20150088525A1 (en) * | 2013-09-24 | 2015-03-26 | Tencent Technology (Shenzhen) Co., Ltd. | Method and apparatus for controlling applications and operations on a terminal |
US20150127335A1 (en) * | 2013-11-07 | 2015-05-07 | Nvidia Corporation | Voice trigger |
WO2016164842A1 (en) * | 2015-04-10 | 2016-10-13 | Angel.Com Incorporated | Visual interactive voice response system |
US9530410B1 (en) * | 2013-04-09 | 2016-12-27 | Google Inc. | Multi-mode guard for voice commands |
US9769550B2 (en) | 2013-11-06 | 2017-09-19 | Nvidia Corporation | Efficient digital microphone receiver process and system |
US10147444B2 (en) | 2015-11-03 | 2018-12-04 | Airoha Technology Corp. | Electronic apparatus and voice trigger method therefor |
JP2020510852A (en) * | 2017-03-31 | 2020-04-09 | アリババ・グループ・ホールディング・リミテッドAlibaba Group Holding Limited | Voice function control method and device |
WO2020262811A1 (en) * | 2019-06-26 | 2020-12-30 | Samsung Electronics Co., Ltd. | System and method for complex task machine learning |
US11321048B2 (en) | 2020-02-25 | 2022-05-03 | Motorola Solutions, Inc. | Method and apparatus for temporary hands-free voice interaction |
Families Citing this family (225)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8645137B2 (en) | 2000-03-16 | 2014-02-04 | Apple Inc. | Fast, language-independent method for user authentication by voice |
US8677377B2 (en) | 2005-09-08 | 2014-03-18 | Apple Inc. | Method and apparatus for building an intelligent automated assistant |
US9318108B2 (en) | 2010-01-18 | 2016-04-19 | Apple Inc. | Intelligent automated assistant |
US8977255B2 (en) | 2007-04-03 | 2015-03-10 | Apple Inc. | Method and system for operating a multi-function portable electronic device using voice-activation |
US10002189B2 (en) | 2007-12-20 | 2018-06-19 | Apple Inc. | Method and apparatus for searching using an active ontology |
US9330720B2 (en) | 2008-01-03 | 2016-05-03 | Apple Inc. | Methods and apparatus for altering audio output signals |
US8996376B2 (en) | 2008-04-05 | 2015-03-31 | Apple Inc. | Intelligent text-to-speech conversion |
US20100030549A1 (en) | 2008-07-31 | 2010-02-04 | Lee Michael M | Mobile device having human language translation capability with positional feedback |
US8676904B2 (en) | 2008-10-02 | 2014-03-18 | Apple Inc. | Electronic devices with voice command and contextual data processing capabilities |
US10241752B2 (en) | 2011-09-30 | 2019-03-26 | Apple Inc. | Interface for a virtual digital assistant |
US10241644B2 (en) | 2011-06-03 | 2019-03-26 | Apple Inc. | Actionable reminder entries |
US10255566B2 (en) | 2011-06-03 | 2019-04-09 | Apple Inc. | Generating and processing task items that represent tasks to perform |
US9431006B2 (en) | 2009-07-02 | 2016-08-30 | Apple Inc. | Methods and apparatuses for automatic speech recognition |
US10276170B2 (en) | 2010-01-18 | 2019-04-30 | Apple Inc. | Intelligent automated assistant |
US8682667B2 (en) | 2010-02-25 | 2014-03-25 | Apple Inc. | User profiling for selecting user specific voice input processing information |
US9262612B2 (en) | 2011-03-21 | 2016-02-16 | Apple Inc. | Device access using voice authentication |
US8731936B2 (en) * | 2011-05-26 | 2014-05-20 | Microsoft Corporation | Energy-efficient unobtrusive identification of a speaker |
US10057736B2 (en) | 2011-06-03 | 2018-08-21 | Apple Inc. | Active transport based notifications |
US8994660B2 (en) | 2011-08-29 | 2015-03-31 | Apple Inc. | Text correction processing |
US8768707B2 (en) * | 2011-09-27 | 2014-07-01 | Sensory Incorporated | Background speech recognition assistant using speaker verification |
US8996381B2 (en) * | 2011-09-27 | 2015-03-31 | Sensory, Incorporated | Background speech recognition assistant |
US9031847B2 (en) * | 2011-11-15 | 2015-05-12 | Microsoft Technology Licensing, Llc | Voice-controlled camera operations |
US10134385B2 (en) | 2012-03-02 | 2018-11-20 | Apple Inc. | Systems and methods for name pronunciation |
US9280610B2 (en) | 2012-05-14 | 2016-03-08 | Apple Inc. | Crowd sourcing information to fulfill user requests |
US10417037B2 (en) | 2012-05-15 | 2019-09-17 | Apple Inc. | Systems and methods for integrating third party services with a digital assistant |
US9721563B2 (en) | 2012-06-08 | 2017-08-01 | Apple Inc. | Name recognition system |
US9547647B2 (en) | 2012-09-19 | 2017-01-17 | Apple Inc. | Voice-based media searching |
US9275637B1 (en) * | 2012-11-06 | 2016-03-01 | Amazon Technologies, Inc. | Wake word evaluation |
KR101990037B1 (en) * | 2012-11-13 | 2019-06-18 | 엘지전자 주식회사 | Mobile terminal and control method thereof |
US9424405B2 (en) * | 2012-11-28 | 2016-08-23 | Apple Inc. | Using receipts to control assignments of items of content to users |
US9704486B2 (en) * | 2012-12-11 | 2017-07-11 | Amazon Technologies, Inc. | Speech recognition power management |
CN103871408B (en) * | 2012-12-14 | 2017-05-24 | 联想(北京)有限公司 | Method and device for voice identification and electronic equipment |
US9047857B1 (en) * | 2012-12-19 | 2015-06-02 | Rawles Llc | Voice commands for transitioning between device states |
AU2021202255B9 (en) * | 2013-02-07 | 2022-06-09 | Apple Inc. | Voice trigger for a digital assistant |
EP2954514B1 (en) * | 2013-02-07 | 2021-03-31 | Apple Inc. | Voice trigger for a digital assistant |
US10652394B2 (en) | 2013-03-14 | 2020-05-12 | Apple Inc. | System and method for processing voicemail |
US10748529B1 (en) * | 2013-03-15 | 2020-08-18 | Apple Inc. | Voice activated device for use with a voice-based digital assistant |
EP2801974A3 (en) * | 2013-05-09 | 2015-02-18 | DSP Group Ltd. | Low power activation of a voice activated device |
US9026176B2 (en) | 2013-05-12 | 2015-05-05 | Shyh-Jye Wang | Message-triggered voice command interface in portable electronic devices |
US10020008B2 (en) | 2013-05-23 | 2018-07-10 | Knowles Electronics, Llc | Microphone and corresponding digital interface |
EP3575924B1 (en) | 2013-05-23 | 2022-10-19 | Knowles Electronics, LLC | Vad detection microphone |
US9711166B2 (en) | 2013-05-23 | 2017-07-18 | Knowles Electronics, Llc | Decimation synchronization in a microphone |
WO2014197334A2 (en) | 2013-06-07 | 2014-12-11 | Apple Inc. | System and method for user-specified pronunciation of words for speech synthesis and recognition |
US9582608B2 (en) | 2013-06-07 | 2017-02-28 | Apple Inc. | Unified ranking with entropy-weighted information for phrase-based semantic auto-completion |
WO2014197336A1 (en) | 2013-06-07 | 2014-12-11 | Apple Inc. | System and method for detecting errors in interactions with a voice-based digital assistant |
WO2014197335A1 (en) | 2013-06-08 | 2014-12-11 | Apple Inc. | Interpreting and acting upon commands that involve sharing information with remote devices |
WO2014200728A1 (en) | 2013-06-09 | 2014-12-18 | Apple Inc. | Device, method, and graphical user interface for enabling conversation persistence across two or more instances of a digital assistant |
US10176167B2 (en) | 2013-06-09 | 2019-01-08 | Apple Inc. | System and method for inferring user intent from speech inputs |
US9747899B2 (en) | 2013-06-27 | 2017-08-29 | Amazon Technologies, Inc. | Detecting self-generated wake expressions |
CN103309618A (en) | 2013-07-02 | 2013-09-18 | 姜洪明 | Mobile operating system |
GB2553040B (en) * | 2013-07-10 | 2018-04-18 | Cirrus Logic Int Semiconductor Ltd | Sensor input recognition |
GB2526980B (en) * | 2013-07-10 | 2017-04-12 | Cirrus Logic Int Semiconductor Ltd | Sensor input recognition |
US9502028B2 (en) | 2013-10-18 | 2016-11-22 | Knowles Electronics, Llc | Acoustic activity detection apparatus and method |
CN103500578B (en) * | 2013-10-22 | 2016-05-11 | 上海云视科技股份有限公司 | Speech control method and apparatus |
US9147397B2 (en) | 2013-10-29 | 2015-09-29 | Knowles Electronics, Llc | VAD detection apparatus and method of operating the same |
US10296160B2 (en) | 2013-12-06 | 2019-05-21 | Apple Inc. | Method for extracting salient dialog usage from live data |
GB2524222B (en) | 2013-12-18 | 2018-07-18 | Cirrus Logic Int Semiconductor Ltd | Activating speech processing |
EP3084760A4 (en) * | 2013-12-20 | 2017-08-16 | Intel Corporation | Transition from low power always listening mode to high power speech recognition mode |
KR102179506B1 (en) | 2013-12-23 | 2020-11-17 | 삼성전자 주식회사 | Electronic apparatus and control method thereof |
US11132173B1 (en) * | 2014-02-20 | 2021-09-28 | Amazon Technologies, Inc. | Network scheduling of stimulus-based actions |
US9430463B2 (en) | 2014-05-30 | 2016-08-30 | Apple Inc. | Exemplar-based natural language processing |
US9715875B2 (en) | 2014-05-30 | 2017-07-25 | Apple Inc. | Reducing the need for manual start/end-pointing and trigger phrases |
US9966065B2 (en) | 2014-05-30 | 2018-05-08 | Apple Inc. | Multi-command single utterance input method |
US10170123B2 (en) | 2014-05-30 | 2019-01-01 | Apple Inc. | Intelligent assistant for home automation |
US9842101B2 (en) | 2014-05-30 | 2017-12-12 | Apple Inc. | Predictive conversion of language input |
US9633004B2 (en) | 2014-05-30 | 2017-04-25 | Apple Inc. | Better resolution when referencing to concepts |
US9355640B2 (en) | 2014-06-04 | 2016-05-31 | Google Inc. | Invoking action responsive to co-presence determination |
US9338493B2 (en) | 2014-06-30 | 2016-05-10 | Apple Inc. | Intelligent automated assistant for TV user interactions |
US9953646B2 (en) | 2014-09-02 | 2018-04-24 | Belleau Technologies | Method and system for dynamic speech recognition and tracking of prewritten script |
US9818400B2 (en) | 2014-09-11 | 2017-11-14 | Apple Inc. | Method and apparatus for discovering trending terms in speech requests |
US10789041B2 (en) | 2014-09-12 | 2020-09-29 | Apple Inc. | Dynamic thresholds for always listening speech trigger |
US9886432B2 (en) | 2014-09-30 | 2018-02-06 | Apple Inc. | Parsimonious handling of word inflection via categorical stem + suffix N-gram language models |
US10074360B2 (en) | 2014-09-30 | 2018-09-11 | Apple Inc. | Providing an indication of the suitability of speech recognition |
US9668121B2 (en) | 2014-09-30 | 2017-05-30 | Apple Inc. | Social reminders |
US10127911B2 (en) | 2014-09-30 | 2018-11-13 | Apple Inc. | Speaker identification and unsupervised speaker adaptation techniques |
US9646609B2 (en) | 2014-09-30 | 2017-05-09 | Apple Inc. | Caching apparatus for serving phonetic pronunciations |
US9449442B2 (en) * | 2014-10-23 | 2016-09-20 | Vivint, Inc. | Interface of an automation system |
CN105740056B (en) * | 2014-12-08 | 2019-03-29 | 联想(北京)有限公司 | Information processing method and electronic equipment |
US9830080B2 (en) | 2015-01-21 | 2017-11-28 | Knowles Electronics, Llc | Low power voice trigger for acoustic apparatus and method |
US9633661B1 (en) | 2015-02-02 | 2017-04-25 | Amazon Technologies, Inc. | Speech-responsive portable speaker |
US10121472B2 (en) | 2015-02-13 | 2018-11-06 | Knowles Electronics, Llc | Audio buffer catch-up apparatus and method with two microphones |
CN104715754A (en) * | 2015-03-05 | 2015-06-17 | 北京华丰亨通科贸有限公司 | Method and device for rapidly responding to voice commands |
US10152299B2 (en) | 2015-03-06 | 2018-12-11 | Apple Inc. | Reducing response latency of intelligent automated assistants |
US9865280B2 (en) | 2015-03-06 | 2018-01-09 | Apple Inc. | Structured dictation using intelligent automated assistants |
US10567477B2 (en) | 2015-03-08 | 2020-02-18 | Apple Inc. | Virtual assistant continuity |
US9886953B2 (en) | 2015-03-08 | 2018-02-06 | Apple Inc. | Virtual assistant activation |
US9721566B2 (en) * | 2015-03-08 | 2017-08-01 | Apple Inc. | Competing devices responding to voice triggers |
US9899019B2 (en) | 2015-03-18 | 2018-02-20 | Apple Inc. | Systems and methods for structured stem and suffix language models |
WO2016161641A1 (en) * | 2015-04-10 | 2016-10-13 | 华为技术有限公司 | Voice recognition method, voice wake-up device, voice recognition device and terminal |
US9842105B2 (en) | 2015-04-16 | 2017-12-12 | Apple Inc. | Parsimonious continuous-space phrase representations for natural language processing |
US10460227B2 (en) | 2015-05-15 | 2019-10-29 | Apple Inc. | Virtual assistant in a communication session |
US10083688B2 (en) | 2015-05-27 | 2018-09-25 | Apple Inc. | Device voice control for selecting a displayed affordance |
US10200824B2 (en) | 2015-05-27 | 2019-02-05 | Apple Inc. | Systems and methods for proactively identifying and surfacing relevant content on a touch-sensitive device |
US10127220B2 (en) | 2015-06-04 | 2018-11-13 | Apple Inc. | Language identification from short strings |
US10101822B2 (en) | 2015-06-05 | 2018-10-16 | Apple Inc. | Language input correction |
US9578173B2 (en) | 2015-06-05 | 2017-02-21 | Apple Inc. | Virtual assistant aided communication with 3rd party service in a communication session |
US11025565B2 (en) | 2015-06-07 | 2021-06-01 | Apple Inc. | Personalized prediction of responses for instant messaging |
US10186254B2 (en) | 2015-06-07 | 2019-01-22 | Apple Inc. | Context-based endpoint detection |
US10255907B2 (en) | 2015-06-07 | 2019-04-09 | Apple Inc. | Automatic accent detection using acoustic models |
US10001964B2 (en) * | 2015-06-08 | 2018-06-19 | Intel Corporation | Systems, methods, and devices for handling Wi-Fi and bluetooth audio |
KR102296174B1 (en) * | 2015-06-26 | 2021-08-31 | 삼성전자주식회사 | Electronic apparatus and method for converting audio thereof |
US20160378747A1 (en) | 2015-06-29 | 2016-12-29 | Apple Inc. | Virtual assistant for media playback |
US9478234B1 (en) | 2015-07-13 | 2016-10-25 | Knowles Electronics, Llc | Microphone apparatus and method with catch-up buffer |
US10747498B2 (en) | 2015-09-08 | 2020-08-18 | Apple Inc. | Zero latency digital assistant |
US10331312B2 (en) | 2015-09-08 | 2019-06-25 | Apple Inc. | Intelligent automated assistant in a media environment |
US10671428B2 (en) | 2015-09-08 | 2020-06-02 | Apple Inc. | Distributed personal assistant |
US10740384B2 (en) | 2015-09-08 | 2020-08-11 | Apple Inc. | Intelligent automated assistant for media search and playback |
US9697820B2 (en) | 2015-09-24 | 2017-07-04 | Apple Inc. | Unit-selection text-to-speech synthesis using concatenation-sensitive neural networks |
US11010550B2 (en) | 2015-09-29 | 2021-05-18 | Apple Inc. | Unified language modeling framework for word prediction, auto-completion and auto-correction |
US10366158B2 (en) | 2015-09-29 | 2019-07-30 | Apple Inc. | Efficient word encoding for recurrent neural network language models |
US11587559B2 (en) | 2015-09-30 | 2023-02-21 | Apple Inc. | Intelligent device identification |
US10083685B2 (en) * | 2015-10-13 | 2018-09-25 | GM Global Technology Operations LLC | Dynamically adding or removing functionality to speech recognition systems |
US10691473B2 (en) | 2015-11-06 | 2020-06-23 | Apple Inc. | Intelligent automated assistant in a messaging environment |
US10956666B2 (en) | 2015-11-09 | 2021-03-23 | Apple Inc. | Unconventional virtual assistant interactions |
US10049668B2 (en) | 2015-12-02 | 2018-08-14 | Apple Inc. | Applying neural network language models to weighted finite state transducers for automatic speech recognition |
US10223066B2 (en) | 2015-12-23 | 2019-03-05 | Apple Inc. | Proactive assistance based on dialog communication between devices |
US10446143B2 (en) | 2016-03-14 | 2019-10-15 | Apple Inc. | Identification of voice inputs providing credentials |
US9934775B2 (en) | 2016-05-26 | 2018-04-03 | Apple Inc. | Unit-selection text-to-speech synthesis based on predicted concatenation parameters |
US9972304B2 (en) | 2016-06-03 | 2018-05-15 | Apple Inc. | Privacy preserving distributed evaluation framework for embedded personalized systems |
US10049670B2 (en) | 2016-06-06 | 2018-08-14 | Google Llc | Providing voice action discoverability example for trigger term |
US11227589B2 (en) | 2016-06-06 | 2022-01-18 | Apple Inc. | Intelligent list reading |
US10249300B2 (en) | 2016-06-06 | 2019-04-02 | Apple Inc. | Intelligent list reading |
US10049663B2 (en) | 2016-06-08 | 2018-08-14 | Apple, Inc. | Intelligent automated assistant for media exploration |
DK179309B1 (en) | 2016-06-09 | 2018-04-23 | Apple Inc | Intelligent automated assistant in a home environment |
US10509862B2 (en) | 2016-06-10 | 2019-12-17 | Apple Inc. | Dynamic phrase expansion of language input |
US10192552B2 (en) | 2016-06-10 | 2019-01-29 | Apple Inc. | Digital assistant providing whispered speech |
US10490187B2 (en) | 2016-06-10 | 2019-11-26 | Apple Inc. | Digital assistant providing automated status report |
US10586535B2 (en) | 2016-06-10 | 2020-03-10 | Apple Inc. | Intelligent digital assistant in a multi-tasking environment |
US10067938B2 (en) | 2016-06-10 | 2018-09-04 | Apple Inc. | Multilingual word prediction |
DK201670540A1 (en) | 2016-06-11 | 2018-01-08 | Apple Inc | Application integration with a digital assistant |
DK179415B1 (en) | 2016-06-11 | 2018-06-14 | Apple Inc | Intelligent device arbitration and control |
DK179049B1 (en) | 2016-06-11 | 2017-09-18 | Apple Inc | Data driven natural language event detection and classification |
DK179343B1 (en) | 2016-06-11 | 2018-05-14 | Apple Inc | Intelligent task discovery |
US10474753B2 (en) | 2016-09-07 | 2019-11-12 | Apple Inc. | Language identification using recurrent neural networks |
US10043516B2 (en) | 2016-09-23 | 2018-08-07 | Apple Inc. | Intelligent automated assistant |
US10283138B2 (en) * | 2016-10-03 | 2019-05-07 | Google Llc | Noise mitigation for a voice interface device |
JP6659514B2 (en) | 2016-10-12 | 2020-03-04 | 東芝映像ソリューション株式会社 | Electronic device and control method thereof |
US11281993B2 (en) | 2016-12-05 | 2022-03-22 | Apple Inc. | Model and ensemble compression for metric learning |
US9940930B1 (en) | 2016-12-07 | 2018-04-10 | Google Llc | Securing audio data |
US10134396B2 (en) | 2016-12-07 | 2018-11-20 | Google Llc | Preventing of audio attacks |
US10242673B2 (en) | 2016-12-07 | 2019-03-26 | Google Llc | Preventing of audio attacks using an input and an output hotword detection model |
US10593346B2 (en) | 2016-12-22 | 2020-03-17 | Apple Inc. | Rank-reduced token representation for automatic speech recognition |
US11204787B2 (en) | 2017-01-09 | 2021-12-21 | Apple Inc. | Application integration with a digital assistant |
US10332520B2 (en) | 2017-02-13 | 2019-06-25 | Qualcomm Incorporated | Enhanced speech generation |
JP6812843B2 (en) * | 2017-02-23 | 2021-01-13 | 富士通株式会社 | Computer program for voice recognition, voice recognition device and voice recognition method |
KR102389996B1 (en) * | 2017-03-28 | 2022-04-25 | 삼성전자 주식회사 | Electronic device and method for screen controlling for processing user input using the same |
US11250844B2 (en) * | 2017-04-12 | 2022-02-15 | Soundhound, Inc. | Managing agent engagement in a man-machine dialog |
US10748531B2 (en) * | 2017-04-13 | 2020-08-18 | Harman International Industries, Incorporated | Management layer for multiple intelligent personal assistant services |
US10417266B2 (en) | 2017-05-09 | 2019-09-17 | Apple Inc. | Context-aware ranking of intelligent response suggestions |
DK201770383A1 (en) | 2017-05-09 | 2018-12-14 | Apple Inc. | User interface for correcting recognition errors |
US10311870B2 (en) | 2017-05-10 | 2019-06-04 | Ecobee Inc. | Computerized device with voice command input capability |
DK180048B1 (en) | 2017-05-11 | 2020-02-04 | Apple Inc. | MAINTAINING THE DATA PROTECTION OF PERSONAL INFORMATION |
DK201770439A1 (en) | 2017-05-11 | 2018-12-13 | Apple Inc. | Offline personal assistant |
US10395654B2 (en) | 2017-05-11 | 2019-08-27 | Apple Inc. | Text normalization based on a data-driven learning network |
US10726832B2 (en) | 2017-05-11 | 2020-07-28 | Apple Inc. | Maintaining privacy of personal information |
US11301477B2 (en) | 2017-05-12 | 2022-04-12 | Apple Inc. | Feedback analysis of a digital assistant |
DK201770428A1 (en) | 2017-05-12 | 2019-02-18 | Apple Inc. | Low-latency intelligent automated assistant |
DK179745B1 (en) | 2017-05-12 | 2019-05-01 | Apple Inc. | SYNCHRONIZATION AND TASK DELEGATION OF A DIGITAL ASSISTANT |
DK179496B1 (en) | 2017-05-12 | 2019-01-15 | Apple Inc. | USER-SPECIFIC Acoustic Models |
DK201770432A1 (en) | 2017-05-15 | 2018-12-21 | Apple Inc. | Hierarchical belief states for digital assistants |
DK201770431A1 (en) | 2017-05-15 | 2018-12-20 | Apple Inc. | Optimizing dialogue policy decisions for digital assistants using implicit feedback |
US10303715B2 (en) | 2017-05-16 | 2019-05-28 | Apple Inc. | Intelligent automated assistant for media exploration |
US20180336892A1 (en) | 2017-05-16 | 2018-11-22 | Apple Inc. | Detecting a trigger of a digital assistant |
US10311144B2 (en) | 2017-05-16 | 2019-06-04 | Apple Inc. | Emoji word sense disambiguation |
US10403278B2 (en) | 2017-05-16 | 2019-09-03 | Apple Inc. | Methods and systems for phonetic matching in digital assistant services |
DK179560B1 (en) | 2017-05-16 | 2019-02-18 | Apple Inc. | Far-field extension for digital assistant services |
US10657328B2 (en) | 2017-06-02 | 2020-05-19 | Apple Inc. | Multi-task recurrent neural network architecture for efficient morphology handling in neural language modeling |
EP3435642A1 (en) * | 2017-07-29 | 2019-01-30 | Advanced Digital Broadcast S.A. | A system and method for remote control of appliances by voice |
US11282528B2 (en) * | 2017-08-14 | 2022-03-22 | Lenovo (Singapore) Pte. Ltd. | Digital assistant activation based on wake word association |
KR102489914B1 (en) * | 2017-09-15 | 2023-01-20 | 삼성전자주식회사 | Electronic Device and method for controlling the electronic device |
US10235353B1 (en) * | 2017-09-15 | 2019-03-19 | Dell Products Lp | Natural language translation interface for networked devices |
US10445429B2 (en) | 2017-09-21 | 2019-10-15 | Apple Inc. | Natural language understanding using vocabularies with compressed serialized tries |
US10755051B2 (en) | 2017-09-29 | 2020-08-25 | Apple Inc. | Rule-based natural language processing |
US10636424B2 (en) | 2017-11-30 | 2020-04-28 | Apple Inc. | Multi-turn canned dialog |
US10733982B2 (en) | 2018-01-08 | 2020-08-04 | Apple Inc. | Multi-directional dialog |
US10733375B2 (en) | 2018-01-31 | 2020-08-04 | Apple Inc. | Knowledge-based framework for improving natural language understanding |
US10789959B2 (en) | 2018-03-02 | 2020-09-29 | Apple Inc. | Training speaker recognition models for digital assistants |
US10592604B2 (en) | 2018-03-12 | 2020-03-17 | Apple Inc. | Inverse text normalization for automatic speech recognition |
US11127405B1 (en) * | 2018-03-14 | 2021-09-21 | Amazon Technologies, Inc. | Selective requests for authentication for voice-based launching of applications |
US10877637B1 (en) | 2018-03-14 | 2020-12-29 | Amazon Technologies, Inc. | Voice-based device operation mode management |
US10885910B1 (en) | 2018-03-14 | 2021-01-05 | Amazon Technologies, Inc. | Voice-forward graphical user interface mode management |
US10818288B2 (en) | 2018-03-26 | 2020-10-27 | Apple Inc. | Natural assistant interaction |
US10909331B2 (en) | 2018-03-30 | 2021-02-02 | Apple Inc. | Implicit identification of translation payload with neural machine translation |
US10928918B2 (en) | 2018-05-07 | 2021-02-23 | Apple Inc. | Raise to speak |
US11145294B2 (en) | 2018-05-07 | 2021-10-12 | Apple Inc. | Intelligent automated assistant for delivering content from user experiences |
US10984780B2 (en) | 2018-05-21 | 2021-04-20 | Apple Inc. | Global semantic word embeddings using bi-directional recurrent neural networks |
US11386266B2 (en) | 2018-06-01 | 2022-07-12 | Apple Inc. | Text correction |
DK180639B1 (en) | 2018-06-01 | 2021-11-04 | Apple Inc | DISABILITY OF ATTENTION-ATTENTIVE VIRTUAL ASSISTANT |
DK201870355A1 (en) | 2018-06-01 | 2019-12-16 | Apple Inc. | Virtual assistant operation in multi-device environments |
US10892996B2 (en) | 2018-06-01 | 2021-01-12 | Apple Inc. | Variable latency device coordination |
DK179822B1 (en) | 2018-06-01 | 2019-07-12 | Apple Inc. | Voice interaction at a primary device to access call functionality of a companion device |
US10496705B1 (en) | 2018-06-03 | 2019-12-03 | Apple Inc. | Accelerated task performance |
KR102563314B1 (en) | 2018-08-30 | 2023-08-04 | 삼성전자주식회사 | Electronic Device and the Method for Generating Short cut of Quick Command |
US10861444B2 (en) * | 2018-09-24 | 2020-12-08 | Rovi Guides, Inc. | Systems and methods for determining whether to trigger a voice capable device based on speaking cadence |
US11010561B2 (en) | 2018-09-27 | 2021-05-18 | Apple Inc. | Sentiment prediction from textual data |
US11462215B2 (en) | 2018-09-28 | 2022-10-04 | Apple Inc. | Multi-modal inputs for voice commands |
US11170166B2 (en) | 2018-09-28 | 2021-11-09 | Apple Inc. | Neural typographical error modeling via generative adversarial networks |
US10839159B2 (en) | 2018-09-28 | 2020-11-17 | Apple Inc. | Named entity normalization in a spoken dialog system |
US11475898B2 (en) | 2018-10-26 | 2022-10-18 | Apple Inc. | Low-latency multi-speaker speech recognition |
JP7202853B2 (en) * | 2018-11-08 | 2023-01-12 | シャープ株式会社 | refrigerator |
US11100925B2 (en) | 2018-12-06 | 2021-08-24 | Comcast Cable Communications, Llc | Voice command trigger words |
US11638059B2 (en) | 2019-01-04 | 2023-04-25 | Apple Inc. | Content playback on multiple devices |
US11348573B2 (en) | 2019-03-18 | 2022-05-31 | Apple Inc. | Multimodality in digital assistant systems |
US11475884B2 (en) | 2019-05-06 | 2022-10-18 | Apple Inc. | Reducing digital assistant latency when a language is incorrectly determined |
DK201970509A1 (en) | 2019-05-06 | 2021-01-15 | Apple Inc | Spoken notifications |
US11423908B2 (en) | 2019-05-06 | 2022-08-23 | Apple Inc. | Interpreting spoken requests |
US11307752B2 (en) | 2019-05-06 | 2022-04-19 | Apple Inc. | User configurable task triggers |
US11140099B2 (en) | 2019-05-21 | 2021-10-05 | Apple Inc. | Providing message response suggestions |
DK201970511A1 (en) | 2019-05-31 | 2021-02-15 | Apple Inc | Voice identification in digital assistant systems |
DK180129B1 (en) | 2019-05-31 | 2020-06-02 | Apple Inc. | User activity shortcut suggestions |
US11496600B2 (en) | 2019-05-31 | 2022-11-08 | Apple Inc. | Remote execution of machine-learned models |
US11289073B2 (en) | 2019-05-31 | 2022-03-29 | Apple Inc. | Device text to speech |
US11468890B2 (en) | 2019-06-01 | 2022-10-11 | Apple Inc. | Methods and user interfaces for voice-based control of electronic devices |
US11360641B2 (en) | 2019-06-01 | 2022-06-14 | Apple Inc. | Increasing the relevance of new available information |
KR20210031265A (en) * | 2019-09-11 | 2021-03-19 | 삼성전자주식회사 | Electronic device and operating method for the same |
US11488406B2 (en) | 2019-09-25 | 2022-11-01 | Apple Inc. | Text detection using global geometry estimators |
KR20210055347A (en) * | 2019-11-07 | 2021-05-17 | 엘지전자 주식회사 | An aritificial intelligence apparatus |
US11295741B2 (en) | 2019-12-05 | 2022-04-05 | Soundhound, Inc. | Dynamic wakewords for speech-enabled devices |
US11043220B1 (en) | 2020-05-11 | 2021-06-22 | Apple Inc. | Digital assistant hardware abstraction |
US11061543B1 (en) | 2020-05-11 | 2021-07-13 | Apple Inc. | Providing relevant data items based on context |
US11755276B2 (en) | 2020-05-12 | 2023-09-12 | Apple Inc. | Reducing description length based on confidence |
US11490204B2 (en) | 2020-07-20 | 2022-11-01 | Apple Inc. | Multi-device audio adjustment coordination |
US11438683B2 (en) | 2020-07-21 | 2022-09-06 | Apple Inc. | User identification using headphones |
US11620999B2 (en) | 2020-09-18 | 2023-04-04 | Apple Inc. | Reducing device processing of unintended audio |
US11652655B1 (en) * | 2022-01-31 | 2023-05-16 | Zoom Video Communications, Inc. | Audio capture device selection for remote conference participants |
Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20020193989A1 (en) * | 1999-05-21 | 2002-12-19 | Michael Geilhufe | Method and apparatus for identifying voice controlled devices |
US20060019613A1 (en) * | 2004-07-23 | 2006-01-26 | Lg Electronics Inc. | System and method for managing talk burst authority of a mobile communication terminal |
US20070260456A1 (en) * | 2006-05-02 | 2007-11-08 | Xerox Corporation | Voice message converter |
US20080154612A1 (en) * | 2006-12-26 | 2008-06-26 | Voice Signal Technologies, Inc. | Local storage and use of search results for voice-enabled mobile communications devices |
US20100312547A1 (en) * | 2009-06-05 | 2010-12-09 | Apple Inc. | Contextual voice commands |
US20110223893A1 (en) * | 2009-09-30 | 2011-09-15 | T-Mobile Usa, Inc. | Genius Button Secondary Commands |
US20130191122A1 (en) * | 2010-01-25 | 2013-07-25 | Justin Mason | Voice Electronic Listening Assistant |
US8798995B1 (en) * | 2011-09-23 | 2014-08-05 | Amazon Technologies, Inc. | Key word determinations from voice data |
Family Cites Families (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6212541B1 (en) * | 1994-10-24 | 2001-04-03 | Microsoft Corporation | System and method for switching between software applications in multi-window operating system |
KR20010094229A (en) | 2000-04-04 | 2001-10-31 | 이수성 | Method and system for operating a phone by voice recognition technique |
US6539358B1 (en) * | 2000-05-24 | 2003-03-25 | Delphi Technologies, Inc. | Voice-interactive docking station for a portable computing device |
JP4317834B2 (en) * | 2005-06-01 | 2009-08-19 | 株式会社カシオ日立モバイルコミュニケーションズ | Sound output device and sound output control program |
US20090222270A2 (en) * | 2006-02-14 | 2009-09-03 | Ivc Inc. | Voice command interface device |
US8060249B2 (en) * | 2006-11-22 | 2011-11-15 | Senticare Inc. | Medication dispenser with integrated monitoring system |
US8296142B2 (en) * | 2011-01-21 | 2012-10-23 | Google Inc. | Speech recognition using dock context |
-
2012
- 2012-09-15 US US13/621,068 patent/US8452597B2/en active Active
- 2012-09-27 US US14/347,456 patent/US20140244253A1/en not_active Abandoned
- 2012-09-27 WO PCT/US2012/057576 patent/WO2013049358A1/en active Application Filing
- 2012-09-27 CN CN201280056118.5A patent/CN103930945B/en active Active
- 2012-09-27 EP EP12773459.8A patent/EP2761615A1/en not_active Ceased
-
2016
- 2016-02-16 US US15/045,128 patent/US20160180851A1/en not_active Abandoned
Patent Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20020193989A1 (en) * | 1999-05-21 | 2002-12-19 | Michael Geilhufe | Method and apparatus for identifying voice controlled devices |
US20060019613A1 (en) * | 2004-07-23 | 2006-01-26 | Lg Electronics Inc. | System and method for managing talk burst authority of a mobile communication terminal |
US20070260456A1 (en) * | 2006-05-02 | 2007-11-08 | Xerox Corporation | Voice message converter |
US20080154612A1 (en) * | 2006-12-26 | 2008-06-26 | Voice Signal Technologies, Inc. | Local storage and use of search results for voice-enabled mobile communications devices |
US20100312547A1 (en) * | 2009-06-05 | 2010-12-09 | Apple Inc. | Contextual voice commands |
US20110223893A1 (en) * | 2009-09-30 | 2011-09-15 | T-Mobile Usa, Inc. | Genius Button Secondary Commands |
US20130191122A1 (en) * | 2010-01-25 | 2013-07-25 | Justin Mason | Voice Electronic Listening Assistant |
US8798995B1 (en) * | 2011-09-23 | 2014-08-05 | Amazon Technologies, Inc. | Key word determinations from voice data |
Cited By (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10891953B2 (en) | 2013-04-09 | 2021-01-12 | Google Llc | Multi-mode guard for voice commands |
US9530410B1 (en) * | 2013-04-09 | 2016-12-27 | Google Inc. | Multi-mode guard for voice commands |
US10181324B2 (en) | 2013-04-09 | 2019-01-15 | Google Llc | Multi-mode guard for voice commands |
US20150088525A1 (en) * | 2013-09-24 | 2015-03-26 | Tencent Technology (Shenzhen) Co., Ltd. | Method and apparatus for controlling applications and operations on a terminal |
US9769550B2 (en) | 2013-11-06 | 2017-09-19 | Nvidia Corporation | Efficient digital microphone receiver process and system |
US20150127335A1 (en) * | 2013-11-07 | 2015-05-07 | Nvidia Corporation | Voice trigger |
US9454975B2 (en) * | 2013-11-07 | 2016-09-27 | Nvidia Corporation | Voice trigger |
WO2016164842A1 (en) * | 2015-04-10 | 2016-10-13 | Angel.Com Incorporated | Visual interactive voice response system |
US10270908B2 (en) | 2015-04-10 | 2019-04-23 | Genesys Telecommunications Laboratories, Inc. | Visual interactive voice response system |
US10147444B2 (en) | 2015-11-03 | 2018-12-04 | Airoha Technology Corp. | Electronic apparatus and voice trigger method therefor |
JP2020510852A (en) * | 2017-03-31 | 2020-04-09 | アリババ・グループ・ホールディング・リミテッドAlibaba Group Holding Limited | Voice function control method and device |
US10991371B2 (en) | 2017-03-31 | 2021-04-27 | Advanced New Technologies Co., Ltd. | Voice function control method and apparatus |
WO2020262811A1 (en) * | 2019-06-26 | 2020-12-30 | Samsung Electronics Co., Ltd. | System and method for complex task machine learning |
US11875231B2 (en) | 2019-06-26 | 2024-01-16 | Samsung Electronics Co., Ltd. | System and method for complex task machine learning |
US11321048B2 (en) | 2020-02-25 | 2022-05-03 | Motorola Solutions, Inc. | Method and apparatus for temporary hands-free voice interaction |
Also Published As
Publication number | Publication date |
---|---|
WO2013049358A1 (en) | 2013-04-04 |
CN103930945B (en) | 2017-10-31 |
CN103930945A (en) | 2014-07-16 |
US8452597B2 (en) | 2013-05-28 |
EP2761615A1 (en) | 2014-08-06 |
US20160180851A1 (en) | 2016-06-23 |
US20130085755A1 (en) | 2013-04-04 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US8452597B2 (en) | Systems and methods for continual speech recognition and detection in mobile computing devices | |
AU2019246868B2 (en) | Method and system for voice activation | |
US10332524B2 (en) | Speech recognition wake-up of a handheld portable electronic device | |
US10381010B2 (en) | Voice control user interface during low power mode | |
US10381001B2 (en) | Voice control user interface during low-power mode | |
EP3062309B1 (en) | Low power detection of an activation phrase | |
US10452348B2 (en) | Systems and methods for communicating notifications and textual data associated with applications | |
CN109994108B (en) | Full duplex communication techniques for conversational conversations between chat robots and people | |
US9026176B2 (en) | Message-triggered voice command interface in portable electronic devices | |
EP3078021B1 (en) | Initiating actions based on partial hotwords | |
US20170213553A1 (en) | Voice Control User Interface with Progressive Command Engagement | |
US8924219B1 (en) | Multi hotword robust continuous voice command detection in mobile devices | |
KR101605481B1 (en) | Automatically monitoring for voice input based on context | |
US9530409B2 (en) | Event-triggered hands-free multitasking for media playback | |
US20180204569A1 (en) | Voice Assistant Tracking And Activation | |
TWI535258B (en) | Voice answering method and mobile terminal apparatus | |
US10880833B2 (en) | Smart listening modes supporting quasi always-on listening | |
AU2013252518A1 (en) | Embedded system for construction of small footprint speech recognition with user-definable constraints | |
WO2020001165A1 (en) | Voice control method and apparatus, and storage medium and electronic device | |
US20190304460A1 (en) | Voice control user interface with progressive command engagement | |
US20170206901A1 (en) | Voice Control User Interface with Progressive Command Engagement | |
CN106878294B (en) | Method and device for recording voice information by interphone |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: GOOGLE INC., CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:BRINGERT, BJORN ERIK;PIETRYKA, PAWEL;HODGSON, PETER JOHN;AND OTHERS;SIGNING DATES FROM 20120827 TO 20120914;REEL/FRAME:033678/0203 |
|
AS | Assignment |
Owner name: GOOGLE LLC, CALIFORNIA Free format text: CHANGE OF NAME;ASSIGNOR:GOOGLE INC.;REEL/FRAME:044129/0001 Effective date: 20170929 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |