US20060106613A1 - Method and system for evaluating automatic speech recognition telephone services - Google Patents
Method and system for evaluating automatic speech recognition telephone services Download PDFInfo
- Publication number
- US20060106613A1 US20060106613A1 US11/319,989 US31998905A US2006106613A1 US 20060106613 A1 US20060106613 A1 US 20060106613A1 US 31998905 A US31998905 A US 31998905A US 2006106613 A1 US2006106613 A1 US 2006106613A1
- Authority
- US
- United States
- Prior art keywords
- voice recognition
- user
- sample utterances
- evaluation
- recognition service
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/01—Assessment or evaluation of speech recognition systems
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/28—Constructional details of speech recognition systems
- G10L15/30—Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
Definitions
- the present invention relates generally to the evaluation of automated speech recognition, and more specifically relates to the evaluation of the effectiveness of automated speech recognition in providing a telephone service.
- ASR Automatic speech recognition
- ASR uses processor intensive evaluation of digitized voice signals to recognize human speech. For instance, ASR compares a digitized voice signal against a glossary, also known as a vocabulary, of expected responses and identifies the digitized voice signal as an expected response if a match is found with a great enough confidence.
- glossaries of expected responses are typically fine tuned to adapt as much as possible to variations in human voices and noise signals for a likely set of commands.
- ASR technology has steadily improved in terms of reliability and speed as processing capability and processing techniques have improved so that ASR technology is growing increasingly popular as a user friendly interface for businesses.
- Voice recognition offers a friendly alternative to touch tone services provided through DTMF signals and also reduces the cost otherwise associated with live operator support of customer inquiries.
- voice recognition based telephone services have grown increasingly popular in providing services through mobile devices such as wireless or cell phone networks because users are able to access information “hands off” making cell phone use safer, such as in driving conditions.
- an increasing number of services have become available ranging from obtaining driving directions, weather information, flight information and reservations and even stock quotes.
- Cingular wireless offers a variety of services supported by voice recognition through Cingular's VOICE CONNECT service.
- voice recognition technology offers clear advantages for inputting requests to a telephone system compared with touch pad DTMF signaling and offers considerable cost advantages over the use of live operators.
- voice recognition fails or performs unreliably, voice recognition introduces considerable user frustration.
- voice recognition applications are typically tuned for a given set of expected commands and conditions. For instance, within a given service, separate glossaries of responses are often used to improve reliability by increasing the likelihood that a voice request will be recognized, with each glossary designed to address a set of commands. Further, glossaries are fine tuned periodically to adapt to changing conditions and respond to reliability problems. These fine tunings are in addition to changes implemented for menu items and additional services.
- the FIGURE depicts a block diagram of a system for evaluating automatic speech recognition services provided through a telephone network.
- Voice recognition glossaries are typically designed and applied to optimize recognition of a set of expected commands, such as names of cities.
- a voice recognition service having a variety of menu nodes, a number of different glossaries are typically used with each menu node.
- a context having a wide variety of combinations of expected commands is possible so that voice utterances intended for recognition by one glossary may have an impact on recognition by other glossaries associated with a menu node.
- an accurate measure of the usability of a service that uses voice recognition is difficult to obtain from abstract testing of individual nodes or glossaries.
- the present invention uses sample utterances in different contexts to determine error and recognition rates. For instance, a sample utterance evaluates recognition of a voice command at different menu nodes of a voice recognition service so that evaluation of the response to the command is within a context of glossaries applied at that node.
- a block diagram depicts a system for evaluating automatic speech recognition services provided by a telephone network.
- the system evaluates voice recognition services by interfacing with the service through the telephone network and submitting speech sample utterances and determining recognition and error rates for the voice recognition service.
- the evaluation is performed in either a manual or an automated mode by comparing expected responses to sample utterances against actual responses to identify errors and determine system reliability.
- An evaluation engine 10 performs the evaluation of voice recognition services by sending selected sample utterances through the telephone network, receiving responses from the voice recognition service and determining error and recognition rates for the sample utterances.
- a configuration engine 12 interacts with a user to establish a test configuration 14 and to provide the sample utterances of test configuration 14 to the voice recognition service through a telephone system interface 16 .
- Responses from the voice recognition service are received at telephone system interface 16 and are provided to an error and recognition assessment engine 18 .
- Error and recognition assessment engine 18 compares received results against expected results from the sample utterance sent according to test configuration 14 . Error and recognition rates are determinable either through user interaction of comparing recorded sample utterances and recorded voice recognition service responses or by automated comparisons that track sample utterances and voice recognition service responses by error and recognition occurrences.
- Telephone system interface 16 communicates with telephone network 22 through a physical interface 20 , such as a hybrid coupler phone tap.
- evaluation engine 10 resides on a personal computer having a phone tap physical interface 20 that allows evaluation engine 10 to directly dial through network 22 to communicate with a voice recognition service 24 .
- Direct communication between evaluation engine 10 and voice recognition service 24 allows emulation of voice commands so that evaluation engine 10 is able to navigate through a voice recognition service menu either by following a test configuration 14 or by manual manipulation through a user interface.
- a technician may manually navigate through the nodes of the menu with a variety of sample utterances to evaluate the extent of the difficulty or may design a test configuration that provides an automated navigation of the menu and reports error and recognition rates.
- an evaluation engine module 26 associated with voice recognition service 24 establishes a logical link with telephone system interface 16 to allow coordination with test configuration 14 .
- evaluation engine module 26 brings voice recognition service 24 to a menu node corresponding to a menu node identified in configuration engine 14 so that sample utterances are submitted for evaluation without having to follow the menu tree between nodes of the voice recognition service.
- telephone system interface 16 may send sample utterances associated with one or more predetermined menu nodes in a repeated manner not bound by the menu of the voice recognition service with evaluation engine module 26 bringing voice recognition service 24 to the predetermined node before each sample utterance is sent.
- An evaluation graphical user interface 28 allows user interaction with evaluation engine 10 to establish and run test configurations 14 .
- Evaluation graphical user interface 28 is, for example, created with Visual Basic to operate on a Windows based personal computer, although other embodiments may use alternative programming applications and operating systems.
- Evaluation graphical user interface 28 applies a map 30 of voice recognition service 24 and a library 32 of sample utterances, such as digitized voice samples stored as wave files having a “.wav” extension, to design a test configuration 14 in a test configuration window 34 .
- Service buttons 36 allow the design of a test configuration 14 for a selected voice recognition service 24 and allows establishment of basic contact information, such as the telephone number to dial for the voice recognition service 24 that is selected.
- Speaker buttons 38 allow the selection of sample utterances classified by the speaker that generated the utterances.
- Noise buttons 40 allow a test configuration 14 to include simulated levels of noise such as static, road noise and/or crowd noise.
- a go button 42 initiates testing.
- Map window 30 and library window 32 access configuration engine 12 to allow selection of a test configuration 14 through test configuration engine 34 .
- Configuration engine 12 presents a voice recognition service menu on map window 30 and a library of stored digital sample utterances in library 32 from a voice recognition service menu data base 44 and sample utterance library data base 46 .
- Menu data base 44 includes a series of nodes corresponding to the menu items of voice recognition service 24 .
- main menu node of menu data base 44 provides a user with options to select children nodes including driving directions, weather, flight information, or stock quotes.
- the user selects an appropriate child node from the main menu by saying “go to driving directions”, “go to weather”, “go to flight information”, or “go to stock quotes”, as depicted by the utterances of library data base 46 .
- the selections available from the main menu node are often global selections that a user may state from any child node to proceed automatically to a selected child node or the parent main menu node. For instance, a user who selects flight information may automatically proceed to weather information from the flight information child node by stating the utterance of the main menu node “go to weather”.
- the child nodes of the main menu node in turn have child nodes that aid callers in determining specific information. For instance, the flight information child node allows a user to select an airline, destination and arrival city, as well as destination and arrival times.
- the weather child node allows a user to select a city.
- the driving directions child node allows a user to select a location, possibly a city or a landmark within a city.
- the stock quotes child node allows a user to select a company such as Southeastern Bell Corporation by either the company's name or ticker symbol, SBC.
- the voice recognition service 24 applies one or more glossaries at each node of menu data base 44 to identify appropriate information for a caller. For instance, each node is tuned for voice recognition of expected requests of a caller to improve efficiency and reliability of the voice recognition service.
- One difficulty with the use of different glossaries is that one or more utterances may overlap between different nodes of the menu leading to reduced service reliability. For instance, a node that relates to stock quotes may fail to recognize global glossary utterances due to the relationship between the utterances for stock quotes available through the service and the utterances associated with a global menu node, such as the main menu node. In such a situation, a caller at the stock quotes node who commands “go to main menu” instead could receive an unrequested stock quote, resulting in caller frustration and an inability to proceed to the main menu.
- Test configuration window 34 provides a drop and drag environment for creating a test configuration 14 by selecting nodes from map window 30 and sample utterances for the node from library window 32 .
- a user contacts voice recognition service 24 through telephone system interface 16 and manually selecting sample utterances from library window 32 based on a speaker selected from button 38 .
- voice recognition service 24 For instance, once telephone system interface 16 establishes contact with voice recognition service 24 a user selects “go to driving directions” from library window 32 stated by a speaker selected by button 38 . In this manner, the user may navigate the menu of voice recognition service 24 as a normal caller but with sample utterances and simulated noise conditions.
- the error or recognition results of the response are tracked by error and recognition engine 18 which provides an automated comparison to expected voice recognition service responses, records responses for future comparison or tabulates error or recognition results based upon a manual determination made by the user.
- test configuration window 34 automates a test configuration 14 for evaluation engine 10 to run in cooperation with voice recognition service 24 .
- the test configuration depicted in window 34 of the FIGURE illustrates navigation through four voice recognition service nodes with selected sample utterances at each node.
- Evaluation engine 10 automates interaction with voice recognition service 24 according to test configuration 14 as designed in test configuration window 34 so that, for instance, a desired test configuration may be repeatedly run with different speaker and noise conditions.
- Error and recognition assessment engine 18 tracks responses to voice recognition service 24 and tabulates results based on a comparison of actual and expected responses by voice recognition service 24 to sample utterances.
- Evaluation engine module 26 automates the navigation of voice recognition service 24 to enable a more rapid navigation through nodes to be tested by avoiding the need to navigate by voice commands.
- test configurations 14 allows the testing of speech recognition and error rates based on context. For example, the error and recognition rates associated with a particular glossary or glossaries are tested within the context of the voice recognition service. As glossaries are updated and tuned for a node or nodes of a voice recognition menu, the impact of such updates or tuning is tested so that the response of a voice recognition service in different contexts is determined. For instance, the addition of a new stock for stock quotes to a voice recognition service glossary may have unintended impacts on a global glossary such as the main menu so that a caller at the stock quotes node who states “go to main menu” has a greater likelihood of voice recognition error in the stock quote context than in the main menu context. Indeed, as voice recognition service menus grow more complicated, it becomes more difficult to design glossaries for a particular context so that the glossaries take into account the myriad of other menu items that may be available to callers of a voice recognition service in the context of that node.
- evaluation engine 10 provides a practical testing tool that identifies potential problems with a voice recognition service in the actual context of the service as opposed to separate testing of the glossaries.
- test configurations run by evaluation engine 10 allow a determination of the effect of changes in the actual context of the voice recognition service.
- evaluation engine 10 provides a basis for improving node and glossary design for a voice recognition service as a whole.
- evaluation engine 10 provides a user-friendly testing platform to evaluate the effectiveness of a voice recognition service provided through a telephone network. For instance, complaints by telephone network users about particular menu node or voice command failures may be tested through a simulated interaction that emulates the conditions of the reported failure. Automated test configurations with sample utterances from a range of speakers and conditions allows the pinpointing of problem areas to provide specific areas for improvement, thus reducing the cost and improving the results of future updates.
- evaluation engine 10 is flexible to adapt to a variety of services, including services provided by different vendors. For instance, because evaluation engine 10 interfaces with services through a telephone network, it provides a base testing platform for comparing services provided by different vendors by initiating interaction with each service as a customer.
Abstract
A method and system for evaluating telephone services provided by speech recognition interfaces an evaluation engine with a voice recognition service over a telephone system to submit speech utterance samples to the voice recognition service, receive the response of the voice recognition service to the sample utterances, and determine error and recognition of the sample utterances by the voice recognition service by comparing actual voice recognition service responses to expected responses. The evaluation engine permits evaluation of a voice recognition service for plural glossaries in different contexts, such as through predetermined nodes of a voice recognition service menu having plural glossaries.
Description
- The present invention relates generally to the evaluation of automated speech recognition, and more specifically relates to the evaluation of the effectiveness of automated speech recognition in providing a telephone service.
- Automatic speech recognition (ASR) technology interacts with human users by recognizing speech commands and responding with some action, such as providing users with information. ASR uses processor intensive evaluation of digitized voice signals to recognize human speech. For instance, ASR compares a digitized voice signal against a glossary, also known as a vocabulary, of expected responses and identifies the digitized voice signal as an expected response if a match is found with a great enough confidence. In order to improve the reliability of an ASR system, glossaries of expected responses are typically fine tuned to adapt as much as possible to variations in human voices and noise signals for a likely set of commands. ASR technology has steadily improved in terms of reliability and speed as processing capability and processing techniques have improved so that ASR technology is growing increasingly popular as a user friendly interface for businesses.
- One application for ASR technology that is gaining wide acceptance is the use of voice recognition for providing services through a telephone network. Voice recognition offers a friendly alternative to touch tone services provided through DTMF signals and also reduces the cost otherwise associated with live operator support of customer inquiries. In particular, voice recognition based telephone services have grown increasingly popular in providing services through mobile devices such as wireless or cell phone networks because users are able to access information “hands off” making cell phone use safer, such as in driving conditions. As the quality of voice recognition applications has improved, an increasing number of services have become available ranging from obtaining driving directions, weather information, flight information and reservations and even stock quotes. For instance, Cingular wireless offers a variety of services supported by voice recognition through Cingular's VOICE CONNECT service.
- When it works, voice recognition technology offers clear advantages for inputting requests to a telephone system compared with touch pad DTMF signaling and offers considerable cost advantages over the use of live operators. However, when voice recognition fails or performs unreliably, voice recognition introduces considerable user frustration. Thus, to improve reliability, voice recognition applications are typically tuned for a given set of expected commands and conditions. For instance, within a given service, separate glossaries of responses are often used to improve reliability by increasing the likelihood that a voice request will be recognized, with each glossary designed to address a set of commands. Further, glossaries are fine tuned periodically to adapt to changing conditions and respond to reliability problems. These fine tunings are in addition to changes implemented for menu items and additional services.
- One significant difficulty with updating and improving the reliability of services supported by voice recognition is that changes and updates to voice recognition glossaries to support menu changes will have an effect on the service as a whole, for instance by altering recognition rates where glossaries are applied in different contexts. When voice recognition is deployed to a telephone service the overall impact of fine tuning of a glossary is difficult to predict for the application of the glossary in different contexts, such as in combination with other glossaries, especially when real live factors like noise and variations in voices are taken into account.
- A more complete understanding of the present embodiments and advantages thereof may be acquired by referring to the following description taken in conjunction with the accompanying drawing, in which like reference numbers indicate like features, and wherein:
- The FIGURE depicts a block diagram of a system for evaluating automatic speech recognition services provided through a telephone network.
- Preferred embodiments of the present invention are illustrated in the figures, like numerals being used to refer to like and corresponding parts of the various drawings.
- Voice recognition glossaries are typically designed and applied to optimize recognition of a set of expected commands, such as names of cities. However, in a voice recognition service having a variety of menu nodes, a number of different glossaries are typically used with each menu node. Thus, at any given node, a context having a wide variety of combinations of expected commands is possible so that voice utterances intended for recognition by one glossary may have an impact on recognition by other glossaries associated with a menu node. In other words, an accurate measure of the usability of a service that uses voice recognition is difficult to obtain from abstract testing of individual nodes or glossaries.
- In order to evaluate voice recognition services using different combinations of one or more glossaries, the present invention uses sample utterances in different contexts to determine error and recognition rates. For instance, a sample utterance evaluates recognition of a voice command at different menu nodes of a voice recognition service so that evaluation of the response to the command is within a context of glossaries applied at that node.
- Referring now to the FIGURE, a block diagram depicts a system for evaluating automatic speech recognition services provided by a telephone network. The system evaluates voice recognition services by interfacing with the service through the telephone network and submitting speech sample utterances and determining recognition and error rates for the voice recognition service. The evaluation is performed in either a manual or an automated mode by comparing expected responses to sample utterances against actual responses to identify errors and determine system reliability.
- An
evaluation engine 10 performs the evaluation of voice recognition services by sending selected sample utterances through the telephone network, receiving responses from the voice recognition service and determining error and recognition rates for the sample utterances. Aconfiguration engine 12 interacts with a user to establish atest configuration 14 and to provide the sample utterances oftest configuration 14 to the voice recognition service through atelephone system interface 16. Responses from the voice recognition service are received attelephone system interface 16 and are provided to an error andrecognition assessment engine 18. Error andrecognition assessment engine 18 compares received results against expected results from the sample utterance sent according totest configuration 14. Error and recognition rates are determinable either through user interaction of comparing recorded sample utterances and recorded voice recognition service responses or by automated comparisons that track sample utterances and voice recognition service responses by error and recognition occurrences. -
Telephone system interface 16 communicates withtelephone network 22 through aphysical interface 20, such as a hybrid coupler phone tap. For instance,evaluation engine 10 resides on a personal computer having a phone tapphysical interface 20 that allowsevaluation engine 10 to directly dial throughnetwork 22 to communicate with avoice recognition service 24. Direct communication betweenevaluation engine 10 andvoice recognition service 24 allows emulation of voice commands so thatevaluation engine 10 is able to navigate through a voice recognition service menu either by following atest configuration 14 or by manual manipulation through a user interface. Thus, for instance, if a problem is noted with a voice recognition service, a technician may manually navigate through the nodes of the menu with a variety of sample utterances to evaluate the extent of the difficulty or may design a test configuration that provides an automated navigation of the menu and reports error and recognition rates. - In one embodiment, in addition to the voice emulation interaction with
voice recognition service 24, anevaluation engine module 26 associated withvoice recognition service 24 establishes a logical link withtelephone system interface 16 to allow coordination withtest configuration 14. For instance,evaluation engine module 26 bringsvoice recognition service 24 to a menu node corresponding to a menu node identified inconfiguration engine 14 so that sample utterances are submitted for evaluation without having to follow the menu tree between nodes of the voice recognition service. Thus, as an example,telephone system interface 16 may send sample utterances associated with one or more predetermined menu nodes in a repeated manner not bound by the menu of the voice recognition service withevaluation engine module 26 bringingvoice recognition service 24 to the predetermined node before each sample utterance is sent. - An evaluation
graphical user interface 28 allows user interaction withevaluation engine 10 to establish and runtest configurations 14. Evaluationgraphical user interface 28 is, for example, created with Visual Basic to operate on a Windows based personal computer, although other embodiments may use alternative programming applications and operating systems. Evaluationgraphical user interface 28 applies amap 30 ofvoice recognition service 24 and alibrary 32 of sample utterances, such as digitized voice samples stored as wave files having a “.wav” extension, to design atest configuration 14 in atest configuration window 34.Service buttons 36 allow the design of atest configuration 14 for a selectedvoice recognition service 24 and allows establishment of basic contact information, such as the telephone number to dial for thevoice recognition service 24 that is selected.Speaker buttons 38 allow the selection of sample utterances classified by the speaker that generated the utterances.Noise buttons 40 allow atest configuration 14 to include simulated levels of noise such as static, road noise and/or crowd noise. Ago button 42 initiates testing. -
Map window 30 andlibrary window 32access configuration engine 12 to allow selection of atest configuration 14 throughtest configuration engine 34.Configuration engine 12 presents a voice recognition service menu onmap window 30 and a library of stored digital sample utterances inlibrary 32 from a voice recognition servicemenu data base 44 and sample utterancelibrary data base 46.Menu data base 44 includes a series of nodes corresponding to the menu items ofvoice recognition service 24. - For instance, when a caller calls
voice recognition service 24 the call is initially handled at a main menu node which provides generalized areas of inquiry that allow the user to select more specific information from children nodes of the main menu node. As an example, main menu node ofmenu data base 44 provides a user with options to select children nodes including driving directions, weather, flight information, or stock quotes. The user selects an appropriate child node from the main menu by saying “go to driving directions”, “go to weather”, “go to flight information”, or “go to stock quotes”, as depicted by the utterances oflibrary data base 46. - The selections available from the main menu node are often global selections that a user may state from any child node to proceed automatically to a selected child node or the parent main menu node. For instance, a user who selects flight information may automatically proceed to weather information from the flight information child node by stating the utterance of the main menu node “go to weather”. The child nodes of the main menu node in turn have child nodes that aid callers in determining specific information. For instance, the flight information child node allows a user to select an airline, destination and arrival city, as well as destination and arrival times. The weather child node allows a user to select a city. The driving directions child node allows a user to select a location, possibly a city or a landmark within a city. The stock quotes child node allows a user to select a company such as Southwestern Bell Corporation by either the company's name or ticker symbol, SBC.
- The
voice recognition service 24 applies one or more glossaries at each node ofmenu data base 44 to identify appropriate information for a caller. For instance, each node is tuned for voice recognition of expected requests of a caller to improve efficiency and reliability of the voice recognition service. One difficulty with the use of different glossaries is that one or more utterances may overlap between different nodes of the menu leading to reduced service reliability. For instance, a node that relates to stock quotes may fail to recognize global glossary utterances due to the relationship between the utterances for stock quotes available through the service and the utterances associated with a global menu node, such as the main menu node. In such a situation, a caller at the stock quotes node who commands “go to main menu” instead could receive an unrequested stock quote, resulting in caller frustration and an inability to proceed to the main menu. -
Test configuration window 34 provides a drop and drag environment for creating atest configuration 14 by selecting nodes frommap window 30 and sample utterances for the node fromlibrary window 32. In the most simple example, a user contactsvoice recognition service 24 throughtelephone system interface 16 and manually selecting sample utterances fromlibrary window 32 based on a speaker selected frombutton 38. For instance, oncetelephone system interface 16 establishes contact with voice recognition service 24 a user selects “go to driving directions” fromlibrary window 32 stated by a speaker selected bybutton 38. In this manner, the user may navigate the menu ofvoice recognition service 24 as a normal caller but with sample utterances and simulated noise conditions. The error or recognition results of the response are tracked by error andrecognition engine 18 which provides an automated comparison to expected voice recognition service responses, records responses for future comparison or tabulates error or recognition results based upon a manual determination made by the user. - In an alternative embodiment,
test configuration window 34 automates atest configuration 14 forevaluation engine 10 to run in cooperation withvoice recognition service 24. For example, the test configuration depicted inwindow 34 of the FIGURE illustrates navigation through four voice recognition service nodes with selected sample utterances at each node.Evaluation engine 10 automates interaction withvoice recognition service 24 according totest configuration 14 as designed intest configuration window 34 so that, for instance, a desired test configuration may be repeatedly run with different speaker and noise conditions. Error andrecognition assessment engine 18 tracks responses tovoice recognition service 24 and tabulates results based on a comparison of actual and expected responses byvoice recognition service 24 to sample utterances.Evaluation engine module 26 automates the navigation ofvoice recognition service 24 to enable a more rapid navigation through nodes to be tested by avoiding the need to navigate by voice commands. - One advantage of
evaluation engine 10 is thattest configurations 14 allows the testing of speech recognition and error rates based on context. For example, the error and recognition rates associated with a particular glossary or glossaries are tested within the context of the voice recognition service. As glossaries are updated and tuned for a node or nodes of a voice recognition menu, the impact of such updates or tuning is tested so that the response of a voice recognition service in different contexts is determined. For instance, the addition of a new stock for stock quotes to a voice recognition service glossary may have unintended impacts on a global glossary such as the main menu so that a caller at the stock quotes node who states “go to main menu” has a greater likelihood of voice recognition error in the stock quote context than in the main menu context. Indeed, as voice recognition service menus grow more complicated, it becomes more difficult to design glossaries for a particular context so that the glossaries take into account the myriad of other menu items that may be available to callers of a voice recognition service in the context of that node. - One example of an advantage of
evaluation engine 10 is that it provides a practical testing tool that identifies potential problems with a voice recognition service in the actual context of the service as opposed to separate testing of the glossaries. Thus, as services are updated to include additional nodes, changes to nodes or fine-tuning of glossaries, test configurations run byevaluation engine 10 allow a determination of the effect of changes in the actual context of the voice recognition service. By identifying potential recognition errors in the context of the voice recognition service,evaluation engine 10 provides a basis for improving node and glossary design for a voice recognition service as a whole. - Another example of an advantage of
evaluation engine 10 is that it provides a user-friendly testing platform to evaluate the effectiveness of a voice recognition service provided through a telephone network. For instance, complaints by telephone network users about particular menu node or voice command failures may be tested through a simulated interaction that emulates the conditions of the reported failure. Automated test configurations with sample utterances from a range of speakers and conditions allows the pinpointing of problem areas to provide specific areas for improvement, thus reducing the cost and improving the results of future updates. - Another example of an advantage is that
evaluation engine 10 is flexible to adapt to a variety of services, including services provided by different vendors. For instance, becauseevaluation engine 10 interfaces with services through a telephone network, it provides a base testing platform for comparing services provided by different vendors by initiating interaction with each service as a customer. - Although the present invention has been described in detail, it should be understood that various changes, substitutions and alterations can be made hereto without the parting from the spirit and scope of the invention as defined by the appended claims.
Claims (21)
1-20. (canceled)
21. A system for evaluating a voice recognition service, comprising:
an evaluation graphical user interface (GUI) operable to facilitate the generation of a test configuration for evaluating a voice recognition service, the evaluation GUI including:
a node selection interface allowing the user to select from a plurality of voice recognition menu nodes a particular node to be evaluated; and
a utterance selection interface allowing the user to select one or more sample utterances for the evaluation of the voice recognition service; and
an evaluation engine communicatively coupled to the evaluation GUI and configured to cooperate with the evaluation GUI to generate the test configuration, the evaluation engine including:
a voice recognition service interface for communicating to the voice recognition service the one or more selected sample utterances and for receiving from the voice recognition service one or more actual responses to the one or more selected sample utterances; and
an assessment engine for comparing the one or more actual responses to one or more expected responses to determine recognition rates of the voice recognition service.
22. A system according to claim 21 , wherein:
the node selection interface comprises a map interface configured to display a map of the plurality of voice recognition menu nodes indicating relationships between particular nodes; and
the utterance selection interface comprises a library interface configured to display a library of sample utterances from which the user can select the one or more sample utterances.
23. A system according to claim 21 , wherein the node selection interface comprises a map interface configured to display a map of the plurality of voice recognition menu nodes indicating the relationships between particular nodes and to allow the user to use the map to select the particular node to be evaluated.
24. A system according to claim 21 , wherein the evaluation GUI further includes a speaker selection interface allowing the user to select a particular speaker from a plurality of different speakers; and
wherein the utterance selection interface comprises a library interface configured to display a library of sample utterances generated by the selected particular speaker and to allow the user to select the one or more sample utterances from the displayed library.
25. A system according to claim 21 , wherein the evaluation GUI further includes a noise selection interface allowing the user to select one or more simulated background noises.
26. A system according to claim 25 , wherein the one or more simulated background noises comprise at least one of static noise, road noise, and crowd noise.
27. A system according to claim 21 , further comprising a telephony interface for communicating the one or more selected sample utterances to the voice recognition service.
28. A method for evaluating a voice recognition service, comprising:
receiving a user selection of a particular voice recognition menu node to be evaluated, the particular node selected from a plurality of voice recognition menu nodes;
receiving a user selection of one or more sample utterances for an evaluation of the particular node;
communicating the one or more selected sample utterances to a voice recognition service;
receiving one or more actual responses to the one or more selected sample utterances from the voice recognition service; and
comparing the one or more actual responses to one or more expected responses to determine recognition rates of the voice recognition service.
29. A method according to claim 28 , wherein the user selection of the particular node and the one or more sample utterances are provided by an evaluation graphical user interface.
30. A method according to claim 28 , further comprising:
displaying a map of the plurality of voice recognition menu nodes indicating relationships between particular nodes; and
displaying a library of sample utterances from which the user can select the one or more sample utterances.
31. A method according to claim 28 , further comprising:
generating an evaluation graphical user interface (GUI) that displays a map of the plurality of voice recognition menu nodes indicating the relationships between particular nodes; and
receiving the user selection of the particular node to be evaluated via the evaluation GUI.
32. A method according to claim 28 , further comprising generating an evaluation graphical user interface (GUI), the evaluation GUI:
allowing a user to select a particular speaker from a plurality of different speakers;
displaying a library of sample utterances generated by the selected particular speaker; and
allowing the user to select from the displayed library the one or more sample utterances.
33. A method according to claim 28 , further comprising receiving a user selection of one or more simulated background noises.
34. A method according to claim 33 , wherein the one or more simulated background noises comprise at least one of static noise, road noise, and crowd noise.
35. A method according to claim 28 , wherein the one or more selected sample utterances are communicated to the voice recognition service through a telephony interface.
36. Computer instructions encoded in computer-readable media and executable by a processor, comprising:
logic for receiving a user selection of a particular voice recognition menu node to be evaluated, the particular node selected from a plurality of voice recognition menu nodes;
logic for receiving a user selection of one or more sample utterances for an evaluation of the particular node;
logic for communicating to a voice recognition service the one or more selected sample utterances;
logic for receiving from the voice recognition service one or more actual responses to the one or more selected sample utterances; and
logic for comparing the one or more actual responses to one or more expected responses to determine recognition rates of the voice recognition service.
37. Computer logic according to claim 36 , further comprising:
logic for displaying a map of the plurality of voice recognition menu nodes indicating relationships between particular nodes; and
logic for displaying a library of sample utterances from which the user can select the one or more sample utterances.
38. Computer logic according to claim 36 , further comprising:
logic for generating an evaluation graphical user interface (GUI) that displays a map of the plurality of voice recognition menu nodes indicating the relationships between particular nodes; and
logic for receiving the user selection of the particular node to be evaluated via the evaluation GUI.
39. Computer logic according to claim 36 , further comprising logic for generating an evaluation graphical user interface (GUI) including:
logic for allowing the user to select a particular speaker from a plurality of different speakers;
logic for displaying a library of sample utterances generated by the selected particular speaker; and
logic for allowing the user to select the one or more sample utterances from the displayed library.
40. Computer logic according to claim 36 , further comprising logic for communicating the one or more selected sample utterances to the voice recognition service through a telephony interface.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/319,989 US20060106613A1 (en) | 2002-03-26 | 2005-12-28 | Method and system for evaluating automatic speech recognition telephone services |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US10/106,074 US7016842B2 (en) | 2002-03-26 | 2002-03-26 | Method and system for evaluating automatic speech recognition telephone services |
US11/319,989 US20060106613A1 (en) | 2002-03-26 | 2005-12-28 | Method and system for evaluating automatic speech recognition telephone services |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/106,074 Continuation US7016842B2 (en) | 2002-03-26 | 2002-03-26 | Method and system for evaluating automatic speech recognition telephone services |
Publications (1)
Publication Number | Publication Date |
---|---|
US20060106613A1 true US20060106613A1 (en) | 2006-05-18 |
Family
ID=28452461
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/106,074 Active 2024-08-18 US7016842B2 (en) | 2002-03-26 | 2002-03-26 | Method and system for evaluating automatic speech recognition telephone services |
US11/319,989 Abandoned US20060106613A1 (en) | 2002-03-26 | 2005-12-28 | Method and system for evaluating automatic speech recognition telephone services |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/106,074 Active 2024-08-18 US7016842B2 (en) | 2002-03-26 | 2002-03-26 | Method and system for evaluating automatic speech recognition telephone services |
Country Status (1)
Country | Link |
---|---|
US (2) | US7016842B2 (en) |
Cited By (16)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060212291A1 (en) * | 2005-03-16 | 2006-09-21 | Fujitsu Limited | Speech recognition system, speech recognition method and storage medium |
US20080010280A1 (en) * | 2006-06-16 | 2008-01-10 | International Business Machines Corporation | Method and apparatus for building asset based natural language call routing application with limited resources |
US20080151921A1 (en) * | 2002-09-30 | 2008-06-26 | Avaya Technology Llc | Packet prioritization and associated bandwidth and buffer management techniques for audio over ip |
US20090136014A1 (en) * | 2007-11-23 | 2009-05-28 | Foncloud, Inc. | Method for Determining the On-Hold Status in a Call |
US20090202050A1 (en) * | 2007-11-23 | 2009-08-13 | Foncloud, Inc. | System and method for deep dialing phone systems |
US20090207996A1 (en) * | 2007-11-23 | 2009-08-20 | Foncloud, Inc. | System and method for eliminating hold-time in phone calls |
US20090207980A1 (en) * | 2007-11-23 | 2009-08-20 | Foncloud, Inc. | System and method for externally mapping an interactive voice response menu |
WO2009114498A2 (en) * | 2008-03-10 | 2009-09-17 | Foncloud, Inc. | System and method for externally mapping an interactive voice response menu |
US7978827B1 (en) | 2004-06-30 | 2011-07-12 | Avaya Inc. | Automatic configuration of call handling based on end-user needs and characteristics |
US8150020B1 (en) * | 2007-04-04 | 2012-04-03 | At&T Intellectual Property Ii, L.P. | System and method for prompt modification based on caller hang ups in IVRs |
US8218751B2 (en) | 2008-09-29 | 2012-07-10 | Avaya Inc. | Method and apparatus for identifying and eliminating the source of background noise in multi-party teleconferences |
US20130229410A1 (en) * | 2006-11-28 | 2013-09-05 | Samsung Electronics Co., Ltd. | Rendering apparatus and method |
US8593959B2 (en) | 2002-09-30 | 2013-11-26 | Avaya Inc. | VoIP endpoint call admission |
US8670972B1 (en) * | 2007-06-07 | 2014-03-11 | West Corporation | Method and apparatus for voice recognition unit simulation |
US8983845B1 (en) * | 2010-03-26 | 2015-03-17 | Google Inc. | Third-party audio subsystem enhancement |
US9386151B2 (en) | 2007-11-23 | 2016-07-05 | Foncloud, Inc. | System and method for replacing hold-time with a call-back in a contact center environment |
Families Citing this family (18)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20050125486A1 (en) * | 2003-11-20 | 2005-06-09 | Microsoft Corporation | Decentralized operating system |
KR20060111673A (en) * | 2004-01-08 | 2006-10-27 | 보이스 시그널 테크놀로지스, 인코포레이티드. | Automated testing of voice recognition software |
US7885817B2 (en) * | 2005-03-08 | 2011-02-08 | Microsoft Corporation | Easy generation and automatic training of spoken dialog systems using text-to-speech |
US7599861B2 (en) | 2006-03-02 | 2009-10-06 | Convergys Customer Management Group, Inc. | System and method for closed loop decisionmaking in an automated care system |
US7535367B2 (en) * | 2006-04-12 | 2009-05-19 | Nitesh Ratnakar | Airplane lavatory reservation system |
US7809663B1 (en) | 2006-05-22 | 2010-10-05 | Convergys Cmg Utah, Inc. | System and method for supporting the utilization of machine language |
US8379830B1 (en) | 2006-05-22 | 2013-02-19 | Convergys Customer Management Delaware Llc | System and method for automated customer service with contingent live interaction |
US9635154B1 (en) * | 2008-02-08 | 2017-04-25 | West Corporation | Real-time monitoring of caller experience for a group of steps in a call flow |
EP2287835B1 (en) * | 2009-07-10 | 2012-05-23 | Deutsche Telekom AG | Automated evaluation of the usability of a speech dialogue system |
US9349365B2 (en) * | 2013-03-14 | 2016-05-24 | Accenture Global Services Limited | Voice based automation testing for hands free module |
US9772919B2 (en) | 2013-03-14 | 2017-09-26 | Accenture Global Services Limited | Automation of D-bus communication testing for bluetooth profiles |
EP2778927B1 (en) * | 2013-03-14 | 2017-02-22 | Accenture Global Services Limited | Voice based automation testing for hands free module |
US20150031416A1 (en) | 2013-07-23 | 2015-01-29 | Motorola Mobility Llc | Method and Device For Command Phrase Validation |
US9444935B2 (en) * | 2014-11-12 | 2016-09-13 | 24/7 Customer, Inc. | Method and apparatus for facilitating speech application testing |
CN110164474B (en) * | 2019-05-08 | 2021-09-14 | 北京百度网讯科技有限公司 | Voice wake-up automatic test method and system |
CN111242455A (en) * | 2020-01-07 | 2020-06-05 | 北京百度网讯科技有限公司 | Method and device for evaluating voice function of electronic map, electronic equipment and storage medium |
EP3968296A1 (en) | 2020-09-09 | 2022-03-16 | Schweizerische Bundesbahnen SBB | Method for monitoring a system, monitoring system and monitoring module |
CN113593551B (en) * | 2021-07-01 | 2023-07-25 | 中国人民解放军63892部队 | Objective evaluation method for interference effect of voice communication based on command word recognition |
Citations (38)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5572570A (en) * | 1994-10-11 | 1996-11-05 | Teradyne, Inc. | Telecommunication system tester with voice recognition capability |
US5632002A (en) * | 1992-12-28 | 1997-05-20 | Kabushiki Kaisha Toshiba | Speech recognition interface system suitable for window systems and speech mail systems |
US5634086A (en) * | 1993-03-12 | 1997-05-27 | Sri International | Method and apparatus for voice-interactive language instruction |
US5638431A (en) * | 1995-05-01 | 1997-06-10 | Mci Corporation | Calling card validation system and method therefor |
US5640446A (en) * | 1995-05-01 | 1997-06-17 | Mci Corporation | System and method of validating special service calls having different signaling protocols |
US5715369A (en) * | 1995-11-27 | 1998-02-03 | Microsoft Corporation | Single processor programmable speech recognition test system |
US5737489A (en) * | 1995-09-15 | 1998-04-07 | Lucent Technologies Inc. | Discriminative utterance verification for connected digits recognition |
US5765179A (en) * | 1994-08-26 | 1998-06-09 | Kabushiki Kaisha Toshiba | Language processing application system with status data sharing among language processing functions |
US5809462A (en) * | 1995-04-24 | 1998-09-15 | Ericsson Messaging Systems Inc. | Method and apparatus for interfacing and training a neural network for phoneme recognition |
US5835565A (en) * | 1997-02-28 | 1998-11-10 | Hammer Technologies, Inc. | Telecommunication system tester with integrated voice and data |
US5920837A (en) * | 1992-11-13 | 1999-07-06 | Dragon Systems, Inc. | Word recognition system which stores two models for some words and allows selective deletion of one such model |
US5943649A (en) * | 1997-10-29 | 1999-08-24 | International Business Machines Corporation | Configuring an audio interface for different microphone types |
US5960447A (en) * | 1995-11-13 | 1999-09-28 | Holt; Douglas | Word tagging and editing system for speech recognition |
US6006183A (en) * | 1997-12-16 | 1999-12-21 | International Business Machines Corp. | Speech recognition confidence level display |
US6044346A (en) * | 1998-03-09 | 2000-03-28 | Lucent Technologies Inc. | System and method for operating a digital voice recognition processor with flash memory storage |
US6091802A (en) * | 1998-11-03 | 2000-07-18 | Teradyne, Inc. | Telecommunication system tester with integrated voice and data |
US6092043A (en) * | 1992-11-13 | 2000-07-18 | Dragon Systems, Inc. | Apparatuses and method for training and operating speech recognition systems |
US6163768A (en) * | 1998-06-15 | 2000-12-19 | Dragon Systems, Inc. | Non-interactive enrollment in speech recognition |
US6219645B1 (en) * | 1999-12-02 | 2001-04-17 | Lucent Technologies, Inc. | Enhanced automatic speech recognition using multiple directional microphones |
US6246980B1 (en) * | 1997-09-29 | 2001-06-12 | Matra Nortel Communications | Method of speech recognition |
US6299452B1 (en) * | 1999-07-09 | 2001-10-09 | Cognitive Concepts, Inc. | Diagnostic system and method for phonological awareness, phonological processing, and reading skill testing |
US20020041659A1 (en) * | 1998-09-24 | 2002-04-11 | Antony Paul Beswick | Embedded phonetic support and tts play button in a contacts database |
US6377662B1 (en) * | 1997-03-24 | 2002-04-23 | Avaya Technology Corp. | Speech-responsive voice messaging system and method |
US20020055845A1 (en) * | 2000-10-11 | 2002-05-09 | Takaya Ueda | Voice processing apparatus, voice processing method and memory medium |
US6434521B1 (en) * | 1999-06-24 | 2002-08-13 | Speechworks International, Inc. | Automatically determining words for updating in a pronunciation dictionary in a speech recognition system |
US20020156626A1 (en) * | 2001-04-20 | 2002-10-24 | Hutchison William R. | Speech recognition system |
US20030069729A1 (en) * | 2001-10-05 | 2003-04-10 | Bickley Corine A | Method of assessing degree of acoustic confusability, and system therefor |
US20030083883A1 (en) * | 2001-10-31 | 2003-05-01 | James Cyr | Distributed speech recognition system |
US20030125945A1 (en) * | 2001-12-14 | 2003-07-03 | Sean Doyle | Automatically improving a voice recognition system |
US20030156706A1 (en) * | 2002-02-21 | 2003-08-21 | Koehler Robert Kevin | Interactive dialog-based training method |
US6757657B1 (en) * | 1999-09-03 | 2004-06-29 | Sony Corporation | Information processing apparatus, information processing method and program storage medium |
US6766294B2 (en) * | 2001-11-30 | 2004-07-20 | Dictaphone Corporation | Performance gauge for a distributed speech recognition system |
US6839669B1 (en) * | 1998-11-05 | 2005-01-04 | Scansoft, Inc. | Performing actions identified in recognized speech |
US6916291B2 (en) * | 2001-02-07 | 2005-07-12 | East Carolina University | Systems, methods and products for diagnostic hearing assessments distributed via the use of a computer network |
US20050169453A1 (en) * | 2004-01-29 | 2005-08-04 | Sbc Knowledge Ventures, L.P. | Method, software and system for developing interactive call center agent personas |
US7177814B2 (en) * | 2002-02-07 | 2007-02-13 | Sap Aktiengesellschaft | Dynamic grammar for voice-enabled applications |
US7191133B1 (en) * | 2001-02-15 | 2007-03-13 | West Corporation | Script compliance using speech recognition |
US7225132B2 (en) * | 2000-03-14 | 2007-05-29 | British Telecommunications Plc | Method for assigning an identification code |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5832565A (en) * | 1997-03-06 | 1998-11-10 | Water Bonnet Mfg., Inc. | Hinge arrangement for marine windows and other pivoting structures |
-
2002
- 2002-03-26 US US10/106,074 patent/US7016842B2/en active Active
-
2005
- 2005-12-28 US US11/319,989 patent/US20060106613A1/en not_active Abandoned
Patent Citations (39)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5920837A (en) * | 1992-11-13 | 1999-07-06 | Dragon Systems, Inc. | Word recognition system which stores two models for some words and allows selective deletion of one such model |
US6092043A (en) * | 1992-11-13 | 2000-07-18 | Dragon Systems, Inc. | Apparatuses and method for training and operating speech recognition systems |
US5632002A (en) * | 1992-12-28 | 1997-05-20 | Kabushiki Kaisha Toshiba | Speech recognition interface system suitable for window systems and speech mail systems |
US5634086A (en) * | 1993-03-12 | 1997-05-27 | Sri International | Method and apparatus for voice-interactive language instruction |
US5765179A (en) * | 1994-08-26 | 1998-06-09 | Kabushiki Kaisha Toshiba | Language processing application system with status data sharing among language processing functions |
US5572570A (en) * | 1994-10-11 | 1996-11-05 | Teradyne, Inc. | Telecommunication system tester with voice recognition capability |
US5809462A (en) * | 1995-04-24 | 1998-09-15 | Ericsson Messaging Systems Inc. | Method and apparatus for interfacing and training a neural network for phoneme recognition |
US5638431A (en) * | 1995-05-01 | 1997-06-10 | Mci Corporation | Calling card validation system and method therefor |
US5640446A (en) * | 1995-05-01 | 1997-06-17 | Mci Corporation | System and method of validating special service calls having different signaling protocols |
US5737489A (en) * | 1995-09-15 | 1998-04-07 | Lucent Technologies Inc. | Discriminative utterance verification for connected digits recognition |
US5960447A (en) * | 1995-11-13 | 1999-09-28 | Holt; Douglas | Word tagging and editing system for speech recognition |
US5715369A (en) * | 1995-11-27 | 1998-02-03 | Microsoft Corporation | Single processor programmable speech recognition test system |
US5835565A (en) * | 1997-02-28 | 1998-11-10 | Hammer Technologies, Inc. | Telecommunication system tester with integrated voice and data |
US6377662B1 (en) * | 1997-03-24 | 2002-04-23 | Avaya Technology Corp. | Speech-responsive voice messaging system and method |
US6246980B1 (en) * | 1997-09-29 | 2001-06-12 | Matra Nortel Communications | Method of speech recognition |
US5943649A (en) * | 1997-10-29 | 1999-08-24 | International Business Machines Corporation | Configuring an audio interface for different microphone types |
US6006183A (en) * | 1997-12-16 | 1999-12-21 | International Business Machines Corp. | Speech recognition confidence level display |
US6044346A (en) * | 1998-03-09 | 2000-03-28 | Lucent Technologies Inc. | System and method for operating a digital voice recognition processor with flash memory storage |
US6163768A (en) * | 1998-06-15 | 2000-12-19 | Dragon Systems, Inc. | Non-interactive enrollment in speech recognition |
US20020041659A1 (en) * | 1998-09-24 | 2002-04-11 | Antony Paul Beswick | Embedded phonetic support and tts play button in a contacts database |
US6091802A (en) * | 1998-11-03 | 2000-07-18 | Teradyne, Inc. | Telecommunication system tester with integrated voice and data |
US6839669B1 (en) * | 1998-11-05 | 2005-01-04 | Scansoft, Inc. | Performing actions identified in recognized speech |
US6434521B1 (en) * | 1999-06-24 | 2002-08-13 | Speechworks International, Inc. | Automatically determining words for updating in a pronunciation dictionary in a speech recognition system |
US6299452B1 (en) * | 1999-07-09 | 2001-10-09 | Cognitive Concepts, Inc. | Diagnostic system and method for phonological awareness, phonological processing, and reading skill testing |
US6757657B1 (en) * | 1999-09-03 | 2004-06-29 | Sony Corporation | Information processing apparatus, information processing method and program storage medium |
US6219645B1 (en) * | 1999-12-02 | 2001-04-17 | Lucent Technologies, Inc. | Enhanced automatic speech recognition using multiple directional microphones |
US7225132B2 (en) * | 2000-03-14 | 2007-05-29 | British Telecommunications Plc | Method for assigning an identification code |
US20020055845A1 (en) * | 2000-10-11 | 2002-05-09 | Takaya Ueda | Voice processing apparatus, voice processing method and memory medium |
US6916291B2 (en) * | 2001-02-07 | 2005-07-12 | East Carolina University | Systems, methods and products for diagnostic hearing assessments distributed via the use of a computer network |
US7191133B1 (en) * | 2001-02-15 | 2007-03-13 | West Corporation | Script compliance using speech recognition |
US20020156626A1 (en) * | 2001-04-20 | 2002-10-24 | Hutchison William R. | Speech recognition system |
US20030069729A1 (en) * | 2001-10-05 | 2003-04-10 | Bickley Corine A | Method of assessing degree of acoustic confusability, and system therefor |
US20030083883A1 (en) * | 2001-10-31 | 2003-05-01 | James Cyr | Distributed speech recognition system |
US6766294B2 (en) * | 2001-11-30 | 2004-07-20 | Dictaphone Corporation | Performance gauge for a distributed speech recognition system |
US7103542B2 (en) * | 2001-12-14 | 2006-09-05 | Ben Franklin Patent Holding Llc | Automatically improving a voice recognition system |
US20030125945A1 (en) * | 2001-12-14 | 2003-07-03 | Sean Doyle | Automatically improving a voice recognition system |
US7177814B2 (en) * | 2002-02-07 | 2007-02-13 | Sap Aktiengesellschaft | Dynamic grammar for voice-enabled applications |
US20030156706A1 (en) * | 2002-02-21 | 2003-08-21 | Koehler Robert Kevin | Interactive dialog-based training method |
US20050169453A1 (en) * | 2004-01-29 | 2005-08-04 | Sbc Knowledge Ventures, L.P. | Method, software and system for developing interactive call center agent personas |
Cited By (44)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8593959B2 (en) | 2002-09-30 | 2013-11-26 | Avaya Inc. | VoIP endpoint call admission |
US20080151886A1 (en) * | 2002-09-30 | 2008-06-26 | Avaya Technology Llc | Packet prioritization and associated bandwidth and buffer management techniques for audio over ip |
US20080151921A1 (en) * | 2002-09-30 | 2008-06-26 | Avaya Technology Llc | Packet prioritization and associated bandwidth and buffer management techniques for audio over ip |
US7877500B2 (en) | 2002-09-30 | 2011-01-25 | Avaya Inc. | Packet prioritization and associated bandwidth and buffer management techniques for audio over IP |
US8370515B2 (en) | 2002-09-30 | 2013-02-05 | Avaya Inc. | Packet prioritization and associated bandwidth and buffer management techniques for audio over IP |
US8015309B2 (en) | 2002-09-30 | 2011-09-06 | Avaya Inc. | Packet prioritization and associated bandwidth and buffer management techniques for audio over IP |
US7877501B2 (en) | 2002-09-30 | 2011-01-25 | Avaya Inc. | Packet prioritization and associated bandwidth and buffer management techniques for audio over IP |
US7978827B1 (en) | 2004-06-30 | 2011-07-12 | Avaya Inc. | Automatic configuration of call handling based on end-user needs and characteristics |
US20060212291A1 (en) * | 2005-03-16 | 2006-09-21 | Fujitsu Limited | Speech recognition system, speech recognition method and storage medium |
US8010359B2 (en) * | 2005-03-16 | 2011-08-30 | Fujitsu Limited | Speech recognition system, speech recognition method and storage medium |
US20080010280A1 (en) * | 2006-06-16 | 2008-01-10 | International Business Machines Corporation | Method and apparatus for building asset based natural language call routing application with limited resources |
US8370127B2 (en) * | 2006-06-16 | 2013-02-05 | Nuance Communications, Inc. | Systems and methods for building asset based natural language call routing application with limited resources |
US20130229410A1 (en) * | 2006-11-28 | 2013-09-05 | Samsung Electronics Co., Ltd. | Rendering apparatus and method |
US9060061B2 (en) | 2007-04-04 | 2015-06-16 | At&T Intellectual Property Ii, L.P. | System and method for prompt modification based on caller hang ups in IVRs |
US9621723B2 (en) | 2007-04-04 | 2017-04-11 | At&T Intellectual Property Ii, L.P. | System and method for prompt modification based on caller hang Ups in IVRs |
US8150020B1 (en) * | 2007-04-04 | 2012-04-03 | At&T Intellectual Property Ii, L.P. | System and method for prompt modification based on caller hang ups in IVRs |
US9756183B2 (en) | 2007-04-04 | 2017-09-05 | At&T Intellectual Property Ii, L.P. | System and method for prompt modification based on caller hang ups in IVRs |
US9357068B2 (en) | 2007-04-04 | 2016-05-31 | At&T Intellectual Property Ii, L.P. | System and method for prompt modification based on caller hang ups in IVRs |
US8488771B2 (en) | 2007-04-04 | 2013-07-16 | At&T Intellectual Property Ii, L.P. | System and method for prompt modification based on caller hang ups in IVRS |
US8781098B2 (en) | 2007-04-04 | 2014-07-15 | At&T Intellectual Property Ii, L.P. | System and method for prompt modification based on caller hang ups in IVRs |
US9497304B1 (en) | 2007-06-07 | 2016-11-15 | West Corporation | Method and apparatus for voice recognition unit simulation |
US10181319B1 (en) | 2007-06-07 | 2019-01-15 | West Corporation | Method and apparatus for voice recognition unit simulation |
US11087737B1 (en) * | 2007-06-07 | 2021-08-10 | West Corporation | Method and apparatus for voice recognition unit simulation |
US8990063B1 (en) | 2007-06-07 | 2015-03-24 | West Corporation | Method and apparatus for voice recognition unit simulation |
US8670972B1 (en) * | 2007-06-07 | 2014-03-11 | West Corporation | Method and apparatus for voice recognition unit simulation |
US10445441B1 (en) | 2007-06-07 | 2019-10-15 | West Corporation | Method and apparatus for voice recognition unit simulation |
US8917817B1 (en) | 2007-06-07 | 2014-12-23 | West Corporation | Method and apparatus for voice recognition unit simulation |
US9288316B2 (en) | 2007-11-23 | 2016-03-15 | Foncloud, Inc. | System and method for eliminating hold-time in phone calls |
US9386151B2 (en) | 2007-11-23 | 2016-07-05 | Foncloud, Inc. | System and method for replacing hold-time with a call-back in a contact center environment |
US8774373B2 (en) | 2007-11-23 | 2014-07-08 | Foncloud, Inc. | System and method for externally mapping an interactive voice response menu |
US9014351B2 (en) | 2007-11-23 | 2015-04-21 | Foncloud, Inc. | System and method for deep dialing phone systems |
US20090202050A1 (en) * | 2007-11-23 | 2009-08-13 | Foncloud, Inc. | System and method for deep dialing phone systems |
US8605868B2 (en) | 2007-11-23 | 2013-12-10 | Foncloud, Inc. | System and method for externally mapping an interactive voice response menu |
US9270817B2 (en) | 2007-11-23 | 2016-02-23 | Foncloud, Inc. | Method for determining the on-hold status in a call |
US20090207980A1 (en) * | 2007-11-23 | 2009-08-20 | Foncloud, Inc. | System and method for externally mapping an interactive voice response menu |
US20090136014A1 (en) * | 2007-11-23 | 2009-05-28 | Foncloud, Inc. | Method for Determining the On-Hold Status in a Call |
US8908847B2 (en) | 2007-11-23 | 2014-12-09 | Foncloud, Inc. | System and method for deep dialing phone systems |
US10284726B2 (en) | 2007-11-23 | 2019-05-07 | Foncloud, Inc. | System and method for replacing hold-time with a call-back in a contact center environment |
US20090207996A1 (en) * | 2007-11-23 | 2009-08-20 | Foncloud, Inc. | System and method for eliminating hold-time in phone calls |
US8515028B2 (en) | 2007-11-23 | 2013-08-20 | Foncloud, Inc. | System and method for externally mapping an Interactive Voice Response menu |
WO2009114498A3 (en) * | 2008-03-10 | 2009-12-17 | Foncloud, Inc. | System and method for externally mapping an interactive voice response menu |
WO2009114498A2 (en) * | 2008-03-10 | 2009-09-17 | Foncloud, Inc. | System and method for externally mapping an interactive voice response menu |
US8218751B2 (en) | 2008-09-29 | 2012-07-10 | Avaya Inc. | Method and apparatus for identifying and eliminating the source of background noise in multi-party teleconferences |
US8983845B1 (en) * | 2010-03-26 | 2015-03-17 | Google Inc. | Third-party audio subsystem enhancement |
Also Published As
Publication number | Publication date |
---|---|
US7016842B2 (en) | 2006-03-21 |
US20030187639A1 (en) | 2003-10-02 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7016842B2 (en) | Method and system for evaluating automatic speech recognition telephone services | |
US7996220B2 (en) | System and method for providing a compensated speech recognition model for speech recognition | |
US6601029B1 (en) | Voice processing apparatus | |
US7219063B2 (en) | Wirelessly delivered owner's manual | |
US6757544B2 (en) | System and method for determining a location relevant to a communication device and/or its associated user | |
CA2756140C (en) | Service oriented speech recognition for in-vehicle automated interaction | |
US7783305B2 (en) | Method and system for providing menu tree assistance | |
US20140112465A1 (en) | Providing information regarding interactive voice response sessions | |
US9082414B2 (en) | Correcting unintelligible synthesized speech | |
US7881938B2 (en) | Speech bookmarks in a voice user interface using a speech recognition engine and acoustically generated baseforms | |
US20120253823A1 (en) | Hybrid Dialog Speech Recognition for In-Vehicle Automated Interaction and In-Vehicle Interfaces Requiring Minimal Driver Processing | |
US6671668B2 (en) | Speech recognition system including manner discrimination | |
US20060095268A1 (en) | Dialogue system, dialogue method, and recording medium | |
US7552221B2 (en) | System for communicating with a server through a mobile communication device | |
US20060009974A1 (en) | Hands-free voice dialing for portable and remote devices | |
US8428241B2 (en) | Semi-supervised training of destination map for call handling applications | |
US20200211560A1 (en) | Data Processing Device and Method for Performing Speech-Based Human Machine Interaction | |
US20060020471A1 (en) | Method and apparatus for robustly locating user barge-ins in voice-activated command systems | |
US7474958B2 (en) | Information providing method for vehicle and information providing apparatus for vehicle | |
EP1151431B1 (en) | Method and apparatus for testing user interface integrity of speech-enabled devices | |
EP1524870A1 (en) | Method for communicating information in a preferred language from a server via a mobile communication device | |
JPH10207486A (en) | Interactive voice recognition method and device executing the method | |
CA2737850A1 (en) | Wirelessly delivered owner's manual | |
Attwater et al. | Data Adaptive Dialog Systems | |
KR20060098128A (en) | Call-center system by multi-level speech recognition |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: SBC TECHNOLOGY RESOURCES, INC., TEXAS Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MILLS, SCOTT H.;REEL/FRAME:017886/0019 Effective date: 20020325 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |
|
AS | Assignment |
Owner name: NUANCE COMMUNICATIONS, INC., MASSACHUSETTS Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:AT&T INTELLECTUAL PROPERTY I, L.P.;REEL/FRAME:041504/0952 Effective date: 20161214 |