WO2001082264A1

WO2001082264A1 - An interactive, computer-aided speech education method and apparatus

Info

Publication number: WO2001082264A1
Application number: PCT/US2001/013258
Authority: WO
Inventors: John R. Lee; Alfredo Alvendia
Original assignee: Jrl Enterprises, Inc.
Priority date: 2000-04-26
Filing date: 2001-04-25
Publication date: 2001-11-01
Also published as: AU2001257233A1

Abstract

An educational method and system for executing the method, to improve the efficiency of individual learning by monitoring the student's progress and pacing the course material to the student's ability to comprehend and learn. This method and system also reduces the teachers's administrative and paperwork burden. Interactive multimedia technology is combined with unique courseware development to provide a flexible teaching tool and student monitoring system. The method further comprises speech education which provides examples, stores and evaluates responses and provides suitable feedback.

Description

AN INTERACTIVE. COMPUTER-AIDED

SPEECH EDUCATION METHOD AND APPARATUS

BACKGROUND

The present invention relates generally to educational methods at the classroom and individual level, as well as systems for implementing such methods. More particularly, the present invention relates to an educational method which combines audio-visual and computer technology to shift the teacher's role away from mass pupil instruction and paperwork and back to the more traditional role of helping each individual student with his or her unique educational difficulties. Of particular significance in this application are methods and apparatus directed to teach speech/language skills to students.

Classroom teaching methods have changed very little since the early days of our country. The bedrock of education remains the teacher or professor lecturing to groups of students, commonly in conjunction with a text. Homework is assigned to reinforce the lessons learned in class and tests are administered to measure students' aptitude and retention of the material. At the early levels of education, the setting for this instruction has and continues to be the schoolroom, public or private, where a teacher is responsible for a group of, for example, fifteen to forty students.

The conventional classroom method of teaching a subject typically involves a lesson plan which includes a plurality of lectures each having an associated written homework assignment for the student to reinforce the lecture. The teacher collects and evaluates the homework and occasionally gives tests to measure the students' retention of the material. The teacher is also required to perform a multitude of other subsidiary tasks, such as report card generation and parent-teacher conferences, to keep the community satisfied with both the students' and the teacher's performance. Thus under the conventional classroom scenario, teachers must split their time to assume the roles of administrator and public relation officer, as well their supposedly primary role of teacher.

Educators have long recognized that while the classroom scenario provides economies of scale, one drawback is that students of different ages and aptitudes learn at differing rates and with the need for varying amounts of individual attention. This recognition led to the availability of private tutoring, for those who could afford such, provides more flexibility to meet an individual student's needs. On the other hand, private tutoring has also been criticized since it does not provide the student with the opportunity to develop social skills which is inherent in the classroom setting. Although this traditional method of education has changed little over the years, the amount of information which students need to assimilate to be considered well educated continues to increase geometrically. This fact is graphically illustrated by the technological wonders which surround us such as microcomputers, DNA mapping, videophones, the Internet, etc. The increase in information puts a correspondingly greater burden on teachers to try to cram more learning into the same amount of time, which leads to an associated increase in the amount of paperwork to be handled. Moreover, as fiscal resources grow more and more scarce, teachers have been required to handle even larger groups of students resulting in even less individual attention. Although these problems are rapidly becoming more acute, they have been acknowledged by educators for many years. Many countries have reacted by adding more days to the school year or sending children to school at an earlier age so that students have more time to acquire the additional knowledge necessary. This solution, however, will only provide short term relief since there is only a finite amount of additional learning time which can be reasonably added to students' schedules, while our knowledge base will always continue to expand. Additionally, increasing the school year further aggravates the fiscal problem of education by requiring even greater economic resources and putting greater pressure on teachers.

Educators have also experimented with alternate teaching methods, such as self-study programs and videotaped lectures, to both more efficiently use teachers' time and to provide an element of individuality to a student's study program. Although these types of methods are somewhat successful at higher levels of education, they are generally ineffective at lower levels where younger students lack the discipline to teach themselves .

Another approach has been outlined in the afore-incorporated by reference patents to John Lee et al. Therein, interactive educational methods and systems are described which provide a flexible alternative to the conventional classroom educational method. These educational methods integrate technology and learning in a manner which is specifically designed to make learning more effective for both students and teacher. For example, the computer-aided techniques describe therein relieve the teacher of many of the non-teaching burdens discussed above, while also giving the student the opportunity to learn at his or her own pace. These systems also provide an interactive, audio-visual system which will encourage each student to learn at his or her own individual pace and using teaching techniques that are adapted to promote learning according to each individual's strengths and weaknesses. Moreover, each student is provided with instructional reinforcement in the form of questions and his or her performance will be monitored by the system. If a student has difficulty with a particular area, the system will alert the teacher who can then provide additional help.

Another area of teaching in which it would be desirable to provide creative, computer- aided solutions is speech/language education. This type of education focuses on teaching students how to correctly pronounce various words and/or phrases. Recent technological advances in the area of voice signal processing and speech recognition have provided the building blocks upon which computer-aided educational techniques can advance into the realm of speech/language education. Actually teaching students how to pronounce words, however, is more challenging than simply training a computer to recognize speech that is input thereto. The present invention addresses this challenge and provides methods and systems for interactive, computer-aided speech education.

SUMMARY In accordance with the present invention, an educational speech teaching method using an interactive audio-visual computer system includes outputting, by a computer system, at least one example of a speech object; prompting, by the computer system, a student to repeat the speech object; storing the repeated speech object in the computer system; evaluating the stored, repeated speech object using a speech evaluation engine to generate grades associated with the stored, repeated speech object on a phoneme level; and determining, by the computer system, whether to provide another example of the speech object or to output a positive reinforcer based on the grades generated by the speech evaluation engine.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other advantages and objects of the invention will become more apparent from the following detailed description of the preferred embodiments when read in conjunction with the attached Figures in which:

Figure 1 illustrates a block diagram of the elements of a system for implementing the educational method according to an embodiment of the present invention; Figure 2 sets forth a flow chart which indicates an overall sequence of events according to a preferred embodiment of an educational method according to the present invention;

Figures 3 and 4 illustrate flowcharts describing subprocesses of the sequence described in Figure 2; Figure 5 is a flow chart illustrating another exemplary embodiment of the present invention;

Figures 6-16 depict exemplary interface screens of a teacher's workstation according to exemplary embodiments of the present invention;

Figure 17 is a flowchart describing an exemplary speech education method according to an embodiment of the present invention; and

Figure 18 is a block diagram depicting the general hardware and software elements associated with implementing the method of Figure 17.

DETAILED DESCRIPTION

In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular circuits, circuit components, techniques, etc. in order to provide a thorough understanding of the present invention. However, it will be apparent to one skilled in the art that the present invention may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known methods, devices, and circuits are omitted so as not to obscure the description of the present invention.

A system for implementing the educational method according to an exemplary embodiment of the present invention will be described with reference to Figure 1. In this exemplary embodiment it is envisioned that each student will be provided with a workstation 10 that may comprise all of the devices indicated in block form within the dotted lines of Figure 1 linked together via system bus 12. The workstation 10 can, for example, be microcomputer and the CPU 14 could be that which is part of the microcomputer. For example, an IBM PC having an Intel 80386, 80486 or Pentium^* microprocessor could provide the platform for the workstation. Alternately, more or less powerful computer systems could be used. The workstation 10 can be integrated within or around an ergonomic environment. Each station can include an ergonomically designed chair (not shown) so that the student can remain comfortably seated for extended periods of time. The display 16, which can be a conventional VGA monitor attached via a serial port to the system bus 12, can be mounted within a student's desk under glass at about a 45 degree angle. By providing workstations having the display 16 mounted under glass, the classroom as a whole is uncluttered and the teacher is able to readily observe both students and their respective workstation displays. Alternately, the display can simply rest on the desktop. In some preferred embodiments of the invention, the display will be touch sensitive so that a student can enter data and respond to questions from the computer via the screen.

The keyboard 18, floppy drives 20, and hard drive 22 are all conventional components attached via suitable interfaces to the CPU 14. The auxiliary input device 24 can be any type of pointing device such as a mouse, digitizing tablet or light pen. The CD-ROM device 26 and laser disk 28, for providing text data and video data, respectively, are also of a conventional design and can be either built-in to the workstation or provided as stand-alone models. Full screen video presentations, or partial screen windows or video presentations, can be stored on CD or laser disk using, for example, an IMPEG compression technique.

Each of the TV input module 30, audio interface 32, and stereo input module 34 can be off-the-shelf boards which are designed to plug into the expansion slots of a microcomputer in a known manner. The TV input module 30 is a board which allows a computer to receive television signals via either antenna or cable and convert those signals into image signals which can be displayed on all or part of the display 16. The audio interface 32 is an input/output board which allows the computer to send sounds, including voice and music, to a student via headphones (not shown) which plug into the board and receive such sounds via a microphone (not shown). Alternately, the audio interface could simply be a speaker which would be connected to a sound board and/or a voice synthesizer as is well known in the art. The stereo input module 34 is a board which allows the microcomputer to receive radio transmissions or signals from a tape player, CD player or any other type of audio device, which a student could listen to as background music. Printer 36 can be any type of microcomputer compatible printer, including dot matrix, inkjet or laser printer, which communicates with the CPU in a known manner. All of the student workstations are in constant communication with a teacher's workstation 40 via a LAN interface 42 and local area network (LAN) 44. As described in more detail below, this real-time communication between student workstation and teacher workstation allows the teacher to be informed of the students' progress and activities as well as allowing the teacher to tailor instructional programs for each student.

The foregoing description will be understood by one skilled in the art as merely an example of one system which could be used to implement the various embodiments of the educational method of the present invention described in detail below and that various other configurations and devices could also be used. Reference to the conventionality of the various I/O devices described above is made merely to emphasize that one skilled in the art will recognize that the system described in Figure 1 can be implemented using known interfaces and software techniques .

Initially, one advantageous feature of the educational method according to the present invention is that courseware can be uniquely developed for each class, school, school system or any other segment of a student population so as to provide a curriculum desired by the educators using the system and method of the present invention. The term "courseware" as used throughout this specification is meant to comprise the combination of control programs and data which provide each lesson to the student at a workstation and includes, for example, both the information stored on the hard drive, laser disk, and CD-ROM, as well as that input through the TV input module, and the programmed presentation of that data. The phrase

"lesson segment" is used to denote a block of material presented to a student which is intended to be viewed and interacted with in one sitting, typically 15-40 minutes. A lesson segment can also be a test which evaluates the student's retention of one or more lesson segments or a whole semester's worth of material. As a simplified example, history courseware dealing with the American Revolution might comprise a plurality of lesson segments. One lesson segment might include, for example, a narrative story of George Washington's role in the revolution. The text of the story, read from the CD-ROM, might occupy one half of the display while illustrations, from the laser disk, occupy the other half of the screen. After each segment of the lesson, the program could test the student's comprehension by asking the student to answer a series of questions via the keyboard which relate to the previous segment of the lesson. Thus the teacher or school system can select material including, for example, text, illustrations, length of lesson and questions to be answered, to comprise the courseware for any subject. This courseware development step in the educational method according to the present invention advantageously provides teachers and parents with the opportunity to create and update the educational material for presentation to the students.

One skilled in the teaching art will appreciate that many other teaching tools could be integrated into the courseware, in addition to those mentioned above, to present various topics. For example, some topics, such as biology, are particularly conducive to teaching via film presentation. This material can be input via the TV input module, or presented as computer video decompressed from a CD, and displayed while an accompanying voice description is listened to by the student on the headphone set.

Next, Figure 2 illustrates an exemplary preferred embodiment of an overview of an educational method according to the present invention wherein the student is using the workstation and the system has been loaded with the developed courseware. In block 100, the teacher initializes the system and then the students enter their homework assignments from the previous day into the system at block 110. The homework assignment can be entered manually by the student typing the answers on the keyboard. Alternately, an optical scanning device (not shown) can be provided which is used to scan the homework paper and digitize the answers. In either case, the homework answers are compared to the expected answers by the CPU of each workstation and the results transmitted via the LAN to the teacher's workstation as indicated by block 120. The homework answers are stored in database files corresponding to each student name or ID number in the hard drive at the teacher's station for future use.

The teacher and/or a system program can then use the results of the homework assignments in conjunction with each student's progress, which has been stored in another database file at the end of the previous day, to assign lesson segments to each student as the process flow continues to block 130. Alternatively, lesson assignments can be determined solely on the basis of each student's progress and homework can be used either as a measure of retention or as a pretest. This assignment process allows the teacher and/or a system program to determine how much and what type of material each student can access for a given period of time, and provides the first of several opportunities for the teacher and/or system program to tailor each student's individual learning program. Within the system, this assignment process is controlled by the CPU of the teacher's station which downloads the control programs corresponding to the lesson segments selected by the teacher and/or a system program from the hard drive or other storage device of the teacher's workstation to the selected student's station through the LAN. Alternately, a single control program could be downloaded to, or stored on, the students' workstations which will allow selection of lesson segments stored on the hard drive of each student's workstation. When run by the student these control programs access the various information storage devices to retrieve the audio and visual data created for each lesson segment.

In block 140, the process flow moves to the next step of displaying the main screen at each student work station. In addition to lesson segments, this screen can also include selections such as games, movies or educational video programs, and creative writing or drawing activities. These selections can also be chosen by the teacher, student, and/or a system program and can be downloaded through the LAN to each student workstation. As mentioned briefly above, depending on the educational level at which the system and method are being implemented, the student can make selections in a variety of ways. Younger students, for example, can make a selection by touching color-coded figures displayed on a touch-sensitive screen. More advanced students can make a selection by using the workstation's pointing device to click on the appropriate icon.

In decision blocks 141, 143, 145, and 147, the process flow checks to see which selection is made from the main screen. For exemplary purposes only, the choices of

LESSON SEGMENT, GAME, VIDEO and END are illustrated in Figure 2, however it is to be understood that many other different types of selections can be provided. Based on the determined selection, an appropriate subroutine 142, 144, 146, or 148 is executed. The LESSON SEGMENT selection will now be described with reference to Figure 3 which illustrates an exemplary flowchart breaking down some of the steps of a lesson segment subroutine as diagrammatically indicated at block 142.

A control program corresponding to the selected function which was previously stored on the hard drive or other storage device of the student's workstation during the assignment process is initiated in block 152. Prior to presenting a lesson, a pretest can be performed to determine the student's current level of comprehension of a subject. If the student's comprehension is sufficiently high, then the lesson presentation may be skipped and the student can be quizzed on this topic. Otherwise, the student then watches and listens to the audio-visual presentation created by the control program as the process flow continues in block 153. For example, a series of mathematics problems may be presented on display 16 using computer-driven video of a teacher's prerecorded lesson. The control program will also provide software control options for the student such as adjustment of speed or volume of the presentation and the ability to stop and restart the presentation so that the student can take a break or ask the teacher a question as discussed below.

At decision block 154 the control program checks to see if it has presented all of the material for the lesson segment. If so, the subroutine ends and the control program goes to block 180 in Figure 3 to execute a quizzing subprocess described below. If not, the control program polls various flag variables associated with the commands available to the student during the presentation as shown at decision block 156. If a flag has not been set, the control program loops back and continues displaying material at block 153 and then again checks if the lesson segment is over. If a flag has been set, indicating that the student wants to change or stop the presentation, the flow proceeds to a decision tree to implement the corresponding subroutine.

A first such exemplary subroutine is shown beginning at decision block 158, wherein it is asked whether the flag representing the STOP command was set. If not, the flow proceeds to the next command checking decision block 160. If the student has activated the stop command, the process flows to decision block 159 where the student is asked if he or she wishes to save the current position in the lesson segment and return to the main screen. If so, a pointer indicating a last displayed time frame in the lesson segment is stored and the process goes back to block 140 in Figure 2. Otherwise the presentation is temporarily frozen at block 162. Next, the control program checks to see if the student has activated the restart command at decision block 164. If so, the process flow returns to block 153 and continues to display material in that loop. If not, a counter is incremented at block 166 and the count is compared, at block 168, to a predetermined number, for example, a number equal to 5 minutes, to see if the teacher should be notified of the delay. If the current number is equal to the predetermined number, a message is sent over the LAN to notify the teacher at block 170, otherwise, the process flow loops back to again check if the restart command has been given. The SPEED and VOLUME control commands are checked for and implemented at blocks 160, 172, 174, and 178 in a similar manner. Since these functional subroutines are not themselves part of the present invention and are well known in the art, they are not described in further detail herein. Moreover, one skilled in the art will appreciate that a plurality of other commands can be made available to the student by simply adding such subroutines to the decision tree and providing icons or areas of a touch-sensitive screen which will set a flag corresponding to that function. After a lesson segment has finished being presented, the control flow according to this exemplary embodiment of the present invention goes from decision block 154 to block 180 which executes a quiz subroutine illustrated in more detail in Figure 4. At the end of each lesson segment, which can be designed at the courseware level to be of an optimal attention span length for the intended grade level of the student, typically there will be a series of questions retrieved from a database of questions associated with the particular lesson segment for the student to answer at block 182. The student enters his or her responses via the keyboard and the workstation CPU compares these responses to the correct answers stored in the database at blocks 184 and 186. The quiz can be time limited as discussed below. If desired, a grade on the student's responses can be generated and transmitted to the teacher's station for storage in the student's file.

At decision block 186, if the student has correctly answered all of the questions the flow loops back to the main screen block 140 of Figure 2, whereat the student can access another lesson segment, request recreational material, or end for the day. If the student has incorrectly answered some questions, the process flow loops back to block 153 of Figure 3 and the program retrieves and replays only material relating to those questions which were missed. This replayed material could be excerpted from the original presentation or it could be new material specifically designed to explain the correct answer to each incorrectly answered question. The process of replaying this material includes all of the steps discussed above with respect to subroutine block 142 and the subroutines of Figure 3. Alternatively, the presentation of remedial material might occur only when a student's quiz score is below a predetermined threshold.

After playing this remedial material, the same or different questions as those previously answered incorrectly will again be displayed and answered at blocks 182 and 184. The workstation CPU will again check the answers at decision block 186 and transmit the results to the teacher's station for storage. If all of the answers are correct, the process flow returns to the main screen block 140. If, however, after a predetermined number of tries the student still fails to grasp the material and answers some questions incorrectly, the process flows to block 190 and the student's workstation will send a message to the teacher's workstation indicating which material the student is having problems with. The teacher can then use his or her own methods to personally help the student to grasp the material.

According to another exemplary embodiment of the present invention, a curriculum database provides different ways of presenting the concepts contained in lesson segments. For example, the same lesson segment can correspond to plural presentations stored in the curriculum database, each of which is based on a different teaching style, format and/or media. Thus, each lesson segment will have corresponding presentations stored in the curriculum database which emphasize, for example, voice or other audio information, pictures, animation, symbolism, analogy, repetition, and combinations thereof. Of course those skilled in the art will appreciate that these are only a few of the different media and different teaching styles which can be combined to provide different presentations for a particular lesson segment.

An exemplary usage of a curriculum database according to the present invention will now be described. Suppose that a student is working on a lesson segment in Algebra that is intended to convey the relationship between the equation y = mx + b and a line on a graph. The student first works with an animated presentation that illustrates how the equation y = x + 1 corresponds to a line which has slope 1 and intersects the y-axis at coordinate (0, 1). For whatever reason, the student does not master this topic after working with this lesson segment, this lack of mastery indicated, for example, by the quizzing subroutine of Figure 4.

The system then branches back to block 153 of Figure 3, as earlier described, for the presentation of additional material. In this case, the additional material comprises all or part of a different presentation which corresponds to this Algebra lesson segment. For example, the system could provide a presentation which uses an audio and "hands on" approach, taking the student step-by-step through a process that asks the student to draw a graph on a piece of paper, write down the equation y = x + 1 and draw in points which come from the equation. This particular student, who happens to grasp things better by actually doing them, now understands the concepts which eluded him or her after the first presentation and subsequently passes the next quizzing segment so that he or she can move on to new challenges. Of course the foregoing is merely an example, and these presentations could have been described as being in the opposite order, the point being that different approaches to presenting the same material are provided by the curriculum database to prevent frustration on the part of the student and accelerate the learning process. Moreover, the system can learn the idiosyncracies of the individual students and present lesson segments accordingly. Thus, if for example some students grasp material more quickly based on audio presentations, their first presentation of a lesson segment can be one which has been predetermined to emphasize audio stimuli. If, on the other hand, other students react better to visual presentations, their first presentation can be one which has been predetermined to emphasize visual stimuli.

The system can learn the idiosyncracies of the students by conventional means, e.g. , artificial intelligence, expert systems, simply generating statistics based on the number of questions answered correctly after working with each category of presentation type, etc. This information can be compiled and stored in the LAN 44 or the teacher's workstation 40 and used to generate a preference ranking that tailors the order of initial presentation and remedial presentations for each student. Alternately, or in combination with this automatic method, the teacher can designate this order or any part thereof for all or any number of students. The teacher can also temporarily override the presentation preferences of each student. If, for example, it is desired to test a new presentation package which has been added to the curriculum database, then the teacher could override the presentation ranking so that each student first accesses the new package for a particular lesson segment.

In addition to lesson segments, blocks 144 and 146 generally indicate game and video subroutines, respectively, which can be selected by a student from the main screen if they have been sent to the student workstation by the teacher and/or a system program. These selections can be used to reward the student or merely to provide a break from the more educational materials. Like the lesson segment function described above, the selection of these functions initiates a control program which allows the student to play a game or watch a program for a predetermined time. After the predetermined time has elapsed, the program will return the student to the main screen where only lesson segment selections will be available so that the student can complete the assigned lessons for the day. Further detail regarding particular games and video programs and their control are beyond the scope of this disclosure and can be readily implemented using conventional means. Again it should be noted that games and videos are detailed herein only as examples of other functions which can be integrated into the system and method according to the present invention and that many other selections and functions can be readily implemented by one skilled in the art. The END function which is recognized by decision block 147 and implemented at block 148 of Figure 2 can be selected by the student at the end of the day or as a logoff feature if other students need to use the workstation. Housekeeping software will update the database files at the teacher's station regarding the student's activity during the session being terminated. Homework for the following day corresponding to the lesson segments completed during the session can be can be retrieved from memory and generated by the printer. Additionally, progress reports or remedial notes can be generated at the printer for the student to take home to his or her parents.

The teacher's workstation 40 can comprise a system similar to that of the student workstation 10 which may be enhanced to provide sufficient processing resources to control the LAN and provide monitoring capabilities of each of the student workstations tied into the LAN. The teacher's workstation can include a more powerful CPU, larger on-board memory and a larger hard drive to store, for example, all of the various databases used to store information received from the students' workstations. In addition to the functions discussed above, the teacher's workstation also provides the teacher with other capabilities including, editing the databases associated with the materials and questions of the lesson segments and report generation features which allow the teacher to easily generate grade reports on any or all of the students. An exemplary teacher's workstation will now be described with respect to Figures 6-16 which show exemplary interface screens. Figure 6 depicts a first menu from which a teacher may select various submenus, the classroom shutdown command or the quit command. The class shutdown command provides a way in which the teacher can signal the students' workstations that, for example, the end of a class period is approaching and that students should not be permitted to access additional lesson segments which might extend beyond the end of the classroom period. The classroom setup submenu is illustrated in Figure 7. Therein various functions including adding or changing a student's information, deleting a student, printing the classroom lists, and printing logon names are provided. An exemplary screen for adding or changing a student appears as Figure 8.

The lesson setup or lesson maintenance menu is illustrated as Figure 9. Each of the first five options listed from the top of this menu have examples thereof shown in Figures 10-

14. By clicking on the print lesson information option, the teacher has the ability to capture information regarding the lessons in printed form. As can be seen in Figure 10, each lesson segment can be presented to the students in any order desired by the teacher. For example, the teacher could change the numbers indicated in the "lesson order" column to change the order of lesson presentation based upon his or her teaching experience.

Figure 11 shows an exemplary chapter test location menu which allows the teacher to selectively determine when quizzing will be performed relative to the presentation of the lessons. Additionally, a teacher can exclude certain lessons from the quiz as desired. Figure 12 shows an exemplary interface which can be used for the teacher to determine what score on a quiz will allow a student to proceed to the next lesson. For example, a threshold score can be entered in the "Grade" column which will allow a student to proceed to the next lesson. Each quiz can have its own score for passing individually determined by the teacher or a universal threshold could be provided for all quizzes.

Figure 13 displays an interface which can be used at the supervisory station to provide a maximum quiz time within which period a student must finish taking a quiz. At the end of this maximum time, the system would then evaluate the student's current responses and provide a grade or score. The homework editor screen of Figure 14 can be used by the teacher to associate various assignments with each lesson presented by the system. Thus, at the end of a class (as indicated by the teacher shutting down the classroom) homework can be generated based upon the entries and changes made by teacher on this screen.

Figure 15 illustrates a gradebook menu screen which allows the teacher to easily output grades either as a hard copy or to the teacher's display. Although the screen shows two examples of ways in which grade information can be output, those skilled in the art will readily appreciate that the grade database can be queried in a number of different ways to produce different reports. One exemplary report is illustrated as Figure 16. In this screen, the students are alphabetized and their scores and progress through each lesson segment is both graphically and numerically portrayed so that a teacher can use this gradebook screen to quickly determine which student or students are having problems with which particular lesson segments, i.e. , by looking at the color coded indications on the screen.

According to another exemplary embodiment of the present invention, speech education can also be provided as a feature of the aforedescribed system. As mentioned above, speech education according to the present invention refers to techniques which are more interactive and dynamic than conventional speech recognition. An example will be provided using the flowchart depicted in Figure 17 as well as the conceptual block diagram of Figure 18 which identifies some of the hardware and software components involved in the speech education techniques according to the present invention.

First, speech education according to the present invention provides the ability for the system to provide one or more visual and/or audible examples of phonemes, syllables, words, phrases or sentences that are the object of the lesson. This can be accomplished by, for example, an object word "theatrical" being displayed on display 16 and a correct pronunciation thereof output as synthesized speech from audio interface 32 (e.g. , via speaker (s) 1800). As with the earlier described embodiments, the manner in which examples are provided by the system, e.g., a straightforward visual representation of the word, a picture reinforcement illustrating the word, a visual representation of the word made up of its composite phonemes (as in the dictionary, typically to the right of the word itself), an audio segment of a speech professional saying the word or an audio segment of a speech professional saying each phoneme that makes up the word individually, can be established by the teacher using the teacher's workstation 40 to set the parameters for the speech education class, at the student's own workstation 10, or by the system based upon the student's own preferred learning style as reflected by a pattern of correct/incorrect responses. For example as in the previous embodiments, the system can initiate the lesson using a first style of example (audible repetition of the object word) and, for failed attempts, can repeat the example with a reinforcing example which emphasizes a different stimuli (e.g., a visual or audible phonetically broken down version of the word). Moreover the number of repetitions of each example is also variable by the system, e.g. , under the control of the teacher at the teacher's workstation 40.

The example or examples can be provided to the student in conjunction with the the courseware's display of educational aids on the display 16 via graphical user interface (GUI) 1804. In this instance the courseware is referred to as application software layer 1806 to refer to its interface and control functionality for speech engine 1808 (described below) and GUI 1804. The speech objects of interest can, for example, appear to be pronounced by an educational character displayed on display 16. Once the example or examples have been provided, the student will be prompted to repeat a speech object at step 1702. This prompt can also be visual, audible or both. Additionally, it may be useful to prompt the student to repeat the speech object multiple times, e.g. , in order to allow the system to grade and average multiple responses to provide a more accurate assessment of the student's ability to properly pronounce the object. The student's responses are captured by the system, e.g. , by a microphone 1802 which is included in audio interface 32 of the student's workstation. Depending upon the ambient conditions, various approaches and techniques can be implemented to enhance the process of capturing an accurate electrical representation of the responses uttered by the student. For example, the microphone 1802 can be implemented as a so-called horn microphone having a conically-shaped input element designed to funnel the speech input into the transducer which captures the speech information. Additionally, or in the alternative, echo cancelling techniques and other noise reduction signal processing techniques, which signal processing techniques are well known in other arts, e.g., cellular telephones, can also be added to improve the captured speech information. Once captured, the student's responses are recorded e.g. , in any of the memory devices associated with the student's workstation 10 (e.g. , hard drive 22), the LAN 44 or the teacher's workstation.

Having captured the student's response(s), the next step 1706 in interactive speech education techniques according to the present invention is to evaluate those responses. This can be accomplished using any suitable software speech engine 1808 designed for the purposes of speech evaluation. However, Applicants have determined that it is preferable that the software engine be able to analyze the speech object responses at a phoneme level rather than a word level. Accordingly the software engine which is selected for this task should be capable of breaking down the captured responses into their individual phonemes and grading the pronunciation of each phoneme. An example of this type of software engine is that produced by Idioma Ltd. , an Israeli company having a place of business at 15 Maskit Street, Herzeliya, 46733, Israel. The interested reader can find additional information about this exemplary software engine at http://www.idioma.co.il, the disclosure of which is expressly incorporated here by reference. Those skilled in the art will appreciate, however, that other speech evaluation engines could also be used to implement the present invention.

The output of the software engine 1806 is further processed by the application software 1804 running on workstation 10 to determine an appropriate reinforcer based on the evaluated responses. For example, the software engine 1806 may provide as an output a numerical grade of each phoneme in the recorded version or versions of the speech object. This output can then be evaluated by the application software to provide a suitable reinforcer and/or to determine and perform a repetitive exercise intended to correct a mispronounced word. For example, suppose that the software engine 1806 evaluates the phonemes in two captured versions of the word "theatrical" and outputs averaged numerical grades of 90% , 92% , 84% and 99% for the four phonemes which comprise this speech object. The application software 1804 would receive these numerical grades and compare them with a threshold, which may be settable by the teacher at his or her workstation. In this example, suppose that the threshold is 80% such that this was considered by the system to be correctly pronounced. The application software 1804 would make this determination and command the GUI 1802 to output a positive reinforcer at step 1708, e.g. , one of the educational characters saying "Good Job Ellen! " . Alternatively, suppose that the first phoneme "the" in the speech object "theatrical" is mispronounced (i.e., below a certain threshold of identity as compared with the correct enunciation of this phoneme) as determined by application software based on its evaluation of the output of software engine 1806, but that the remaining phonemes are pronounced correctly. In this case, the application software 1806 can command GUI 1804 to output a repetitive reinforcer, e.g. , by having an educational character repronounce the word

"theatrical" with emphasis on the first phoneme and/or displaying the word broken down into its phonemes with the first phoneme highlighted.

As mentioned earlier, the decision-making as to the manner and stimuli in which examples and remedial examples are output on the part of application software 1806 can also be influenced by a student's recorded history of exercise-mistake-corrective example-response to enhance the learning process. For example, if this particular student has a recorded history of correcting mistakes with fewer iterations using audible emphasis for incorrectly pronounced phonemes, then this type of remedial reinforcement can be provided at step 1708. Application software 1806 can also decide to perform another iteration of the flow illustrated in Figure 17 if the grades output by speech engine 1808 warrant retrying that particular speech object.

Although the foregoing description has been directed to an exemplary system for implementing educational methods according to the present invention, one skilled in the art will appreciate that many other systems or variations of the system described above could readily be configured to implement an educational method according to the present invention. For example, other types of memory storage devices and communication systems linking the student workstations to the teacher workstation could be readily implemented. Methods according to the present invention are also not limited to a single group of student terminals linked to a single teacher's workstation, but could include an entire school's workstations or more.

Moreover, one skilled in the art will readily appreciate that emerging multimedia technologies and computer developments readily lend themselves to integration with various embodiments of the present invention. For example, expert systems could be used as part of the courseware so that the system develops its own questioning and remedial presentation techniques. Moreover, the advent of computer voice synthesis and recognition provides another alternative for both student input and system audio output.

The foregoing description which sets forth exemplary preferred embodiments of a novel educational method and system for carrying out such a method is merely intended to be illustrative of the present invention rather than restrictive or limitive thereof. On the other hand, the scope of the invention is set forth in the appended claims which are intended to encompass what is directly set forth therein as well as reasonable equivalents thereof including those expressly and implicitly set forth in the disclosure.

Claims

What is claimed is:

1. An educational speech teaching method using an interactive audio-visual computer system comprising the steps of: outputting, by said computer system, at least one example of a speech object; prompting, by said computer system, a student to repeat said speech object; storing said repeated speech object in said computer system; evaluating said stored, repeated speech object using a speech evaluation engine to generate grades associated with said stored, repeated speech object on a phoneme level; and determining, by said computer system, whether to provide another example of said speech object or to output a positive reinforcer based on said grades generated by said speech evaluation engine.

2. The method of claim 1 , wherein said step of outputting at least one example of a speech object further comprises the step of: displaying an educational character that pronounces said speech object and simultaneously outputting an audible example of said speech object.

3. The method of claim 1 , wherein said step of outputting at least one example of a speech object further comprises the step of: displaying a visual representation of said speech object broken down into its composite phonemes.

4. The method of claim 1, wherein said step of outputting further comprises the step of: repeating the output of said speech object a plurality of times.

5. The method of claim 1 , wherein said step of storing said repeated speech object in said computer system further comprises the step of: capturing said repeated speech object by microphone; and recording an electrical representation of said speech object in a memory device associated with said computer system.

6. The method of claim 1 , wherein said step of determining, by said computer system, whether to provide another example of said speech object or to output a positive reinforcer based on said grades generated by said speech evaluation engine further comprises the step of: comparing values of said grades generated by said speech evaluation engine with a threshold; and selectively outputting said positive reinforcer when all of said grades exceed said threshold.

7. The method of claim 1 , wherein said step of determining, by said computer system, whether to provide another example of said speech object or to output a positive reinforcer based on said grades generated by said speech evaluation engine further comprises the step of: comparing values of said grades generated by said speech evaluation engine with a threshold; and selectively outputting a remedial example of said speech object when at least one of said grades is less than said threshold.

8. The method of claim 7, wherein said remedial example is different than said at least one example.

9. The method of claim 8, wherein a style of said remedial example is selected based upon an historical performance of said student.

10. A computer system for implementing a computer-aided, speech teaching method comprising: means for outputting, by said computer system, at least one example of a speech object; means for prompting, by said computer system, a student to repeat said speech object; means for storing said repeated speech object in said computer system; means for evaluating said stored, repeated speech object using a speech evaluation engine to generate grades associated with said stored, repeated speech object on a phoneme level; and means for determining, by said computer system, whether to provide another example of said speech object or to output a positive reinforcer based on said grades generated by said speech evaluation engine.

11. The system of claim 10, wherein said means for outputting at least one example of a speech object further comprises: means for displaying an educational character that pronounces said speech object and simultaneously outputting an audible example of said speech object.

12. The system of claim 10, wherein said means for outputting at least one example of a speech object further comprises: means for displaying a visual representation of said speech object broken down into its composite phonemes.

13. The system of claim 10, wherein said means for outputting further comprises: means for repeating the output of said speech object a plurality of times.

14. The system claim 10, wherein said means for step of storing said repeated speech object in said computer system further comprises: a microphone for capturing said repeated speech object; and a memory device for recording an electrical representation of said speech object.

15. The system of claim 10, wherein said means for determining, by said computer system, whether to provide another example of said speech object or to output a positive reinforcer based on said grades generated by said speech evaluation engine further comprises: means for comparing values of said grades generated by said speech evaluation engine with a threshold; and means for selectively outputting said positive reinforcer when all of said grades exceed said threshold.

16. The system of claim 10, wherein said means for determining, by said computer system, whether to provide another example of said speech object or to output a positive reinforcer based on said grades generated by said speech evaluation engine further comprises: means for comparing values of said grades generated by said speech evaluation engine with a threshold; and means for selectively outputting a remedial example of said speech object when at least one of said grades is less than said threshold.

17. The system of claim 16, wherein said remedial example is different than said at least one example.

18. The system of claim 17, wherein a style of said remedial example is selected based upon an historical performance of said student.

19. An educational speech teaching method using an interactive audio-visual computer system comprising the steps of:

(a) outputting, by said computer system, at least one example of a speech object;

(b) repeating, by a student, said speech object;

(c) storing said repeated speech object in said computer system;

(d) evaluating said stored, repeated speech object to generate grades associated with said stored, repeated speech object; and

(e) determining, by said computer system, whether to repeat steps (a)-(d) for said speech object or to output a positive reinforcer based on said grades generated by said computer system.

20. The method of claim 19, wherein if steps (a)-(d) are repeated, then a remedial example is output in step (a), which remedial example is different than said at least one example.