WO2001043107A1 - Computer-based, interactive learning method and system - Google Patents

Abstract

Methods and systems for assisting individuals learn subject matter and, in particular, to prepare for tests, preferably standardized tests. The inventive methods and systems employ multiple teaching techniques which are all geared to the individual's level, abilities, goals and/or circumstances. The methods and systems are advantageous because they also allow users to learn to take practice exams at their own pace while accommodating any personal circumstances.

Description

TITLE OF THE INVENTION

Computer-Based, Interactive Learning Method and System RELATED APPLICATIONS

This application is related to U.S. Application Serial No. 09/457,621, filed December 8, 1999, hereby incorporated by reference.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates generally to improved computer-based, interactive learning methods and systems for teaching or educating users and to improved methods and systems for evaluating user performance and or learning method or system performance. Description of Related Art

Several publications are referenced in this application. The references describe the state of the art to which this invention pertains and are hereby incorporated by reference.

The education an individual receives throughout life is frequently thought of as the single most important factor in determining the opportunities available to the individual, as well as whether the individual will have the ability to take advantage of such opportunities. Educational advancement often depends on the education received early in life. For example, the performance of the individual during grade school level and later in high school, will typically determine which colleges or universities the individual may be accepted. In turn, the individual's performance at the college level will typically determine which graduate schools the individual may be accepted and/or which careers the individual may enter or pursue.

Evaluating an individual's performance or predicting the individual's ability to perform in the future was typically done using the grades the individual received during classroom study and instructor recommendations. However, an increasingly important factor in assessing an individual's performance and ability to perform in the future is the individual's score on standardized tests. Standardized tests are increasingly used to gauge and improve student performance. If the individual receives a high score on such tests, opportunities are created. In contrast, there are real-world consequences for those students who do not "measure up", such as the inability to attend colleges or universities of choice and/or lost scholarship opportunities. It is well known that a "perfect score" on the SATs will allow the student to enter almost any college, whereas an unusually low score may foreclose many opportunities.

Standardized tests such as the Scholastic Aptitude Test (SAT), Graduate Record Examination (GRE), Law School Admissions Test (LSAT) and Medical College Admission Test (MCAT) are used to test the intelligence or aptitude of millions of students each year. Moreover, many states and municipalities are developing similar tests for use in public education systems to assist in evaluating educational techniques or whether a student should be elevated to the next grade level or allowed to matriculate to a "charter" or "magnet" school. Such tests are also employed to determine the quality of the institution where the tested individuals receive their education. As a result, the standardized test format has become a pervasive tool for evaluating individual performance and determining which opportunities, educational and otherwise, will be made available to the individual. The Big Test: The Secret History of the American Meritocracy. Nicholas Lemann (Farrar, Straus & Giroux, 1999).

Even though these tests have been criticized as being biased, unfair or an inaccurate measure of an individual's abilities, the results of these tests remain - at the very least - an important factor in determining whether an individual will be allowed to matriculate to certain colleges and universities. Thus, an individual's preparation for such tests is a critical factor that may determine which colleges or universities the individual will be able to attend. However, given the limitations of such tests, some admissions offices have used standardized test scores for evaluation purposes in addition to other factors such as the individual's demographics, race, nationality, etc. If a standardized test is perceived as biased against one group of individuals, an admissions office may compensate for such a bias by taking this into account when reviewing the scores of such individuals. As discussed in more detail below, an individual's inability to gauge the existence or extent of such compensation may inhibit that individual from taking advantage of all possible opportunities.

Thus, not surprisingly, the ability to perform well on any given standardized test is critical and of the utmost importance. As a result, numerous practice books and commercial instructional courses have been created to assist the individual in improving his or her scores. Many individuals purchase practice exams, video or audio tapes, attend classes and take other actions to improve their performance. However, many of these methods are costly and criticized as being ineffective in helping the individual achieve a higher score. Thus, it would be desirable to develop a system and method to improve an individual's performance on a standardized test in an inexpensive and efficient manner.

There are a variety of techniques for teaching subject matter relating to one or more topics to an individual. There are also a variety of techniques for testing the individual's knowledge of one or more topics to determine how much the individual has learned and/or to determine the abilities of the individual. Such tests include written answers, multiple choice, essay questions, etc.

Individuals can learn passively by simply reading information or listening to a lecture regarding a particular subject. Other methods include interactive methods where the student asks questions or is asked questions to stimulate learning and better focus on the student's strengths and weaknesses. The review of previously learned subject matter is important in any educational or test preparation system. The ability to review subject matter both before and after taking a test, for example, allows an individual to learn subject matter, test the user's knowledge based on that subject matter and then review subject matter where the test shows the individual may have a weakness.

Thus, educational computer-based systems that test a user's knowledge, coupled with the immediate scoring of answers, are well-known in the art. The computer has greatly increased the ability of any individual to learn new subjects. The computer not only provides a means for providing a vast amount of information for a user to absorb, but also allows for the development of interactive programs which permit users to ask questions relating to a subject. U.S. Patent No. 5,934,910 to Ho, et al.

In most of the national standardized tests, such as the Scholastic Aptitude Test (SAT), the questions are of multiple choice, and are automatically graded to produce a report. Usually, the grading and report-generating are computerized. The SAT report includes two scores, the first indicating a student's mathematics skill and the second, the student's verbal skill. It is impossible to decipher from the two numbers the areas where the student is weak. For example, if the student's mathematics score is low, it is impossible to determine from the score alone if the low score is a result of the student being weak in algebra, geometry or both.

If the student is not satisfied with his scores, he may go to the remedial classes described above. Typically, in such a class, the student takes practice tests with many other students. The practice tests and study materials provided for all the students in the class are the same since it is difficult to manually tailor each test or study material to the understanding of each student. After a practice test, a student may compare his test results with his previous results to see if he has improved. Again, the comparison is quite superficial. The student's ability to determine his strengths and weaknesses is limited to reviewing the different scores for each section. If the recent test score is higher than a previous test score, then he assumes that he has improved. This type of feedback on one's performance is not optimal.

Thus, improved methods and systems of improving a student's performance on standardized tests are desired.

There are several existing computer-based methods and systems for intelligent tutoring. Some programs provide users with a series of questions which the user is tested and later provided with the answer. Certain home education computer-based systems allow users to be tutored by the system before having to answer a specific question or solve a problem. Currently, there are numerous stand-alone software programs for test preparation and a few on-line test preparation centers. Typically, these programs or websites allow a user to take an exam and review their performance. This allows the user to identify weak areas that require additional preparation. Some of these sites allow users to do practice problems in mathematics or verbal areas separately, for example, so that they can hone their skills in each area independently.

Known systems for assisting test takers in standardized tests such as the SAT include: on-line learning centers which provide test-taking tips, discussion groups, and other resources for students. Such centers typically attempt to develop the students test taking skills by providing a series of sample tests for practice and then providing the answers to the tests. Some systems evaluate the student's test taking ability and indicate the student's strengths or weakness.

Methods and systems for test preparation and evaluation include the following:

U.S. Patent No. 5,180,309 by Egnor issued January 19, 1993 relates to a method and system having the objective of automatically allotting partial credit for a user's response that is less than perfect but is within a pre-selected tolerance that is associated with the preprogrammed answer to the question. U.S. Patent No. 4,759,717 discloses a student response system intended to collect data of a limited nature (e.g., responses to multiple-choice questions) from a number of classrooms.

U.S. Patent No. 5,947,747 relates to a system for providing a computer-based assessment of an individual's educational performance.

U.S. Patent No. 5,597,312 to Bloom et al. relates to a computer-based intelligent tutoring method and system for providing tutoring to a student in an interactive application. The method and system are for teaching customer service representatives (CPRS) by working through either abstract or concrete simulations of on-the-job scenarios. The system monitors the student's performance, provides feedback on his performance, provides hints on expert responses during problem solving sessions, and employs several strategies to ensure that the student is continually but not excessively challenged, including tailoring the style of instruction and choice of scenarios to the individual and skimming over well known parts of a scenario during problem solving. The system tests the student's verbal responses to sample customer questions and evaluates his ability to process orders and the like.

U.S. Patent No. 5,934,910 by Ho et al. relates to a system and method for computer learning based on question asking. That is, the method and system teaches a subject based on the user's natural language questions.

U.S. Patent No. 5,934,909 by Ho et al. relates to an educational method and system that automatically assesses and enhances a student's understanding in a subject. The system provides individually-tailored tests whose difficulties are geared towards the student's level of understanding in the subject. The student can use the system to prepare for an examination and can also use the tests to learn the subject. In one embodiment, the assessment and the enhancement take into account the student's past performance. In another embodiment, the system takes into account the time in which the test is taken compared with previous tests to determine the student's forgetfulness. The system also includes a means for generating a report which provides an assessment of the student'sstrengths and weaknesses in different areas of the tested subject. The report may further provide action items indicating ways for the student to improve on the subject. For example, the report may suggest that the student spend more time on simple multiplication and division. However, the system includes only the test taker's identity. Thus, a truly individualized and personalized system for preparing an individual according to the present invention is not provided.

U.S. Patent No. 5,306,154 by Ujita et al. relates to an educational system, and more particularly, to an educational system which is suitable for teaching operational procedures in a plant, and physical behaviors in the plant by using a simulator. An object of the system is to provide an educational system which is capable of implementing an optimized re-education capable of reflecting idiosyncrasies of learners in their understanding capabilities. The portion of the curriculum requiring re-education is selected in accordance with the degree of understanding of the learner and accounting for his/her idiosyncrasy in learning, so that a pertinent re-education optimized for each learner is capable of being executed, and a time for implementing such reeducation is capable of being minimized.

The previous methods and systems suffer from several disadvantages. In the typical computer-aided educational system, there is no review. Those systems assume the students remember everything they have learned, and just keep on teaching new areas. If a student wants to review certain subject matter, he can always go back and search for it. However, most students generally do not go back; they typically proceed forward and believe that once they have learnt a subject, they will never forget. As forgetfulness and lack of understanding start to build up, not before long the typical student will become confused and unable to further leam the subject matter. Another disadvantage of previous systems and methods is that each fails to fully evaluate the individual user's test preparation level and personal circumstances continually or dynamically. More specifically, the prior methods of preparing a user for a standardized test evaluate the user after a single test.

One additional drawback of the existing educational system is that it cannot take into account relative norms (e.g., comparison). In most computer-based systems, each user exists in isolation from all other players. This arrangement may be suitable for informal, isolated learning, but it lacks the comprehensive data collection/comparison features needed for formal test administration. For example, without an on-line database of scores, there is no way for a user to compare his score with other user's scores.

Yet another disadvantage of the previous methods and systems is their narrow focus to attempt to improve the score of each individual user, rather than provide useful information which may be derived during the preparation for such tests and the student user's performance on the standardized tests.

Thus, it would be desirable to provide an improved system and method for learning, particularly for standardized test preparation, which overcomes the above described disadvantages. More specifically, it would be desirable to provide a method and system which provides personalized, dynamic interactive teaching which not only increases the individual's ability to perform well on a standardized test, but also provides other useful information to better equip the individual to make what could be some of the most important decisions in his or her life and/or allow researchers to better evaluate the weaknesses and biases of standardized tests. OBJECTS OF THE INVENTION

It is an object of the invention to overcome the above-identified deficiencies.

It is another object of the invention to provide a test preparation or learning method and system which is flexible and allows for personalized, interactive and dynamic assistance to student users.

It is a further object of the invention to provide a method and system which provides a more valuable evaluation of the performance of the student user.

It is a still further object of the invention to provide a method and system which facilitates the student user's selection of college/university and application to the same.

It is a still further object of the invention to provide a learning method or system which allows the authors or developers of the specific learning method or system to better evaluate and improve the learning experience for the user.

It is a still further object of the invention to provide a method and system which facilitates the evaluation and development of standardized tests or test questions or the evaluation of groups or subgroups of individuals.

The foregoing and other objects and advantages of the invention will be set forth in or apparent from the following description.

SUMMARY OF THE INVENTION

The present invention relates improved methods and systems for assisting individuals learn subject matter and, in particular, to prepare for tests, preferably standardized tests. The inventive methods and systems employ multiple teaching techniques which are all geared to the individual's level, abilities, goals and/or circumstances. According to one embodiment, the method and system may employ: (1) a personalized study plan, (2) a personalized list or database of study materials or study action items (herein called "study basket"), (3) a personalized test taking analysis and evaluation, (4) additional study materials such as practice exams, practice questions, test taking tips, SAT dictionary, Roots & Stems, vocabulary builder, problem builder and personal study notes, (5) a test taking pace analysis, (6) nervousness indicator, (7) statistics module to assist the student user in evaluating their performance compared with other student users, and assist the author or developer of the learning environment or experience to evaluate the effectiveness of the method or system (e.g., effectiveness of study materials, etc.), (8) a rule- based system that allows the authors, developers, creators or administrators of the study material to create or provide personalized feedback for student users, (9) allow collaboration by sharing study aids among users and allowing for the searching of such shared study aids by the users, (10) enables the systematic and automatic development of an "expert community" (e.g., users who perform above a specified level during the learning process and thus identified as "experts" capable of assisting other users), (11) creation and development of an expert center where individual users can submit questions and obtain answers, (12) a personalized university center to organize and assist the individual in the preparation for the test and any application process which utilizes the test score, (13) personalized "college counselor" (e.g., list of colleges or university offering specified majors or evaluations of probability of acceptance to specified colleges or universities, etc.) or combinations thereof. The method and system are advantageous because they also allow users to learn and take practice exams at their own pace.

One main objective of the invention is to provide personalized instruction. This involves not only customizing the type and extent of instruction based on the student's performance during practice or actual tests, but also taking into account as many relevant circumstances as possible such as when the student plans to take the standardized test, how many hours per day can the student dedicate to preparing for the test, any target test score the student hopes to achieve, etc. Moreover, preferably, the customization is dynamic in that it can be changed as circumstances change. This allows a truly personalized study plan, for example, to be provided to the student

which takes these factors into account.

Another preferred embodiment of the invention provides a valuable resource for students as well as other non-student users such as college/university admissions offices, standardized test developers and evaluators, psychologists, sociologists, etc. More specifically, the method and system preferably include the formation of a database of information relating to the profiles of students. The profiles can include a variety of useful profile information such as the student's age or sex, race or nationality, demographics, etc. Preferably, the student information which may be input according to the method or system is divided into confidential information such as student name and social security number (which is not disclosed to non-student users) and student profile information (such as age or sex, race or nationality, demographics, etc) which forms an anonymous student profile database which may be utilized by student users and/or nonstudent users.

Other objects as well as aspects, features and advantages of the present invention will become apparent from a study of the present specification, including the drawings, claims and specific examples.

BRIEF DESCRIPTION OF THE DRAWINGS

Fig. 1 illustrates a block diagram of a system according to one embodiment of the present invention;

Fig. 2 illustrates a block diagram of the server of Fig. 1 ;

Fig. 3 illustrates a block diagram of the student user terminal of Fig. 1;

Fig. 4 shows a flow diagram according to one preferred aspect of the invention;

Fig. 5 shows a flow diagram of one embodiment of the invention;

Fig. 6 shows a flow diagram of another embodiment of the invention; Fig. 7 shows a flow diagram according to yet another embodiment of the invention;

Fig. 8 shows a flow diagram according to a still further embodiment of the invention; and

Fig. 9-33 depict typical output displays of the method and system according to preferred embodiments of the invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

The present invention broadly relates to methods and systems for teaching one or more individuals subject matter of an educational and/or professional nature. Although the invention is particularly applicable for assisting a student user prepare for a standardized test, the invention also relates to generally assisting individuals learn any type of subject matter. The term "student user" refers to both students and other individuals who seek to learn additional subject matter.

One aspect of the invention relates to methods and systems for preparing student users for educational tests, particularly standardized tests, such as the GED, SAT, ACT, GRE, LSAT, MCAT, GMAT, Optometry Admission Test, Dental Admission Test, MAT, PCAT, TOEFL, TOEIC, NTE, HERBIE, Basic Competency (Elementary School), Proficiency, New York Reagents Exam, Catholic High School Entrance Exam, etc. The invention also relates to assisting individuals prepare for professional tests such as CPA, Insurance Broker, Insurance Agent, Investment, Series 6, Series 7, Multi-State Bar Exam, Professional Engineer (PE), Real Estate Broker, Police Officer, etc.

The invention also relates to methods and systems for assisting companies or other entities train employees. For example, a company may have a vast amount of information available relating to company procedures, company manufacturing or production plans, company services, etc. Not all employees would need to leam all information relating to the company. Certain types of employees may need to leam subject matter relating to topics that other types of employers do not need to learn and vice versa. At the same time, there may be overlapping subject matter that each employee needs to learn. For example, a new employee in the financial department may need to learn about the company's financial procedures, but not about the specifications for its products. Conversely, a newly hired engineer may need to leam the opposite. At the same time, there may be certain types of information that both employees should learn (e.g., travel reimbursement procedures, company anti-harassment policies, use of company computer network, etc.).

A preferred aspect of the invention relates to methods and systems for preparing student users for standardized tests which are given to a large number of students. The SAT, for example, is given to almost 2 million high school students a year, with most students taking the test in the spring of their junior year and/or the fall of their senior year of high school. The SAT is designed to allow college admissions officers to judge all students using a common yardstick. The SAT is intended to compensate for the unreliability surrounding high school grades (e.g., school to school differences, grade inflation, quality of teaching, etc.). Thus, the student's score on the SAT is an important factor in determining which colleges the student may attend.

As stated above, the problem with previous test preparation methods and systems is that they generally treat all users in the same or similar manner (i.e., a high scoring student is taught in the same manner as a low scoring student, a student with two months to study is treated the same as a student with two weeks to study, etc.). As a result, the instruction is not optimized for the individual and therefore does not provide the best method for preparing a student for an examination. Some students may learn at one pace, others at another pace. Some students may have a strong understanding of geometry, others may not. Some students may have one month to prepare, others may have three months. Some students may be able to dedicate two hours a day for preparing, while others can devote six hours. The present invention removes many of the limitations of prior methods and systems by adding flexibility and personalized assistance so that each individual is treated not only based on their performance on each practice test, but also based on their individual circumstances. This allows the instruction and other assistance provided to the student to be optimized. For example, the method and system may not only provide the student user with an analysis of his strengths and weaknesses, but may also provide a detailed daily study plan designed to improve the student's performance.

Another aspect of the invention is the optimized use of computers for learning. The term "computer" is intended to include a device having a digital processor or the like such as a desktop, notebook or handheld computer, cellular phone or the like. The inventive methods and systems are employed using computers, preferably networked computers such as those linked via the Internet. One preferred embodiment of the invention relates to practicing the inventive method and/or constructing the inventive system using a computerized network such as a LAN (local area network), Internet, Intranet or the like. Preferably, the invention incorporates the use of the Internet to allow an unlimited number of student users to readily access the inventive service. Not only does using a computer network such as the Internet more efficiently provide the service to an unlimited number of users, it also provides further advantages to student users or nonstudent users not previously provided with previous educational methods and systems including previous computer based methods and systems.

The explosive growth of the Internet, and specifically, the growth of the World Wide Web ("the Web"), has allowed a number of new services and products to be provided in an efficient and cost effective manner. The Web is built around a network of "server" computers which exchange requests and data from each other using hypertext transfer protocol ("http"). A designer designs the layout of a Web page, for example, using Hypertext Markup Language ("HTML"). Several versions of HTML are currently in existence. Examples include HTML versions 2.0 and 3.0, as specified by the WWW Consortium of MIT.

A user views a Web page using one of a number of commercially available "browser" programs. The browser submits an appropriate http request to establish a communications link with a Web server of the network. A typical http request references a Web page by its unique Uniform Resource Locator ("URL"). A URL identifies the Web server hosting that Web page, so that an http request for access to the Web page can be routed to the appropriate Web server for handling. Web pages can also be linked graphically to each other.

Information presentations on Web pages are often abbreviated or shortened in order to save space and to produce better layouts. Typical examples of abbreviated or shortened information includes the use of icons. A user may merely click the icon, for example, to retrieve additional information and or web pages. Figures 9-33 depict several typical output displays of a web-based method and system according to several embodiments of the invention.

Fig. 1 illustrates a system 100 according to one embodiment of the invention. System 100 includes student user terminals 101 A, 101B, 101C in which the teaching methods according to the invention may be executed. A student user receives and inputs information provided on student user terminal 101 A which is connected to server 103 via network 102 through two-way communication links. Network 102 may be any suitable network, preferably a public switched phone network, such as the Internet, an Intranet, LAN or the like. Network 102 may include a standard frame-relay network such as is operated by AT&T or MCI. At various locations, preferably geographically remote from that of server 103, a number of intermediate local servers may connect server 103 via network 102 to student user terminals 101 A, 101B and 101C. A student user preferably accesses server 103 via network 102 using student user terminal 101 A. System 100 may further comprise nonstudent user terminal 104 which is also connected to server 103 via network 102. Server 103 preferably controls the flow of data to and from student user terminals 101 A, 101B and 101C and nonstudent user terminal 104. There are no special requirements for student user computer 101 A or nonstudent user terminal 104, other than that each have some means of assessing the information from server 103, preferably some means of assessing the information via network 102. Terminals 101 A, 101B and 101C and nonstudent user terminal 104 may be structurally the same or different, except student user terminals 101 A, 101B and 101C are used by student users, whereas nonstudent user terminal 104 is used by nonstudent users such as college admissions officers, test developers or evaluators, researchers, etc. System 100 may include any number of student user or nonstudent user computers. System 100 may include server 103 or means for conveying information to and from server 103 when server 103 is external to system 100.

Figure 2 illustrates a block diagram of server 103 of Figure 1. Server 103 includes CPU 201, cryptographic processor 202, RAM 203, ROM 204, network interface 205 and data storage device 210. Network interface 205 links server 103 to network 102. CPU 201 is preferably connected to each of the elements of server 103. Server 103 may comprise one or more servers (not shown). CPU 201 executes program instructions stored in RAM 203, ROM 204 and data storage device 210 to perform various functions of the present invention.

Data storage device 210 preferably includes a combination of a plurality of databases such as test preparation database 211, student user account database 212, student profile database 213 and nonstudent user account database 214, as well as program instructions (not shown) for CPU 201. Test preparation database 211 may contain practice exams, practice questions, tips, vocabulary words, etc. Student user account 212 may include student user information such as account status, personal information, personal study plan, personal study basket, My Notes, etc. Student profile database 213 may include student user profiles, statistics, etc. The databases in data storage device 210, such as databases 211, 212, 213 and 214, are preferably implemented as standard relational databases capable of supporting searching and storing multimedia information such as text, audio, photographs, video, QuickTime movies, etc.

According to one prefened embodiment, the mailboxes are integrated mailboxes which provide integrated e-mail, facsimile and/or voice mail services which allows the user to be facsimiled or voice mailed information rather then e-mailed. This may be particularly helpful, for example, when the information being communicated relates to mathematics problems. Voice-mail would be advantage when an oral explanation is more beneficial than a written explanation.

CPU 201 is also preferably programmed to search databases including databases 211, 212, 213 and 214 and transmit information in response to a student user or nonstudent user search request. CPU 201 receives a search request containing certain criteria (e.g., test score, age, race, etc.) and searches the databases to find one or more matches. Based upon the search, CPU 201 releases certain information to the student user or nonstudent user requestor. Preferably, CPU 201 may assign pseudonyms to each student user, if requested, to maintain anonymity. There are a number of search techniques that can be used including keyword, fuzzy logic and natural language search tools. According to one embodiment, CPU 201 compares the criteria against each student user profile stored in system 100 using one or more search algorithms and transmits to the requestor the student user profiles or group or subgroup of student user profiles identified.

CPU 201 is preferably also programmed to provide communications, preferably anonymous communications, between one or more student users and one or more nonstudent users. In one embodiment, CPU 201 can receive and store electronic mail messages in electronic mailboxes assigned to each student user and/or nonstudent user, thereby establishing a non-real- time communications channel between student users and education researchers, for example. According to one preferred embodiment, the personal mailboxes are integrated mailboxes which provide integrated e-mail, facsimile and/or voicemail services which allows the student user to receive and transmit information by facsimile and/or voicemail in addition to by email. Communication via facsimile may be advantageous, for example, when the information being communicated relates to mathematics such as geometry. Likewise, communications by voicemail may be advantageous when an oral explanation to a problem is more desirable than a written explanation.

CPU 201 preferably comprises a conventional high-speed processor capable of executing program instructions to perform the functions described herein. Although server 103 is described as being implemented with a single CPU 201, in alternative embodiments, server 103 could be implemented with a plurality of processors operating in parallel or in series.

Data storage device 210 preferably comprises static memory capable of storing large volumes of data, such as one or more floppy disks, hard disks, CDs, or magnetic tapes.

Network interface 205 connects CPU 201 to network 103. Network interface 205 receives data streams from CPU 201 and network 103 formatted according to respective communication protocols. Network interface 205 reformats the data streams appropriately and relays the data streams to network 103 and CPU 201, respectively. Network interface 205 preferably accommodates several different communication protocols.

Cryptographic processor 202 is programmed to encrypt, decrypt and authenticate the stored data in one or more of the databases described above. The student user's name, for example, may be encrypted in the student profile database 213 so that nonstudent users can review the information in the database without learning the student user's identification. RAM 203 and ROM 204 preferably comprise standard commercially-available integrated circuit chips.

Figure 3 depicts a block diagram of student user terminal 101 A of Figure 1. Student user terminal 101 A provides the student user with an interface to system 100. Preferably, student user terminal 101 A allows a student user to enter data and transmit the data to server 103 via network 102 and also receive data from server 103. Student user terminal 101 A preferably includes CPU 301, which is connected to ROM 302, RAM 303, video driver 304, cryptographic processor 305, communication port 306, input device 307, and data storage device 308. Input device 307 may include a keyboard, touchpad, mouse, any other known input device or combinations thereof. Video monitor 310 is connected to video driver 304 and modem 311 is connected to communication port 306 and network 102.

CPU 301 executes program instructions stored in RAM 303, ROM 302, and data storage device 308 to carry out various functions associated with student user terminal 101 A. In a preferred embodiment, CPU 301 is programmed to receive data from input device 307, receive data from communication port 306, output queries and received data to video driver 304 for display on video monitor 310, and output data to communication port 306 for transmission by modem 311 to network 102. CPU 301 preferably transmits the data to cryptographic processor 305 for encryption before outputting data to communication port 306 for transmission to network 102. When CPU 301 receives encrypted data, CPU 301 transmits the encrypted data to cryptographic processor 305 for decryption.

CPU 301 preferably comprises a high-speed processor capable of performing the functions described herein. Although student user terminal 101 A is described as being implemented with a single CPU 301, in alternative embodiments, student user server 101 A could be implemented with a plurality of processors operating in parallel or in series. Video driver 304 relays received video and text data from CPU 301 to video monitor 310 for display.

Data storage device 308 preferably comprises static memory capable of storing large volumes of data, such as one or more floppy disks, hard disks, CDs, or magnetic tapes.

Communication port 306 relays data between CPU 301 and modem or network card 311 in accordance with conventional techniques.

Cryptographic processor 305 is programmed to encrypt and decrypt data in accordance with conventional encryption/decryption techniques and is preferably capable of decrypting code encrypted by cryptographic processor 202. The student user's name, for example, may be encrypted in the student profile database 213 so that nonstudent users can review the information in the database without learning the student user's identification.

Modem 311 preferably comprises a high-speed data transmitter and receiver. Input device 307 preferably comprises any data entry device for allowing a user to enter data, such as a keyboard, a mouse, a video camera, a touch or writing pad, key pad or microphone.

RAM 303 and ROM 302 preferably comprise standard commercially-available integrated circuit chips.

There are a number of ways a student user or nonstudent user may input information into the system and/or make a selection using the student user terminal 101 A including the use of selected strokes on a keyboard, the clicking of a mouse after moving the pointer to a particular icon or graphic or the like on the video screen, and/or by touching the video screen, touch pad or the like. Such methods and devices for allowing a user to input information or make selections into a computer-based system are well known in the art.

Using this invention, remotely-located student users at a number of student user terminals 101 A, 10 IB and 101C can simultaneously use the invention and compare their results with other remotely located student users. The test preparation can be done on an ongoing basis with no need for an on-line connection where the student user downloads the information. Alternatively, an on-line connection (e.g., Internet, World Wide Web, American Online, CompuServe, Prodigy, etc.) can be used for even greater flexibility in test question distribution and control.

Whether on-line or off-line, the students user's scores and other personnel information are securely associated with the student user's identity (such as by using a secured personal identification number, See U.S. Patent No. 5,971,272), which in turn is associated with the student user's profile. Preferably, the system can provide the student user with the ability to determine how the student user performed compared with other student users generally or other student users having certain profiles, without learning the other student user's identities. According to one embodiment, the student user can determine how he compared with other student users of the same age, from the same school district, of the same geographic region and/or of the same ethnic background. The transmitted test results and other related information are configured to reliably associate the student user with his test results, using encoding, user identification, or coπoborative techniques to deter confusion, eπors, fraud and insure confidentiality.

Thus, the student user's identity is preferably confidential. One form of advantageous anonymity useful in the invention involves "shielded identity", where a trusted agent (e.g., the operator of the server 103) knows the identity of the masked party (e.g., student user), but does not reveal that identity to others except under certain circumstances. U.S. Patent No. 5,884,272.

Figure 9 illustrates a web page display according to one embodiment of the invention which prompts the student user to input personal information. The personal information such as name, address, E-mail address and phone number which is input on this page is preferably kept confidential. Thus, the server's network interface is preferably able to support multiple simultaneous data connections with multiple users. In a preferred embodiment, a server or central computer is accessible over the Internet or commercial on-line service such as America Online, CompuServe, or Prodigy, allowing multiple test-takers to access the central computer via simultaneous on-line connections. In another embodiment of the invention, the test questions can be downloaded directly from the central computer to the external terminals.

According to one preferred embodiment of the invention, each individual student user has a personal web page which includes test preparation and evaluation tools. Preferably, the tools are displayed in the form of text, icons and/or other suitable graphics. Graphics user interface operating systems, such as Microsoft Corporation's Windows™ and any related products, have greatly improved the ease with which tasks can be accomplished on a computer. Instead of requiring the user to remember the file name and path of an application program in order to execute it, as conventionally is the case in text-based operating systems, in a graphic operating system, the user need only activate an icon that represents the application. The graphic icon is preferably activated by positioning a cursor over it and then "double clicking" on the icon with a pointing device that is used to control the cursor. Since the properties assigned to the graphic icon are linked to the executable file and specify its complete directory path, there is no need for the user to recall that information after the icon properties are initially set up to run the program. See, PCT patent publication No. WO 96/39654.

Pull-down menus or keystroke-mouse combinations may also be provided to allow the student user to view additional options. Additionally, interface objects are preferably grouped according to functionality. For instance, objects relating to test preparation including practice exams, study plan and study basket may be grouped together and may also be kept separate from objects concerned with personal profile, statistics module, the college/university applications process, etc.

Figure 10 illustrates a web page display according to one embodiment of the invention which includes icons and/or text representing the various features provided by the invention. Figure 11 illustrates a web page according to another embodiment. Figure 12 illustrates a web page for a particular user for modifying or reviewing account or profile information.

According to one preferred embodiment of the invention, the student user is provided with an options format in the form of user settings. For example, the student user can preferably select which language the information is conveyed (e.g., English, French, etc). That is, the system is preferably multi-lingual. As another example, the student user may select preferences with respect to learning techniques (e.g., more practice exams in place of practice questions or detailed answers versus summary answers). The user settings may also provide the option of study reminders, how many times a word or question is answered correctly before a topic is removed from study basket, personal web page format/content/layout, etc. A default format is also preferably provided should the student user not employ the user settings.

Thus, one aspect of the invention relates to a method and system which advantageously provides one or more of the following computer-based features to the student user (each of which is discussed in further detail below): (1) a personalized study plan, (2) a personalized list or database of study materials or study action items (herein called "study basket"), (3) a personalized test taking analysis and evaluation, (4) additional study materials such as practice exams, practice questions, test taking tips, SAT dictionary, Roots & Stems, vocabulary builder, problem builder and personal study notes, (5) a test taking pace analysis, (6) nervousness indicator, (7) statistics module to assist the student user in evaluating their performance compared with other student users, and assist the author or developer of the learning environment or experience to evaluate the effectiveness of the method or system (e.g., effectiveness of study materials, etc.), (8) a rule-based system that authors, developers, creators or administrators of the study material to create or provide personalized feedback for student users, (9) allow collaboration by sharing study aids among users and allowing for the searching of such shared study aids by the users, (10) enables the systematic and automatic development of an "expert community" (e.g., users who perform above a specified level during the learning process and thus identified as "experts" capable of assisting other users), (1 1) creation and development of an expert center where individual users can submit questions and obtain answers, (12) a personalized university center to organize and assist the individual in the preparation for the test and any application process which utilizes the test score, (13) personalized "college counselor" (e.g., list of colleges or university offering specified majors or evaluations of probability of acceptance to specified colleges or universities, etc.) or combinations thereof. Each of the functions is designed to organize and assist the student user in the preparation for a test such as a standardized test and/or for any application process which utilizes the standardized test score such as applying to college or university.

Figure 4 depicts a block step diagram according to one preferred embodiment of the invention illustrating how the different features may interact. More specifically, the study plan is the central feature of this embodiment, where information from features such as the practice exams, practice problems, vocabulary builder and math builder & test tips are fed directly into the study plan and/or indirectly to the study plan via the study basket. For example, a student user may engage in test preparation by taking a practice exam. The results of the practice exam may produce an evaluation of strengths and weaknesses on a topic by topic basis. Weak topics may then be transferred to the study basket which automatically generates appropriate additional study materials. The additional study materials are then added to and incorporated into an updated study plan. Alternatively, the weak topics may be directly transferred to the study plan which may automatically generate and distribute the additional study materials for the student user.

Figure 5 depicts a block step diagram for another embodiment of the invention relating to the incorporation of study materials such as study topics, user created problems and/or personal study notes ("My Notes"). More specifically, after the student user selects and reads a particular topic, such as geometry, the student user may either (a) choose to add the topic to the study basket and/or update the study plan, (b) create a personal study note regarding the topic, (c) select or create problems, solve the problems and review the answers and or (d) request personal assistance, such as by e-mail, fax and/or voice-mail. These options will be described more fully below.

Other permutations of the invention would be readily recognized by one of ordinary skill in the art. More specifically, the different features of the invention, which may be used in any combination, are discussed in more detail below.

1. Personalized Study Plan

One embodiment of the invention relates to a method and system which includes a customized, personalized study plan. The improved study plan is achieved by using information generated by the system and/or by collecting certain additional information from the student user and constructing an efficient study plan based on this information. Unlike previous methods and systems, the invention utilizes the ability of computers to collect, store and retrieve a vast variety and amount of information to provide an advantageously customized study plan.

Such additional information may include: (a) the date the student user intends to begin preparing for the test and the actual date of the test, (b) the amount of time the student can study each day (e.g., four hours a day on weekdays/eight hours a day on weekends or four hours a day the first half of the test preparation period, six hours a day the second half, etc.), (c) any target goals the student user may have, (d) choice of study technique (e.g., practice questions versus practice exams, etc.), (e) previous test scores, etc.

The customized study plan can be initially generated by asking the student user some key pieces of information such as the day the user wants to start test preparation, the date the user wants to be done preparing for the examination, the number of times the student user wants to study per week, the days of the week the student user prefers to study and so on. Figure 13 illustrates a display prompting a student user to answer such questions. Based on the answers to such questions, the number of days the preparation that are available to the student user can be determined. The amount of material the student user needs to study is calculated and then distributed over the days/weeks available based on a predefined criteria which would be different for different exams that the user may be preparing for.

According to one embodiment, the study plan incorporates test preparation for more than one test, e.g. a student user may be planning to take the LSAT, GMAT and MCAT over four month period or the student user may have five examinations to study for over a three week period.

One advantage of an individualized study plan is the ability to take into account not only the individual's current test taking strengths and weaknesses, but also any other factors the individual may need to consider when preparing for an exam. For example, an individual who has two months to study for an exam and can accommodate six hours of studying a day should have a different study plan then an individual who has one month to study and can only accommodate four hours of studying a day. Accordingly, one embodiment of the invention allows the student user to input time parameters such as the date of the actual test, which days the student can study and/or how many hours the student can study per day, etc. One preferred embodiment of the invention provides a "study plan calendar" for the student user which sets for a detailed study plan including the dates and times certain topics should be studied, practice exams or questions taken, etc. Preferably, the study plan calendar can be viewed by day, by week and/or by month.

According to one preferred embodiment, the study plan is dynamic in that it can be updated automatically and/or by the student user. That is, the invention provides a dynamic individualized study plan that can be updated for the student user based on a number of factors including practice test or question performance, study materials covered, pace of progress, change of circumstances (e.g. student user has more or less time to study), varying pace of learning depending on topic, etc. For example, the method or system may automatically revise the study plan by adding supplemental study topics if the practice tests indicate that the student user needs additional instruction regarding that topic. Alternatively, the study plan may be automatically revised by removing certain items from the calendar if the student user's test results show that such instructional items are no longer necessary in view of improvements in test performance in those areas.

The "automatic" adjustment to the study plan can either be performed with or without the student user's prior consent. Preferably, any "automatic" adjustments to the study plan are first suggested to the student user who can then either agree to the suggestion or disagree by using the appropriate input action.

For example, after the initial customized study plan has been generated, there may be two sources of input that can alter the customized study plan. The first is input from the student user and the second is input generated by the system based on the user's performance on various areas. In the case of feedback from the system software itself, the system may evaluate the student user's performance and find topics which the student is having trouble with and add these topics to multiple days so that the user releams the material consistently to help himself improve upon those areas. The criteria as to where to go about making the additional entries and so on depends on the material under consideration and the test under consideration.

The dynamic study plan can preferably also be adjusted by the student user. For example, the student may adjust the study plan by increasing or decreasing the time allocated for studying and/or the amount of emphasis given to any particular topic. As another example, the student user may decide that he can spend eight hours, rather than four on a particular day and may adjust the study plan accordingly. As yet another example, if the student user should fall behind in the action items in the study plan, the plan should be adjusted accordingly.

As another example of a factor which may be included in the study plan is a target score the student user seeks to achieve on the test. A student user may desire a particular score on the SAT for example. Using information including the student's scores on prior tests, the study plan is formulated to help the student user achieve the target score. As another example, one individual may be planning to take the SAT and may want to attend a technical undergraduate institution instead of a liberal arts college and may wish to perform better on the math portions of the SAT compared to the verbal portions. This aspect of the invention would allow the individual to emphasize the math preparation rather then the verbal preparation. According to one embodiment, the user may input a desired target score for a standardized test and the individualized study plan produced takes the target test score into account when formulating the individualized study plan. Preferably, the student user can change the target test score more than once.

Thus, this aspect of the invention provides an individualized study plan, preferably a dynamic individualized study plan, for the individual user. Preferably, the study plan is designated on a student user's personal web page (discussed further below) as text, an icon or any other suitable graphic. The student user merely has to select the study plan to view its contents (e.g., calendar, etc.) and/or make any changes.

According to one preferred embodiment of the invention, the method and system are web-based and are designed to assist a student user prepare for an examination and the structure of the study plan is broken down into four different study phases. Phase 1 - Preliminary Study Phase Phase 2 - Intermediate Study Phase Phase 3 - Advanced Study Phase Phase 4 - Final Preparation Phase

Phase 1 : In this phase the method and system guides the student user through at least one diagnostic or practice test, half of the practice problems, all of the math builder web pages, all of the test tips web-pages, and at least one cycle through all of the vocabulary words. The length of this phase is approximately 1/5 of the entire study period minus 30 days.

Phase 2: Information from Phase 1 is used to determine what the student user's weaknesses are and in which areas the individual needs the most help in. This phase of the study plan is customized based on this analysis. The purpose of this phase is to make the student go through all of the necessary material and gain a solid understanding of the subject matter which will be tested on the examination as well as the associated subject matter. The length of this phase is approximately 3/5 of the entire study period minus 30 days.

Phase 3: By this phase, the student user should feel well prepared for the examination. During this phase, the intensity of the teaching is increased and any remaining weaknesses on the part of the student user are ironed out. If the student is weak in more areas than the study plan provides time to cover, then the method and system automatically covers only the most important topics (e.g., topics tested most heavily on the given test) and drops the rest unless the student shows faster then expected progress in which case other items may be covered as well. The length of this phase is approximately 1/5 of the entire study period minus 30 days.

Phase 4: At this point the teaching of subject matter is stopped and the student is made to take about one exam every 3 days. The student user takes the exam on one day and then uses the other two days to go through the entire exam and studying any mistakes that were made and the reasons the mistakes were made and studying any materials that were added to the study basket by the software, etc. The length of this final phase is exactly 30 days.

The length of each one of the phases can be changed by the student if it is so desired. 2. Performance Tracker

Another embodiment of the invention relates to a method and system which analyzes the performance of the student user during the test preparation. The analysis can include the students performance on practice exams, practice questions or practice problems and the like.

According to one preferred embodiment, the method or system includes tracking the performance of the student user during practice tests, practice questions, etc. Preferably, each question is associated with a particular topic and/or subtopic. For example, a question may involve both geometry and algebra and will thus be associated with geometry and algebra. If the user gets the question incorrect, the system will automatically add these topics to the study plan and/or study basket or suggest that such an addition be made. Alternatively, if the student user gets these types of questions correct, the method or system may remove or reduce these topics in the study plan and/or study basket. Preferably, any addition or removal is first suggested to the student user.

Preferably, the method and system determines the student user's strength in various topics independent of the score weights given to questions. Score weights on individual questions relate to the difficulty of the question, and do not present an accurate determination of the actual understanding of the subject material by the student. Such determination allows further customization of the student user's study plan since a more accurate assessment of the user's strengths and weaknesses is provided.

For example, one embodiment of the invention defines and uses a new metric herein referred to as the "Topic Proficiency" of the student user for each topic and subtopic, to determine the student user's understanding of each topic. The Topic Proficiency for any topic may be computed as follows:

Number of questions related to topic answered correctly

Topic Proficiency = * 100%

Number of questions related to topic that are presented

When a topic has subtopics (and when a subtopic has sub-subtopics), both the numerator and denominator in the above equation include all questions related to that topic as well as all subtopics in the entire sub-tree below that topic. For example, if topic A has subtopics B and C, and subtopic C has subtopics D and E, the Topic Proficiency of A includes questions related to topics A, B, C, D and E. As another example, if a course has topics P and Q and the test has questions 1, 2, 3 and 4 and Question 1 and 2 are associated with topic P, and questions 2, 3 and 4 are associated with topic Q. If a user has answered questions 1 , 2, 3 correctly and question 4 incorrectly, then

Proficiency in topic P - ( 2 * 100 ) / 2 = 100 %

Proficiency in topic Q = ( 2 * 100 ) / 3 = 66 %

The above metric can be generalized as a function that weights different questions differently indicating their relative importance of determining proficiency in a topic. The ability to more accurately determine a user's strengths and weaknesses regarding a particular topic or subtopic allows for an improved personalized learning experience. Preferably, the author or developer of the learning experience (e.g., the learning experience created using the inventive

methods and systems) and/or the user can specify a desired proficiency for each topic in a course. A topic where a user has attained the specified proficiency can be removed.

Preferably, a desired proficiency (e.g., "target" proficiency) may be specified for each topic. According to one embodiment, a topic where a user has attained the desired proficiency in a subject area such as topic P, the study materials relating to topic P may be removed from the study plan.

Furthermore, based on whether topic questions are presented and answered prior to certain study materials relating to the topic they relate to, or after, Topic Proficiency is preferably further classified into Pre- Assessment Topic Proficiency and Post- Assessment Topic Proficiency.

Pre- Assessment Topic Proficiency is the ratio of the number of questions related to the topic that were presented and answered correctly before presenting the study materials relating to the topic, to the total number of questions related to the topic study materials that were presented before presenting the topic study materials, expressed as a percentage.

Post- Assessment Topic Proficiency is the ratio of the number of questions related to the topic that were presented and answered coπectly after presenting the topic study materials, to the total number of questions related to the topic that were presented after presenting the topic study Materials, expressed as a percentage.

By computing the difference between Post-Assessment Topic Proficiency and Pre- Assessment Topic Proficiency for a topic, the invention determines a user's Proficiency improvement in a topic, which is a numeric value of the improvement in a user's understanding of a topic and/or a numerical value indicating the effectiveness of the study materials relating to the topic. Another embodiment relates to a computer-aided method for assisting at least one student user leam subject matter using sample test results generated by the student user taking a sample test and student user information from the student user comprising:

(a) receiving the student user information from the student user;

(b) analyzing the sample test results generated by the student user; and

(c) providing one or more of a performance evaluation, study plan or study basket based on the sample test results and the student user information.

Another embodiment relates to a computer-based learning method for teaching a user one or more subjects comprising the steps of:

(a) providing a series of questions to the user on a display;

(b) retrieving user answers from the user for the series of questions; and

(c) evaluating the answers by comparing the user answers with coπect answers thereby determining the strengths and weaknesses of the user,

wherein the method further comprises the step of retrieving user information from the user sufficient to form a user profile.

Yet another embodiment relates to a computer-aided learning method for assisting a student user leam subject matter, the method comprising:

(a) generating a sample test comprising a multiplicity of questions relating to one or more subjects from a question database, the multiplicity of questions being associated with a predetermined coπect answer;

(b) transmitting the sample test to the student user;

(c) receiving student user answers for the multiplicity of questions from the student user; (d) evaluating the student user answers compared with the predetermined coπect answers thereby determining which questions the student user answered coπectly and which questions the student user answered incoπectly thereby providing test results; and

(e) assisting the student user by providing the student user with the test results and an evaluation of the test results, wherein the method further comprises the step of receiving student user information from the student user and the step of assisting the student user comprises utilizing the student user information.

A still further embodiment of the invention relates to a computer-aided learning method for assisting a student user prepare for a standardized test, the method comprising:

(a) transmitting a sample test comprising a multiplicity of questions relating to one or more standardized test subjects from a question database, each of the multiplicity of questions being associated with a predetermined coπect answer;

(b) receiving student user answers for the multiplicity of questions from the student user;

(c) evaluating the student user answers compared with the predetermined coπect answers thereby determining which questions the student user answered coπectly and which questions the student user answered incoπectly thereby providing test results; and

(d) providing the student user with the test results and an evaluation of the test results; wherein the method further comprises the step of receiving student user information from the student user including at least two or more of the following: (i) student user identification selected from the group consisting of name, social security number, credit or debit card number, telephone number or other identification, (ii) date of standardized test or how many days until the standardized test, (iii) time the student user can allocate for preparing for the standardized test, (iv) student user standardized test score target goal, and (v) previous standardized test scores, if any.

The invention evaluates the student user's performance and displays the analysis to the user. Preferably, the tracker generates a list of all the weaknesses in descending order so that the student user can readily determine which is his greatest weaknesses. For example, the performance tracker may display summaries of the scores received in each of the practice exams so the user can review improvements in performance.

According to one prefened embodiment, the performance tracker also provides summaries of these scores received on the exams compared with other student users taking the similar exam.

Preferably, the performance tracker enables the user to review the details of any of the exams, for example, by clicking on an icon or other graphic representing the exam. Figure 14 depicts a display with an icon for each of Exams 1 through 30. A selected exam, Exam 1, is highlighted to illustrate a possible selection by a student user. Preferably, there is either a separate web page for exams which have already been taken by the student user and a page for the exams which have not been taken. Alternatively, both exams may be on the same web-page, but differentiated from one another by shading, marking or the like.

If the selected practice exam has not been taken, the student user is given an opportunity to take the exam by making an appropriate selection. Figure 15 depicts a typical display shown to the student user after selecting the take the exam according to one embodiment of the invention. According to one embodiment, the student user is provided with a test score after each test section. Figure 16 depicts a display for such a section score report according to one embodiment of the invention. Figure 17 depicts a display after the exam is completed according

to one embodiment of the invention.

After the student user completes an exam, he is preferably given an opportunity to review his performance results. Moreover, when the particular exam which has already been taken by the student user is opened or selected by the user, a detailed report is also preferably available and displayed including how many answers in each category of the questions were missed, answered correctly or unanswered. Figure 18 depicts a typical display of such a report on a section by section basis. Figure 19 depicts a display for a performance analysis page for a student user according to one embodiment of the invention which provides a variety of options to the student user. Preferably, the user is also allowed to pull up individualized questions for review. Analysis by test section may also be given (e.g., the SAT has six sections). The detailed report may also provide a list of the main weaknesses of the user and/or how much time the student took to answer each question.

Thus, the system may also include a weakness analysis tool which provides detailed analysis of the user's weaknesses. The weakness analysis tool may list weaknesses in descending order by category. It may also mention how many questions and which tests each of the concepts were missed, answered coπectly or unanswered. The function of the weakness analysis tool is to help point out to the user how he or she can approach solving certain problems. The weakness analysis tool may also update and/or customize the test type taking tips, the study basket and/or the study plan.

The unique part of the performance tracker is that it not only performs evaluation of the students weaknesses but it also preferably allows the user to know how many times they have seen a certain question while taking the exam, the number of times they changed the answer to the question, and finally the amount of time they spent on each question. This analysis helps the student know if they are wasting too much time on a question or a certain type of question and/or

part of a question, whether they are missing many of the questions they are uncertain about (e.g., student is guessing on certain types of questions and getting each one incoπect), and whether they are being inefficient test takers by returning to a question too many times. The performance analysis may also include a test taking pace analysis (discussed further below).

Moreover, the student user can evaluate how they are doing in comparison to the average of all the other students. Figures 20-22 depict typical displays, using different formats, providing an evaluation of an individual student user's performance compared with all student users according to one embodiment of the invention.

All of the analysis in this feature may be performed by question number, question category (multiple choice, Grid Ins, etc), and or question type (exact topic(s) being tested). It can also be based on broad subject areas such as math vs. verbal or subtopics such as geometry, algebra, etc.

3. Study Basket

Another aspect of the invention provides an individualized study basket. The study basket is a database or memory or the like which essentially holds study topics which the student user can review at a future time.

Preferably, the study basket is designated on a student user's personal page as an appropriate icon. Figures 10 and 11 depict web-based displays which illustrate a personal page containing a number of icons and text including a graphic icon for a study basket. Preferably, if the student user selects the study basket icon, a display or list is provided which includes the study materials the student user is assigned such as practice exams, practice questions, etc.

Study materials may be placed into the study basket in a number of ways. First, the student user may choose to add a study topic to the study basket for future review. For example, the student user may lack confidence in one area such as geometry even though the student scores well in this area. Thus, the system may allow the student user to add additional instruction in geometry.

Second, the system itself may automatically determine which topics the student user is weak in and add them to the study basket, for example, after the user takes a practice exam or practice question (See Figure 4). Preferably, each question may be associated with one or more topics. Thus, if the user gets a question wrong, each of the topics associated with answering the particular question would be put into the study basket.

For example, the various types of study materials can be labeled or designated as relating to particular topics or subtopics. According to one embodiment, the system includes a database which contains three tables. The first table contains the questions and coπesponding answers. The second table includes a list of all the possible topics which may be tested on the particular exam and an associated LD. for each of these topics. The LD. could be numerical and/or text in nature. A third table would contain information associating each question with one or more of the topic ID's in the second table. If multiple topics are covered by the question then there will be two or more ID's associated with this question number in the third table. Thus, each entry will make a link to at least one topic. Once the three tables are constructed, the system software determines which question number is being tested and whether the student user has answered the question coπectly using the first table. Then, if the student user answers the question incoπectly, the system immediately determines the topics which are associated with the question using the third table and finally using the second table to determine the title of the topic itself. Once this is done, the system generates a list of topic titles which may be reviewed by the user. These topic titles can then be transfeπed to the study basket for future review by the user. These topic titles, in turn, may be associated with a separate database(s) which contains relevant study packets containing detailed or summary reviews of the topic(s), practice questions or problems relating to

the topic(s), etc.

The system may include a database which contains a table that keeps track of the contents of each user's study basket. Preferably, the study basket is designated on the student user's personal page as an appropriate icon. The student user merely has to select the study basket to view its contents and/or make any changes.

4. Personalized Study Materials: Practice Exams,

Study Packets, Test Taking Tips and Questions and Problems

According to another embodiment of the invention, the method and system also provides additional study materials including: (a) study packets, (b) practice exams, (c) practice questions/problems, (d) test taking tips, (e) vocabulary builder, (f) smart problems, (g) problem builder, (h) personal study notes (e.g., "My Notes"), etc. The additional study materials may be added to the study basket and/or study plan. The additional study materials may also be separately designated on the student user's personal page as an appropriate icon or the like. See Figures 10 and 11.

The study materials may provide information, such as answers to questions, in detail or summary format. Preferably, the student user is able to select which format is displayed by adjusting the appropriate user settings.

Preferably, the additional study materials contain information associated with one or more study topics. Thus, when the performance tracker determines that more or less instruction is necessary for any given topic, the appropriate action is taken by either adding or removing the additional study material which is associated with the topic from the study plan, study basket, etc. The study packets may include brief or detailed descriptions of subject matter including text, graphics, audio, video, etc. Figure 23 depicts a display containing study materials relating to fractions and decimals. Figure 24 depicts a display containing study materials relating to properties of integers. The study packets are contained in a database and are preferably categorized by topic. Thus, when a student user's study plan or study basket is being updated to include additional study materials relating to algebra, for example, suitable study packets are retrieved and included in the study basket or study plan.

The practice exams may also be stored in a database and are preferably as close as possible to the actual exam the student user is preparing for. Even more preferably, the practice exams are "real exams". Figure 6 illustrates the steps involved in a user taking practice exams and how this activity can dynamically update the study plan and/or study basket.

The practice questions (or problems), like the practice exams, are also stored in a database and are preferably as "realistic" as possible. Like the study packets, the practice questions are preferably categorized by topic. Thus, when a student user requires additional instruction regarding geometry, for example, suitable geometry questions are retrieved and included in the study basket and/or study plan. Figure 7 illustrates an embodiment using practice question during test preparation.

Other types of study materials may include test taking tips, preferably customized test taking tips. General tips may include strategies for taking a particular exam (e.g., better to skip a question rather than guess, etc.). The materials may also include a SAT dictionary or Roots & Stems, etc. The invention may also provide customized tips. Figure 25 depicts a display containing test taking tips according to one embodiment of the invention. An example shall be used to explain how this feature could be implemented. If a student user misses a lot of geometry questions relating to triangles while practicing problems or while taking a practice exam, the system software will notice this, for instance, by using a table in the database to keep track of how many questions of each kind that the user has answered incorrectly with regards to practice exams and also how many with regards to practice problems. In addition to this, in the case of practice problems, it would be helpful to keep track of how many consecutive questions of each kind that the user has missed. This latter piece of data would be very helpful in determining the level of trouble the student is having. The system can set a criteria internal to the software which requires that when the number of consecutively missed questions of a certain type of problem is three or more, then it is added to the personal test taking tips table of the database. Information about this will also be added the study plan and/or study basket as well so that those features could perform their updated functions as well. One could simply make the entry with the usemame and the ID of the topic the student is having trouble with. Then, when the user enters the customized test taking tips area, the software simply refers to the test taking tips table and determine if there are any entries under the present user's name. If one or more are found, advice associated with such a topic is displayed for the user to read and lea from.

Another embodiment of the invention provides a smart vocabulary builder. This tool helps the user improve vocabulary skills. Preferably, the smart vocabulary builder presents a list of words that the user can click on and spend time learning each one. It is in essence a flash card technique being employed in the software. For example, once the user has completed learning the words, he clicks on an icon which begins a quiz on those words. Preferably, there is a definition and at least one question that involves each of the words in the vocabulary list. The software keeps careful track of the user's answers to each question and performs analysis on any missed questions and the possible reasons as to why they were missed by the user. It further also makes the determination as to which words the user knows and which words he does not know. Those words which are known are marked as such and those which are not known are marked as unknown. The student user is preferably allowed to change the status of any of the words from known to unknown and vice-versa. Upon the completion of the quiz, the user can start learning a new set of words or end the vocabulary building session.

Figure 8 illustrates a flow diagram for a vocabulary builder according to one embodiment of the invention.

As an example of a prefened embodiment, one can create a database with two tables, one or which has the words, the word types, the definitions, and at least one sample sentence using the word. The second table can contain the word and the question and answer associated with this word for the vocabulary quiz part of the feature. If one has multiple questions for the same word, then these should be made as different entries. Having multiple questions per word is useful so that the questions can be chosen at random when a certain word is tested rather than always asking the same exact question for any given word.

According to one embodiment, the vocabulary builder first displays a small set of words. The user is then allowed to study these words one word at a time. Upon the completion of the studying, the user can then move onto taking the quiz. At this point, the table with the questions is referred to and the questions that are associated with each of the words that the user has just studied are transmitted to the user. Of course, only one question needs to be asked per word. Each word can be graded and the result can be reported to the user immediately so he can leam from any mistakes. At the end of the quiz all those words which were answered coπectly can be marked as known and the others unknown. Preferably, the user can have the option of changing the status of any of the words from known to unknown or vice-versa. This allows the user to give his personal opinion on his understanding of the words thus acting as a check upon the system software. This criteria is preferably only applied when the user is going through a list of words for the first time. After all the words have been traversed once, the system may cycle through the words again but preferably only going through those words which are still marked as unknown. From the second cycle onwards, a word is preferably not marked as known unless the user gets the answer coπectly on three consecutive quiz questions on the same word. Thus it will take at least three more cycles through these words to eliminate any one of them. This criteria can preferably be adjusted by the user using the option settings.

Another embodiment of the invention provides a feature that allows a student user to input a particular word to receive a definition and sample sentence using the word. Figure 26 depicts a display for such a feature according to one embodiment of the invention.

Yet another embodiment provides "smart practice problems". The word "Smart" comes into play here because these problems not only give practice to the student user in various areas of study, but also helps the software decide the student's weaknesses and at some level determine possible problem solving approaches the user is taking which may be incoπect. Furthermore, this feature preferably provides that those problems which the users answers coπectly are marked as problems the user can solve and the rest are marked as missed so that they can be reviewed again in the future. Figure 27 depicts a display which provides a student user with an option of either mathematics or verbal practice problems according to one embodiment of the invention. Figure 28 depicts a display providing a student user with different options according to another embodiment of the invention.

The implementation of this feature is quite similar to the vocabulary builder. There is a table in the database which contains the problems to be given to the user. The problems are categorized by type. Thus geometry problems are of one type, algebra of another, verbal of another, and so on. If a question posed to the user is answered coπectly, the problem is marked as known so that the user does not have to worry about these problems in the future since they already know how to deal with them. Those which are answered incoπectly are reviewed again at a future date. The frequency with which problems of a particular category are being missed preferably causes updates to take place in the study plan and/or study basket as well. Problems with which the user is having a lot of trouble with are automatically emphasized more in the study plan and/or study basket.

Another embodiment of the invention allows the student user to construct problems and then try to determine the answer which the system determines whether it is coπect or incoπect. The uniqueness here is that of having a section where the student user can design and solve his own problems. For example, if the student user is reading a geometry topic involving determining the area of a triangle, the student user can simply click on a button on this page which will then provide a picture of a triangle. The student user can then set up the problem by typing in the dimensions of the triangle and then trying to solve it and enter the answer into a text box. The system software can then calculate what the answer should be and inform the student if he did it coπectly. If the problem is missed, then a step by step explanation may be given as to how to go about doing the problem coπectly.

Yet another embodiment of the invention includes a "personal notes" or "my notes" feature. For example, on many of the web pages generated during the method such as the math builder and test taking tips pages, the student user is preferably given the functionality to add personal notes to any page that they wish. This is analogous to adding notes on different pages of a book. The note may include, for example, a personal explanation for solving a problem or for an answer to a problem in the student user's own words. Thus every time the student user revisits the page after making a note, he will see his note at the bottom of the screen (or any other position on the page) in a "personal notes" section of the page. Figure 29 depicts a display for a personal note according to one embodiment of the invention. The user can also preferably come to the personal notes section in order to manage these notes as well as download a personalized copy of their notes in any suitable text and/or have the operator of the server print out the notes into a book format and send it out to them.

The personal notes feature also preferably includes various study aids such as allowing for highlights or markup of content, where a user can select text to be highlighted, which then appears in a highlighted color. Other study aids include pointers to reference material including Internet web pages or Uniform Resource Locators (URLs), audio and video clips, and bookmarks. A bookmark allows a user to mark a study material for quick retrieval later on. Every time a user revisits a page, all the study aids created by the user are preferably available for review by that user.

Another prefened embodiment of the invention relates to allowing sharing of personal study materials between student users to facilitate collaboration among peer users. Personal study materials that can be shared may include (a) personal notes, (b) highlights, (c) pointers to related reference materials such as URLs, (d) topics and questions of interest relating to the subject materials, etc. This allows a student user that is having difficulty with a certain topic to refer to notes from another student user. The users can also interact with each other through notes and markups in the topic content to further enrich their learning experience.

Another prefened embodiment relates to the ability of a student user to filter personal study materials that are shared with other selected student users. This allows student users to restrict the information they view as they progress through their course.

Yet another prefened embodiment allows student users to search through all of the personal study materials they have access to including their own and/or the ones shared with them by their peers. The search feature allows searching the contents of personal notes, the names of URL links, and the content of highlighted text. 5. Pace Tracker

Another embodiment of the invention relates to a test pace tracker. The pace tracker allows the student to know if they are moving through the exam at a fast enough pace to finish the exam or if the student is taking the test too fast. This helps them to adjust test taking habits to the right pace thus making them better test takers.

The pace tracker may not only provide information to the student user after the test, but preferably also allows the student user to review his pace throughout the test. It may be beneficial for a student user to know, for example, that he is taking too long on a particular type of question, etc. At the same time, the test developer and evaluators would be interested in this information as well.

Thus, one embodiment further comprises providing the student user with a test taking pace. The pace analysis may preferably be provided on a test section by test section basis and/or on a question by question basis. Preferably, the test taking pace includes a pace analysis based on topic. Advantageously, the student user is provided with an indication during the sample test of whether the test taking pace is slower or faster than a predetermined pace.

According to one prefened embodiment of the invention, the pace tracker is a small bar in the bottom left hand comer of the screen. It has two states. In one state, it maybe displayed as a green colored bar with the note "Just Right" written on top of it and in the other state it may be seen as a red colored bar with the note "Too Slow" written on it. The green bar is shown when the student is moving along at a fast enough pace to complete the exam and the red bar is shown when the user is moving too slowly. A third state could also be introduced which may be a blue bar with the writing "Too Fast" on top to let the user know that they are moving along too fast and thus are notably prone to making mistakes. One of ordinary skill in the art would readily recognize that the pace tracker may be displayed to the student user in a number of ways. The pace tracker not only provides information to the student user during the test, but preferably also allows the student user to review his pace after the test. That is, it may be advantageous for a student user to know, for example, that he is taking too much time on a particular type of question, etc. At the same time, nonstudent users such as test developers and test evaluators would be interested in this information as well. For example, nonstudent users may be interested in determining whether a particular type of question takes too long for students to answer because of poor design, poor wording, improper length, or the like.

6. Nervousness Indicator

Another aspect of the invention includes providing a nervousness indicator for the student user. It is known that being nervous during a test or exam can greatly effect a student's performance. Thus, the ability to determine if the student user's performance may be effected by nervousness would be helpful. For example, if it is determined that a student user's performance is being decreased because of nervousness, the student user can receive assistance or tips for decreasing the nervousness and thus improving test taking performance. This aspect of the invention essentially analyzes a student user's performance and/or actions during an exam to determine the student user's level of nervousness. Many factors may be used to determine the level of nervousness. Such factors, particularly when compared with the average test taker, which may be measured to determine the level of nervousness include the student user's test taking pace, the number of times answers are changed, how long each question takes and physical manifestations of nervousness such as pulse-rate, eye movements, etc. For example, if the user is viewing many of the questions very frequently (e.g., keeps returning to certain questions more often than the average student) this would indicate nervousness on the part of the student user. Other factors are the amount of time spent per question compared to the average student user and the amount of times the user changes the answer to the question compared to the average student user. Each of these factors can carry different amounts of weight in determining

a nervousness value. For example, one may determine that a student who changes answers too often displays greater nervousness than one who simply spends too much time per question and thus one may assign a greater weight value to the former in comparison to the latter.

According to one embodiment, a numerical value between 1 to 10 is determined for each of the three factors mentioned above. That is, an individual who views 10 out of 25 of the questions more often than the average student may have a rating of 4 while a student who views 16 out of 25 of the questions more often than the average student may have a rating of 7. After these three ratings are determined, the numerical values can be used along with the weight values assigned to each criterion to come up with a final "nervousness indicator" value. Appropriate additional test-taking assistance can be provided to the student user depending on the level of nervousness. Using the same factors, an analogous indicator known as the "confidence indicator" can be determined in addition to or instead of the "nervousness indicator".

7. Statistics Module

Another aspect and advantage of the inventive method and system is simultaneously providing a unique College/University application tool for the student user and/or a valuable research tool for test developers and evaluators, educators, psychologists, etc. One advantage of an on-line test preparation system or similar computer-based network is the ability to form a database with information for a large number of users. For example, information relating to each of the student users may be formulated in a database to form a student user profile database. The user profile may include the student's age, sex, race, residence, family history (e.g., single parent, divorced parents, etc.), school size, etc. The student profile may also include information relating to extra curricular activities such as sports, student government, etc. The profile may also include information relating to the user's test preparation including when they started preparation, how many times they have taken the particular exam, what other test preparation

courses they attended, etc. The formation of this database would help compare the data of all the users based on multiple criteria.

One advantageous use for such data would be for researchers who analyze and research a variety of topics including the fairness of standardized tests, disparity of performance based on demographics, impacts of various factors on test performance, etc. For example, the user's profile may include information relating to the highest level of education obtained by the user's parents. Thus, a researcher could analyze the possible impact of the parent's educational level on the student's performance. Such information can include whether the student was bo in a foreign country or whether the student user was raised by a single parent. The type of information in the profile is unlimited, dependent only on the student user agreeing to input such information.

The invention also preferably tracks various other pieces of information which can be analyzed and researched by the authors or developers of the learning experience or environment (e.g., practice tests and study materials) to determine the effectiveness of the content of the tests and study materials employed by the method or system. For example, the invention preferably tracks the location and number of study aids used by users (e.g., personal notes, highlights, URL links and bookmarks), which can be used by an author or developer to understand where users are taking the most notes or other study aids. The invention also preferably tracks the improvement in proficiency of a user, which can be used by an author to determine the study areas where a user is weak and then they can attempt to improve the content in those areas. The invention also preferably tracks user questions and answers and the study material that the question originated from. This information gives the author or developer a chance to see the kind of question that was raised from each study material and perhaps coπect deficiencies in the learning experience. The invention also preferably tracks the number of times each personal advice rule (feedback mles are discussed in Section 10 below) was triggered. This will help the author understand the frequency with which his feedback mles are being triggered.

One embodiment of the invention relates to a method for providing a customized, on-line test preparation service to a student user at a student user computer and for providing a research tool for nonstudent users, the student user computer running a web browser application configured to receive information including web pages from a web server, to present the web pages to the student user and to navigate among web pages by sending navigational commands to the web server, the method comprising:

(a) providing a web server associated with at least one database containing a plurality of student user profiles and student user personalized web pages;

(b) receiving a student user profile database search request from a nonstudent user computer; and

(c) communicating information including one or more web pages bearing one or more student profiles, portions thereof or listings thereof for display at the non-student user computer in response to the student profile search request.

Another embodiment relates to a method for searching a database of student subscriber information in order to find at least one student subscriber or class of student subscribers matching student subscriber search criteria, the student subscriber information including a student subscriber profile, the method comprising:

(a) searching the database for records matching the student subscriber search criteria; and (b) relaxing at least some of the search criteria, until at least a predetermined number of matching student subscriber records are found.

The following are factors or information which may be included in the user's profile: name (preferably confidential, i.e., only available to the system administrator), social security number (preferably confidential), age, sex, race, ethnicity, nationality, grade level, residence (including address, city, state, email, etc., - preferably confidential), school district, grade school(s) attending/attended, high school(s) attending/attended, college or university attending/attended, grade point average, economic background (e.g., family income/assets), individual's present economic state (e.g. need for financial aid), disabilities, any drugs taken by student and amounts taken (e.g., prescription or otherwise), diet (e.g., vegetarian, etc.), number of siblings, place in family (e.g., oldest child, youngest child), marital status of student user (e.g., single, married, divorced, remarried), marital status of parents, parent's educational level, parent's major in school, parent's occupation, student's academic interests, present or intended major, athletic interests and abilities, extracurricular interests or activities, colleges/universities student is interested in applying to, has applied to, has been accepted to and/or plans to matriculate, the date the student plans to take the exam, number of months student has been preparing for the test and approximate number of hours, level of math education or any other subjects that may be relevant to a given exam, actual scores received on practice and any previous exam, reasons for student's acceptance of college/university, any scholarships and/or financial aid received or seeking educational opportunities offered, sports opportunities offered, scores on each exam, combined score analysis (e.g., the time student spend on each section of an exam, student's answer to each question, date a given test was taken), etc. Preferably, at least one of the above-listed factors or information is included in the student user profile, more preferably at least two, even more preferably at least three, and most prefened at least four. According to one particularly prefened embodiment, at least five are included, preferably at least six, more preferably at least seven, even more preferably at least eight and most prefened at least ten of the factors or information are included in the student user profile. According to a still further prefened embodiment, at least eleven are included, preferably at least twelve, more preferably at least thirteen, even more preferably at least fourteen and most prefened at least fifteen of the factors or information are included in the student user profile. Higher numbers of greater than twenty, greater than twenty-five, greater than fifty, greater than seventy-five or greater than 100 factors or items are also included within the scope of the invention.

The same or different numbers of factors may also be included in the student user or nonstudent user search requests for searching the student user profile database.

According to another embodiment, third parties such as educators, admission officers or researchers may request that additional information be included in the profile in which case the method or system would include the additional request in the original profile request and/or submit a profile update including the request to the student users. For example, a researcher may request information comparing the performance of student users from one state compared with student users from another state. Figure 30 depicts a display which may be provided in response to such a request according to one embodiment of the invention. One prefened embodiment relates to an automated system which allows the researcher to modify the profile by adding additional criteria without first submitting a request to the system administrator. For example, a researcher may wish to add another question to the student profile such as "How regularly does the student exercise?". Rather than first submitting the additional question to the system as a request, the additional question may be automatically included in the student profile information request form. If a student user has already provided a profile, the student is preferably requested to update the profile with the additional requested information.

Alternatively, another embodiment requires that the researcher first submit the additional question as a request.

Significantly, this database would also be useful for students who wish to evaluate their performance compared with other student users. This feature would better enable the student user to determine which colleges they should apply to based on how they perform compared with similarly situated individuals. This is advantageous since the college application process is time consuming and costly. For example, college admissions decisions may be based on a variety of factors. An elite college may desire a student body including individuals from throughout the country and possibly throughout the world. Thus, an applicant from a remote area may be able to better evaluate his or her chances of gaining admission to such an elite College or University by evaluating how the user performed not against all potential applicants, but against similarly situated individuals, e.g., individuals from the same remote area.

Figure 31 depicts a display according to one embodiment of the invention which may be provided in response to a request for information comparing the performance of a student user, John, compared with a subgroup of student users from a particular state, New Jersey. This information may be helpful, for example, if the student user is interested in applying to state colleges in New Jersey and believes he is competing against other applicants from New Jersey. Figure 32 depicts the information using a graphical representation according to one embodiment of the invention. Figure 33 depicts the information using a table representation according to another embodiment of the invention where the student user is compared with students from California. As another example, when the user chooses that he wants to know what his score would have been if compared only to students taking the exam on a certain date, for example, the system isolates all those students that fit the criteria and then determines the raw scores of each of these students and then statistically curves these exams in comparison to each other and come up with a new set of scores.

Thus, the ability of a method and system to construct a large detailed database including information relating to the users could be valuable not only to researchers, educators, test developers and evaluators and the like but also to the student users.

8. University Center

As stated above, according to a prefened embodiment of the invention, the student user has an individual personalized web page which may include a variety of information relating to the student user's test preparation including icons representing the study basket, the customized study plan, the performance tracker/weakness analysis tool, the score comparison module, etc.

Moreover, the student user's web page may also include information relating to the user's application to schools including undergraduate, graduate and post-doctoral schools. For example, the web page may include an icon labeled "college applications". Upon clicking this icon, a list of colleges which the student user is interested in applying to or has applied to may be displayed. Additional information including whether a particular college application has been requested by the student user, received by the student user, completed and/or submitted may be provided along with any additional documents which need to be submitted to complete any given application (e.g., essays, recommendations, etc.) and/or the application process (e.g., scheduling or dates of campus interviews or tours, etc.). In this manner, the student user can keep track what is typically a complicated and stressful application process. Preferably, this forms the student user's personalized desktop to the academic world. Here the user could place whatever links he wants on the desktop and the system displays that very same desktop every time the user logs on. For example, the user may desire links to certain colleges such as Stanford or Cornell on the desktop so that the next time the user returns he will see the links to these two pages on the personalized university center.

Furthermore, the university center preferably allow users to apply to most schools of their choice on-line. Most college/universities provide new applications on-line. Thus, the individualized university center preferably provides links to such colleges/universities such that the application forms may be downloaded and filled out and then submitted to the particular colleges/universities the user is interested in.

According to one prefened embodiment, the system includes a database containing the individual user's application information such that the system automatically fills in different application forms using information already input into the database by the student user.

Moreover, the system also preferably automatically reports to the educational testing service (ETS) or any other appropriate testing service when the specified user requires scores sent to a particular school. ETS will then process the request and upon completion notify the system which in turn will notify the student user via e-mail or the like.

Preferably, the student user can also check on the status of their application at the university center. For example, the student user can check to see if the application has been forwarded to the appropriate schools, which of the schools have received the application, which schools have been sent the scores, which schools have received the scores, which of the schools have responded that the application is complete, and whether any other schools have given a final decision, etc. Moreover, with the permission of the user, the university center can further collaborate

with colleges/universities and provide them with a list of students that may fit their criteria. For example, if the university desires students who seem to be able to score 1400 or above on the SAT, then the system can compile such a list and report to the universities. Alternatively, to maintain the student's confidentiality, the system may e-mail or otherwise contact the student regarding a college or university's possible interest and allow the student user to decide if he or she wants to contact or have their profile or portion thereof forwarded to the interested university.

Thus, this feature involves the creation of appropriate database tables and storing data given by the user and other sources and retrieving it again at a later time as has been the case with some of the other features listed above.

One embodiment includes desktop links to universities. The student user is provided with a list of colleges/universities to choose from. The user chooses the colleges he is most interested in. The links to these colleges can the be stored away in a table of the database where one would enter the student usemame, the title of the college, and the URL of the college. From this point on whenever the student user enters the university center, there may be one icon for every entry in the database table with their usemame. Under each icon will be the title of the college and the link on the icon is set to the link that was stored in the database. The student user can also type in his own links to the Internet and this information can also be stored in the table and retrieved whenever the user views his desktop.

As an example, the implementation of the college/university application feature can be done by creating a database where one can store the usemame, the name of the college/university being applied to, the date the user applies, information with regards to all of the various confirmations from ETS, colleges, etc., and all of the other appropriate columns. Once the database is completely set up, the data is stored in the appropriate fields and retrieved when necessary to be displayed to the student, or when it is necessary to notify ETS or the college/university. The application information can be stored in a different database for each university depending on the application format that the universities prefer. Once the user fills in the general application form at the site, the database associated with all the colleges he is interested in is retrieved and then the student user's information is formatted into an application pursuant to the requirements of each college and then the properly formatted application is forwarded to the college/university. If the look and not just the information needs to be in a certain format, then this can be achieved by using Word document templates with one template defined for each university. The appropriate template is opened and completed and then forwarded to the university by printing it out and mailing or by attaching it to an e-mail, depending on what the university may prefer.

Thus, according to one prefened embodiment of the invention, the student user fills in only one application template and it is formatted into the various formats for each school the student user is interested in. This way each college/university (institution) can still have its own unique application.

According to another embodiment, the student user sends a copy of their test scores and transcripts to the system after taking the test. The results are stored on file and later supplied to the institutions of interest. The student user will not be required to send a copy of their scores and transcripts to every school, thus cutting down on their expense. This would also allow the admissions officer to avoid all the extra paper work.

According to another embodiment, the system provides customized filters for every school that wishes to have one. These filters will filter out all student users who do not fit the institutions criteria for admission. For example, an institution may want us to eliminate all students who do not have a certain score and GPA or perhaps a certain level of aptitude in a subject, etc. These criterion will preferably be kept confidential and not released to the public. The option can further be given to the institutions to create a filter of their own by visiting a web site or the like.

According to one prefened embodiment, all communications between the institution and the student can pass through the university center. If the student user wishes to submit a letter to the university it would be e-mailed or mailed or posted on their university center and then the system would forward that information electronically to the university. Preferably, encrypted e- mail technology is used to ensure security.

Preferably, the student user can check the status of their application at the center and can see if an application is complete, whether a final decision has been reached, the expected date of final decision, and other such vital information which is dynamically updated by the institutions at the university center web site or through customized software.

Moreover, electronic acceptance and rejection letters could be provided through e-mail thus cutting down on the cost of creating such letters and mailing them out. The content of the letter could be customized to the institution's needs.

Preferably, the student user can also find out the average GPA, Test Score, etc., of students who are cuπently applying to any particular school of their interest or have been accepted in the past. This will help them get a feel for where they stand and thus making the necessary decisions for their academic future.

Once accepted to an institution, the student can preferably submit their decision of accepting or declining the admission offer through the university center before the deadline given by the institution. This will ensure timely responses from the student as well as make it very easy for the student to respond. Once the student has accepted a school he can inquire as to the other students who have also accepted the school. Of course, only information from students who have provided their consent will be released. This would allow the students to communicate with each other and find roommates and talk about any other vital information before they attend the institution itself.

Preferably, the invention also provides a roommate matching system to help the students find roommates of their choice, advantageously this is achieved using the student user information. The students can further fill out the housing, meal, and all other college applications from this center and can also register for the classes of their choice.

9. Electronic College Counselor

According to another prefened embodiment, the method and system provides an electronic college counselor to assist the student user decide which colleges/universities the student user should apply based on the user's interests, grades, performance on tests, etc. There are hundreds of different colleges or universities an individual may attend to obtain a higher education. Determining which to apply is sometimes a daunting task. Accordingly, one embodiment of the present invention advantageously provides suggestions or recommendations to the student user based on information collected from the student user and information collected and/or readily available regarding possible options. The "college counselor" can either be a stand alone method and system or is incorporated with one or more of the features described above.

According to one embodiment, the student user inputs relevant information such as (a) geographic preference (e.g., northeast, particular state etc.), (b) prefened type of college (large/small, private/public, urban setting/rural setting, etc.), (c) range of affordable tuition, (d) intended major, post-graduate studies (e.g. history, law school, etc.), (e) extracurricular interests (e.g., football, hockey, fencing, crew, sailing, etc), (f) intended career (e.g. journalist, accountant, engineer, fashion designer, etc.), etc. Using this information, the invention provides a list of possible options to the student user.

According to one prefened embodiment, the test preparation, student profiles, university center and electronic counselor are combined to provide an advantageous method and system for assisting the student users in achieving their goals. For example, using the invention, the student user can prepare for a standardized test such as the SAT, compare his performance not only to all student users, but to similarly situated student users to enable the student user to properly rate himself, determine which possible college/universities to apply, prepare and submit an application to selected colleges/universities and keep track of the application throughout the college admissions process. The invention not only allows the student user to more efficiently get through the college application process, but also better understand the options available and be better able to take advantage of those options.

10. Author Defined Personalized Feedback to the User

Another embodiment of the invention relates to a rich rule-based system that empowers the authors or developer of the learning environment to create highly personalized feedback to the student users as part of their courses, and associate the feedback with a set of conditions or patterns that are recognized during the actual learning process of the student. This is achieved by providing the author or developer with a rule editing capability as part of building the course where the author can either select one of several pre-defined rules in the system or create their own mles and provide their feedback in terms of the action that needs to take place when the condition or pattern defined in the rule occurs during the learning process. Unlike the existing rule based learning systems which use static mles and feedback hardwired in the system itself, the invention truly enables the author or developer to write their own personalized feedback for users or define an appropriate action against particular conditions or patterns they are concerned about and deliver an experience to each student in a scalable way where the student could feel that the author is personally watching over the student's shoulder and providing valuable personal feedback at the right time and place in the learning process.

The rule can be any condition or pattern that can be recognized based on the vast variety of information that may be tracked during the leaming process of a student user. Such tracked information may include: (a) the student's cunent progress, (b) the student's completion status, (c) whether the student is on or behind or ahead of the schedule, (d) the student's performance and score in different tests, (e) the amount of time spent by the student on different topics and test questions, (f) the student's subject matter strengths and weaknesses identified by the system based on how well they answered specific questions that are associated with one or more specific topics or subtopics, (g) the amount and nature of study aids used by the student on different topics, (h) the number of questions raised by the student in the context of different topics, (i) the student's leaming patterns and preferences identified by the system over time, and (j) whether the student leverages all the capabilities of the underlying system to improve the efficiency of his or her learning process. Examples of leaming patterns identified by the system over time may include things like if the student tends to skip practice quizzes and tests, if the student tends to rush through the study material, if the student tends to not review or analyze the areas where he or she did not perform well, etc. Examples of leaming preferences identified by the system over time may include things like if the student tends to leam better when the study material is delivered in an audio form or text form or through visual representations or through hands-on simulation, etc. Rules can be based on any combination of the above information which may be tracked for the student user as well as any comparison of these information with the conesponding information averaged over the students peers in the system. There are several types of actions that can be specified by the author or developer against any rule (i.e., actions which are triggered by rule). The action could be to send an email to the student with specific feedback, or display the feedback text at the appropriate juncture in the student's learning process or display feedback when the user explicitly requests the personal feedback from the system, or add/remove specific course content, or specify altemate content to the student for specific topics, or notify the students parents or managers or mentors through email, or suggest alternatives or related courses that might be useful for the student.

As an example, the condition represented in the rule might be "Daily time commitment required in future to complete the rest of the course on time is less than or equal to the cunent average daily time commitment of the user" and the coπesponding action defined by the author might be to email or display the following text to the student: "%UserName%! At your cunent rate you are on schedule to complete %MilestoneName% on or before %MilestoneDate%". Both the rule and the action can include one or more variables tracked by the system.

The variables can be specified by the author or developer as %VariableName% in the middle of a rule or feedback and the system will automatically substitute the actual value (e.g., user's name) while evaluating the rule and taking the coπesponding action. This allows the feedback text to be personalized for each user even after the method or system has been developed.

Another example of a pattern represented by the rule might be "The student has spent at least 50% less time on %TypeOfQuestion% on the past three practice tests compared to the student's peers and he has scored at least 25% less in %Type of Question% on all three practice tests compared the peers" and the coπesponding action might be to provide the feedback "%UserName%! Compared to your peers, you tend to rash through %TypeOfQuestion% when taking tests. Slowing down a bit and spending more time on understanding this type of question before answering could help you improve your performance." The above examples of the invention are intended to be illustrative and not limiting.

11. Systematic and Automatic Creation of an Expert Community with reliable validation of expertise.

Another embodiment of the invention relates to the automatic and systematic creation of an Expert Community. This is achieved by automatically identifying students who have performed exceptionally well on specific subjects and making their expertise available to the rest of the leaming community. Advantageously, other students who are learning that subject matter could ask questions and get help from these peer experts. Preferably, being identified as an "expert" could be used as an award to motivate learning.

Existing methods of creating expert communities either require the experts to be manually added into the system by an authority or require providing an electronic method for an expert to add himself or herself into the system by declaring his or her expertise in a certain subject. The biggest drawback of these methods is that there is no clear way to automatically validate the expertise of the person when they are granted an expert status. The invention removes this shortcoming by granting expert status to selected students in the system who have achieved greater than a specified high proficiency in the subject. This is made possible in the system because of the fact that the system tracks a rich variety of information about each student as they leam a subject and thereby has the ability to use this tracked information to validate and determine if the student should be granted an expert status in the subject. The author or developer of the study materials and tests can specify a cutoff bar for each study material and test, and a student whose performance as measured by score or proficiency is above the metric may be selected or identified as a potential expert. Unlike the previous expert community models, the invention truly enables building an expert community from scratch by automatically locating and identifying expertise among students through a reliable method by tracking their actual performance in the subject. Once located, the student becomes a candidate for becoming a peer expert in that subject and, preferably, is given the option to participate in the community as an expert. If the student is willing, the system grants that student a peer expert status and automatically advertises the student's expertise to the community of other students in the system. In essence, the student's expertise is really evolved and promoted by the system into a community resource that can be used by other students. This in turn helps other students to achieve better knowledge and expertise in that subject and potentially become peer experts themselves.

The above description of the invention is intended to be illustrative and not limiting. Various changes or modifications in the embodiments described may occur to those skilled in the art. These can be made without departing from the spirit or scope of the invention.

Claims

CLAIMS;

1. A computer-aided method for assisting at least one student user leam subject matter using sample test results generated by said student user taking a sample test and student user information from said student user, said method comprising:

(a) receiving said student user information from said student user;

(b) analyzing said sample test results generated by said student user; and

(c) providing a performance evaluation, a study plan, a study basket or combinations thereof based on said sample test results and said student user information.

2. A computer-based leaming method for teaching a user one or more subjects comprising the steps of:

(a) providing a series of questions to the user on a display;

(b) retrieving user answers from said user for said series of questions; and

(c) evaluating said answers by comparing the user answers with conect answers thereby determining the strengths and weaknesses of said user, wherein said method further comprises the step of retrieving user information from said user sufficient to form a user profile.

3. A computer-aided learning method for assisting a student user leam subject matter, the method comprising:

(a) generating a sample test comprising a multiplicity of questions relating to one or more subjects from a question database, said multiplicity of questions being associated with a predetermined coπect answer;

(b) transmitting said sample test to said student user;

(c ) receiving student user answers for said multiplicity of questions from said student user; (d) evaluating said student user answers compared with said predetermined coπect answers thereby determining which questions said student user answered conectly and which questions said student user answered inconectly thereby providing test results; and

(e) assisting said student user by providing said student user with said test results and an evaluation of said test results, wherein said method further comprises the step of receiving student user information from said student user and said step of assisting said student user comprises utilizing said student user information.

4. A computer-aided learning method for assisting a student user prepare for a standardized test, the method comprising:

(a) transmitting a sample test comprising a multiplicity of questions relating to one or more standardized test subjects from a question database, each of said multiplicity of questions being associated with a predetermined coπect answer;

(b) receiving student user answers for said multiplicity of questions from said student user;

(c) evaluating said student user answers compared with said predetermined conect answers thereby determining which questions said student user answered coπectly and which questions said student user answered incoπectly thereby providing test results; and

(d) providing said student user with said test results and an evaluation of said test results; wherein said method further comprises the step of receiving student user information from said student user including at least two or more of the following: (i) student user identification selected from the group consisting of name, social security number, credit or debit card number, telephone number or other identification, (ii) date of standardized test or how many days until the standardized test, (iii) time the student user can allocate for preparing for the standardized test, (iv) student user standardized test score target goal, and (v) previous standardized test scores, if any.

5. A method for providing a customized, on-line test preparation service to a student user at a student user computer and for providing a research tool for nonstudent users, said student user computer running a web browser application configured to receive information including web pages from a web server, to present said web pages to said student user and to navigate among web pages by sending navigational commands to said web server, said method comprising:

(a) providing a web server associated with at least one database containing a plurality of student user profiles and student user personalized web pages;

(b) receiving a student user profile database search request from a nonstudent user computer; and

(c) communicating information including one or more web pages bearing one or more student profiles, portions thereof or listings thereof for display at the non-student user computer in response to the student user profile search request.

6. A method for searching a database of student subscriber information in order to find at least one student subscriber or class of student subscribers matching student subscriber search criteria, the student subscriber information including a student subscriber profile, the method comprising:

(a) searching the database for records matching said student subscriber search criteria; and (b) relaxing at least some of the search criteria, until at least a predetermined number of matching student subscriber records are found.

7. The method of claim 1, further comprising the step of password protecting said student user information.

8. The method of claim 1, wherein said method assists said student user leam subject matter in preparation for a test.

9. The method of claim 1, wherein said method assists said student user leam subject matter in preparation for a standardized test.

10. The method of claim 1, further comprising repeating steps (a) through (c) at least once, wherein after each sample test, said student user is provided an updated performance evaluation based on information including said test results, additional student user information or combinations thereof.

11. The method of claim 1, further comprising receiving from said student user a request for taking said sample test prior to steps (a) - (c).

12. The method of claim 1, further comprising receiving from said student user a request for taking an additional sample test after step (c).

13. The method of claim 1, further comprising providing said student user with a test taking pace.

14. The method of claim 13, wherein said test taking pace includes a pace analysis on a test section by test section basis.

15. The method of claim 13, wherein said test taking pace includes a pace analysis on a question by question basis.

16. The method of claim 13, wherein said test taking pace includes a pace analysis based on topic.

17. The method of claim 13, wherein said student user is provided with an indication during said sample test of whether the test taking pace is slower or faster than a predetermined pace.

18. The method of claim 1, further comprising the step of providing said student user with a study plan based on said test results and said student user information including the scheduled date of said test.

19. The method of claim 18, wherein said study plan includes a schedule for one or more of: (a) taking practice tests, (b) studying test topics, (c) taking practice questions, or (d) achieving predetermined study goals.

20. The method of claim 1, wherein said multiplicity of questions are each associated with one or more study topics tested on a standardized exam.

21. The method of claim 20, wherein said evaluation and any updated evaluation includes analysis of said student user test performance is based on said one or more topics.

22. The method of claim 1, further comprising the step of providing said student with study topics based on said test results.

23. The method of claim 1, further comprising the step of providing said student with test taking tips based on said test results.

24. The method of claim 1, further comprising the step of providing said student with practice questions based on said test results.

25. The method of claim 24, wherein said student user requests said practice questions prior to said practice questions being provided.

26. The method of claim 24, wherein each practice question is associated with a topic.

27. The method of claim 1, further comprising providing said student user with a vocabulary building study session comprising the steps of:

(i) providing said student user with one or more study words;

(ii) allowing said student user to study said one or more study words;

(iii) providing said student user with a study word question;

(iv) receiving a student user answer for said study word question;

(v) providing said student user with a predetermined coπect answer to said study word question;

(vi) allowing said student user to continue with another study word question and, if selected, repeating steps (iii)-(v); and

(vii) updating one or more of said student user's study plan or said student user's study basket.

28. The method of claim 27, wherein said updating comprises designating words as learnt or unlearnt.

29. The method of claim 27, wherein said student user may designate or redesignate said words as learnt or unlearnt.

30. The method of claim 27, wherein said unlearnt words are added to one or more of said study basket or said study plan.

31. The method of claim 28, wherein said learnt words are added to a database containing other learnt words.

32. The method of claim 1, further comprising the step of receiving a student user question or request from said student user.

33. The method of claim 32, further comprising the step of providing an answer to said student user question or request.

34. The method of claim 1, wherein the method provides a study plan.

35. The method of claim 34, wherein said study plan includes a calendar.

36. The method of claim 34, wherein said study plan includes a schedule for taking practice tests, practice questions or problems, studying topics, or combinations thereof.

37. The method of claim 34, wherein said student user information includes the date of an actual test or how many days until the actual test.

38. The method of claim 34, wherein said student user information includes the amount of time per day said student user can study.

39. The method of claim 34, wherein said student user information includes the amount of time per week said student user can study.

40. The method of claim 1 , further comprising providing said student user with a secured identification including a personal identification, a personal password or combinations thereof.

41. The method of claim 40, further providing a personalized web page for said student user, said personalized web page being associated with said secured identification and said personalized web page displaying or providing links to information including one or more of the following: a student profile, student user study plan, student user study basket, practice exams, practice exam scores, actual exam scores, test taking tips, study topics or test preparation calendar.

42. The method of claim 40, wherein said personalized web page may be customized by said student user.

43. The method of claim 40, further providing a personalized web page comprising information relating to, links to information relating to or links to one or more of the following: colleges, universities, financial assistance organizations, evaluations or ratings of colleges or universities, admissions standards for colleges/universities, admissions offices of

colleges/universities, or educational testing services.

44. The method of claim 43, wherein said personalized web page further comprises one or more of the following: (a) information relating to colleges/universities of interest to student user; (b) applications for colleges/universities, (c) information relating to colleges/universities applied to by the student user, (d) status of applications, (e) offers of matriculation, (f) financial aid, or (g) acceptances by student user.

45. The method of claim 1, further comprising receiving student user information from said student user and storing said student user information in a student information database.

46. The method of claim 45, wherein said student user information includes at least one student profile item selected from the group consisting of: name, social security number, age, sex, race, ethnicity, nationality, grade level, residence, school district, grade school(s) attending/attended, high school(s) attending/attended, college or university attending/attended, grade point average, economic background, individual's present economic state, disabilities, drags taken by student and amounts taken, diet, number of siblings, place in family, marital status of student user, marital status of parents, parent's educational level, parent's major in school, parent's occupation, student's academic interests, present or intended major, athletic interests and abilities, extracurricular interests or activities, colleges/universities student user is interested in applying to, has applied to, has been accepted to and/or plans to matriculate, the date the student user plans to take the exam, number of months said student user has been preparing for the test and approximate number of hours, level of math education or any other subjects that may be relevant to a given exam, actual scores received on practice and any previous exam, reasons for student's acceptance of college/university, any scholarships and/or financial aid received or seeking educational opportunities offered, sports opportunities offered,

scores on each exam and combined score analysis.

47. The method of claim 45, further comprising the step of receiving a search request for said student information database from a nonstudent user.

48. The method of claim 45, further comprising the step of receiving a search request for said student information database from a student user.

49. The method of claim 47, wherein said student profile database search request includes at least one criteria selected from said student profile items.

50. The method of claim 47, wherein said student profile database search request includes at least two criteria selected from said student profile items.

51. The method of claim 47, wherein said student profile database search request includes at least three criteria selected from said student profile items.

52. The method of claim 47, wherein said student profile database search request includes at least four criteria selected from said student profile items.

53. The method of claim 47, wherein nonstudent user is selected from the group consisting of: (a) admissions officer, (b) college application counselor, (c) standardized test developer, (d) standardized test evaluator, (e) education researcher or (f) psychologist.

54. A computer-readable media tangibly embodying a test preparation database, a student user profile database and a program of instractions executable by a computer to perform the method of claim 1.

55. A computer system for assisting one or more student users lea subject matter using test results and student user information comprising: (a) a student user computer having local memory and a display for displaying an interactive environment that allows the student user to access secure and non-secure areas on a remote server, said remote server having memory for storing a database and having a secure area and a non-secure area, said secure area including student user information, student user study plan, study basket or combinations thereof and said non-secure area including test preparation information, practice questions, practice tests, study topics or combinations therefore;

(b) a communication network for coupling said remote server with said student user computer for allowing communication therebetween.

56. A computer system for assisting one or more student users leam subject matter using test results and student user information and for assisting one or more nonstudent users evaluate said one or more student users, said computer system comprising:

a server having a memory and a student user profile database defined in the memory, said student user profile including student user information; and

a teaching computer which includes a teaching process which, when executed, performs the steps of:

(i) presenting stimuli to said student user, said stimuli selected from the group consisting of: questions, tests, study topics or combinations thereof;

(ii) receiving student user response signals generated by the student user by physical manipulation of one or more user input devices in response to the stimuli;

(iii) storing, in a student user response database, student user response data representing the student user response signals; (iv) forming evaluation data representing a conelation between the student user response data and predetermined conect response data for the stimuli; and

(v) transmitting the evaluation data to said student user profile database for storage therein and to said student user;

57. The system of claim 56, further comprising a server computer which includes a server process which, when executed, performs the steps of:

(i) receiving a student user profile search request from a nonstudent user computer;

(ii) searching said student user profile database in response to said request; and

(iii) sending student user profile search results to said nonstudent user.

58. The system of claim 56, further comprising a server computer which includes a server process which, when executed, provides said student user with a comparative evaluation of said evaluation data compared with other student users using said student user profile database.

59. A system for assisting one or more student users leam subject matter comprising:

(a) a server having a memory and a database defined in the memory containing student user information;

(b) a student user computer in communication with the server, the server in operation communicating to the student user computer a plurality of selectable graphics, requesting a selection by the student user, receiving a selection from the student user, and adding the selection to the database, the server in operation further communicating to the student user computer a request for information identifying the student user, receiving the information from the student user computer, adding the information to the database, and associating in the database the student user with a personal web page selected by the student user.

60. A system for communicating between student users and a service providing test preparation assistance and college or university admissions assistance on a computer network, comprising:

a student user computer including a computer display, the student user computer being one of a plurality of student user computers on the computer network;

a graphical user interface for displaying a desktop and one or more icons representing programs, files and resources or links to resources including one or more of the following: student user profile, student user study plan, student user study basket, practice exams, student user practice exam scores, student user actual exam scores, test taking tips, study topics, student user test preparation calendar, colleges, universities, financial assistance organizations, evaluations or ratings of colleges or universities, admissions standards for colleges or universities, admissions offices of colleges or universities, educational testing services, applications for colleges or universities, colleges or universities applied to by the student user, status of applications submitted by student user, offers of matriculation to student user, financial aid offered to student user, or acceptances by student user; and

a student user options page displayed on the computer display, the student user options page having student-user selected criteria for controlling which icons representing said programs, files and resources or links to resources available on the computer network are displayed on the desktop.

61. The method of claim 1, where the analysis of test results is based on a proficiency metric.

62. The method of claim 3, wherein said evaluating of said results is rule based, wherein a rale includes a condition and wherein when said condition met, an action is triggered.

63. The method of claim 62, wherein said action comprises an email notification to said student user.

64. The method of claim 62, wherein said action comprises displaying feedback text, or displaying audio or video clips to said student user.

65. The method of claim 62, wherein said action comprises the addition or removal of study material, and/or providing alternate or related study material to said student user.

66. The method of claim 64, wherein the feedback text contains variables to enable personalization and delivery of tracked information as part of said feedback by substituting with a real cunent value before displaying feedback.

67. The method of claim 1, further comprising identifying at least one expert student user within a group of student users based on the performance evaluation of said at least one expert student user.

68. The method of claim 67, further comprising allowing at least one student user to submit questions to said at least one expert student user.

69. The method of claim 67, wherein said at least one expert student user is identified using a performance metric based on topic proficiency, pre-assessment proficiency, post- assessment proficiency or combinations thereof.

70. The method of claim 2, further comprising identifying at least one expert student user within a group of student users based on the performance evaluation of said at least one expert student user.

71. The method of claim 70, further comprising allowing at least one student user to submit questions to said at least one expert student user.

72. The method of claim 3, further comprising identifying at least one expert student user within a group of student users based on the performance evaluation of said at least one expert student user.

73. The method of claim 72, further comprising allowing at least one student user to submit questions to said at least one expert student user.

74. The computer system of claim 55, wherein said system automatically identifies potential expert student users by selecting students who have performed well on tests and/or study materials.

75. The computer system of claim 56, wherein said system automatically identifies potential expert student users by selecting students who have performed well on tests and/or study materials.

76. The method of claim 67, further comprising specifying a minimum user proficiency that said student user needs to achieve to become eligible for being recognized as a student expert and selecting said expert student user based on said minimum user proficiency.

77. The method of claim 1, further comprising said at least one student user to share personalized study materials with other student users.

78. The method of claim 77, further comprising allowing said at least one student user to filter specified personalized study materials shared by other student users.

79. A method for allowing sharing personal study aids among two or more students users, the method comprising:

(a) designating personal study aids of a first student user as shared study aids accessible by other student users; and

(b) allowing said other student users to access said shared study aids.

80. The method of claim 79, further comprising allowing at least one of said other student users to filter out specified shared study aids.

81. The method of claim 79, further comprising allowing said first student user to restrict which personal study aids may be shared with other student users.

82. The method of claim 79, further comprising allowing at least one student user to search the contents of study aids that are shared by other student users and/or allowing at least one student user to search the contents of said student user's personal study aids.

83. The method of claim 1, further comprising tracking performance and/or questions of said at least one student user and revising said study plan based on said tracking.

84. A computer-readable media tangibly embodying a subject matter database, a user profile database and a program of instractions executable by a computer to perform the method of claim 79.

85. A method for providing feedback to an author or administrator of a computer based learning experience for student users, said feedback comprising information regarding the effectiveness and clarity of the content of test and study material employed in the learning experience, the method comprising:

(a) tracking specified information including: (i) type and number of study aids used by at least one student user, (ii) improvement in proficiency of at least one student user in one or more study areas, (iii) at least one student user question and the study material from which said question originated, (iv) the number of times a personal advice rule was triggered, (v) or combinations thereof; and

(b) evaluating said learning experience based on said tracking.

86. The method of claim 84, further comprising revising said learning experience based on said evaluating.