WO2008127337A1

WO2008127337A1 - Intellectual property examination

Info

Publication number: WO2008127337A1
Application number: PCT/US2007/066698
Authority: WO
Inventors: Ray J. Mueller; Andrew Van Luchene; Dean Alderucci
Original assignee: Leviathan Entertainment
Priority date: 2007-04-16
Filing date: 2007-04-16
Publication date: 2008-10-23

Abstract

The present disclosure provides various novel means for selecting and assigning patent applications for examination. The disclosure describes various automated and non-automated means for selecting a qualified examiner based on a number of criteria including the examiner's history, current workload and the subject matter of the filed application. The disclosure further describes means for adjusting examiners workloads and ensuring the quality of examined applications.

Description

Intellectual Property Examination

Background

[0001] Protecting intellectual property through patent systems is a vital part of most countries' national economies and well as the global economy. However, many patent systems are facing a number of challenges due to the increased technical complexity of patent applications as well as with the challenge of hiring and training new patent examiners to cope with the increasing number of applications being filed. [0002] In 2000, 311,807 patent applications were filed in the U.S. This number increased to 409,532 applications in 2005. Globally, 145,300 applications were filed under the Patent Cooperation Treaty in 2006, representing a 6.4% growth over the previous year. This trend has held steady since 1995 with the number of applications filed increasing every year.

[0003] The problems in the protection of intellectual property rights are further compounded by virtual reality games. Hundreds of thousands of players access games known as massive multi-player online games (MMOGs) and massive multi-player online role playing games (MMORPGs). Players of these games customarily access a game repeatedly (for durations typically ranging from a few minutes to several days) over a given period of time, which may be days, weeks, months or even years. Many of these games purport to give intellectual property rights to the players in their virtual creations. However, these games lack a structured system for evaluating and granting such rights.

[0004] Given the increasing number of applications being filed and the increased demand for protection of intellectual property, it would be advantageous to provide alternate methods for assigning and distributing applications for examinations. Such alternate methods would relieve some of the pressure on patent systems, allowing examiner's to focus on the aspects of their duties that require human involvement.

Brief Description of the Drawings

[0005] Fig. 1 is a block diagram depicting a system 100 of an embodiment of the present invention.

[0006] Fig. 2 is an embodiment of a method of assigning an application to an examiner according to one embodiment of the application.

[0007] Fig. 3 is a block diagram depicting a system 200 of an embodiment of the present invention.

[0008] Fig. 4 is an embodiment of a method of a method of examining an application according to an embodiment of the invention.

[0009] Fig. 5 is a block diagram depicting a system 300 of an embodiment of the present invention.

[0010] Fig. 6 is a block diagram depicting a system 400 of an embodiment of the present invention.

[0011] Fig. 7 is a block diagram depicting a system 500 of an embodiment of the present invention.

[0012] Fig. 8 is an embodiment of a method of assigning an application to an examiner according to one embodiment of the application.

[0013] Fig. 9 is a block diagram depicting a system 600 of an embodiment of the present invention.

[0014] Fig. 10 is an embodiment of a method of reviewing an application or patent according to an embodiment of the application.

[0015] Fig. 11 is a block diagram depicting a system 700 according to an embodiment of the present invention.

[0016] Fig. 12 is a block diagram of a system 800 according to an embodiment of the present invention.

Detailed Description

[0017] A patent is a means for protecting the rights of an inventor. It is a property right granted to an inventor by a governing entity or by a regional office or other third party acting for a governing entity or group of government entities. This right allows the inventor to exclude anyone else from commercially exploiting the inventor's invention for a set time period. What is granted is not the right to make, use, offer for sale, sell or import, but the right to exclude others from making, using, offering for sale, selling or importing the invention.

[0018] By granting an exclusive right, patents provide incentives to individuals, offering them recognition for their creativity and material reward for their marketable inventions.

These incentives encourage innovation. Due to the steady increase in the number of applications being filed, the burden on various patent offices has increased. This increase has resulted in delays in reviewing applications and increased workloads for examiners. There have been many efforts to streamline the patent application process, shorten examination times and ensure the quality of issued patents. However, the sheer number of applications submitted can be overwhelming.

[0019] Various embodiments of the present invention address this issue by providing methods and systems for assigning applications to an examiner for examination. Such a system may be used to select the examiner with the most relevant expertise, distribute workloads more evenly, evaluate efficiency and accuracy, as well as increase the resources available for evaluating applications and issued patents. These systems and methods may be used in the real world as well as in virtual worlds and provide a means for automating all or part of the selection process.

[0020] According to various embodiments:

[0021] Abstract of the Invention-includes that part of a patent application that is the abstract as defined by the USPTO guidelines.

[0022] Agent- includes the agent responsible for filing a patent application.

[0023] Alternate Language- includes words that can be used as alternates for words in a patent application.

[0024] Artificial Intelligence- includes any computer program that uses neural nets and genetic algorithms.

[0025] Assignee Name- includes the meaning defined by the USPTO guidelines.

[0026] Assignee City- includes the meaning defined by the USPTO guidelines.

[0027] Assignee State- includes the meaning defined by the USPTO guidelines.

[0028] Assignee Country- includes the meaning defined by the USPTO guidelines.

[0029] Attorney- includes the attorney responsible for drafting and/or filing a patent application. [0030] Attorney Name- includes the meaning defined by the USPTO guidelines.

[0031] Attorney Address- includes the meaning defined by the USPTO guidelines.

[0032] Attorney State- includes the meaning defined by the USPTO guidelines.

[0033] Attorney Country- includes the meaning defined by the USPTO guidelines.

[0034] Background of the Invention- includes that part of a patent application that is background as defined by the USPTO guidelines.

[0035] Claims- includes that part of a patent application that is claims as defined by the

USPTO guidelines.

[0036] Date Stamp- includes an electronic, unalterable stamp on an electronic file indicated the date that the file was created or received by a computer system.

[0037] Date of Invention- includes the date a patent application has with a first time stamp.

[0038] Degree of infringement- includes the statistically measured amount that a product or technical white paper infringes an issued patent application.

[0039] Description of the Invention- includes that part of a patent application that is description as defined by the USPTO guidelines.

[0040] Draftsperson- includes the meaning defined by the USPTO guidelines.

[0041] Dollar Value- includes a dollar amount that is defined as the value of a patent license of a patent.

[0042] End User- includes any user of a system including an inventor, researcher, attorney, or agent who is interacts wiuth the system, e.g., by creating, enhancing, researching, filing, prosecuting, licensing, or invalidating a patent application. An end user may be required to be a member of a central system.

[0043] Electronic notification- includes an email or other means of digitally sending a message with a date and time stamp to an electronic address.

[0044] Errors and Omissions- includes the meaning defined by the USPTO guidelines

[0045] Examiner- includes a patent examiner.

[0046] Issued Patent- includes the meaning defined by the USPTO guidelines.

[0047] Filing Date- includes the time stamp of the date that a patent application was submitted to the patent office.

[0048] Filed Patent- includes a patent application that is filed with the USPTO. [0049] File Wrapper- includes all files associated with a patent application including but not limited to: the patent application, a certified search, notes of distinguishing language, notes of rejection, notes of additional distinguishing language, record of interview, additional prior art references, and all electronic notifications associated with a patent application.

[0050] First Office Action- includes the meaning described in the USPTO guidelines.

[0051] Genetic Algorithm- includes a computer algorithm that is capable of modifying and improving itself over time.

[0052] Infringement- includes that a product or technical white paper practices the invention protected by the claims of an issued patent.

[0053] Interview- includes an electronically recorded conversation between an end user and a patent examiner.

[0054] Invention Class- includes the meaning described in the USPTO guidelines.

[0055] Invention Subclass- includes the meaning described in the USPTO guidelines.

[0056] Invention Figures- includes the meaning described in the USPTO guidelines.

[0057] Invention Claims- includes the meaning described in the USPTO guidelines.

[0058] Inventor Name- includes the meaning described in the USPTO guidelines.

[0059] Inventor City- includes the meaning described in the USPTO guidelines.

[0060] Inventor State- includes the meaning described in the USPTO guidelines.

[0061] Inventor Country- includes the meaning described in the USPTO guidelines.

[0062] Issued Patent- includes the meaning described in the USPTO guidelines.

[0063] Literature Prior Art- includes prior art for a patent application other than patents.

[0064] Missing Parts- includes the meaning described in the USPTO guidelines.

[0065] Non-Obviousness Score- includes a score given to a patent application by a central system that relates the obviousness of the invention disclosed by the patent application to prior art cited by the central system.

[0066] Notice of allowance- includes the meaning defined by the USPTO guidelines.

[0067] Notes- includes any language added to a prior art record by an end user.

[0068] Note of distinguishing language- includes notes provided by end users in response to the prior art cited in a certified search. These notes distinguish a patent application submitted by the end user over the prior art references contained in the certified search provided by the central system. [0069] Note of additional distinguishing language- includes notes provided by end users in response to a second office action conducted by a patent examiner.

[0070] Note of rejection- includes the notes contained in a second office action provided by a patent examiner.

[0071] Novel— includes the meaning described in the USPTO guidelines.

[0072] Novelty Score- includes a score given to a patent application by a central system that related the novelty of the invention disclosed in the patent application to prior art cited by the central system.

[0073] Obvious- includes the meaning described in the USPTO guidelines.

[0074] Office Action— includes the meaning described in the USPTO guidelines.

[0075] Online Chat Room- includes any electronic correspondence medium that allows for a real time, electronic conversation between a patent examiner and an end user.

[0076] Patent Application- includes any document created to describe and invention by an end user.

[0077] Patent Application Data- includes data contained in a patent application.

[0078] Patent Application Date- includes the time stamped date that a patent application was entered into a central system.

[0079] Patent Examiner- includes a person responsible for reviewing the patent application and deciding if the patent can be issued.

[0080] Patent examination queue- includes the queue of patent applications that are assigned to a patent examiner that require office actions or reexaminations.

[0081] Patent invalidator- includes an end user who is attempting to invalidate an issued patent.

[0082] Patent License- includes a legal right to use an invention disclosed in an issued patent.

[0083] Patent Licensee- includes an end user who is licensing an issued patent.

[0084] Patent Office- includes the United State Patent and Trademark Office and any other Intellectual Property authority, virtual or real in this or any other world.

[0085] Patent Prior Art- includes prior art that is filed and issued patents.

[0086] Patent Value score- includes a score assigned by an artificial intelligence system that demonstrates the strength of the claims of an issued patent in light of prior art.

[0087] PCT Information - includes the meaning described in the USPTO guidelines. [0088] Potential Licensee- includes an end user who may want to license an issued patent.

[0089] Prior Art- includes any document with a time stamp prior to the time stamp of a patent application.

[0090] Prior Art Data- includes data that is prior art.

[0091] Priority Date— includes the meaning described in the USPTO guidelines.

[0092] Product- includes a created thing that can be protected by or that can infringe the claims of an issued patent.

[0093] Published Prior Art-includes prior art that is available for review by the general public.

[0094] Reexamination- includes a second examination of a patent after it has been issued.

[0095] Relevance Score- includes a score assigned by an end user or by a central system to a particular piece of prior art as it relates to a particular patent application.

[0096] Research Report- includes a report assemble by a researcher or a central system that contains prior art related to a patent application.

[0097] Researcher- includes a person who manually researches prior art databases to find prior art related to a patent application.

[0098] Score- includes a numerical value assigned to something as it relates to something else.

[0099] Second Office Action— includes the meaning described in the USPTO guidelines.

[00100] Second examination- includes reexamination.

[00101] Status Change- includes a change in status of a patent application as it moves through the patent process. Changes in status can include but are not limited submitting the application for examination, receiving a certified search for the application, placing the patent application in an examiner queue, receiving an office action for the patent application, receiving a notice of allowance for the patent application, receiving a notice of missing parts for the patent application; receiving a patent number for the patent application, and receiving an indication of interest from a potential licensee for the patent application. [00102] Submitted Patent Application- includes a patent application that an end user submits to the central system for examination.

[00103] Subsequent Patent Application- includes an application that comes after a patent application.

[00104] Technical white paper- includes a text description of a product that describes the parts of the product and how they work together.

[00105] Time Stamp- includes an unalterable recording of the time a document was created by, entered into, or received by a system.

[00106] Title— includes the meaning described in the USPTO guidelines.

[00107] Web-Based Application- includes an application that is accessible on the

World Wide Web via a web browser such as Microsoft's Internet Explorer. The application will be stored on a central server and accessed via other computers.

[00108] Web-Based Form- includes an electronic form used to enter information by and end user into a web-based application.

[00109] Unpublished Prior Art- includes prior art that is not available to the general public, but that can be viewed by employees of the central system.

[00110] Useful— includes the meaning described in the USPTO guidelines.

[00111] Usefulness Score- includes a score given to a patent application based on its usefulness as defined by the USPTO guidelines.

[00112] Generally, patent applications are received by a patent office and then assigned to a technology center or other art group suited to that field. The applications are then classified and assigned to an examiner by a senior examiner or other reviewing entity. Given the increasing number of applications and the number of examiners working in the patent office, and the increasing specializing of particular technological fields, it may be difficult for a supervisor or other reviewing authority to retain the relevant details regarding the experience and history of all of the examiners.

[00113] Automating the process in whole or in part may streamline the application process, save examiners' time and increase productivity by allowing examiners to focus on higher value added activities such as examination or training rather than administrative tasks. Furthermore, spending large amounts of time on administrative duties such as assigning applications can erode skills, reduce efficiency and impair employee mentoring and development. Automating will increase the time available for other tasks, helping to address some of the issues such as a backlog of applications or delays in examination that are currently plaguing patent offices. [00114] In some embodiments, a database may be created of all of the examiners and their histories. Such information may be used to create a profile of a patent examiner. This profile may be used to determine the examiner(s) with the most relevant experience in regards to each application received. Profiles may include information regarding previous applications that the examiner has examined, the patent applications in the examiner's queue for examination, the examiner's efficiency rating, the prior art cited in the previous and/or waiting applications, the examiner's education, the examiner's particular area of expertise, the length of time it takes the examiner to examine an application, the examiner's grade or level, the examiner's current workload, the examiner's seniority, the examiner's previous experience, the examiner's training, the number of reissues in previously filed applications, the number of appeals filed, the results of appealed applications, or any combination thereof.

[00115] In some embodiments, the elements of a profile may be assigned a number and a total score for an examiner may be generated. In some embodiments, the elements may be weighted. In further embodiments, the weighting of the elements may change depending on the use of the score. For example, the weighting may change depending on the information in the application. Some areas of technology may require more outside knowledge than others. In those instances, the examiner's previous education may carry more weight than other elements of a profile. In technology areas that require less specific knowledge, other elements of the profile may carry more weight. In some embodiments, the score generated may be a relevance score relating to the examiner's area of expertise in relation to a particular application. [00116] When a new application is submitted to a patent office for examination, the field of invention, class, subclass, abstract, claims, or any combination thereof is reviewed for subject matter and an area of expertise for review is determined. Such a review may be run using a simple table based method, a rules based system, or artificial intelligence techniques, for example neural net, Bayesian algorithm, genetic algorithms, pattern recognition, expert systems, case based reasoning, fuzzy systems, hybrid intelligent systems, evolutionary computation, concept processing or any combination thereof. [00117] In some embodiments, the submitted application is compared to the score generated by the patent examiner profile. Examiners whose profiles are comprised of those patent applications and prior art references that are most relevant to the patent application being submitted are given the highest relevance scores from the submitted patent application.

[00118] In other embodiments, the patent examiners with the highest relevance scores are then reviewed for workload. In some embodiments, workload is evaluated prior to determining relevance of experience. For example, examiners may not be eligible to receive an application for examination if the score they receive for their current workload is too high. Workload may be evaluated based on the number of applications the examiner is currently reviewing, the average length of time the examiner takes to process an application, the expected length of time to process a new application, or any combination thereof. In some embodiments, determinations may be made by the examiner, his peers or the public. In other embodiments, determinations may be made using artificial intelligence and genetic algorithms. In some embodiments, scores from different sources may be aggregated. Such an aggregation may or may not be weighted. In additional embodiments, the workload scores of examiners should be effectively equal. In order to address an imbalance or inequity, applications that are waiting to be examined may be reassigned. In additional embodiments, continued inequities in workloads may influence hiring decisions. For example, a supervisor may be notified if there is a continuing increase in applications in a particular area. Such notification may trigger a hiring listing or further investigation into the trend. [00119] The examiner with the most closely related relevance score and the workload score indicating the most availability may then be assigned the application. In some embodiments, the comparison of the relevance score and workload score may be weighted. In other embodiments, for example in particularly specialized fields, the workload score may be disregarded.

[00120] In some embodiments, the system may be designed to receive feedback.

Such feedback may be used to increase the accuracy of the assignment and the weighting of the scoring. For example, the assigned examiner may submit their time estimate for reviewing the application. This time estimate may be shorter or longer than that calculated by the system. In another embodiment, the examiner could determine that he or she does not have the necessary expertise in the field of the application and may request reassignment of the application. Such feedback could be incorporated into the algorithm to add to the efficiency of the system. Feedback may be used to continuously update the system, update it periodically, or may be used as part of a multivariate or regression analysis to alter the way information is processed.

[00121] In a further embodiment, patent applications and/or office actions may be reviewed by a peer or other third party. For example, a patent attorney that is unaffiliated with a particular case could review an examiner's work and provide feedback in the form of a numeric score and/or comments. Applications could be assigned to patent attorneys who volunteer for such service, or all patent attorneys may be required to review a set number of applications in a specific time period. In some embodiments, the patent attorneys who provide such services may be compensated.

Such compensation may be payment of a fee, reduction of patent office fees for the applications that patent attorney files, acceleration of examination for that patent attorney's applications, or any combination thereof.

[00122] In additional embodiments an issued patent may be submitted for review by a potential licensee or patent invalidator. Such a request may provide assurance for the licensee as to the validity and/or strength of a particular patent.

[00123] Fig. 1 provides an exemplary system 100 that may be used to provide the embodiment described above. As shown, system 100 may include examiner server 102,

Application server 104 and Patent attorney server 106.

[00124] Examiner server 102 may include programs such as patent examiner profile generation programs 110, and various associated databases such as patent examiner database 112, and patent examiner queue 114.

[00125] Application server 104 may include programs such as patent application profile generation program 120 and patent application assignment program 122 as well as databases such as patent application database 124.

[00126] Patent attorney server 106 may include programs such as patent attorney profile generation program 130, patent application assignment program 132 and payment program 134 as well as patent attorney database 136.

[00127] Applications for examination, whether initial examination or subsequent review, may be submitted by any means possible. In some embodiments, applications may be submitted electronically. In other embodiments, applications may be submitted on paper. In a further embodiment, applications may be submitted by facsimile. All applications may be assigned using the system described above, or such a system may be used for particular classes of applications.

[00128] Information regarding applications and/or the application themselves may be stored, for example, in patent application database 124. Such a database may include information such as application ID, application data, certified search ID, distinguishing language data, application class, application subclass, end user ID, file date, application score, application length, and application claims. [00129] Once submitted, a profile of the application may be generated, for example using patent application profile generation program 120. A profile score for a patent application may be calculated, for example, using some or all of the following steps:

1. Receive patent application.

2. generate score for patent application based on patent application data.

3. store patent application score.

[00130] The profile for an application may then be compared to the profile for an examiner. Information regarding examiners may be stored, by any means applicable, for example in patent examiner database 112. Patent examiner database 112 may include information such as, but not limited to, patent examiner history which may include information such as, examiner ID, examiner profile, examiner work load, previous examinations, examiner education, examiner experience, examiner training and examiner score.

[00131] A profile for an examiner may be calculated, for example, using patent examiner profile generation program 110. Patent examiner profile generation program 110 may be configured, for example, to use some or all of the following steps:

1. retrieve patent examiner history.

2. retrieve patent applications previously examined by examiner.

3. generate a score for patent examiner based on applications previously examined by examiner and patent examiner history.

4. store patent examiner score. [00132] Once the profile for an examiner has been compiled, the scores of a patent application and the examiner may be compared to locate the examiner with the most relevant experience. An application may be assigned to an examiner using a program such as patent application assignment program 122. Such a program may, for example, be configured to use some or all of the following steps:

1. retrieve patent application score.

2. generate list of patent examiners with scores closest to patent application

3. assign patent application to patent examiner based on score.

[00133] In some embodiments, the current workload of an examiner may be taken into consideration. Such information may be stored, for example in patent examiner queue database 114. Patent examiner queue database 114 may include information such as examiner ID, application ID 1-N, target completion date 1-N, priority fee 1-N, queue score. The current workload of an examiner may be taken into consideration using, for example, some or all of the following steps:

1. Retrieve patent application score.

2. Generate list of patent examiners with scores closest to patent application

3. Retrieve patent examiner queue scores.

4. Assign patent application to patent examiner based on score and queue score.

[00134] In other embodiments, workload may be taken into consideration prior to the selection of examiners with the closes relevance scores. For example, using some or all of the following steps:

1. Calculate examiner queue score.

2. Retrieve patent application score.

3. Retrieve patent examiner relevance scores.

4. Disregard examiners with highest queue score.

5. Assign application to examiner with closest remaining relevance score. [00135] In one embodiment, an application may be assigned to an examiner using some or all of the steps in Fig. 2 wherein when an application is received its class is determined. Once the class is determined, the profiles of examiners in that field are accessed and the relevance scores of those examiners are retrieved. The examiners with the most relevant experience are determined and their queue scores are retrieved. If the examiners have comparable workloads, the queue scores are ignored. If the examiners have differing queue scores, the application is assigned to the examiner with the lowest queue score and the most relevant experience unless there is a significant difference in the relevance scores in which case the queue scores may be disregarded. [00136] In some embodiments, examination of an application may benefit from collaboration. Such a collaborative system may add to the strength of issued patents by providing additional insight and critique. Examined applications may be reviewed by a peer or a third party at any point during the examination process. In some embodiments, an application may be reviewed, initial comments made and the application reassigned to a second examiner for additional input. In other embodiments, an application may be reviewed just prior to issuance of a notice of allowance. In a further embodiment, issued patents may be reviewed and subject to recall during a specific time period, for example during the first six months or the first five years after issuance.

[00137] Additional review may be performed by another examiner or an outside party such as a patent attorney. Some patent attorneys may focus their practice entirely on the review of filed applications or may do so in conjunction to their own practice. A profile of each patent attorney willing to review applications may be compiled, for example, in patent attorney database 136. Patent attorney database 136 may include information such as, but not limited to, patent attorney ID, patent attorney profile, applications profiled, experience, education, years in practice, patent attorney score, previous applications reviewed, timeliness of review, conflicts, current clients, firm affiliation, or any additional information considered relevant to assigning an application to a particular patent attorney.

[00138] A profile for a patent attorney may be calculated, for example, using patent attorney profile generation program 130. Patent attorney profile generation program 130 may be configured, for example, to use some or all of the following steps:

1. Retrieve patent attorney record.

2. Retrieve applications previously filed by patent attorney.

3. Retrieve patent applications previously examined by patent attorney.

4. Generate a score for patent attorney based on applications previously examined and previously filed by patent attorney. 5. Store patent attorney score.

[00139] Once a patent attorney profile score is generated, it may be compared to the score of a particular application. Patent attorneys with the most relevant score may then be assigned an application for review using, for example, patent application assignment program 132.

[00140] In some embodiments, patent attorneys may receive remuneration for applications they review. Remuneration may include payment of a fee, reduction of patent office fees, acceleration of examination of pending applications, or any combination thereof. In other embodiments, review of a certain number of applications in a specific time period may be mandatory. In further embodiments, review may be voluntary. Information regarding the payments received or the type or amount of payment may be calculated, for example, using payment program 134. [00141] Protection of intellectual property is becoming increasingly important in virtual worlds as well as in the real world. Massive multi player online games (MMOGs) or massive multi-player role-playing games (MMORPGs) are computer game which are capable of supporting hundreds, thousands, or millions of players simultaneously. Typically, this type of game is played in a giant persistent world where the game continues playing regardless of whether or not real players are logged in. Players commonly access these games through a network such as the Internet, and may or may not be required to purchase additional software or hardware in order to play the game. Such networks allow for people all over the world to participate and interact with each other in a virtual environment.

[00142] Individuals involved in playing and participating in virtual environments spend a great deal of time and energy developing and executing ideas, concepts, and designs to be used in a virtual environment. Some games give intellectual property rights in the concepts, designs or ideas in a virtual environment, however these games generally lack a systematic system for registering such concepts, designs or ideas. [00143] Fig. 2 provides an exemplary system 200 that may be used to register virtual ideas, concepts and designs as described above. As shown, system 200 may include a patent office server 202, and a game environment server 204. [00144] Patent Office Server 202 may include patent examination and registration program 210, and patent examiner profile generation program 212. Patent Office Server 202 may further include a plurality of databases such as registered patent database 224, a patent examiner queue database 214, and examiner database 226. [00145] Game environment server 204 may include programs such as item creation and patent registration program 216, create item from patent program 218, patent application profile generation program 220 and patent application assignment program 240 as well as various databases such as player database 228, player character database 230, available skills database 232, available NPC database 234, available natural resources database 236, and patent application database 238. [00146] Patents may be used as blueprints for the design and building of any object to be used in a virtual environment. In some embodiments, the types of patents that may be registered and the type of virtual objects that may be built may depend in part on the virtual environment in which the character resides. Each design for a virtual object may generate one or more patents. Patents may contain all or some of the design elements of a concept or may contain a general outline of the object sought to be replicated. In some embodiments, patents may be based on a combination of digital images, information from the design database, other blueprints, and/or computer readable code.

[00147] Each patent application created may be stored, for example, in patent application database 238, which may include information such as creator, game environments in which it may be used, ID, assignee, class, status, content, registration date, expiration date, number of times used, allowable quantity, skills needed, import and export restrictions, and materials needed. Once an application is examiner and registered, it may be stored, for example in registered patent database 224. Registered patent database 224 may include, for example, creator, licensing structure, royalty payments, valuation, game environments in which it may be used, game environments in which it has been used, ID, assignee, class, status, content, registration date, expiration date, number of times used, allowable quantity, skills needed, import and export restrictions, and materials needed

[00148] The registration of a patent application may require examination of the patent application in order to determine that it does not infringe anyone else's patent, and/or to verify that it does not infringe any virtual or real world patents, registrations, copyrights or trademarks. Such an examination may occur by any means applicable, for example through a governing entity or through a patent office. In one embodiment, a patent may be analyzed using various rules based expert systems or genetic algorithms to determine the degree of difference between one design and another. If the degree of difference is not of an adequate percentage, the patent application can be rejected by the patent office system of the game server. Such an analysis may take place, for example, using patent examination and registration program 210.

[00149] In another embodiment, system 200 may be configured to determine if an application can be registered by performing steps such as:

1. Receive an application filing, including player character inventor ID, virtual entity assignee ID, application design, application class and subclass, required resources and skills to assemble the invention in the application, and required other patents to assemble the invention.

2. Compare filed application to existing patents and generate a similarity rating.

3. If similarity rating is greater than allowable threshold, flag application as requiring further examination.

4. If similarity rating is less than allowable threshold, generate a registration number.

5. Assign registration number to application record.

6. Notify owner that application has been registered.

7. Post application in patent office.

[00150] If the system is unable to make a determination as to the originality of an application, a determination may be made by players, characters, or other third parties. [00151] In another embodiment, examination is performed by an examiner. In one embodiment, examiners are hired and governed by laws and rules, such as the laws and rules of the United States of America, the USPTO, Paris Convention for the Protection of Industrial Property, European Patent Convention, European Patent Office, Japan Patent Office, World Intellectual Property Organization, African Regional Intellectual Property Organization, Canadian Intellectual Property Office, Eurasian Patent Organization, European Patent Office, German Patent Office, Indian Patent Office, IP Australia, Japan Patent Office, Organisation Africaine de Ia Propriete Intellectuelle, Irish Patent Office, State Intellectual Property Office of China, Intellectual Property Office of Taiwan, Intellectual Property Office of Singapore, Intellectual Property Office of Philippines, United Kingdom Patent Office or by laws, rules and regulations established by the game manufacturer, one or more players in the game designated for such purpose, any other body elected by the players and/or appointed by the game manufacturer, and/or any other entity that is duly authorized to appoint and/or hire examiners.

[00152] In some embodiments, the role of examiner may be performed by NPCs.

In other embodiments, examiners may be other characters. In a further embodiment, examiners may be players. According to one embodiment, a virtual or real fee can be charged to a player character who wants to register an application. Such fees may be fixed or variable or fixed and variable at different points in the game and may be determined arbitrarily, based on the resources needed to make the object in the application, the complexity of the application, the number of applications the character has previously registered, the availability of examiners, the backlog for registration and/or examination, market prices, real world examination fees, or any combination thereof. In some embodiments, there may be fees paid to accelerate examination. A portion of this fee can be paid to other player characters who are willing to examine the filed application for registerability. In another embodiment, examiners can be volunteers.

[00153] Once submitted, a profile of the application may be generated, for example using patent application profile generation program 220. A profile score for a patent application may be calculated, for example, using some or all of the following steps:

1. Receive patent application.

2. Generate score for patent application based on patent application data.

3. Store patent application score.

[00154] The profile for an application may then be compared to the profile for an examiner. Information regarding examiners may be stored, by any means applicable, for example in patent examiner database 226. Patent examiner database 226 may include information such as, a patent examiner history which may include information such as, but not limited to, examiner ID, examiner profile, examiner work load, previous examinations, examiner ID, examiner class, examiner skills, examiner qualifications, examiner subclass, examiner education, examiner experience, and examiner score. [00155] A profile for an examiner may be calculated, for example, using patent examiner profile generation program 212. Patent examiner profile generation program 212 may be configured, for example, to generate a profile using some or all of the following steps:

1. Retrieve patent examiner history.

2. Retrieve patent applications previously examined by examiner.

3. Generate a score for patent examiner based on applications previously examined by examiner.

4. Store patent examiner score.

[00156] Once the profile for an examiner has been compiled, the scores of a patent application and the examiner may be compared to locate the examiner with the most relevant experience. An application may be assigned to an examiner, for example, using some or all of the following steps:

1. Retrieve patent application score.

2. Generate list of patent examiners with scores closest to patent application

3. Assign patent application to patent examiner based on score.

[00157] In some embodiments, the current workload of an examiner may be taken into consideration. Such information may be stored, for example in patent examiner queue database 214. Patent examiner queue database 214 may include information such as examiner ID, application ID 1-N, target completion date 1-N, priority fee 1-N, queue score. The current workload of an examiner may be taken into consideration using some or all of the following steps:

1. Retrieve patent application score.

2. Generate list of patent examiners with scores closest to patent application

3. Retrieve patent examiner queue scores.

4. Assign patent application to patent examiner based on score and queue score

[00158] A patent application may be assigned to an examiner using a combination of the relevance score and the queue score, or just the relevance score or just the queue score. In some embodiments, applications may be reassigned if an examiner's queue score gets too high. Assignment of an application to a particular examiner may take place using, fur example, patent application assignment program

240.

[00159] Once assigned, an application may be examined. In some embodiments, examination may take place using a combination of examiners and algorithms. For example, the server may select the most closely related patents and present them to an examiner for the final determination regarding registerability. Such a method may use some or all of the following steps:

1. Output a patent that has a similarity rating higher than allowable threshold to a player character.

2. Receive opinion from player character that application can be registered.

3. Generate a registration number.

4. Assign registration number to blueprint record.

5. Notify application owner that application has been registered.

6. Post application in patent office.

[00160] In another embodiment, examination may occur using a method such as that outlined in Fig. 4 where an initial screening is made comparing an application to registered patents. If the threshold of similarity is not exceeded, the application is registered. If the threshold is exceeded, an examiner is selected to review the application and the application to be registered and the similar patents(s) are given to an examiner to the selected examiner. If the examiner disagrees with the assessment, the examiner may register the application. If the examiner agrees with the assessment that the application and patents are too closely related, a rejection may be sent to the creator or other character seeking to register the application. The creator or other character seeking to register the application may then present arguments regarding the differences between the application and patents. If the examiner is convinced, the application may be registered. If the examiner is not convinced, registration may be denied. [00161] Patents may be used to create virtual objects in the virtual environment.

In some embodiments only registered patents may be used. In other embodiments, there may be ways around the registration requirement. In some embodiments, only the creator of the patent or characters designated by the creator of the patent may request objects be made using the patent. When a request is made to construct a virtual object, the requesting character's assets may be inventoried to determine if they possess the necessary materials and skills to make the requested virtual object. In some embodiments, players and/or characters may need to have particular types of accounts in order to assemble objects from patents. Information regarding the character and the player controlling the character may be stored, for example in player database 228 and player character database 230, respectively. Player database 228 may include information such as, but not limited to, player ID, the character(s) controlled by the player, blueprints imported, design concepts, objects created, billing information, account information and personal information. Player character database 230 may include information such as, but not limited to, character ID, player ID, assets, skills, obligations, objects created, objects requested, raw materials, natural resources, attributes, rates for use of skills, patents invented, patents owned, and game environment access.

[00162] If they do not have the necessary materials, the name of a supplier may be requested or proffered In some embodiments, virtual objects may only be constructed with natural resources and raw materials that are found in that game environment. In other embodiments, additional natural resources and raw materials may be brought in from other game environments. Information regarding natural resources and raw materials may be stored, for example in available natural resources database 236. Available natural resources database 236 may include information such as, but not limited to, resource ID, resource descriptor, last market value, maximum allowed, issued to date, remaining to be issued, permit price, available date range, renewability, and resource attributes 1-n.

[00163] Characters may also be inventoried to determine if they have the necessary skills to construct a virtual object. If they do not have the necessary skills, the requesting character may request the game server, an NPC or another character assemble the object. Information regarding the skills and NPCs available in a particular environment may be stored for example, in skill database 232 and NPC database 234 respectively. Skill database 232 which may contain information such as the skill ID, type, conditions for use, available era(s), characters with skills, NPCs with skills, skill levels, and use of skills. NPC database 234 may include information such as NPC ID, type, location, conditions for use, license or permit fee, available eras, costs for use, and skills. In some embodiment, the particular characters or NPCs with the necessary skills may not exist in that game environment.

[00164] Once the necessary patents, materials, skills and NPCs are acquired, an object may need to be assembled. Assembly may take place using any means applicable, for example item creation and patent registration program 216. In one embodiment, a character may only be able to request the formation of virtual objects that they have the ability to assemble. In another embodiment, a player character may only be able to request the formation of virtual objects that they can use. In a further embodiment a player character may request the formation of any virtual object. Virtual objects may be created by any means applicable. For example, in one embodiment, virtual objects may be created using create item from patent program 218.

[00165] Generally, after a patent application is received by a patent office or other examining entity, it is assigned to a technology center, art group, or other subsection of the receiving office which specializes on the particular field of the invention. The application is then more specifically classified and assigned to an Examiner by a senior examiner or other reviewing entity. The Examiner plans a search by identifying the field of search, selecting the tool(s) to perform the search and determining the appropriate search strategy. The field of search includes local and international patents and patent applications as well as nonpatent literature. Given the increasing number of applications as well as the increasing amount of prior art available for searching, it would be advantageous to decrease the amount of examiner time required to perform a search allowing examiners to concentrate on patentability determinations rather than spending a significant amount of time searching. [00166] According to an embodiment of the present disclosure, when a new application is submitted to a patent office or other receiving office for examination, a profile may be generated taking into consideration the field of invention, class, subclass, abstract, claims, inventorship, any additional relevant information or any combination thereof. Such a review may be manual or automated. In some embodiments, a review may be run using a simple table based method, quantitative and qualitative approaches to decision analysis, a rules based system, or artificial intelligence techniques, for example neural net, Bayesian algorithm, genetic algorithms, pattern recognition, expert systems, case based reasoning, fuzzy systems, hybrid intelligent systems, evolutionary computation, concept processing or any combination thereof.

[00167] The review may generate a score for the application which may be used in determining a search strategy. Scores may include factors such as the subject matter of the invention, the class, the area of technology, the newness of the technology, the art cited in the application, the art cited in any parent application, art submitted in an information disclosure statement, other applications filed by the inventor(s), publications by the inventor(s), citations in the publications of the inventor(s), art in other applications filed by the inventor(s), key words in the application, field of the application, any other relevant material, or any combination or subset thereof. [00168] The score of the application may then be compared to the scores of other applications, or to the scores of databases available for searching. In some embodiments, the search strategy or file wrappers for similar applications or applications in the same art may be reviewed to determine appropriate databases or search strategies.

[00169] Once a search strategy has been composed, it may be run in any of a number of databases including databases containing patent documents, foreign patent documents and non-patent literature. Profiles of databases may be determined based on any of a number of criteria such as the field of a particular database, the appropriateness of particular databases, the contents of particular databases, the accuracy of particular databases, how current particular databases are, the number of times other examiners have used particular databases, the breadth of a particular database, the focus of a particular database, the ease of use of a particular database or any combination thereof. Databases may be selected manually or using automated means. In some embodiments, numeric scores for databases may be calculated using simple table based methods, quantitative and qualitative approaches to decision analysis, a rules based system, or artificial intelligence techniques, for example neural net, Bayesian algorithm, genetic algorithms, pattern recognition, expert systems, case based reasoning, fuzzy systems, hybrid intelligent systems, evolutionary computation, concept processing or any combination thereof. Such scores may be relevance scores, or scores for other purposes. [00170] In some embodiments, particular databases may be more appropriate for particular fields of art. For example, in certain fields, most of the inventions may be modifications on prior patented inventions and therefore searches may be limited to patent databases. In other fields, there may be relatively few existing patents and therefore searches may be performed in databases that include non-patent literature. Some of these databases may focus on particular areas of technology, for example, Pubmed was developed by the National Center for Biotechnology Information at the U.S. National Institute for Health. It would therefore be an appropriate database to search regarding technology in medicine, nursing, dentistry, veterinary medicine, the health care system, and the preclinical sciences. Pubmed would, however, receive a low relevance score in relation to an application regarding vehicle engines. [00171] Once the database is selected, the search may be run to locate relevant prior art. In one embodiment, prior art may be scored as relevant using simple table based method, quantitative and qualitative approaches to decision analysis, a rules based system, or artificial intelligence techniques, for example neural net, Bayesian algorithm, genetic algorithms, pattern recognition, expert systems, case based reasoning, fuzzy systems, hybrid intelligent systems, evolutionary computation, concept processing or any combination thereof. Such methods may evaluate multiple criteria including common words and phrases in the patent application and the prior art; the class and sub class of the patent application and the prior art; the amount of time the prior art was considered by the end user or patent examiner on the same or similar patent applications; the notes an end user or patent examiner has attached to the prior art; the number of times the prior art has been cited by other publications in the field; the office actions performed by the system or patent examiners utilizing the same prior art on similar patent applications; the patent application data of the patent application and the data of the prior art; the number of examiners and/or end users who have reviewed or cited the prior art for similar patent applications in the past; the seniority, grade, or ranking of patent examiners who have cited the prior art in similar patent applications; the number of times the prior art data is cited in similar patent applications; the number of times the prior art data is accessed by end users and patent examiners to conduct searches on similar patent applications, or any combination thereof. In some embodiments, the search may be run by the submitter or an external service prior to submission of the application to the patent office. In other embodiments, the search may be run prior to assigning the application to an examiner. [00172] In some embodiments, the elements of any particular score may be weighted. In further embodiments, the weighting of the elements may change depending on the use of the score. For example, the weighting may change depending on the information in the application. With relatively new references, the number of examiners and/or end users who have reviewed or cited the prior art may be discarded as irrelevant, or weighted more heavily if a high number of examiners and/or end users have accessed a new reference. In some embodiments, the score of the prior art may be used to determine its relevance. The results may be compiled in a certified prior art search with a ranked listing of the most relevant prior art. Such prior art searches may be supplemented by manual searches performed by an examiner. In some embodiments, a manual search may be performed initially and then supplemented by an automated search.

[00173] In some embodiments, prior art searches are performed when an application is received by the patent office. In other embodiments, prior art searches may be performed prior to submission of the application to the patent office. In some embodiments the submitter of the application may be required to provide comments to distinguish the invention over the prior art located whether the prior art is identified before or after submission of the application. In further embodiments, a search is run periodically during the pendency of the application to ensure that all potential references have been found, to ensure that no new art has been entered into the system, and to take into consideration any additional comments or references that have been submitted in reference to the application.

[00174] In the event that new prior art is located, the end user can be notified automatically or at the discretion of the examiner. In some embodiments, the end user may be given the opportunity to distinguish the new prior art over the invention prior to receiving a formal rejection. In other embodiments, such commentary may only be provided if an examiner has not yet issued any formal action on the application. Such commentary may be used to refine prior art searches and/or may alter the relevancy scores of the prior art that has already been identified.

[00175] In some embodiments, manual searches may be conducted that supplement or add to the search performed by the automated system. References identified in a manual search that have not been identified in an automated search may be added to the automated search providing feedback and a means for refining the algorithms used to perform automated searches.

[00176] Artificial intelligence systems or neural nets can acquire the necessary tools to perform prior art searches by any means applicable. In some embodiments, such refinement is acquired by reviewing the search strategies of previously filed applications. File wrappers of previous patent applications including prior search strategies can be examined to develop rules for conducting searches on subsequently submitted applications. In another embodiment, techniques such as crossover and mutation may be used to create a self-improving search tool.

[00177] Once an application is filed, it may become prior art for subsequently filed applications. In some embodiments pending applications may be published. In other embodiments, pending applications may be unpublished. Either or both published and unpublished applications may be used as references against a pending application. [00178] When the system determines that an unpublished application is relevant prior art, the end user may be informed that such an application exists. In some embodiments, the end user must rely on the patent examiner to examine the unpublished prior art and notify him if it is an issue for the pending application. In other embodiments, the end user can request that the unpublished application be published. In further embodiments, the end user may request a second examiner compare the unpublished prior art and the currently pending application. In additional embodiments, if unpublished references are used, they could be automatically reviewed by two or more examiners prior to citation in a rejection. In some embodiments, the application and the unpublished reference may be randomly assigned to another examiner in the same art unit. Review by additional examiners may or may not be anonymous. For example, in some embodiments the initial or subsequent examiner may know the identity of the other examiner(s). In other embodiments, the review may be blind. In the event that the review is blind, the examiners may still be able to discuss the application and the unpublished prior art through blind drops, anonymous email, anonymous instant messaging or any other means of communication which would conceal their identities.

[00179] In some embodiments, it may be useful to have reviews or comments on prior art. Such comments may be submitted by examiners, end users submitting the references with an application, or third parties. Comments may concern the relation of the references to a particular application, summaries of the references, general observations of the references, or any combination thereof. In some embodiments such comments may be public. In other embodiments, comments may be available to other examiners. In further embodiments, comments may be accessed by the system in determining the relevance of a reference as prior art against a pending application. [00180] In some embodiments, comments may be reviewed by other inventors, researchers, patent attorneys, the general public, and/or examiners who may challenge the application or comments by submitting their own comments. The value of comments and the weight they receive from the system may depend on the submitter. For instance, comments from submitters who have established a track record of insightful submissions may be given more weight than comments from first time submitters. In other embodiments, comments from established practitioners, judges, professors, retired examiners, experts in a particular field, etc. may be given more weight than comments from the general public. The credentials of a particular commentator could be submitted with the comments and verified by the system or obtained independently, for example through a review of databases or other electronic information. In another embodiment, examiners could evaluate the usefulness of any particular comment and such feedback could increase or decrease the weight of the comment from any particular commentator. Feedback may be used to continuously update the system, update it periodically, or may be used as part of a multivariate or regression analysis to alter the way information is processed.

[00181] In further embodiments, applications may be submitted to the inventors or authors of relevant prior art for review. The comments provided by other inventors or researchers in the field could be used to distinguish the prior art or as the basis for a rejection in a subsequent action issued by the patent office. In some embodiments, any comments submitted regardless of the source could be weighted. Weighting may depend on the number of applications the inventor has filed, the number of articles authored, the journals in which such articles were published, the value of previously submitted comments, the institution with which the inventor or author is associated, the number of applications the examiner has examined, the seniority of the examiner, the number of cases a judge has tried, the education of the examiner, the education of the inventor, the education of the author, any other variable which may effect the quality of the comments by the inventor, or any combination thereof.

[00182] Fig. 5 provides an exemplary system 300 that may be used to provide the embodiment described above. As shown, system 300 may include Examiner server 302, Application server 304 and Search server 306.

[00183] Examiner server 302 may include a review program 314, and a patent examiner profile generator 316. Examiner server 302 may additional included databases such as examiner database 312.

[00184] Application server 304 may include programs such as patent application profile generator 324 and databases such as application database 320 and end user database 322. Search server 306 may include information such as certified search program 330, as well as various databases such as certified search database 332 and prior art database 334.

[00185] Applications for examination, whether initial examination or subsequent review, may be submitted by any means possible. In some embodiments, applications may be submitted electronically. In other embodiments, applications may be submitted on paper. In a further embodiment, applications may be submitted by facsimile. [00186] Information regarding the application received may be stored, for example, in application database 320. Application database 320 may include information such as application ID, application data, certified search ID, distinguishing language data, application class, application subclass, end user ID, file date, application score, application length, and application claims. Such information may be used to compile a profile for the application and a score using, for example, patent application profile generator program 324. A profile score for a patent application may be calculated, for example, using some or all of the following steps:

4. Receive patent application.

5. Generate score for patent application based on patent application data.

6. Store patent application score.

[00187] The score for the patent application may be used in selecting the most appropriate database for selecting the most appropriate databases in which to run a search for prior art as well as in making determinations as to the relevance of prior art. Such searches may be performed before or after an application is formally submitted and or assigned to an Examiner.

[00188] The search system for identifying prior art may use simple table based method, quantitative and qualitative approaches to decision analysis, a rules based system, or artificial intelligence techniques, for example neural net, Bayesian algorithm, genetic algorithms, pattern recognition, expert systems, case based reasoning, fuzzy systems, hybrid intelligent systems, evolutionary computation, concept processing or any combination thereof. Prior art searches may be run in any database available. In some embodiments, searches may be run in a database such as prior art database 334. Prior art database 334 may include information such as prior art ID, prior art type, prior art date, prior art contents, prior art notes, note taker ID, and prior art note dates. [00189] In some embodiments, the certified search program 330 may be programmed to perform searches using file wrapper data from previously examined applications. Such information may be transmitted using some or all of the following steps:

1. Receive File Wrapper Data.

2. Use File Wrapper Data To Train Genetic Algorithm to Conduct Certified Searches.

In another embodiment, the results of a prior art search may be improved by inputting the results of a manual search into the program such as certified search program 330. Such improvements may be made using some or all of the following steps:

1. Output Patent Application Data and Certified Search.

2. Receive Additional Prior Art References.

3. Use Additional Prior Art References to Enhance Genetic Algorithm that created Certified Search.

[00190] Searches may be run before or after an application is assigned. In one embodiment, search as conducted using a program such as certified search program 330. Such searches may be conducted using some or all of the following steps:

1. Receive Patent Application Data.

2. Retrieve Prior Art.

3. Compare Patent Application Data to Prior Art.

4. Identify Relevant Prior Art Documents. 5. Store Relevant Prior Art Documents with Patent Application and Time Stamp. The results of a search may be stored by any means possible. In some embodiments, such information may be stored in certified search database 332. Such a database may include information such as search ID, researcher ID and a list of the prior art references found.

[00191] In some embodiments, a search and first office action may be issued prior to the application being assigned to an Examiner. For example, an automated system could process the application, input the relevant information and output a set of relevant prior art documents. The end user may be required to respond or distinguish the application from the prior art identified by the system prior to further processing of the application or review by an examiner though there may also be a means for appeal. In some embodiments, lesser fees may be applied for the initial search. End users may determine not to pursue the application further based on the results of the initial, automated search. In other embodiments, additional fees may be charged to have the application reviewed by a human examiner.

[00192] In one embodiment, the results of the search may be given along with the application to the appropriate Examiner. Examiners may be assigned by any means applicable. In some embodiments, profiles of examiners are created and the relevancy of their experience is matched to appropriate applications. Information on particular examiners may be stored, for example in patent examiner database 312. Such a database may include information such as examiner ID, examiner profile, examiner work load, previous examinations, examiner education, examiner experience, examiner training and examiner score, or any combination thereof.

[00193] A score for an examiner may be calculated, for example, using patent examiner profile generation program 316. Patent examiner profile generation program 316 may be configured, for example, to use some or all of the following steps:

1. Retrieve patent examiner history.

2. Retrieve patent applications previously examined by examiner.

4. Store patent examiner score. [00194] The score for an examiner may be compared to the score for an application to determine the most appropriate examiner for a particular application. [00195] The results of the search and or any comments by an examiner are sent to the end user. Information regarding the end user may be stored, for example, in the end user database such as end user database 322. Such a database may include information such as end user ID, end user profile, end user billing information, end user correspondence and end user score.

[00196] An end user may respond to the prior art produced by the search as well as any comments by the Examiner. Once a response is received to the first office action, whether generated by the system or by an Examiner, a subsequent search may be run to ensure that all relevant prior art has been found. Particularly in areas of high technology, there may be a lag in the publication of literature and its entry into a database. A refreshing of the search results may be run using, for example, some or all of the following steps:

1. Retrieve Patent Application Data and Certified Search.

2. Compare Patent Application Data to Prior Art Data.

3. Generate list of Relevant Prior Art.

4. Compare list to Certified Search.

5. If list has additional prior art references, store references with Patent Application Data.

6. Output notice to end user that patent application has received additional prior art references.

[00197] In the event that new art is found, the first office action may be supplemented or reissued. Such an issuance may be automated or performed by an Examiner using some or all of the following steps:

1. Receive request to review additional prior art.

2. Output Patent Application Data and Additional Prior Art.

3. Receive Additional Distinguishing Language over Additional Prior Art.

4. Store Additional Distinguishing Language with Patent Application and Time Stamp.

5. Determine Fee for filing Additional Distinguishing Language.

6. Apply Fee to End User Account. [00198] In the event that an examiner discovers additional relevant art, or an end user submits art that was not identified in a prior art search, such information may be added to the certified search results. The addition of these documents may further refine the search criteria, adding to its sophistication and accuracy. Such an addition may take place using some or all of the following steps:

1. Output Patent Application Data and Certified Search.

2. Receive Additional Prior Art References.

3. Store Additional Prior Art References as Part of Certified Search and Time Stamp.

[00199] In some embodiments, the prior art located may be unpublished. In such an event, the end user may request that the prior art be published in order to be used by the Examiner, and/or a second review of the application and the prior art by an additional examiner may be requested or automatically initiated. For example, such additional examination could be initiated using review program 314. Review program 314 may use some or all of the following steps:

1. Receive Request to Examine Patent Application.

2. Ouput Patent Application, Relevant Prior Art, and Distinguishing Language.

3. Receive Response to Distinguishing Language.

4. Receive Additional Relevant Prior Art.

5. Receive Comments on Additional Prior Art.

6. Store Response to Distinguishing Language, Additional Relevant Prior Art and Comments on Additional Prior Art with Patent Application and Time Stamp.

7. If Additional Prior Art Cited was unbpublished, determine a Second Examiner and Place Patent Application in Second Examiner Queue.

8. Output notification to End User that Patent Application is Receiving Second Examination because Additional Prior Art cited by first examiner was unpublished.

[00200] In other embodiments, an additional review of unpublished materials cited against an application must be requested by the end user. Such a request must be submitted by the end user. In some embodiments, such a request may trigger some or all of the following steps:

1. Receive request to review Response to First Office Action. 2. Output Patent Application, Response to Distinguishing Language, Additional Relevant Prior Art, and Comments on Additional Prior Art.

3. Receive Request for Reexamination if Additional Prior Art Cited was Unpublished.

[00201] Subsequent office actions may follow similar patterns with additional searches or comments from the examiner.

[00202] In some embodiments, it may be useful to receive commentary on the prior art as well as the pending application. Such commentary may be supplied by the end user, examiners or third parties. In some embodiments, applications may be published for comment prior to issuance. In other embodiments, examiners who use particular references in rejecting an application may append commentary to the reference such as a summary, the particular relevance of the prior art, or any additional information that may be useful in other searches, or as part of the record of the application being examined. Commentary may also be provided by third parties. Third parties may additionally submit prior art for consideration after reviewing application along with comments regarding its similarity to the prior art. This may be particularly useful in areas of rapidly evolving technology where it may be difficult for an examiner to remain appraised of all of the changes in the art.

[00203] Comments may be ranked or weighted depending on the commentator. Such a weighting may depend on the standing of the commentator and/or the usefulness or frequency of previous comments submitted by that commentator. Commentary may be published or unpublished by the central system and may or may not be accessible to the general public.

[00204] Fig. 6 provides an exemplary system 400 that may be used to provide the embodiment described above. As shown, system 400 may include Examiner server 402,

Application server 404, Search server 406 and Commentary server 408.

[00205] Examiner server 402 may include a patent examiner profile generator

416. Examiner server 402 may additionally include one or more databases such as

Examiner database 412.

[00206] Application server 404 may include programs such as application profile generator 424 and databases such as application database 420 and end user database

422. [00207] Search server 406 may include information such as certified search program 430, as well as myriad databases such as certified search database 432 and prior art database 434.

[00208] Commentary server 408 may include programs such as commentary ranking 444 and commentator ranking 446 as well as databases such as commentary database 440 and commentator database 442.

[00209] Applications for examination, whether initial examination or subsequent review, may be submitted by any means possible. In some embodiments, applications may be submitted electronically. In other embodiments, applications may be submitted on paper. In a further embodiment, applications may be submitted by facsimile. [00210] In some embodiments, applications may be published upon receipt. In other embodiments, applications may remain unpublished until issuance. In the event that applications are published, they may be opened for commentary. Commentary may occur at any point during the application process or may be limited to particular points in the application process. For example, commentary may be submitted just prior to issuance, at the beginning of the application process, after the first office action has been issued, or at any other relevant time period. In some embodiments, commentary may be limited to the claims. In other embodiments, commentary may be limited to the body of the application.

[00211] In some embodiments, commentary may be stored in commentary database 440 which may include information such as application or literature commented on, comments, commentator ID, commentator ranking, comment ranking, usefulness of comment, or any other additional information which would increase the usefulness of the comments received. In other embodiments, information regarding the application received including any commentary may be stored, for example, in application database 420. Application database 420 may include information such as application ID, application data, certified search ID, distinguishing language data, application class, application subclass, end user ID, file date, application score, application length, commentary and application claims. Such information may be used to compile a profile for the application and a score using, for example, patent application profile generator program 424. A profile score for a patent application may be calculated, for example, using some or all of the following steps: 1. Receive patent application.

2. Generate score for patent application based on patent application data.

3. Store patent application score.

[00212] In some embodiments, the application may be sent out for commentary.

The score of the patent application may be used in determining the most appropriate expert or experts to whom the application should be submitted for review. Such experts may volunteer, be elected or appointed to such positions. In some embodiments, such positions may be paid.

[00213] Information regarding commentators may be stored by any means applicable. For example, information may be stored in commentator database 442. Commentator database 442 may include information such as commentator ID, education, experience, inventions, publications, employer, applications filed, applications reviewed, usefulness of previous comments, current research, ranking, or any combination thereof.

[00214] Commentators and or their comments may receive a ranking. For example, such a ranking may be based on commentator education, experience, inventions, publications, employer, applications filed, applications reviewed, usefulness of previous comments, current research, number of previous comments, accuracy of previous comments, or any combination thereof. Such information may be used to score the usefulness of a particular comment or comments from a particular commentator using commentary ranking program 444 and commentator ranking program 446.

[00215] In other embodiments, commentators may be able to submit prior art which may be useful in determining the patentability of the invention. Such information may or may not be reviewed by an examiner for usefulness. The prior art may be stored, for example in prior art database 434 and tagged to be connected with the application. In some embodiments, the end user may be required to distinguish the prior art submitted by the commentator. In other embodiments, the identified prior art may be used to refine search algorithms for prior art searches.

[00216] The score for the patent application may additionally be used to select the most appropriate database for selecting the most appropriate databases in which to run a search for prior art as well as in making determinations as to the relevance of prior art. Such searches may be performed before or after an application is formally submitted and or assigned to an Examiner.

[00217] The search system for identifying prior art may use simple table based method, quantitative and qualitative approaches to decision analysis, a rules based system, or artificial intelligence techniques, for example neural net, Bayesian algorithm, genetic algorithms, pattern recognition, expert systems, case based reasoning, fuzzy systems, hybrid intelligent systems, evolutionary computation, concept processing or any combination thereof. Prior art searches may be run in any database available. In some embodiments, searches may be run in a database such as prior art database 434. Prior art database 434 may include information such as prior art ID, prior art type, prior art date, prior art contents, prior art notes, note taker ID, and prior art note dates. [00218] In some embodiments, the certified search program 430 may be programmed to perform searches using file wrapper data from previously examined applications. Such information may be transmitted using some or all of the following steps:

1. Receive File Wrapper Data.

[00219] In another embodiment, the results of a prior art search may be improved by inputting the results of a manual search, commentary, or third party submissions into a program such as certified search program 430. Such improvements may be made using some or all of the following steps:

1. Output Patent Application Data and Certified Search.

2. Receive Additional Prior Art References and/or comments.

[00220] Searches may be run before or after an application is assigned. In one embodiment, search as conducted using a program such as certified search program 430. Such searches may be conducted using some or all of the following steps:

1. Receive Patent Application Data.

2. Retrieve Prior Art.

3. Compare Patent Application Data to Prior Art. 4. Identify Relevant Prior Art Documents.

5. Store Relevant Prior Art Documents with Patent Application and Time Stamp.

[00221] The results of a search may be stored by any means possible. In some embodiments, such information may be stored in certified search database 432. Such a database may include information such as search ID, researcher ID, commentator ID, examiner ID and a list of the prior art references found.

[00222] In some embodiments, a search and first office action may be issued prior to the application being assigned to an Examiner. For example, an automated system could process the application, input the relevant information and output a set of relevant prior art documents. The prior art documents could be supplemented by information or comments from commentators. The end user may be required to respond or distinguish the application from the prior art identified by the system and respond to the commentator's comments prior to further processing of the application or review by an examiner though there may also be a means for appeal. In some embodiments, lesser fees may be applied for the initial search. End users may determine not to pursue the application further based on the results of the initial, automated search and/or based on the information received from commentators. In other embodiments, additional fees may be charged to have the application reviewed by a human examiner. [00223] In other embodiments, the results of the search may be given along with the application to the appropriate Examiner. Examiners may be assigned by any means applicable. In some embodiments, profiles of examiners are created and the relevancy of their experience is matched to appropriate applications. Information on particular examiners may be stored, for example in patent examiner database 412. Such a database may include information such as examiner ID, examiner profile, examiner work load, previous examinations, examiner education, examiner experience, examiner training and examiner score, or any combination thereof.

[00224] A score for an examiner may be calculated, for example, using patent examiner profile generation program 416. Patent examiner profile generation program 416 may be configured, for example, to use some or all of the following steps:

1. Retrieve patent examiner history.

2. Retrieve patent applications previously examined by examiner. 3. Generate a score for patent examiner based on applications previously examined by examiner and patent examiner history.

4. Store patent examiner score.

[00225] The score for an examiner may be compared to the score for an application to determine the most appropriate examiner for a particular application. [00226] The results of the search and or any comments by an examiner or third party are sent to the end user. Information regarding the end user may be stored, for example, in the end user database such as end user database 422. Such a database may include information such as end user ID, end user profile, end user billing information and end user score.

[00227] An end user may respond to the prior art produced by the search as well as any comments by the Examiner and/or commentator. Once a response is received to the first office action, whether generated by the system or by an Examiner, a subsequent search may be run to ensure that all relevant prior art has been found. Particularly in areas of high technology, there may be a lag in the publication of literature and its entry into a database. A refreshing of the search results may be run using, for example, some or all of the following steps:

1. Retrieve Patent Application Data and Certified Search.

2. Compare Patent Application Data to Prior Art Data.

3. Generate list of Relevant Prior Art.

4. Compare list to Certified Search.

[00228] In other embodiments, an initial search may be run prior to publication.

In such an event, once the end user has responded to the initial search, an application may be opened for commentary. Such commentary may be used to supplement the search and may include additional prior art submitted by commentators. In the event that new art is found, the first office action may be supplemented or reissued. Such an issuance may be automated or performed by an Examiner using some or all of the following steps: 1. Receive request to review additional prior art.

2. Output Patent Application Data and Additional Prior Art.

3. Receive Additional Distinguishing Language over Additional Prior Art.

5. Determine Fee for filing Additional Distinguishing Language.

6. Apply Fee to End User Account.

[00229] In the event that an examiner or commentator discovers additional relevant art, or an end user submits art that was not identified in a prior art search, such information may be added to the certified search results. The addition of these documents may further refine the search criteria, adding to its sophistication and accuracy. Such an addition may take place using some or all of the following steps:

1. Output Patent Application Data and Certified Search.

2. Receive Additional Prior Art References.

[00230] The responses of the end user may overcome the prior art or may result in the generation of a second Office action. A second Office action may be drafted by the preliminary examiner or by a subsequent examiner. A second Office action may be generated using some or all of the following steps:

1. Receive Request to Examine Patent Application.

2. Ouput Patent Application, Relevant Prior Art, and Distinguishing Language.

3. Receive Response to Distinguishing Language.

4. Receive Additional Relevant Prior Art.

5. Receive Comments on Additional Prior Art.

[00231] Any subsequent office actions or reviews may follow similar steps.

[00232] Decentralizing patent examination may decrease costs, increase competitiveness, decrease bureaucracy, and relievo the backlog of applications that is frustrating many patent systems. Such a system may be used to distribute workloads more evenly, evaluate efficiency and accuracy, as well as increase the resources available for evaluating applications and issued patents,

[00233] Decentralizing examination would allow for increased review of applications and a fluidity in the examination process that would be adaptable to increasing and/or decreasing demands on a patent system without the problems created by having a central administration directly recruit, hire and train new examiners. The biggest barrier to decentralizing the examination process is the confidentiality of patent applications. However, since most applications publish within eighteen months of filing, this is becoming less of an issue. In some embodiments, assignment to an outside examiner could be limited until after the application is published. In other embodiments, applications which will not publish may only be examined by centralized examiners. In a further embodiment, all information in an application may be confidential until the application issues. In some embodiments, in the event that an application is unpublished, all inventor or assignee data may be purged from the application before it is assigned to an outside examiner.

[00234] In some embodiments, decentralized examination may take place by directly assigning applications to outside examiners on a contract basis or other flexible means. In other embodiments, outside agencies may be staffed with examiners and contracts may be created between a centralized agency and the outside agency which would be responsible for all hiring and employment related matters. [00235] In some embodiments, entities that wish to examine applications may register with a central office. Entities may submit information regarding the areas of art in which they have qualified examiners, the types of patents they are willing to examine or reexamine, rates, the number of applications they can handle or any other relevant information. In some embodiments, a database may be created of all of the examination entities and their histories. Such information may be used to create a profile of examining entities. This profile may be used to determine the examining entity with the examiners with the most relevant experience, the efficiency of an examining entity, the turnover rate, the appeal rate and the workload of any particular examining entity. Histories of examining entities may include information regarding applications the examining entity has previously examined, current employees, education of current employees, area of specialization, timeliness, number of applications reviewed, results of reviews, number of applications appealed, the results of appeals, current applications being considered, efficiency ratings or any combination thereof.

[00236] In other embodiments, databases may be created of all examiners and their place of employment, whether independent or through an agency. Profiles of specific examiners may include previous applications that the examiner has examined, the patent applications in the examiner's queue for examination, the examiner's efficiency rating, the prior art cited in the previous and/or pending applications, the examiner's education, the examiner's particular area of expertise, the length of time it takes the examiner to examine an application, the examiner's grade or level, the examiner's seniority, the examiner's previous experience, the examiner's training, the number of reissues in previously examined applications, the number of reviews, the results of reviewed applications, the number of appeals filed, the results of appealed applications, or any combination thereof. In some embodiments, an examiner's effectiveness may be evaluated and included in a profile. Effectiveness may be determined by any combination of variables such as the speed of the examiner, the accuracy of the examination, the number of times a ruling is appealed or overturned, the number of times an application is invalidated, ranking submitted by applicants, peers, and other interested parties, or any combination thereof. In some embodiments, all examiners may be required to be certified by a centralized agency. In other embodiments, training may be undertaken by the examining entities. The levels of certification and/or the types of certification may be included in the profile of each examiner.

[00237] In some embodiments, the elements of a profile may be assigned a number and a total score for an examining entity and/or examiner may be generated. In some embodiments, the elements may be weighted. In further embodiments, the weighting of the elements may change depending on the use of the score. For example, the weighting may change depending on the information in the application. Some areas of technology may require more outside knowledge than others. In those instances, the examiner's previous education may carry more weight than other elements of a profile. In technology areas that require less specific knowledge, other elements of the profile may carry more weight. In some embodiments, the score generated may be a relevance score relating to the examiner's area of expertise in relation to a particular application. [00238] When a new application is submitted to a patent office for examination, the field of invention, class, subclass, abstract, claims, or any combination thereof is reviewed for subject matter and an area of expertise for review is determined. Such a review may be run using a simple table based method, a rules based system, or artificial intelligence techniques, for example neural net, Bayesian algorithm, genetic algorithms, pattern recognition, expert systems, case based reasoning, fuzzy systems, hybrid intelligent systems, evolutionary computation, concept processing or any combination thereof.

[00239] In some embodiments, the submitted application is compared to the score generated by the examining entity profile and/or the patent examiner profile. Examining entities and/or examiners whose profiles are comprised of those patent applications and prior art references that are most relevant to the patent application being submitted are given the highest relevance scores.

[00240] In other embodiments, the current queues of examining entities and examiners are reviewed for workload and a queue score is determined. In some embodiments, workload is evaluated prior to determining relevance of experience. For example, examiners may not be eligible to receive an application for examination if the score they receive for their current workload is too high. Workload may be evaluated and a queue score may be calculated based on the number of applications the examiner is currently reviewing, the average length of time the examiner takes to process an application, the expected length of time to process a new application, or any combination thereof. In some embodiments, determinations may be made by the examiner, his peers or the public. In other embodiments, determinations may be made using artificial intelligence and genetic algorithms. In some embodiments, scores from different sources may be aggregated. Such an aggregation may or may not be weighted. In additional embodiments, the workload scores of examiners should be effectively equal. In order to address an imbalance or inequity, applications that are waiting to be examined may be reassigned to other examiners.

[00241] The examining entity or examiner with the most closely related relevance score(s) and the workload score indicating the most availability may then be assigned the application. In some embodiments, such a decision may be made by the central office. In other embodiments, applications may be assigned to a particular examining entity and the examining entity may control distribution to a particular examiner. In some embodiments, the comparison of the relevance score and workload score may be weighted. In other embodiments, for example in particularly specialized fields, the workload score may be disregarded.

[00242] In some embodiments, the system may be designed to receive feedback.

Such feedback may be used to increase the accuracy of the assignment and the weighting of the scoring. For example, the assigned examiner may submit their time estimate for reviewing the application. This time estimate may be shorter or longer than that calculated by the system. In another embodiment, the examiner could determine that he or she does not have the necessary expertise in the field of the application and may request reassignment of the application. An examiner or examining entity could also indicate the appropriateness of the assignment. Such feedback could be incorporated into the algorithm to add to the efficiency of the system. Feedback may be used to continuously update the system, update it periodically, or may be used as part of a multivariate or regression analysis to alter the way information is processed. [00243] Once an application is assigned to an examiner or examining entity, the end user may be informed. In some embodiments, the end user may be informed as to the examining entity or examiner charged with examining the application. In other embodiments, the identity of the examining entity and/or the examiner may be anonymous. The end user may additionally be informed of the number of applications ahead of their application. In some embodiments, applications can be moved ahead in the queue. Acceleration of examination may take place through the payment of additional fees, through the provision of services for the patent office, for example, through a peer review program, or some combination thereof.

[00244] In many research areas, it is not uncommon for researchers to continue to improve their invention. Each improvement or portion thereof may be filed as a separate application or a continuation in part of a currently pending application. In some embodiments, an end user may file a series of continuation-in-part applications prior to the examination of the first application. The filing of the first application may secure the application's place in an examiner's queue. In some embodiments, each subsequent related application may be stored with the first application. When the examiner reaches the parent application in the examination queue, the end user can be notified and may select any of the continuation-in-part applications for examination. The remaining applications may be abandoned, or placed back in the examiner's queue. [00245] During examination, an end user may desire to communicate with an examiner. Such communication may take place by any means possible. In some embodiments, such communication may be in person. In other embodiments, such communication may be in writing, by phone, facsimile, electronic mail, in a chat room, or in any other medium for communication. In some embodiments, the examiner may be anonymous. Anonymity may be preserved through the use of aliases, online chat rooms, anonymous e-mail encryption, or any combination thereof. In some embodiments, the request for real time communication may be submitted to a central system. An end user may supply a list of times available for the communication, or may wait to receive a list of times available from the examiner. A transcript of the interview and/or a summary of the interview may become part of the application file. [00246] During examination, an examiner may issue one or more rejections of the claims in the application in view of the prior art. In some embodiments, one examiner may be responsible for the examination of the application. In other embodiments, once an initial series of rejections has been issued by an examiner, the application may be sent to a second examiner for review or additional commentary. The number of rejections that may be issued may vary depending on the application, the fees paid, the number of claims, a combination thereof, or may be constant.

[00247] In some embodiments, an issued patent may be reexamined. Such a reexamination may take place at the request of the end user, a potential licensee, a patent invalidator, or other third party. In some embodiments, additional prior art may be submitted with the request for review. Such a request may provide assurance for the owner and/or licensee as to the validity and/or strength of a particular patent. In some embodiments, the review is undertaken by the initial examiner. In other embodiments, the review is undertaken by a different examiner. In further embodiments, the application may be assigned to an examiner at a different examining entity then the original examiner. In another embodiment, the application may be assigned to a different examiner at the same examining entity. In further embodiments, such requests may be handled by a central patent examination office. Requests for reexamination may be accompanied by a fee. Such a fee may be constant for all requestors, or may vary depending on the requestor. In some embodiments, there may be graduated fees depending on the number of times an application has been reviewed. Such fees may increase or decrease depending on the number of times an application has been reviewed. In some embodiments, there may be a cap on the fees. In other embodiments, an application may only be reviewed a certain number of times. In further embodiments, each licensee or other third party may request a review. [00248] In the event that the subsequent examination determines that an application is invalid, the patent application may be assigned to an additional examiner for review. If the additional examiner determines that the application is valid, the subsequent examiner is overruled. If the additional examiner determines that the patent is invalid then the initial examiner is overruled. In some embodiments, the subsequent and/or initial examiner may be able to defend their decision regarding patentability. In another embodiment, review of patentability may be made by the courts. In the event a patent is invalidated, such information may be entered into the central system. [00249] In the event that a patent is found to be invalid, a note may be made in the examiner's file. In some embodiments, the examiner may receive a demerit. If a certain number of invalidations occur, an examiner may be required to receive retraining. In some embodiments, an examiner may be prohibited from examining applications in a particular field. In a further embodiment, an examiner may be prohibited from examining applications. In additional embodiments, if an examining entity has a certain number of patents invalidated, they may be barred from receiving more applications.

[00250] If a patent is found to be valid, a note may be made in the examiner's file. Such examiners and/or the examining agency may receive merit bonuses or other recognition for their examination. Such information may additionally be added to the profile of the examiner and/or examining agency.

[00251] In a further embodiment, patent applications and/or office actions may be reviewed by a peer or other third party. Requests for such review may be made by holders of prior art cited against an application, the applicant or end user, the system or a third party. Such requests for review may be sent to other examiners or non- examiners. For example, a patent attorney that is unaffiliated with a particular case could review an examiner's work and provide feedback in the form of a numeric score and/or comments. Applications could be assigned to patent attorneys who volunteer for such service, or all patent attorneys may be required to review a set number of applications in a specific time period. In some embodiments, the patent attorneys who provide such services may be compensated. Such compensation may be payment of a fee, reduction of patent office fees for the applications that patent attorney files, acceleration of examination for that patent attorney's applications, or any combination thereof.

[00252] In some embodiments, such a request for a review may be made by the system as a means of verifying the examination process and ensuring fidelity of applications. Determinations as to which applications to submit for outside review may be based on the disparity in the relevancy scores between the examiner and the application, the examiner's efficiency rating, the complexity, length and technology of the application, the number of office actions received or continuations filed, notes attached to the application, at random, or any combination thereof. In some embodiments, information received from the review may be used for training automated systems used in assigning, searching and distributing applications.

[00253] Fig. 7 provides an exemplary system 500 that may be used to provide the embodiment described above. As shown, system 500 may include central server 502,

End user server 504, Application server 506, Examiner server 508, and Review server

110.

[00254] Central server 502 may include programs such as patent application assignment and reassignment 512, interview program 514, certified search program 518 and various databases such as examination agency database 516.

[00255] End user server 504 may include an end user database 520 and end user application database 522.

[00256] Application server 506 may include programs such as patent application profile generation program 530, patent application management program 534 and patent application database 532. [00257] Examiner server 508 may include programs such as examiner profile generation program 540, as well as various databases such as examiner database 542, and examiner queue database 544.

[00258] Applications for examination, whether initial examination or subsequent review, may be submitted by any means possible. In some embodiments, applications may be submitted electronically. In other embodiments, applications may be submitted on paper. In a further embodiment, applications may be submitted by facsimile. [00259] Information regarding applications and/or the application themselves may be stored, for example, in patent application database 532. Such a database may include information such as application ID, application data, certified search ID, distinguishing language data, application class, application subclass, end user ID, file date, application score, application length, priority, related applications, and application claims.

[00260] Once submitted, a profile of the application may be generated, for example using patent application profile generation program 530. A profile score for a patent application may be calculated, for example, using some or all of the following steps:

1. Receive patent application.

2. Generate score for patent application based on patent application data.

3. Store patent application score.

[00261] The score for a patent application may then be compared to the score for an examination agency. Information on an examination agency may be stored, for example in examination agency database 516. Examination agency database 516 may include information such as agency ID, agency subject matter, the efficiency of an examining entity, the turnover rate, applications the examining entity has previously examined, current employees, education of current employees, area of specialization of examining entities, number of applications reviewed, results of reviews, number of applications appealed, the results of appeals, current applications being considered, efficiency ratings or any combination thereof.

[00262] In some embodiments, an application may be assigned to a specific examiner in an agency by comparing the profile or relevance score of an examiner with the profile of an application. Information regarding examiners may be stored, by any means applicable, for example in patent examiner database 542. Patent examiner database 542 may include information such as, but not limited to, patent examiner history which may include information such as, examiner ID, examiner profile, examiner work load, previous examinations, examiner education, examiner experience, examiner training and examiner score.

[00263] A profile for an examiner may be calculated, for example, using patent examiner profile generation program 540. Patent examiner profile generation program 540 may be configured, for example, to use some or all of the following steps:

1. Retrieve patent examiner history.

2. Retrieve patent applications previously examined by examiner.

4. Store patent examiner score.

[00264] Once the profile for an examiner has been compiled, the scores of a patent application and the examiner may be compared to locate the examiner with the most relevant experience. An application may be assigned to an examiner by a central office or by the examining agency. In some embodiments, applications may be assigned using patent application assignment and reassignment program 512. Such a program may, for example, be configured to use some or all of the following steps:

1. Retrieve patent application score.

2. Generate list of examiners with scores closest to patent application.

3. Assign patent application to examiner based on score.

The location of each application and its status may be tracked using, for example, patent application management program 534.

[00265] In some embodiments, the current workload of an examiner may be taken into consideration. Such information may be stored, for example in patent examiner queue database 544. Patent examiner queue database 544 may include information such as examiner ID, application ID 1-n, target completion date 1-n, priority fee 1-n, queue score. The current workload of an examiner may be taken into consideration using, for example, some or all of the following steps:

1. Retrieve patent application score.

2. Generate list of patent examiners with scores closest to patent application 3. Retrieve patent examiner queue scores.

4. Assign patent application to patent examiner based on score and queue score.

[00266] In other embodiments, workload may be taken into consideration prior to the selection of examiners with the closes relevance scores. For example, using some or all of the following steps:

1. Calculate examiner queue score.

2. Retrieve patent application score.

3. Retrieve patent examiner relevance scores.

4. Disregard examiners with highest queue score.

5. Assign application to examiner with closest remaining relevance score. [00267] Assignment of an application may be done by a central office or by an examining agency or entity. In some embodiments, the current workload of an examining entity may be taken into consideration by the central office prior to assigning an application or group of applications to the examining entity. In one embodiment, applications may be assigned to an examining agency using some or all of the steps in Fig. 8. As seen in Figure 8, an application is received by a central office. Its class and subclass is determined and an application score is generated. An application score may be generated manually, or using a variety of automated means including, but not limited to, simple table based method, a rules based system, or artificial intelligence techniques, for example neural net, Bayesian algorithm, genetic algorithms, pattern recognition, expert systems, case based reasoning, fuzzy systems, hybrid intelligent systems, evolutionary computation, concept processing or any combination thereof.

[00268] Once an application is assigned, an end user may be notified.

Information regarding an end user may be stored, for example in end user database 520. Such a database may include information such as end user ID, assignee ID, end user profile, end user score and end user billing information. Information regarding the applications an end user has filed may be stored, for example in end user application database 522. End user application database 522 may include information such as end user ID and application ID. In some embodiments, an end user may be informed regarding the number of applications in the queue ahead of the end user's application, for example using information from examiner queue database 544. In other embodiments, the end user may be informed of the examiner and/or examining entity examining the application. In further embodiments, such information may be kept confidential.

[00269] During the examination process, the end user may seek to discuss the application with the examiner. Such a discussion may take place using, for example interview program 514. Interview program 514 may use some or all of the following steps to schedule the interview:

1. Receive request for an interview from an end user for a patent application.

2. Determine available times for examiner.

3. Output available times to end user.

4. Receive acceptance of a time.

5. Initiate interview at time specified.

[00270] In the event that the identity of the examiner is to be kept secret, interview program 514 may facilitate anonymous discussions between the end user and the examiner. For example, such discussions may be facilitated using some or all of the following steps:

1. Receive end user log in.

2. Receive examiner log in.

3. Conduct interview.

4. Store interview results.

[00271] In some embodiments summaries of the interview may be kept with the application. In other embodiments, transcripts of the interview may be kept with the application. In further embodiments, both a summary and a transcript of the interview may be kept with the application.

[00272] Searches of prior art may be run before or after an application is assigned. In one embodiment, a search can be conducted using a program such as certified search program 518. Such searches may be conducted using some or all of the following steps:

1. Receive Patent Application Data.

2. Retrieve Prior Art.

5. Store Relevant Prior Art Documents with Patent Application and Time Stamp.

[00273] In some embodiments, the results of searches may be stored with an application in patent application database 532.

[00274] The results of the search and or any comments by an examiner are sent to the end user. Information regarding the end user may be stored, for example, in the end user database such as end user database 522. An end user may respond to the prior art produced by the search as well as any comments by the examiner. Subsequent rejections may follow similar patterns with additional searches or comments from the examiner until the application is determined to be allowable and issues.

[00275] In some embodiments, it may be desirous to verify the validity and/or strength of a patent. A review system may be used to ensure the high quality of granted patents. In some embodiments, a review system may be used to reassure potential investors, licensees, the owner of the patent, or other third parties. For example, individuals or companies may wish to know the strength of a patent as part of the due diligence process. Such due diligence could be performed in preparation for an initial public offering, or on behalf of a licensee or acquirer of a patent portfolio. Such third parties may wish to have an application reexamined by a subsequent examiner to verify the decisions made by the initial examiner.

[00276] Fig. 9 provides an exemplary system 600 that may be used to provide the embodiment described above. As shown, system 600 may include Central server 602,

End user server 604, Application server 606, and Examiner server 608.

[00277] Central server 602 may include programs such as patent application assignment and reassignment 612, interview program 614, demerit/merit assignment program 616, patent review program 618 and various databases such as examination agency database 614.

[00278] End user server 604 may include an end user database 620 and end user application database 622.

[00279] Application server 606 may include programs such as patent application profile generation program 630, patent application management program 634 and patent application database 632. [00280] Examiner server 608 may include programs such as examiner profile generation program 640, as well as various databases such as examiner database 642, and examiner queue database 644.

[00281] A review system may be initiated to ensure the high quality of granted patents. In some embodiments, a review system may be initiated to review applications just prior to issuance or after issuance. Such a review may be particularly useful in areas of rapidly evolving technology, providing prior art that an examiner may not have located independently. Review may also be requested by patent invalidators, potential licensees, or as part of other due diligence processes. In some embodiments, review may be requested by the owner of the patent who may have identified a new piece of prior art they wish to have considered.

[00282] Information regarding applications and/or the application themselves may be stored, for example, in patent application database 632. Such a database may include information such as application ID, application data, certified search ID, distinguishing language data, application class, application subclass, end user ID, file date, application score, application length, priority, related applications, prior art, application claims, and date of issuance.

[00283] In some embodiments, requests for review may be submitted by the system. Such requests may be based on a combination of variables such as the difference in the relevancy score between the examiner and the application, the examiner's efficiency rating, the examiner's experience level, the complexity of the application, the newness of the technology, the length of the application, the number of amendments made to the claims, the prior art, the extent of notes by the examiner, randomly, or any combination thereof.

[00284] In other embodiments, holders of a patent may identify new prior art that they wish to have considered. In further embodiments, requests may be submitted by third parties such as potential purchasers or licensees.

[00285] Requests for review may require the payment of additional fees. Such fees may be fixed or variable. In some embodiments, the fees may increase or decrease depending on the number of times the patent has been reviewed and/or the type of requestor. For example, requests from the end user or applicant may be more expensive then requests from a potential licensee or vice versa. [00286] Once a request for review is received, the history of the patent including prior examiners is retrieved. Such a history may include the file wrapper of the patent as well as any previous reviews or unconsidered prior art that may have been submitted or discovered during the prosecution of the application. In some embodiments, the request for review may be sent to the initial examiner. In other embodiments, the request for review may be sent to a subsequent examiner. For example, in the event the review is a request from the patent owner regarding a newly identified piece of prior art, the request may be sent to the initial examiner. In the event that the request is from a third party regarding the validity of the application, the request may be sent to a subsequent examiner. Decisions regarding who may examine the patent and/or application or management of the review(s) of a patent and/or application may be controlled, for example, using patent application management program 234. [00287] In some embodiments, the profile of a patent may be recalculated, for example using patent application profile generation program 630. Such recalculations may take into account refinements in the profile process, permitting advancement and retraining of the methods used to create the profile including, but not limited to, automated means including, but not limited to, simple table based method, a rules based system, or artificial intelligence techniques, for example neural net, Bayesian algorithm, genetic algorithms, pattern recognition, expert systems, case based reasoning, fuzzy systems, hybrid intelligent systems, evolutionary computation, concept processing or any combination thereof. A profile score for a patent may be calculated, for example, using some or all of the following steps:

1. Receive patent for review.

2. Generate score for patent based on patent data.

3. Store patent score.

[00288] In some embodiments, the request for review may be sent to the same examination agency. In other embodiments, requests for review must be sent to a different examination agency. In some embodiments, there may be agencies that specialize or which only perform reviews of issued patents.

[00289] In order to determine the appropriate examination agency, the score for a patent may be compared to the score for an examination agency. Information on an examination agency may be stored, for example in examination agency database 614. Examination agency database 614 may include information such as agency ID, agency subject matter, the efficiency of an examining entity, the turnover rate, applications the examining entity has previously examined, current employees, education of current employees, area of specialization of examining entities, number of applications appealed, the results of appeals, current applications being considered, efficiency ratings or any combination thereof.

[00290] In some embodiments, a patent may be assigned to a specific examiner in an agency. For example, in one embodiment, only senior examiners may perform such a review. Information regarding examiners may be stored by any means applicable, for example in patent examiner database 642. Patent examiner database 642 may include information such as, but not limited to, patent examiner score, and patent examiner history which may include information such as, examiner ID, examiner profile, examiner work load, previous examinations, examiner education, examiner experience, and examiner training.

[00291] A profile for an examiner may be calculated, for example, using patent examiner profile generation program 640. Patent examiner profile generation program 640 may be configured, for example, to use some or all of the following steps:

1. Retrieve patent examiner history.

2. Retrieve patent applications previously examined by examiner.

4. Store patent examiner score.

[00292] Once the profile for an examiner has been compiled, the scores of the patent and the examiners may be compared to locate the examiner with the most relevant experience. A patent may be assigned to an examiner by a central office or by the examining agency. In some embodiments, applications may be assigned using patent application assignment and reassignment program 610. Such a program may, for example, be configured to use some or all of the following steps:

1. Retrieve application score of patent.

2. Generate list of examiners with scores closest to patent.

3. Retrieve requirements of review.

4. Assign patent to examiner based on score and requirements of review. [00293] In some embodiments, assignment may take place using some or all of the steps in Figure 10. As seen in Figure 10, a request to an application or issued patent reviewed is received by the central office. The application score of a patent is compared to the scores of available examiners. In some embodiments, there may be a conflict that prevents an examiner from reviewing a particular patent. If the examiner(s) does not have a conflict, the workload of the examiner may be taken into consideration. In some embodiments, if an examiner has a large backlog of applications to examine and/or review, they may be removed from consideration for this review. The patent or application may then be assigned to the examiner with the lowest queue score. [00294] Once a patent is assigned for review, the requestor may be notified.

Information on the requestor may be stored, for example in end user database 620. Such a database may include information such as end user ID, inventor ID, assignee ID, end user profile, end user score and end user billing information. Information regarding the applications an end user has filed or requested to be reviewed may be stored, for example in end user application database 622. End user application database 622 may include information such as end user ID and application ID. In some embodiments, an end user may be informed regarding the number of patents or applications in the queue ahead of the end user, for example using information from examiner queue database 644 which may include information such as the number of applications waiting to be examined and the average length of time for examination. In other embodiments, the end user may be informed of the examiner and/or examining entity examining the application. In further embodiments, such information may be kept confidential. [00295] Review of a patent may take place using, for example patent review program 618. Such a program may use some or all of the following steps to perform a review.

1. Receive request to reexamine patent.

2. Determine fee for reexamination.

3. Apply fee to end user account.

4. Determine examiner to conduct reexamination.

5. Place patent application in queue of examiner.

6. Receive reexamination results.

7. Store reexamination results with time stamp. 8. If reexamination results are different than first examination results, determine tie breaker examiner and store patent application in tie breaker examiner queue.

[00296] During the review process, the end user may seek to discuss the patent with the Examiner, for example to specifically indicate or debate areas of concern. Such a discussion may take place using, for example interview program 612. Interview program 612 may use some or all of the following steps to schedule the interview:

1. Receive request for an interview from an end user for a patent.

2. Determine available times for examiner.

3. Output available times to end user.

4. Receive acceptance of a time.

5. Initiate interview at time specified.

In the event that the identity of the examiner is to be kept secret, interview program 612 may facilitate anonymous discussions between the end user and the examiner. For example, such discussions may be facilitated using some or all of the following steps:

1. Receive end user log in.

2. Receive examiner log in.

3. Conduct interview.

4. Store interview results.

[00297] In some embodiments summaries of the interview may be kept with the application and/or patent. In other embodiments, transcripts of the interview may be kept with the application.

[00298] In the event the examiner determines the patent is invalid, a review by an additional examiner may be initiated. The additional examiner may be from the same or a different examining entity then the initial and additional examiners, and may be selected using methods similar to those described above for selecting other examiners. In some embodiments, the tiebreaker examiner must have additional training or experience. For example, in some embodiments, the tiebreaker examiner must be a primary examiner. A review by the additional examiner selected to break the tie may be preformed, for example, using patent review program 618 which may be configured to use some or all of the following steps.

1. Output reexamined patent to tiebreaker examiner. 2. Receive tiebreaker examination results.

3. Store results with Patent Application

4. Output notification to end user of tiebreaker examination results. [00299] If it is determined that a patent is invalid, demerits may be given to the initial examiner. If the demerits exceed a certain number, the examiner may be required to receive retraining or may be prevented from examining applications in a particular field. The awarding of demerits may be included in the examiner's history and become part of the examiner's profile. In some embodiments, demerits may be awarded using demerit/merit assignment program 616. Demerit/merit assignment program 616 may use some or all of the following steps:

1. If reexamination and/or tiebreaker examination generate different results from first examination, retrieve examiner record of the first examination.

2. Determine demerit score.

3. Apply demerit score to examiner record.

[00300] If the patent is upheld, demerits may be awarded to the subsequent examiner and a merit award may be given to the initial examiner. The awarding of merit may be included in the examiner's history and become part of the examiner's profile. In some embodiments, merit may be awarded using demerit/merit assignment program 616. Demerit/merit assignment program 616 may use some or all of the following steps:

1. If reexamination validates first examination, retrieve examiner record of the first examination.

2. Determine merit score

3. Apply merit score to examiner record

[00301] In some embodiments, there may be an appeal from the decision of the additional examiner, for example through the court system.

[00302] In some embodiments, examination of an application may benefit from peer review. Such review may be by another examiner, or by a third party. In other embodiments, it may be useful to have reviews or comments on prior art. Such comments may be submitted by examiners, end users submitting the references with an application, or third parties. Comments may concern the relation of the references to a particular application, summaries of the references, general observations of the references, or any combination thereof. In additional embodiments, reviews of applications and/or office actions may be undertaken by patent attorneys or agents.

[00303] Fig. 11 provides an exemplary system 700 that may be used to provide the embodiment described above. As shown, system 700 may include central server

702, End user server 704, Application server 706, examiner server 708, and review server 710.

[00304] Central server 702 may include programs such as patent application assignment and reassignment 712, application review program 718, certified search program 714 and various databases such as examination agency database 716.

[00305] End user server 704 may include an end user database 720 and end user application database 722.

[00306] Application server 706 may include programs such as patent application profile generation program 730, patent application management program 734 and patent application database 732.

[00307] Examiner server 708 may include programs such as examiner profile generation program 740, as well as various databases such as examiner database 742, and examiner queue database 744.

[00308] Review server 710 may include various programs such as patent attorney profile generator program 750, payment program 754, patent application assignment program 756 and databases such as patent attorney database 752.

[00309] In some embodiments, examination of an application may benefit from collaboration. Such a collaborative system may add to the strength of issued patents by providing additional insight and critique. Examined applications may be reviewed by a peer or a third party at any point during the examination process. In some embodiments, an application may be reviewed, initial comments made and the application reassigned to a second examiner for additional input. In other embodiments, an application may be reviewed just prior to issuance of a notice of allowance. In a further embodiment, issued patents may be reviewed and subject to recall during a specific time period, for example during the first six months or the first five years after issuance.

[00310] When applications are received, they are processed and a profile is generated, for example using patent application profile generation program 730. A profile score for a patent application may be calculated, for example, using some or all of the following steps:

1. Receive patent application.

2. Generate score for patent application based on patent application data.

3. Store patent application score.

[00311] Patent application data may be stored, for example, in patent application database 732. Such a database may include information such as application ID, application data, certified search ID, distinguishing language data, application class, application subclass, end user ID, file date, application score, application length, priority, related applications, and application claims.

[00312] The score for a patent application may then be compared to the score for an examination agency. Information on an examination agency may be stored, for example in examination agency database 716. Examination agency database 716 may include information such as agency ID, agency subject matter, the efficiency of an examining entity, the turnover rate, applications the examining entity has previously examined, current employees, education of current employees, area of specialization of examining entities, number of applications appealed, the results of appeals, current applications being considered, efficiency ratings or any combination thereof. [00313] In some embodiments, an application may be assigned to a specific examiner in an agency by comparing the profile or relevance score of an examiner with the profile of an application. Information regarding examiners may be stored, by any means applicable, for example in patent examiner database 742. Patent examiner database 742 may include information such as, but not limited to, examiner score and patent examiner history which may include information such as, examiner ID, examiner profile, examiner work load, previous examinations, examiner education, examiner experience, and examiner training. A profile for an examiner may be calculated, for example, using patent examiner profile generation program 740.

[00314] Once the profile for an examiner has been compiled, the scores of a patent application and the examiner may be compared to locate the examiner with the most relevant experience. An application may be assigned to an examiner by a central office or by the examining agency. In some embodiments, applications may be assigned using patent application assignment and reassignment program 712. [00315] Once an application is assigned, an end user may be notified.

Information regarding an end user may be stored, for example, in end user database 720. Such a database may include information such as end user ID, assignee ID, end user profile, end user score and end user billing information. Information regarding the applications an end user has filed may be stored, for example in end user application database 722. End user application database 722 may include information such as end user ID and application ID. In some embodiments, an end user may be informed regarding the number of applications in the queue ahead of the end user, for example using information from examiner queue database 744 which may include information such as examiner ID, number of applications waiting to be processed and average length of time for examination. In other embodiments, the end user may be informed of the examiner and/or examining entity examining the application. In further embodiments, such information may be kept confidential.

[00316] Searches of prior art may be run before or after an application is assigned. In one embodiment, search as conducted using a program such as certified search program 714. Such searches may be conducted using some or all of the following steps:

1. Receive Patent Application Data.

2. Retrieve Prior Art.

3. Compare Patent Application Data to Prior Art.

4. Identify Relevant Prior Art Documents.

5. Store Relevant Prior Art Documents with Patent Application and Time Stamp.

[00317] In some embodiments, the results of searches may be stored with an application in patent application database 732.

[00318] The results of the search and or any comments by an examiner are sent to the end user. Information regarding the end user may be stored, for example, in the end user database such as end user database 722. An end user may respond to the prior art produced by the search as well as any comments by the Examiner. Subsequent rejections may follow similar patterns with additional searches or comments from the examiner until the application is determined to be allowable and issues. [00319] In some embodiments, an application or a patent may be flagged for additional review using for example application review program 718. Additional review may be performed by another examiner or an outside party such as a patent attorney. Some patent attorneys may focus their practice entirely on the review of filed applications or may do so in conjunction to their own practice. A profile of each patent attorney willing to review applications may be compiled, for example, in patent attorney database 752. Patent attorney database 752 may include information such as, but not limited to, patent attorney ID, patent attorney profile, applications profiled, experience, education, years in practice, patent attorney score, previous applications reviewed, timeliness of review, conflicts, current clients, firm affiliation, or any additional information considered relevant to assigning an application to a particular patent attorney.

[00320] A profile for a patent attorney may be calculated, for example, using patent attorney profile generation program 750. Patent attorney profile generation program 750 may be configured, for example, to use some or all of the following steps:

1. Retrieve patent attorney record.

2. Retrieve applications previously filed by patent attorney.

3. Retrieve patent applications previously examined by patent attorney.

4. Generate a score for patent attorney based on applications previously examined and previously filed by patent attorney.

5. Store patent attorney score.

[00321] Once a patent attorney profile score is generated, it may be compared to the score of a particular application. Patent attorneys with the most relevant score may then be assigned an application for review using, for example, patent application assignment program 756. The placement of each patent and its status may be tracked using, for example patent application management program 734. In some embodiments, the reviewer may be sent a summary of the application or patent, or the application or patent itself if it has published in order to run a conflict check. In the event that the reviewer has a conflict, they must refuse the assignment.

[00322] In some embodiments, patent attorneys may receive remuneration for applications they review. Remuneration may include payment of a fee, reduction of patent office fees, acceleration of examination of pending applications, or any combination thereof. In other embodiments, review of a certain number of applications in a specific time period may be mandatory. In further embodiments, review may be voluntary. Information regarding the payments received or the type or amount of payment may be calculated, for example, using payment program 754. [00323] The comments from the review may be viewed by the examiner, the examiner's supervisor, a central authority, or may be made public. In some embodiments, the examiner may use the comments to provide further rejections and/or refine an office action. In other embodiments, the comments from a reviewer may call into question the validity of a patent, requiring reexamination.

[00324] According to various embodiments, the present disclosure provides a system for ex parte or inter partes patent invalidation. In certain embodiments, prior art submissions, especially when a hyperlink or other reference is submitted instead of an actual document, are retrieved by the system, either manually or automatically. The source or referenced hyperlinked document, including any and all relevant documents associated with such source or referenced hyperlinked document may be stored using any suitable means, including, for example, in a time and date stamped and/or encrypted file(s). This allows for the capture and future identification of the state and content of such documents at or nearly at the time of submission or at the earliest opportunity thereafter. Accordingly, in the event that the source document is deleted or modified, the saved record(s) can be accessed to determine the state of the document at the time of submission.

[00325] In certain embodiments, if the source document referenced by a hyperlink cannot be found, validated, imported, or saved, the system may either reject the assertion of the prior art and/or may warn the end user and/or request that the end user supply such missing or invalid files. Alternatively, the system may proceed and only storing the hyperlink or other references, but may also make a recorded note of the fact that the document could not be found, validated and/or imported. In such cases, the patent examiner, for example, may not consider such prior art and/or may only consider it in searching for other similar art and/or may consider such submissions, for example, based upon the reputation and past performance / reliability of the end user submitting such prior art data. [00326] In certain embodiments, existing or new search tools may be provided to permit end users to search for submitted notes, documents, prior art and/or comments. In some cases, end users may desire to use a search engine that has been modified to provide specific features relating to searching for prior art. Exemplary methods for providing patent and prior art searches are disclosed in U.S. Patent Application Nos. 11/671,380, "Automated Patent Searches" filed February 5, 2007; 11/693,555 "Providing Certified Patent Searches Conducted by Third Party Researchers" filed March 29, 2007; and 11/697,447 entitled "Enhanced Patent Prior Art Search Engine," filed April 6, 2007; each of which is hereby incorporated by reference. [00327] In certain other embodiment, additional features and functions may be desirable. Additional methods to create or modify search engines are well known and understood within the prior art and by any person of ordinary skill. For example, methods to design and build a search engine are disclosed and discussed by the authors of the following books, including, for example "Understanding Search Engines: Mathematical Modeling and Text Retrieval (Software, Environments, Tools), Second Edition, by Michael W. Berry and Murray Browne, which is incorporated by reference. Methods to create web pages, hyperlinks and hypertext are well known in the prior art and any person with ordinary skill in the art can design and create such hyperlinks. Methods to design and create hypertext and/or hyperlinks are discussed and disclosed by the authors of the following reference and other materials, including, for example: "Intelligent Hypertext: Advanced Techniques for the World Wide Web (Lecture Notes in Computer Science), by Charles Nicholas and James Mayfield," "Information Architecture for the World Wide Web: Designing Large-Scale Web Sites [ILLUSTRATED], by Louis Rosenfeld (Author), Peter Morville," Creating Web Pages with HTML Simplified, by Sherry Willard Kinkoph (Author)," "Master Visually Web Design (With CD-ROM) by Carrie F. Gatlin and Michael S. Toot," and "Creating Internet Intelligence: Wild Computing, Distributed Digital Consciousness, and the Emerging Global Brain (IFSR International Series on Systems Science and Engineering), by Ben Goertzel." Each of which is hereby incorporated by reference. [00328] In one embodiment, the prior art documents that are submitted, and/or the fact of their submission, cannot be viewed or accessed by anyone except the examiner assigned to the case and/or authorized end users. In other embodiments, the submitted prior art documents, and the fact of their submission, may be made public or made available to only certain parties or individuals. Who is granted access to this information may or may not depend on factors such as whether or not a particular application has published, issued, or otherwise been made public, or whether the application was submitted via an alternative or experimental system, such as a "fast track" or "expedited examination" program.

[00329] In certain embodiments, end users or any person or third party may submit prior art, hyperlinks to prior art, and/or commentary regarding such prior art submissions and/or notes regarding the issued or pending patent application. Exemplary methods to provide attachment of notes into documents and/or associate notes with documents, or words, or other data are disclosed in US Patent Application Nos. 11/690,095 "Facilitating Certified Prior Art Note Taking and Method for Using Same," filed March 22, 2007; 11/697,480 entitled "Note Overlay System," filed April 6, 2007; and 11/697,486 entitled "Document Examiner Comment System," filed April 6, 2007; each of which is incorporated herein by reference. In addition or in the alternate, a database of such submissions may be created and/or a blog may be established for such purposes.

[00330] In some embodiments, a patent examiner or other designated party reviews the application for patentability and/or reviews the prior art documents and/or notes or commentary submitted by end users or any third parties. If the examiner elects to use or cite a prior art document and/or note or commentary to reject a patent application, the first end user or entity and/or others who submitted the prior art and/or notes or commentary may be paid a fee and/or such end user or entity may receive higher priority, respect, or trust when submitting subsequent prior art, notes and/or commentary. By providing a fee or other benefits to end users or entities, an environment conducive and supportive of prior art submissions is thereby established. [00331] In certain embodiments, fees are paid to one or more end users and/or entities that submit prior art, opinions, notes and/or commentary. In come cases, fees are higher or different for those whose prior art, notes and/or commentary are cited, used, or are instrumental in isolation and/or in combination with other prior art, notes and/or commentary, in the examination and/or rejection, and/or invalidation of a pending or issued patent. Such fees may be determined by any applicable means, including, for example, a set fee per occurrence or usage or recitation of any such prior art, note or commentary, and/or variable fees, and/or free market forces, and/or by rule, law or regulation passed or imposed by any duly authorized governing body. For example, a rule or regulation imposed by the USPTO or other governing agency, including, for example, a foreign patent agency, and/or fees may be established via a learning system designed for such purposes.

[00332] For example, a genetic algorithm may be implemented that tests various fee structures for different forms, quantities and qualities of prior art, notes and/or commentary submissions, as the system tries various pricing models, the genetic algorithm may use a "fitness test" to determine if there is any relationship between such fees and the resulting quantity and/or quality of any such prior art, notes and/or commentary submissions. If such a relationship exists, a genetic algorithm could exploit this information to optimize the results and pricing models accordingly. In certain embodiments, fees may be shared among two or more persons, end users, or entities, e.g., the USPTO and a web site hosting company.

[00333] Use and applications of rules based, expert systems and/or genetic algorithms are well known in the prior art and may be implemented using any applicable means. For example, methods to develop rules, expert systems and/or genetic algorithms are discussed and disclosed in various issued and pending patents and reference and other materials, including the following books entitled: "Genetic Algorithms in Search, Optimization, and Machine Learning", by David E. Goldberg, and "An Introduction to Genetic Algorithms," by Melanie Mitchell, and "Expert Systems: Design and Development," by John Durkin," and "Logical Foundations for Rule-Based Systems (Studies in Computational Intelligence)," by Antoni Ligeza, each of which are incorporated herein by reference.

[00334] In certain embodiments, end users that submit prior art may be scored based upon any applicable means, including, for example, on the number of times the prior art and/or notes or commentary they submit is used in a patent application examination and/or is cited and/or is principally or secondarily responsible for issued or pending claims being rejected or invalidated. Such scores may be assigned manually by, for example, patent examiners, and/or via automated means. For example, as, or if, a patent examiner submits an opinion, and such opinion cites any such prior art, notes or commentary as supporting documentation or as a source or foundation of any such opinion, then the system could record any such citation and/or opinion and/or its source or foundation and/or supporting arguments. When storing such information, such a system could also include information about the end user or entity or person that originally or first submitted such materials and/or other information about any one or more end users that submitted such materials. Such scores could be used for a variety of beneficial purposes, including, providing any future examiner(s) with additional information about information supplied by any such end user or entity. This additional information may encourage or discourage an examiner from using the end user's or entities' submissions. For example, if a given end user has proven reliable in providing prior art, notes and/or commentary, patent examiners may be encouraged to review such reliable end user's submissions first and/or exclusively and/or ahead of other less reliable sources. In addition or in the alternate, if a given end user or entity has shown to be generally unreliable, such end user or entities may be barred from further submissions and/or such submissions may be rejected or discounted by the system and/or such poor scores or historical underperformance may be made available to future examiners so that such examiners may opt to not review such materials and/or if such a review is conducted, it may be conducted with benefit of such knowledge of such past poor performance.

[00335] In certain embodiments, scores and/or rankings are determined by any suitable means, including by submissions made by any authorized party, which may include any one or more of: patent examiners, patent attorneys, end users, entities, third parties, peer review committees, organizations or groups created for such purposes, etc. Scores may be recorded individually or in the aggregate, for example, there may be scores assigned by patent examiners that are kept separate from all other scores, and/or patent attorneys may also have a separate score, and/or examiner and attorney scores may be combined or averaged, while layperson scores or rankings may be held separately.

[00336] In another embodiment, all, some, or groups of submitted or calculated scores are held separately, and/or in detail such that subsequent review for such scores and their accuracy can be determined. For example, if a certain end user's prior art turns out to be instrumental in overturning or rejecting a patent, the system could review the scores previously attached to such end user. If the scores accurately reflected the eventual quality or relevancy of such prior art as submitted by end users, then the person or application providing such score or rank may be determined as a more reliable source or application to provide such ranking.

[00337] In other embodiments, scores are only assigned when an outcome is know, i.e., after the end user's submission has either been used or rejected for use by, for example, a patent examiner.

[00338] In yet other embodiments, relevancy scores may be determined, in whole or in part, through the use of automated means. In addition to the novel relevancy ranking methods disclosed herein, other methods to determine relevancy between and among documents and/or websites are well known within the prior art, including, for example, the methods discussed in the book entitled "Text Databases and Document Management: Theory and Practice, by Amita Goyal Chin, which is incorporated by reference.

[00339] In some embodiments, examiners can request and/or pay for submissions from prior art submitters that are based, in part, on their scores. Fees for such requests may be determined by any applicable means, including, free market forces and/or based upon, in whole or in part on the scores. In certain cases, such fees may be shared between or among two or more persons, entities and/or organizations, e.g., the USPTO and the end user that submitted prior art.

[00340] In certain embodiments, end users, e.g., individuals that submit prior art, may have more than one score associated with their submissions and themselves. For example, an individual that submits prior art in more than one field of use may prove more or less reliable depending upon the field of use. Therefore, a database of scores may be established and tracked in order to better understand and predict past and future prior art, notes and/or commentary usefulness and/or relevancy.

[00341] In certain embodiments, whenever an end user submits prior art and/or an examiner or other third party submits an opinion, and/or any other changes, submissions, opinions, notes, prior art documents, hyperlinks, or other data changes, is required, is submitted, is approved, or is altered or should be altered, any one or more affected and/or interested parties may be notified of such events via any applicable means such as via e-mail or an alert. For example, if a third party end user submits prior art to a published pending patent application, the inventor(s) and/or assignees and/or inventor's attorneys may be sent an alert or notice of such submission. [00342] According to an embodiment, an AI system can use the prior art submitted by third party submitters to provide search tool enhancements to a search engine used by examiners and the public when examining and drafting patent applications. An exemplary tool for preparing submitting documents via the Internet is disclosed for example, in U.S. Patent Application No. 11/627,263 "Automated Web- Based Application Preparation and Submission" filed January 25, 2007, which is incorporated herein by reference.

[00343] In certain embodiments, merchants or advertisers may desire to associate one or more advertisements to any patent application or prior art information, including, for example, links or advertisements tied to any one or more of the following, any data, words, figures, images, graphics, icons, etc. Merchants or advertisers may wish to include advertising messages or other marketing content using hyperlinks or other methods. Exemplary methods for including advertising, including charging for such advertising, including the rental, license, purchase or placement of hyperlinks or other applications, modules or other information in such documents are disclosed in U.S. Patent Application Nos. 11/668,586 "Targeted Advertising Based on Invention Disclosures," filed January 30, 2007; 11/668,596, "Keyword Advertising in Invention Disclosure Documents," filed January 30, 2007; 11/697,437 "Merchant Tool for Embedding Advertisement Hyperlinks to Words in a Database of Documents" filed April 6, 2007; and 11/697,443 "Self Teaching Thesaurus" filed April 6, 2007; each of which is hereby incorporated by reference.

[00344] In certain embodiments, before displaying any prior art, patent application, advertisement and/or before presenting a list of prior art, patent applications, including words and/or documents, e.g., from a lexicon of words, and/or any other search results information or data, it may be desirable to ascertain certain additional information about such search request regarding, e.g., prior art, an end user, a patent application, advertisement and/or other request for information. In such cases, the system may determine that it is necessary, desirable or generally useful to present one or more survey questions to aid in determining which prior art, patent applications, words, documents, advertisements, or other information should be presented, e.g.,. to help determine which prior art or advertisement might yield generally better results, and/or which prior art, word or synonym is generally more relevant given the information known about the end user and/or collected by using and/or displaying and/or gathering results from one or more such survey questions. For example, when an end user enters the word "case" into a search tool designed to retrieve prior art relating to use of such word or words, the system might ask the end user the following question or questions: e.g., are you an attorney, are you interested in travel, or are you seeking legal advice. Based upon the end user's response, e.g., if the end user responded in the affirmative to the last question, the system might either provide a definition of "case" to include legal cases, and/or the system may also provide an advertisement for one or more attorneys seeking clients, and/or the system may present prior art that includes the word "case" and/or has relevancy to a patent application within a given field of use, etc. Based upon the response to one or more questions, the system may present additional qualifying questions, i.e., additional questions to further narrow the search results and or the sort display results.

[00345] Exemplary methods to provide for survey questions and gathering of data are disclosed by applicants in U.S. Patent Application Nos. 60/774,177, entitled "Survey Based Qualification of Keyword Searches," 11/278,123, also entitled "Survey Based Qualification of Keyword Searches" 11/562,738 "Survey Based Qualification of Keyword Searches" and 11/608,150, entitled "Map and Inventory Based On-Line Purchases" which applications are incorporated herein by this reference. [00346] According to some embodiments, end users can create a short or other name for a title of a prior art submission. The search engine looks for other prior art documents or submissions that refer to that name and provides links to them as part of the GUI when displaying that prior art document and/or hyperlink. [00347] According to another embodiment, end users can also link articles to a patent application or prior art submission. For example, an interested third party or end user may wish to submit a magazine article to a patent examiner that is presently reviewing a patent application. Such submission may be in the form of a note, which may include a hyperlink to the source materials. [00348] In another embodiment, the system could compare two or more prior art submissions and remove or delete duplicate entries, and/or reference the two and/or sort them such that they appear generally adjacent to one another.

[00349] According to an embodiment, prior art contributed to different sections of a patent can have different relevance weights for subsequent searches by end users and spiders or web crawlers. For example, prior art contained within the prior art and claims sections may carry a generally higher weight or ranking than prior art associated with the abstract section of the patent application.

[00350] According to another embodiment, the number of times a document is downloaded, cross referenced, hyperlinked, or contributed to may have an effect on its search relevance to spiders, web crawlers, end users, and/or to its ranking. For example, if within a single prior art submission, there are five hundred entries, but numerous end users repeatedly review entry numbers: 7, 212, and 327, such three entries may be move up partially to the top of the prior art submission or to the top of the submission and/or such entries may be flagged as potentially being generally more relevant than other less frequently accessed/reviewed entries. The order of such entries may be determined, in whole or in part, by the number of times accessed, the nature or qualifications or other attributes of those accessing such entries, e.g., entries accessed more frequently by a patent examiner may carry more weight than by third parties.

[00351] In certain embodiments, certain end users may have special privileges or restrictions regarding access to or use of the notes system and/or one or more of its features and/or benefits. For example, an Inventor/Assignee/Attorney may have a special log in that allows them to post prior art submissions, which other end users or third parties cannot. The system validates that the end user has a special relationship to the prior art submission, document, action or method stop and/or is otherwise authorized to perform a given task or tasks and provides senior editing and prior art, notes and document adding or other privileges / restrictions to that end user log on. Such privileges and/or restrictions include, but are not limited to, the ability to:

1. Log into the hyperlink, prior art, document or note creation or editing program(s)

2. Access to or the ability to create and/or submit prior art, and/or a hyperlink, source document, opinion or note 3. Make changes or delete a prior art submission, hyperlink, source document, opinion or note.

4. Rank or score a prior art document, hyperlink, opinion, mapping information or note

5. Provide biographical information

6. Provide supporting or detracting information, documents, hyperlinks or notes to any prior art document, hyperlink, document or note or any combination of these.

7. Establish and/or modify and/or submit and/or review rules, regulations or programs that control submission, review and/or approval of any prior art, documents, hyperlinks, rules or other information or programs

8. Create, modify or remove any function or practice any of the invention as disclosed herein and/or by any of the applications incorporated herein by reference.

9. Establish, create or otherwise determine a price or fee or tax to be levied, accrued, charged or otherwise collected relating to any submission of any prior art, document, hyperlink or note or other information, and/or any of the forgoing actions, procedures and/or for granting or accepting any such authorizations or permissions and/or to make use of any document, hyperlink or note.

10. Make, suggest or submit corrections to any of the above

11. Access, or the right to grant the ability to permit others to access, or to grant end users to any of the above-mentioned systems and/or to authorize others to perform or obtain or grant any permissions or authorizations or any combination of the above.

12. Or any combination of the above.

[00352] In another embodiment, the system can permit an end user to attach prior art and/or a note to a group of documents, e.g. patent applications. For example, in addition to providing prior art and/or a note for a single patent application, an end user can select a group of patents and/or sections of that group of patents and provide prior art and/or notes for the group or sections from that group. In this fashion, a single prior art submission and/or note or part of a note can be simultaneously associated with or otherwise linked to more than one document.

[00353] In certain embodiments, all or some prior art, documents, notes, hyperlinks, entries and revisions shall carry a time and date stamp. A time and date stamp and/or change tracking may be encrypted to prevent unauthorized or fraudulent modifications. In such cases, change tracking would permit end users and/or only certain authorized end users to determine the various states of any such prior art, documents, definitions, notes, images, video, audio, documents, or text, etc., at the time of each such change, update or modification. In this fashion, the priority of ideas, e.g., within a patent application may be determined by an authorized end user, e.g., a patent examiner or a court of competent jurisdiction. In addition or in the alternate, such change tracking may retain a copy of the data before and after any such change or update.

[00354] According to another embodiment, spiders or WebCrawlers or other applications that scan prior art, documents, opinions, notes or web pages, can comb through, i.e., examine, digital articles and documents and automatically create notes for a document based on how the document is discussed in other documents and/or notes. [00355] In certain embodiments, end users may desire to search prior art, notes, opinions, words, documents or databases, for example, a patent database, to find relevant prior art, opinions, notes, words, documents, e.g., patents and/or prior art that may require lexicon updates and/or definitions, synonyms and/or antonyms. Exemplary methods for providing patent and prior art searches are disclosed in U.S. Patent Application Nos. 11/671,380, "Automated Patent Searches" filed February 5, 2007; 11/693,555 "Providing Certified Patent Searches Conducted by Third Party Researchers" filed March 29, 2007; and 11/697,447 entitled "Enhanced Patent Prior Art Search Engine," filed April 6, 2007; each of which is hereby incorporated by reference. [00356] In other embodiments, end users may desire to prioritize the processing of their prior art or other submissions, documents, notes, opinions, reviews, commentary or other tasks or items submitted to a queue. In such cases, methods to provide for prioritization may be desirable. Exemplary methods for priority queuing documents are disclosed for example in U.S. Patent Application Nos. 11/462,621, "Fee-Based Priority Queuing for Insurance Claim Processing," filed August 4, 2006; 11/611,024 "System and Method for Prioritizing Items in a Queue" filed December 14, 2006; and PCT Application No. PCT/US06/340347, "Insurance Form Priority Queuing;" each of which are incorporated herein by reference.

[00357] In certain embodiments, surveys, notes, advertisements and hyperlinks may be interchangeable terms, e.g., a survey can be a note, and a note can be a survey or a survey can include an advertisement or vice versa.

[00358] In certain embodiments, a visual representation of relationships and/or relevancy between two or more of: patent applications, prior art, notes, opinions, documents and/or other data, may be created to aid end users in accessing, searching, reviewing or analyzing any such information, documents and/or data. For example, a prior art map may be produced showing, e.g., at the top, the name or ID of a pending patent application, then, a map to all submitted prior art, and/or other documents. Such map may include relevancy information, e.g., more relevant items may be color coded, or a line connecting such more relevant items may have a thicker line or may be sorted or appear closer to the source patent application document, etc. A visual map showing such representations may be made by any applicable means, including methods described herein and/or methods disclosed in US Patent Application No. 11/697,443 entitled "Self-Teaching Thesaurus," filed April 6, 2007.

[00359] In some embodiments, when end users submit prior art, such prior art may undergo a review and/or approval process. Such processes may be manual and/or automated. For example, prior to attaching or associating any prior art to any given patent application or other document(s), such prior art may first require a review by a patent examiner or other designated or authorized third party, and/or such prior art may be accessed, e.g., via a hyperlink, to determine if it exists and if it can be captured, encrypted, imaged and/or stored with an optional time/date stamp. Such approval process may be accomplished via any applicable means, including those disclosed / described herein.

[00360] In certain embodiments, the disclosed invention may be practiced in the real or virtual world. For example, a video game may include a virtual patent office, wherein such patent office may review one or more patent applications, and wherein one or more end users, players, or player characters and/or other third parties, may submit one or more sources, documents, hyperlinks, etc., that may serve as prior art. Exemplary methods and systems for providing protection of intellectual property in a virtual environment are disclosed, for example, in U.S. Patent Application Nos. 11/428,263, "Video Game Environment" filed June 30, 2006; 11/620,563 "Copyright of Digital Works in a Virtual Environment," filed January 5, 2007; 11/689,977, "Digital Rights Management in a Virtual Environment," filed March 22, 2007; 11/671,373 "Video Game with Control of Quantities of Raw Materials" filed February 5, 2007; 11/680,960 "System for the Creation and Registration of Ideas and Concepts in a Virtual Environment," filed March 1, 2007; each of which is incorporated herein by reference. Accordingly, the disclosed invention may be applied to such virtual environment, world or video game(s) or any combination of the forgoing. For example, commentary and opinions and/or scoring, such as those disclosed herein may be created, used and/or delivered in the virtual world. Furthermore, virtual patent examiners (which may or may not be real world patent examiners too), may be used to provide patent opinions regarding a player's or player character's patent application for a virtual object. [00361] The disclosed invention could be also be used for the creation of agreements between or among real or virtual end users, players, player characters or other third parties. In such cases, methods to ensure that agreements are enforceable and that advertising fees are collected in such virtual environments are desirable. Exemplary methods for providing such contract enforcement and collection of fees are disclosed, for example, in U.S. Patent Application Nos. 11/279,991 "Securing Virtual Contracts with Credit," filed April 17, 2006; 11/624,662 "Securing Contracts in a Virtual World," filed January 18, 2007; 11/559158 "Financing Options in a Virtual World" filed November 13, 2006; 11620,542 "Satisfaction of Financial Obligations in a Virtual Environment Via Virtual and Real World Currency," filed January 5, 2007; 11/421,025 "Financial Institutions and Instruments in a Virtual Environment," filed May 30, 2006, and 11/380,489 "Multiple Purchase Options for Virtual Purchases," filed April 27, 2006; each of which are hereby incorporated herein by reference. [00362] In other embodiments, comments, opinions and/or notes may also be used to provide feedback regarding game play, enjoyment, features, desired features, discovered errors, and/or any other form of communication and/or ranking information. [00363] It will be appreciated that all embodiments herein which refer to a patent are equally applicable to a patent application, and vice versa, unless explicitly stated otherwise with respect to a particular embodiment. The references to a patent (or to a patent application) are for reasons of brevity only.

[00364] Those having skill in the art will recognize that there is little distinction between hardware and software implementations. The use of hardware or software is generally a choice of convenience or design based on the relative importance of speed, accuracy, flexibility and predictability. There are therefore various vehicles by which processes and/or systems described herein can be effected (e.g., hardware, software, and/or firmware) and that the preferred vehicle will vary with the context in which the technologies are deployed.

[00365] At least a portion of the devices and/or processes described herein can be integrated into a data processing system with a reasonable amount of experimentation. Those having skill in the art will recognize that a typical data processing system generally includes one or more of a system unit housing, a video display device, memory, processors, operating systems, drivers, graphical user interfaces, and application programs, interaction devices such as a touch pad or screen, and/or control systems including feedback loops and control motors. A typical data processing system may be implemented utilizing any suitable commercially available components to create the gaming environment described herein.

[00366] Accordingly, as shown in Fig. 12, a system 800 may comprise a plurality of various hardware and/or software components such as those described below. It will be appreciated that for ease of description, the variously described hardware and software components are described and named according to various functions that it is contemplated may be performed by one or more software or hardware components within the system. However, it will be understood that the system may incorporate any number of programs configured to perform any number of functions including, but in no way limited to those described below. Furthermore, it should be understood that while, for ease of description, multiple programs and multiple databases are described, the various functions and/or databases may, in fact, be part of a single program or multiple programs running in one or more locations. [00367] Exemplary programs include:

1. Prior Art Submission Program 811

2. Patent Application Filing Program 812 3. Billing and Payment Program 813

4. Prior Art and Searcher Scoring Program 814

5. AI Search Improvement Program 815 [00368] Exemplary database architecture includes:

[00369] Patent Application Database 821, which may be configured to collect, store, and interrelate data such as:

1. Patent Application ID

2. Application Serial Number

3. Pending Application Number

4. Issued Patent Number

5. Patent Applicant Name(s) 1 - N

6. Applicant, i.e., User ID

7. Inventor City

8. Inventor State

9. Inventor Country

10. Assignee Name

11. Assignee City

12. Assignee State

13. Assignee Country

14. Title

15. Abstract

16. Application Date

17. Short Description

18. Long Description

19. Claims

20. Attorney or Agent Name

21. Attorney or Agent ID

22. Specifications / Descriptions

23. Filed of Use

24. Current US Classification

25. Current International Classification

26. Primary Examiner 27. Assistant Examiner

28. Parent Case Information

29. Related Application Data (e.g. US)

30. Referenced By

31. Reissue Data

32. Foreign References

33. Foreign Priority

34. Other References

35. PCT Information

36. Government Interest

37. Application Type

38. Hyperlinks (e.g., document locations) 1 - N

39. Class 1 - N

40. Subclass 1 - N

41. Type 1 - N

42. Subtype 1 - N

43. Additional Date / Time Stamps / Change Tracking Data

44. Submitted / Found / Indexed On

45. Submitted / Found / Indexed By ID or Hyperlink

46. Published y/n

47. Published Date

48. Revised On 1 - N

49. Revised By 1 - N

50. Before Image 1 - N

51. After Image 1 - N

52. Rejected / Issued?

53. Rejected / Issued Date

54. Notes 1 - N

[00370] End User Database 822, which may be configured to collect, store, and interrelate data such as:

1. User ID

2. Name 3. Account Type

4. Description

5. Terms and Conditions ID

6. Credit Card Information

7. Preferred Card Number

8. Preferred Card Holder

9. Preferred Card Type

10. Name

11. Expiration Date

12. Security Code

13. Additional Cards 1 - N

14. Card Number

15. Card Holder (e.g., Bank Name)

16. Card Type (e.g., Visa)

17. Name

18. Expiration Date

19. Security Code

20. Areas of practice / Fields of Use (if attorney) 1 - N

21. Contact Information

22. Qualifications 1 - N

23. Skills 1 - N

24. Current Cases ID 1 - N

25. Prior Cases ID 1 - N

26. Notes 1 - N

27. Relevancy / Performance Data ID 1 - N

28. Relevancy / Performance Data

29. ID

30. Description

31. Type

32. Summary Relevancy Score

33. Detailed Relevancy Transaction Data

34. Transaction ID 35. Transaction Type

36. Document, Patent Application, Note or other ID

37. Score

38. Score Provided By System or ID 1 - N

39. On Date 1 - N

40. Contest Transaction ID

[00371] Prior Art Database 823, which may be configured to collect, store, and interrelate data such as:

1. Prior Art ID

2. Patent Application ID 1 - N

3. Submitted By, i.e., User ID 1 - N

4. Prior Art Hyperlink IDs (e.g., document locations) 1 - N

5. Prior Art Attached Documents, Figures, Images, etc. 1 - N

6. Prior Art Source Descriptions 1 - N

7. Relevancy Scores 1 - N

8. Relevant Prior Art Submissions - Prior Art IDs 1 - N

9. Group 1 - N

10. Class 1 - N

11. Subclass 1 - N

12. Type 1 - N

13. Subtype 1 - N

14. Additional Date / Time Stamps / Change Tracking Data

15. Submitted / Found / Indexed On

16. Submitted / Found / Indexed By ID or Hyperlink

17. Published y/n

18. Published Date

19. Revised On I - N

20. Revised By 1 - N

21. Before Image 1 - N

22. After Image 1 - N

23. Rejected / Issued?

24. Rejected / Issued Date 25. Notes 1 - N

[00372] Group Database 824, which may be configured to collect, store, and interrelate data such as:

1. Group ID

2. Description

3. Includes Sub-Groups / Sub-Class IDs 1 - n

4. Notes 1 - N

5. Class

6. Class ID

7. Description

8. Includes Sub-Class IDs 1 - N

9. Notes 1 - N

10. Sub Class

11. Subclass ID

12. Description

13. Notes 1 - N

14. Note Class

15. Note Class ID

16. Description

17. Includes Sub-Class IDs 1 - N

18. Notes 1 - N

19. Note Subclass

20. Note Subclass ID

21. Description

22. Notes 1 - N

[00373] Hyperlink Database 843, which may be configured to collect, store, and interrelate data such as:

1. Hyperlink ID

2. Description

3. Security Data

4. User ID

5. Password 6. Encryption Key(s) 1 - N

7. Hyperlink (e.g., URL)

8. Submitted by User ID

9. Date Submitted On

10. Date Revised On I - N

11. Description

12. Notes 1 - N

[00374] Notes (opinions and commentary) Database 825, which may be configured to collect, store, and interrelate data such as:

1. Note ID

2. Hyperlinks 1 - N

3. Note Description Short

4. Note Description Long

5. Note Group ID

6. Note Class ID

7. Note Subclass ID

8. Note and/or Note Attachments 1 - N

9. Submitted By ID

10. Original Submission Date / Time

11. Notes 1 - N

12. Modifications 1 - N

13. Submitted By ID

14. Modification Submission Date

15. Short Description

16. Long Description

17. Submitted By ID

18. Original Submission Date / Time

19. Hyperlinks 1 - N

20. Change Images 1 - N

21. Before Change Image

22. After Change Image

23. Notes 1 - N [00375] Billing Database 826, which may be configured to collect, store, and interrelate data such as:

1. Billing Transaction ID

2. Date / Time Stamp

3. Type

4. Billing Method ID

5. Bill From ID 1 - N

6. Bill To ID 1 - N

7. Fee ID 1 - N

8. Amount 1 - N

9. Description 1 - N

10. Transaction Detail IDs 1 - N

11. Terms and Conditions 1 - N

12. Billing Rules 1 - N

13. Fees Database

14. Fee ID

15. Fee Type

16. Description

17. Fee Rules 1 - N

[00376] Examiner Database 827, which may be configured to collect, store, and interrelate data such as:

1. Examiner ID

2. Name

3. Areas of practice / Fields of Use 1 - N

4. Contact Information

5. Qualifications 1 - N

6. Skills 1 - N

7. Current Cases ID 1 - N

8. Prior Cases ID 1 - N

9. Docket Numbers - 1 - N

10. Relevancy / Performance Data ID 1 - N

11. Notes 1 - N [00377] Qualifications Database 828, which may be configured to collect, store, and interrelate data such as:

1. Qualification ID

2. Description

3. Qualification Type

4. Years Experience

5. Fields of Use Applicable 1 - N

6. Notes ID l - N

[00378] Skills Database 829, which may be configured to collect, store, and interrelate data such as:

1. Skill ID

2. Description

3. Type

4. Years Experience

5. Fields of Use Applicable 1 - N

6. Notes ID 1 - N

[00379] Billing Terms and Conditions Database 830, which may be configured to collect, store, and interrelate data such as:

1. Billing Method ID

2. Billing Type

3. Description

4. Billing Frequency

5. Due by # days

6. Late by # days

7. Interest Rate Fixed

8. Interest Rate Variable

9. Interest Accrues after days

10. Notes 1 - N

[00380] Accounts Receivable Database 831 , which may be configured to collect, store, and interrelate data such as: 1. User ID

Total Amount Owed 2. Transaction Detail Records 1 - N

3. Date of Transaction

4. Type

5. Document ID

6. Word ID

7. Hyperlinks 1 - N

8. Amount

9. Notes 1 - N

[00381] Transaction Database 832, which may be configured to collect, store, and interrelate data such as:

1. Transaction ID

2. Description

3. Date / Time

4. Type

5. User ID

6. Examiner ID

7. Transaction Details

8. Document, Patent, Note, Opinions, Prior Art ID 1 - N

9. Relevancy Data

10. Submitted By

11. Submitted On Date / Time

12. Hyperlinks 1 - N

13. Attachments 1 - N

14. Original Image Capture

15. Capture Date / Time

16. Modified Image(s) 1 - N

17. Modifϊcation(s) Submitted By 1 - N

18. Date Submitted 1 - N

19. Before Image 1 - N

20. Before Image Date / Time 1 - N

21. After image 1 - N

22. After Image Date / Time 1 - N 23. Billing T&C s 1 -N

24. Billing Method ID

25. Billing Rules 1 -N

26. Transaction Amount

27. Notes 1-N

[00382] Examiner Rules Database 833, which may be configured to collect, store, and interrelate data such as:

1. RuIeID

2. Rule Description

3. Rules 1-N

4. Notes 1-N

5. Security Rules ID 1 - N

[00383] Billing Rules Database 834, which may be configured to collect, store, and interrelate data such as:

1. RuIeID

2. Rule Description

3. Rules 1-N

4. Notes 1-N

5. Security Rules ID 1 - N

[00384] Fees - Rules Database 835, which may be configured to collect, store, and interrelate data such as:

1. RuIeID

2. Rule Description

3. Rules 1-N

4. Notes 1-N

5. Security Rules ID 1 - N

[00385] Thesaurus Notes Database 836, which may be configured to collect, store, and interrelate data such as:

1. Thesaurus Note ID

2. Notes 1-N

[00386] Alert Event Rules Database 837, which may be configured to collect, store, and interrelate data such as: 1. Alert Event Rule ID

2. Alert Event Description

3. Alert Event Rules 1 - N

4. Event Condition

5. Alert Recipient ID 1 - N

6. Alert Method 1 - N

7. Alert Database ID 1 - N

8. Notes 1 - N

[00387] Search Database 838, which may be configured to collect, store, and interrelate data such as:

1. Document ID

2. Document Location / Hyperlink

3. Notes 1 - N

[00388] Map Database 839, which may be configured to collect, store, and interrelate data such as:

1. Map ID #

2. Description

3. Notes 1 - N

4. Source Document (e.g., prior art) ID #

5. Source Words ID # 1 - N

6. Map references (sources) ID 1 - N

7. Map references (nodes) ID 1 - N

8. Map references (other) ID 1 - N

9. Related Document or Word ID # 1 - N

10. Type ID (e.g., word or document, etc.)

11. Related Document or Word ID # 1 - N

12. Related Document or Word ID Relevancy Score, %, or Rank

13. Map references (sources) ID 1 - N

14. Map references (nodes) ID 1 - N

15. Map references (other) ID 1 - N

16. Notes 1 - N [00389] Alert Database 840, which may be configured to collect, store, and interrelate data such as:

1. Alert Database ID

2. Alert Contents, one or more of:

3. Text

4. Variable Data

5. Executable

6. Notes 1 - N

[00390] Alert Methods Database 841, which may be configured to collect, store, and interrelate data such as:

1. Alert Method ID

2. Method Type

3. Delivery Method (cell phone, pager, e-mail, PDA, database, executable, etc.)

4. Notes 1 - N

[00391] Alert Recipient Database 842, which may be configured to collect, store, and interrelate data such as:

1. Alert Recipient ID (e.g., end user ID)

2. Description

3. Alert Method Preferences ID 1 - N

4. Notes 1 - N

[00392] It will be appreciated that the various software and hardware components described above will be configured to perform a variety of functions and methods. Listed below are some exemplary methods that might be performed by the systems as described herein: [00393] Attach Prior Art to Published Application

1. Receive a request to attach a prior art data to a published document

2. Output prior art attachment form

3. Receive prior art data

4. Store prior art data with published document

5. Review Attached Prior Art

6. Retrieve a document record 7. Retrieve prior art data attached to record

8. Flag prior art data as "reviewed"

[00394] Pay End Users if Prior Art Was Used to Reject Patent

1. Generate/Receive an office action

2. Determine if office action sites prior art data attached to published document

3. Retrieve user payment information associated with prior art data

4. Pay user [00395] Score Prior Art

1. Retrieve prior art data attached to a published document

2. Score prior art data base on relevancy rules

3. Store prior art data with scores

4. Score End User

[00396] Retrieve prior art data submitted by an end user

1. Determine usefulness of data base on rules

2. Score end user based on usefulness of data

3. Store end user score

[00397] Improve Search Engine Based on Prior Art Submissions

1. Retrieve prior art data submissions

2. Apply genetic algorithm to prior art data submissions

3. Enhance genetic algorithm based on prior art data submissions [00398] Event Driven Model

[00399] Load Databases

1. Initially populate or create empty databases

2. Update Databases

[00400] Primary Application / Watchdog

1. Load Database(s)

2. Determine if one or more sub-applications should be executed

3. Execute appropriate sub-applications (see below)

4. Update Database(s)

5. Repeat Process as Necessary / Desired / Indicated [00401] User Interface Application 1. Load database(s)

2. Display graphical user interface for each application / feature as requested / desired

3. Receive input from end users

4. Receive data from sources, e.g., via hyperlinks

5. Execute functions as requested / required and/or load additional applications / GUIs

6. Update databases [00402] Security Application

1. Load Database(s)

2. Determine if requested action and/or end user is permitted

3. If not, notify application and/or end user

4. If yes, permit requested step and/or loading of application or other authorized action(s)

5. Update Database(s) [00403] Opt In / Sign Up Application

1. Load Databases

2. Receiving Indication of new user sign up

3. Record any and all or available information regarding one or more patent applicant's, end users, examiners, attorneys and/or third parties

4. Update databases

[00404] End User Preferences Application

1. Load Databases

2. Present Preferences GUI if required

3. Receive End User Preferences / Feedback / Usage Tracking Information, including: a. Filter Criteria or Rules b. Sort Criteria or Rules c. Relevancy Information d. Weighting Factors, Criteria or Rules e. Security Preferences f. Feedback / Tracking Preferences g. Notes h. Usage habits / patterns i. Display preferences [00405] Create / Maintain Prior Art (Document) Database

1. Load Databases

2. Determine available or participating documents

3. Periodically search all available documents

4. Create / update index for all found (or participating) documents

5. Receive indication of or end user request to add / change / delete prior art

6. If required, queue and review request(s)

7. If required, approved or reject request(s)

8. Create / update prior art document databases

9. Update databases

[00406] Prior Art, Document, Opinion, Notes Search / Indexing Program

1. Load Database(s)

2. Determine Search / Index Procedure is necessary or desired

3. Search World Wide Web or all accessible or participating databases

4. Index Prior Art, Documents, Opinions, Notes and Hyperlinks

5. Store Results

6. Update Database(s)

[00407] Prior Art Document Submission / Filing Application

1. Load Database(s)

2. Receive indication / request to submit prior art document(s) or hyperlink(s)

3. Capture image of all relevant materials, including then current definitions, along with Time / Date stamp information

4. If desired, encrypt any or all output materials, e.g., prior art, notes, patent applications, definitions, words, synonyms, antonyms, figures and/or related documents and/or supporting materials to prevent or otherwise control subsequent access and/or modifications

5. Update Database(s) [00408] End User Contest Application 1. Load Database(s)

2. Receive Indication that one or more end users and/or third parties, e.g., patent examiner, contests one or more prior art submissions, notes, opinions and/or other documents, maps and/or supporting materials

3. Determine relevancy / validity of the contest by any one or all of the following if desired / applicable

4. Solicit other end user / third party votes / scores / ranking

5. Use GA

6. Submission to authorized end user or third party

7. Preponderance of feedback

8. If contest is determined valid, accept requested changes

9. Otherwise reject requested changes

10. Update Database(s)

[00409] Word Search / Indexing Program

1. Load Database(s)

2. Determine Search / Index Procedure is necessary or desired

3. Search World Wide Web or all accessible or participating databases / words

4. Index Words and Hyperlinks

5. Store Results

6. Update Database(s)

[00410] Opinion / Note Attachment Program

1. Load Database(s)

2. Provide Attachment Creation GUI

3. Receive New Opinion / Note from End User, e.g. Examiner

4. Create Opinion / Note

5. Create Opinion / Note Hyperlink

6. Associate Opinion / Note with Document, e.g., patent application and/or prior art, Word and/or Hyperlink (as applicable), by, e.g., inserting or otherwise associating Note Hyperlink with Prior Art, and/or Document, Word and/or Hyperlink

7. Update Database(s) [00411] Opinion / Note Modification Program

1. Load Database(s)

2. Provide Modification GUI

3. Receive Opinion / Note Change / Delete Request from End User, e.g., examiner

4. Create Opinion / Note Modification

5. If required, Create Revised Opinion / Note Hyperlink

6. Associate Revised Opinion / Note with Prior Art or Document, Word and/or Hyperlink, by inserting or otherwise associating Note Hyperlink with Prior Art, Document, Word and/or Hyperlink

7. Else, if required, delete Opinion / Note Hyperlink

8. Update Database(s)

[00412] Opinion / Note Access / Use Program

1. Load Database(s)

2. Provide Access / Use GUI

3. Receive opinion / note access / use / activation request from end user (or application), e.g., patent applicant or attorney

4. Apply Relevancy Filter (if applicable / requested / desired)

5. Determine action steps, e.g., execute program or hyperlink:

6. If applicable, perform one or more of the following:

7. Display appropriate opinion / note contents

8. Display like notes, opinions, prior art, documents or hyperlinks to like documents, and/or words, hyperlinks, etc.

9. Execute program or hyperlink

10. Display opinion, document, prior art, note and/or advertisement

11. If desired / applicable, open new window to display opinion / note contents, prior art or advertisement and/or GUI's

12. Execute opinion / note attachment program

13. Update Database(s)

[00413] Prior Art / Opinion / Note Attachment Program

1. Receive indication of new or modified or deleted prior art / opinion / note

2. Load Database(s) 3. If desired, capture before / after change images

4. Create or update or remove hyperlink(s) as required

5. Update database(s)

[00414] Find Like Prior Art, Opinions, Notes, Documents, Words, Hyperlinks

Program

1. Load database(s)

2. Receive indication any new or modified prior art, opinion, note, document, or hyperlink has been indexed

3. Search for relevant prior art, opinions, notes, documents, or hyperlinks

4. Index results

5. Update database(s)

[00415] Prior Art / Opinion / Note Relevance Program

1. Load Database(s)

2. Receive Relevancy Input from End Users

3. Or use automated application to determine relevancy, e.g., via GA

4. Associate Relevancy with Prior Art / Opinions / Notes and/or Documents

5. Update Database(s)

[00416] Prior Art / Opinion / Note Search Review Program

1. Load Databases

2. Present Search GUI

3. Receive Prior Art, Opinions, Notes or Documents or other Search String Request

4. If desired, needed or requested, retrieve synonyms and display in separate search string box

5. Receive indication that end user prefers or clicks on synonym or other hyperlink

6. Determine if additional information and/or a survey is needed desired

7. If needed or desired, execute survey program

8. Determine if advertisement should be displayed

9. Display advertisement if desired, needed, requested 10. Based upon available information, e.g., search string, synonyms and/or survey results, Search any or all available and/or participating databases and/or data warehouses

11. Retrieve results including opinions, notes, prior art, other documents, synonyms, antonyms, advertisements, notes, hyperlinks, cases, and other search results data based upon any one or more of the forgoing and/or other search criteria

12. Determine weights, sort, filter and other system and/or end user search criteria of end user requesting search

13. Determine relevancy of results text / data / documents, etc. based upon any one or more criteria including: a. Opinion / Note Type, Group, Class or Subclass b. Prior Art, Patent Application, and/or Document Type, Group, Class or Subclass c. User Type d. Security Privileges - Permissions or denials e. User Preferences, weighting criteria f. Computer Type g. Search Engine Type or Provider Preferences h. Relevancy Conditions / Information i. Document results section weighting j. Survey Questions and/or responses k. Past or present end user feedback

14. Determine if results data should be displayed in one or more separate page(s), popup or other window(s)

15. Display results, in whole or in part, based upon relevancy, weighting factors, document section information, and/or in sorted / filtered order and/or store results in certified or encrypted database for subsequent user or examiner or third party access, and/or other available relevancy, sorting, display options criteria

16. Display one or more of the following, in whole or in part, if indicated, requested, needed or otherwise desired including, but not limited to: a. Prior Art b. Patent Application Information c. Opinions d. Notes e. Comments f. Relevancy information g. End user weighting, criteria, sort, filter and/or display and/or other preferences or system settings h. Mapping information i. Synonyms and/or antonyms j. Definitions k. Figures 1. Text m. One or more Documents n. Hyperlinks o. Advertisements p. Or any other search or database results data q. Any or all other data as desired / requested / necessary 17. Update Databases [00417] Feedback and Performance Improvement Application

1. Load Databases

2. Receive indication of end user or system activity

3. Determine if end user feedback is indicated, required, necessary offered or is otherwise submitted or provided

4. Determine feedback category, including any one or more of the following categories/items, including the relevancy, accuracy, usefulness, completeness, effectiveness or appeal of any one or more of the following system settings, and/or data including, but not limited to: a. Prior Art b. Results data c. Opinions d. Notes e. Comments f. Relevancy information g. End user weighting, criteria, sort, filter and/or display or other preferences or system settings h. Mapping information i. Synonyms and/or antonyms j. Definitions k. Figures

1. Text m. One or more Documents n. Hyperlinks o. Advertisements p. Ease of application or feature use q. Any or all other data as desired / requested / necessary

5. Request feedback and/or changes to and/or opinions regarding or relating to one or more affected end users regarding one or more feedback categories as defined / determined above and receive feedback information including at least one or more of the following, including, but not limited to: a. Relevancy rankings b. Scores c. Weighting factors or weights d. Sorting preferences e. Filtering preferences f. Display preferences g. Subjective criteria h. Notes

6. Use on screen feedback option or survey to solicit feedback

7. Receive end user feedback

8. Determine and update relevancy, weighting criteria and/or other scores

9. If feedback warrants, or so indicates, request additional feedback on the feedback 10. Modify applicable / affected criteria including, but not limited to any relevant settings such as those relating to any one or more or part or all of a/an/the: a. Genetic or other learning algorithms b. Relevancy or scoring algorithms c. System, end user and/or other settings, weights, preferences, sort, selection, display criteria. d. End user or system weighting, criteria, sort, filter and/or display and/or other preferences or system settings e. Prior Art f. Mapping information g. Opinions h. Synonyms and/or antonyms i. Definitions j. Figures k. Text

1. Documents m. Hyperlinks n. Advertisements o. Notes p. Any or all other data as desired / requested / necessary

11. Update databases

[00418] Usage Tracking and Optimization Program

1. Load Databases

2. Receive indication of end user or system activity

3. Store / analyze activity

4. When / if requested generate usage tracking / activity reports

5. Display reports and/or export data as requested / desired / needed

6. Determine if activity affects or is related to and/or is otherwise correlated to/with and/or could improve any results data including or system performance, including for example: a. Relevancy and/or scoring calculation methods or algorithms b. Accuracy and/or quality of prior art submissions / opinions / notes / comments, and/or c. Advertising results d. Click through results e. Conversion rates f. End user feedback g. End user skills h. Search methods or algorithms i. Hyperlink use or relevancy j. Sort and/or filter methods, calculations and/or options

7. Provide performance data to genetic or other algorithm(s)

8. Modify methods and/or algorithms and/or end user or other options based upon performance data

9. Update Databases [00419] Billing Program

1. Load Database(s)

2. Receive indication that billing activity has occurred

3. Determine affected parties, e.g., payer and payee

4. Determine billing rules, terms and conditions

5. Determine billing amounts due

6. Create Invoice and A/P or A/R notices / entries

7. Send Invoices and notices

8. Update Databases

9. Await Payment

10. Receive payment indication

11. Apply payments

12. Notify A/P or A/R systems / and/or affected parties

13. Determine if payments are timely / sufficient

14. If not, execute collections program

15. Update Database(s) [00420] Collections Program

1. Receive indication payments are late and/or insufficient 2. Load Database(s)

3. If applicable, execute one or more of the following steps :

4. Send late notice

5. Send insufficient payment or funds notice

6. Limit or prevent further use until payment terms are partially or fully satisfied, each according to billing terms and conditions and/or rules

7. Collect funds due from primary and/or secondary credit cards on file.

8. Notify affected parties

9. Update Database(s) [00421] Mapping Program

1. Load Databases

2. Receive indication that one or more patent applications, prior art, opinions, notes, comments, words, synonyms, antonyms and/or other documents or hyperlinks or notes have been added or changed or removed from one or more databases

3. Receive or determine relevancy information

4. Determine mapping relationships among any one or more of the forgoing

5. Monitor patent application, prior art, notes, opinions, commentary, word, synonym, antonym, and/or other documents and/or mapping usage

6. Receive feedback from end users and/or determine change in mapping relationships and/or relevancy

7. If desired or required, submit any such changes for review / approval

8. If approved, update mapping relationship data accordingly

9. Update Databases [00422] Survey Program

1. Load Databases

2. Receive indicator that relevancy information should be updated and/or search results may be improved with survey results data

3. And/or periodically submit one or more survey questions to one or more end users

4. Determine questions based upon survey database rules and/or based upon prior effectiveness of one or more survey questions 5. Determine respondent or target end users

6. Submit questions to respondent(s)

7. Receive results

8. Determine new relevancy scores

9. Update relevancy information and/or modify hyperlinks, prior art submissions, opinions, performance data, mapping information or advertisements based upon new or revised relevancy scores and/or other end user feedback

10. And/or use GA to determine relevancy scores and/or hyperlink and/or advertisements

11. Update databases [00423] Alerts Program

1. Load Database(s)

2. Determine if Alert Event has occurred

3. Determine Alert Contents based upon alert rules

4. Determine Alert Recipients and Contents and Delivery Method(s)

5. Send Alert(s)

6. Update Database(s)

[00424] It will be appreciated that while, for the sake of discussion, various databases have been described separately, the data in these and any other suitable databases could be merged into a single large databases and/or maintained separately in additional databases, or in other structures besides a database. Moreover, any such databases could be independent or linked, and the data in these databases could be stored centrally on a server or separately on game devices.

[00425] The present disclosure provides numerous systems and methods related to examination systems for real world patent systems as well as patent systems in virtual environments in online computer games. It should be appreciated that numerous embodiments are described in detail and that various combinations and subcombinations of these embodiments are contemplated by the present disclosure. [00426] A reference to "another embodiment" in describing an embodiment does not necessarily imply that the referenced embodiment is mutually exclusive with another embodiment (e.g., an embodiment described before the referenced embodiment), unless expressly specified otherwise.

[00427] The terms "include", "includes", "including", "comprising" and variations thereof mean "including but not limited to", unless expressly specified otherwise.

[00428] The term "consisting of and variations thereof includes "including and limited to", unless expressly specified otherwise. The terms "a", "an" and "the" mean

"one or more", unless expressly specified otherwise.

[00429] The term "plurality" means "two or more", unless expressly specified otherwise.

[00430] The term "herein" means "in this patent application, including anything which may be incorporated by reference", unless expressly specified otherwise.

[00431] The phrase "at least one of, when such phrase modifies a plurality of things (such as an enumerated list of things) means any combination of one or more of those things, unless expressly specified otherwise. For example, the phrase "at least one of a widget, a car and a wheel" means either (i) a widget, (ii) a car, (iii) a wheel, (iv) a widget and a car, (v) a widget and a wheel, (vi) a car and a wheel, or (vii) a widget, a car and a wheel.

[00432] The phrase "based on" does not mean "based only on", unless expressly specified otherwise. In other words, the phrase "based on" describes both "based only on" and "based at least on".

[00433] The term "represent" and like terms are not exclusive, unless expressly specified otherwise. For example, the term "represents" does not mean "represents only", unless expressly specified otherwise. In other words, the phrase "the data represents a credit card number" describes both "the data represents only a credit card number" and "the data represents a credit card number and the data also represents something else".

[00434] The term "whereby" is used herein only to precede a clause or other set of words that express only the intended result, objective or consequence of something that is previously and explicitly recited. Thus, when the term "whereby" is used in a claim, the clause or other words that the term "whereby" modifies do not establish specific further limitations of the claim or otherwise restricts the meaning or scope of the claim.

[00435] The terms "such as", "e.g." and like terms means "for example", and thus does not limit the term or phrase it explains. For example, in the sentence "the computer sends data (e.g., instructions, a data structure) over the Internet", the term "e.g." explains that "instructions" are an example of "data" that the computer may send over the Internet, and also explains that "a data structure" is an example of "data" that the computer may send over the Internet. However, both "instructions" and "a data structure" are merely examples of "data", and other things besides "instructions" and "a data structure" can be "data".

[00436] The term "determining" and grammatical variants thereof (e.g., to determine a price, determining a value, determine an object which meets a certain criterion) is used in an extremely broad sense. The term "determining" encompasses a wide variety of actions and therefore "determining" can include calculating, computing, processing, deriving, investigating, looking up (e.g., looking up in a table, a database or another data structure), ascertaining and the like. Also, "determining" can include receiving (e.g., receiving information), accessing (e.g., accessing data in a memory) and the like. Also, "determining" can include resolving, selecting, choosing, establishing, and the like. It does not imply certainty or absolute precision, and does not imply that mathematical processing, numerical methods or an algorithm process be used. Therefore "determining" can include estimating, predicting, guessing and the like. [00437] It will be readily apparent to one of ordinary skill in the art that the various processes described herein may be implemented by, e.g., appropriately programmed general purpose computers and computing devices. Typically a processor (e.g., one or more microprocessors, one or more microcontrollers, one or more digital signal processors) will receive instructions (e.g., from a memory or like device), and execute those instructions, thereby performing one or more processes defined by those instructions.

[00438] A "processor" may include one or more microprocessors, central processing units (CPUs), computing devices, microcontrollers, digital signal processors, or like devices or any combination thereof. Thus a description of a process is likewise a description of an apparatus for performing the process. The apparatus can include, e.g., a processor and those input devices and output devices that are appropriate to perform the method. Further, programs that implement such methods (as well as other types of data) may be stored and transmitted using a variety of media (e.g., computer readable media) in a number of manners. In some embodiments, hard- wired circuitry or custom hardware may be used in place of, or in combination with, some or all of the software instructions that can implement the processes of various embodiments. Thus, various combinations of hardware and software may be used instead of software only. [00439] The term "computer-readable medium" includes any medium that participates in providing data (e.g., instructions, data structures) which may be read by a computer, a processor or a like device. Such a medium may take many forms, including but not limited to, non-volatile media, volatile media, and transmission media. Nonvolatile media include, for example, optical or magnetic disks and other persistent memory. Volatile media include dynamic random access memory (DRAM), which typically constitutes the main memory. Transmission media include coaxial cables, copper wire and fiber optics, including the wires that comprise a system bus coupled to the processor. Transmission media may include or convey acoustic waves, light waves and electromagnetic emissions, such as those generated during radio frequency (RF) and infrared (IR) data communications. Common forms of computer-readable media include, for example, a floppy disk, a flexible disk, hard disk, magnetic tape, any other magnetic medium, a CD-ROM, DVD, any other optical medium, punch cards, paper tape, any other physical medium with patterns of holes, a RAM, a PROM, an EPROM, a FLASH-EEPROM, any other memory chip or cartridge, a carrier wave as described hereinafter, or any other medium from which a computer can read. [00440] Various forms of computer readable media may be involved in carrying data (e.g. sequences of instructions) to a processor. For example, data may be (i) delivered from RAM to a processor; (ii) carried over a wireless transmission medium; (iii) formatted and / or transmitted according to numerous formats, standards or protocols, such as Ethernet (or IEEE 802.3), SAP, ATP, Bluetooth™, and TCP/IP, TDMA, CDMA, and 3 G; and / or (iv) encrypted to ensure privacy or prevent fraud in any of a variety of ways well known in the art.

[00441] Thus a description of a process is likewise a description of a computer- readable medium storing a program for performing the process. The computer-readable medium can store (in any appropriate format) those program elements which are appropriate to perform the method.

[00442] Just as the description of various steps in a process does not indicate that all the described steps are required, embodiments of an apparatus include a computer / computing device operable to perform some (but not necessarily all) of the described process.

[00443] Likewise, just as the description of various steps in a process does not indicate that all the described steps are required, embodiments of a computer-readable medium storing a program or data structure include a computer-readable medium storing a program that, when executed, can cause a processor to perform some (but not necessarily all) of the described process.

[00444] Where databases are described, it will be understood by one of ordinary skill in the art that (i) alternative database structures to those described may be readily employed, and (ii) other memory structures besides databases may be readily employed. Any illustrations or descriptions of any sample databases presented herein are illustrative arrangements for stored representations of information. Any number of other arrangements may be employed besides those suggested by, e.g., tables illustrated in drawings or elsewhere. Similarly, any illustrated entries of the databases represent exemplary information only; one of ordinary skill in the art will understand that the number and content of the entries can be different from those described herein. Further, despite any depiction of the databases as tables, other formats (including relational databases, object-based models and / or distributed databases) are well known and could be used to store and manipulate the data types described herein. Likewise, object methods or behaviors of a database can be used to implement various processes, such as the described herein. In addition, the databases may, in a known manner, be stored locally or remotely from any device(s) which access data in the database. [00445] Various embodiments can be configured to work in a network environment including a computer that is in communication (e.g., via a communications network) with one or more devices. The computer may communicate with the devices directly or indirectly, via any wired or wireless medium (e.g. the Internet, LAN, WAN or Ethernet, Token Ring, a telephone line, a cable line, a radio channel, an optical communications line, commercial on-line service providers, bulletin board systems, a satellite communications link, or a combination of any of the above). Each of the devices may themselves comprise computers or other computing devices, such as those based on the Intel® Pentium® or Centrino™ processor, that are adapted to communicate with the computer. Any number and type of devices may be in communication with the computer.

[00446] In an embodiment, a server computer or centralized authority may not be necessary or desirable. For example, the present invention may, in an embodiment, be practiced on one or more devices without a central authority. In such an embodiment, any functions described herein as performed by the server computer or data described as stored on the server computer may instead be performed by or stored on one or more such devices.

[00447] Those having skill in the art will recognize that there is little distinction between hardware and software implementations. The use of hardware or software is generally a choice of convenience or design based on the relative importance of speed, accuracy, flexibility and predictability. There are therefore various vehicles by which processes and/or systems described herein can be effected (e.g., hardware, software, and/or firmware) and that the preferred vehicle will vary with the context in which the technologies are deployed.

[00448] At least a portion of the devices and/or processes described herein can be integrated into a data processing system with a reasonable amount of experimentation. Those having skill in the art will recognize that a typical data processing system generally includes one or more of a system unit housing, a video display device, memory, processors, operating systems, drivers, graphical user interfaces, and application programs, interaction devices such as a touch pad or screen, and/or control systems including feedback loops and control motors. A typical data processing system may be implemented utilizing any suitable commercially available components to create the environment described herein.

[00449] Of course it will be appreciated that the systems methods described herein are provided for the purposes of example only and that none of the above systems methods should be interpreted as necessarily requiring any of the disclosed components or steps nor should they be interpreted as necessarily excluding any additional components or steps. Furthermore, it will be understood that while various embodiments are described, such embodiments should not be interpreted as being exclusive of the inclusion of other embodiments or parts of other embodiments. [00450] The invention is described with reference to several embodiments.

However, the invention is not limited to the embodiments disclosed, and those of ordinary skill in the art will recognize that the invention is readily applicable to many other diverse embodiments and applications as are reflected in the range of real world financial institutions, instruments and activities. Accordingly, the subject matter of the present disclosure includes all novel and nonobvious combinations and subcombinations of the various systems, methods configurations, embodiments, features, functions, and/or properties disclosed herein.

[00451] Where a limitation of a first claim would cover one of a feature as well as more than one of a feature (e.g., a limitation such as "at least one widget" covers one widget as well as more than one widget), and where in a second claim that depends on the first claim, the second claim uses a definite article "the" to refer to the limitation (e.g., "the widget"), this does not imply that the first claim covers only one of the feature, and this does not imply that the second claim covers only one of the feature (e.g., "the widget" can cover both one widget and more than one widget). [00452] Each claim in a set of claims has a different scope. Therefore, for example, where a limitation is explicitly recited in a dependent claim, but not explicitly recited in any claim from which the dependent claim depends (directly or indirectly), that limitation is not to be read into any claim from which the dependent claim depends. [00453] When an ordinal number (such as "first", "second", "third" and so on) is used as an adjective before a term, that ordinal number is used (unless expressly specified otherwise) merely to indicate a particular feature, such as to distinguish that particular feature from another feature that is described by the same term or by a similar term. For example, a "first widget" may be so named merely to distinguish it from, e.g., a "second widget". Thus, the mere usage of the ordinal numbers "first" and "second" before the term "widget" does not indicate any other relationship between the two widgets, and likewise does not indicate any other characteristics of either or both widgets. For example, the mere usage of the ordinal numbers "first" and "second" before the term "widget" (1) does not indicate that either widget comes before or after any other in order or location; (2) does not indicate that either widget occurs or acts before or after any other in time; and (3) does not indicate that either widget ranks above or below any other, as in importance or quality. In addition, the mere usage of ordinal numbers does not define a numerical limit to the features identified with the ordinal numbers. For example, the mere usage of the ordinal numbers "first" and "second" before the term "widget" does not indicate that there must be no more than two widgets. [00454] When a single device or article is described herein, more than one device

/ article (whether or not they cooperate) may alternatively be used in place of the single device / article that is described. Accordingly, the functionality that is described as being possessed by a device may alternatively be possessed by more than one device / article (whether or not they cooperate).

[00455] Similarly, where more than one device or article is described herein

(whether or not they cooperate), a single device / article may alternatively be used in place of the more than one device or article that is described. For example, a plurality of computer-based devices may be substituted with a single computer-based device. Accordingly, the various functionality that is described as being possessed by more than one device or article may alternatively be possessed by a single device / article. [00456] The functionality and / or the features of a single device that is described may be alternatively embodied by one or more other devices which are described but are not explicitly described as having such functionality / features. Thus, other embodiments need not include the described device itself, but rather can include the one or more other devices which would, in those other embodiments, have such functionality / features.

[00457] Numerous embodiments are described in this patent application, and are presented for illustrative purposes only. The described embodiments are not, and are not intended to be, limiting in any sense. The presently disclosed invention(s) are widely applicable to numerous embodiments, as is readily apparent from the disclosure. One of ordinary skill in the art will recognize that the disclosed invention(s) may be practiced with various modifications and alterations, such as structural, logical, software, and electrical modifications. Although particular features of the disclosed invention(s) may be described with reference to one or more particular embodiments and / or drawings, it should be understood that such features are not limited to usage in the one or more particular embodiments or drawings with reference to which they are described, unless expressly specified otherwise.

[00458] The present disclosure is neither a literal description of all embodiments of the invention nor a listing of features of the invention which must be present in all embodiments.

[00459] Neither the Title (set forth at the beginning of the first page of this patent application) nor the Abstract (set forth at the end of this patent application) is to be taken as limiting in any way as the scope of the disclosed invention(s). An Abstract has been included in this application merely because an Abstract of not more than 150 words is required under 37 C. F. R. § 1.72(b).

[00460] The title of this patent application and headings of sections provided in this patent application are for convenience only, and are not to be taken as limiting the disclosure in any way.

[00461] Devices that are described as in communication with each other need not be in continuous communication with each other, unless expressly specified otherwise.

On the contrary, such devices need only transmit to each other as necessary or desirable, and may actually refrain from exchanging data most of the time. For example, a machine in communication with another machine via the Internet may not transmit data to the other machine for long period of time (e.g. weeks at a time). In addition, devices that are in communication with each other may communicate directly or indirectly through one or more intermediaries.

[00462] A description of an embodiment with several components or features does not imply that all or even any of such components / features are required. On the contrary, a variety of optional components are described to illustrate the wide variety of possible embodiments of the present invention(s). Unless otherwise specified explicitly, no component / feature is essential or required.

[00463] Although process steps, algorithms or the like may be described in a sequential order, such processes may be configured to work in different orders. In other words, any sequence or order of steps that may be explicitly described does not necessarily indicate a requirement that the steps be performed in that order. On the contrary, the steps of processes described herein may be performed in any order practical. Further, some steps may be performed simultaneously despite being described or implied as occurring non-simultaneously (e.g., because one step is described after the other step). Moreover, the illustration of a process by its depiction in a drawing does not imply that the illustrated process is exclusive of other variations and modifications thereto, does not imply that the illustrated process or any of its steps are necessary to the invention, and does not imply that the illustrated process is preferred. [00464] Although a process may be described as including a plurality of steps, that does not imply that all or any of the steps are essential or required. Various other embodiments within the scope of the described invention(s) include other processes that omit some or all of the described steps. Unless otherwise specified explicitly, no step is essential or required.

[00465] Although a product may be described as including a plurality of components, aspects, qualities, characteristics and / or features, that does not indicate that all of the plurality are essential or required. Various other embodiments within the scope of the described invention(s) include other products that omit some or all of the described plurality.

[00466] Unless expressly specified otherwise, an enumerated list of items (which may or may not be numbered) does not imply that any or all of the items are mutually exclusive. Therefore it is possible, but not necessarily true, that something can be considered to be, or fit the definition of, two or more of the items in an enumerated list. Also, an item in the enumerated list can be a subset (a specific type of) of another item in the enumerated list. For example, the enumerated list "a computer, a laptop, a PDA" does not imply that any or all of the three items of that list are mutually exclusive - e.g., an item can be both a laptop and a computer, and a "laptop" can be a subset of (a specific type of) a "computer".

[00467] Likewise, unless expressly specified otherwise, an enumerated list of items (which may or may not be numbered) does not imply that any or all of the items are collectively exhaustive or otherwise comprehensive of any category. For example, the enumerated list "a computer, a laptop, a PDA" does not imply that any or all of the three items of that list are comprehensive of any category.

[00468] Further, an enumerated listing of items does not imply that the items are ordered in any manner according to the order in which they are enumerated. [00469] In a claim, a limitation of the claim which includes the phrase "means for" or the phrase "step for" means that 35 U.S. C. § 112, paragraph 6, applies to that limitation.

[00470] In a claim, a limitation of the claim which does not include the phrase

"means for" or the phrase "step for" means that 35 U.S. C. § 112, paragraph 6 does not apply to that limitation, regardless of whether that limitation recites a function without recitation of structure, material or acts for performing that function. For example, in a claim, the mere use of the phrase "step of or the phrase "steps of in referring to one or more steps of the claim or of another claim does not mean that 35 U.S. C. § 112, paragraph 6, applies to that step(s).

[00471] With respect to a means or a step for performing a specified function in accordance with 35 U.S. C. § 112, paragraph 6, the corresponding structure, material or acts described in the specification, and equivalents thereof, may perform additional functions as well as the specified function.

[00472] Computers, processors, computing devices and like products are structures that can perform a wide variety of functions. Such products can be operable to perform a specified function by executing one or more programs, such as a program stored in a memory device of that product or in a memory device which that product accesses. Unless expressly specified otherwise, such a program need not be based on any particular algorithm, such as any particular algorithm that might be disclosed in this patent application. It is well known to one of ordinary skill in the art that a specified function may be implemented via different algorithms, and any of a number of different algorithms would be a mere design choice for carrying out the specified function. [00473] Therefore, with respect to a means or a step for performing a specified function in accordance with 35 U.S. C. § 112, paragraph 6, structure corresponding to a specified function includes any product programmed to perform the specified function. Such structure includes programmed products which perform the function, regardless of whether such product is programmed with (i) a disclosed algorithm for performing the function, (ii) an algorithm that is similar to a disclosed algorithm, or (iii) a different algorithm for performing the function.

[00474] The present disclosure provides, to one of ordinary skill in the art, an enabling description of several embodiments and / or inventions. Some of these embodiments and / or inventions may not be claimed in this patent application, but may nevertheless be claimed in one or more continuing applications that claim the benefit of priority of this patent application. Applicants intend to file additional applications to pursue patents for subject matter that has been disclosed and enabled but not claimed in this patent application.

I l l

Claims

We claim:

1. A method of selecting the appropriate examiner to examine a patent application comprising: a) determining a profile for a patent examiner; b) determining a profile for the subject matter of a patent application; c) assigning the examiner with the closest profile to the profile of the subject matter of a patent application to examine the application.

2. The method of claim 1 further comprising evaluating the current workload of the most appropriate examiners and assigning the application to the examiner with the least workload.

3. The method of claim 2, wherein evaluating the current workload comprises: a)determining the amount of time it will take an examiner to examine the application; b)determining the number of pending applications in the examiner's queue; c)determining the amount of time required to examine the pending applications; and d)creating a queue score based on the number of applications.

4. The method of claim 1, wherein determining a profile for a patent examiner comprises: a) retrieving a patent examiner history; b) retrieving the patent applications previously examined by the examiner; c) generating a score for an examiner based on applications previously examined by the examiner.

5. The method of claim 4, wherein determining a profile for a patent examiner further comprises retrieving the number of applications that have been appealed.

6. The method of claim 4, wherein a patent examiner history comprises the patent examiner's education, seniority, number of applications examined, and training.

7. The method of claim 4, wherein the score is generated using artificial intelligence techniques.

8. The method of claim 7, wherein the artificial intelligence techniques are neural net, Bayesian algorithm, genetic algorithms, pattern recognition, expert systems, case based reasoning, fuzzy systems, hybrid intelligent systems, evolutionary computation, or concept processing.

9. The method of claim 1 , wherein determining a profile for the subject matter of a patent application comprises: a)evaluating the field of the invention; b)evaluating the class of the invention;

^evaluating the abstract of the invention; devaluating the claims of the invention; e)generating a score for the application based on the evaluation.

10. The method of claim 9, wherein the evaluation is made using artificial intelligence techniques.

11. The method of claim 10, wherein the artificial intelligence techniques are neural net, Bayesian algorithm, genetic algorithms, pattern recognition, expert systems, case based reasoning, fuzzy systems, hybrid intelligent systems, evolutionary computation, or concept processing.

12. The method of claim 1 , wherein the patent is submitted in electronic format.

13. The method of claim 1, wherein the examiner assigned an application can request that it be reassigned.

14. A method for adjusting examiner workload comprising: a)determining the length of time it should take to examine an application; previewing the number of applications waiting in a queue to be examined by a particular examiner; c)determining the length of time it should take to examine the applications in the examiner's queue; d)generating a queue score based on the length of time to examine applications and the number of applications waiting in a queue; e)comparing the queue scores of examiner's in a technology center; f)reassigning applications so that all examiner's queue scores are approximately equal.

15. The method of claim 14, wherein the determination of the length of time it should take to examine an application is determined using artificial intelligence.

16. The method of claim 14, wherein a supervisor may reassign applications.

17. The method of claim 14, wherein applications are reassigned automatically.

18. The method of claim 14, wherein applications are reassigned based on an examiner's relevance score.

19. The method of claim 18, wherein an examiner's relevance score is determined by: a) retrieving a patent examiner profile; b) retrieving the patent applications previously examined by the examiner; c) generating a score for an examiner based on applications previously examined by the examiner.

20. The method of claim 14, wherein when the queue scores of the examiner's in a particular technology center reach a particular threshold, a job listing for additional examiners is posted.

21. A method of performing a prior art search on an application comprising: a. determining a profile for an application; b. comparing the profile for an application with the profile for a database; c. performing a search of prior art in the most relevant database; d. scoring the relevance of the prior art; and e. ranking the prior art based on relevance.

22. The method of claim 21 , wherein a profile of an application comprises the field of invention, class, subclass, abstract, and claims.

23. The method of claim 22, wherein the profile is converted to a numeric score.

24. The method of claim 21 , wherein the search is performed prior to submission of the application to a patent office.

25. The method of claim 24, wherein an end user must distinguish the relevant prior art from the application when the application is submitted to the patent office.

26. The method of claim 21, wherein the search is performed after submission of the application to a patent office.

27. The method of claim 21 , wherein the database may contain published or unpublished art.

28. The method of claim 21 , wherein the profile of a database comprises the contents of particular databases, the accuracy of particular databases, how current particular databases are, and the number of times other examiners have used particular databases.

29. The method of claim 28, wherein the profile is converted to a numerical score.

30. The method of claim 21 , wherein the relevance of the prior art may be determined by comparing one or more criteria comprising common words and phrases, the class and sub class of the patent application and the prior art, the amount of time the prior art was considered by the end user or patent examiner on the same or similar patent applications, the notes an end user or patent examiner has attached to the prior art, the office actions performed by the system or patent examiners utilizing the same prior art on similar patent applications, the number of times the prior art has been cited, the seniority of the examiners who have cited the prior art for similar applications in the past, or the number of times the prior art data has been accessed in similar searches.

31. The method of claim 30, wherein the criteria may be weighted.

32. A method of increasing the review of an application comprising: a. publishing an application for comment; b. receiving commentary from third parties; and c. evaluating the commentary for relevance.

33. The method of claim 32, wherein the commentary is weighted.

34. The method of claim 33, wherein the weight of commentary is based on one or more criteria comprising the number of times the commentator has submitted the comments, the usefulness of previous comments, the number of publications the commentator has, or the credentials of the commentator.

35. The method of claim 32, wherein the commentary is available to the public.

36. The method of claim 32, wherein the commentary is available to examiners.

37. A method of increasing the accuracy of a search of prior art comprising: a. determining a profile for a patent application; b. inputting references cited in the application; c. inputting references cited in an information disclosure statement; d. reviewing the search strategy used for previous applications with similar profiles; and e. initiating a search based on the information in the search strategy used for previous applications with similar profiles cross-referenced with references cited in the application.

38. The method of claim 37, further comprising the inputting of references discovered by an examiner in a manual search.

39. The method of claim 37, wherein the search technique is self improving.

40. The method of claim 37, wherein the search can be performed using artificial intelligence, table based method, a rules based system, neural net, Bayesian algorithm, genetic algorithms, pattern recognition, expert systems, case based reasoning, fuzzy systems, hybrid intelligent systems, evolutionary computation, or concept processing.

41. A method of outsourcing patent applications for examination comprising: a) determining a profile for the subject matter of a patent application; b) determining a profile for an examination agency; and c) assigning the application to the examination agency with the closest profile to the profile of the subject matter of a patent application to examine the application.

42. The method of claim 41 further comprising creating a profile of examiners at an examination agency.

43. The method of claim 42, wherein determining a profile for an examiner comprises: a) retrieving examiner history; b) retrieving the patent applications previously examined by the examiner; and c) generating a score for an examiner based on the examiner history and the applications previously examined by the examiner.

44. The method of claim 43, wherein a patent examiner history comprises the examiner's education, seniority, number of applications examined, and training.

45. The method of claim 43, wherein the score is generated using artificial intelligence techniques.

46. The method of claim 46, wherein the artificial intelligence techniques are neural net, Bayesian algorithm, genetic algorithms, pattern recognition, expert systems, case based reasoning, fuzzy systems, hybrid intelligent systems, evolutionary computation, or concept processing.

47. The method of claim 41 , wherein determining a profile for the subject matter of a patent application comprises: a) evaluating the field of the invention; b) evaluating the class of the invention; c) evaluating the abstract of the invention; d) evaluating the claims of the invention; and e) generating a score for the application based on the evaluation.

48. The method of claim 47, wherein the evaluation is made using artificial intelligence techniques.

49. The method of claim 41 , wherein determining a profile for an examination agency comprises: a) retrieving the area of specialization of the agency; b) retrieving the subject matter of applications previously examined by an agency; c) retrieving the expertise of the examiners employed by the agency; and d) generating a score for the examining agency.

50. The method of claim 49, wherein the evaluation is made using artificial intelligence techniques.

51. A method of reviewing patents comprising: a) receiving a request to review a patent; b) retrieving the history of the issuance of the patent; c) generating a profile for the patent; d) comparing the profile of the patent to the profile of an examiner; and e) assigning the patent to the examiner with the closest profile.

52. The method of claim 51 , wherein the review is requested by the system.

53. The method of claim 51 , wherein the review is requested by a third party.

54. The method of claim 51 , wherein the examiner may not be the same examiner that initially examined the patent.

55. The method of claim 51 , wherein the examiner must be a senior examiner.

56. The method of claim 51, wherein if the examiner determines that the patent is invalid, the patent is reviewed by an additional examiner.

57. A method of reviewing applications comprising: a) selecting an application for review; b) generating a patent application score for the selected application; c) generating a patent attorney score; d) comparing the patent application score to the patent attorney score; and e) assigning the application to the patent attorney with the closest score.

58. The method of claim 57, wherein the comparison is made using artificial intelligence techniques.

59. The method of claim 57, wherein the artificial intelligence techniques are neural net, Bayesian algorithm, genetic algorithms, pattern recognition, expert systems, case based reasoning, fuzzy systems, hybrid intelligent systems, evolutionary computation, or concept processing.

60. The method of claim 57, wherein the patent attorney may receive remuneration for the review.

61. A method comprising : providing an electronic database of patent documents to a plurality of end users; providing a user interface configured to: enable the end users to view the patent documents in the database; enable the end users to submit prior art in reference to one or more of the patent documents in the database; identifying the prior art submitted by a given end user; and determining the frequency with which the prior art submitted by the given end user is used to invalidate a patent document.

62. The method of claim 61 further comprising bestowing a benefit upon the end user if the frequency is determined to be above a given threshold.

63. The method of claim 62 wherein the benefit is increased significance placed on future prior art submissions submitted by the end user.

64. The method of claim 62 wherein there is a fee associated with submitting prior art documents and the benefit is a reduced fee.

65. The method of claim 62 wherein the benefit is monetary compensation.

66. The method of claim 62 wherein the benefit is expedited examination of a patent application submitted by the end user.

67. The method of claim 61 further comprising penalizing the end user if the frequency is determined to be below a given threshold.

68. The method of claim 67 wherein the penalty is decreased significance placed on future prior art submissions submitted by the end user.

69. The method of claim 67 wherein the penalty is a ban from submitting prior art in connection with other patent documents.

70. The method of claim 67 wherein there is a fee associated with submitting prior art documents and the benefit is an increased fee.

71. A method comprising : providing a database of patent documents to a plurality of end users; receiving a prior art document from an end user, wherein the prior art document is submitted in conjunction with a patent document in the database, reviewing the prior art document; determining that the patent document is invalid based on the prior art document; providing a tangible benefit to the end user.

72. The method of claim 71 wherein the tangible benefit is monetary compensation.

73. The method of claim 71 wherein the tangible benefit is expedited examination of a patent application submitted by the end user.

74. The method of claim 71 further comprising receiving commentary from the end user identifying why the prior art is believed to invalidate the patent document.

75. The method of claim 71 further comprising tracking the frequency with which prior art documents submitted by the end user are used to invalidate a prior art document.

76. The method of claim 75 further comprising bestowing a benefit upon the end user if the frequency is determined to be above a given threshold.

77. The method of claim 72 wherein the benefit is increased significance placed on future prior art submissions submitted by the end user.

78. The method of claim 76 wherein there is a fee associated with submitting prior art documents and the benefit is a reduced fee.

79. The method of claim 75 further comprising penalizing the end user if the frequency is determined to be below a given threshold.

80. The method of claim 79 wherein the penalty is decreased significance placed on future prior art submissions submitted by the end user.