WO2001069490A1 - Method of conducting secure clinical trials and guaranteeing the occurrence of an event - Google Patents

Method of conducting secure clinical trials and guaranteeing the occurrence of an event Download PDF

Info

Publication number
WO2001069490A1
WO2001069490A1 PCT/US2001/007533 US0107533W WO0169490A1 WO 2001069490 A1 WO2001069490 A1 WO 2001069490A1 US 0107533 W US0107533 W US 0107533W WO 0169490 A1 WO0169490 A1 WO 0169490A1
Authority
WO
WIPO (PCT)
Prior art keywords
trial
clinical
data
transactions
members
Prior art date
Application number
PCT/US2001/007533
Other languages
French (fr)
Inventor
Glen M. De Vries
Edward F. Ikeguchi
Alexis E. Te
Original Assignee
Medidata Solutions, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Medidata Solutions, Inc. filed Critical Medidata Solutions, Inc.
Priority to AU2001240128A priority Critical patent/AU2001240128A1/en
Publication of WO2001069490A1 publication Critical patent/WO2001069490A1/en

Links

Classifications

    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H10/00—ICT specially adapted for the handling or processing of patient-related medical or healthcare data
    • G16H10/20—ICT specially adapted for the handling or processing of patient-related medical or healthcare data for electronic clinical trials or questionnaires
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/10—Office automation; Time management

Definitions

  • This invention relates to life cycles of clinical trials and more specifically to authenticating that a recorded event occurred in a proscribed manner and was performed by authorized personnel, the authentication leading to the acceptance or rejection of trial data in a clinical trial.
  • CROs Contract Research Organizations
  • a clinical trial is carried out in the following manner.
  • the company sponsoring the trial or the CROs contracted for the trial (the "sponsor") will select a number of trial investigators.
  • These trial investigators are usually doctors who specialize in the area of medicine relevant to the new medical technology under study. The trial investigators are chosen based upon several criteria including:
  • the trial monitor is a person who is usually hired by the CROs in order to verify that the data reported by the trial investigator corresponds to the source documentation, i.e., the trial participants clinic record.
  • the trial monitor serves as an auditor of the trial investigators in order to police the integrity of the data collected.
  • the trial monitor also assures that all documentation, such as the trial participant informed consent is properly signed and that the trial investigator stay true to the procedure set forth in the trial protocol.
  • the trial monitor also checks to assure that the reports forwarded to the sponsor correspond to the actual medical record documented by the trial investigator in the trial participant's chart.
  • the sponsor is made cognizant of the overall status of the trial. For example, if an inordinate number of complications arise from the medical treatment, it is the sponsor's responsibility to know of the problem and to react. Sometimes such reactions may prematurely halt the trial. Conversely, it is not uncommon for the new medical technology to be so effective that the reaction and the responsibility of the sponsor is to offer the new medical technology to more trial participants.
  • the key event leading to the generation of data is that between two individuals: the trial participant, e.g., a patient, and the trial investigator, e.g., a doctor.
  • this interaction is the event that is documented in the trial participant's medical record and is a reflection of the trial participant's physical examination, reports of subjective complaints, inte ⁇ retations of objective testing, and a synthesized analysis of the trial participant's information as a function of the trial investigator's professional training.
  • Each transaction of the inventive method is performed by one of the members initiating a session by establishing a data from a computing device equipped with various authentication hardware and/or software, to the computing device on which the inventive method is implemented.
  • the invention determines what level of access is allowed and what level of authentication is required for the logged in member.
  • the members who are the trial participants and the trial investigators are shown a plurality of tests to choose from. When the required test is selected, the trial data entry may begin.
  • the trial participants and the trial investigators are asked to reply to a plurality of questions or enter information corresponding to various elements of the test. Each of the entered values is immediately evaluated to determine if values entered are outside of the exception limits.
  • the exception limits may be default or custom set by the trial administrator. If the exception limits are exceeded, the trial participants and the trial investigators are asked to verify if the values are correct.
  • Figure la is a network topology diagram, showing connectivity of various parts of the system of the present invention.
  • Figure 2 is a diagram of the hardware components of computing devices used by the system of the present invention.
  • Figures 4b-4f is a sample screen of the clinical trial management program listing various test and intervals at which these tests are to be performed.
  • Figure 5 is a flow diagram of the clinical trial management program of the present invention.
  • the common bus 30 is further connected to
  • peripheral interface 38 to the peripherals 58, such as the keyboard, the mouse, navigational buttons, e.g., on a digital phone, a touch screen, and/or writing screen on full size and hand held devices, e.g., a palm pilot TM;
  • the communications interface 44 e.g., a plurality of modems
  • a network connection 60 e.g., an Internet Service Provider (ISP) and to other services, which is in turn connected to the network 20, whereby a data path is provided between the network 20 and the computing devices 12, 14, 15, 16, and 18 ( Figure 1) and, in particular, the common bus 30 of these computing devices; and
  • ISP Internet Service Provider
  • the trial administrator defines every aspect of any clinical trial, which may be administered by the inventive system.
  • the time intervals table 74 comprises timing information regarding individual tests may be defined, e.g.,
  • the data entry vehicle used i.e., checkbox, text field, a pointer to a data file such as a music format or a digital image, or an interfaced piece of hardware utilized to obtain data.
  • Figure 3b as defined by its elements in related tables 78 ( Figure 3b) is reusable, i.e., may be used to perform multiple clinical trials. In this manner, basic elements commonly found in clinical trials could be made available in a custom generated clinical trial. For example, the trial administrator trying to create a clinical trial may be presented with a panel of trial parts such as patient, doctor, blood test, heart exam, etc. Any trial element may be inco ⁇ orated into a trial in a "drag-and-drop" or other fashion.
  • the trial administrators desiring to create or design clinical trials may do so by establishing a datapath from the computing device 15 ( Figure 1) to the computing device 12, via the network 20 to access the clinical trial management program of the present invention.
  • Figures 4b and 4c show the clinical trial 72 comprising intervals 74 and tests 76.
  • Test elements 78 Figure 3b
  • the exceptions 80 may be displayed. Description "done" 106 indicates that the particular test 76 for the particular interval 74 has been performed.
  • the trial administrators will be able establish or modify an existing list of trial investigators and trial monitors in the personnel database 1 lb ( Figure 1) for the particular trial being developed. Moreover, the trial investigators will be able to enter in to the personnel database 1 lb ( Figure 1), lists of names of trial participant/patient selected to participate in the particular study after the trial is developed and is being carried out.
  • the trial investigator will have access to patient specific information in the database l ie ( Figure 1), by patient name and by patient number as well as to summary data within the scope of his or her own results.
  • the trial administrator may also be able to define the type of investigator enrollment, e.g., an open enrollment allowing anyone qualified to act as the trial investigator to sign up or register as the trial investigator in the personnel database 1 lb ( Figure 1).
  • the trial monitor will have access to patient specific information saved in the trial data database l ie ( Figure 1) and identified by patient number, i.e., no patient names are revealed.
  • Trial Once the particular trial has been completely defined or created in the application data database 11a ( Figure 1) the trial may commence. After the trial is activated, the parameters defining the trial in the database 11a ( Figure 1) can no longer be manipulated. To facilitate creation of new trials, the entire existing trial may be copied to create a new trial and that trial's parameters may be modified.
  • the trial adnrinistrator contracts the trial investigators to select the trial participants, begin patient enrollment and data collection regarding these trial participants.
  • the trial administrator further contracts the trial monitors to monitor the veracity of the trial data.
  • the trial participants are selected and lists of names of the trial participants are entered into the personnel database 1 lb ( Figure 1).
  • the clinical trial management program of the present invention shown in Figure 5, may read in the parameters of the trial established in the application data database 11a ( Figure 1) and automatically generate the appropriate questionnaires and forms to allow the trial participants of the trial to fill in the results of tests 77 ( Figure 4f).
  • the clinical trial management program 200 may be accessed by authorized personnel in step 202 through the login screen 102 ( Figure 4a).
  • step 204 judging by the login and determining the level of access through the use of the personnel database 1 lb ( Figure 1), a dete ⁇ nination is made of what level of authentication should be performed, or whether authentication is necessary all together. It may be determined that further or additional authentication is required, such as biometrics authentication of a single or multiple trial participants, e.g., in situations where the authentication is used to guarantee the physical interaction between a trial investigator and a trial participant.
  • the concurrent authentication of a plurality of members is utilized by the present invention in situations where the authentication is used to guarantee the physical interaction between the trial investigators and the trial participants.
  • step 210 the trial investigators and the trial participants will be shown a list of possible tests 72 ( Figure 4c).
  • the trial investigators and the trial participants may be presented with screens, e.g., 76a-c ( Figures 4d-4f), and may start in step 212 to enter test result data.
  • test screens 76a-c are dynamically generated by the clinical trial management program 200.
  • step 210 the actions required, i.e., defined in the application data database 11a ( Figure 1), are evaluated and only these required are presented. Furthermore, data entered in response may lead to additional options to be displayed. Actions that have been previously completed are displayed as such, giving instant feedback regarding what was previously entered into the system.
  • step 214 a logout path may be taken. An in-depth description of the logout procedure will be discussed below.
  • a transaction of the clinical trial management program 200 begins following the login in step 202 and the authentication in step 204. All the information entered during the session after the login and before the logout is considered a part of this transaction. The transaction will not be committed, i.e., be made a part of the permanent storage 52 ( Figure 2) until the whole session is completed.
  • data elements 77 Figures 4d-4f
  • the clinical trial management program 200 dynamically executes scripts to perform the first level of validation of the data values 77 ( Figures 4d-4f) entered.
  • the executed scripts may validate entered data values 77 ( Figures 4d-4f) and may be executed by the browser program, e.g., Microsoft ExplorerTM, Netscape NavigatorTM, using its built in functionality on the computing devices 14, 15, 16, and 18 ( Figure 1). Moreover, the executed scripts may validate entered data values 77 ( Figures 4d-4f) on the computing device 12 ( Figure 1) to provide two levels of data validation.
  • the browser program e.g., Microsoft ExplorerTM, Netscape NavigatorTM
  • step 216 saves these values 77 ( Figures 4d-4f) in the current transaction, executes the scripts to perform the second level of validation as described above.
  • the values 77 ( Figures 4d-4f), entered for each element are once again compared with the exception ranges 80 ( Figure 3b) and with the acceptable data change rate.
  • the data change rate is determined by comparing values 77 ( Figures 4d-4f) to these entered for the same element at previous intervals. Should the values 77 ( Figures 4d-4f) be either out of the exception range 80 ( Figure 3b) or change at a rate beyond the preset exception change rate for the given element, an exception record 270 ( Figure 6) is added to the current transaction.
  • step 218 it is determined if one or more exceptions are generated at the given interval for the given element. If exceptions are generated, in step 220 an exception record adds new tests to the trial participant's complete trial record and the elements within that new text, which must be completed to close the trial participant's participation in the trial. At this point it is possible to offer an option of completing the exception data immediately or to queue the exception for a later time. If in step 222 it is determined that the exception is to be completed now, the test form for the exception is presented and the data entry loop starting in the step 212 is repeated. Otherwise, in step 224 the exception is queued, and will then appear in the appropriate data entry options of the step 210.
  • Each record 270 comprises the following fields:
  • a trial key 270b for pointing to the specific trials table 72 ( Figure 6a) of the application data database 11a ( Figure 1) with which the present record 270 identifies or belongs.
  • a data certification code field 270i comprising a plurality of random digits.
  • the trial investigator's computing device 16 may print a record of the activities of each patient handled by the trial investigator during any given login episode. This record will act as a legal record of the investigator's actions during a log in episode and be clearly marked with a statement affi ⁇ ning that the investigator has read the record and agrees with it. The investigator will then be asked to sign the record. The signed record can then be utilized as the source documentation since it contains a record of patient data with a verifying signature. The document may be appropriately formatted for placement into the trial participant's chart.
  • An actual data record 2701 of the 77 ( Figures 4d-4f) with data of various types, e.g., text, integer, binary, real, floating, date/time, images, biometrics, etc., keeping their appropriate data type.
  • each record 270 is recorded in the transaction that covers the entire session from login in step 202 to logout in step 214.
  • I-n step 226 it is determined from the personnel database 1 lb ( Figure 1) or from the application data database 11a ( Figure 1) that a logout authentication is required.
  • step 228 it is determined which authentication is to be performed, the biometrics, e.g., iris or finge ⁇ rint scan, retinal scan, facial recognition, DNA imprinting using hair, saliva, etc., in step 230 or other, e.g., entering social security number, special I-D codes, in step 232 may be required before committing the transaction.
  • the biometrics e.g., iris or finge ⁇ rint scan, retinal scan, facial recognition, DNA imprinting using hair, saliva, etc.
  • step 230 or other, e.g., entering social security number, special I-D codes, in step 232 may be required before committing the transaction.
  • the authentication may be performed as follows:
  • step 234 it is determined if the required authentication at the end of the session was properly performed or failed. If the authentication failed, in step 236 the transaction is rolled back and the data does not become a permanent part of the trial data. In the alternative the transaction may become a permanent part of the trial data with a recorded indication that the logout authentication failed, or permanently logged outside of the trial data, however the transaction will be rolled back and the data will not become a part of the trial data in the trial data database l i e ( Figure 1). In step 238, the transaction is committed, i.e., recorded as a permanent record in the trial data database lie ( Figure 1).
  • the trial monitor is the entity, person, or persons that constantly monitor the integrity of the trial insofar as the data entered and the conduct of the trial investigators and the trial personnel is concerned.
  • the trial administrator often contracts the trial monitors as part of the CROs or as independent consultants.
  • the trial monitor maintains a record of the activities of the trial investigators and makes certain that the data submitted by the trial investigator corresponds to the source documentation, i.e., the patient's medical record.
  • the trial monitor verifies whether the trial investigator has provided the proper informed consent and that the trial participant's characteristics conform to the inclusion and exclusion criteria specified in the trial protocol.
  • Trial monitors will perform declared and undeclared visits to the trial investigator sites to audit the operations of the trial investigator in-person.
  • the inventive system may provide the trial monitor with access to limited information in the trial data database lie ( Figure 1). While the trial monitors may be free to view all trial participant information from the personnel database 1 lb ( Figure 1), they may not be able to identify these trial participants except by a unique code assigned to each trial participant for pu ⁇ oses of the trial.
  • the trial monitor may be asked to authenticate him or herself in steps 206 or 208. This authentication is accomplished using either a form of biometrics measurement in step 206 or by username and password method in step 208.
  • These reports may be created for each trial participant according to each time point, of the intervals 74 ( Figure 3b), tracked as per the trial protocol. Furthermore, these reports may enable the trial monitor to verify the trial investigator's source documentation against the data shown in the report. Separate reports may be generated for data that is missing, entered late, changed by the trial investigator, or is out of range. This may allow the trial monitor to address each specific issue with the trial investigator and to reconcile the problem in the trial data. Separate reports may further be generated for every adverse event reported by the trial investigator. Each report may list the adverse event and the follow-up necessary to be performed by the trial investigator in order to document the ultimate consequences to the trial participant.
  • Summary reports of the trial monitor's verified values 77 may be generate each time a trial monitor completes a review of either a single test element 78 ( Figure 3b) or group of tests 76 ( Figure 3b).
  • the verified values 77 will also be associated with identifying stamps such as date, time, trial monitor's biometrics authentication, etc. This report may ultimately be printed for approval by the trial investigator and signature.
  • the trial administrator may be given special access to certain forms of data while being restricted to others. For example, while the trial administrator may be able to view overall summary data about a trial, they will be unable to access patient-specific information in order to maintain patient privacy.
  • the application will have special functions available to the trial administrator.
  • the trial administrator will be able to have a real-time "bird's eye view" of the ongoing clinical trial.
  • the trial administrator After logging in step 202, the trial administrator will be asked to authenticate him or herself as described above with reference to steps 204, 206 and 208.
  • J-n step 242 the trial administrator will be enabled to obtain summary data of the trial and application-specific tools to allow manipulation of the summary data.
  • a summary information area allows the trial administrator access to updates on the status of various trial investigator sites. In doing so, the trial administrator is able to see the performance of each trial investigator via parameters such as the number of patients enrolled or the number of completed patients or the number of delinquencies or the number of adverse events or by the answers to satisfaction- questionnaires about the trial investigators rated by patients/trial participants.
  • Reporting tools may be provided such that the trial administrator can generate reports.
  • a pharmaceutical company testing a new drug may want a report that conforms to the outline of a new drug application; the trial administrator, for academic reasons might want a report that is later submitted as a clinical paper to a medical journal.
  • Templates for various kinds of reports will be available to the trial administrator.
  • the trial administrator will be able to customize their own template, which may consist of a series of headings and hierarchical subheadings to format a document.
  • the reports created may be integrated with the analysis tools such that graphs generated dynamically from the trial data will be inco ⁇ orated into the report.
  • the application will allow for collaborative authoring and tracking of a document using a system by which parts of a document may be "checked-in” or “checked-out” by multiple trial administrators.
  • the reports created will be able to be output in multiple formats compatible with other word processing software or document readers, i.e., Adobe Acrobat, or Microsoft Word.
  • the trial administrator may immediately transfer the document electronically to the end viewer such as a peer review committee or a governmental regulatory board.

Abstract

The present invention is a method of conducting clinical trials (172). Each clinical trial includes members comprising trial administrators, trial monitor, trial investigators, and trial participants or patients. The present invention allows the conduct of the clinical trial to take place over a network, e.g., the Internet or the telephone grid. The participants use computing devices connected to the network to perform transactions of the clinical trial. Each of these tests comprises one or more periods of execution or intervals (174), test elements (178), and a plurality of exception (180) limits. All the trial participants are listed in a database and are assigned access and validation levels. The executed transactions are stored in a database. Any time during the performance of the clinical trial the stored transactions can be queried at random to assure veracity of the clinical trial and to notify the trial administrator of discrepancy to prevent fraud.

Description

METHOD OF CONDUCTING SECURE CLINICAL TRIALS AND GUARANTEEING THE OCCURRENCE OF AN EVENT
BACKGROUND OF THE INVENTION
Field of the Invention
This invention relates to life cycles of clinical trials and more specifically to authenticating that a recorded event occurred in a proscribed manner and was performed by authorized personnel, the authentication leading to the acceptance or rejection of trial data in a clinical trial.
Description of the Background of the Invention
Annually, companies in the pharmaceutical and biotechnology industries funnel huge monetary investments into research and development (R&D) of new medical technology, i.e. drugs, medical devices, and new method or technique to diagnose and treat a medical disorder. A life cycle of any new medical technology generally originates in a laboratory. Following the laboratory, the new medical technology undergoes several phases of investigation to prove its safety and efficacy. Once a sufficient body of evidence has been amassed to assure such safety and efficacy, the new medical technology is sent before a regulatory committee for review. If the medical technology is subsequently approved, it can then be marketed for sale and sold, enabling the companies investing in that new medical technology to begin to realize the return on their investment in the development of the new technology. The life cycle for most of the new medical technology is lengthy, for example, in 1997 the process of developing a new medical technology from laboratory to marketing took an average of 10 to 12 years. Over the past decade, a new industry has arisen as an adjunct to the clinical research process of developing new medical technologies. R&D service providing organizations, known as Contract Research Organizations (CROs) arose out of a growing need among the pharmaceutical and biotechnology companies to curb costs. Rather than hiring full-time research staff, CROs became a resource for these companies to outsource the adrninistrative and coordinating responsibilities for clinical research. Over time, CROs compounded other value-added services to encompass a spectrum of services including clinical trial coordination, monitoring of data collection, identification of quality research trial investigators and sites, and centralized laboratory testing. The development of the CROs was also logical for other reasons including the favorable implications for the pharmaceutical and biotechnology companies to have their new medical technology tested under the unbiased eye of a third-party organization.
One of the greatest claims made by the CROs is the ability of their trained staff professionals to shorten the amount of time required to complete a research project. This ability would in turn shorten the amount of time required to pass regulatory scrutiny. As can be recognize from the protracted 10 - 12 year development period described above, such shortening of the life cycle time is very encouraging for most pharmaceutical and biotechnology companies. That is because, while a newly invented drag or medical device may be patented upon discovery, with the 20 year patent term commencing on the date of filing of the patent application, the profitability of such new medical device does not take effect until it is approved, marketed, and sold. And this, as described above, can take 10 - 12 years, which period usually coincides with a large portion of the limited life of the patent.
It is not inconceivable that after spending millions on R&D and after facing a tough regulatory approval process, the pharmaceutical and biotechnology companies would have but 2 - 5 years of exclusivity to exploit the sale of the new medical technology. Such outcome is ultimately detrimental to all, it discourages R&D in the medical field, encourages hasty clinical research to expedite the regulatory process, and places smaller to mid-sized companies at a distinct disadvantage to the established "health care giants." The protracted pre-marketing time, particularly in the United States, has also been criticized as being a barrier to the passage of vital new medical technologies. In many cases, life-saving drugs can only be obtained in settings of research or in markets outside the control of the United States' Government.
Despite the emergence of the CROs, their services are still severely under-utilized. In the United States, it is estimated that less then 20% of the medical R&D market is captured by the CROs. These figures are far smaller outside of the United States, particularly in Europe and Japan.
Life Cycle
Typically, a clinical trial is carried out in the following manner. At the initiation of a clinical trial, the company sponsoring the trial or the CROs contracted for the trial (the "sponsor") will select a number of trial investigators. These trial investigators are usually doctors who specialize in the area of medicine relevant to the new medical technology under study. The trial investigators are chosen based upon several criteria including:
1. The number of trial participants seen and the ability of the trial investigator to accrue trial participants into the study.
2. The facility and support staff available to the trial investigator. 3. The trial investigator's ability to collect and maintain data in a secure fashion without compromising trial participant confidentiality or care.
A trial protocol will be formulated to achieve the desired goals of the trial, and the protocol will be presented for review and approval before an Institutional Review Board (IRB). The IRB is a committee consisting of peers and people with experience in the research field. It may consist of physicians, nurses, PhD's, bio-statisticians, bio-ethicists, and others qualified to evaluate research on human. The IRB evaluates research protocols to assure scientific integrity while mamtaining the trial participant's safety and privacy within a standard of ethics acceptable for human experimentation. Once a clinical trial is under way, the trial investigators do the following: 1. Begin recruiting trial participants that fit the inclusion/exclusion criteria of the protocol.
2. Explained the risks and benefits of the trial to the trial participants.
3. Ask the trial participants to give an informed consent. 4. Apply new medical technology, i.e., administer medical treatments.
5. Initiate and continue trial data collection.
The trial data is collected by the trial investigators in the form of reports that are then forwarded to the sponsor. Reporting generally takes the form of paper sheets that are handwritten and transmitted via fax or placed into digital form with the use of an electronic scanner. These reports generally serve as the data collection vehicle with various parameters studied as outlined in the particular protocol. The parameters studied in a clinical trial may include subjective findings such as trial participant's complaints, satisfaction, or symptoms. Objective parameters may also be studied, these include physical examination, laboratory or radiological tests, and other measured findings. Finally, a separate parameter followed in almost all trials is incidence of adverse events or complications from the treatment.
In the course of a clinical trial, an important responsibility is that of a trial monitor. The trial monitor is a person who is usually hired by the CROs in order to verify that the data reported by the trial investigator corresponds to the source documentation, i.e., the trial participants clinic record. The trial monitor serves as an auditor of the trial investigators in order to police the integrity of the data collected. The trial monitor also assures that all documentation, such as the trial participant informed consent is properly signed and that the trial investigator stay true to the procedure set forth in the trial protocol. The trial monitor also checks to assure that the reports forwarded to the sponsor correspond to the actual medical record documented by the trial investigator in the trial participant's chart.
As the clinical trial progresses, the sponsor is made cognizant of the overall status of the trial. For example, if an inordinate number of complications arise from the medical treatment, it is the sponsor's responsibility to know of the problem and to react. Sometimes such reactions may prematurely halt the trial. Conversely, it is not uncommon for the new medical technology to be so effective that the reaction and the responsibility of the sponsor is to offer the new medical technology to more trial participants.
At the end of the trial, a stage of trial "closing" is conducted. During this stage all reports are finalized and any missing data is reconciled by the trial investigators. After this, the sponsor consolidates the data collected during the trial for statistical analysis. The results of such analysis are then made available for reporting to the regulatory bodies and/or academic publishing.
Detriments
There are many instances where the life cycle described above falls short of its optimal potential. In fact negative reports and issues of the shortcomings of medical R&D are in the news daily. In January 2000, all human gene therapy experiments carried out at an Ivy League University and sponsored by the United States government was halted after the death of a trial participant. After further review, there was evidence that the trial participant was not provided proper informed consent. Outside of simple human error, there are ample situations where monetary pressures may lead to potential fraud by the trial investigator and/or the sponsor. This f aud may include the falsification of the trial data, loose inteφretation of the trial protocols to allow into a study trial participants who may not be proper candidates, statistical manipulations to allow results to appear better than actual, under-reporting of adverse events, etc. There have even been situations where entire group of trial participants reported upon in the trial, did not even exist.
In the early 1990's the National Surgical Adjuvant Breast and Bowel Project ran a clinical trial evaluating benefits of performing mastectomy for trial participants with breast cancer versus lumpectomy with or without radiation. During this trial, one of the site directors, the equivalent of the trial investigator, falsified the dates of certain events of the trial, in order to allow ineligible trial participants to appear eligible for the research. This ultimately resulted in a massive government investigation costing millions of dollars, a delay in the availability of the trial results affecting thousands of patients with breast cancer, skepticism concerning the trial results, loss of the trial investigator credibility, and the derailment of the careers of several prominent academic figures.
The potential for similar situations to re-occur forces the increased stringency of regulatory processes, thus adding to the already long life cycle period required before marketing and sale of the new medical technology.
Evolution Given the pressures placed upon the sponsors in terms of time expenditure while sustaining the scientific rigor, several CROs and industry specialists have begun implementing digital formats for data collection. Digital data collection and collaboration of research over digital networks have the potential for multiple advantages. These include the ability to consolidate the trial data into a single database as the trial data is being collected, thus allowing for the analysis of data in real time. In addition, data can be validated upon entry, ultimately resulting in less time spent at the end of the trial to reconcile "loose ends" in the data collection process. These and other advantages of electronic data collection are clearly superior to conventional methods of handwritten forms, scanning, and faxing. However, many concerns still exist over issues of data security, trial participant privacy, and veracity of the collected data.
Furthermore, while electronic data collection has the potential to improve upon the time expenditure in the clinical trial process, it does not resolve the need to vigorously monitor the trial for fraud. In other words, while the electronic data collection makes the clinical trial process faster, it does not improve the scientific rigor with which the trial investigators collect and report upon the trial data.
In any clinical research setting, the key event leading to the generation of data is that between two individuals: the trial participant, e.g., a patient, and the trial investigator, e.g., a doctor. In conventional methods, this interaction is the event that is documented in the trial participant's medical record and is a reflection of the trial participant's physical examination, reports of subjective complaints, inteφretations of objective testing, and a synthesized analysis of the trial participant's information as a function of the trial investigator's professional training.
The conventional methods of clinical research rely upon a trial investigator's signature on a paper document. This creates plenty of room for fraud and an obvious need for strict monitoring. As the clinical research field looks more to digital data capture and transmission, there is a greater need to authenticate the information. While various methods have been proposed for digital authentication of individuals, what is needed is the authentication that a recorded event occurred in a proscribed manner and was performed by authorized personnel. This event may include the interaction between two or more authorized personnel, such as the doctor and patient. Moreover, such authentication must lead to the acceptance or rejection of the trial data in a clinical study or trial.
SUMMARY OF THE J-NVENTION
The present invention is a method of conducting clinical trials. Each clinical trial includes members comprising trial administrators, trial monitors, trial investigators, and trial participants or patients. Additionally these members may include reporters, scientists, and others interested in sampling the results of the study as it progresses. The trial administrator selects the trial monitors, persons responsible with assuring the veracity of the study, and the trial investigators, persons like nurses and doctors who will actually administer the study or trial. The trial administrator will further define procedural guidelines for the performance of the clinical trial. The trial investigators in turn will select the trial participants or patients to be enrolled in the clinical trial. The trial monitors observe the conduct of the clinical trials to detect any deviations from the procedural guidelines previously established by the trial administrator. The present invention allows the conduct of the clinical trial to take place over a network, e.g., the Internet or the telephone grid. The participants using computing devices connected to the network to perform transactions of the clinical trial. Each of the tests comprising one or more periods of execution or intervals, test elements, and a plurality of exception limits. All the trial participants are listed in a database and are assigned access and validation levels. The clinical trial is performed according to the procedural guidelines by carrying out all of the transactions of the clinical trial. The executed transactions are stored in a database. Any time during the performance of the clinical trial the stored transactions can be queried at random to assure veracity of the clinical trial and to notify the trial administrator if any discrepancy is found to prevent fraud.
Each transaction of the inventive method is performed by one of the members initiating a session by establishing a data from a computing device equipped with various authentication hardware and/or software, to the computing device on which the inventive method is implemented. As the connection is established and the member has logged in, the invention determines what level of access is allowed and what level of authentication is required for the logged in member. After performing the required authentication method, the members who are the trial participants and the trial investigators are shown a plurality of tests to choose from. When the required test is selected, the trial data entry may begin.
The trial participants and the trial investigators are asked to reply to a plurality of questions or enter information corresponding to various elements of the test. Each of the entered values is immediately evaluated to determine if values entered are outside of the exception limits. The exception limits may be default or custom set by the trial administrator. If the exception limits are exceeded, the trial participants and the trial investigators are asked to verify if the values are correct.
The login authentication process initiates or starts the clock running on a time range within which data entry or the current session must conclude. In situations where the interaction of the members of the clinical trial is authenticated, the login authentication process may include the input of more than one authorized personnel or member of the clinical trial during the established time range. Regardless of the number of authenticated parties or members of the clinical trial, the data entry may occur as follows:
1. before the authentication of all necessary parties or members of the clinical trial; 2. sandwiched in between authentications, e.g., at least one authentication followed by data entry which is followed by at least one authentication or re-authentication and so on; and 3. after the authentication of all necessary parties or members of the clinical trial. Transactions failing that test may not be committed to the database, i.e., the results will be discarded.
After logging in, the members who are the trial monitors and the trial administrators may be allowed to execute various reports and queries on the information collected by the clinical trial up to date. Such reports and queries may assist these members in assuring that the clinical trial adheres to its procedural guidelines and therefore the data collected is beyond reproach
BRIEF DESCRIPTION OF DRAWINGS
The foregoing objects and advantages of the present invention may be more readily understood by one skilled in the art with reference being had to the following detailed description of a preferred embodiment thereof, taken in conjunction with the accompanying drawings wherein like elements are designated by identical reference numerals throughout the several views, and in which:
Figure la is a network topology diagram, showing connectivity of various parts of the system of the present invention. Figure 2 is a diagram of the hardware components of computing devices used by the system of the present invention.
Figure 3 a is a flow diagram of a database entry program of the present invention, used to populate the application database.
Figure 3b is a logical relationship design diagram of elements comprising the records of the application database.
Figure 4a is a sample login screen for gaining access to the clinical trial management program of the present invention.
Figures 4b-4f is a sample screen of the clinical trial management program listing various test and intervals at which these tests are to be performed.
Figures 4g is a sample report screen of the clinical trial management program listing for the trial monitor various information indicating test performance status.
Figure 5 is a flow diagram of the clinical trial management program of the present invention.
Figure 6 is a logical relationship design diagram of elements comprising the records of the trial database and their relationship to elements of the application database.
DETAILED DESCRIPTION OF A PREFERRED EMBODIMENT OF THE INVENTION
As shown in Figure 1, the inventive system 10 comprises computing devices 14 for use by trial participants and trial investigators, computing devices 16 for use by trial monitors, computing devices 15 for use by a trial administrator, and computing devices 18 for use by laboratories to connect via the network 20 to computing devices 12, where the trial management program of the present invention is executed. The computing devices 12 maintain and utilize databases 11 which include an application database 1 la for defining trials, i.e., scopes of studies, a personnel database 1 lb for defining persons authorized to access the inventive system residing on the computing devices 12, and a trial database lie for saving and mamtøining information collected with reference to the trials of the applications database 11a. The databases 11 do not have to be physically distinct, any distinction is made herein only for the puφose of clarity of description, in fact these databases 11 may be subdivided in to more discreet units. The computing devices 12 gather the trial data submitted through the computing devices 14 and 16 for the puφose of collecting that information in the trials database lie. The network 20 may be any type of a network including the telephone network, the local area network, e.g., the Intranet, and the wide area networks, e.g., the Internet.
The computing devices 12, 14, 15, 16, and 18 may take the configuration of any computer ranging from mainframes and personal computers (PCs) to digital telephones and hand held devices, e.g., palm pilots™. I-n one illustrative embodiment of this invention shown in Figure 2, such computing devices may comprise a bus 30, which is connected directly to each of the following: 1. a central processing unit (CPU) 32;
2. a memory 34;
3. a system clock 36;
4. a peripheral interface 38 ;
5. a video interface 40; 6. an input/output (I/O) interface 42;
7. a communications interface 44; and
8. a multimedia interface 46.
The common bus 30 is further connected
1. by the video interface 40 to a display 50; 2. by the I/O interface 42 to a storage device 52, which may illustratively take the form of memory gates, disks, diskettes, compact disks (CD), digital video disks (DVD), etc.;
3. by the multimedia interface 46 to any multimedia component 56; 4. by peripheral interface 38 to the peripherals 58, such as the keyboard, the mouse, navigational buttons, e.g., on a digital phone, a touch screen, and/or writing screen on full size and hand held devices, e.g., a palm pilot ™;
5. by the communications interface 44, e.g., a plurality of modems, to a network connection 60, e.g., an Internet Service Provider (ISP) and to other services, which is in turn connected to the network 20, whereby a data path is provided between the network 20 and the computing devices 12, 14, 15, 16, and 18 (Figure 1) and, in particular, the common bus 30 of these computing devices; and
6. furthermore, by the communications interface 44 to the wired and/or the wireless telephone system 54.
Application Data
Before clinical trials are carried out they are defined by values that may be placed into the application database 11a (Figure 1). The entity, person, or persons creating, authoring, or instigating the trial driven by the inquiry into the hypothesis under study in a particular clinical trial is called the trial administrator. The trial administrator defines every aspect of any clinical trial, which may be administered by the inventive system.
Each aspect of the clinical trial of the inventive system is defined by one of a series of database tables 70, shown in Figure 3b. These tables 70 may be developed and managed with the help of any modern database management software, such as Oracle, Sybase, Microsoft Access™, and others. After being defined, the tables 70 are stored in the application database 11a (Figure 1). Each of these tables 70 references other tables 70 by the virtue of a relational architecture. The inventive system provides a method to enable the creation of the tables 70 to define the anticipated application data. The hierarchical sequence of the method is shown in Figure 3a. There, in step 172, the table defining parts of a clinical trial 72 (Figure 3b) is defined, including the structure of the trial, i.e., randomized, prospective, placebo-controlled, etc.; and the number of patients or trial participants to be enrolled in the trial.
Each of the records entered into the table 72 (Figure 3b) would then be further defined in other related tables in the database 11a (Figure 1). Thus, at step 174 the time intervals table 74 (Figure 3b) comprises timing information regarding individual tests may be defined, e.g.,
1. the actual intervals of the study;
2. the margin of flexibility allowable in defining the time intervals; and
3. the test determinations made at each interval.
The tests table 76 (Figure 3b), in turn, is defined in step 176 by listing the tests, e.g., the blood; the urine; the weight; and a quality of life questionnaire; to be performed and replied to. Elements of each test 76 (Figure 3b) may be further defined in step 178 in the elements table 78 (Figure 3b), which may comprise test parameters such as: 1. test name; 2. upper and lower limits of normal;
3. normal rates of change;
4. optional versus mandatory status of the test;
5. the type of data entry required, i.e., date, number, Boolean, text, etc.; and
6. the data entry vehicle used, i.e., checkbox, text field, a pointer to a data file such as a music format or a digital image, or an interfaced piece of hardware utilized to obtain data.
Each test element 78 (Figure 3b) may be determined to have particular exceptions. Exceptions are values or data points entered outside the predefined protocol intervals and tests. They are implemented in order to handle data for patient complications or adverse events, or other points of planned or unplanned data entry. In other words exceptions are a determination by the trial administrator of what is to be considered abnormal, e.g., body temperature below 96° and above 106° Fahrenheit. In step 180 exceptions 80 (Figure 3b) will be defined for each data element 78 (Figure 3b) of each test 76 (Figure 3b) calling for such exceptions. Not every element 78 (Figure 3b) or test 76 (Figure 3b) may require an exception.
The present application enables each part of one of the clinical trials in tables 72
(Figure 3b) as defined by its elements in related tables 78 (Figure 3b) is reusable, i.e., may be used to perform multiple clinical trials. In this manner, basic elements commonly found in clinical trials could be made available in a custom generated clinical trial. For example, the trial administrator trying to create a clinical trial may be presented with a panel of trial parts such as patient, doctor, blood test, heart exam, etc. Any trial element may be incoφorated into a trial in a "drag-and-drop" or other fashion.
The trial administrators desiring to create or design clinical trials may do so by establishing a datapath from the computing device 15 (Figure 1) to the computing device 12, via the network 20 to access the clinical trial management program of the present invention.
This can be accomplished by using an Internet-based browser program, e.g., Microsoft Explorer™, Netscape Navigator™. After connecting to the computing device 12 (Figure 1), as shown in Figure 4a, and entering the username/password combination 100, on the login screen 102 of the inventive system, the trial administrator may create a new trial and begin defining the various elements of the trial. Furthermore, although after the start of the trial modifications may not be accepted, to create new trials the trial administrator may modify and delete any existing trials. For example, Figures 4b and 4c show the clinical trial 72 comprising intervals 74 and tests 76. By clicking screen buttons 104 further components of the trial record 72, e.g., test elements 78 (Figure 3b) and the exceptions 80 may be displayed. Description "done" 106 indicates that the particular test 76 for the particular interval 74 has been performed.
Personnel Data As part of development of the clinical trial, the trial administrators will be able establish or modify an existing list of trial investigators and trial monitors in the personnel database 1 lb (Figure 1) for the particular trial being developed. Moreover, the trial investigators will be able to enter in to the personnel database 1 lb (Figure 1), lists of names of trial participant/patient selected to participate in the particular study after the trial is developed and is being carried out.
Each entry, i.e., a person's name and pertinent data, of the database lie (Figure 1) will have an associated set of pre-established permission standards, different permission standards for the trial participants, trial investigators, and trial monitors. These permission standards define the level of access available to each of the trial participants, trial investigators, and trial monitors involved in the trial to the application data in the database 1 la (Figure 1), the personnel data in the personnel database 1 lb (Figure 1), and the trial data in the database 1 lc (Figure 1). For example: 1. The trial participant patient would have access to subjective questionnaires in the application data in the database 11a (Figure 1) without access to any other data collected by the inventive system.
2. The trial investigator will have access to patient specific information in the database l ie (Figure 1), by patient name and by patient number as well as to summary data within the scope of his or her own results. The trial administrator may also be able to define the type of investigator enrollment, e.g., an open enrollment allowing anyone qualified to act as the trial investigator to sign up or register as the trial investigator in the personnel database 1 lb (Figure 1).
Alternatively, in an enrollment by invitation only, the trial administrator may choose to develop a trial whereby the trial investigators may have to make a telephone call, e-mail, or mail correspondence to the trial acfoiinistrator to request and to be furnished the login information. Before furnishing such login information, the trial administrator would update the entries of the personnel database 1 lb (Figure 1) to give these invited trial investigators access to the inventive system. The receipt of the login information would enable the trial investigator to sign up in the trial investigator registration saved in the personnel database 1 lb (Figure 1) on the computing device 12 (Figure 1).
3. The trial monitor will have access to patient specific information saved in the trial data database l ie (Figure 1) and identified by patient number, i.e., no patient names are revealed.
4. The trial administrator, in addition to full access to the application data database l ie
(Figure 1) will have access only to summary data of the trial database 1 lc (Figure 1) for the entire trial without specific patient information.
Trial Once the particular trial has been completely defined or created in the application data database 11a (Figure 1) the trial may commence. After the trial is activated, the parameters defining the trial in the database 11a (Figure 1) can no longer be manipulated. To facilitate creation of new trials, the entire existing trial may be copied to create a new trial and that trial's parameters may be modified.
To run a trial, the trial adnrinistrator contracts the trial investigators to select the trial participants, begin patient enrollment and data collection regarding these trial participants. The trial administrator further contracts the trial monitors to monitor the veracity of the trial data. The trial participants are selected and lists of names of the trial participants are entered into the personnel database 1 lb (Figure 1). At this point the clinical trial management program of the present invention, shown in Figure 5, may read in the parameters of the trial established in the application data database 11a (Figure 1) and automatically generate the appropriate questionnaires and forms to allow the trial participants of the trial to fill in the results of tests 77 (Figure 4f). As shown in Figure 5, the clinical trial management program 200 may be accessed by authorized personnel in step 202 through the login screen 102 (Figure 4a). In step 204, judging by the login and determining the level of access through the use of the personnel database 1 lb (Figure 1), a deteπnination is made of what level of authentication should be performed, or whether authentication is necessary all together. It may be determined that further or additional authentication is required, such as biometrics authentication of a single or multiple trial participants, e.g., in situations where the authentication is used to guarantee the physical interaction between a trial investigator and a trial participant. In such case authentication steps such as biometrics, e.g., iris or finger print scan, facial recognition, voice print, retinal scan, facial recognition, etc., and/or DNA authentication, e.g., the blood, urine, hair, saliva, tissue sampling, etc., in step 206 or other, e.g., entering social security number, special ID codes, in step 208 may be required before proceeding. Please note that the authentication may be performed as follows:
1. on the individual members of the clinical trials, e.g., the trial investigators and the trial participants;
2. on the plurality of members of the clinical trials concurrently, e.g., a plurality of the trial investigators, a plurality of the trial participants, a combination of the trial investigators and trial participants.
The concurrent authentication of a plurality of members is utilized by the present invention in situations where the authentication is used to guarantee the physical interaction between the trial investigators and the trial participants.
The authentication tests may use flexible timeouts defined by the trial administrator for single and multiple interactive sessions with the trial participant. Moreover, these sessions are location independent. Each computing device 14, 15, 16, and 18 (Figure 1) used in the authentication may be identified by a unique identification number, therefore the physical location of such components need not be fixed for the authentication puφoses.
As described above, the preferred embodiment of the present invention contemplates but is not limited to personnel with predefined purposes. Accommodation of personnel for many diverse puφoses, e.g., television, magazine, and newspaper reporters, colleges, hospitals, competitors, statisticians, insurance companies, etc., not described in the preferred embodiment may be easily provided by the inventive system and may be as easily created by these skilled in the art.
Participants/Investigators
After logging in step 202 and being authenticated in steps 206 or 208, in step 210 the trial investigators and the trial participants will be shown a list of possible tests 72 (Figure 4c). After selecting a particular entry 76a (Figure 4c) from the screen 72 (Figures 4c), the trial investigators and the trial participants may be presented with screens, e.g., 76a-c (Figures 4d-4f), and may start in step 212 to enter test result data.
These test screens 76a-c (Figures 4d-4f) are dynamically generated by the clinical trial management program 200. In step 210 the actions required, i.e., defined in the application data database 11a (Figure 1), are evaluated and only these required are presented. Furthermore, data entered in response may lead to additional options to be displayed. Actions that have been previously completed are displayed as such, giving instant feedback regarding what was previously entered into the system.
DATA STORAGE
Of course, if after glancing at the list of required actions displayed in step 210 the trial investigator or the trial participant decides not to proceed, in step 214 a logout path may be taken. An in-depth description of the logout procedure will be discussed below.
A transaction of the clinical trial management program 200 begins following the login in step 202 and the authentication in step 204. All the information entered during the session after the login and before the logout is considered a part of this transaction. The transaction will not be committed, i.e., be made a part of the permanent storage 52 (Figure 2) until the whole session is completed. As data elements 77 (Figures 4d-4f) are entered in to the test screens 76a- 76c (Figures 4d-4f), the clinical trial management program 200 dynamically executes scripts to perform the first level of validation of the data values 77 (Figures 4d-4f) entered. The executed scripts may validate entered data values 77 (Figures 4d-4f) and may be executed by the browser program, e.g., Microsoft Explorer™, Netscape Navigator™, using its built in functionality on the computing devices 14, 15, 16, and 18 (Figure 1). Moreover, the executed scripts may validate entered data values 77 (Figures 4d-4f) on the computing device 12 (Figure 1) to provide two levels of data validation.
These scripts compare the values 77 (Figures 4d-4f) entered, with previously set exception limits 80 (Figure 3b) defined in the application data database 11a (Figure 1). In the event where an entered value is outside of these exception limits the inventive clinical trial management program 200 will alert the trial investigator or the trial participant entering the data values 77 (Figures 4d-4f) and may request or require additional data to be entered. For example, a confirmation dialog, to insure that the out-of-range value is in fact correct will be inteφosed in step 216 and the trial investigators and the trial participants may be asked or forced to reply before proceeding.
After all the test values 77 (Figures 4d-4f) are completely filled in, step 216 saves these values 77 (Figures 4d-4f) in the current transaction, executes the scripts to perform the second level of validation as described above. The values 77 (Figures 4d-4f), entered for each element are once again compared with the exception ranges 80 (Figure 3b) and with the acceptable data change rate. The data change rate is determined by comparing values 77 (Figures 4d-4f) to these entered for the same element at previous intervals. Should the values 77 (Figures 4d-4f) be either out of the exception range 80 (Figure 3b) or change at a rate beyond the preset exception change rate for the given element, an exception record 270 (Figure 6) is added to the current transaction.
In step 218 it is determined if one or more exceptions are generated at the given interval for the given element. If exceptions are generated, in step 220 an exception record adds new tests to the trial participant's complete trial record and the elements within that new text, which must be completed to close the trial participant's participation in the trial. At this point it is possible to offer an option of completing the exception data immediately or to queue the exception for a later time. If in step 222 it is determined that the exception is to be completed now, the test form for the exception is presented and the data entry loop starting in the step 212 is repeated. Otherwise, in step 224 the exception is queued, and will then appear in the appropriate data entry options of the step 210.
After saving values 77 (Figures 4d-4f) in the transaction, and completing or queuing tests generated from exceptions, the values 77 (Figures 4d-4f) may be stored. Please direct your attention to Figure 6b showing the record for storing values 77 (Figures 4d-4f). Each record 270 comprises the following fields:
1. A unique key 270a used to identify the transaction.
2. A trial key 270b, for pointing to the specific trials table 72 (Figure 6a) of the application data database 11a (Figure 1) with which the present record 270 identifies or belongs.
3. An intervals key 270c for identifying the particular interval on the intervals table 74 (Figure 6a) of the application data database 11a (Figure 1) which corresponds to the specific value 77 (Figures 4d-4f) of the present trial identified by the trial key 270a (Figure 6).
4. A test key 270d for identifying the particular tests table 76 (Figure 6a) that the values 77 (Figures 4d-4f) are associated with.
5. An element key 270e for identifying or pointing back to the element in the elements table 78 (Figure 6a) that the values 77 (Figures 4d-4f) correspond to.
6. An identification key 270f for identifying the trial personnel who entered the values 77 (Figures 4d-4f), such personnel was previously defined in the personnel database 1 lb (Figure 1) as described above.
7. A patient identification key 270g associating the values 77 (Figures 4d-4f) with the particular trial participant, such trial participant was previously defined in the personnel database 1 lb (Figure 1) as described above.
8. A date and time created field 270h for storing the year, month, date, hour, minute, second and millisecond that the values 77 (Figures 4d-4f) were recorded.
9. A data certification code field 270i comprising a plurality of random digits.
The current methods of administering a clinical trial involve monitoring the trial centers to assure research integrity. The basic role of the trial monitor is to verify that the data entered by the clinical investigator corresponds to the records taken in the source documentation. The source documentation denotes the trustworthiest record of the data. This is because it contains records of the interactions between the trial investigator and the trial participant and the resultant outcome of any tests taken. The source documentation is held in such high regard because it contains the trial investigator's signature. Usually the source documentation is regarded as the trial participant's medical chart. The present invention describes a method by which the source documentation can be generated by the computer application. All new data values entered into the trial database are grouped according to the date, time, trial participant, computer address, and login episode. For each and every unit of data entered into the trial database l ie (Figure 1), a multi-digit unique identification number is generated at random. After each log in episode, the trial investigator's computing device 16 (Figure 1) may print a record of the activities of each patient handled by the trial investigator during any given login episode. This record will act as a legal record of the investigator's actions during a log in episode and be clearly marked with a statement affiπning that the investigator has read the record and agrees with it. The investigator will then be asked to sign the record. The signed record can then be utilized as the source documentation since it contains a record of patient data with a verifying signature. The document may be appropriately formatted for placement into the trial participant's chart.
Moreover, the inventive method makes the job of the trial monitor considerably easier. Prior to auditing a site where the trial tests are performed, the trial monitor may print out a list of data transactions and corresponding identification numbers. The trial monitor may then make certain that the events documented in the trial investigator's source documentation contains the correct identification numbers.
Since both the investigator source documentation and monitor transaction list are generated from the same server-side database, there is virtually no possibility for fraud or forgery in this. The trial investigator that tries to change the data values or other parameters entered into the database will create a discrepancy between the source documentation and the data transaction list output for the monitor.
10. A unique identifier field 270j for the identity of the hardware such as the IP address of a network 20 (Figure 1) connection 60 (Figure 2) of the computing devices 12, 14, 15, 16, and 18 (Figure 1).
11. An authentication stamp field 270k generated by the authentication method in steps 206 and 208 (Figure 5) for the session in progress.
12. An actual data record 2701 of the 77 (Figures 4d-4f) with data of various types, e.g., text, integer, binary, real, floating, date/time, images, biometrics, etc., keeping their appropriate data type.
13. Any supplemental data 270m entered along with the values 77 (Figures 4d-4f) as defined by the element corresponding to the values 77 (Figures 4d-4f), for example, an 'other' field in a multiple-choice question. Returning now to Figure 5, in step 216 each record 270 is recorded in the transaction that covers the entire session from login in step 202 to logout in step 214. I-n step 226 it is determined from the personnel database 1 lb (Figure 1) or from the application data database 11a (Figure 1) that a logout authentication is required. If authentication is required, in step 228 it is determined which authentication is to be performed, the biometrics, e.g., iris or fingeφrint scan, retinal scan, facial recognition, DNA imprinting using hair, saliva, etc., in step 230 or other, e.g., entering social security number, special I-D codes, in step 232 may be required before committing the transaction. Please note that the authentication may be performed as follows:
' 1. on the individual members of the clinical trials, e.g., the trial investigators and the trial participants; 2. on the plurality of members of the clinical trials concurrently, e.g., a plurality of the trial investigators, a plurality of the trial participants, a combination of the trial investigators and trial participants.
In step 234 it is determined if the required authentication at the end of the session was properly performed or failed. If the authentication failed, in step 236 the transaction is rolled back and the data does not become a permanent part of the trial data. In the alternative the transaction may become a permanent part of the trial data with a recorded indication that the logout authentication failed, or permanently logged outside of the trial data, however the transaction will be rolled back and the data will not become a part of the trial data in the trial data database l i e (Figure 1). In step 238, the transaction is committed, i.e., recorded as a permanent record in the trial data database lie (Figure 1).
Please note that additional determinations may indicate whether transaction comprising the values 77 (Figures 4d-4f) will be rolled back in step 236 or committed in step 238. A session may end because the timeout period specified for a particular or a group of the trial participants and trial investigators stored in the personnel database 1 lb (Figure 1) has been exceeded. Laboratories
A particularly useful feature of the present invention is its ability to accept results for individual trial participants as well as bulk results for groups of trial participants from laboratories connected by computing devices 18 (Figure 1) to the computing devices 12 (Figure 1) via the network 20 (Figure 1) or by a direct digital interface between the laboratory computing devices 18 (Figure 1) and the computing device 12 (Figure 1). Where a plurality of samples of the trial participant's specimens, e.g., blood, saliva or urine, are tested by a particular laboratory, the results may be securely transferred to the trial data database lie (Figure 1). The trial participants' privacy may be maintained by providing the laboratories only with the assigned trial participant unique number. The laboratory computing device 18 (Figure 1) may then cross-reference the trial database 11c (Figure 1) using the trial participant unique number and input the appropriate laboratory test result. This will eliminate the need for human intervention in inputting the test results and therefore the possibility for human-error or fraud
Monitors
As was previously described in the description of the background of the invention section, the trial monitor is the entity, person, or persons that constantly monitor the integrity of the trial insofar as the data entered and the conduct of the trial investigators and the trial personnel is concerned. The trial administrator often contracts the trial monitors as part of the CROs or as independent consultants. Typically the trial monitor maintains a record of the activities of the trial investigators and makes certain that the data submitted by the trial investigator corresponds to the source documentation, i.e., the patient's medical record. The trial monitor verifies whether the trial investigator has provided the proper informed consent and that the trial participant's characteristics conform to the inclusion and exclusion criteria specified in the trial protocol. Trial monitors will perform declared and undeclared visits to the trial investigator sites to audit the operations of the trial investigator in-person. The inventive system may provide the trial monitor with access to limited information in the trial data database lie (Figure 1). While the trial monitors may be free to view all trial participant information from the personnel database 1 lb (Figure 1), they may not be able to identify these trial participants except by a unique code assigned to each trial participant for puφoses of the trial. Upon entering the inventive system in step 202, the trial monitor may be asked to authenticate him or herself in steps 206 or 208. This authentication is accomplished using either a form of biometrics measurement in step 206 or by username and password method in step 208. Once the trial monitor is properly logged into the application running on the computing device 12 (Figure 1), they will have access to the entries of the trial participant related data stored in the personnel databases 1 lb (Figure 1) and in the trial data database l ie (Figure 1) referenced according to specific trial investigators. In step 240, the trial monitors will be able to get complete reports of transactions performed by individual trial investigators, e.g., 84 (Figure 4g), identified by data stamps including the following:
1. date and time of patient visit 86 (Figure 4g);
2. interaction and data entry 88 (Figure 4g);
3. Internet protocol address from which data was entered (not shown);
4. trial investigator identification 90 (Figure 4g); and
5. a unique encryption stamp for each transaction 87 (Figure 4g).
These reports may be created for each trial participant according to each time point, of the intervals 74 (Figure 3b), tracked as per the trial protocol. Furthermore, these reports may enable the trial monitor to verify the trial investigator's source documentation against the data shown in the report. Separate reports may be generated for data that is missing, entered late, changed by the trial investigator, or is out of range. This may allow the trial monitor to address each specific issue with the trial investigator and to reconcile the problem in the trial data. Separate reports may further be generated for every adverse event reported by the trial investigator. Each report may list the adverse event and the follow-up necessary to be performed by the trial investigator in order to document the ultimate consequences to the trial participant.
Summary reports of the trial monitor's verified values 77 (Figures 4c-4f) may be generate each time a trial monitor completes a review of either a single test element 78 (Figure 3b) or group of tests 76 (Figure 3b). The verified values 77 (Figures 4c-4f) will also be associated with identifying stamps such as date, time, trial monitor's biometrics authentication, etc. This report may ultimately be printed for approval by the trial investigator and signature.
Administrator The trial administrator may be given special access to certain forms of data while being restricted to others. For example, while the trial administrator may be able to view overall summary data about a trial, they will be unable to access patient-specific information in order to maintain patient privacy.
The application will have special functions available to the trial administrator. In general, the trial administrator will be able to have a real-time "bird's eye view" of the ongoing clinical trial. After logging in step 202, the trial administrator will be asked to authenticate him or herself as described above with reference to steps 204, 206 and 208. J-n step 242 the trial administrator will be enabled to obtain summary data of the trial and application-specific tools to allow manipulation of the summary data.
General functions available to the trial administrator could include summary information, analysis tools, and reporting tools. A summary information area allows the trial administrator access to updates on the status of various trial investigator sites. In doing so, the trial administrator is able to see the performance of each trial investigator via parameters such as the number of patients enrolled or the number of completed patients or the number of delinquencies or the number of adverse events or by the answers to satisfaction- questionnaires about the trial investigators rated by patients/trial participants.
Analysis tools may also be available to the trial administrator such that raw summary data representing the current state of the trial will be able to be manipulated. For example, the trial administrator may be able to apply various statistical analysis functions, i.e., averages, standard deviations, control for data variables, etc., to generate graphs and charts. The trial administrator would also be able to use analysis tools to compare two or more trial investigator sites in terms of performance and view the results in graph or chart format.
Reporting tools may be provided such that the trial administrator can generate reports. For example, a pharmaceutical company testing a new drug may want a report that conforms to the outline of a new drug application; the trial administrator, for academic reasons might want a report that is later submitted as a clinical paper to a medical journal. Templates for various kinds of reports will be available to the trial administrator. The trial administrator will be able to customize their own template, which may consist of a series of headings and hierarchical subheadings to format a document. The reports created may be integrated with the analysis tools such that graphs generated dynamically from the trial data will be incoφorated into the report. The application will allow for collaborative authoring and tracking of a document using a system by which parts of a document may be "checked-in" or "checked-out" by multiple trial administrators. The reports created will be able to be output in multiple formats compatible with other word processing software or document readers, i.e., Adobe Acrobat, or Microsoft Word. Ultimately upon completion of the report, the trial administrator may immediately transfer the document electronically to the end viewer such as a peer review committee or a governmental regulatory board.
While the invention has been particularly shown and described with respect to illustrative and preferred embodiments thereof, it will be understood by those skilled in the art that the foregoing and other changes in form and details may be made therein without departing from the spirit and scope of the invention that should be limited only by the scope of the appended claims.

Claims

CLAIMSHaving thus described our invention, what we claim as new, and desire to secure by Letters Patent is: 5
1. A method of conducting one or more clinical trial, the one or more clinical trial having a plurality of members, the plurality of members using one or more computing devices connected to a network to perform a plurality of transactions of the one or more clinical trial, said method comprising the steps of: 0 a) establishing procedural guidelines and the one or more clinical trial; b) perfoπning the one or more clinical trial according to the procedural guidelines by carrying out the plurality of transactions of the one or more clinical trial; c) authenticating said plurality of members and accepting only these of the plurality of transactions that are performed by said plurality of members that are authenticated; and 5 d) issuing a warning if a discrepancy is found when any of the plurality of transactions is carried out.
2. The method of claim 1, wherein the plurality of members comprises at least one trial administrator, at least one trial monitor, at least one trial investigator, and one or more 0 trial participants, the at least one trial administrator selecting the at least one trial monitor and the at least one trial investigator, the at least one trial investigator selecting the one or more trial participants to be enrolled in the one or more clinical trial, the at lest one trial monitor observing the one or more clinical trial to detect any deviations from the procedural guidelines.
-5
3. The method of claim 2, wherein the plurality of members further comprises: one or more reporters; one or more scientists, and one or more publishers.
4. The method of claim 3, wherein the at least one trial administrator selects said at 0 least one trial investigator according to an open enrollment, in said open enrollment anyone qualified to act as a trial investigator is permitted to register as the at least one trial investigator.
5. The method of claim 4, wherein the at lest one trial administrator selects said at least one trial investigator according to a by-invitation-only enrollment, in said by invitation only enrollment the at least one trial administrator registers said at least one trial investigator.
6. The method of claim 5, wherein each said one or more clinical trial comprises: an identification of the at least one trial administrator, and an identification of the one or more trial participants to be enrolled in the clinical trial; a schedule of a one or more periods of execution during which period a one or more test values are accepted; a plurality of tests for each of said one or more periods on said schedule; a plurality of elements for each of said plurality of tests; and a plurality of exceptions for setting limits on said one or more test values.
7. The method of claim 6, wherein each of the plurality of members has an associated set of pre-established permission standards, said permission standards define a level of access by each of the plurality of members to said one or more test values.
8. The method of claim 7, further including a step of allowing access by the at least one trial investigator to said one or more test values through a use of said identification of the one or more trial participants.
9. The method of claim 8, further including a step of allowing access by the at least one trial monitor to said one or more test values through a use of said identification of the one or more trial participants.
10. The method of claim 9, wherein said carrying out of the plurality of transactions comprises the following steps: determining a level of access allowed and a level of authentication necessary for a one or more of the plurality of members performing the plurality of transactions; performing an authentication method corresponding to said determined level of authentication, each of the plurality of transactions comprises steps performed by said one or more of the plurality of members after said performance of said authentication method; exhibiting said plurality of tests and said plurality of elements and permitting entry of a one or more test values in satisfaction of said exhibited plurality of elements; and accepting said one or more test values and determining if said accepted one or more test values are outside of said limits.
11. The method of claim 10, wherein said authentication uses biometrics and non biometrics methods.
12. The method of claim 11, wherein if said authentication method fails said plurality of transactions is not permanently stored.
13. The method of claim 12, wherein if said authentication method fails said plurality of transactions is marked as failed and is permanently stored.
14. The method of claim 13, wherein said authentication method starts the clock running on a time range within which the plurality of transactions must be carried out.
15. The method of claim 14, wherein the plurality of transactions is prevented from being carried out if said time range is exceeded, said time range is specified by said at lest one trial administrator for each of said plurality of members.
16. The method of claim 15, wherein said plurality of elements are dynamically generated for said one or more periods of execution, whereby only these of said plurality of elements that require said one or more test values to be entered are exhibited.
17. The method of claim 16, further comprising steps of exhibiting a confirmation dialog to insure that said entered one or more test values determined to be outside of said limits are correct, and if not correct then insuring that said entered one or more test values be corrected.
18. The method of claim 17, wherein said entered one or more test values comprise: a unique key for identifying a transaction; a trial key for identifying the clinical trial; a test key for identifying said first test of said plurality of tests of the clinical trial identified by said trial key; an intervals key for identifying each of said one or more periods of execution of said first test; an element key for identifying each of said plurality of elements; an identification key for identifying said at least one trial investigator and said at least one trial participant who entered said information value; a patient identification key for identifying each said at least one trial participant; a date and time created field for storing temporal values of recording of said information value; a data certification code field comprising a plurality of random digits; a unique identifier field to identify the computing devices; and an authentication field for identifying said authentication method performed.
19. The method of claim 18, further comprising a step of producing a plurality of reports of the plurality of transactions, said plurality of reports are created from said one or more test values and retrieved according to one or more keys comprising: a date and time of each of said plurality of transactions; a network address of the computing devices from which information was entered; an identification of the at least one trial investigator; an identification of the computing device on which said plurality of transactions was entered; and a unique encryption stamp for each of said plurality of transactions.
20. The method of claim 19, wherein said step of producing said plurality of reports enables finding said discrepancy and thereby verify veracity of the one or more clinical trial.
21. The method of claim 20, wherein said step of producing said plurality of reports .enables evaluation of performance of the at least one trial investigator.
22. The method of claim 21, wherein the network is Internet.
23. The method of claim 21, wherein the network is a telecommunications network.
24. A method of authenticating performance and recordation of a transaction during execution of a clinical trial to guaranty that the transaction occurred in a proscribed manner and was performed by authorized personnel members, the transaction being performed by the authorized personnel members from a plurality of computing devices connected to a network by using data entry programs, the transaction being recorded on an at least one computing device executing a trial management program, said method comprising the steps of: a) establishing in the clinical trial a schedule of a plurality periods of execution during which period a plurality of test values are accepted; a plurality of tests for each of said plurality of periods, a plurality of elements for each of said plurality of tests, and a plurality of exceptions for setting limits on said one or more test values; b) the authorized personnel members accessing the at least one computing device executing the trial management program and identifying themselves; c) determining a level of access allowed and a level of authentication necessary for the authorized personnel members; d) presenting said plurality of tests and said plurality of elements and initiating a timer on a time range within which the transaction must be completed; e) accepting a plurality of test values in satisfaction of said plurality of elements; and f) recording said plurality of test values of the transaction if the transaction was completed with in said time range.
25. The method of claim 24, wherein said authentication uses biometrics and non biometrics methods.
26. The method of claim 25, further comprising a step of determining if said accepted plurality of test values are outside of said limits.
27. The method of claim 26, further comprising steps of exhibiting a confirmation dialog to insure that said plurality of test values determined to be outside of said limits are correct, and if not correct then accepting a new plurality of test values.
28. A method of authenticating the execution and recordation of at least one clinical trial, said method comprising the steps of: a) constructing a first database with a set of first records defining the at least one clinical trial to be executed, a set of second records defining dates on which tests of the at least one clinical trial will be executed, a set of third records defining respectively said tests executed corresponding to said dates, and a set of fourth records defining respectively elements of said tests, said elements are parameters to be tested by the corresponding tests, b) constructing a second database to list only those trial participants that are permitted to enter and access data of the at least one clinical trial; c) requesting permission to enter trial data by providing identifying information of said trial participant who will enter said trial data and on whose behalf said data is being entered and comparing said identifying information with the list of said trial participants in said second database and authenticating and permitting entry of said trial data if there is a match; d) accepting said trial data from said authenticated trial participant, said trial data comprising timing information of duration of entry of said trial data and the parameters to which the trial data relates; and e) testing said accepted trial data by comparing said timing information of duration of entry and the parameters with those timing information of duration and parameters in the first database and, if there is a match, accepting said trial data.
29. The method of claim 28, wherein said first database is further constructed with a set of fifth records defining value ranges of said elements, and testing whether the trial data related to the parameters falls within said value ranges.
30. The method of claim 29, wherein said accepted trial data is stored in a third database.
31. A method of authentication of the conduct and recordation of one or more clinical trials, the one or more clinical trials having a plurality of members including at least one trial administrator, at least one trial monitor, at least one trial investigator, and one or more trial participants, the at least one trial administrator selecting the at least one trial monitor and the at least one trial investigator and defining procedural guidelines for the one or more clinical trials, the at least one trial investigator selecting the one or more trial participants to be enrolled in the one or more clinical trials, the at lest one trial monitor observing the one or more clinical trials to detect any deviations from the procedural guidelines, the plurality of members using one or more computing devices connected to a network to perform a plurality of transactions of the one or more clinical trials, the plurality of transactions being stored in a third database, the method comprising the steps of: a) establishing in a first database a plurality of tests and procedural guidelines for each of the one or more clinical trials, each of said plurality of tests comprising a one or more periods of execution, a plurality of elements, and a plurality of exception limits and defining in a second database the plurality of members; b) recruiting and registering the plurality of members in said second database, each of the plurality of members being assigned level of access to a third database; c) performing the one or more clinical trials according to the procedural guidelines by carrying out all of the plurality of transactions making up the one or more clinical trials; and d) querying said third database at random to assure veracity of the one or more clinical trials and notifying the at least one trial administrator if discrepancy is found, the at least one trial administrator being in a position to prevent fraud.
32. A method of authenticating performance of a transaction during execution of a clinical trial to guaranty that the transaction occurred in a proscribed manner, the transaction being entered through a data entry program executing on a first at least one computing device into a third database managed by a trial management program executing on a second at least one computing device, said first at least one computing device and said second at least one computing device are connected to a network, said method comprising the steps of: a) establishing in a first database progression of transactions of the clinical trial comprising a schedule of a plurality of periods of execution during which period a plurality of test values are accepted; a plurality of tests for each of said plurality of periods, a plurality of elements for each of said plurality of tests, and a plurality of exceptions for setting limits on said one or more test values; b) establishing in a second database a list of a plurality of trial members enrolled to perform said progression of transactions; c) said trial members performing said progression of transactions by executing the data entry program, the trial management program identifying said trial members and determining a level of access allowed and performing a level of authentication necessary; d) displaying said plurality of elements and accepting a plurality of test values in satisfaction of said plurality of elements; and f) recording said plurality of test values if said plurality of test values was received with in a time range within which the transaction must be completed.
PCT/US2001/007533 2000-03-10 2001-03-09 Method of conducting secure clinical trials and guaranteeing the occurrence of an event WO2001069490A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
AU2001240128A AU2001240128A1 (en) 2000-03-10 2001-03-09 Method of conducting secure clinical trials and guaranteeing the occurrence of an event

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US52355100A 2000-03-10 2000-03-10
US09/523,551 2000-03-10

Publications (1)

Publication Number Publication Date
WO2001069490A1 true WO2001069490A1 (en) 2001-09-20

Family

ID=24085472

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2001/007533 WO2001069490A1 (en) 2000-03-10 2001-03-09 Method of conducting secure clinical trials and guaranteeing the occurrence of an event

Country Status (2)

Country Link
AU (1) AU2001240128A1 (en)
WO (1) WO2001069490A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1376444A1 (en) * 2002-06-27 2004-01-02 Abb Research Ltd. System and method for conducting a clinical trial
EP1434156A2 (en) * 2002-12-23 2004-06-30 Siemens Aktiengesellschaft A method for conducting a clinical study

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5991731A (en) * 1997-03-03 1999-11-23 University Of Florida Method and system for interactive prescription and distribution of prescriptions in conducting clinical studies
US6081786A (en) * 1998-04-03 2000-06-27 Triangle Pharmaceuticals, Inc. Systems, methods and computer program products for guiding the selection of therapeutic treatment regimens

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5991731A (en) * 1997-03-03 1999-11-23 University Of Florida Method and system for interactive prescription and distribution of prescriptions in conducting clinical studies
US6081786A (en) * 1998-04-03 2000-06-27 Triangle Pharmaceuticals, Inc. Systems, methods and computer program products for guiding the selection of therapeutic treatment regimens

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
DATABASE ABI/INFORM [online] APPLEBY CHUCK: "A web-flattened playing field has drug firms converging onto the continum of care", XP002940429, retrieved from 02116386 accession no. Dialog Database accession no. 67331945 *
E-HEALTH & PHARMACEUTICALS, HEALTH FORUM JOURNAL, vol. 44, no. 1, January 2001 (2001-01-01) - February 2001 (2001-02-01), pages 17 - 21 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1376444A1 (en) * 2002-06-27 2004-01-02 Abb Research Ltd. System and method for conducting a clinical trial
EP1434156A2 (en) * 2002-12-23 2004-06-30 Siemens Aktiengesellschaft A method for conducting a clinical study
EP1434156A3 (en) * 2002-12-23 2007-04-18 Siemens Aktiengesellschaft A method for conducting a clinical study

Also Published As

Publication number Publication date
AU2001240128A1 (en) 2001-09-24

Similar Documents

Publication Publication Date Title
US7596541B2 (en) Method and apparatus of assuring informed consent while conducting secure clinical trials
US11568963B2 (en) Method for creating and using registry of clinical trial participants
US7532942B2 (en) Method and apparatus for generating a technologist quality assurance scorecard
US11798660B2 (en) Methods and systems for data analysis
US20160125550A1 (en) Apparatus and method for processing and/or for providing healthcare information and/or healthcare-related information
US20070067189A1 (en) Method and apparatus for screening, enrollment and management of patients in clinical trials
US20100106531A1 (en) System and method for distributed analysis of patient records
US20010037219A1 (en) Systems, methods and computer program products for facilitating one-to-one secure on-line communications between professional services providers and remotely located clients
CN103688263A (en) Patient-interactive healthcare system and database
US20040193448A1 (en) Touch-screen applications for outpatient process automation
US20230177463A1 (en) System and Method for Screening Potential Test Subjects for Participation in Recent Trials
US8688475B2 (en) Method and system for clinical trial compliance
US20150012284A1 (en) Computerized exercise equipment prescription apparatus and method
Dünnebeil et al. Modular architecture of value-added applications for German healthcare telematics
US10510437B2 (en) Method for creating and using registry of clinical trial participants
WO2001069490A1 (en) Method of conducting secure clinical trials and guaranteeing the occurrence of an event
US20220270729A1 (en) Systems, devices, and methods for generating and displaying medical report interfaces and reports for communication to a third party
CAMACHO KNOWLEDGE AND OUTCOMES MANAGEMENT
Lackner et al. STUDY PROTOCOL
Roland et al. Clinical trial operations
Pae Jr et al. Database: relevant or not
Jones et al. Information is power: measure it, manage it
Alawaji Privacy and security risks for national health records systems
Mwangi Strengthening The Quality Of Hiv Data In Kenya: Tracing Of Patients Lost To Follow Up And Reduction Of Low Linkage
Thompson Physician report cards: now and in the future.(Value-Based Health Care)

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY BZ CA CH CN CR CU CZ DE DK DM DZ EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT TZ UA UG US UZ VN YU ZA ZW

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE TR BF BJ CF CG CI CM GA GN GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
122 Ep: pct application non-entry in european phase
NENP Non-entry into the national phase

Ref country code: JP