WO2000031666A1 - Computer network based system and method for collecting and reporting data - Google Patents

Computer network based system and method for collecting and reporting data Download PDF

Info

Publication number
WO2000031666A1
WO2000031666A1 PCT/US1999/027695 US9927695W WO0031666A1 WO 2000031666 A1 WO2000031666 A1 WO 2000031666A1 US 9927695 W US9927695 W US 9927695W WO 0031666 A1 WO0031666 A1 WO 0031666A1
Authority
WO
WIPO (PCT)
Prior art keywords
survey
data
database
report
method recited
Prior art date
Application number
PCT/US1999/027695
Other languages
French (fr)
Other versions
WO2000031666A9 (en
Inventor
Daniel Short
Molly Riness
Wes Kempfer
Original Assignee
Information Works, Llc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Information Works, Llc filed Critical Information Works, Llc
Priority to AU19190/00A priority Critical patent/AU1919000A/en
Publication of WO2000031666A1 publication Critical patent/WO2000031666A1/en
Publication of WO2000031666A9 publication Critical patent/WO2000031666A9/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/02Marketing; Price estimation or determination; Fundraising

Definitions

  • the present invention relates to computerized market research tools, and in particular to a computer network based system and method for collecting and reporting data, preferably market research data, such as survey data.
  • a second example is 'SurveySite', which uses a pop-up invitation implemented in Java which acts as a link to an HTML survey hosted at a separate site A report is generated and delivered to the organization commissioning the research at the end of the survey's life, e g after a pre-determined number of surveys have been responded to
  • a final example, which employs a very different methodology to the previous examples, is 'RelevantKnowledge'/Nielsen Group A group of people (called a web panel or research panel) agree in advance to have their Internet usage monitored and then periodically a log of which Web Sites these people have visited is uploaded to a central collection point, and a report is generated
  • the present invention provides a computer network based system and method for collecting and reporting data, preferably market research data in real-time Whatever the application to which the invention is applied, particularly where it is market research oriented, the real-time feedback it provides can enable timely response to emerging issues, can enhance training/feedback for support staff and can give the ability to promptly see problem areas through the users' eyes From one aspect, the invention provides a computer network based, preferably
  • a software implemented data collection tool is deployed to a client on the computer network (for example a Web browser) to elicit data from a user accessing the computer network through that client
  • the collection tool returns the elicited data to a research database, which stores the elicited data
  • the research database is made accessible to one or more designated persons, typically a commissioner of the research work and/or someone with their authority
  • the data collection tool can be deployed to successive users of the computer network, accessing the network via the same or, more typically, a different client
  • the deployment of the data collection tool is initiated by the user visiting a particular pre-defined trigger page of information on the computer network, or any one of a number of such pre-defined trigger pages
  • each trigger page will be associated with a particular information site of the organization commissioning the research For example, where the system is implemented on
  • the system preferably has the ability to delay the actual commencement of data ehcitation
  • the data collection tool may have the ability to linger within the client for a period of time after its deployment, during which time the client device can be operated to perform other functions
  • the data collection tool will provide a survey or other data form, presenting the user with a series of questions to be answered For some questions the user may be presented with a series of possible responses from which they must choose one, or in some cases one or more Other questions may require a response by way of a value on a continuous scale, and another option is to allow the user to enter free text answers Where there is some control placed on the response to a question such that the response is predictable, the system may be designed to use the response to that question to decide whether a subsequent question in the survey be used Once all of the answers to the survey have been provided, the data collection tool delivers the answers to the research database for access by the reporting tool at the report site In addition to the answers themselves, the data collection tool may also collect and return other types of information, for example information indicating the state of completion of a survey, or other similar sample incidence information In some embodiments of the system, the data collection tool conducts one or more analyses of the data to produce one or more report results which are stored in the database The analysis or analyses may be used.
  • the research database is preferably accessible via a report site on the computer network which, if desired, can provide secure access to the database
  • the report site may be set up so that a pre-arranged password is required before the report site enables the research database to be accessed, the password only being made available to pre- autho ⁇ zed designated persons
  • the results of the market research, or other data collection process can be made available only to the desired parties, for example the commissioner of the research or persons with their express consent
  • the report site preferably also provides a user friendly front end to the database, and comprises or works in conjunction with a software implemented report tool to extract relevant report results from the database and present them to the designated person(s) as an appropriately formatted report
  • the results will be presented graphically as bar charts, pie charts, tables or the like, but any one of a large number of other graphical or textual formats are possible
  • the report tool has the capability to manipulate the report results and/or carry out one or more analyses of them, in order to present the designated person(s) with a variety of differently focused reports
  • the system may allow the designated person to select one of a number of focused reports, or even to define the focus of the report themselves
  • the data collection tool can be deployed from a server (referred to hereinafter a "survey server") associated with the computer network, other than a server (referred to hereinafter as an "information server") on which the information site containing the trigger page or pages is located
  • a server referred to hereinafter as an "information server”
  • the research database is also preferably resident on a server (referred to hereinafter as a "report server") other than the information server, as is the report site and report tool
  • the research database, the report site and the report tool are preferably resident on a single report server which conveniently may be the same server as the survey server from which the data collection tool is deployed
  • the data collecting and reporting system elicits information from the users of a plurality of distinct information sites on the computer network, which may be resident on the same or different information servers For each information site the same data collection tool and/or the same report tool may be employed Alternatively, custom versions of one or both of these tools may be provided for use with one or more of the information sites served If different, customized collection and report tools are used for respective ones of the information sites served bv the system, thev are preferably each based on a respective one of a master data collection tool and a master report tool, which provide common functions generally required by most, if not all market research applications or, if applicable, other data collection applications to which the system applies In this way, the effort required to provide a custom data collection and/or report tool for any particular application can be minimized
  • all of the data elicited may be stored in a single research database, but access to the data is preferably controlled so that the report results associated with the distinct information sites are only accessible to the respective designated person or persons
  • each designated person may be given a unique access name and corresponding password which they use to enter the report site, the report site and report tool utilizing the unique access name, if verified with the corresponding password, to determine which report results should be made accessible to a particular designated person Brief Description of the Drawings
  • FIGURE 1 schematically illustrates a market research data collection and reporting system that operates in accordance with an embodiment of the present invention
  • FIGURE 2 is a block diagram showing the structure of the client side component of the data collection tool of the system of FIGURE 1 ,
  • FIGURE 3 schematically illustrates the procedure by which the system of FIGURE 1 collects data
  • FIGURES 4 to 17 show screen shots from an exemplary survey conducted over the Internet using the inventive system
  • FIGURES 18 to 28 show pages from an exemplary report site according to the present invention Detailed Description of the Preferred Embodiment
  • the present invention is a computer network based system and method for collecting and reporting data, preferably market research data in real time
  • data preferably market research data
  • the primary application of the invention is seen to be in the field of gathering customer feedback regarding the quality of services, information and products that are being provided by an organization doing business on the World Wide Web, and the following description of an exemplary illustrates such an application
  • the invention is also applicable to gathering other types of data, whether market research data or other informational data in any of a number of computer network environments In this sense it can be considered as a general purpose market research tool
  • a data collection and reporting system is implemented on the World Wide Web, data exchange between various components of the system being effected over the Internet 2
  • Visitors to a specific information site 4 on the Web typically a Web site of the research commissioner, are presented with a survey including a number of questions
  • the answers to these questions given by the visitors are stored in a database 6 and made immediately accessible to a designated person or persons, typically the research commissioner, by way of one or more user friendly survey reports available on-line over the Internet 2 at a report Web site 8
  • the survey remains active for a finite period defined bv the number of visitors to take the survey an overall time duration or a combination of the two
  • the results remain accessible to the designated person(s) for a further pre-defined post-survey time period
  • the survey results can be down-loaded from the survey database 6, in order that a permanent record may be kept by the research commissioner Additionally, any of the graphical user's page, or the post-survey period.
  • a single surve) server 14 operating the inventive system may be used to simultaneously conduct more than one survey, each survey being associated with the same or a different Web site or page on the World Wide Web For each Web site or page the same fundamental data collection tool is employed, although an expanded survey (including additional questions) or otherwise customized survey (e g including revised questions and/or graphic images) may be used Where multiple surveys are being conducted, the survey results from each survey are stored in the database 6 in a manner identifying them with their respective survey, in order that the report site 8 and report tool 10 can distinguish them It is also likely that the designated person(s) will be different for the different surveys Consequentlv, each designated person is assigned a unique user name, verified by a password, for accessing the report site 8 and hence the survey results stored in the survey database 6 In this way it is possible to restrict access to the survey results associated with particular surveys only to respective authorized designated persons
  • the survey applet 16 conducts a validation operation to determine whether to proceed with the survey, in which it is determined, for example (a) whether the implementation of the survey is supported by the visitor's Web browser 18, (b) whether the survey has been previously executed in the current browser session, and (c) whether the IP address from which the visitor is accessing the network is on a list of addresses excluded from participation in the survey If these and/or other validation tests are passed, that is to say, for example, the visitor's browser 18 will support the survey, the survey has not been previously executed and the visitor is not excluded, the applet 16 proceeds to the next step in the procedure If any one of the tests is failed, the survey applet 16 exits without ever presenting itself to the user Also in this preferred implementation, the frequency at which the survey is presented to visitors to the trigger page 12 can be controlled That is to say the survey need not be presented to each visitor who passes the above validation operation This frequency control is
  • the survey applet 16 also has the capability to provide a further front end filter to the survey If desired rather than launching straight in to the full survey itself, the visitor can be presented with an invitation to take the survey This invitation can give the visitor the opportunity to decline to take the survey (such a decision to decline itself being recorded as the survey data), can be used to "screen" for a specific type of visitor (e g visitors with a specific connection speed or other screening information), and can also be conveniently used to offer the visitor an incentive for completing the survey
  • the survey applet 16 presents the visitor with a "START SURVEY” icon, along with instructions on the completion of the survey and, if desired, a suggestion to them that they complete their business at the Web site before taking the survey, in order that their responses might be based on a more complete review of the Web site
  • the "START SURVEY” icon exists in its own window separate from the main browser window, so it may be minimized or lowered while the visitor continues to browse the information Web site 4
  • they When the user is ready to commence answering the survey questions, they simply activate the "START SURVEY” icon as they would any other icon (e g by clicking on it with their mouse), and they are presented with the first question
  • the data collection tool comprises both server side and client side components, which communicate via the Internet 2 using the standard HTTP protocol
  • the server side components are resident on, and are executed by the survey server 14 They are currently implemented as a series of servlets, which run under the JavaWebServer 1 1 environment Thev could however, be implemented to run under any HTTP server that supports Java servlets or other equivalent applets
  • the survey servlet 20 which, at the request of the client side component 16, extracts the appropriate survey configuration data from the survey database 6 and posts it to client side component 16
  • the survey servlet 20 is first passed a pre-assigned unique survey job code by the client side component 16
  • the client side component 16 itself obtains the unique survey job code from the trigger on the trigger page 12, the code being one component of the trigger
  • the survey servlet 20 retrieves the approp ⁇ ate configuration data from the survey database 6, puts it in name-value pair format and posts it to the client side component 16
  • the configuration data includes the following elements
  • “Survey class name” this is a class file that determines the question set for each survey Each different survey may be associated with its own unique survey class, in which case the text of the questions, which make up the survey can be coded into the class However, in a preferred implementation a generic -13-
  • survev class is used for a number of different surveys, the generic class configuring itself for any particular survey by retrieving the respective question set from the survey database
  • the survey class name will be the same for all surveys sharing the generic survey class "Survey Job Code" - this is the key to retrieving the correct survey configuration for a particular survey It is passed to the client side component 16 of the data collection tool as a parameter from the trigger
  • the visitor is presented the survey questions in a question frame which contains two panels (1) a question panel in which the question text is displayed, and ( ⁇ ) a response panel in which the visitor makes their response
  • the "Question Color” parameter determines the background color of the question panel
  • the second server side component of the data collection tool is a data grabber servlet 22
  • This servlet accepts the survey data from the client side component 16 and updates the survey database 6, which is preferably a relational database, with this data More specifically, the client side component 16 sends the survey data to the survey server 14 over the Internet 2 as survey data post requests The data is posted in name-value pair format
  • the data grabber servlet 22 accepts the HTTP request, extracts the survey data and posts it to the survey database 6 using, in the currently preferred embodiment, the JDBC API (a standard feature of JDK1 1)
  • a question servlet 24 which accepts question text requests from the client side component 16, extracts the approp ⁇ ate question text from the survey database 6 and returns the question text to the client side component 16
  • the question data is returned to the client side component 16 in string format via an HTTP response
  • the client side component 16 parses the string formatted data and uses it to initialize Java question objects
  • the question text itself is designed with a view to the particular survey being undertaken
  • the currently envisaged application for which the exemplary system according to the invention is designed is the gathering of customer feedback regarding the quality of services, information, and products that are being provided by an organization doing business on the World Wide Web Suitable standard, core questions for this application are given in Appendix I, although in any particular case it may be desirable to add further, custom questions, or to modify some of the standard questions to provide the desired focus for the market research being undertaken
  • it may be returned using object serialization if the programming environment allows, for example if the survey applet is running under JDK1 1 Object serialization allows an object instance to
  • the final server side component of the data collection tool is a counter servlet 26 which accepts data packages from the client side component 16 containing survey progress data, that is data indicating the progress of the survey, rather than data representing the survey answers themselves It is envisaged that this data could be used to detect when a visitor reaches particular milestones in the completion of the survey, and would enable analysis of the actions of visitors who start the survey but drop out prior to completion
  • the survey server 14 also holds a copy of the client side component 16 of the data collection tool, which is deployed to, and executed within the visitor's Web browser 18
  • the client side component 16 is implemented as an applet written in the Java language and will be referred to as the survey applet 16 It provides the interface with the visitor through which the survey data is collected More particularly, the survey applet 16 once downloaded and executed within the visitor's browser 18, presents the visitor with a series of questions one at a time, in a window separate from the browser 18 The visitor responds to each question in turn and once they have committed a response the applet 16 does not allow the response to be changed
  • the applet 16 initially records each response on the client side, and then when all of the survey questions have been answered, the responses (l e survey data) are posted to the survey server 14
  • a further advantage is that since the server applet 16 runs in a separate window, it can be minimized allowing the user to explore the Web site 4 they are visiting before starting the survey This may increase the likelihood of a more informed evaluation of the site by the visitor
  • the use of the Java applet also provides a more versatile and robust user interface, for example it is not affected by connectivity problems once up and running, and offers a superior development environment that results in both source code that is easier to debug, maintain, and modify incrementally and research results that are methodologically more reliable than in other known on-line survey tools.
  • FIGURE 2 is an object diagram for a preferred implementation of the survey servlet
  • the APPLET LOADER 30 is the component initially downloaded from the survey server 14 by the visitor's browser 18 when the visitor requests the trigger page 12, containing the trigger, from the information server 4 Its functions are to download from the survey server the necessary Java classes, to present the invitation to take the survey to the visitor, and to instantiate the CLIENT SPECIFIC SURVEY 32 and call the doSurveyQ method, when the INVITER 34 indicates a positive answer by a visitor to the invitation
  • CLIENT SPECIFIC SURVEY 32 for which the WEBMETRICS class 36 is a superclass overrides the doSurveyQ method, and instantiates the QUESTION FRAME 38, the concrete QUESTION objects 40 and CGI POST 42 In addition it passes, in turn, a reference to each QUESTION object 40 to the QUESTION FRAME howf) method, to cause each question of the survey to be presented in turn
  • the survey may be configured so that the presentation of certain QUESTION objects 40 is dependent on an answer to a previous question For example, one might only wish to ask the visitor what industry sector they work in if they have already indicated that they were visiting the information site of the research commissioner for business purposes Skip logic such as this, and other similar question controls are not available in conventional HTML survey tools
  • CLIENT SPECIFIC SURVEY 32 calls the getResponseQ method of QUESTION 40 and the addPostVah ⁇ e() method of CGI POST 42 to retrieve the visitor's response to the question and update CGI POST 42 with the response Finally, once all of the survey questions have been answered, CLIENT SPECIFIC SURVEY 32 calls the CGI POST postQ method to return the answers to the survey server 14
  • QUESTION FRAME 38 contains QUESTION 40 and displays QUESTION 40 when its showQ method is called by CLIENT SPECIFIC SURVEY 32 QUESTION FRAME 38 also handles action events from NEXT BUTTON 44, which accepts the visitor's indication that they have answered the current question A NEXT BUTTON event handler within QUESTION FRAME 38 calls the sioreResponseQ method of QUESTION 40 and updates PROGRESS METER 46 as each question is answered If the visitor attempts to move on to the next question without having competed the answer to the current question, the NEXT BUTTON handler causes an appropriate error message to be displayed to the visitor, or takes other corrective action if the survey methodology so indicates QUESTION 40 contains a QUESTION PANEL 48 in which the question text is presented to the visitor, and a RESPONSE PANEL 50 which presents question response choices to the visitor and interacts with the visitor to capture their responses When its storeRespomeO method is called by QUESTION
  • CGI POST 42 functions to send the captured responses in name-value pair format to the data grabber servlet 22 of the survey server 14 from ts post () method
  • the data collection process is initiated when the visitor requests a trigger page 12 on a Web site 4 associated with an enabled survey, as they would request a page on any other Web site
  • the trigger page 12 can be any ordinary HTML page on the Web site 4, additionally containing the survey trigger
  • this survey trigger is an applet tag, which the web master can simply insert into the HTML source for one or more of their web pages
  • this applet tag, or one like it can be small (for example only 5 by 5 pixels) and colored to match the Web page 12 it resides on, thus avoiding any disruption of the appearance of the page
  • FIGURE 3 illustrates the complete process
  • FIGURES 4 to 17 show screen shots taken during the process
  • the visitor 52 accessing an information web site ("customer Web site") 4 via a Web browser 18, requests the trigger page 12 from the Web site 4 (typically the Web site of the research commissioner) 2)
  • the Web site 4 returns the requested trigger page 12 which contains an applet tag similar to the example given above 3)
  • the codebase parameter for the applet tag on the trigger page 12 directs the visitor's browser 18 to request the survey applet ("WebMet ⁇ cs applet) 16 from the survey server (“WebMet ⁇ cs server”) 14 4)
  • the survey applet 16 is returned to the browser 18 by the survey server 14 along with related Java classes
  • the survey applet 18 requests survey configuration information from the survey server 14 The configuration information is requested on the basis of the unique survey codebase included in the applet tag -20-
  • the applet 16 executes a validation sequence to determine whether or not to present the visitor 52 with an invitation to take the survey It checks whether or not it is supported under the visitor's browser 18, whether it has been previously executed in the current browser session, and whether or not the visitor ' s IP address has been blocked If all those tests pass, the survey applet 16 finally determines whether or not to present a survey invitation to the visitor based on the configured invitation frequency, / which is downloaded as part of the survey configuration in step 7
  • the value of/ is an integer that determines the probability that the visitor 52 will be presented with an invitation — the greater the value of/ the lower the probability If/ is equal to 1 , then every visitor whose browser and IP address pass the initial validation will be presented with an invitation If the validation sequence fails, the applet quietly exits 9) If the validation succeeds, the visitor 52 is presented with an invitation 60 to take the survey (see FIGURE 4)
  • the survey applet 16 posts the fact that an invitation 60 was presented to the survey server 14 (step 9a)
  • the invitation 60 may or may not
  • the requested classes are downloaded to the survey applet 16 from the survey server 14
  • the visitor 52 is presented with a status window for the normally brief duration of the download Delaying downloads of some of the classes up until this point reduces the time taken to initially download the survey applet 16 -21-
  • the survey server 14 returns the survey questions to the survey applet 16
  • the visitor 52 is presented with a status window for the normally brief duration of this download Delaying question download up until this point reduces the initial download time of the survey applet 16
  • the visitor 52 is presented with survey instructions 66 (see FIGURE 5) followed by a start survey icon 68 (see FIGURE 6) The visitor is instructed and encouraged to continue with their visit to the information web site 4 before continuing with the survey
  • the visitor 52 explores the information web site 4
  • the start survey icon 58 meanwhile exists as a separate window on the visitor's desktop As such, it may be minimized or lowered while the visitor interacts with the browser (see FIGURE 7), and effectively "floats" with them as they browse 17)
  • the visitor 52 is presented with the question frame, also in a window separate from the browser 18
  • the question frame 70 presents the user with one question at a time, examples of which are seen in FIGURES 8 to 13
  • Each question frame displays a banner 72, in which the name of the research commissioner can be displayed for example, a question panel 74, in which the question text is displayed, and a response panel 76 in which the response options are given where applicable and in which the visitor enters their response
  • the question frame 70 displays a progress bar 78 indicating the visitor's progress through the survey, and a "NEXT" button 80 which the visitor clicks, having made a response, to move on to the next question
  • FIGURES 8 to 13 shows a list of mutually exclusive possible responses alongside respective option buttons 82 that the visitor can click on to make their selection of response
  • FIGURE 9 the response panel displays a free format text box 84 in which the visitor is invited to type comments
  • FIGURE 10 illustrates a response panel with two sliders
  • Step 19a The visitor enters a response to each question in turn and after responding clicks the "NEXT" button 80 to move to the next question
  • the response is stored locally, for example in the memory or other storage device associated with the device on which the visitor's browser is running, and the question frame 70 presents the next question and awaits an answer This question and answer procedure continues until the last question is answered Step 19a shows mid-point progress updates being posted to the survev server 14
  • the visitor clicks the "NEXT" button 80 without having completed a response they are presented with an appropriate error message For example, they might be told they have not entered a response at all (see FIGURE 14), or that an element of a multi-part response (see FIGURE 15)
  • the visitor is also prompted to enter “Importance” before entering “Performance” if they attempt to move the "Performance” slider without having moved the “Importance” slider first (see FIGURE 16
  • FIGURE 17 When the user clicks the "NEXT" button 80a, the survey servlet 16 posts all of the question responses to the survey server 14 where they are used to update the survey database 6 The survey servlet 14 then exits, leaving the visitor
  • the counter servlet 26 also makes/updates a record of the following attributes (referred to as sample incidence information) number of hits to the trigger applet tag, number of qualified hits (by browser type), 1 e those which pass the initial validation step, number of invitations produced, number of invitations accepted, and number of surveys offered that were completed.
  • sample incidence information referred to as sample incidence information
  • number of hits to the trigger applet tag number of qualified hits (by browser type)
  • 1 e those which pass the initial validation step
  • number of invitations produced number of invitations accepted, and number of surveys offered that were completed
  • the report site 8 allows designated persons, typically the research commissioner to review the results of an on-going, or recently completed survey via a web browser 1 10 (FIGURE 1 ) In this way, remote access to the real time results is possible from anywhere with Internet access
  • the report site 8 is implemented entirely in the Practical Extraction and Report Language (PERL) and relies on CGI It retrieves data from the relational database 6 and formats it to present the visitor with meaningful reports
  • the report site effectively encompasses a report tool 10, because it is able to manipulate the survey results stored in the database 6 to present the desired format and focus of the report
  • FIGURES 18 to 28 When a designated user accesses the report site 8, they are first presented with a login page 1 12 (FIGURE 18) Each designated person is given a user name and password with which to login, and on entering these in the respective boxes 1 14, 1 16
  • the second report type available from the Executive Summary contents page is "Site Performance" Clicking on the respective button 126 takes the designated person to the report seen in FIGURES 24A and 24B This report displays the visitors' "importance” and “performance” ratings for seven key basic elements relevant to most, if not all Web sites
  • the final report accessible from the Executive Summary contents page is the "Demographics" report This is a full report of the demographic indicators of the visitors who responded to the survey, including indicators such as job occupation, frequency of Internet usage, age and income Each indicator is rated as a percentage of total responses This report is seen in its entirety in FIGURES 25A through D
  • the second option is "Counts and frequencies"
  • Clicking on this button 134 takes the designated person to a report giving a question-by-question display of answers to all questions in the survey, with the exception of free text answers
  • the answers are displayed in an appropriate graphical form, for example where the question gave the option of a limited choice of answers, the percentage of each answer chosen as a percentage of total answers to the question is displayed as a bar graph
  • FIGURE 26 shows an extract from the "Counts and frequencies" report
  • the next report type available on the main contents page is the "Verbatim" report 136
  • the visitor in response to some questions in the survey, the visitor is given the opportunity to give a free text, open-ended response
  • the answers to these questions are stored verbatim in the survey database, and are accessible through the "Verbatim" report, question by question
  • the final report option on the main contents page is the "Crosstabs" report 138
  • This report provides the designated person with a real-time interactive tool, which gives them the opportunity to select specific questions to generate a crosstab
  • the designated person first selects the "banner” from a scrollable list 140 (see FIGURE 27), which may be any one of the non-free text survey questions, and then selects the "stub", which can be any other one of the non-free text questions from the survey
  • the "stub” answers are then displayed crosstabbed with the "banner” answers, in both table 142 and bar graph 144 form with approp ⁇ ate statistical notations where applicable (e g mean, median standard deviation and/or standard error), as seen in FIGURES 28A and 28B respectively
  • the report site provides a very versatile, interactive tool for reviewing the survey results Furthermore, since the results are updated with the data from each newly completed survey, the report site provides real time feed back on the performance of the Web site
  • the report site may also provide access to a survey editor facility
  • the fundamental component of this facility is essentially a data entry tool via which the question data and/or survev configuration data in the survey database can be entered and/or amended Consequently, this facility enables, through means known to those of ordinaiy skill, the designated person(s) to edit an existing survey, or even create their own survey, for use with the data collection tool disclosed herein
  • the survey editor is a front-end component that enables the entry and modification of question content and survey configuration Using this front-end component, a user may enter the following information for each question contained in a survey question text, question type (for example, single response, multiple response or open ended), response text, response dependencies, and order of appearance within the survey.
  • question type for example, single response, multiple response or open ended
  • response text for example, response, multiple response or open ended
  • response dependencies for example, single response text
  • order of appearance within the survey The user can also enter or modify survey specific configuration data such as question and response color banner and start-up images, invitation frequency, and IP address blocking
  • Adopting a survey editor such as this will allow someone without skills in the relevant computer programming fields to create and modify questionnaires/surveys In this way, the commissioner of the research or other data collection procedure will be able to create their own surveys using this tool
  • the system described above can be implemented on conventional Internet compatible hardware
  • the server side components of the data collection tool reside on a survey server or servers connected to the Internet
  • the survey database, the report site and the report tool all reside on a report server or servers connected to the Internet
  • the survey server and report server are illustrated as being separate servers and may be resident on separate physical platforms, it is also feasible for a single physical platform and /or server to perform both functions
  • the currently preferred server configuration is a Sun SPARC hardware platform, running a Solaris 2 x operating system, but other hardware/OS platforms can be used, so long as they meet with the primary requirements of offering support for a robust RDBMS (such as those mentioned below), JDK 1 x, a web server that supports the Java servlet API, and PERL or equivalent platforms JavaWebServer 1 x is the currently preferred web server, although any web server that supports the Java servlet API could be used
  • the survey database is preferably an RDBMS such as Sybase 11, Oracle, Informix or Progress
  • the client to and on which the data collection tool is deployed will typically be a Java supporting Web browser such as Microsoft Internet Explorer or Netscape Navigator running in a Windows environment, although the invention is also applicable to other browsers and/or operating systems
  • the performance requirements for the information server are dependent on the characteristics of the information site itself, rather than the characteristics of the system of the invention which, in the implementation described here, does not use any of the resources of the information server, only those of the client and the survey server
  • the data collection tool may be used as the front end to data reporting tools other than the inventive one described herein
  • the inventive data reporting tool and report site describe herein may be used in conjunction with a database that is populated with data elicited using a data collection tool other than the inventive tool described herein
  • Exemplary Core Survey Questions The following sets forth a set of exemplary survey questions suitable for gathering customer feedback regarding the quality of services and information that are being provided by an organization doing business on the World Wide Web Some of the questions are only to be asked if a particular response was made to a preceding question This is indicated with a Boolean statement at the beginning of the question See for example questions 1 1 and 12, which are only to be asked if the person taking the survey has selected the first or the third response option to question 10 ql Prior to beginning this survey, about how long did you spend exploring this site 7
  • the site ' s organization makes information easy to find.
  • the site contains valuable supplementary or educational resources
  • the site looks attractive
  • the site is easv to navigate

Abstract

A computer network based system and method for collecting and reporting data, preferably market research data in real-time. Real-time feedback enables timely response to emerging issues, enhances training/feedback for support staff and gives the ability to promptly see problem areas through the users' eyes. The present invention collects and reports market research data. A software implemented data collection tool (16) deploys to a client (18) over a computer network (2) to elicit data from the user. The collection tool (16) returns the elicited data to a research database (6), which stores the elicited data. The research database (6) is accessible to one or more designated persons, typically a commissioner of the research work and/or someone with their authority.

Description

COMPUTER NETWORK BASED SYSTEM AND METHOD FOR COLLECTING
AND REPORTING DATA
Field of the Invention The present invention relates to computerized market research tools, and in particular to a computer network based system and method for collecting and reporting data, preferably market research data, such as survey data.
Background of the Invention As computer network technology, in particular Internet technology, has rapidly developed in recent years and businesses and other organizations have found or been presented with more effective ways to take advantage of this Internet technology, the World Wide Web has taken on ever increasing importance in the world of commerce. More and more organizations are using the Internet as a primary channel of communication, supplementing or even replacing traditional lines of communication such as face-to-face meetings, mail, fax and phone. A recent study by Forrester Research, Inc estimates that Internet commerce will grow forty fold over a period of five years, from US$8 billion in 1997 to US$327 billion in goods and services by the year 2002
Even now the Internet plays a very significant role as a distribution channel, a point of contact with customers, a marketing and advertising medium and a channel for services and support, and the significance of this role will only increase in the future Concordant with this growing significance of commerce over the Internet, in order that organizations can maintain and improve on the standards of information they share via the Internet, there is a rapidly growing need for feedback on the organizations' Internet presence, and thus a need for tools which can quickly and efficiently collect and report relevant market research data characterizing and evaluating the organizations' Internet efforts
Traditionally, market research has been conducted by way of a survey completed during an interview between a market researcher and a consumer, either face-to-face or over the telephone As an alternative, the survey may be mailed on paper or on computer diskette to the consumer for them to complete and return The survey contains a series of questions designed to elicit relevant information from the consumer, which can be collated and analyzed with answers to the survey from a number of other consumers to provide a characterization and/or an evaluation of particular facets of the organization, services or products of the entity for whom the survey is being conducted
Many companies or businesses initially developed on line survey tools in-house More recently, computerized survey tools have been provided, whereby survey or other market research information can be elicited over the Internet One example of such a tool is 'SurveySez', which allows an organization to create their own HTML-based survey as a page or pages hosted on the SurveySez Internet Web Site The organization then includes a link to the survey on their own
A second example is 'SurveySite', which uses a pop-up invitation implemented in Java which acts as a link to an HTML survey hosted at a separate site A report is generated and delivered to the organization commissioning the research at the end of the survey's life, e g after a pre-determined number of surveys have been responded to
Another example is 'CLTResearch', offering a survey in flat HTML Similarly to
'SurveySite' a report is generated at the conclusion of the complete survey period Yet another example of an HTML survey with a report generated on completion is 'KB&P' In the latter example, the respondents to the survey mav either be visitors to a particular Web Site, as in the above examples, or previously registered participants
A final example, which employs a very different methodology to the previous examples, is 'RelevantKnowledge'/Nielsen Group A group of people (called a web panel or research panel) agree in advance to have their Internet usage monitored and then periodically a log of which Web Sites these people have visited is uploaded to a central collection point, and a report is generated
Summary of the Invention The present invention provides a computer network based system and method for collecting and reporting data, preferably market research data in real-time Whatever the application to which the invention is applied, particularly where it is market research oriented, the real-time feedback it provides can enable timely response to emerging issues, can enhance training/feedback for support staff and can give the ability to promptly see problem areas through the users' eyes From one aspect, the invention provides a computer network based, preferably
Internet based, system and method for collecting and reporting data, preferably market research data A software implemented data collection tool is deployed to a client on the computer network (for example a Web browser) to elicit data from a user accessing the computer network through that client The collection tool returns the elicited data to a research database, which stores the elicited data The research database is made accessible to one or more designated persons, typically a commissioner of the research work and/or someone with their authority The data collection tool can be deployed to successive users of the computer network, accessing the network via the same or, more typically, a different client Each time the collection tool is deployed and data returned, the research database is updated, and thus updated, real time results are made accessible to the designated person(s) Preferably, the deployment of the data collection tool is initiated by the user visiting a particular pre-defined trigger page of information on the computer network, or any one of a number of such pre-defined trigger pages Typically, each trigger page will be associated with a particular information site of the organization commissioning the research For example, where the system is implemented on the Internet, each trigger page may be a page of their Web site If there are two or more trigger pages at an information site, the system is preferably controlled so as to suppress deployment of the data collection tool more than once during any single visit to the information site by a user For instance the data collection tool can function to detect whether or not it has previously been run duπng the current browser session in order to prevent multiple deployments In some embodiments of the invention, further control is exercised over the deployment of the data collection tool For instance, a first stage in the deployment of the collection tool may be the issue of an invitation to the user, for example giving them the option to decline to take the survey or other data collection means if they -wish If desired any such decision to decline to take a survey may be recorded in the research database A further possibility is to control the frequency at which the data collection tool is deployed That is to say, the tool need not be deployed each time a user visits a trigger page, but rather can be presented only to every nth user to visit the page, or to a random one of every n users (where n is any real integer greater than 1 ) and/or only after lapse of a predetermined time interval following a preceding deployment of the tool Yet a further possibility, where the users of the computer network are identified with a user name or other identifier such as machine or source IP address, is to place limitations on the identity of the users to whom the data collection tool is presented For instance, it may be desirable to exclude certain named/identified users from the market research or other data collection process, for example employees of the research commissioner In addition or alternatively, particularly on computer networks such as the Internet where individual users do not necessarily require positive identification as a condition of access, specific network addresses (e g IP addresses on the Internet) can be blocked to provide a similar control over the persons to whom the data collection tool is presented
Once the data collection tool has been deployed, the system preferably has the ability to delay the actual commencement of data ehcitation For example, the data collection tool may have the ability to linger within the client for a period of time after its deployment, during which time the client device can be operated to perform other functions For example it may be desirable to deploy the data collection tool when a user first arrives at an information site, but to delay ehcitation of the market research data, or other data, until after the user has had an opportunity to explore the site more fully
In a typical implementation of the invention, the data collection tool will provide a survey or other data form, presenting the user with a series of questions to be answered For some questions the user may be presented with a series of possible responses from which they must choose one, or in some cases one or more Other questions may require a response by way of a value on a continuous scale, and another option is to allow the user to enter free text answers Where there is some control placed on the response to a question such that the response is predictable, the system may be designed to use the response to that question to decide whether a subsequent question in the survey be used Once all of the answers to the survey have been provided, the data collection tool delivers the answers to the research database for access by the reporting tool at the report site In addition to the answers themselves, the data collection tool may also collect and return other types of information, for example information indicating the state of completion of a survey, or other similar sample incidence information In some embodiments of the system, the data collection tool conducts one or more analyses of the data to produce one or more report results which are stored in the database The analysis or analyses may be relatively simple, for example calculating occurrences of a particular class of answer as a percentage of total answers to a question or relatively complex, for example deriving a report result based on answers to a number of different questions The data collection tool may also directly store the raw information elicited as report results in the database, for example answers to one of more of the questions presented in a survey may be stored without further analysis Furthermore, in preferred embodiments, the at least some report results are also available in an interactive query format, allowing someone accessing the results via the report site to specify an analysis based on a combination of any two or possibly more than two, of the non-free text questions of a survey
The research database is preferably accessible via a report site on the computer network which, if desired, can provide secure access to the database For instance, the report site may be set up so that a pre-arranged password is required before the report site enables the research database to be accessed, the password only being made available to pre- authoπzed designated persons In this way, the results of the market research, or other data collection process, can be made available only to the desired parties, for example the commissioner of the research or persons with their express consent
The report site preferably also provides a user friendly front end to the database, and comprises or works in conjunction with a software implemented report tool to extract relevant report results from the database and present them to the designated person(s) as an appropriately formatted report Typically, the results will be presented graphically as bar charts, pie charts, tables or the like, but any one of a large number of other graphical or textual formats are possible It is particularly preferred that the report tool has the capability to manipulate the report results and/or carry out one or more analyses of them, in order to present the designated person(s) with a variety of differently focused reports For example, the system may allow the designated person to select one of a number of focused reports, or even to define the focus of the report themselves
Advantageously, the data collection tool can be deployed from a server (referred to hereinafter a "survey server") associated with the computer network, other than a server (referred to hereinafter as an "information server") on which the information site containing the trigger page or pages is located This arrangement, unlike prior survey systems, can avoid placing additional load on the information server The research database is also preferably resident on a server (referred to hereinafter as a "report server") other than the information server, as is the report site and report tool The research database, the report site and the report tool are preferably resident on a single report server which conveniently may be the same server as the survey server from which the data collection tool is deployed
In a development of the present invention, the data collecting and reporting system elicits information from the users of a plurality of distinct information sites on the computer network, which may be resident on the same or different information servers For each information site the same data collection tool and/or the same report tool may be employed Alternatively, custom versions of one or both of these tools may be provided for use with one or more of the information sites served If different, customized collection and report tools are used for respective ones of the information sites served bv the system, thev are preferably each based on a respective one of a master data collection tool and a master report tool, which provide common functions generally required by most, if not all market research applications or, if applicable, other data collection applications to which the system applies In this way, the effort required to provide a custom data collection and/or report tool for any particular application can be minimized
Where multiple information sites are served, all of the data elicited may be stored in a single research database, but access to the data is preferably controlled so that the report results associated with the distinct information sites are only accessible to the respective designated person or persons For example, each designated person may be given a unique access name and corresponding password which they use to enter the report site, the report site and report tool utilizing the unique access name, if verified with the corresponding password, to determine which report results should be made accessible to a particular designated person Brief Description of the Drawings
The foregoing aspects and many of the attendant advantages of this invention will become more readily appreciated as the same becomes better understood by reference to the following detailed description, when taken in conjunction with the accompanying drawings, wherein FIGURE 1 schematically illustrates a market research data collection and reporting system that operates in accordance with an embodiment of the present invention,
FIGURE 2 is a block diagram showing the structure of the client side component of the data collection tool of the system of FIGURE 1 ,
FIGURE 3 schematically illustrates the procedure by which the system of FIGURE 1 collects data,
FIGURES 4 to 17 show screen shots from an exemplary survey conducted over the Internet using the inventive system, and
FIGURES 18 to 28 show pages from an exemplary report site according to the present invention Detailed Description of the Preferred Embodiment
The present invention is a computer network based system and method for collecting and reporting data, preferably market research data in real time Currently, the primary application of the invention is seen to be in the field of gathering customer feedback regarding the quality of services, information and products that are being provided by an organization doing business on the World Wide Web, and the following description of an exemplary illustrates such an application However, the invention is also applicable to gathering other types of data, whether market research data or other informational data in any of a number of computer network environments In this sense it can be considered as a general purpose market research tool
Referring to FIGURE 1 , a data collection and reporting system is implemented on the World Wide Web, data exchange between various components of the system being effected over the Internet 2 Visitors to a specific information site 4 on the Web, typically a Web site of the research commissioner, are presented with a survey including a number of questions The answers to these questions given by the visitors are stored in a database 6 and made immediately accessible to a designated person or persons, typically the research commissioner, by way of one or more user friendly survey reports available on-line over the Internet 2 at a report Web site 8 The survey remains active for a finite period defined bv the number of visitors to take the survey an overall time duration or a combination of the two At the end of the complete survey period the results remain accessible to the designated person(s) for a further pre-defined post-survey time period At any time duπng pendency of the survey or duπng the post-survey period, the survey results can be down-loaded from the survey database 6, in order that a permanent record may be kept by the research commissioner Additionally, any of the graphical reports at the report site 8 can be captured using common methods for HTML capture and reposted to any other Web site (for example in order that the research commissioners intranet can be used to share the information within an organization) or the graphics can be incorporated into any compatible tool for presentation The main elements of a currently preferred embodiment of the system are (a) a data collection tool, having both server side and client side components, for presenting the user with the survey and collecting their answers as survey data, (b) a survey database 6 for storing the survey data, either in raw form or following one or more analyses or manipulations, as survey results, and which also contains survey configuration data and survey question text, (c) a report site 8 through which the designated person can access the survey results, (d) a report tool 10 which is implemented within the report site 8 to present the designated person with the survey results in the form of one or more focused survey reports, and (e) a trigger, resident on a trigger page 12 of an information site 4, typically a Web page of the research commissioner, to initiate the survey when the visitor requests that trigger page 12
Advantageously, a single surve) server 14 operating the inventive system may be used to simultaneously conduct more than one survey, each survey being associated with the same or a different Web site or page on the World Wide Web For each Web site or page the same fundamental data collection tool is employed, although an expanded survey (including additional questions) or otherwise customized survey (e g including revised questions and/or graphic images) may be used Where multiple surveys are being conducted, the survey results from each survey are stored in the database 6 in a manner identifying them with their respective survey, in order that the report site 8 and report tool 10 can distinguish them It is also likely that the designated person(s) will be different for the different surveys Consequentlv, each designated person is assigned a unique user name, verified by a password, for accessing the report site 8 and hence the survey results stored in the survey database 6 In this way it is possible to restrict access to the survey results associated with particular surveys only to respective authorized designated persons
In the preferred implementation of the data collection tool, there is a capability to suppress presentation of the survey to the visitor in certain circumstances More particularly when mitialh deployed the survey applet 16 conducts a validation operation to determine whether to proceed with the survey, in which it is determined, for example (a) whether the implementation of the survey is supported by the visitor's Web browser 18, (b) whether the survey has been previously executed in the current browser session, and (c) whether the IP address from which the visitor is accessing the network is on a list of addresses excluded from participation in the survey If these and/or other validation tests are passed, that is to say, for example, the visitor's browser 18 will support the survey, the survey has not been previously executed and the visitor is not excluded, the applet 16 proceeds to the next step in the procedure If any one of the tests is failed, the survey applet 16 exits without ever presenting itself to the user Also in this preferred implementation, the frequency at which the survey is presented to visitors to the trigger page 12 can be controlled That is to say the survey need not be presented to each visitor who passes the above validation operation This frequency control is implemented by setting an invitation frequency (f) within the survey applet 16 The value/ is an integer which determines the probability that a user will be presented with the survey I f is set to 1, every visitor who passes the validation will be presented with the survey, whereas integers greater than 1 will reduce the probability that a given visitor will be presented with a survey For instance, the survey applet 16 can be configured to randomly present the survey to one out of every n visitors, where / = n In the current implementation, a visitor is presented with an invitation if the results of floor(R xf) equals 0 where R is a pseudo random number between 0 and 1 The floor() function truncates the product of R xfi that is to say subtracts the fractional part of the product, to give an integer value In the preferred implementation, the value/ can be changed during the course of a survey, allowing control of the period during which data collection is completed, providing the unique ability to ensure collection of a representative sample of site traffic
The survey applet 16 also has the capability to provide a further front end filter to the survey If desired rather than launching straight in to the full survey itself, the visitor can be presented with an invitation to take the survey This invitation can give the visitor the opportunity to decline to take the survey (such a decision to decline itself being recorded as the survey data), can be used to "screen" for a specific type of visitor (e g visitors with a specific connection speed or other screening information), and can also be conveniently used to offer the visitor an incentive for completing the survey
Whatever front end filters are used, if and when the full survey is launched, the survey applet 16 presents the visitor with a "START SURVEY" icon, along with instructions on the completion of the survey and, if desired, a suggestion to them that they complete their business at the Web site before taking the survey, in order that their responses might be based on a more complete review of the Web site The "START SURVEY" icon exists in its own window separate from the main browser window, so it may be minimized or lowered while the visitor continues to browse the information Web site 4 When the user is ready to commence answering the survey questions, they simply activate the "START SURVEY" icon as they would any other icon (e g by clicking on it with their mouse), and they are presented with the first question
Considering the data collection tool in more detail As mentioned above, the data collection tool comprises both server side and client side components, which communicate via the Internet 2 using the standard HTTP protocol The server side components are resident on, and are executed by the survey server 14 They are currently implemented as a series of servlets, which run under the JavaWebServer 1 1 environment Thev could however, be implemented to run under any HTTP server that supports Java servlets or other equivalent applets Currently there are four server side components
First, there is a survey servlet 20 which, at the request of the client side component 16, extracts the appropriate survey configuration data from the survey database 6 and posts it to client side component 16 Where the system is set up to handle multiple surveys, in order that the survey servlet 20 can extract and post the correct survey configuration data for the survey that is to be activated, the survey servlet 20 is first passed a pre-assigned unique survey job code by the client side component 16 The client side component 16 itself obtains the unique survey job code from the trigger on the trigger page 12, the code being one component of the trigger Using the unique survey job code, the survey servlet 20 retrieves the appropπate configuration data from the survey database 6, puts it in name-value pair format and posts it to the client side component 16 In currently preferred embodiments, the configuration data includes the following elements
"Survey class name" - this is a class file that determines the question set for each survey Each different survey may be associated with its own unique survey class, in which case the text of the questions, which make up the survey can be coded into the class However, in a preferred implementation a generic -13-
survev class is used for a number of different surveys, the generic class configuring itself for any particular survey by retrieving the respective question set from the survey database With this latter implementation, the survey class name will be the same for all surveys sharing the generic survey class "Survey Job Code" - this is the key to retrieving the correct survey configuration for a particular survey It is passed to the client side component 16 of the data collection tool as a parameter from the trigger
"Question Image (banner)" - this is an image that is displayed at the top of each question frame and may, for example, contain the name or other identifying symbol of the research commissioner
"Start Image (logo)" - this is an image that appears with the "START SURVEY" icon
"Question Color" - in the exemplary implementation, as will be explained further below, the visitor is presented the survey questions in a question frame which contains two panels (1) a question panel in which the question text is displayed, and (π) a response panel in which the visitor makes their response The "Question Color" parameter determines the background color of the question panel
"Response Color" - this parameter determines the background color of the response panel in the question frame
"Frequency" - this is the value/ which determines the probability that a user will be presented with an invitation
"Blocked IP" - this is a string containing a comma separated list of IP network addresses for which the applet is to be blocked, 1 e addresses at which the survey will not be presented to visitors "Invitation Text" - this final element of the configuration data contains text that appears in an invitation window, inviting the visitor to take the survey
The second server side component of the data collection tool is a data grabber servlet 22 This servlet accepts the survey data from the client side component 16 and updates the survey database 6, which is preferably a relational database, with this data More specifically, the client side component 16 sends the survey data to the survey server 14 over the Internet 2 as survey data post requests The data is posted in name-value pair format The data grabber servlet 22 accepts the HTTP request, extracts the survey data and posts it to the survey database 6 using, in the currently preferred embodiment, the JDBC API (a standard feature of JDK1 1)
Third, there is a question servlet 24 which accepts question text requests from the client side component 16, extracts the appropπate question text from the survey database 6 and returns the question text to the client side component 16 The question data is returned to the client side component 16 in string format via an HTTP response The client side component 16 parses the string formatted data and uses it to initialize Java question objects The question text itself is designed with a view to the particular survey being undertaken The currently envisaged application for which the exemplary system according to the invention is designed is the gathering of customer feedback regarding the quality of services, information, and products that are being provided by an organization doing business on the World Wide Web Suitable standard, core questions for this application are given in Appendix I, although in any particular case it may be desirable to add further, custom questions, or to modify some of the standard questions to provide the desired focus for the market research being undertaken As an alternative to returning the question data in string format, it may be returned using object serialization if the programming environment allows, for example if the survey applet is running under JDK1 1 Object serialization allows an object instance to be encoded mto a byte stream, which can then be written to a file or network socket The byte stream can be subsequently read from the file, or the other end of the network socket, and used to instantiate another copy of the original object With this object serialization approach it becomes possible to simplify both the question servlet 24 and the client side component 16 because they are no longer required to convert question data to and from string format
The final server side component of the data collection tool is a counter servlet 26 which accepts data packages from the client side component 16 containing survey progress data, that is data indicating the progress of the survey, rather than data representing the survey answers themselves It is envisaged that this data could be used to detect when a visitor reaches particular milestones in the completion of the survey, and would enable analysis of the actions of visitors who start the survey but drop out prior to completion
The survey server 14 also holds a copy of the client side component 16 of the data collection tool, which is deployed to, and executed within the visitor's Web browser 18 In this exemplary embodiment, the client side component 16 is implemented as an applet written in the Java language and will be referred to as the survey applet 16 It provides the interface with the visitor through which the survey data is collected More particularly, the survey applet 16 once downloaded and executed within the visitor's browser 18, presents the visitor with a series of questions one at a time, in a window separate from the browser 18 The visitor responds to each question in turn and once they have committed a response the applet 16 does not allow the response to be changed The applet 16 initially records each response on the client side, and then when all of the survey questions have been answered, the responses (l e survey data) are posted to the survey server 14
This approach has a number of advantages over prior survey techniques Firstly, the ability to prevent answers being changed once made is generally desirable, and using a Java applet, or an equivalent applet enables this to be achieved whilst keeping the number of transactions with the survey server 14 to a minimum Thus the survey applet 16 collects all user responses, question by question, and posts them to the survey server 14 in a single transaction, whereas the known HTML implemented survey tools would require either a transaction after each question response or putting all questions on a single form In the first case the user may experience frustrating delays between document downloads which would tend to increase the survey drop out rate thereby jeopardizing the confidence intervals and/or statistical validity of the survey results The latter case would require HTML and server side CGI code, which would be very difficult to maintain In either of these HTML implementations, the user can easily go back to change previous responses The ability of the present invention to control these aspects of behavior on the part of the survey tool and the actual survey respondent helps to ensure the validity of the methodology and the integrity of the data collected This is an important methodological improvement offered by the present invention over other known survey tools
A further advantage is that since the server applet 16 runs in a separate window, it can be minimized allowing the user to explore the Web site 4 they are visiting before starting the survey This may increase the likelihood of a more informed evaluation of the site by the visitor The use of the Java applet also provides a more versatile and robust user interface, for example it is not affected by connectivity problems once up and running, and offers a superior development environment that results in both source code that is easier to debug, maintain, and modify incrementally and research results that are methodologically more reliable than in other known on-line survey tools The structure of the survey servlet 16 will now be discussed in greater detail, with reference to FIGURE 2, which is an object diagram for a preferred implementation of the survey servlet
The APPLET LOADER 30 is the component initially downloaded from the survey server 14 by the visitor's browser 18 when the visitor requests the trigger page 12, containing the trigger, from the information server 4 Its functions are to download from the survey server the necessary Java classes, to present the invitation to take the survey to the visitor, and to instantiate the CLIENT SPECIFIC SURVEY 32 and call the doSurveyQ method, when the INVITER 34 indicates a positive answer by a visitor to the invitation
CLIENT SPECIFIC SURVEY 32, for which the WEBMETRICS class 36 is a superclass overrides the doSurveyQ method, and instantiates the QUESTION FRAME 38, the concrete QUESTION objects 40 and CGI POST 42 In addition it passes, in turn, a reference to each QUESTION object 40 to the QUESTION FRAME howf) method, to cause each question of the survey to be presented in turn If desired, the survey may be configured so that the presentation of certain QUESTION objects 40 is dependent on an answer to a previous question For example, one might only wish to ask the visitor what industry sector they work in if they have already indicated that they were visiting the information site of the research commissioner for business purposes Skip logic such as this, and other similar question controls are not available in conventional HTML survey tools
As each question is answered, CLIENT SPECIFIC SURVEY 32 calls the getResponseQ method of QUESTION 40 and the addPostVahιe() method of CGI POST 42 to retrieve the visitor's response to the question and update CGI POST 42 with the response Finally, once all of the survey questions have been answered, CLIENT SPECIFIC SURVEY 32 calls the CGI POST postQ method to return the answers to the survey server 14
QUESTION FRAME 38 contains QUESTION 40 and displays QUESTION 40 when its showQ method is called by CLIENT SPECIFIC SURVEY 32 QUESTION FRAME 38 also handles action events from NEXT BUTTON 44, which accepts the visitor's indication that they have answered the current question A NEXT BUTTON event handler within QUESTION FRAME 38 calls the sioreResponseQ method of QUESTION 40 and updates PROGRESS METER 46 as each question is answered If the visitor attempts to move on to the next question without having competed the answer to the current question, the NEXT BUTTON handler causes an appropriate error message to be displayed to the visitor, or takes other corrective action if the survey methodology so indicates QUESTION 40 contains a QUESTION PANEL 48 in which the question text is presented to the visitor, and a RESPONSE PANEL 50 which presents question response choices to the visitor and interacts with the visitor to capture their responses When its storeRespomeO method is called by QUESTION FRAME 38, QUESTION 40 interprets the state of the RESPONSE PANEL 50 to capture the visitor's response and stores the captured response as a string value The question response choices themselves can take any one of a number of different forms Examples include a list of mutually exclusive response options, presented for instance as option buttons or a single selection list box, a list of non-exclusive response options of which the visitor can select one or more, presented, for instance, as check boxes or a multiple selection list box, one or more sliders or the like which allow a response to be given on a continuous scale, used, for instance, to respond to a question requesting an indication of the importance and/or performance of a particular feature, and one or more text fields allowing free hand entry of responses by the visitor, used, for instance, when a question asks for the visitor's comments, or to elicit contact details from the visitor Specific examples of these various response choice types are illustrated in FIGURES 8 to 13, which are discussed below
Finally, CGI POST 42 functions to send the captured responses in name-value pair format to the data grabber servlet 22 of the survey server 14 from ts post () method
As already discussed above, the data collection process is initiated when the visitor requests a trigger page 12 on a Web site 4 associated with an enabled survey, as they would request a page on any other Web site The trigger page 12 can be any ordinary HTML page on the Web site 4, additionally containing the survey trigger In the preferred implementation, this survey trigger is an applet tag, which the web master can simply insert into the HTML source for one or more of their web pages An exemplary applet tag is shown below <applet wιdth=5 heιght=5 code="ιworks jvl02 WebMetπcs AppletLoader class" codebase- 'http //demo informationworks net/wmdemo/classes"> <ρaram name="name" value="demo"> </applet> Advantageously, this applet tag, or one like it, can be small (for example only 5 by 5 pixels) and colored to match the Web page 12 it resides on, thus avoiding any disruption of the appearance of the page
To give a better understanding of the data exchange between the various components of the system, and the visitor's interaction with the system, a step wise example of the process of data collection using the inventive system will now be described with reference to FIGURE 3, which illustrates the complete process, and FIGURES 4 to 17 which show screen shots taken during the process The numbered steps described below correspond to the numbered arrows in FIGURE 3
1) The visitor 52, accessing an information web site ("customer Web site") 4 via a Web browser 18, requests the trigger page 12 from the Web site 4 (typically the Web site of the research commissioner) 2) The Web site 4 returns the requested trigger page 12 which contains an applet tag similar to the example given above 3) The codebase parameter for the applet tag on the trigger page 12 directs the visitor's browser 18 to request the survey applet ("WebMetπcs applet) 16 from the survey server ("WebMetπcs server") 14 4) The survey applet 16 is returned to the browser 18 by the survey server 14 along with related Java classes
5) The visitor's browser 18 executes the survey applet 16, which runs within the browser 18
6) The survey applet 18 requests survey configuration information from the survey server 14 The configuration information is requested on the basis of the unique survey codebase included in the applet tag -20-
7) The applicable survey configuration is returned to the survey applet 16 by the survey server 14
8) The applet 16 executes a validation sequence to determine whether or not to present the visitor 52 with an invitation to take the survey It checks whether or not it is supported under the visitor's browser 18, whether it has been previously executed in the current browser session, and whether or not the visitor's IP address has been blocked If all those tests pass, the survey applet 16 finally determines whether or not to present a survey invitation to the visitor based on the configured invitation frequency, / which is downloaded as part of the survey configuration in step 7 The value of/ is an integer that determines the probability that the visitor 52 will be presented with an invitation — the greater the value of/ the lower the probability If/ is equal to 1 , then every visitor whose browser and IP address pass the initial validation will be presented with an invitation If the validation sequence fails, the applet quietly exits 9) If the validation succeeds, the visitor 52 is presented with an invitation 60 to take the survey (see FIGURE 4) The survey applet 16 posts the fact that an invitation 60 was presented to the survey server 14 (step 9a) The invitation 60 may or may not offer an incentive for the visitor to take the survey It allows the visitor to respond yes or no by clicking the appropriate button 62, 64 10) The visitor responds yes or no If the response is no, the survey applet quietly exits
1 1) If the visitor's response to the invitation is yes, the survey applet requests the Java classes required to complete the survey from the survey server 14
12) The requested classes are downloaded to the survey applet 16 from the survey server 14 The visitor 52 is presented with a status window for the normally brief duration of the download Delaying downloads of some of the classes up until this point reduces the time taken to initially download the survey applet 16 -21-
13) The survey questions are requested from the survey serve 14r by the survey applet 16
14) The survey server 14 returns the survey questions to the survey applet 16 The visitor 52 is presented with a status window for the normally brief duration of this download Delaying question download up until this point reduces the initial download time of the survey applet 16
15) The visitor 52 is presented with survey instructions 66 (see FIGURE 5) followed by a start survey icon 68 (see FIGURE 6) The visitor is instructed and encouraged to continue with their visit to the information web site 4 before continuing with the survey
16) The visitor 52 explores the information web site 4 The start survey icon 58 meanwhile exists as a separate window on the visitor's desktop As such, it may be minimized or lowered while the visitor interacts with the browser (see FIGURE 7), and effectively "floats" with them as they browse 17) The visitor 52 clicks the start survey icon 58 when ready to begin the survey
18) The visitor 52 is presented with the question frame, also in a window separate from the browser 18 The question frame 70 presents the user with one question at a time, examples of which are seen in FIGURES 8 to 13 Each question frame displays a banner 72, in which the name of the research commissioner can be displayed for example, a question panel 74, in which the question text is displayed, and a response panel 76 in which the response options are given where applicable and in which the visitor enters their response In addition, the question frame 70 displays a progress bar 78 indicating the visitor's progress through the survey, and a "NEXT" button 80 which the visitor clicks, having made a response, to move on to the next question Various formats of question response may be used, as illustrated by FIGURES 8 to 13 FIGURE 8 shows a list of mutually exclusive possible responses alongside respective option buttons 82 that the visitor can click on to make their selection of response In FIGURE 9 the response panel displays a free format text box 84 in which the visitor is invited to type comments FIGURE 10 illustrates a response panel with two sliders 86, 88 that can be dragged by the visitor to their choice of response on the sliding scales shown In FIGURE 1 1, the visitor enters their response by clicking on the desired response in a scrollable, single choice list box 90 FIGURE 12 shows an example of a checkbox list 92, in which the user has the option to select more than one response Additionally, the response panel seen in FIGURE 12 includes a text box 94 adjacent the "Other Internet technology" option, to enter what "other" technology they have FIGURE 13 shows a further example of a response panel including text boxes 96, here to elicit contact details from the visitor 52
19) The visitor enters a response to each question in turn and after responding clicks the "NEXT" button 80 to move to the next question The response is stored locally, for example in the memory or other storage device associated with the device on which the visitor's browser is running, and the question frame 70 presents the next question and awaits an answer This question and answer procedure continues until the last question is answered Step 19a shows mid-point progress updates being posted to the survev server 14 If at any time the visitor clicks the "NEXT" button 80 without having completed a response, they are presented with an appropriate error message For example, they might be told they have not entered a response at all (see FIGURE 14), or that an element of a multi-part response (see FIGURE 15) In a similar way, in the case of the multipart "Importance / Performance" response illustrated in FIGURE 10, the visitor is also prompted to enter "Importance" before entering "Performance" if they attempt to move the "Performance" slider without having moved the "Importance" slider first (see FIGURE 16) 20) After the last question is answered, the visitor is presented with a "Thank you" 98 in the question panel 74, and a "NEXT" button 80a to end the survey (see
FIGURE 17) When the user clicks the "NEXT" button 80a, the survey servlet 16 posts all of the question responses to the survey server 14 where they are used to update the survey database 6 The survey servlet 14 then exits, leaving the visitor
52 to continue browsing if they wish If, however, they return to the trigger page
12, they will not be presented with a further survey, because they will fail the validation step (step 9 above) After the last question is answered, the counter servlet 26 also makes/updates a record of the following attributes (referred to as sample incidence information) number of hits to the trigger applet tag, number of qualified hits (by browser type), 1 e those which pass the initial validation step, number of invitations produced, number of invitations accepted, and number of surveys offered that were completed This combination of features is comparable to the type of sample incidence tracking used to validate traditional market research efforts, and is one unique advantage offered by the invention over other known on-line research methods
Having explained the process by which survey information is collected, the report site
8 and its interaction with the survey database 6 to present the designated person with the desired reports will now be discussed As already mentioned above the report site 8 allows designated persons, typically the research commissioner to review the results of an on-going, or recently completed survey via a web browser 1 10 (FIGURE 1 ) In this way, remote access to the real time results is possible from anywhere with Internet access In the preferred implementation described, the report site 8 is implemented entirely in the Practical Extraction and Report Language (PERL) and relies on CGI It retrieves data from the relational database 6 and formats it to present the visitor with meaningful reports Using PERL, the report site effectively encompasses a report tool 10, because it is able to manipulate the survey results stored in the database 6 to present the desired format and focus of the report A representative session at the report site 8 is now described with reference to FIGURES 18 to 28 When a designated user accesses the report site 8, they are first presented with a login page 1 12 (FIGURE 18) Each designated person is given a user name and password with which to login, and on entering these in the respective boxes 1 14, 1 16 on the login page and clicking the "Start" button 118 the designated person is presented with a drop down menu 120 (FIGURE 19) of all active or recently completed surveys that are accessible to them (this may be a subset of the complete set of active surveys, some only being accessible by other designated users) The designated person selects the survey for which they wish to see the results and they are then presented with a contents page (FIGURE 20) that allows them to select one of several reports namely "Executive Summary" "Counts and Frequencies" "Verbatims" or "Crosstabs" or alternatively they can select to export data from the database for example for use with a local spreadsheet or database program
By clicking on the first menu item, "Executive Summary" 122 the designated person is taken to an Executive Summary contents page (FIGURE 21), where they are presented with the choice of three report types, "Overall", "Site Performance", and "Demographics", the designated person clicking on the respective button 124, 126, 128 to be taken to the report of their choice As seen in FIGURES 22 A and 22B, which respectively show top and bottom halves of the relevant report site page, the "Overall" report gives three general indicators of satisfaction of the visitors to the site overall satisfaction with the site likelihood of revisiting the site, and likelihood of recommending the site to a friend or colleague In each case, as with all of the other reports described below, the results represent an up to date view of visitors responses to the survey The "Overall" report also provides a link to specific question answers in the "Verbatims" report (descπbed below) by clicking on the "poor evaluation verbatims" button 130 Specifically, this links to the free text answers of the visitors who have rated their overall satisfaction as "very dissatisfied" or "somewhat dissatisfied", and who have taken the opportunity to comment more fully on their reasons for this poor rating A further feature of the "Overall" report is that it provides the opportunity to view the results by pre-defined cuts or demographic segment (gender age, income, Internet experience, frequency of visitation to site) To switch the report format to view the results by a particular demographic segment, the designated person simply clicks on the respective button 132 at the foot of the report (see FIGURE 22B) FIGURES 23A and 23B respectively show extracts from the top and bottom halves of an example of one such demographic segment report based on income
The second report type available from the Executive Summary contents page (FIGURE 21) is "Site Performance" Clicking on the respective button 126 takes the designated person to the report seen in FIGURES 24A and 24B This report displays the visitors' "importance" and "performance" ratings for seven key basic elements relevant to most, if not all Web sites
The final report accessible from the Executive Summary contents page is the "Demographics" report This is a full report of the demographic indicators of the visitors who responded to the survey, including indicators such as job occupation, frequency of Internet usage, age and income Each indicator is rated as a percentage of total responses This report is seen in its entirety in FIGURES 25A through D
Returning to the main contents page (FIGURE 20), the second option is "Counts and frequencies" Clicking on this button 134 takes the designated person to a report giving a question-by-question display of answers to all questions in the survey, with the exception of free text answers The answers are displayed in an appropriate graphical form, for example where the question gave the option of a limited choice of answers, the percentage of each answer chosen as a percentage of total answers to the question is displayed as a bar graph FIGURE 26 shows an extract from the "Counts and frequencies" report
The next report type available on the main contents page is the "Verbatim" report 136 As discussed above, in response to some questions in the survey, the visitor is given the opportunity to give a free text, open-ended response The answers to these questions are stored verbatim in the survey database, and are accessible through the "Verbatim" report, question by question
The final report option on the main contents page is the "Crosstabs" report 138 This report provides the designated person with a real-time interactive tool, which gives them the opportunity to select specific questions to generate a crosstab The designated person first selects the "banner" from a scrollable list 140 (see FIGURE 27), which may be any one of the non-free text survey questions, and then selects the "stub", which can be any other one of the non-free text questions from the survey The "stub" answers are then displayed crosstabbed with the "banner" answers, in both table 142 and bar graph 144 form with appropπate statistical notations where applicable (e g mean, median standard deviation and/or standard error), as seen in FIGURES 28A and 28B respectively
Thus, as illustrated by the above the report site provides a very versatile, interactive tool for reviewing the survey results Furthermore, since the results are updated with the data from each newly completed survey, the report site provides real time feed back on the performance of the Web site
The report site, or an alternative site accessible via the Internet, may also provide access to a survey editor facility The fundamental component of this facility is essentially a data entry tool via which the question data and/or survev configuration data in the survey database can be entered and/or amended Consequently, this facility enables, through means known to those of ordinaiy skill, the designated person(s) to edit an existing survey, or even create their own survey, for use with the data collection tool disclosed herein
In more detail, the survey editor is a front-end component that enables the entry and modification of question content and survey configuration Using this front-end component, a user may enter the following information for each question contained in a survey question text, question type (for example, single response, multiple response or open ended), response text, response dependencies, and order of appearance within the survey The user can also enter or modify survey specific configuration data such as question and response color banner and start-up images, invitation frequency, and IP address blocking
Adopting a survey editor such as this will allow someone without skills in the relevant computer programming fields to create and modify questionnaires/surveys In this way, the commissioner of the research or other data collection procedure will be able to create their own surveys using this tool
The system described above can be implemented on conventional Internet compatible hardware The server side components of the data collection tool reside on a survey server or servers connected to the Internet The survey database, the report site and the report tool all reside on a report server or servers connected to the Internet Although in FIGURE 1 the survey server and report server are illustrated as being separate servers and may be resident on separate physical platforms, it is also feasible for a single physical platform and /or server to perform both functions
Particularly when using a single physical platform, the currently preferred server configuration is a Sun SPARC hardware platform, running a Solaris 2 x operating system, but other hardware/OS platforms can be used, so long as they meet with the primary requirements of offering support for a robust RDBMS (such as those mentioned below), JDK 1 x, a web server that supports the Java servlet API, and PERL or equivalent platforms JavaWebServer 1 x is the currently preferred web server, although any web server that supports the Java servlet API could be used The survey database is preferably an RDBMS such as Sybase 11, Oracle, Informix or Progress
The client to and on which the data collection tool is deployed will typically be a Java supporting Web browser such as Microsoft Internet Explorer or Netscape Navigator running in a Windows environment, although the invention is also applicable to other browsers and/or operating systems The performance requirements for the information server are dependent on the characteristics of the information site itself, rather than the characteristics of the system of the invention which, in the implementation described here, does not use any of the resources of the information server, only those of the client and the survey server
While the preferred embodiment of the invention has been illustrated and described, it will be appreciated that \ aπous changes can be made therein without departing from the spirit and scope of the invention For instance, whilst the system has been described consistently as including modules both for collecting data and for reporting the collected data, each of these modules may have independent utility in appropπate circumstances For example, the data collection tool may be used as the front end to data reporting tools other than the inventive one described herein Likewise, the inventive data reporting tool and report site describe herein may be used in conjunction with a database that is populated with data elicited using a data collection tool other than the inventive tool described herein
APPENDIX I
Exemplary Core Survey Questions The following sets forth a set of exemplary survey questions suitable for gathering customer feedback regarding the quality of services and information that are being provided by an organization doing business on the World Wide Web Some of the questions are only to be asked if a particular response was made to a preceding question This is indicated with a Boolean statement at the beginning of the question See for example questions 1 1 and 12, which are only to be asked if the person taking the survey has selected the first or the third response option to question 10 ql Prior to beginning this survey, about how long did you spend exploring this site7
l O Less than a minute
2 O One to three minutes
3 O Four to ten minutes
4 O Over ten minutes
99 O Not sure
q2 In your own words, please describe your purpose for visiting this site
Figure imgf000031_0001
IMPORTANCE/PERFORMANCE
In the following questions we will present you with attributes that describe various characteristics of a web site. q3. First, please indicate how important each of these elements are to you when you are online.
q4 Then, we'd like to know your opinion as to how well (JOB DESCRIPTION) performed on each of these criteria.. RANDOMIZE ORDER.
The site's organization makes information easy to find.
Not No Very Not Very important preference important easy at all easy DK
1 2 3 4 5 6 7 1 2 3 4 5 6 7 99 o o o o o o o o o o o o o o o
02 The site contains valuable links.
Not No Very No valuable Very valuable important preference important links links DK
1 2 3 4 5 6 7 1 2 3 4 5 6 7 99 O O O O O O O O O O O O O O O
03 The site contains valuable supplementary or educational resources
Not No Very Resources Resources important preference important not valuable very
1 2 3 4 5 6 7 at all valuable DK
O O O O O O O 1 2 3 4 5 6 7 99
O O O O O O O o
04 The latest, most current information is available.
Not No Very Not Very important preference important current at all up-to-date DK
1 2 3 4 5 6 7 1 2 3 4 5 6 7 99
O O O O O O O O O O O O O O O -31-
The site loads quickly
Not No Verv Loads Loads important preference important verv siowlv λervtast DK
1 2 3 4 5 6 7 1 2 3 4 5 6 7 99 O O O O O O O O O O O O O O o
The site looks attractive
Not No Verv Not Verv important preferenc important attractive attracts e DK
1 2 3 4 5 6 7 1 2 3 4 5 6 7 99
O O O O O O O O O O O O O O o
The site is easv to navigate
Not No Veπ Not Ven
Important preference important easv at all eas\ DK
1 2 3 4 5 6 7 1 2 3 4 5 6 7 99
O O O O O O O o o o o 0 o o O
nsidering all factors, how satisfied are you with this site7 o Very satisfied o Somewhat satisfies
0 Neutral o Somewhat dissatisfied
0 Verv dissatisfied
IF q5=Somewhat or Very dissatisfled: Please tell us why you give this Web site a poor rating.
Figure imgf000033_0001
q7 How likely are you to visit this site again7
1 O Very likely to visit
2 O Somewhat likely to visit
3 O Neither likely nor unlikely to visit
4 O Somewhat unlikely to visit
5 O Very unlikely to visit q8 How likely are you to recommend that a friend or colleague visit this site7
1 O Very likely to recommend
2 O Somewhat likely to recommend
3 O Neither likely nor unlikely to recommend
4 O Somewhat unlikely to recommend
5 O Very unlikely to recommend
q9 What can we add to make this site more useful to you7
Figure imgf000034_0001
DEMOGRAPHICS
qio. Are you visiting this Web site for business or recreation7
1 O Business
2 O Recreation
3 O Both business and recreation
ql l IF qio="l" OR "3": Which one of the following best describes your industry 7
Figure imgf000035_0001
R "3":Which department do you work in'
Figure imgf000036_0001
-35-
How did you most recently get to (JOB DESCRIPTION)7
O Search engine o Bookmark o Destination site o Linked from another site
IF q!3 SEARCH ENGINE: Which search engine did you link from7
0 Infoseek
O Lycos o HotBot o Yahoo o Excite o Alta Vista
0 Web Crawler o Other 1
IF ql3 LINKED FROM ANOTHER SITE: What site did you link from7
Figure imgf000037_0001
How frequently do you access the Internet7 o Hourly o Daily o A few times a week o A few times a month o Less frequently
Have you ever purchased anything over the Internet7
O Yes O No
ow would you evaluate your level of expertise regarding the Internet7 O Novice user (new to the Internet) O Intermediate user (frequent browser, use plug-ins) O Expert user (programmer, heavy user) qi9 How often have vou visited (JOB DESCRIPTION) before today7
1 O Frequently (more than ten times)
2 O Once in a while (fewer than ten times)
3 O Never before
q20 Please tell me at what speed you are connecting to the Internet
1 o 14 4
2 o 28 8
3 o 33 6
4 o 56 6
5 o ISDN
6 0 T-l
7 o T-3
98 0 Other, SPECIFY
99 o Don't know
Please indicate which of the following Interne! technologies vou have on your computer or RealVideo
Figure imgf000038_0001
Are vou single or married7
O Single O Married
q23 How many children under the age of 18 live in your household7
0 One
0 Two o Three o Four or more o None q24 Are you male or female7
O Male O Female
q25 What is your age7
Under 18
18-24
25-34
35-44
45-54
55-64
65 and over
q26 Please indicate the highest level of education you've completed
1 o Less than high school graduate
2 o High school graduate
3 o One or two years of college
4 o College graduate
5 o Graduate school/advanced degree
q27 Which of the following represents your total household income7
Under $10,000 $10,000 to $24,999 $25,000 to $49,999 $50,000 to $74,999 $75,000 to $99,999 Over $100,000
Thank you very much1 Please complete the following so that we can enter you in the sweepstakes drawing for
Name.
E-mail Address
Phone (work)
Phone (home)
Address
q98 We often complete different types of Internet surveys with respondents like yourself Research subjects range for studies regarding computer usage, to Internet habits, to gathering feedback on new product ideas Please check here if you'd be willing to have us contact you for other research
1 O Yes, contact me again
2 O No, don't contact me again
q99 Overall, how would you rate your experience on this Internet survey compared to other surveys you may have taken in the past
1 o Much better experience
2 o About the same as others
3 o Much worse experience

Claims

Claims
1 A computer network based method for collecting and reporting user data to a client, the method comprising deploying a data collection tool to a client on the computer network, eliciting data from a user accessing the computer network through the client, storing the elicited data in a research database, and providing access to the elicited data to the client
2 The method recited in claim 1 , wherein deploying a data collection tool is initiated by a user visiting a predetermined trigger page
3 The method recited in claim 2 wherein predetermined trigger page is one of a plurality of predetermined trigger pages
4 The method recited in claim 3, wherein deploying a data collection tool further comprises detecting the deploying of the data collection tool, and controlling the deploying as a function of the results of the detecting
5 The method recited in claim 1 wherein deploying a data collection tool further comprises controlling the deploying as a function of the status of one or more test conditions
6 The method recited in claim 5, wherein test conditions comprise at least one of a response to an invitation to respond to a survey, the quantity of responses to the survey relative to a predetermined frequency control function, and the identity of the user
7 The method recited in claim 1 , wherein storing the elicited data in the research database further comprises updating the research database 8 The method recited in claim 1 , further comprising extracting one or more report results from the research database, and wherein providing access to the client further comprises presenting one or more of the report results to the client
9 The method recited in claim 8 wherein extracting one or more report results from the research database further comprises forming one or more reports
10 The method recited in claim 8, wherein presenting one or more of the report results to the client further comprises providing access to one or more of the report results in an interactive query format
1 1 The method recited in claim 1 , wherein storing is performed upon an ending of each the eliciting data, and providing access to elicited data further comprises posting the elicited data to a server and providing access to the server to the client
12 A computer network based method implemented on the Internet for collecting and reporting user data to a client over the Internet, the method comprising presenting an invitation to the user to take an on-line survey executing the on-line survey as a function of the user s response to the invitation, executing the on-line survey comprising sequentially presenting a plurality of questions, storing the user s response to each question, posting all the responses to a survey server for updating a survey database after all the questions are presented, providing access to all responses to the client upon completion of each instance of updating of the survey database
13 The method recited in claim 12, wherein presenting an invitation to take an on-line survey further comprises validating the user 14 The method recited in claim 13, wherein validating comprises determining one or more of the ability of the user's browser to support the survey, prior execution of the survey duπng the instant browser session, and a block of the user's internet protocol address
15 The method recited in claim 14 wherein validating further comprises determining the status of a frequency control function
16 The method recited in claim 15, wherein presenting an invitation to the user to take an on-line survey further comprises presenting a trigger page to a user upon request by the user, the trigger page including a hyperlink to a survey applet
17 The method recited in claim 16, further comprising updating a database of one or more predetermined incidence information data and wherein providing access to all responses to the client further comprises providing access to database of incidence information
18 The method recited m claim 17, wherein incidence information data comprises one or more of a quantity of users accessing the hyperlink, a quantity of the users satisfying the validating, a quantity of invitations presented, a quantity of invitations accepted and a quantity of surveys completed
19 The method recited in claim 12, wherein presenting an invitation to take an on-line survey further comprises storing an indication that the presenting an invitation to take an on- line survey was performed
20 The method recited in claim 12, wherein executing the on-line survey further comprises presenting one or more predetermined questions as a function of the user's response to a first presented one or more of the questions 21 The method recited in claim 12, wherein posting all responses to a survey server occurs in a single transaction
22 A user data collecting and reporting system implemented on the Internet the system comprising a data collection tool for presenting a survey to a user and collecting the user s responses in the form of survey data, a trigger coupled to data collection tool for initiating the survey, a survey database coupled to the data collection tool for storing the survey data, a report site coupled to the survey database for accessing the survey data, and a report tool coupled to the data collection tool for detecting the end of the collecting and coupled to the report site for presenting the survey data after detecting the end
23 The system recited in claim 22, wherein the report site further comprises a front end coupled to the survey database and the report tool for extracting and formatting one or more of the survey data
24 The system recited in claim 22, further comprising an information site, the trigger resident on the information site
25 The svstem recited in claim 22 wherein the data collection tool comprises a server side and a client side communicating with the server side via the Internet
26 The system recited in claim 25, wherein the server side comprises a survey servlet for extracting data defining a configuration of the survey from the survey database. a data grabber servlet for accepting a request for one or more of the survey data, extracting the one or more survey data, and posting the one or more survey data to the survey database, and a question servlet for accepting question text requests, extracting predetermined question text from the survey database, and returning the question text
27 The system recited in claim 26, wherein the server side further comprises a counter servlet for accepting survey progress data
28 The system recited in claim 22, wherein the data collection tool further comprises a survey applet, the survey applet performing a validation function for determining whether to present the survey to the instant user
29 The system recited in claim 22, wherein the trigger comprises a hyperlink to the data collection tool
30 The system recited in claim 22, further comprising a survey editor tool for editing the survey
31 The system recited in claim 22, wherein the survey database, report site, and report tool reside on one or more report servers connected to the Internet
32 The system recited in claim 22, wherein the survey database report site and report tool reside on two or more different servers connected to the Internet
33 A computer network based method implemented on the Internet for collecting visitor data in real-time, the method comprising extracting survey configuration data defining a configuration of a survey from a survey database, the survey including a set of survey questions, presenting the survey to a visitor, collecting visitor's responses to all the survey questions in the form of survey data. extracting one or more of the survey data, and posting the one or more extracted survey data to the survey database in single transaction
34 The method recited in claim 33, wherein presenting the survey to a visitor further comprises executing the survey on the visitor's web browser, whereby interface with the visitor is provided for collecting the survey data
35 The method recited in claim 33, wherein the survey configuration data is a subset of a plurality of configuration data defining a plurality of survey configurations, each of the survey configuration including a subset of a plurality of survey questions
36 The method recited in claim 35 wherein the survey configuration data is determined as a function of a unique descriptive code
37 The method recited in claim 33, wherein presenting the survey to a visitor further comprises determining which of the questions is presented as a function of the user's response to an earlier presented question
38 The method recited in claim 33, wherein collecting further comprises collecting one or more predetermined incidence information data, and posting further comprises updating a database of the incidence information
39 The method recited in claim 33, wherein collecting further comprises collecting survey progress data, and posting further comprises posting the survey progress data to the survey database
40 A computer network based method implemented on the Internet for reporting collected visitor data to a client in real-time, the method comprising storing collected data in a research database upon completion of collection of visitor data, and providing access to the collected data to the client upon completion of each instance of the storing
41 The method recited in claim 40, further comprising extracting one or more report results from the research database
42 The method recited in claim 41 , wherein providing access to the collected data further comprises providing the report results in an interactive query format
43 The method recited in claim 41 wherein extracting one or more report l esults from the research database further comprises storing raw information collected as report results in the database
44 The method recited in claim 41, wherein extracting one or more report results from the research database further comprises forming one or more reports
45 The method recited in claim 44, wherein forming one or more reports further comprises analyzing the report results and forming one or more focused reports
46 The method recited in claim 44, wherein forming one or more reports further comprises manipulating the report results and forming one or more focused reports
47 The method recited in claim 40, wherein providing access to the collected data is performed as a function of matching one of one or more predetermined access names with a corresponding predetermined password 48 The method recited in claim 47, wherein providing access to the collected data further comprises determining which one or more of a plurality of report results are accessible as a function of the password.
PCT/US1999/027695 1998-11-25 1999-11-22 Computer network based system and method for collecting and reporting data WO2000031666A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
AU19190/00A AU1919000A (en) 1998-11-25 1999-11-22 Computer network based system and method for collecting and reporting data

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US10990898P 1998-11-25 1998-11-25
US60/109,908 1998-11-25

Publications (2)

Publication Number Publication Date
WO2000031666A1 true WO2000031666A1 (en) 2000-06-02
WO2000031666A9 WO2000031666A9 (en) 2001-05-17

Family

ID=22330213

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US1999/027695 WO2000031666A1 (en) 1998-11-25 1999-11-22 Computer network based system and method for collecting and reporting data

Country Status (2)

Country Link
AU (1) AU1919000A (en)
WO (1) WO2000031666A1 (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2001055945A2 (en) * 2000-01-31 2001-08-02 E-Steel Corporation System and method for uploading product data to a commodity exchange server
FR2812422A1 (en) * 2000-07-20 2002-02-01 Ge Medical Tech Serv SECURE PREPARATION AND DELIVERY OF DATA REPORTS
EP1342222A2 (en) * 2000-10-11 2003-09-10 Shlomo Lampert Reaction measurement method and system
EP1412889A1 (en) * 2000-06-28 2004-04-28 Right Angle Research LLC Performance measurement method for public relations, advertising and sales events
US7769626B2 (en) 2003-08-25 2010-08-03 Tom Reynolds Determining strategies for increasing loyalty of a population to an entity
US9589475B2 (en) 2001-08-08 2017-03-07 Geoffrey S. Lycas Method and apparatus for personal awareness and growth

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4603232A (en) * 1984-09-24 1986-07-29 Npd Research, Inc. Rapid market survey collection and dissemination method
US5784562A (en) * 1995-10-10 1998-07-21 U S West Advanced Technologies, Inc. System for using a dialog session context to process electronic forms data on the world wide web

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4603232A (en) * 1984-09-24 1986-07-29 Npd Research, Inc. Rapid market survey collection and dissemination method
US5784562A (en) * 1995-10-10 1998-07-21 U S West Advanced Technologies, Inc. System for using a dialog session context to process electronic forms data on the world wide web

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
POWELL TIMOTHY: "Do it yourself survey software", MARKETING COMPUTERS, vol. 16, no. 2, February 1996 (1996-02-01), pages 30, XP000901271 *
PROCTOR, ROBERT W.: "World Wide Web survey research: Benefits, potential problems and solutions", BEHAVIOR RESEARCH METHODS, INSTRUMENTS & COMPUTERS, vol. 29, no. 2, May 1997 (1997-05-01), pages 274 - 279, XP002923876 *
READ WALTER H.: "Gathering Opinion On-line", HRMAGAZINE, vol. 36, no. 1, 1991, pages 51 - 53, XP000901272 *

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2001055945A2 (en) * 2000-01-31 2001-08-02 E-Steel Corporation System and method for uploading product data to a commodity exchange server
WO2001055945A3 (en) * 2000-01-31 2003-01-03 Steel Corp E System and method for uploading product data to a commodity exchange server
EP1412889A1 (en) * 2000-06-28 2004-04-28 Right Angle Research LLC Performance measurement method for public relations, advertising and sales events
EP1412889A4 (en) * 2000-06-28 2005-07-20 Right Angle Res Llc Performance measurement method for public relations, advertising and sales events
FR2812422A1 (en) * 2000-07-20 2002-02-01 Ge Medical Tech Serv SECURE PREPARATION AND DELIVERY OF DATA REPORTS
US7085927B1 (en) 2000-07-20 2006-08-01 Ge Medical Systems, Inc. Secure data report preparation and delivery
EP1342222A2 (en) * 2000-10-11 2003-09-10 Shlomo Lampert Reaction measurement method and system
EP1342222A4 (en) * 2000-10-11 2007-03-28 Assessment Systems Ltd Reaction measurement method and system
US9589475B2 (en) 2001-08-08 2017-03-07 Geoffrey S. Lycas Method and apparatus for personal awareness and growth
US7769626B2 (en) 2003-08-25 2010-08-03 Tom Reynolds Determining strategies for increasing loyalty of a population to an entity

Also Published As

Publication number Publication date
WO2000031666A9 (en) 2001-05-17
AU1919000A (en) 2000-06-13

Similar Documents

Publication Publication Date Title
Punter et al. Conducting on-line surveys in software engineering
US6741967B1 (en) Full service research bureau and test center method and apparatus
US20040039990A1 (en) Automated form and data analysis tool
EP2270731A1 (en) Enhancing virally-marketed facilities
US20080228560A1 (en) System and Method for Creating Compiled Marketing Research Data Over A Computer Network
Matejka et al. IP-QAT: in-product questions, answers, & tips
US20020107726A1 (en) Collecting user responses over a network
US20040019688A1 (en) Providing substantially real-time access to collected information concerning user interaction with a web page of a website
US20060015390A1 (en) System and method for identifying and approaching browsers most likely to transact business based upon real-time data mining
US20030200137A1 (en) Novel system and method for polling a group
WO2003003160A2 (en) System and method for addressing a performance improvement cycle of a business
JPWO2006137479A1 (en) Web Advertising System and Web Advertising Program
Forgionne et al. Management support system effectiveness: Further empirical evidence
WO2004012044A2 (en) Receiving and reporting page-specific user feedback concerning one or more particular web pages of a website
WO2000031666A1 (en) Computer network based system and method for collecting and reporting data
US20030120687A1 (en) System and method of providing customized marketing data
CN114969303A (en) Questionnaire investigation method, device, equipment and storage medium
Fang Web site design factors and usability: a navigation structure experiment
JP2002329035A (en) Electronic totaling system and electronic totaling program
US20030200248A1 (en) Methods and apparatus for design of processes and monitoring performance of those processes
Nasirin et al. Geographical Information Systems (GIS) success factors amongst UK food retailers: Comparisons between market leaders and followers
Degen et al. JIET design process for E-business applications
Adikari Usability Modelling For Requirements Engineering
Pijnenburg et al. Developing a Business Process Family Model for the Integration of an Online Sales Channel
Hiott Collecting and using networked statistics: Current status, future goals

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AE AL AM AT AU AZ BA BB BG BR BY CA CH CN CU CZ DE DK EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MD MG MK MN MW MX NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT UA UG UZ VN YU ZA ZW

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): GH GM KE LS MW SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE BF BJ CF CG CI CM GA GN GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
REG Reference to national code

Ref country code: DE

Ref legal event code: 8642

AK Designated states

Kind code of ref document: C2

Designated state(s): AE AL AM AT AU AZ BA BB BG BR BY CA CH CN CU CZ DE DK EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MD MG MK MN MW MX NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT UA UG UZ VN YU ZA ZW

AL Designated countries for regional patents

Kind code of ref document: C2

Designated state(s): GH GM KE LS MW SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE BF BJ CF CG CI CM GA GN GW ML MR NE SN TD TG

COP Corrected version of pamphlet

Free format text: PAGES 1/35-35/35, DRAWINGS, REPLACED BY NEW PAGES 1/35-35/35; DUE TO LATE TRANSMITTAL BY THE RECEIVING OFFICE

122 Ep: pct application non-entry in european phase