CA2396766A1 - Storing and retrieving the visual form of data - Google Patents

Abstract

Data representing a visual form of data is received. The data including content data and format data indicating the manner in which the content data is to be visually represented. At least some of the content data is identifi ed in accordance with a template, which includes one or more extraction instructions. The retrieved data may then be stored in the identified conten t data. Or, in response to the data, an action may be taken such as initiating a process. A template may be applied to several data files, storing visual for ms of data and the information from those files may be stored in a database in association with the visual forms of data. A user may input an extraction instruction by visually identifying a region of a sample visual form of data and selecting a manner to extract the content data in reference to the identified region.

Description

STORING AND RETRIEVING THE VISUAL FORM OF DATA
Background This invention relates to storing the visual form of data. Computer programs generally maintain data in a variety of formats. There usually is one format that is 1 o unique, and typically proprietary, to each computer program in which raw data is stored persistently. This format usually is designed to reduce the amount of information actually stored and, in some cases, to restrict the ability of a third party to access the data. Data in this format generally is created by a "save" function of the computer program. The save function formats the raw data and stores the formatted raw data in yet another format, called a "file," that is defined by the operating system for which the computer program is designed. Data that is being processed by a computer program is stored in another format, also typically proprietary, called a "data structure," which generally is stored in volatile or working memory during execution of the computer program. A data structure usually is designed to permit the 2 o data to be processed efficiently by the computer program, while minimizing the amount of memory needed to represent the data.
With many computer programs, the most useful form of the data from the perspective of the user is its visual form, e.g., what is displayed on a computer display or what is printed. However, this form of the data often is not captured into 2 5 permanent or persistent storage, unless it is printed and the printed form is electronically scanned. In particular, the file format used by a computer program often does not maintain data in a visual form for several reasons. The visual form of the data generally requires more information to be represented and can be reconstructed from raw data that requires less information to be represented.

3 o Therefore storing the visual form of the data generally is considered unnecessary.
Part of the visual form of data produced by a computer program is generated, for example, from environmental data (such as the date and time) or user selected data that is being processed, and is not recoverable from the file format, but only from the data structures of the computer program. Although some data structures represent the visual form of the data, often there is no mechanism to retain the visual form of the data other than by printing. Some operating systems permit displayed data to be copied from one computer program to another using a "cut-and-paste" operation.
But this operation generally requires the other computer program to be in operation on the same machine. Some computer programs also do not have these operations available to the user. For some computer programs, the printed form of the data, not the displayed data, is most useful and this operation does not provide access to the printed data.
Even if the visual form of data from a computer program is stored, as new versions of the computer program are used, or if the computer program is no longer available, access to that data is impeded. Also, another computer program still might not be able to access the data if the data is stored in a proprietary format.
This lack of access to the visual form of the data from a computer program creates a variety of problems when this form of the data is desired for creating compound documents from multiple sources of data, particularly if the data is created, used and shared over a period of time by multiple different users with multiple different computer programs that are dispersed geographically. As a particular 2 0 example, in the pharmaceutical industry, data may be acquired from many laboratory instruments in geographically dispersed laboratories over a significant period of time, and then may be combined to produce reports, for example, for regulatory compliance. The inability to centrally access an electronic visual form of the data from these instruments adds a significant cost to regulatory compliance.
Summary In one aspect, the invention features receiving data representing a visual form of data including content data and format data indicating the manner in which the content data is to be visually represented; identifying at least some of the content data 3 0 in accordance with a template; and storing the identified content data.

Embodiments of this aspect of the invention may include one or more of the following features.
The data representing the visual form of data may be normalized in accordance with a displayed form of the visual form of data. The visual form of data may be characterized by a plurality of dimensions characterized by at least two coordinate systems where normalizing the data representing the visual form of data includes converting values expressed in the two coordinate systems into a common coordinate system. The common coordinate system may be the coordinate system of a displayed form of the visual form of data.
The template includes at least one extraction instruction for identifying at least some of the content data from the received data, and the extraction instruction includes information indicating location of at least some of the content data based on the common coordinate system. The data representing the visual form of data includes data in a format required by an operating system layer for outputting the visual form of data by a printer. The operating system layer may be Windows operating system and the data representing the visual form of data may be a Windows metafile.
The template includes at least one extraction instruction for identifying at least some of the content data from the received data. The visual form of data may be 2 0 characterized by a plurality of dimensions characterized by a coordinate system and the extraction instruction may include information indicating location of the desired data based on the coordinate system. The visual form of data may be characterized by a plurality of dimensions and the extraction instruction may include information with respect to location of a reference marker and a direction in one of the plurality of 2 5 dimensions where identifying at least some of the content data includes searching in the direction for identifying at least some of the content data in the direction.
A sample visual form of data may be displayed. Data may be received from a user indicating the location of data selected by the user in the displayed sample visual form of data, and the extraction instruction may be formed based on location data 3 o identifying the location of the data selected by the user.

The extraction instruction may stored in association with data representing the sample visual form of data.
The received data further may represent a plurality of visual forms of data where storing the identified content data further includes storing the identified content data in association with data representing a corresponding one of a plurality of visual forms of data.
In another general aspect, the invention features a graphical user interface including a region for displaying a sample visual form of data, a region enabling a user to input location data identifying a location of data selected by the user, and a region causing a computer program to form an extraction instruction using the location data identifying the location of the data selected by the user.
In another general aspect, the invention features receiving data representing a visual form of data including content data and format data indicating the manner in which the content data may be to be visually represented; identifying at least some. of the content data in accordance with a template; and initiating performance of an action based on results of the identifying of at least some of the content data.
In yet another general aspect, the invention features a computer implemented technique of receiving information defining a parsing criterion including displaying a graphical user interface for displaying a multi-dimensional document containing 2 o multiple units of information; and receiving first information from a user identifying a location within the displayed document, and second information specifying a desired unit of information based on a location of the desired unit of information relative to the identified location, where the information defining the parsing criterion includes the first and second information.
2 5 Embodiments of this aspect of the invention may include one or more of the following features.
A plurality of documents can be parsed to identify units of information based on the parsing criterion. The identified units of information may be stored on a computer readable medium. The document may be parsed based on the parsing 3 0 criterion to identify the desired unit of information. The identified information may be processed to arrive at new information. Information identifying at least one user-definable action to be performed on the identified information may be received.
In yet another aspect, the invention features a computer implemented technique of receiving information defining a parsing criterion including displaying a graphical user interface for displaying a mufti-dimensional document containing multiple units of information; and receiving first information from a user identifying a visual maxker within the displayed document, and second information defining a desired unit of information within the document by specifying a relative position of the unit of information with respect to the marker, where the information defining the 1 o parsing criterion includes the first and second information.
In yet another aspect, the invention features a computer implemented technique of receiving information defining~a parsing criterion including displaying a graphical user interface for displaying a mufti-dimensional document containing multiple units of information displayed in a mufti-dimensional space; and receiving first information a user identifying a region within the displayed document, and second information defining a desired unit of information within the document by specifying a relative position of the unit of information with respect to the region, where the information defining the parsing criterion includes the first and second information. In some embodiments, the second information may indicate that the 2 0 desired unit of information overlaps with the identified region or that the desired unit of information may be contained within the identified region.
In yet another aspect, the invention features displaying a graphical user interface for displaying a mufti-dimensional document containing multiple units of information displayed in a mufti-dimensional space; and receiving first information 2 5 from a user defining a desired unit of information within the document by specifying a relative position of the unit of information and second information identifying an action to be executed depending on the existence or non-existence of the unit of information within the document.
In another aspect, the invention features displaying a graphical user interface 3 o for displaying a mufti-dimensional document containing multiple units of information displayed in a mufti-dimensional space; receiving first information from a user defining a desired unit of information within the document by specifying a relative position of the unit of information and second information identifying an action to be executed depending on the existence or non-existence of the unit of information within a selected region of the document.
All publications, patent applications, patents, and other references mentioned herein are incorporated by reference in their entirety. In case of conflict, the present specification, including definitions, will control. In addition, the methods, and examples are illustrative only and not intended to be limiting.
Other features and advantages of the invention will be apparent from the following detailed description, and from the claims.
Brief Description of the Drawings FIG. 1 is a schematic diagram illustrating an embodiment of a system for storing the visual form of data from an application into a database.
FIG. 2 is a block diagram illustrating operations of the system of FIG. 1.
FIG. 3 is a block diagram illustrating operations of a template builder module of the system of FIG. 1.
FIG. 4 is an illustration of a graphical user interface for managing projects in a project manager module of the system of FIG. 1.
2 o FIG. 4A is an illustration of a graphical user interface for selecting and applying templates.
FIG. 5 is an illustration of a graphical user interface for creating or editing templates in the template builder module of the system of FIG. 1.
FIG. 6 is an illustration of a graphical user interface for creating or editing an 2 5 extraction instruction for searching for a tag within a rectangle in the template builder module of the system of FIG. 1.
FIG. 7 is an illustration of a graphical user interface for creating or editing an extraction instruction for searching for a tag next to a marker in the template builder module of the system of FIG. 1.

FIG. 8 is an illustration of a graphical user interface for ereatittg or editing an extraction instruction for searahi~g for a word match in the template builder rnoc~ulc ofthe system afp'TG. 1, FT.G. 9 is ara illustration of a graphical user interface for creating or editing an extraction instruction for deriving a tag from searahecl. fot- values irt the template builder module of the system of FIG. 1.
1L IG. 10 is a schematic dia~;rat~i of an embodiment of a data structure of a template.
FIG. 1 t is an illustration of a gaphiaal user interface for listing extraction instructions of a template ire the template builder xnadule ofthe system of FIG. 1.
FIG. ;L2 is an illtcstration of a graphical user inter.Eacc for displaying an audit trail in the templaxe builder rxlodule of the system of FT.G. 1.
T'lt~, 13 is a knock diagrm illustrating opcratiorts of a template runner module of the system of FIG. 1.
1 S ~'IG. 14 is a schematic cliagcam of an embvdizlent of a data structure of a database storing rcpc~rts in ass~aciatiorr with. tags.
Detailed Desct-igtiQn Referring to FIG. 1, system 10 includes a plurality of report generatirxg 2 0 alrplicatiQns 12 rur~rling on cc~znp~tters 7 2a. Computers 12a are couxtc~Cied thtaugh a network 14 to a computer t 6a rutitxing a project managex module 16, a computer 7 8a running a database manager 18, and a computer 24a running a user application 24.
DatRbase mdnag~r 18 is connectod to a database 20. $ach one of report generating applications 1 Z is capahte of generating visual forms of data, typically intended t'rar 2 5 printers. !n the case of computers 12a running operating systems rr~earketed under the 'Ulrindows trademark by Microsoft Corporation of l~odmvnd, Vlr'ashin.gton, the visual forms of data, are stored as windows metafilcs. Such metafiles and visuttE.E
torrns of data will artso be referred to as reports.
Applications 12 cart output the reports over network 1~- to database manager 3 cJ 18 as described in. patent application U.S. Serial lVa. 091213,(119, fled December Lfi, 1998 ("'U19 application"}, for exarngle, by tnatcing function calls to ($e operating RECT(FIE~ SHEET (RULE 9i) tSAIEP

systems of computers I2a.. The ~furtetion ca113 cause the aperatittg systems to create data files {for o~arnple, Wutdaws metahles) representing the visr~al forms otthe data.
(that is, the reports)_ each data file includes at least two types of data:
content data which is the data to be represented visually, and. fot~aat data whfoh includes ' cornma~ds and other information :far causing the content data to be visually represented ire. a particular manner. The function calls generally are provided by' the operatin:,~ systems in ardor to perrrlit applications such as report generating applications 1~ to print data to a printer, ha system 10, the data generated by these applications are received by database iuanager 18 and stored in database ~0, To 20 increase ease a~td efJieieney of organising and searchin,~ through the reports stored in database 20, each report is stared in association with one or more tags. Tags arc fields of d to which can stare data in as~iation with the reports. As taught in the '0l 9 application, a user may input the data ~ the tags, Its system 1 (?, tags may also be generated based on operations performed on 7. 5 the reports in accot'danee with a ienrplate. ~1 temptate is a cullectian of ono or mare cxtract9vn iristnxctions for extracting data from a repoxt. ~d. extraction ittstntcaion, is tine or more commands or criteria four seloatiz~g and oxtractin,~ content data (with or without assooiat~l. format data) from a data file storug data representing a visual form of the deco. A template may be applied to a selected batch of reports to generate lags z o for each one of the reports in the batch to be stored ire association u~iif~ each one of chose records. In orae embodiment, templates specifically store instructions for retrieving the data based on the visual aspects of the generated reports. such visual aspects may include the location c~fthe data within the report of within a selected portion of the report. The instructions may also, for exarxa.ple, indycate the general 2 5 direcdan of a desyred data compared to a selected reference ~narlter in the report.
To allow a user to crate or edit a template, lxrajcct manager module 7 6 pravit~es the user with various graphical user interfaces to input the e~ctraction instructions. In one emboditrient, project manager ruodule 16 allows tht user to select a satnplc or base report based on which the user may ge~aerate a template. The 3 0 selected report is displayed to the user. The user may input the extraction instructions by visually indicating to the computer the location where a desired data should appear RECT(FIE~ SHEET (RULE 9i) tSAIEP

in a report. The user also may visually specify a location of a reference marker and a general direction in which a desired data, should appear in a report relative to the masker, gccause reports are mold-dim,ensfonaX, in that data within them is net only displayed in two dimension hut also in various orientations or ire mare than one page, then the extraction instructions naay be applied in one, or mare dixnousio~ns of the visual forjt~ of data.
The generated. templates then can be applied by project manager module 1 ti, or any afher application progr.~n having the capability of applying templates, to a batch of reports stored in database 2Q or, in realtime, as the reports are being outputted by x 0 report generating applications 2~ to database manager 18.
As is apparent from the above description, system 10 cam be used to integrate various component systctns oFlarge cuterpriscs. ForexampJe, in a priarruaccutical enterprise, each one ofvarnputers 12~e can be rescdrch and development computers ogeratix~g within a single laboratory or multiple laboratories spread across various ~.5 locations ofan enterprise. By printing to a common database, database manager i8 can centrally store tlia reports to retrieve the ittforruation from within those reports and provide them for further pratcssing to at'her applicatiatl programs such as user application program 24.
Each one of eamptiter$ I2a, T6a,18a, and 24a can include a main unit connected to bath. an output devico which displays infoxmation to a user and au input device whicli receives input from a user_ The main unit generally includes a processor cannvci;ed to a memory system via an irnterco~~ction mecharxism. The input devicer and output device also are connected to the proeessoc~ and memory system. via the interGantxection mechanism.
2 5 Qrle car mare output devices may be 4unnected to ttte computers. Example output devices include a cathode ray tube (CI~T) display, Jiquiti crystal displays (LGI~) and other video autpttt devices, printers, corarnunication devices such as a modem, storage devices such as a desk or tape, and audio output. One or more input devices may be connected to the computer system. Example input devices include a 3 o keybaa~xd, keypad, track ball, nzouse, pen and talylet, communication devico, and data input devices such as audio and video capture devices. The invention is not limited ~
RECT(FIE~ SHEET (RULE 9i) tSAIEP

the particular input or output devices used in combination with the computer system or to those described herein.
Each one of the computers may be a general purpose computer system which is programmable using a computer programming language, such as C++, Java, or other language, such as a scripting language or assembly language. The computer system may also include specially programmed, special purpose hardware, or an application specific integrated circuit (ASIC). In a general purpose computer system, the processor is typically a commercially available processor, of which the series x86, Celeron, and Pentium processors, available from Intel, and similar devices from AMD
1 o and Cyrix, the 680X0 series microprocessors available from Motorola, the PowerPC
microprocessor from IBM, the Alpha-series processors from Digital Equipment Corporation, and the MIPS microprocessor from MIPS Technologies are examples.
Many other processors are available. Such a microprocessor executes a program called an operating system, of which windows family of operating systems including Windows NT, and Windows 95 or 98, Linux, UNIX, IRIX, DOS, VMS MAC OS and OS8 are examples, which controls the execution of other computer programs and provides scheduling, debugging, input/output control, accounting, compilation, storage assignment, data management and memory management, and communication control and related services. The processor and operating system define a computer 2 0 platform for which application programs in high-level programming languages are written.
A memory system typically includes a computer readable and writeable nonvolatile recording medium, of which a magnetic disk, a flash memory CD-ROM
(rewriteable), and tape are examples. The magnetic disk may be removable, known as 2 5 a floppy disk, or permanent, known as a hard drive. A magnetic disk has a number of tracks in which signals are stored, typically in binary form, i.e., a form interpreted as a sequence of one and zeros. Such signals may define an application program to be executed by the microprocessor, or information stored on the disk to be processed by the application program. Typically, in operation, the processor causes data to be read 3 0 from the nonvolatile recording medium into an integrated circuit memory element, which is typically a volatile, random access memory such as a dynamic random access memory (DRAM) or static memory (SRAM). The integrated circuit memory element allows for faster access to the information by the processor than does the disk. The processor generally manipulates the data within the integrated circuit memory and then copies the data to the disk after processing is completed. A variety of mechanisms are known for managing data movement between the disk and the integrated circuit memory element, and the invention is not limited thereto.
The invention is not limited to a particular memory system.
Various computer platforms, processors, or high-level programming languages can be used for implementation. Additionally, the computer system may be a multiprocessor computer system or may include multiple computers connected over a computer network. Each computer program modules (e.g. 12, 16, 18 and 24) in FIG.
1 may be separate modules of a computer program, or may be separate computer programs. Such modules may be operable on separate computers. Data may be stored in a memory system or transmitted between computer systems. The plurality of computers or devices may be interconnected by a communication network, such as a public switched telephone network or other circuit switched network, or a packet switched network such as an Internet protocol (IP) network. The network may be wired or wireless, and may be public or private.
Such a system may be implemented in software or hardware or firmware, or 2 0 any combination thereof. The various elements of the system, either individually or in combination may be implemented as a computer program product tangibly embodied in a machine-readable storage device for execution by a computer processor.
Various steps of the process may be performed by a computer processor executing a program tangibly embodied on a computer-readable medium to perform functions by operating 2 5 on input and generating output. Computer programming languages suitable for implementing such a system include procedural programming languages, object-oriented programming languages, and combinations of the two.
The claims axe not limited to a particular computer platform, particular processor, or particular high-level programming language. Additionally, the computer 3 o system may be a multiprocessor computer system or may include multiple computers connected over a computer network. Various possible configurations of computers in a network permit many users to participate in an auction, even if they are dispersed geographically.
Using the Windows95, Windows98 and WindowsNT operating systems, the data files or reports from report generating applications 12 representing the visual form of the data output by the operating system in response to function calls from an application to print the data is in a Windows Metafile format, according to Microsoft.
A metafile is a vector image, or, a list of commands, draw objects, text, and commands to control style. Theoretically, a metafile may be used in any Windows application. A Windows metafile (WMF) is a 16-bit metafile that is supported by Windows 3.1. An enhanced metafile (EMF) is a 32-bit enhanced metafile that is supported by Windows 95, Windows 98, and Windows NT having a super set of WMF commands.
The operation of printing in the Windows operating systems and its use to capture the visual form of data from an application into a database will now be described. In order to print on a printer in a Windows environment, the printer has an associated print driver. When the printer is installed, the operating system is informed of the location of the print driver, i.e., its file name. The print driver specifies the characteristics of the printer to the operating system.
An application 12 (as in FIG. 1) permits a user to select a printer through a 2 o user interface, such as a graphical user interface with menus. The selected printer also may have various printing options that may be selected. Through a function call made by the application in response to user input, the user may invoke a user interface for the print driver to permit the user to specify user information and printing preferences.
Given a selected printer, preferences and information to be printed, the application 12 2 5 issues function calls to a Graphics Device Interface (GDI-32), which is part of the Windows operating system. The GDI-32 requests the selected print driver and its user interface for information about the printer that in turn is given back to the application 12, and is retained by the GDI-32, to assist in the process of generating a correct sequence of function calls to the operating system to print the selected information.
3 o The GDI-32 outputs data into spool files and makes function calls with the names of the spool files to the spooler process to queue the spool files for printing on their designated printers. A spool filc in the 'windows operating system is designated as a Wirxdows metaf 1e by Microsafr. A printer spool ale is not a true metaftle, however, because it aettxally contains prizxtcr setup datra irt aclditian to atly xefetet~ced or embedded metafles. In Windvws9~, a spool hle contains the file names of any metaiiles. In 'V~indowsNT, the metafxles arc embedded in the spool fsle. In both cases, there is oxce spool file per printed. doctcxnerlt, and each page of a docutuaut has a separate metafile.
The spoolcr prc>4ess is informed about the location of a print processor associated with the selected print driver. The spaaler process calls tlxe print prpcessor to process any spool rtes thst the spaalar process has queued fc~.c the print prcyccssor, Gauerally, a typycal print processor receives the spool file frnrn the spooler process and converts it td a format used by the printer, such a punter control language (FCI~), foatScripl: or ether, typically pr~op~xetary, format. lt~tcad of printing, the print proaeaa~or causes the vector image data produced by the operating system to be IS formatted, associated with tags, and stored in a database.
More details about metafiles, print dxiveCS, print processors, spooler processes and spool ftles are available through the Miorcxsal~l Developer Network and iho Micrc~sop Tyevelo~pmetat Network >;.ibrary accessible thrcxugh the Internet.
Database 20 and database manager I S may be any lcitzd of database, including 2 fl a relational database, object-oriented database, unstructured database or other database. Example relational databases i~cludc Oracle $i from Oracle Corporation of Redwood City, California, Tnfatmix Dynamic Ser~rpr from ltzFormix Sof3tware, Inc. of Menla Park, California, I~.B~ from International .l~usine$s Machines of Yarktown ~Ieights, New York, and Access from Microsoft Co~orativn afRedmond, 25 ~fashingtcyn. An examcple objc:ct~rrier~ied database is UbjeetStore from Object Design of"l~urlington,112assachusetts. An example unstructured database is I~Totes from the Iaotus Corporation, of Cambridge, lViassa.ehusetts. A database also rnay be canstc~xcted using a. flat ftle system, for example by using f les with character-delimited fields, such as its early versions of dBAS~, now lCnovvxx as Visual tI,BAS~
3 Q Cram hiprise Corp. of Secxtts Va~lloy, California, totrne~rly .Borland GGtxternaticxnal Crn~.

RECT(FIE~ SHEET (RULE 9i) tSAIEP

Referring to FIG. 2, having generally described system 10, the structure and operation of project manager module 16 will now be described in more detail.
Project manager module 16 includes two modules: a template builder module 30 and a template runner module 32. Template builder module 30 provides various GUIs (which will be described in reference to FIGs. 5-11) to obtain data 34 from the user.
Using data 34 and a sample or base report 36 selected by the user, template builder module 30 creates a template 38. In one embodiment, the data structure of~template 38 includes two components. One component is an edit template component 40 which includes all the information used for future editing of the template and various l 0 record keeping and security information. Another component is a runtime extraction instructions component 42 which includes the information used to apply the template to a batch of reports. After generating template 38, template builder module 30 sends template 38 to database manager 18 to be stored in database 20.
As stated above, each report generating application 12 can generate a batch of visual forms of data files 44 (that is, reports) to be stored in database 20.
Reports 44 may be sent to the template runner module 32 or database manager 18. If sent to template runner module 32, template runner module 32 retrieves template 38 from database 20 through database manager 18 and applies template 38 to the batch of reports. Template runner module 32 can also retrieve a batch of reports stored in 2 0 database 20 and apply template 38 to the retrieved batch of reports. By applying the template, template runner module 32 generates tags based on data content of the visual forms of data and causes them to be stored in database 20 in association with the reports (an example of the data structure of such a database is shown in FIG. 14).
Template runner module 32 may also associate user input tags with the reports.
The 2 5 tags can then be used for searching and organizing the reports. Also, the tags may be provided, whether in association with the reports or not, for further data processing by various user applications 24.
The structure and operation of template builder module 30 and template runner module 32 will now be described in detail. Referring to FIG. 3, template builder 3 o module 30 includes a graphical user interface engine 50 which interacts with the user to obtain various extraction instructions to be incorporated into template 38.
GUI

iz7terface engine 50 provides xnstructians 52, as they are eritered by the user, to ,an instruction application engine 34. To create a templa~to, a user selects ;~
sample or base report 36 which is normalized by a nor~nalir~ttion engine 58 to getier-ate a nortnah~d report GU which is stored in memory in association with base report 3G.
Instruction apptication origine S4 receives the narmali:~,ed report t~0 anal aprlx~es tile extraction it~structians .frog fiUS. interi~ace engine 54 at~d provides the results 62 of the exi~ra,etion instruciiolt execration prsacess to GLTI interface engine 50. G'tTI itcterface engine Sp displays the results so that the user can dynaanicatly determine whCthcr the extraction iiistructiaris 52 are achieving the desired results. After the user tnishes eitteriucg the extraction insintctioris far generating template 38, 'instCUGtion application engine 54 outputs template 38. The r~pec~ation of GuI interface ~0, uotmalization engine SS, arid iiistruciian apgiicaiio~x ertgirle 54 will craw be dasaribed iu detail.
fGeferring to FIG. 4, project manager module 16 first displays a graphical user interface l a0 to the user. GUI 1 UO yncludes a project ciircctary pane 142 which ~ 5 displays a tree dirootory of the various projects available to the ~ezser.
each protect includes at least one batch c~frcporis, which may be stored in databaso 20 or which are to be generated by report gcrterating applications 1,2. A project can also inGludc cane or more templates. For example, project I Od includes four templates 1 D6 and a set of reports to be generated. Region lOG displays settings associated with the batch of 2 C~ reports in a selected project.
Referring to p 1G, 4A, upon sdlecting a project or a template, project rrtanagcr rt2odulc 16 displays a tag information GUI 201) to the user. GUT 2U0 displays in region 202, the project rianic and the various fiuictioris which may be performed on a project by pressing say rune ofbe~ttorts 204. Tag information GUf 20U farther includes 2 5 a template region ~Q4 which ptvvides a drop dovau rueri~r Za6 which includes various templates associated With the project. A button 208 allows a user to select to apply a template to a selected hatch of reports. Region 214 displays the information associated wish the sarxtple or base reporC of the selected template. Region includes the carious tag& for which the template extraction insiructi4ns are used to 3 0 extract data.

RECT(FIE~ SHEET (RULE 9i) tSAIEP

ItefeTring to FIG. 5, template builder module 3t) displays a template btiildcr module graphical user interface 3D0 to the mer For creating or editing a template. To create a templates the user first selects a report based on which the template is created.
The selected rc,~nrt is Misplayed in a report display pane 342, To g~erate the template, the user then inputs the odious extraction urstructions for ext~actiu~g data frazn the report.
In the described eznbodimctxt, the userinay input one or more extraat~ion instxuetions &electad among at least four typGS of extraction ~instruo~iaz~s.
A first type or cxtractit~n instruction allows finding data in a patticnlar axes in the report and inserting the data info a selected. tag iti association with that report, To do so, a user can select a rectangle 3U4 by usitlg a. mouse and painter combination and right clicking vrt the mouse to select the appropriate option From a displayed menu (not shown). Template bui.Ider module 30 then causes computer I6a to display a tag within a GUt 40Q (shown. ilz FIG. 6) to the user. Refe~c~-tng also to FIG. &, G~ 40W
includes a rectangle region 402 in which the cootdinates of the rectangle in the displayed form of the report are displayed.. In region 4U4, GUI 400 displays the page number wvithin the report in which rectangle 304 appe~lrs. IrTote that the number of gages in a report can be thought as a dimen5ian of the report. GUl ~QQ then provides the user with the option of including orals the text chat is car~~pletely wrtthi~,~ the rectangle or alI text 2 i~ which intersecrts the rectangle (region 406). W tag field region 40$, GUI
4U0 provides a drag down .menu form which the user ~ select the namo aFthe tag where the retrieved information will be inserted. In region 4I0, GUl 440 provides the user with the aptxon of not failing the template (i.e., the option of continuing to apply the temlstate despite an armor condition), oven if no text is found within the designated 2 5 rectangle far a particular report and allowing madilicakon of extracted tag vetoes in a tag dialogue G'UI 200 shaven in FIG, 4A.
Another type of extraction insl~otion allows finding data located in a direction relative to a selected reference point in the report and inserting the data into a selected tag in association with the report . A usc,~r can select a rectmgle 30$ by 3 ~ using a ~notxsa and pointer combination and tight clicking on the mouse to select the appropriate option iivm a displayed »zenu (not shown). Taraplate bui.Idcr module 30 RECT(FIE~ SHEET (RULE 9i) tSAIEP

than causes computer I6a to display ~ tag next to marker GUI 700 ~shawn in p'IG. 7) to the user. Referring also to p'IG. 7, CiCTI 700 inchtdes a marker designation region 74~ ins which a user aura select tfic trxanner in which the marker should be searched for in the report. 'l~~e user may select to search for the x~tarker in tlxe entire rapart or its the s selected rc~tangte 308 m the ~reprart. GUi 700 further itrcludcs a region 7G4 in which the direction of the location aI'the text to be included in the tag is relative to the marker can be dcsi,gna$ed. Direction can he designated in any one of the rnultaple dimensiaxts oFthe report. Template builder raodulo 3(~ uses the direction and a set of predetermined insinzctions to end text nnosfi likely intended by the user iv be included z o in the tag field. In region 704, the user is provided with a ~pnll down menu in which the user can select the tag field isa which the text should be inserted. Tai region 7(lC, GUI. 70G provides the user with the oritiou. of not failing the template even iI'na text is found within the designated reolaxtgle for a particular report and allowing uwdif~catiQn of extracted tag values iri a tag dialogue ~ dialogete GUI Z00 shown in F'IG.4A.
Another type of extraction intstruction allows determining whether a particx~lar word or phrase appears in a report and setting a Boolean lag in association vvltE~ the report. To do ao, a user can select a rectangle 308 by using a mouse and pointer combination and right cliaIdng on the manse to select the appropriate option from a 2 0 displayed rrteriu (not shown). Template builder module 3~ then causes computer 7 6a to display a tag next to marker GUI S00 (Shown in FIG. 8) to 'the user.
~aferring to rlG. 8, GU18ta0 includes a region 80~ for inserting the particular word ox phrase to be searched for. Tn region 804'fhe marker may be se~,rclaed for in the entire report or in the selected rectangle 3a8 in the report. f'he user may selcc;t in region 806 whether to 2 5 search only within the rcctatTgle or to search at~.y text that intersects the rectangle, as was the vase with the first typo of extraction irystruction. ~ldditianally, the user may select to ignore t~pperfiawer case differences.
Armther type o:F extraction instruction ailov~s inserting data into a sel~ted tag in association with the report based on a value deriv~:c3 from data in the report, To do 3 0 sc~, a user' can right click on the mouse to select the appropriate option i'rvm a displayed menu. {not shown). Template buiidL~r modurc ~0 then causes coaaputer lea RECT(FIE~ SHEET (RULE 9i) tSAIEP

to display a tag from derived value GUI 91)0 ~sbo~cvn iu3 FIf's 9) to the user. ltefcrring iu FIG, 9, in GUI 9UU, the user can select irt region 902 ho~v the value is derided and in region 904 the tag field in which the derived r~alt~e should be inserted.
The value fnay, far exaxraple, be derived based ot~ functions sucft as mathematical functions performed on data extracted from the report.
Itefetrir~g tQ 1~JG. 14, a. user can review the extraction t'ules of a template in a template details GUI 1000 ~u~d select any one of sham to edit.
Rcfcrnx~g to FIGS. 3 and 6, as the usor inputs cacti extraction i~~st~cuction, instxuetion application engine S~. applies tlxe instruction to the sample report and supplies the t~esttlts 62 to Gt3t cngizie 50. GUi engine 50 in turn displays the results pano 306 of Ct,~rUl 3fl0. 'This allows the user to monitor whether floe extraction l nstrtrction resulted in the correct data being extracted from the sample report and whether the extraction instruction should be mc~di.~ed.
Using these GUIs, the user can input W a extraction instntctic~ns foz a t~plate . 5 to be used in processing a batch of r~:ports. It should be noted that the same GUIs cork be used far editing a template. Aiier cor~apleting creating or editing a template, iztstruction application engine 54 outputs template ~8. FTG. 11 is a blook diagram of the data structuro of template 313. As mentioned above in one embodiment, the data represonti~ng template 38 is structured to include a template edit component 40 and a 2 o template nu~tirne component 42. Template edit cc~mponertt 4a iztcludes a plurality of records 70. C?ns record, which is the header record, stores ideritiilcatiou information with respect to the template such as its name, date of creation, and so on.
The header record also stores information with respect to the data structure of tl~e template edit corrtponextt such as the rtumher pf records in the template edit compartent.
Another Z 5 record 70 stares user .input comments which may provide a description of the template. Another record stores the sample record to assist with future editing ofthc template. Yet another record stores the e~ctraction iristruetlons. FIG. 1 d Shows a. CCUI
window 1 Oa0 for displaying the list of extraction instn~ctions input by the user . 'ftte list shows the tat; Held to rwhich an instruction applies and the cxtractiotx instruction 3 o for that tat; field. 3:'he user c-an select any one of the instructions and edit the iz~struotioos.
is RECT(FIE~ SHEET (RULE 9i) tSAIEP

Yet another record 7I1 in template edit component ~0 stores an audit trail which is record. of all the changes and edits erode to the tet~aplate. This record allows heightened security for ensuring data integrity which is important for regulatory purfc~sc as, for example, in pharmaceutical industry for gaining FI3A
approvals, 1~'IG.
L~ shows a GLII window 1 IOU for displaying the audit trail.
Ftefarring bacts to FIG. ~, havia,g described in detail various graphical user interfaces used by template builder module 3Q to intoract with the user to obtain the template extraction instructions, we will norw descxibe the rrtanner in which tem~plale instruction appIicatioti cttgine $4 processes the user input infbrmatic~n to genc:rt~te the template. One of the di~frcuities in generating tliv leKnplate is i,'arraixig a correspondence between areas of the report selected Ity thv user ari a computer display and the data. ~tle representing the visual Form of data,. The si a and coordinate values associated with the data in the data file, which scores the visual fiorm of data for cxaruple as a vectored image, does not necessarily correspond to the size and coartlinate values associated with the displayed visual form o~Cdata tin the computer display. For example, a f le representing visr~al forms of data includes a plurality of text strings. Each .text string includes text and formaitiirg information four displaying the text as part ofthe visual .form of data. The Carmatting information generally deterrnincs the location of the text string yn a displayed visual form of data, the si~:e of 2 o the ~ont, anel ether such formatting, information. The formatting information also defines a rectangular space within the visual form of data far displaying the string.
However, this informt~tson does n4t always correspond to the si~,c and coordinates o't' the displayed visual farm of data, 'fhe marmot irt which, the data. f~Je stores data also does not always support 2 5 efficient and accurate searching or size and coordinate conversion. For examplo, we have observed that application programs 12 commonly generate witbirr a single report strings with differing fonts, sizes, and coordinates. 'text which one would expect to be inciudul in one text string also may be split between two strings. p'crr example, a single word may t~s split and placed into different text stings. A user seleetin.g a 3 p rectangle Qn the camguter screen, as discussed in reference to F7.G. 6, also may assume he is selecting a single ward or phrase, However, the text strip storing that t~
RECT(FIE~ SHEET (RULE 9i) tSAIEP

single word may store the wood with trait~ng or leading white spaces. 1-Ience, if the user, far example, selects to include only text that is completely within a selected reciaf~.gle, the text string in #ho data r~lc rnay appear to be outside of the selected rectangle, i,Ience, to s~xpport proper creation, cditirtg, and application of templates, the reparis ate I'~erst noruxali2ed.. The first step in normalizing the report is to ensure that all of the toardinate systems in the report use the cootdiaate systems used by lice di splay system. fn that case, perfarniing matches between the text solccted on the semen by the user and the text in the visual faun of data can readily be dine.
'fo do I o so, i~ormali~atio~t engine 58 scans the report and translates ala of the coordinate refcren4es to those used by the display.
The second step pcrforrned by the naraializatian engine 58 is to ensure that the size aFthe text string are scaled to correspond to the display size on the display device. Some operatir~~ systems, Such as the WindovYS brand operating systems may nit provide accurate informatio~t arydlor accux'ate teehniQues for ~crfoxm~.g such scaliutg. ~Ience, for such operating systems, the scaling is optixniacd to correspond is tt~,e scale a~ the display stri~~g of the display d4viGe.
The third step perfornncd by normalization engine 58 involves joining and splitting text stxings so ti~at each text string will contain a logica'1 unit of tcxi. k'ar 2 0 example, rxormalization engine 5& ensures that alI wards vVhioh have been split will be contained within single strings. In addition, normalization engine 58 ensures that words separated by xr~are than three spaces are separated from each other.
icading andlax traiLng spaces area are placed in their own individual taxt strings.
Other instructions may be used to decide an whether to slit or join text.
Ailer performing these steps, nornaalization engine 58 generates a normalized reparl 60 which includes the hose or sample report and a list oFalT
nQrrnali~ed strings attd their associated ncyrrnali~ed size and lacatio~n in the visual form o~f daia. This allows a quick application of extraction instructions S2 by inslruotion application engine ~~.
3 d As cash extraction instzuction 52 is input by the user, instnxotion application engine S4 compares the various coordinate and location iaiformation of the itlstructio~t RECT(FIE~ SHEET (RULE 9i) tSAIEP

to the coordinate and. location u~f'ormation associated with the strings in the list of narmali2ed rcpor! 6U to find those text strings which arc in the vicinity oft;oordinate itzformation associated with the extraction instruction 52. After f~nditl8 those text strings, instruction application engine 54 finds those tent strings which satisfy the cotidilions of the it~tru~tiott. Instruction application eligine 54 applies a #lexible standard in fult"xlli~g the condition, 6vCause the manner in v~hioh the ~caorrdinates are specified by the user on the screen typically inal~ede some error. Makching the coordinates of the visual form of data ai3er, normali7,ation to the coordinates on the screen also includes a degree c~f error. Hence, requirin,8 only, for example, a 90°l0 or 9~% rnatt;h, ensures that user extraction instructions are applied properly.
Referring to FfC'rs. 3 and 13, having described the n~nner in which the templates are gcn~eraied, we will now describe the structure anal operation of template builder module 30. 'fert~plate runner module 32 shares sorx2e ofthe modules of tcmplttte builder module 30. For example, template buitdet~ module includes the x5 normalization engine 58 a~xd the instruction application engine 54. As a'batoh of reports 44a are received, normalization engine 58 normalizes each of t~hc reports and provides each an;e of the normalized reports fro to insir~uotion application etlgine ~4.
Iristr<ictiou application engine j4 retrieves runtime tcruplate component 42 of template 38 frr~m database 20 and applies the extraction insteuciioos to ouch. report.
1i' any 2 t~ errors ocCUr doting the application of template, insiructic~n application enginr~ 54. may displa~r the error #o the user and request for all input by the user. After applying the template to a report, instruction application engine 54 outputs the final it~tructinzt application results to the database manager I8. The final result can be a database record ,far each One of the records (shown in FIG. 14) which associates the generated 25 tags with the report from which the tags overt; generated.
F'or example, template ~8 can not only include cxtraction~ instructions far extracting data from the repoxt, but ran also iuclutie information input by tlzc ust'r to be inptzi info tags in accordance with methods and tcr~chniques described zn the '019 application.
3 p Template runner module 32 eanrun as an independent application program which does not support ereaiitig anti editing templates, but supports manipulating RECT(FIE~ SHEET (RULE 9i) tSAIEP

projects and applying templates to batches of reports. Hence, templates can generated at a central location at an enterprise and applied by various users (for example, user 26 in FIG. 1). Vendors and developers can generate templates and sell runtime components of those templates to various enterprises.
In FIG. 1, although components of system 10 are shown to be connected via network 14, the components ma be configured differently. For example, components of system 10 may operate on a single computer. Or, project manager module 16 and database manager 18 may operate on the same computer. Additionally, template runner module 32 may operate on the same computer with either project manager module 16 or database manager 18, or both. Network 14 can be an intranet (such as a local area network, a wide area network, or various combinations thereof), or the Internet, or combinations thereof.
Having now described a few embodiments, it should be apparent to those skilled in the art that the foregoing is merely illustrative and not limiting, having been presented by way of example only. Numerous modifications and other embodiments are within the scope of one of ordinary skill in the art and are contemplated as falling within the scope of the claims.
Other embodiments are also within the scope of the following claims.
For example, components of system 10 may operate on a single computer. Or, 2 o project manager module 16 and database manager 18 may operate on the same computer. Additionally, template runner module 32 may operate on the same computer with either project manager module 16 or database manager 18, or both.
Network 14 can of course be an intranet (such as a local area network, a wide area network, or various combinations thereof), or the Internet, or combinations thereof.

Claims (49)

1. A computer implemented method comprising.
receiving data representing a visual form of data comprising content data and format data indicating the manner in which the content data is to be visually represented; and identifying at least some of the content data in accordance with a template;
and storing the identified content data.

2. The method of claim 1 further comprising normalizing the data representing the visual form of data.

3. The method of claim 2 wherein the data is normalized in accordance with a displayed form of the visual form of data.

4. The method of claim 2 wherein the visual form of data is characterized by a plurality of dimensions characterized by at least two coordinate systems, wherein normalizing the data representing the visual form of data includes converting values expressed in the two coordinate system into a common coordinate system.

5. The method of claim 4 wherein the common coordinate system is the coordinate system of a displayed form of the visual form of data.

6. The method of claim 4 wherein the template includes at least one extraction instruction for identifying said at least some of the content data from the received data, and the extraction instruction includes information indicating location of at least some of the content data based on the common coordinate system.

7. The method of claim 1 wherein the data representing the visual form of data comprises data in a format required by an operating system layer for outputting the visual form of data by a printer.

8. The method of claim 7 wherein the operating system layer is Windows operating system and the data representing the visual form of data is a Windows metafile.

9. The method of claim 1 wherein the template includes at least one extraction instruction for identifying said at least some of the content data from the received data.

10. The method of claim 9 wherein the visual form of data is characterized by a plurality of dimensions characterized by a coordinate system and the extraction instruction includes information indicating location of the desired data based on the coordinate system.

11. The method of claim 9 wherein the visual form of data is characterized by a plurality of dimensions and the extraction instruction includes information with respect to location of a reference marker and a direction in one of the plurality of dimensions, wherein identifying at least some of the content data includes searching in the direction for identifying at least some of the content data in the direction.

12. The method of claim 9 further comprising:~
displaying a sample visual form of data, receiving data from a user indicating location of data selected by the user in the displayed sample visual form of data, and forming the extraction instruction based on location data identifying the location of the data selected by the user.

13. The method of claim 12 further comprising:
storing the extraction instruction.

14. The method of claim 13 further comprising:
storing the extraction instruction in association with data representing the sample visual form of data.

15. The method of claim 1 wherein the received data further represents a plurality of visual forms of data.

16. The method of claim 15 wherein storing the identified content data further includes:
storing the identified content data in association with data representing a corresponding one of a plurality of visual forms of data.

17. Computer readable media containing a computer program comprising instructions for:
receiving data representing a visual form of data comprising content data and format data indicating the manner in which the content data is to be visually represented;
identifying at least some of the content data in accordance with a template;
and storing the identified content data.

18. Computer system comprising:
a input port that receives data representing a visual form of data comprising content data and format data indicating the manner in which the content data is to be visually represented;
a processor that identifies at least some of the content data in accordance with a template; and a storage media that stores the identified content data.

19. A method comprising:
transmitting data representing a computer program comprising instructions for:
receiving data representing a visual form of data comprising content data and format data indicating the manner in which the content data is to be visually represented;
identifying at least some of the content data in accordance with a template; and storing the identified content data.

20. A graphical user interface comprising:
a region for displaying a sample visual form of data, a region enabling a user to input location data identifying a location of data selected by the user, and a region causing a computer program to form an extraction instruction using the location data identifying the location of the data selected by the user.

21. A computer implemented method comprising:
displaying, on a display, a sample visual form of data, displaying, on the display, a region enabling a user to input location data identifying a location of data selected by the user, and displaying, on the display, a region causing a computer program to form an extraction instruction using the location data identifying the location of the data selected by the user.

22. Computer system comprising:
a processor, and a display, the processor executing instructions causing the display to:
display a sample visual form of data, display a region enabling a user to input location data identifying a location of data selected by the user, and display a region causing a computer program to form an extraction instruction using the location data identifying the location of the data selected by the user.

23. Computer readable media storing a program comprising instructions for:
displaying, on a display, a sample visual form of data, displaying, on the display, a region enabling a user to input location data identifying a location of data selected by the user, and displaying, on the display, a region causing a computer program to form an extraction instruction using the location data identifying the location of the data selected by the user.

24. A method comprising:
transmitting data representing a computer program comprising instructions for:
displaying, on a display, a sample visual form of data, displaying, on the display, a region enabling a user to input location data identifying a location of data selected by the user, and displaying, on the display, a region causing a computer program to form an extraction instruction using the location data identifying the location of the data selected by the user.

25. A computer implemented method comprising receiving data representing a visual form of data comprising content data and format data indicating the manner in which the content data is to be visually represented;~
identifying at least some of the content data in accordance with a template;
and initiating performance of an action based on results of said identifying of at least some of the content data.

26. Computer readable media containing a computer program comprising instructions for:
receiving data representing a visual form of data comprising content data and format data indicating the manner in which the content data is to be visually represented;
identifying at least some of the content data in accordance with a template;
and initiating performance of an action based on results of said identifying of at least some of the content data.

27 27. Computer system comprising:
a input port that receives data representing a visual form of data comprising content data and format data indicating the manner in which the content data is to be visually represented; and a processor that identifies at least some of the content data in accordance with a template and initiates performance of an action based on results of said identification of at least some of the content data.

28. A computer implemented method of receiving information defining a parsing criterion comprising displaying a graphical user interface for displaying a multi-dimensional document containing multiple units of information;
receiving first information from a user identifying a location within the displayed document, and second information specifying a desired unit of information based on a location of the desired unit of information relative to the identified location, wherein the information defining the parsing criterion includes the first and second information.

29. The method of claim 28 further comprising:
parsing a plurality of documents to identify units of information based on the parsing criterion.

30. The method of claim 29 further comprising:
storing the identified units of information on a computer readable medium.

31. The method of claim 28 further comprising:
parsing the document based on the parsing criterion to identify the desired unit of information.

32. The method of claim 31 further comprising 28~~

processing the identified information to arrive at new information.

33. The method of claim 31 further comprising:
receiving information identifying at least one user-definable action to be performed on the identified information.

34. Computer readable media containing a computer program for receiving information defining a parsing criterion comprising instructions for:
displaying a graphical user interface for displaying a multi-dimensional document containing multiple units of information;
receiving first information from a user identifying a location within the displayed document, and second information specifying a desired unit of information based on a location of the desired unit of information relative to the identified location, wherein the information defining the parsing criterion includes the first and second information.

35. Computer system for receiving information defining a parsing criterion comprising:
a display that displays a graphical user interface for displaying a multi-dimensional document containing multiple units of information;
an input port receiving first information from a user identifying a location within the displayed document, and second information specifying a desired unit of information based on a location of the desired unit of information relative to the identified location, wherein the information defining the parsing criterion includes the first and second information.

36. A computer implemented method of receiving information defining a parsing criterion comprising displaying a graphical user interface for displaying a multi-dimensional document containing multiple units of information;
receiving first information from a user identifying a visual marker within the displayed document, and second information defining a desired unit of information within the document by specifying a relative position of the unit of information with respect to the marker, wherein the information defining the parsing criterion includes the first and second information.

37. Computer readable media containing a computer program for receiving information defining a parsing criterion comprising instructions for:
displaying a graphical user interface for displaying a multi-dimensional document containing multiple units of information;
receiving first information from a user identifying a visual marker within the displayed document, and second information defining a desired unit of information within the document by specifying a relative position of the unit of information with respect to the marker, wherein the information defining the parsing criterion includes the first and second information.

38. Computer system for receiving information defining a parsing criterion comprising:
a display that displays a graphical user interface for displaying a multi-dimensional document containing multiple units of information;
an input port first information from a user identifying a visual marker within the displayed document, and second information defining a desired unit of information within the document by specifying a relative position of the unit of information with respect to the marker, wherein the information defining the parsing criterion includes the first and second information.

39. A computer implemented method of receiving information defining a parsing criterion comprising displaying a graphical user interface for displaying a multi-dimensional document containing multiple units of information displayed in a multi-dimensional space;
receiving first information a user identifying a region within the displayed document, and second information defining a desired unit of information within the document by specifying a relative position of the unit of information with respect to the region, wherein the information defining the parsing criterion includes the first and second information.

40. The method of claim 39 wherein the second information indicates that the desired unit of information overlaps with the identified region.

41. The method of claim 39 wherein the second information indicates that the desired unit of information is contained within the identified region.

42. Computer readable media containing a computer program for receiving information defining a parsing criterion comprising instructions for:
displaying a graphical user interface for displaying a multi-dimensional document containing multiple units of information displayed in a multi-dimensional space;
receiving first information a user identifying a region within the displayed document, and second information defining a desired unit of information within the document by specifying a relative position of the unit of information with respect to the region, wherein the information defining the parsing criterion includes the first and second information.

43. Computer system for receiving information defining a parsing criterion comprising:
a display that displays a graphical user interface for displaying a multi-dimensional document containing multiple units of information;
an input port receiving first information from a user identifying a visual marker within the displayed document, and second information defining a desired unit of information within the document by specifying a relative position of the unit of information with respect to the marker, wherein the information defining the parsing criterion includes the first and second information.

44. A computer implemented method comprising displaying a graphical user interface for displaying a multi-dimensional document containing multiple units of information displayed in a multi-dimensional space;
receiving first information from a user defining a desired unit of information within the document by specifying a relative position of the unit of information and second information identifying an action to be executed depending on the existence or non-existence of the unit of information within the document.

45. Computer readable media containing a computer program comprising instructions for:
displaying a graphical user interface for displaying a multi-dimensional document containing multiple units of information displayed in a multi-dimensional space;
receiving first information from a user defining a desired unit of information within the document by specifying a relative position of the unit of information and second information identifying an action to be executed depending on the existence or non-existence of the unit of information within the document.

46. Computer system program for receiving information defining a parsing criterion comprising:
a display that displays a graphical user interface for displaying a multi-dimensional document containing multiple units of information displayed in a multi-dimensional space;
an input port that receives first information from a user defining a desired unit of information within the document by specifying a relative position of the unit of information and second information identifying an action to be executed depending on the existence or non-existence of the unit of information within the document.

47. A computer implemented method comprising displaying a graphical user interface for displaying a multi-dimensional document containing multiple units of information displayed in a multi-dimensional space;
receiving first information from a user defining a desired unit of information within the document by specifying a relative position of the unit of information and second information identifying an action to be executed depending on the existence or non-existence of the unit of information within a selected region of the document.

48. Computer readable media containing a computer program comprising instructions for:
displaying a graphical user interface for displaying a multi-dimensional document containing multiple units of information displayed in a multi-dimensional space;
receiving first information from a user defining a desired unit of information within the document by specifying a relative position of the unit of information and second information identifying an action to be executed depending on the existence or non-existence of the unit of information within a selected region of the document.

49. Computer system program for receiving information defining a parsing criterion comprising:
a display that displays graphical user interface for displaying a multi-dimensional document containing multiple units of information displayed in a multi-dimensional space;
an input port that receives first information from a user defining a desired unit of information within the document by specifying a relative position of the unit of information and second information identifying an action to be executed depending on the existence or non-existence of the unit of information within a selected region of the document.