CA2633458C - Method and system for extending keyword searching to syntactically and semantically annotated data - Google Patents

Abstract

Methods and systems for extending keyword searching techniques to syntactically and semantically annotated data are provided. Example embodiments provide a Syntactic Query Engine ("SQE") that parses, indexes, and stores a data set as an enhanced document index with document terms as well as information pertaining to the grammatical roles of the terms and ontological and other semantic information. In one embodiment, the enhanced document index is a form of term-clause index, that indexes terms and syntactic and semantic annotations at the clause level. The enhanced document index permits the use of a traditional keyword search engine to process relationship queries as well as to process standard document level keyword searches. In one embodiment, the SQE comprises a Query Processor, a Data Set Preprocessor, a Keyword Search Engine, a Data Set Indexer, an Enhanced Natural Language Parser ("ENLP"), a data set repository, and, in some embodiments, a user interface or an application programming interface.

Description

METHOD AND SYSTEM FOR EXTENDING KEYWORD SEARCHING
TO SYNTACTICALLY AND SEMANTICALLY ANNOTATED DATA
BACKGROUND OF THE INVENTION
Field of the Invention The present invention relates to a method and system for searching for information in a data set, and, in particular, to enhanced methods and systems for syntactically indexing and performing syntactic searching of data sets using relationship queries to achieve greater search result accuracy.
Background Often times it is desirable to search large sets of data, such as collections of millions of documents, only some of which may pertain to the information being sought. In such instances it is difficult to either identify a subset of data to search or to search all data yet return only meaningful results.
The techniques that have been traditionally applied to support searching large sets of data have fallen short of expectations, because they have not been able to achieve a high degree of accuracy of search results due to inherent limitations.
One common technique, implemented by traditional keyword search engines, matches words expected to found in a set of documents through pattern matching techniques. Thus, the more that is known in advance about the documents including their content, format, layout, etc., the better the search terms that can be provided to elicit a more accurate result. Data is searched and results are generated based on matching one or more words or terms that are designated as a query. Results such as documents are returned when they contain a word or term that matches all or a portion of one or more

2 keywords that were submitted to the search engine as the query. Some keyword search engines additionally support the use of modifiers, operators, or a control language that specifies how the keywords should be combined when performing a search. For example, a query might specify a date filter to be used to filter the returned results. In many traditional keyword search engines, the results are returned ordered, based on the number of matches found within the data. For example, a keyword search against Internet websites typically returns a list of sites that contain one or more of the submitted keywords, with the sites with the most matches appearing at the top of the list. Accuracy of search results in these systems is thus presumed to be associated with frequency of occurrence.
One drawback to traditional keyword search engines is that they do not return data that fails to match the submitted keywords, even though the data may be relevant. For example, if a user is searching for information on what products a particular country imports, data that refers to the country as a "customer" instead of using the term "import" would be missed if the submitted query specifies "import" as one of the keywords, but doesn't specify the term "customer." For example, a sentence such as "Argentina has been the main customer for Bolivia's natural gas" would be missed, because no forms of the word "import" are present in the sentence. Ideally, a user would be able to submit a query and receive back a set of results that were accurate based on the meaning of the query ¨ not just on the specific keywords used in submitting in the query.
Natural language parsing provides technology that attempts to understand and identify the syntactical structure of a language. Natural language parsers ("NLPs") have been used to identify the parts of speech of each term in a submitted sentence to support the use of sentences as natural language queries against data. However, systems that have used NLPs to parse and process queries against data, even when the data is highly structured, suffer from severe performance problems and extensive storage requirements.
Natural language parsing techniques have also been applied to extracting and indexing information from large corpora of documents. By their nature, such systems are incredibly inefficient in that they require excessive storage and intensive computer processing power. The ultimate challenge with such systems has been to find solutions to reduce these inefficiencies in order to create viable consumer products. Several systems have taken an approach to reducing inefficiencies by subsetting the amount of information that is extracted and subsequently retained as structured data (that is only extracting a portion of the available information). For example, NLPs have been used with Information Extraction engines that extract particular information from documents that follow predetermined grammar rules or when a predefined term or rule is recognized, hoping to capture and provide a structured view of potentially relevant information for the kind of searches that are expected on that particular corpus. Such systems typically identify text sentences in a document that follow a particular part-of-speech pattern or other patterns inherent in the document domain, such as "trigger" terms that are expected to appear when particular types of events are present. The trigger terms serve as "triggers" for detecting such events. Other systems may use other formulations for specified patterns to be recognized in the data set, such as predefined sets of events or other types of descriptions of events or relationships based upon predefined rules, templates, etc. that identify the information to be extracted.
However, these techniques may fall short of being able to produce meaningful results when the documents do not follow the specified patterns or when the rules or templates are difficult to generate. The probability of a sentence falling into a class of predefined sentence templates or the probability of a phrase occurring literally is sometimes too low to produce the desired level of recall.
Failure to account for semantic and syntactic variations across a data set, especially heterogeneous data sets, has led to inconsistent results in some situations.
BRIEF SUMMARY OF THE INVENTION
Embodiments of the present invention provide enhanced methods and systems for syntactically indexing and searching data sets to achieve more accurate search results with greater flexibility and efficiency than previously available. Techniques of the present invention provide enhanced indexing techniques that extend the use of traditional keyword searching techniques to relationship and event searching of data sets. In summary, the syntactic and/or semantic information that is gleaned from an enhanced natural language parsing process is stored in an enhanced document index, for example, a term-clause matrix, that is amenable to processing by the pattern (string) matching capabilities of keyword search engines. Traditional keyword search engines, including existing or even off-the-shelf search engines, can be utilized to discover information by pattern (or string) matching the terms of a relationship

3 query, which are associated with syntactic and semantic information, against the syntactically and/or semantically annotated terms of sentence clauses (of documents) that are stored in the enhanced document index. In this manner, the relationship information of an entire corpus can be searched using a keyword search engine without needing to limit a priori the types or number of relationships that are stored.
Example embodiments of the present invention provide an enhanced Syntactic Query Engine ("SQE") that parses, indexes, and stores a data set, as well as performs syntactic searching in response to queries subsequently submitted against the data set. In one embodiment, the SQE
includes, among other components, a data set repository and an Enhanced Natural Language Parser ("ENLP"). The ENLP parses each object in the data set and transforms it into a canonical form that can be searched efficiently using techniques of the present invention. To perform this transformation, the ENLP
determines the syntactic structure of the data by parsing (or decomposing) each data object into syntactic units, determines the grammatical roles and relationships of the syntactic units, associates recognized entity types and/or ontology paths if configured to do so, and represents these relationships in a normalized form. The normalized data are then stored and/or indexed as appropriate in an enhanced document index.
In one aspect, a corpus of documents is prepared for electronic searching by parsing each sentence into syntactic elements, normalizing the parsed structure to a plurality of tagged terms, each of which indicate an association between the term and a type of tag, and then transforming each sentence into a data structure that treats the tagged terms as additional terms of the sentence to be searched by a keyword search engine. In some embodiments, the tags include a grammatical role tag, a part-of-speech tag, an entity tag, an ontology path specification, or an action attribute. Other tags that indicate syntactic and semantic annotations are also supported. In some embodiments, linguistic normalization is performed to transform the sentence.
In another aspect, the SQE supports a syntax and a grammar for specifying relationship searches that can be carried out using keyword search engines. In one embodiment, the syntax supports a base component that specifies a syntactic search, a prepositional constraint component, a keyword (e.g., a document level keyword) constraint component, and a meta-data constraint component. One or more of the components may be optional. In

4 another embodiment, the components are combined using directional operators that identify which query term has a desired grammatical role.
In yet another aspect, the SQE receives a query that specifies a relationship query using a term, tag type, or tag value. The SQE transforms the query into a set of Boolean expressions that are executed by a keyword search engine against the data structure that has been enhanced to include syntactic and/or semantic annotations. Indicators to matching objects, such as clause, sentences, or documents are returned. In one embodiment, the data structure comprises a term-clause index, a sentence index, and a document index.
In another aspect, the SQE performs corpus ingestion and executes queries using parallel processing. According to one embodiment, each query is performed in parallel on a plurality of partition indexes, which each include one or more portions of the entire enhanced document index.
BRIEF DESCRIPTION OF THE DRAWINGS
Figure 1 shows a relationship query and the results returned by an example embodiment of the InFact 2.5 search engine.
Figure 2 is an example block diagram that conceptually represents a term-clause matrix that stores terms and enhanced indexing information for syntactic searching.
Figure 3 is an example block diagram that conceptually represents a traditional term-document index.
Figure 4 is an example block diagram of an example Syntactic Query Engine.
Figure 5 is an overview of the steps performed by a Syntactic Query Engine to process data sets and relationship queries.
Figures 6A-6G are example screen displays that illustrate the general capabilities of the example user interface and the types of queries that can be executed by an example Syntactic Query Engine.
Figures 7A-7F are example display screens of the progression of an example RQL query submitted to a Syntactic Query Engine.
Figures 8A-8F are example screen displays of an interface associated with browsing ontology paths, viewing corpus metadata, and finding synonyms.
Figure 9 is an example screen display of an interface associated with setting preferences for constraining relationship searches.

5 Figure 10 is an example screen display of an interface associated with displaying SQE query history.
Figures 11A-11F are example screen displays from an alternate graphical based interface for displaying and discovering genetic relationships.
Figure 12 is a conceptual block diagram of the components of an example embodiment of a Syntactic Query Engine.
Figure 13 is a block diagram of the components of an Enhanced Natural Language Parser of an example embodiment of a Syntactic Query Engine.
Figure 14 is a block diagram of the processing performed by an example Enhanced Natural Language Parser.
Figure 15 is a block diagram illustrating a graphical representation of an example syntactic structure generated by the natural language parser component of an Enhanced Natural Language Parser.
Figure 16 is a table that conceptually illustrates normalized data that has been annotated with syntactic and semantic tags by the postprocessor component of an Enhanced Natural Language Parser.
Figure 17 is an example block diagram of data set processing performed by a Syntactic Query Engine.
Figure 18 is a block diagram of query processing performed by an Syntactic Query Engine.
Figure 19 is an example flow diagram of relationship query processing steps performed by an example Query Processor of Syntactic Query Engine.
Figure 20 is an example block diagram of a general purpose computer system for practicing embodiments of a Syntactic Query Engine.
Figure 21 is an example block diagram of a distributed architecture for practicing embodiments of a Syntactic Query Engine.
Figure 22 is a block diagram overview of parallel processing architecture that supports indexing a corpus of documents.
Figure 23 is a block diagram overview of parallel processing architecture that supports relationship queries.
Figure 24 is an example block diagram that shows parallel searching of an enhanced document index.
Figure 25 is an example block diagram of an architecture of the partition indexes that supports incremental updates and data redundancy.

6 Figure 26 is an example conceptual diagram of the transformation of a relationship search into component portions that are executed using a parallel architecture.
Figure 27 is an example flow diagram of the steps performed by a build file routine within the Data Set Preprocessor component of a Syntactic Query Engine.
Figure 28 illustrates an example format of a tagged file built by the build_file routine of the Data Set Preprocessor component of a Syntactic Query Engine.
Figure 29 is an example flow diagram of the steps performed by the dissect_file routine of the Data Set Preprocessor component of a Syntactic Query Engine.
Figure 30 is an example conceptual block diagram of a sentence that has been indexed and stored in a term-clause index of a Syntactic Query Engine.
Figure 31 is an example conceptual block diagram of sample contents of a document index of a Syntactic Query Engine.

7 DETAILED DESCRIPTION OF THE INVENTION
It is often desirable to search large sets of unstructured data, such as collections of millions of documents, only some of which may pertain to the information being sought. Traditional search engines approach such data mining typically by offering interactive searches that match the data to one or more keywords (terms) using classical pattern matching or string matching techniques. At the other extreme, information extraction engines typically approach the unstructured data mining problem by extracting subsets of the data, based upon formulations of predefined rules, and then converting the extracted data into structured data that can be more easily searched.
Typically, the extracted structured data is stored in a relational database management system and accessed by database languages and tools. Other techniques, such as those offered by Insightful Corporation's InFact products, offer greater accuracy and truer information discovery tools, because they employ generalized syntactic indexing with the ability to interactively search for relationships and events in the data, including latent relationships, across the entire data set and not just upon predetermined extracted data that follows particular syntactic patterns. InFactas syntactic indexing and relationship searching uses natural language parsing techniques to grammatically analyze sentences to attempt to understand the meaning of sentences and then applies queries in a manner that takes into account the grammatical information to locate relationships in the data that correspond to the query. Some of these embodiments support a natural language query interface, which parses natural language queries in much the same manner as the underlying data, in addition to a streamlined relationship and event searching interface that focuses on retrieving information associated with particular grammatical roles. Other interfaces for relationship and event searching can be generated using an application programming interface ("API").
Insightful's syntactic searching techniques are described in detail in U.S. Provisional Application Nos.
60/312,385 and 60/620,550, and U.S. Application Nos. 10/007,299, and 10/371,399. The techniques described in these patent applications have typically employed the use of complex data bases with a proprietary search technology for performing relationship and event searching.
Embodiments of the present invention provide enhanced methods and systems for syntactically indexing and searching data sets to achieve more accurate search results with greater flexibility and efficiency than previously

8 available. Techniques of the present invention provide enhanced indexing techniques that extend the use of traditional keyword search engines to relationship and event searching of data sets. In summary, the syntactic and semantic information that is gleaned from an enhanced natural language parsing process is stored in an enhanced document index, for example, a form of a term-clause matrix, that is amenable to processing by the more efficient pattern (string) matching capabilities of keyword search engines. Thus, traditional keyword search engines, including existing or even off-the-shelf search engines, can be utilized to discover information by pattern (or string) matching the terms of a relationship query, which are inherently associated with syntactic and semantic information, against the syntactically and semantically annotated terms of sentence clauses (of documents) stored in the enhanced document index. As another benefit, the additional capabilities of such search engines, such as the availability of Boolean operations, and other filtering tools, are automatically extended to relationship and event searching.
Relationship and event searching, also described as "syntactic searching" in U.S. Application Nos. 60/312,385, 10/007,299, 10/371,399, and 60/620,550, supports the ability to search a corpus of documents (or other objects) for places, people, or things as they relate to other places, people, or things, for example, through actions or events. Such relationships can be inferred or derived from the corpus based upon one or more "roles" that each term occupies in a clause, sentence, paragraph, document, or corpus. These roles may comprise grammatical roles, such as "subject," "object," "modifier,"
or "verb;" or, these roles may comprise other types of syntactic or semantic information such as an entity type of "location," "date," "organization," or "person," etc. The role of a specified term or phrase (e.g., subject, object, verb, place, person, thing, action, or event, etc.) is used as an approximation of the meaning and significance of that term in the context of the sentence (or clause).
In this way, a relationship or syntactic search engine attempts to "understand"
the sentence when a query is applied to the corpus by determining whether the terms in sentences of the corpus are associated with the roles specified in the corresponding query. For example, if a user of the search engine desires to determine all events in which "Hillary Clinton" participated in as a speaker, then the user might specify a relationship query that instructs a search engine to locate all sentences/documents in which "Hillary Clinton" is a source entity and "speak" is an action. In response, the syntactic search engine will determine and return indicators to all sentences/clauses in which "Hillary Clinton" has the

9 role of a subject and with some form of the word "speak" (e.g., speaking, spoke) or a similar word in the role of a verb.
For example, Figure 1 shows a relationship query and the results returned by an example embodiment of the InFact 2.5 search engine. In the InFact 2.5 product, a user of the search engine can specify a search for a known "source" or "target" entity (or both) looking for actions or events that involve that entity. The user can also specify a second entity and look for actions or events that involve both the first and second entity. The user can specify a particular action or may specify a type of action or any action. An entity specified as a source entity typically refers to the corresponding term's role as a subject (or subject-related modifier) of a clause or sentence, whereas an entity specified as a target typically refers to the corresponding term's role as an object (or object-related modifier) of a clause or sentence. An action or event typically refers to a term's role as a verb, related verb, or verb-related modifier. Moreover, instead of a specific entity, the user can specify an entity type, which refers to a tag such as an item in a classification scheme such as a taxonomy. A user can also specify a known action or action type and look for one or more entities, or entity types that are related through the specified action or action type. Many other types and combinations of relationship searches are possible and supported as described in the above-mentioned co-pending patent applications.
In the example user interface shown in Figure 1, a value for the first known entity is specified in entity field 102, a value for a known action is specified in action field 105, and a value for the type of the second entity is specified in entity type field 107. The source field 103 and target field 104 indicate whether the first known entity is to be a source of the action or a recipient (target) of the action. The particular query displayed instructs the search engine to look for sentence clauses that describe any person that drives a jeep when the Find Relationships button 106 is pressed. The results are returned in result field 110, which is shown sorted by similarity to the query.
Example embodiments of the present invention provide an enhanced Syntactic Query Engine ("SQE") that parses, indexes, and stores a data set, as well as performs syntactic searching in response to queries subsequently submitted against the data set. In one embodiment, the SQE
includes, among other components, a data set repository and an Enhanced Natural Language Parser ("ENLP"). The ENLP parses each object in the data set (typically a document) and transforms it into a canonical form that can be searched efficiently using techniques of the present invention. To perform this transformation, the ENLP determines the syntactic structure of the data by parsing (or decomposing) each data object into syntactic units, determines the grammatical roles and relationships of the syntactic units, associates recognized entity types if configured to do so, and represents these relationships in a normalized form. The normalized data are then stored and/or indexed as appropriate.
In one set of example embodiments, which were described in U.S.
Application Nos. 60/312,385, 60/620,550 10/007,299, and 10/371,399, normalized data structures are generated by an enhanced natural language parser and are indexed and stored as relational data base tables. The SQE
stores the grammatical relationships that exist between the syntactic units and uses a set of heuristics to determine which additional relationships to encode in the normalized data structure in order to yield greater accuracy in results subsequently returned in response to queries. For example, the SQE may generate relationship representations in the normalized data structure that correspond to more "standard" ways to relate terms, such as the relationship represented by the tuple (subject, verb, object), but may also generate relationships that treat terms with corresponding certain grammatical roles in a non-standard fashion, such as generating a relationship representation that treats a term that is a modifier of the subject as the subject of the sentence itself. This allows the SQE to search for a user specified entity (as a subject) even in sentences that contain the specified entity as a modifier instead of as the subject of the sentence. For example, the clause:
"the young boy bought a dog"
may be parsed and assigned the following grammatical roles:
boy = subject young = modifier bought = verb dog = object Relationship representations that correspond to (boy, bought, dog), as well as a relationship representations that corresponds to (young, bought, dog) may be generated and stored by the SQE. Once the relationship representations are generated, they are stored in a variety of as relational data base tables to facilitate retrieval.

In the example embodiments of the SQE that are described herein, the normalized data, including the grammatical role and other tag information that can be used to discover relationships, are integrated into enhanced versions of document indexes that are typically used by traditional keyword search engines to index the terms of each document in a corpus. A
traditional keyword search engine can then search the enhanced indexing information that is stored in these document indexes for matching relationships in the same way the search engine searches for keywords. That is, the search engine looks for pattern/string matches to terms associated with the desired tag information as specified (explicitly or implicitly) in a query. In one such example system, the SQE stores the relationship information that is extracted during the parsing and data object transformation process (the normalized data) in an annotated "term-clause matrix," which stores the terms of each clause along with "tagged terms," which include the syntactic and semantic information that embodies relationship information. Other example embodiments may provide different levels of organizing the enhanced indexing information, such as an annotated "term-sentence matrix" or an annotated "term-document matrix."
One skilled in the art will recognize that other variations of storage organization are possible, including that each matrix may be comprised of a plurality of other data structures or matrices.
Figure 2 is an example block diagram that conceptually represents a term-clause matrix that stores terms and enhanced indexing information for syntactic searching. The term-clause matrix 201 is an inverted index of tagged terms. That is, the matrix is indexed by the terms of each clause of each sentence of each document and indicates which clauses contain which terms. The diagram is conceptual in that it doesn't imply that what is represented is stored in the SQE precisely in that matter.
Different implementations may store the term separate from its annotations and may be stored as a plurality of data structures that together comprise the term-clause index. For example, terms that correspond to a particular grammatical role, for example, a "subject" may be stored separately than terms that correspond to a different grammatical role, for example an "object." For example, in Figure 2, each row 202 is indexed by a (tagged) term, e.g., ".../COUNTRY/China_subj"
206, and each column, e.g., columns 203, 204, and 205, represents a clause and contains a value that represents the number of times (e.g., a word count) that the clause contains the indexed term. The diagram is conceptual in that it doesn't imply that what is represented is stored in the SQE precisely in that matter. Different implementations may store the term separate from its annotations and may be stored as a plurality of data structures that together comprise the term-clause index. For example, terms that correspond to a particular grammatical role, for example, a "subject" may be stored separately than terms that correspond to a different grammatical role, for example an "object."
For illustrative purposes, Figure 2 shows a partial term-clause index that corresponds to the text of a given Document D1 that includes:
The president of France visited the capital of China in 1948. From 1949 to 1960 China was in alliance with the Soviet Union, although this relationship was already under severe strain in the late 1950s." From 1972 China aligned itself with the US against perceived Soviet expansionism.
The portion shown corresponds to the second and third sentences of the text, which together contain three clauses. (The indexing of the first clause is not shown.) The rows 202 each contain a term from one of these clauses, tag information that has been associated with the term during the data object parsing and transformation phase, and an indication of whether the clause contains the term in the role that is indicated by the associated tag information.
That is, the terms are annotated with syntactic (e.g., grammatical role) and semantic (e.g., entity/ontology tag) information. For example, the tagged term "(ontology root node)/ENTITY/LOCATION/COUNTRY/China_subj" 206 consists of the term from the associated text "China," a grammatical role tag "subj"
that indicates use of the term "China" as a subject, and an ontology path to the an entity tag "COUNTRY," that indicates that the term "China" is known to have an entity type of "COUNTRY" as determined from an ontology, database, dictionary, or similar structure associated with the SQE. The string "(ontology root node)" is a placeholder in the figure for the real indicator (e.g., name) of the root node of whatever ontology is being used. Also, depending upon the particular ontology being used, there may be a series of different nodes that contain the type "COUNTRY" (other than "ENTITY/LOCATION") and the SQE is programmed to take multiple nodes into account, when ingesting the documents and when searching for terms/tags in a relationship query that may be ambiguously expressed. The tagged terms "(ontology root node)/ENTITY/LOCATION/COUNTRY/Soviet Union_obj"
207 and "(ontology root node)/ENTITY/LOCATION/COUNTRY/Soviet Union_prep"

associated with the same document term "Soviet Union" indicate that the term is present in the document in two different grammatical roles ¨ the first clause contains the term as an object and the third clause contains the term as a complement of a prepositional phrase. Note also that several linguistic normalizations have been performed during the data object transformation process to the normalized data. For example, the tense of the verb "was" has been changed to "be" (passive to active) and the verb phrase "was in alliance"

has been changed to the verb "ally" (verbalization).
Several additional aspects are also notable with respect to the conceptual term-clause index illustrated in Figure 2. The index illustrates the use of custom specified portions of an ontology. In this case, in order to add verb sense information for a set of verbs (i.e., group a set of verbs together), a "VERB" node that indicates different types of verb sense information has been added to the ontology. Additional ontology information could be configured by a system administrator, or, alternatively, a user interface for dynamically modifying the ontology could be provided. In the particular portion of the ontology shown, two verb senses "VERB CHANGE" and "VERB STATIVE" are present. When the SQE ingests a verb that has not been categorized by the ontology, the verb is simply added to the index without a semantic annotation, such as the verb "ally," which has been indexed as "ally_verb. The same is true for other terms that correspond to other parts of speech that have not been classified (yet) by the ontology. For example, the nouns "relationship," "strain" and "expansionism" have been indexed with syntactic annotations for their respective grammatical roles, but do not have any associated semantic (ontology path) annotations. One skilled in the art will recognize that a variety of combinations could be represented in the term-clause index. Also note that the concepts of wildcard interpretation can be implemented a variety of ways, including explicitly putting "generic" nodes that correspond to particular types of wildcards (e.g., entity wildcards, physical_object wildcards, verb wildcards, etc.) depending upon the nodes in the ontology.

The integration of the enhanced indexing information into traditional search engine type document indexes (for example, an inverted index) is what supports the use a standard keyword search techniques to find a new type of document information ¨ that is, relationship information ¨ easily and quickly. An end user, such as a researcher, can pose simple Boolean style queries to the SQE yielding results that are based upon an approximation of the meaning of the indexed data objects. Because traditional search engines do not pay attention to the actual contents of the indexed information (they just perform string matching or pattern matching operations without regard to the meaning of the content), the SQE can store all kinds of relationship information in the indexed information and use a keyword search engine to quickly retrieve it.
The SQE processes each query by translating or transforming the query into component keyword searches that can be performed against the indexed data set using, for example, an "off-the-shelf" or existing keyword search engine. These searches are referred to herein for ease of description as keyword searches, keyword-style searches, or pattern matching or string matching searches, to emphasize their ability to match relationship information the same way search terms can be string- or pattern-matched against a data set using a keyword search engine. The SQE then combines the results from each keyword-style search into a cohesive whole that is presented to the user.

For example, suppose a researcher is attempting to discover something about China's relationships. In particular, suppose that the researcher would like to know China's attitude toward other countries. The researcher accordingly enters a relationship query to the SQE, for example, China_subj AND *_verb AND COUNTRY_obj (query 209) which instructs the SQE to find all clauses (sentences and/or documents) in which China is a source entity (used as a subject) along with any action (any verb) and a second entity of entity type "COUNTRY" is the recipient of the action. Note that the syntax of this query is a conceptual example of a specification of a relationship query using the SQE of the present invention.
The SQE will automatically determine that for this particular ontology the node "COUNTRY" is part of a full ontology pathname of "(ontology root node)/ENTITY/LOCATION/COUNTRY." Many different language specifications and user interfaces can be used to effectively communicate this same instruction to the SQE, and one skilled in the art will recognize that other alternatives are contemplated for use with the SQE. (The query specification matches the way the information is stored in the term-clause and other indexes.) Using the example term-clause index shown in Figure 2, the SQE
would respond with at least indicators to the second and third sentences of the Document D1 as they both contain clauses with the term "China" as the subject.
Moreover, the results returned indicate several different relationships, allowing the researcher quickly to discover a lot about China's foreign policy. For example, the following relationships would be quickly discovered:
China (is) ally of the Soviet Union China aligns itself with the United States which upon first glance may appear contradictory. By further drilling down to look at the returned clauses or sentences, the researcher can quickly discover that China's alliance with the Soviet Union ended in 1960.
In contrast to the term-clause index, the document index of a traditional keyword search engine system simply stores each term that is present in the document, along with an indication of the number of times the term appears in each document. Figure 3 is an example block diagram that conceptually represents a traditional term-document index. The term document index 301 includes rows indexed by the terms 302 of the document. Each column, for example columns 303-305, indicates the number of times the indexed term (in each row) appears in the document. In order to pose a query to find out the same information against this document index, the researcher needs to be much smarter about the content of the documents being searched, or, alternatively, willing to end up with a lot of potentially random information to search through. For example, the researcher could search for documents that contain "China" or documents that contain "China" and a list of alternative countries to look for. In any case, because much of the information concerning China's role in each document is lost when stored in this type of traditional document index, the results provided would tend to be less informative.
Figure 4 is an example block diagram of an example Syntactic Query Engine. A document administrator 402 adds and removes data sets (for example, sets of documents), which are indexed and stored within a data set repository 404 of the SQE 401. When used with keyword style searching techniques, the data set repository 404 stores an enhanced document index as described above. In the example shown in Figure 4, a subscriber 403 to a document service submits queries to the SQE 401, typically using a visual interface. The queries are then processed by the SQE 401 against the data sets indexed in the data set repository 404. The query results are then returned to the subscriber 403. In this example, the SQE 401 is shown implemented as part of a subscription document service, although one skilled in the art will recognize that the SQE may be made available in many other forms, including as a separate application/tool, integrated into other software or hardware, for example, cell phones, personal digital assistants ("PDA"), or handheld computers, or associated with other types of existing or yet to be defined services. Additionally, although the example embodiment is shown and described as processing data sets and queries that are in the English language, one skilled in the art will recognize that the SQE can be implemented to process data sets and queries in any language, or any combination of languages.
Figure 5 is an overview of the steps performed by a Syntactic Query Engine to process data sets and relationship queries. Steps 501-505 address the indexing (also known as the ingestion) process, and steps 506-509 address the query process. Note that although much of the discussion herein focuses on ingestion of an entire data set prior to searching, the SQE also handles incremental document ingestion and is described below with respect to an example embodiment of the SQE architecture. Also, the configuration process that permits an administrator to set up ontologies, dictionaries, sizing preferences for indexes and other configuration and processing parameters is not shown.
Specifically, in step 501, the SQE receives a data set, for example, a set of documents. The documents may be received electronically, scanned in, or communicated by any reasonable means. In step 502, the SQE
preprocesses the data set to ensure a consistent data format. In step 503, the SQE parses the data set, identifying entity type tags and the syntax and grammatical roles of terms within the data set as appropriate to the configured parsing level. For the purpose of extending keyword searching to syntactically and semantically annotated data, parsing sufficient to determine at least the subject, object, and verb of each clause is desirable to perform syntactic searches in relationship queries. However, one skilled in the art will recognize that subsets of the capabilities of the SQE could be provided in trade for shorter corpus ingestion times if full syntactic searching is not desired. For example, as described in U.S. Patent Publication No. 2003/0233224 (U.S. Patent Application No. 10/371,399), the parsing level may be configured using a range of parsing levels, from "deep" parsing to "shallow" parsing. Deep parsing decomposes a data object into syntactic and grammatical units using sophisticated syntactic and grammatical roles and heuristics. Shallow parsing decomposes a data object to recognize "attributes" of a portion or all of a data object (e.g., a sentence, clause, etc), such as entity types specified by a default or custom ontology associated with the corpus or the SQE. In step 504, the SQE
transforms the each parsed clause (or sentence) into normalized data by applying various linguistic normalizations and transformations to map complex linguistic constructs into equivalent structures. Linguistic normalizations include lexical normalizations (e.g., synonyms), syntactic normalizations (e.g., verbalization), and semantic normalizations (e.g., reducing different sentence styles to a standard form). These heuristics and rules are applied when ingesting documents and are important to determining how well the stored sentences eventually will be "understood" by the system.
For example, the SQE may apply one or more of transformational grammar rules, lexical normalizations (e.g., normalizing synonyms, acronyms, hypernyms, and hyponyms to canonical or standard terms), semantic modeling of actions (e.g., verb similarity), anaphora resolution (e.g., noun and pronoun coreferencing resolution) and multivariate statistical modeling of semantic attributes. Multivariate statistical modeling of semantic attributes refers to applying the techniques used to determine similar verbs to other parts of speech, such as nouns and adjectives. These techniques as applied to verbs include such determinations as the frequency weight of the primary sense of the verb; the set of troponyms associated to this verb sense (other ways to perform this verb, e.g., "sweep," "carry," and "prevail" are all troponyms of the verb "win" because they express ways to win); the set of hypernyms associated to this verb sense (more generic classes of which this verb is a part, e.g., "win"
is one way to "gain," "get," or "acquire"); and the set of entailments associated with this verb sense (other verbs that must be done before this verb sense can be done, e.g., "winning" entails "competing," "trying," "attempting,"
"contending,"
etc.). The ability to transform a term to alternatives so that similar actions and entities will also be searched for provides one important way to increase the ability of the SQE to "understand" a search query and retrieve more relevant results. Many transformational grammar rules also can be incorporated into the SQE. The transformational grammar rules may take many forms, including, for example, noun, pronoun, adjective, and adverb verbalization transformations.
Verbalization rules convert the designated part of speech to a verb. For example, the clause "X is a producer of Tungsten" can be simplified to the clause "X produces Tungsten." Another example transformation rule is to simplify a clause by changing it from passive to active voice. For example, the clause "the chart was created by Y" can be transformed to the clause "Y
created the chart."
In step 505, the SQE stores the parsed and transformed sentences in a data set repository. As described above, when the SQE is used with a keyword search engine, the normalized data is stored in (used to populate) an enhanced document index such as the term-clause matrix shown in Figure 2. After storing the data set, the SQE can process relationship queries against the data set. In step 506, the SQE receives a relationship query, for example, through a user interface such as that shown in Figures 6A-6G below.
Alternatively, one skilled in the art will recognize that the query may be transmitted through a function call, batch process, or translated from some other type of interface. In step 507, if necessary (depending upon the interface) the SQE preprocesses the received relation query and transforms it into the relationship query language understood by the system. For example, if natural language queries are supported, then the natural language query is parsed into syntactic units with grammatical roles, and the relevant entity and action terms are transformed into the query language formulations understood by the SQE.
In step 508, the SQE executes the received query against the data set stored in the data set repository. The SQE transforms the query internally into sub-queries as appropriate to the organization of the data in the indexes and executes a traditional keyword search engine (or its own version of keyword style searching) to process the query. In step 509, the SQE returns the results of the relationship query, for example, by displaying them through a user interface such as the summary information shown in Figure 6B.
One skilled in the art will recognize that, although the techniques are described primarily with reference to text-based languages and collections of documents, similar techniques may be applied to any collection of terms, phrases, units, images, or other objects that can be represented in syntactical units and that follow a grammar that defines and assigns roles to the syntactical units, even if the data object may not traditionally be thought of in that fashion.
Examples include written or spoken languages, for example, English or French, computer programming languages, graphical images, bitmaps, music, video data, and audio data. Sentences that comprise multiple words are only one example of a phrase or collection of terms that can be analyzed, indexed, and searched using the techniques described herein. One skilled in the art will recognize how to modify the structures and program flow exemplified herein to account for differences in types of data being indexed and retrieved.
Essentially, the concepts and techniques described are applicable to any environment where the keyword style searching is contemplated.
Also, although certain terms are used primarily herein, one skilled in the art will recognize that other terms could be used interchangeably to yield equivalent embodiments and examples. In addition, terms may have alternate spellings which may or may not be explicitly mentioned, and one skilled in the art will recognize that all such variations of terms are intended to be included.
Also, when referring to various data, aspects, or elements in the alternative, the term "or" is used in its plain English sense, unless otherwise specified, to mean one or more of the listed alternatives. For example, the terms "matrix" and "index" are used interchangeably and are not meant to imply a particular storage implementation. Also, a document may be a single term, clause, sentence, or paragraph or a collection of one or more such objects.
For example, the term "query" is used herein to include any form of specifying a desired relationship query, including a specialized syntax for entering query information, a menu driven interface, a graphical interface, a natural language query, batch query processing, or any other input (including API function calls) that can be transformed into a Boolean expression of terms and annotated terms. Annotated terms are terms associated with syntactic or semantic tag information, and are equivalently referred to as "tagged terms."
Semantic tags include, for example, indicators to a particular node or path in an ontology or other classification hierarchy. "Entity tags" are examples of one type of semantic tag that points, for example, to a type of ENTITY node in an ontology. In addition, although the description is oriented towards parsing and maintaining information at the clause level, it is to be understood that the SQE
is able to parse and maintain information in larger units, such as sentences, paragraphs, sections, chapters, documents, etc., and the routines and data structures are modified accordingly. Thus, for ease of description, the techniques are described as they are applied to a term-clause matrix. One skilled in the art will recognize that these techniques can be equivalently applied to a term-sentence matrix and a term-document matrix.
In the following description, numerous specific details are set forth, such as data formats and code sequences, etc., in order to provide a . thorough understanding of the techniques of the methods and systems of the present invention. One skilled in the art will recognize, however, that the present invention also can be practiced without some of the specific details described herein, or with other specific details, such as changes with respect to the ordering of the code flow.
The Syntactic Query Engine is useful in a multitude of scenarios that require indexing, storage, and/or searching of, especially large, data sets, because it yields results to queries that are more contextually accurate than other search engines. An extensive relationship query language ("RQL") is supported by the SQE. The query language is designed to be used with any SQE implementation that is capable of retrieving relationship information from an indexed data set, regardless of whether the SQE uses a relational database implementation with a proprietary search engine or an enhanced document index that supports a keyword search engine. However, some of the operators may be more easily implemented in one environment versus the other, or may not be available in certain situations. One skilled in the art will recognize that variants of the query language are easily incorporated and that other symbols can be equivalently substituted for operators.
In general, the syntax for a relationship query specifies "entities"
and "actions" that are linked via a series of "operators" with one or more constraints such as document level filters.
Entity: An Entity is a noun or noun phrase in the search query or result. It can be the source (initiator of an action), the target (receiver of an action), or the complement of a prepositional phrase. Entities can be multiple words. If they are quoted, the exact phrase is preferably matched by a phrase in a document being searched. Either double quotes or single quotes may be used; if double quotes are used, then synonyms of the quoted expression will not be included in a search. If single quotes are used, synonyms of the quoted expression will be included. Synonyms are typically specified as properties of an ontology related to the corpus or in a dictionary.
Source: The initiator of an action is referred to as the source. For example, in the query [Country] > threaten > USA, "Country" is the source. The query instructs a search for all countries that threaten the US, but not all countries that the US threatens.
Target: The receiver of an action is referred to as the target. For example, in the query USA > investigates > [organization]
"organization" is the target of the action. The query instructs a search for all political organizations that are the target of an investigation, but not those that are initiating an investigation.
Prepositional Complement: An action is often performed with a prepositional complement. For example, in the query Maya > visit > grandmother PREP
CONTAINS Tuesday "Tuesday" is the prepositional complement of the sentence. The query instructs a search for only visits that happened on Tuesdays.
Action: All relationships are based on an action, or verb. For example, in the query Maya > visit > grandmother "visit" is the action.
Operators: The following example operators are supported:
= Action directionality for events: <, >, <> (or alternatively <-, ->, <->) = Boolean: AND, OR, NOT. The default operation for omitted Boolean operators is OR. Booleans do not have to be uppercase.
= Prepositional constraint: PREP CONTAINS (upper or lowercase), or '^' = Document keyword constraint: DOCUMENT
CONTAINS (upper or lowercase), or ';' = Metadata constraint: METADATA CONTAINS (upper or lowercase), or '#' = Wildcards (not within quotes): *, ? (single and multi-character) = Offset indicators: ¨

= Curly braces { } are used for indirect link searches, to search for entities that link other entities together = Brackets H are used to denote types, either an OntologyPath, or, if used with a verb, an ActionType.
Parenthesis can be used to nest portions of the query.
The general format for a relationship query comprises four components:
Syntactic query A Prep constraints; Document keyword constraints # Metadata constraints The syntactic query component is specified in the format Source Entity >
Action > Target Entity. However, it is not necessary to specify all three components, nor do the directional arrows need to point to the right. For example, Bush <*
Bush < * < * =
> * > Bush are all correct specifications of the entity "Bush" as he related to other entities through any action, and there is no difference between the first two or the last two. Although both actions and entities can be represented by a wildcard, the position of the wildcard in the query determines what it represents. Entities preferably do not point to each other directly.
In addition to the basic syntactic search component of the query, there are three optional components that can be added to filter results (constrain the search):
= any prepositional constraints, to filter results by information found in a prepositional phrase;
= any document keyword constraints, to restrict search to documents that have certain keyword(s); (this causes a basic keyword search) = any metadata constraints, to restrict search to documents tagged with specific metadata values or ranges or values.
These clauses can be expressed in either a long or abbreviated format. In the long format, the clauses are separated by the self-explanatory terms "PREP
CONTAINS", "DOCUMENT CONTAINS" and "METADATA CONTAINS". For example, broken up into several lines for easier reading, the relationship query:

Bush > visit > [Country] AND NOT China PREP CONTAINS plane DOCUMENT CONTAINS "foreign service" OR
diplomat METADATA CONTAINS Date>04/2002 specifies a syntactic search for "visit" relationships between the entity "Bush"
and any country except China. The relationship query is constrained by the preposition "plane", meaning that the word plane must be included in a prepositional phrase within this relationship, indicating travel by plane. The query is further constrained by the document keywords/key phrases "foreign service" and "diplomat," meaning that only relationships from documents containing these words should be returned. Finally, the search is constrained by a date range, and instructs the search engine to only search documents written after April 2002. (This assumes that date related metadata has been associated with the documents at time of data set ingestion.) Date and numeric metadata ranges are specified with "=", ">", "<", ">=", and "<=". Put together, this query searches specifically for diplomatic trips that Bush took by plane since April 2002 to foreign countries with the exception of China.
Note that there are two expressions designated in the document filter above: "foreign service" and "diplomat." When a document contains a keyword in adjective form, e.g., "diplomatic," the document is included in the search results responsive to a query that designated the noun form. The SQE
may be configured to automatically extract the stem of the word and search for other forms. Document level queries are also allowed by specifying a keyword or phrase (even without a syntactic search component). For example:
germany AND france AND england will cause the SQE to search for all documents containing these keywords.
Filter clauses (L e., constraint components) can also be entered in a more abbreviated form, in which the terms "PREP CONTAINS", "DOCUMENT
CONTAINS", and "METADATA CONTAINS" are replaced by a '"', ';' and a '#' character respectively, as in:
Syntactic query A Prep constraints; Document keyword constraints #
Metadata constraints The example relationship query described above regarding diplomatic trips that Bush took by plane can be rewritten in abbreviated form as follows:

Bush > visit > [Country] AND NOT China"plane; "foreign service" OR
diplomat # Date>04/2002 Also note that multiple Metadata constraints can be used with complete Boolean expressions and that Boolean expressions can be nested. For example, the query hamas > act >* METADATA CONTAINS Author="Andrew Jackson" OR price=300 and the query england AND NOT (aerospace OR airways) >abandon >
describe valid relationship queries.
RQL formulated queries can also be embedded within a scripting language to provide an ability to execute batch relationship queries, functions having multiple queries, and control flow statements. For example, it may be desirable to encode a query to be executed at certain times each day against a data set that is continually updated and incrementally ingested. One skilled in the art will recognize that many scripting languages could be defined to achieve control flow of multiple relationship queries, and that the scripting language could include conditional statements.Relationship queries formulated using RQL are submitted to the SQE for execution from a variety of interfaces. For example, a web-based interface, similar to that provided by default with the InFact products, can be used to submit relationship queries. In addition, relationship queries can be submitted using a natural language interface to the SQE, which parses the natural language query into syntactic units that can be translated into an RQL formulated query and then executed. Alternatively, the SQE supports an API that allows the development of other code, such as other user interfaces, that can execute relationship queries by submitting RQL
formulated query strings to the SQE. Figures 11A-11F described below exemplify one such interface that provides a more graphical use of relationship queries.
Figures 6A-6G, 7A-7F, and 8A-8F are example screen displays from an example embodiment of a user interface designed to provide relationship and event searching in accordance with the techniques of the present invention. These screen displays emphasize particular features of a query language that has been designed to take advantage of combining the attributes of keyword style searching with syntactic searching. Additional examples of this user interface, query language, and variants thereof are included in Appendices A and B.
Figures 6A-6G are example screen displays that illustrate the general capabilities of the example user interface and the types of queries that can be executed by an example Syntactic Query Engine. Figure 6A is an example initial screen display of a web-based interface for entering a relationship query to the SQE. There are five basic components of this example interface. Pressing the Search tab 6A03 displays (or generates) the page used to enter queries. The user enters an RQL formulated query into free text field 6A01. When ready, a search is initiated by pressing the Search button 6A02. Alternatively, users can enter RQL syntax using a "form" or template.
The Show Query Generator link 6A08 navigates to this alternative interface to build an RQL formulated query. This interface is described further below with respect to Figure BF. Pressing the Corpus tab 604 displays a page used to browse available ontologies, find out more information for a particular ontology path, browse available metadata, and find synonyms that are configured in the system. These capabilities are described further below with respect to Figures 8A-8F. Pressing the Preferences tab 6A05 displays a page used to set search preferences. These capabilities are described further below with respect to Figure 9. Pressing the History tab 6A06 displays a page that shows a history of prior relationship searches. The history page is described further below with respect to Figure 10. Pressing the Help tab displays a web page(s) of tutorial information and assistance. An example help file is included as Appendix A.
Figure 6B is an example screen display of the format for displaying results in response to a relationship query specified using the relationship query language. The query is entered in query input field 61301, and in this case indicates a search for everything that China buys ("china >
buy > *"). A summary of the results of the search is displayed in result area 6B00.
Note that each "row", for example row 61302, represents a particular relationship that is discovered in the corpus. Instances of this relationship may be actually located in more than one sentence or document. Thus, the Action field indicates a count of the number of times the particular relationship occurs in the data currently being displayed and summarized. For example, the first row 6602 indicates that at least 2 instances of China buying (U.S.) wheat exist in the corpus. In one embodiment, the data is "chunked" prior to display. Thus, when used with chunked data, the number of instances of a particular event/relationship is valid only to what is being displayed. Other embodiments that calculate the entire result prior to display may indicate the number of instances a relationship appears over the entire corpus.
Figure 6C is an example screen display of a more complex query that includes a Boolean operator and a document level filter. The query specified in query input field 6C01 includes two Boolean operators in a Boolean expression, "suicide AND (attack OR bombing)" as part of the syntactic search specification and includes a document level filter. Specifically, the user has specified a relationship search that will assist the user to discover all suicide attacks that have killed people in Israel. The results are shown summarized in result area 6000. Clicking on any one of the actions, for example, "kill [5]"
labeled as action 6CO2, will cause the SQE to display the five instances in the clauses/sentences/documents in which the corresponding relationship is found.
Figure 6D is an example screen display of a link search using an entity type. The query specified in query input field 6D01 instructs the SQE
to search for all people or named persons that link Bush and Thatcher. The results displayed in result area 6D00 show each 3rd person that provides a link between Bush and Thatcher. That is, the 3rd person has some relationship to Bush and has some (possibly separate) relationship to Thatcher. To discover the details of these relationships, the user navigates to one of the displayed links such as link 6D02 which indicates that Ronald Reagan is the person in common in the indicated (indirect) relationship.
Figure 6E is an example screen display of a search that specifies an entity type and an action type. The query specified in query input field instructs the SQE to search for all events in which the Pope took some action involving motion (e.g., driving) to some location. As can be seen in the results displayed in result area 6E00, a variety of actions, sorted by similarity using the sort button 6E02, are displayed. Note also, that a nested search button 6E03 can be pressed to cause the next query to be applied to the results from the prior query. This supports an iterative discovery process where a user progressively narrows a search based upon relationship information received at each search level.
Figure 6F is an example screen display of a search that specifies ontology paths in conjunction with a prepositional constraint. The query specified in query input field 6F01 instructs the SQE to search for all corporate acquisitions, specifically as they relate to the amount of money spent. The prepositional constraint specified by "A money" indicates that some amount of money needs be present in a prepositional phrase of each matching clause.

For example, the results shown in result area 6F00 show a first relationship with a target entity 6F02 in which a sawmill was bought for $2.7 million.
Similarly, the results show a second relationship where the preposition phrase that included the money is associated with the action "buy" labeled 6F03.
The ontology path specified in the query, "[organization/name]" is defined by an ontology associated with the system. Ontologies are typically associated with a corpus at system configuration time, although one skilled in the art will recognize that they can be dynamically changed and the portions of the corpus that are affected by the change, re-ingested. An ontology can be a default ontology associated with the SQE or a custom ontology generated for a specific corpus. Ontology paths are enclosed in brackets, as in [person] or [country]. If a bracketed term is found in a relationship query, the SQE
searches the ontology[ies] for all paths matching the term. If there are multiple matches, all matches are included in the search and results are combined. For example, in a search query containing the type [person], the SQE will substitute with [IF/Entity/Person] to indicate use of the default ontology provided with the system. If another path exists in a custom ontology such as "MyOntology/People/Person," this path is also included in the query and the results are combined. Ontology paths can be browsed through an interface provided under the "Corpus" tab, as described further below with respect to Figures 8A-8F.
Figure 6G is an example screen display of the query generator interface. The form displayed in display area 6G00 is provided to assist a user with specifying the components of a relationship query without needing intimate knowledge of the RQL syntax. The fields are labeled accordingly to explain what the user can enter to create a proper RQL formulated query.
Figures 7A-7F are example display screens of the progression of an example RQL query submitted to a Syntactic Query Engine. In Figure 7A, the user submits a query "s6 kinase <>* <>*" in query input field 7A01. When the user presses the Search button 7A02, the SQE displays results in chunked pages of relationship summary information as shown in Figure 7B. Note that the results shown in Figure 7B include relationships that have "s6 kinase" as a subject, e.g., row 71303, and relationships that have "s6 kinase" as an object, e.g., row 7604. By clicking on one of the displayed actions, for example the "abolish" action 7C01 in Figure 7C, the user can navigate to the document (sentence or clause) that shows that relationship. Figure 7D is an example screen display of a document that has been navigated to by selecting an action link in a displayed relationship summary. The highlighted portion (L e., shown as boxed herein) of the document text 7D01 is the information that has been summarized in the search results displayed in Figure 7C. Figure 7E is an example screen display that illustrates how the user might then go back and modify the query based upon information gleaned while drilling down a particular search. In this case, based upon the actions retrieved in the highest level search, the user has decided to drill down and look at "s6kinase" as it blocks or regulates some other entity. Figure 7F is an example screen display that illustrates that the SQE retrieves relationships having similar verbs to the verb sense specified in the query. In this case, the verb "modulate" is searched for as a similar verb to the user specified verb "regulate."
Figures 8A-8F are example screen displays of an interface associated with browsing ontology paths, viewing corpus metadata, and finding synonyms. Figure 8A is an example screen display of navigation used to browse a default ontology path. When a user types a path specification into path input field 8A01 and presses the Find Ontology Paths button 8A02, then the corresponding additional subpaths are displayed in area 8A03. The user can select the "Show Roots" link 8A04 to show the roots of other ontologies available for that particular corpus. Note that an ontology typically includes a hierarchical classification system (a taxonomy) as well as properties associated with the nodes of the ontology and a dictionary.
Figures 8B-8F are example screen displays from a different version of the user interface, and are provided herein to illustrate how different ontologies may be associated with a single corpus. In Figure 8B, several links to root nodes 8602 are displayed. The user can either select one of these nodes and begin browsing or type a specific path into path input field 8601.
In the example shown, the user selects the path "LocusLink" and browses a hierarchy (not shown) by selecting a next node on the path labeled "Gene".
The next ontology level below "Gene" is displayed in subpath area 8CO3 of Figure 8C. Note that according to this version of the interface, available metadata for the corpus is displayed in metadata display area 8C04. Figure 8D
is an example screen display of an interface used to search for synonyms.
Synonyms for a word specified in input field 8D01 are displayed in synonym display area 8D02. Other interfaces may provide links or other user interface components for navigating to the metadata and synonym information. Figures 8E and 8F illustrate the behavior of the interface when the user inputs a specific entity classification into path input field 8E01. In this case, when the user types in the term "steroids," the SQE responds by displaying indications 8F02 of all ontology paths that contain the entity type "steroids."
Figure 9 is an example screen display of an interface associated with setting preferences for constraining relationship searches. There are a number of preference settings associated with a given search that may be customized to constrain search results or improve result display. The following options are illustrated on the Preferences page, and one skilled in the art will recognize that other options can be provided:
= Include negated actions: When this option is enabled, relationships matching both the positive and negative sense of a verb are displayed. If a user performed a search such as "Clinton > visit > Russia", the sentence "Due to heath reasons Clinton did not visit Russia." would only be returned if this setting was set to true. By default Show Negated Actions is disabled, and only positive actions are displayed.
= Search modifiers along with entities: This option specifies whether modifiers should be searched along with sources and/or targets (as subjects and/or objects). In the above example sentence "Bill visits beautiful, green pastures outside Seattle," if this property is set to true, then a search such as "Bill > visit > Seattle" will return the above relationship. If this property is false, then it will not, and only the query "Bill > visit > pasture" would still yield this result.
= Display modifiers: In the sentence "Bill visits beautiful, green pastures outside Seattle," "beautiful, green" is the prefix modifier for pastures, and "outside Seattle" is the posffix modifier. In a search like "Bill > visit > *, with this property set to true the SQE will display modifiers along with pastures in the target entity summary. If this property is set to false, only the word 'pastures' will be displayed as the target in the tabular display.
= Enforce strict bi-directionality: When doing searches with bi-directional arrows, such as "<>", the search can be interpreted in two different ways. For example, with the search query "Clinton <> * <> Bush", one might wish only to view results in which Bush did something to Clinton XOR Clinton did something to Bush. (XOR indicates an exclusive Boolean OR
operation.) Enforcing strict bi-directionality provides this result.
However, one might also wish to see instances in which Bush and Clinton both did something to some other target together.
These additional results are displayed if strict bi-directionality is not enforced.
= Search ontology path name as term: If a user includes an ontology path like Icityl" in a search query, then results with cities are returned. However, the word "city" is not an instance of an item in the ontology itself, and is not associated with the ontology path. Therefore, without setting this preference, one would not see results that contain the word "city." This preference is set to true to include results with the term "city" in them as well as any terms defined by the ontology path "city."
= Number of relationships per page: The user can set the number of relationships to display on a single page of relationship results. The smaller this value, the faster results will be returned.
= Number of documents per page: The user can set the number of documents to display on a single page of document results. The smaller this value, the faster results will be returned.
= Sort scheme: This setting allows users to sort results in a given chunk or batch of results according to one of several sorting schemes, and to set the default sort scheme for all future searches. Note that an individual result set can also be sorted in the result display. If results are sorted using the drop-down selection box on the results page, the setting does not persist for subsequent searches.
= Surrounding sentences to export: This option allows the user to vary how much contextual information from the document is included along with the sentences returned when the user exports a result set to HTML.

Figure 10 is an example screen display of an interface associated with displaying SQE query history. The history page displays a history queue 1000 of all searches performed in the current browser session. If the browser dies, if you use another browser, or if you press the Clear button 1010, the history queue 1000 is reset. Clinking on one of links 1001-1002 for any query in the Query column will navigate to the results page for that particular query.
Clinking on one of the links 1003-1004 in the Documents column will navigate to the set of documents that contain the results of that query. The "Depends On" column 1005 indicates whether a given query depends on a previous query, for example as a result of executing a nested search.
Figures 11A-11F are example screen displays from an alternate graphical based interlace for displaying and discovering genetic relationships.
This interface could be generated, for example, using an API supported by the SQE. One skilled in the art will recognize that many different APIs can be provided to support accessing the functions of an SQE from other code. In Figure 11A, the user can select possible files that correspond to various sets of genes that can be studied to discover relationships between them. In Figure 11B, the user indicates a desire to select the entity list to be displayed. In Figure 11C, the user selects the "genes3.txt" file as the entity file to be displayed. In Figure 11D, the user (optionally) selects an action list file, for displaying specific types of relationships (based upon verbs). Figure 11E and 11F show the results of the relationships between selected genes. Each dot represents a different gene and each line between two genes represents a relationship evidenced by the corpus. Selecting two genes in the graphical user interface results in the specification of an RQL formulated query to the SQE.
Figure 11E illustrates the results of selecting two of the genes in order to display the specific relationships between them. In this case the user has selected the iqgap1 gene 11F03 and the q02248 gene 11E03 and the possible "actions"
between them are displayed in relationship results area 11E01. In this case, the relationships include "interactions," "regulation," and "localization." At this point, the user has gained information for further follow up. In Figure 11F, two different genes (entities) 11F02 and 1103 are selected to display relationships between them. The actions between them are displayed in relationship results area 11F01. Note that the relationship query invokes a search for both genes as source and target in this example.
An SQE as described may perform multiple functions (e.g., data set parsing, data set storage, query transformation and processing, and displaying results) and typically comprises a plurality of components. Figure is a conceptual block diagram of the components of an example embodiment of a Syntactic Query Engine. A Syntactic Query Engine 1201 comprises a Relationship Query Processor 1210, a Data Set Preprocessor 1203, a Data Set Indexer 1207, an Enhanced Natural Language Parser ("ENLP") 1204, a data set repository 1208, and, in some embodiments, a user interface (or an Applications Programming Interface "API") 1313. The Data Set Preprocessor 1203 converts received data sets 1202 to a format that the Enhanced Natural Language Parser 1204 recognizes. The Enhanced Natural Language Parser ("ENLP") 1204, parses the preprocessed sentences, identifying the syntax and grammatical role of each meaningful term in the sentence and the ways in which the terms are related to one another and/or identifies designated entity and other ontology tag types and their associated values, and transforms the sentences into a canonical form ¨ a normalized data representation. The Data Set Indexer 1207 indexes the normalized data into the enhanced document indexes and stores them in the data set repository 1208. The Relationship Query Processor 1210 receives relationship queries and transforms them into a format that the Keyword Search Engine 1211 recognizes and can execute.
(Recall that the Keyword Search Engine 1211 may be an external or 3rd party keyword search engine that the SQE calls to execute queries.) The Keyword Search Engine 1211 generates and executes keyword searches (as Boolean expressions of keywords) against the data set that is indexed and stored in the data set repository 1208. The Keyword Search Engine 1211 returns the search results through the user interface/API 1213 to the requester as Query Results 1212.
In operation, the SQE 1201 receives as input a data set 1202 to be indexed and stored. The Data Set Preprocessor 1203 prepares the data set for parsing by assigning a Document ID to each document that is part of the received data set (and sentence and clause IDs as appropriate), performing OCR processing on any non-textual entities that are part of the received data set, and formatting each sentence according to the ENLP format requirements.
The Enhanced Natural Language Parser ("ENLP") 1204 parses the data set, identifying for each sentence, a set of terms, each term's tags, including potentially part of speech and associated grammatical role tags and any associated entity tags or ontology path information, and transforms this data into normalized data. The Data Set Indexer 1207 indexes and stores the normalized data output from the ENLP in the data set repository 1208. The data set repository 1208 represents whatever type of storage along with the techniques used to store the enhanced document indexes. For example, the indexes may be stored as sparse matrix data structures, flat files, etc. and reflect whatever format corresponds to the input format expected by the keyword search engine. After a data set is indexed, a Relationship Query 1209 may be submitted to the SQE 1201 for processing. The Relationship Query Processor 1210 prepares the query for parsing, for example by splitting the Relationship Query 1209 into sub-queries that are executable directly by the Keyword Search Engine 1211. As explained above, a Relationship Query 1209 is typically comprised of a syntactic search along with optional constraint expressions. Also, different system configuration parameters can be defined that influence and instruct the SQE to search using particular rules, for example, to include synonyms, related verbs, etc. Thus, the Relationship Query Processor 1210 is responsible for augmenting the specified Relationship Query 1209 in accordance with the current SQE configured parameters. To do so, the Relationship Query Processor 1210 may access the ontology information which may be stored in Data Set Repository 1208 or some other data repository. The Relationship Query Processor 1210 splits up the query into a set of Boolean expression searches that are executed by the Keyword Search engine 1211 and causes the searches to be executed. The Relationship Query Processor 1210 then receives the result of each search from the Keyword Search Engine 1211 and combines them as indicated in the original Relationship Query 1209 (for example, using Boolean operators). One skilled in the art will recognize that the Relationship Query Processor 1210 may be comprised of multiple subcomponents that each execute a portion of the work required to preprocess and execute a relationship query and combine the results for presentation. The results (in portions or as required) are sent to the User Interface/API component 1213 to produce the overall Query Result 1212.
The User Interface Component 1213 may interface to a user in a manner similar to that shown in the display screens of Figures 6A-6G and 7A-7F.
Figure 13 is a block diagram of the components of an Enhanced Natural Language Parser of an example embodiment of a Syntactic Query Engine. The Enhanced Natural Language Parser ("ENLP") 1301 comprises a natural language parser 1302 and a postprocessor 1303. The natural language parser 1302 identifies, for each sentence it receives as input, the part of speech for each term in the sentence and syntactic relationships between the terms each clause of the sentence. An SQE may be implemented by integrating a proprietary natural language parser into the ENLP, or by integrating an existing off-the-shelf natural language parser. The postprocessor 1303 examines the natural language parser 1302 output and, from the identified parts of speech and syntactic relationships, determines the grammatical role played by each term in the sentence and the grammatical relationships between those terms.
When entity tags or other types of semantic tags (indicating nodes in an ontology path) are used in addition to or in lieu of the grammatical relationships, the postprocessor 1303 (or the natural language parser 1302 if capable of recognizing such tags) identifies, for each sentence (or clause where relevant), each semantic tag type and its value. For example, the term "China" could be recognized as an entity type of "COUNTRY" having the (fully specified) ontology path indicator of "IF/ENTITY/LOCATION/COUNTRY." The postprocessor 1303 then generates an enhanced data representation from the determined tags, including the entity tags, other ontology node tags, grammatical roles, and syntactic and grammatical relationships.
Figure 14 is a block diagram of the processing performed by an example Enhanced Natural Language Parser. During document ingestion, the natural language parser 1401 receives a sentence 1403 (or portion thereof) as input, and generates a syntactic structure, such as parse tree 1404. The generated parse tree 1404 identifies the part of speech for each term in each clause of the sentence and describes the relative positions of the terms within the clause. In embodiments that support the recognition of entity tags or other types of ontology path information, the parser 1401 (or postprocessor 1402 if the parser is not capable) also identifies in the parse tree (not shown) the semantic tag type for each corresponding term in the sentence. The postprocessor 1402 receives the generated parse tree 1404 as input, determines the grammatical role of each term in the clause and relationships between terms in the clause, and generates a normalized version of the sentence data annotated with the grammatical role tags (syntactic tags) and semantic tags 1405.
Figure 15 is a block diagram illustrating a graphical representation of an example syntactic structure generated by the natural language parser component of an Enhanced Natural Language Parser. The parse tree shown is one example of a representation that may be generated by a natural language parser. The techniques of the methods and systems of the present invention, implemented in this example in the postprocessor component of the ENLP, enhance the representation generated by the natural language processor by determining the grammatical role of each meaningful term, associating these terms with their determined roles and determining relationships between terms.

In embodiments in which the natural language parser cannot support the recognition of semantic tags, one skilled in the art will recognize that the postprocessor component (such as Postprocessor 1303 in Figure 13) can be programmed to enhance the representation with such tags. In Figure 15, the top node 1501 represents the entire sentence, "The president of France visited the capital of China in 1948." Nodes 1502 and 1503 identify the noun phrase of the sentence, "The president of France," and the verb phrase of the sentence, "visited the capital of China in 1948," respectively. The branches of nodes or leaves in the parse tree represent the parts of the sentence further divided until, at each leaf level, each term is singled out and associated with a part of speech. A configurable list of words are ignored by the parser as "stopwords."

The stopword list comprises words that are deemed not indicative of the information being sought. Example stopwords are "a," "the," "and," "or," and "but." In one embodiment, question words (e.g., "who," "what," "where,"
"when,"
"why," "how," and "does") are also ignored by the parser. In this example, after ignoring the determinant "The" (node 1504), nodes 1508 and 1509 identify the noun phrase 1505 as comprising a noun, "president" and a prepositional phrase, "of France." Nodes 1512 and 1513 divide the prepositional phrase 1509 into a preposition, "of," and a noun, "France." Nodes 1506 and 1507 divide the verb phrase 1503 into a verb, "visit," (morphological form of "visited") and a noun phrase, "the capital of China in 1948." Nodes 1510 and 1511 divide the noun phrase 1507 ultimately after several additional steps into a determinant "The" (node 1514), which may be ignored as a stopword; a noun "capital" (node 1515); a preposition "of' (node 1518); a noun "China" (node 1519); a preposition "in" (node 1520); and a noun "1948" (node 1521).
Figure 16 is a table that conceptually illustrates normalized data that has been annotated with syntactic and semantic tags by the postprocessor component of an Enhanced Natural Language Parser. Depending upon the implementation of the ENLP, the normalized data may or may not be stored in an intermediate data structure prior to being indexed and stored in the enhanced document indexes, such as the term-clause index. The example normalized data representation illustrates annotations applied to the sentence that was illustrated in the parse tree of Figure 15. The annotations are of course dependent upon the ontology root node specified (which in this case is a default ontology root node called "IF") and whether the SQE has been configured to parse with semantic tags. Also, one skilled in the art will recognize that the selected roles and relationship information to be stored may be programmatically determined. In the example shown, row 1601 shows the indexing information for the term "president" and specifies that the term is associated with a grammatical role of "subject" and has been tagged as a type of person (relative to the ontology being used). The SQE also recognizes and maintains information that the subject of this clause is associated with a (suffix) modifier term "France," which has been tagged as a type of country. The SQE
maintains modifier information for subjects, objects, and prepositional phrases, because, in some configurations, the SQE can search for specified subject, object, and/or prepositional constraint terms in addition as modifiers, thereby returning documents that potentially may be relevant even though the sentence clauses didn't include the specified terms precisely as subjects, objects, or complement of a preposition. Row 1602 shows the indexing information for the term "visited" and specifies that the term is associated with the grammatical role of "verb." Note that the SQE stores the stemmed form of the verb "visit" so as to potentially match more forms of the verb. Other heuristics could be similarly incorporated. Row 1603 shows the indexing information for the term "capital,"
including that the term is tagged with a grammatical role of "object" and is associated with two suffix modifiers "China" and "1948," the first of which is tagged as a country (and a location and an entity) and the second of which is tagged as a date (and a numeric value and an entity). Note that these terms are maintained by the SQE as modifiers even though they are also maintained as prepositional complements for use in relationship queries that filter based upon prepositional constraints. Row 1604 shows the indexing information for the term "China," including that the term is tagged with a grammatical role of "prepositional complement" and a semantic tag that specifies that the term is a kind of date. Similarly, row 1605 shows the indexing information for the term "1948," including that the term is tagged with a grammatical role of "prepositional complement" and a semantic tag that specifies that the term is a kind of country (and location and entity). Row 1606 shows the additional sentence/clause information, which in this case is an indication that the clause is a "temporal" one. Clause and sentence information may indicate, for example, that the clause relative to other clauses in the sentence is a conditional clause, a causal clause, a prepositional clause, or a temporal clause or that the sentence is a question, a definition, or contains temporal or numerical information. One skilled in the art will recognize that other classifications of interclause relationships and of sentences may also be incorporated. Also, other linguistic heuristics can be used to generate enhanced indexing information indicated by the normalized data produced by the ENLP. For example, in some implementations, the ENLP provides "co-referencing" analysis, which allows the ENLP to replace pronouns with nouns, or nouns, pronoun phrases, noun phrases, aliases, abbreviations, acronyms, etc. with a corresponding identifying noun. This capability allows greater search accuracy, especially when searching for specific entity names.
Note that the normalized data shown in Figure 16 supports many different types of relationship queries. For example, all of the following relationship queries will cause the SQE to return an indicator to the sentence that has been normalized to the data of Figure 16 (assuming modifiers are searched):
* > visits > [country] (Query for information on all visits of all countries.) president <>* (Query for anything a president does.) *> *> China (Query for any relationship with China.) (Note that the SQE returns the sentence because it searches for "China" as a modifier instead of just as an object of the sentence.) *> *> [country] (Query for any relationship with any country.) France <>*<> China (Query for any relationship b/n France &
China.) (Note that the SQE returns the sentence because it searches for "France" and "China" as modifiers instead of just as subjects and/or objects of the sentence.) Thus, the normalized data demonstrated by Figure 16 is supportive of and responsive to a very flexible style of specifying relationship queries.
The Syntactic Query Engine performs two functions to accomplish effective relationship query processing with syntactic searching capabilities.
The first is the parsing, indexing, and storage of a data set (sometimes termed corpus ingestion). The second is the query processing, which according to the example embodiment described herein, results in the execution of keyword searches. These two functions are outlined with reference to Figures 17-19.

Figure 17 is an example block diagram of data set processing performed by a Syntactic Query Engine. As an example, documents that make up a data set 1701 are submitted to the Data Set Preprocessor 1702 (e.g., component 1203 in Figure 12). If the data set comprises multiple files, as shown in Figure 17, in one embodiment the Data Set Preprocessor 1702 creates one tagged file containing the document set. The Data Set Preprocessor 1702 then dissects that file into individual sentences and sends each sentence to the ENLP 1704 (e.g., component 1204 in Figure 12). After the ENLP 1704 parses each received sentence, it sends the generated normalized data that corresponds to each clause of each sentence (e.g., data such as that represented by Figure 16) to the Data Set Indexer 1705 (e.g., component 1207 in Figure 12). The Data Set Indexer 1705 processes the ENLP output, indexing and storing the information in a format that is dependent upon the storage representation of the enhanced document indexes (for example, the term-clause, term-sentence, and term-document indexes). One skilled in the art will recognize that other methods of data set preprocessing, indexing, and storing may be implemented in place of the methods described herein, and that such modifications are contemplated by the methods and systems of the present invention. For example, the data may be indexed according to a variety of schemes and distributed across a plurality of repositories.
In addition to indexing and storing a data set prior initially, in some embodiments, the SQE can incrementally index and store new documents, updating the relevant enhanced document indexes as necessary. In addition, in embodiments that support dynamic changes to an existing ontology, the SQE
can determine a set of affected documents and "re-ingest" a portion of the corpus as needed. Other variations can be similarly accommodated.
After indexing and storing a data set, the SQE may perform its second function, processing relationship queries against the stored data set.
Figure 18 is a block diagram of query processing performed by an Syntactic Query Engine. A user 1801 (or program through an API) submits a relationship query 1810 to the SQE. The Query Processor 1802 component of the SQE
transforms the query into one or more keyword searches 1811 with appropriate syntactic and semantic annotation information included and executes the keyword searches 1811 by invoking one or more keyword search engine processes, for example, keyword search engines 1804-1807. The results of each keyword search 1811 are subsequently returned back to the invoking Query Processor 1802, which then combines the results 1812 as specified in the relationship query 1810 and returns them to the user/program.
Figure 19 is an example flow diagram of relationship query processing steps performed by an example query processor of Syntactic Query Engine. The query processor executes one or more of steps example 1901-1907 for each query that is forwarded from the user interface/API support modules. One skilled in the art will recognize that the precise behaviors of each step depend upon the heuristics and other rules that are encoded, the preferences set for search parameters, and the way the normalized data is actually stored in the term-clause, term-sentence, and term-document indexes.
In step 1901, the query processor receives a relationship query. Recall that the relationship query of the example syntax described above specifies a syntactic search portion (which may be empty), prepositional constraints, document level keyword filters, and meta-data filters. Also, it is possible to specify values for any one of the relationship query components without the others. Depending upon the implementation, the query processor may include a relationship query interpreter or parser (not shown) to parse the received query into its constituent parts and to produce some form of code (internally specified, using a standard programming language, or otherwise) that controls the flow of the keyword searches that are invoked and the combining of the results. This approach is especially useful with a syntax as described that follows a prescribed grammar.
The relationship query is than transformed as necessary in example steps 1902-1907 in accordance with the implementation.
In step 1902, the query is transformed to handle synonyms of any specified subjects and/or objects. In one embodiment, synonyms are handled by searching the ontology structure for synonyms of a specified term, and, if they are present, adding keyword searches for each synonym found. In an alternative embodiment, terms having synonyms are mapped (e.g., at SQE
configuration time) to a common indicator, such as a "concept identifier"
(concept ID). During ingestion, terms are looked up in the map to determine whether they have corresponding synonyms (hence concept IDs), and, if so, the concept IDs are stored as part of the indexing information. Upon receiving a query, a look up is performed to find a corresponding concept ID (if one exists) to a received term. The query is then transformed so that the resultant keyword searches contain the corresponding concept ID as appropriate. One skilled in the= art will recognize that, using either mechanism (or any other implementation), the formatting of the invoked keyword searches needs to correspond to the way the data has been indexed.
In step 1903, the query processor transforms the query to handle ontology path specifications or "types" if provided in the received query string.
For example, a relationship query may provide a subject and/or object list as [entity] or [person] or [location/country], etc., which is interpreted as a type of node in an ontology hierarchy. The amount of the pathname that is specified is matched to the ontology. Thus, the entity specification "[location/country]"
is matched to any ontology path containing that sub-path. Keyword searches are thus specified for each of the matching ontology paths. Similarly, heuristics may be applied that include as additional keyword searches also searches for related terms, such as hypernyms and hyponyms (more generic and more specific classification terms, respectively), if not already accounted for using available synonym logic.
In step 1904, the query processor transforms the query to handle action types (types of verbs) if specified in the relationship query. For example, a query that specifies "president < > [communication]" instructs the SQE to find all relationships that involve a president doing something by any verb that is considered to be a communication verb. Like the implementations for synonyms described above, the query processor can handle this by including additional keyword searches for each verb of that action type, or can use some kind of verb concept identifier. Again, the query processor needs to match whatever form the indexed data is stored.
In step 1905, based upon the additional transformations from steps 1902-1904, the query processor reformulates the relationship query into one or more keyword searches that can be executed by a keyword search engine. In step 1906, the one or more keyword searches are accordingly invoked and executed. If the enhanced document index is stored as one data structure, then it is possible to execute one keyword search. Alternatively, if the indexed data is actually split between several matrices, then a keyword search is executed on each index as appropriate. For example, searches for matching "keywords" as subjects (or modifiers of subjects) are executed on the subject term-clause index. In step 1907, the results of the keyword searches are combined as expressed in the flow of control logic parsed from the relationship query, and then forwarded to an interface for presentation to the user or program that invoked the relationship query. The query processor then returns to the beginning of the loop in step 1901.

The functions of data set processing (data object ingestion) and relationship query processing can be practiced in any number of centralized and/or distributed configurations of client ¨ server systems. Parallel processing techniques can be applied in performing indexing and query processing to substantial increase throughput and responsiveness.
Representative configurations and architectures are described below with respect to Figures 20-25; however, one skilled in the art will recognize that a variety of other configurations could equivalently perform the functions and capabilities identified herein.
Figure 20 is an example block diagram of a general purpose computer system for practicing embodiments of a Syntactic Query Engine. The computer system 2001 contains one or more central processing units (CPUs) 2002, Input/Output devices 2003, a display device 2004, and a computer memory (memory) 2005. The Syntactic Query Engine 2020, including the Query Processor 2006, Keyword Search Engine 2007, Data Set Preprocessor 2008, Data Set Indexer 2011, Enhanced Natural Language Parser 2012, and data set repository 2015, preferably resides in memory 2005, with the operating system 2009 and other programs 2010 and executes on the one or more CPUs 2002. One skilled in the art will recognize that the SQE may be implemented using various configurations. For example, the data set repository may be implemented as one or more data repositories stored on one or more local or remote data storage devices. Furthermore, the various components comprising the SQE may be distributed across one or more computer systems including handheld devices, for example, cell phones or PDAs. Additionally, the components of the SQE may be combined differently in one or more different modules. The SQE may also be implemented across a network, for example, the Internet or may be embedded in another device.
Figure 21 is an example block diagram of a distributed architecture for practicing embodiments of a Syntactic Query Engine. This architecture supports parallel processing of the indexing (ingestion) of each document as well as parallel query processing. The basic organization involves storing a portion of each (term-clause, sentence, and document) index on multiple machines (e.g., servers), with potentially multiple CPUs, in order to achieve greater throughput and accommodate the extensive storage requirements of a very large corpus. For example, typically a large corpus will easily exceed the CPU and storage limits of a single server machine.
Moreover, to provide commercially viable search solutions, the SQE needs to respond to queries in a timely fashion. Thus, the number of servers and CPUs is typically determined by the expected size of the data set and the desired query response time, and is typically set up during SQE configuration.
The unit of organization used to support indexing and searching is termed a "partition." Thus, an enhanced document index (labeled here as a "keyword index") comprises typically a plurality of "partition indexes," each of which stores some portion of the total keyword index. To perform a search on the entire corpus, then, it is necessary to search each of the partition indexes (with the same keyword search string) and thereafter to combine the results as if the search were performed on a single index. Note that the keyword index may be partitioned according to a variety of schemes, including, for example, a percentage of the index based upon the size of the documents indexed, documents that somehow related together by concept or other classification, schemes based upon storing portions of the index based upon a type supported by the ontology, etc. Any such scheme may be implemented by the servers and may be optimized for the application for which the SQE is being deployed.
A variety of servers and services are employed to process the ingestion and searching on the backend so as to present a unified view of the term-clause, sentence, and document indexes. Figure 21 presents one such embodiment, although one skilled in the art will recognize that a variety of other organizations and components can equivalently support and provide the functions and techniques of the SQE. In Figure 21, an index manager 2101 schedules document ingestion for a collection of document 2110 between a plurality of workers 2102a-2102d, each responsible for indexing a portion of the corpus. The work could be divided at a variety of levels including by document, by sentence, etc., and allows the ingestion workload to be processed in parallel, thus decreasing the amount of time required to ingest a corpus. Each worker 2102a-2102d contains an instance of the SQE data set processing components (and others if necessary), including the preprocessor and an instance of the ENLP. Upon parsing a sentence and annotating it with syntactic and semantic tags, the worker 2102a-2102d creates a corresponding temporary keyword index 2103a-2103d, which represents the portion of the corpus that it has processed until stored in the partition indexes 2104-2105. The index manager 2101 is responsible for distributing the temporary keyword indexes 2103a-2103d to the partition indexes 2104 and 2105 to be merged into their respective keyword indexes 2106 and 2107. Note that the index manager 2101 and the workers 2102a-2102d may in some embodiments utilize an additional data base management system 2120 to store recovery information, such as copies of documents, document metadata, sentences, parse trees and a copy of the clause tables, 2130 although this is a convenience and not necessitated by the functions of the SQE.
Figure 22 is a block diagram overview of parallel processing architecture that supports indexing a corpus of documents. This figure shows one arrangement of servers that can be used to effect the parallel processing architecture of Figure 21. Specifically, AdminClient 2201 controls invocation of an IndexManager (server) 2202 which stores working and recovery information in a database 2203 (if part of a particular implementation) and distributes indexing work to one or more IndexWorkers (servers) 2204. When an IndexWorker 2204 completes indexing of an object (document, sentence, etc.), notification is returned to the IndexWorker 2202, which at appropriate times instructs a corresponding PartitionIndex (server) 2205 to store the indexing information in the appropriate clause, sentence, and document indexes. Each IndexWorker 2202 may also communicate with a WebServer 2206 to deliver status and error information.
Figure 23 is a block diagram overview of parallel processing architecture that supports relationship queries. The partition indexes, such as Partition Index A 2104 and Partition Index B 2105 (in Figure 21), may be arranged in a hierarchy of searcher (servers), and more than one partition index may be managed by a single searcher. Typically, it is advised to have a separate partition index for each CPU present in a server machine to take advantage of inherent parallel processing opportunities in a multiple CPU/parallel processor, machine; however, other arrangements are also possible. In Figure 23, a user such as a researcher using a web browser user interface 2301 or an application using the SQE APIL 2302 issues a relationship query to the SQE as described in detail in the other figures via some supported communications protocol, such as HTTP. (Note also that a server side application that resides on the search service server 2311 could also issue a direct request to the search service 2304.) WebServer 2303 receives the relationship query and issues appropriate search requests to the SearchService 2304. Note that depending upon the particular implementation, the various functional components described by Figure 12 and multiple instances of the same components could reside upon one or more of these servers. The query is preferably organized into a plurality of keyword and ontology searches that are distributed to be processed in parallet and then combined before returning a result to the WebServer 2303. (The returned result flow is not shown.) Thus, search service 2304 invokes a "top" level search 2305 which is responsible for conducting the parallel searches to effectuate a search of the entire keyword index. Searcher 2305 is shown communicating via a remote method invocation protocol to a single partition index server 2308. Searcher 2305 instructs (sub)searcher 2307 to also perform part of the search. Searcher 2307 is shown communicating with two partition indexes, 2309 and 2310. The searcher 2305 also communicates with a (possibly hierarchy of) ontology searchers 2306 as needed to search for pathnames in the ontology (and for browsing the ontology as supported by other aspects of an example SQE user interface).
Figure 24 is an example block diagram that shows parallel searching of an enhanced document index. In Figure 24, a search service 2401 receives a search and distributes the requested relationship search to a top level searcher 2402. The top level searcher 2402 then, in parallel, invokes the same relationship search on a plurality of searchers 2403-2405, depending upon the organization of the partition indexes and whether it is required to search all of them for a particular relationship query. For example, if the partition indexes are organized such that a percentage of the corpus is indexed on each (not by entity type or some other organization), then all of the partition indexes are searched in parallel. Searcher 2403 performs the relationship search on partition index 2410, searcher 2404 performs the relationship search on partition indexes 2422 and 2423, up through searcher 2405 performs the relationship search on partition index 2424. Also, if an ontology search (for synonyms, pathnames, etc.) is required, then the top searcher 2402 invokes a top level ontology searcher 2406 to perform (in parallel as required) an ontology search using one or more ontology searchers such as searcher 2407 to search one or more ontology data repositories 2408 and 2409.
As mentioned, it is sometimes desirable to support the indexing of additional corpus information even when the corpus is being searched. This provides the ability to support incremental indexing of data. It is also sometimes desirable to provide fault tolerance, especially in mission critical applications. Figure 25 is an example block diagram of an architecture of the partition indexes that supports incremental updates and data redundancy. The underlying organization involves maintaining several data instances of the partition index, only one of which is "active" for searching at any one time and maintaining a redundant copy of the data instances that comprise the partition index. The "active" partition index data instance provides the view of the data that the initiator of a query believes is current. To update a partition index, the searcher redirects the indicator of the active partition index data instance to a different data instance. In Figure 25, the searcher 2501 maintains a master partition index 2502 and a clone partition index 1203, which is a replica of the master partition index. Each of the partition indexes 2502 and 2503 in turn maintain a plurality of data instances, for example data instances 2510-2512 and 2520-2522. In the diagram, partition index data instance 2511 is indicated as the "active" partition index data instance. While instance 2511 is active, the searcher 2501 can update other data instances 2510 and 2512 thus providing another type of parallelism. Since clone partition index 2503 is a replica of the master partition index 2502, the data instances 2520-2522 are replicas of the information and state of data instance 2510-2512. One skilled in the art will recognize that there are other ways to provide incremental updating and that Figure 25 illustrates one of them.
The architectures described (and others) can be used to support the indexing and searching functions of an example SQE. Figure 26 is an example conceptual diagram of the transformation of a relationship search into component portions that are executed using a parallel architecture. In the example illustrated, the relationship query 2601 is a link search, however one skilled in the art will recognize that the technique described can be applied and extended to a variety of searches including a plurality of relationship searches that are combined by a scripting language or other means of controlling flow.
The query being processed:
Arafat <> {[organization]} <> Abu Nidal Instructs the SQE to find all relationship where there is a 3rd entity that is an organization linking Arafat and Abu Nidal. In this case, the SQE transforms the query into two syntactic sub-searches 2602 and 2603:
Arafat <> * <> [organization]
which will locate all organizations with which Arafat has any kind of relationship;
and Abu Nidal <> * <> [organization]
which will locate all organizations with which Abu Nidal has any kind of relationship. Each of these syntactic searches 2602 and 2603 are executed using, for example, the parallel architecture described with reference to Figures 22-25. The syntactic search 2602 is distributed to a top searcher 2604 to perform one or more syntactic searches on the partition indexes that make up the corpus and one or more ontology searches as required. Note that as part of this process, the various searchers invoke one or more keyword search engines to perform the actual keyword search on the annotated indexes.
Similarly, the syntactic search 2603 is distributed to a top searcher 2605 to perform one or more syntactic searches on the partition indexes that make up the corpus and one or more ontology searches as required. Again, keyword search engines are invoked as part of this process. Once results from the sub-searches are determined, the query processor, for example, one residing in a search service (such as search service 2401 in Figure 24) determines based upon the initial query 2601 how to combine the results. In the example described, the intersection of the resulting clauses provides the overall query result 2607 desired. One skilled in the art will recognize that similar combinations of sub-searches can be accommodated. Those that indicated a desired intersection (as from a Boolean AND operation) are easily specified.
However, to support other types of control flow operations, such as those that require a union of the resultant data, needs to be defined as to what aspects are desired to be combined especially if the sub-searches yield different types of results.
The architectures illustrated (and others) can also support the preprocessing and data storage functions of an example SQE. As described with reference to Figure 17, the Data Set Preprocessor 1702 performs two overall functions ¨ building one or more tagged files from the received data set files and dissecting the data set into individual objects, for example, sentences.
These functions are described in detail below with respect to Figures 27-29.
Although Figures 27-29 present a particular ordering of steps and are oriented to a data set of objects comprising documents, one skilled in the art will recognize that these flow diagrams, as well as all others described herein, are examples of one embodiment. Other sequences, orderings and groupings of steps, and other steps that achieve similar functions, are equivalent to and contemplated by the methods and systems of the present invention. These include steps and ordering modifications oriented toward non-textual objects in a data set, such as audio or video objects.
Figure 27 is an example flow diagram of the steps performed by a build_file routine within the Data Set Preprocessor component of a Syntactic Query Engine. The build_file routine generates text for any non-textual entities within the dataset, identifies document structures (e.g., chapters or sections in a book), and generates one or more tagged files for the data set. In one embodiment, the build_file routine generates one tagged file containing the entire data set. In alternate embodiments, multiple files may be generated, for example, one file for each object (e.g., document) in the data set. In step 2701, the build_file routine creates a text file. In step 2702, the build_file routine determines the structure of the individual elements that make up the data set.

This structure can be previously determined, for example, by a system administrator and indicated within the data set using, for example, HTML tags.

For example, if the data set is a book, the defined structure may identify each section or chapter of the book. These HTML tags can be used to define document level attributes for each document in the data set. In step 2703, the build_file routine tags the beginning and end of each document (or section, as defined by the structure of the data set). In step 2704, the routine performs OCR processing on any images so that it can create searchable text (lexical units) associated with each image. In step 2705, the build_file routine creates one or more sentences for each chart, map, figure, table, or other non-textual entity. For example, for a map of China, the routine may insert a sentence of the form, This is a map of China.
In step 2706, the build_file routine generates an object identifier (e.g., (a Document ID) and inserts a tag with the generated identifier. In step 2707, the build_file routine writes the processed document to the created text file.
Steps 2702 through 2707 are repeated for each file that is submitted as part of the data set. When there are no more files to process, the build_file routine returns.
Figure 28 illustrates an example format of a tagged file built by the build_file routine of the Data Set Preprocessor component of a Syntactic Query Engine. The beginning and end of each document in the file is marked, respectively, with a <DOC> tag 2801 and a </DOC> tag 2802. The build_file routine generates a Document ID for each document in the file. The Document ID is marked by and between a <DOCNO> tag 2803 and a </DOCNO> tag 2804. Table section 2805 shows example sentences created by the build_file routine to represent lexical units for a table embedded within the document.
The first sentence for Table 2805, This table shows the Defense forces, 1996, is generated from the title of the actual table in the document. The remaining sentences shown in Table 2805, are generated from the rows in the actual table in the document. One skilled in the art will recognize that various processes and techniques may be used to identify documents within the data set and to identify entities (e.g., tables) within each document. The use of equivalent and/or alternative processes and markup techniques and formats, including HTML, XML, and SGML and non-tagged techniques are contemplated and may be incorporated in methods and systems of the present invention.
The second function performed by the Data Set Preprocessor component of the SQE is dissecting the data set into individual objects (e.g., sentences) to be processed. Figure 29 is an example flow diagram of the steps performed by the dissect_file routine of the Data Set Preprocessor component of a Syntactic Query Engine. in step 2901, the routine extracts a sentence from the tagged text file containing the data set. In step 2902, the dissect_file routine preprocesses the extracted sentence, preparing the sentence for parsing. The preprocessing step may comprise any functions necessary to prepare a sentence according to the requirements of the natural language parser component of the ENLP. These functions may include, for example, spell checking, removing excessive white space, removing extraneous punctuation, and/or converting terms to lowercase, uppercase, or proper case. One skilled in the art will recognize that any preprocessing performed to put a sentence into a form that is acceptable to the natural language parser can be used with techniques of the present invention. In step 2903, the routine sends the preprocessed sentence to the ENLP. In step 2904, the routine receives as output from the ENLP a normalized data representation of the sentence. In step 2905, the dissect_file routine forwards the original sentence and the normalized data representation to the Data Set Indexer for further processing.
Steps 2901-2905 are repeated for each sentence in the file. When no more sentences remain, the dissect_file routine returns.
The Data Set Indexer (e.g., component 1705 in Figure 17) prepares the normalized data generated from the data set (e.g., as illustrated in Figure 16) to be indexed and stored in the data set repository. One skilled in the art will recognize that the normalized data can be stored in a variety of ways and data structures, yet still achieve the abstraction of maintaining a term-clause matrix, a term-sentence matrix or a term-document matrix. Any data structure that can be understood by the target keyword search engine being used is operable with the techniques of the present invention. In one embodiment, separate indexes exist for each enhanced document (term-clause, term-sentence, and term-document) matrix. In addition, in some embodiments the term-clause index is further divided into a separate index for each grammatical role, so as to allow more efficient keyword searches. The indexes are cross referenced by an internal identifier, which can be used to decipher a document id, sentence id, or a clause id. The tuple (document id, sentence id, clause id) that uniquely identifies each clause in the document corpus. Other divisions and distributions of the data can be accommodated. Table 1 below conceptually illustrates the information that is maintained in an example term-clause index of the present invention.
Field Name Type Description Id (internal) Indexed, document id, sentence id, clause id stored concatenated separated by subject tokenized, contains subjects(s), subject modifiers and indexed entity type(s) for subjects and modifiers.
The modifiers are preferably separated into prefix and suffix. If subject has entity type, the data indexer also stores t_entity (just once). If any modifier has entity type, the data indexer also stores tm_entity (just once). Noun phrases that were recognized by NL parser are also stored with spaces replaced by 'V' The subject field order is:
prefix_subject_mod subject suffix_subject_mod Entity_types NLP_noun_phrases.
object tokenized, contains objects(s), object modifiers and indexed entity type(s) for objects and modifiers The modifiers are preferably separated into prefix and suffix. If object has entity type, the data indexer stores t_entity (just once). If any modifier has entity type, the data indexer also stores tm_entity (just once). Noun phrases that were recognized by NL parser are also stored with spaces replaced by 'V' The object field order is:
prefix_object_mod object suffix_object_mod Entity_types NLP_noun_phrases.
pcomp tokenized, contains pcomp(s), preposition(s), pcomp indexed modifiers and entity type(s) for pcomp, modifiers. The modifiers are preferably separated into prefix and suffix. If pcomp has entity type, the data indexer also store t_entity (just once). If any modifier has entity type, the data indexer also stores tm_entity (just once). Noun phrases that were recognized by NL parser are also stored with spaces replaced by 'V' The pcomp field order is:
preposition pcomp modifiers, pcomp Entity_types NLP_noun_phrases verb tokenized, contains verbs(s), verb modifiers and indexed entity type(s) for verbs and modifiers.
Noun phrases that were recognized by NL
parser are also stored with spaces replaced by A.' The verb field order is:
prefix_verb_mod verb suffix_ verb _mod Entity_types NLP_noun_phrases.
parent_id indexed, clause id(10) stored clause_rel_sent_class tokenized, Contains inter-clause relationships such indexed as:
= conditional_c = causal_c = prepositional_c = temporal_c and Sentence Attributes such as:
= question_s = definition_s = temporal_s = numerical_s.
relationship stored (Encoded clause for display) Table 1 As can be observed from Table 1, a variety of information is indexed to correspond to the term-clause index. "Entity_types" includes whatever types are supported by the ontology. In a default system, several types of entities are supported; however, one skilled in the art will recognize that other categorizations of types could also be supported.
Similarly, particular exemplary sentence and inter-clause relationship types are listed, however other classifications are supported as well.
Figure 30 is an example conceptual block diagram of a sentence that has been indexed and stored in a term-clause index of a Syntactic Query Engine. The example sentence illustrated is "Jane admires sunny Seattle on a busy June 3rd." The id field 3001 is an internal string that can cross-reference to the corresponding clause, sentence, and document. The subject field 3002 includes the term "Jane" (the subject), which has no modifiers, but is a member of two classifications in the ontology: an individual (t_entity/person/any/individual) and a female (t_entity/person/female). The field also stores that the subject has an entity type (indicated as t_entity). The verb field 3003 includes the stemmed form of the verb term "admires" (the verb), followed by a series of suffix modifiers of the verb, which appear also as parts of prepositional phrases in pcomp field 3005. The modifiers (m_on, m_busy, m_June, m_3rd) are stored in the verb field along with the information that at least one of the modifiers has an entity type (indicated by a tm_entity tag) and that the entity type in the modifier list includes a date (tm_entity/temporal/date).
As illustrated, the object field 3004 includes the term "Seattle," along with annotations that it has an entity type (t_entity) of city (t_entity/location/city) and has a series of prefix and suffix modifiers (m_sunny, m_on, m_busy, m_June, m_3rd) that have entity types (tm_entity) including a date (tm_entity/temporal/date). The pcomp (prepositional complement) field 3005 includes the terms in the prepositional phrase "on a busy June 3rd" stored with the phrase "June Td" as the prepositional complement and the other terms as modifiers. The phrase is recognized as an entity, hence the pcomp field includes an entity type (t_entity) of date (t_entity/temporal/date). The parent_id field 3006 indicated the clause id of the parent clause in the sentence if there are multiple clauses. The clause_rel_sent_class field 3007 indicates any inter-clause relationships, such as whether the clause is a conditional phrase, and any sentence attributes such as an annotation that the sentence is, as in this case, a temporal statement. Such classifications enable keyword searching based upon classifications of sentences as well as other syntactic and semantic tags. The relationship field 3008 is used for displaying the clause and is implementation specific.
Table 2 below conceptually illustrates the information that is maintained in an example sentence index of the present invention. Since the terms with syntactic and semantic annotations are stored in the term-clause index, the enhanced indexing information can be identified by the sentence index, but is not typically stored as part of it.
Field Name Type Description sentid indexed Document id sentence id separated by sent_text Stored String content of the sentence Table 2 Table 2 includes an indicator to the entire content of the sentence, and an identifier that will enable cross referencing to the internal clause ids of the clauses that constitute the sentences. The identifier also cross-references to the document that contains the sentence.
Table 3 below conceptually illustrates the information that is maintained in an example document index of the present invention. Since the terms with syntactic and semantic annotations are stored in the term-clause index, the enhanced indexing information can be identified by the document index, but is not typically stored as part of it.
Field Name Type Description doc_id indexed, stored Document id dhs_doc_id stored DHS_doc_id (URL in one embodiment) title Tokenized, Document title Indexed, stored creationDate Indexed, stored Document creation date;

format: yyyy.MM.dd-HH:mnn:ss metatag Tokenized, MetatagName#MetatagValue Indexed, stored content Tokenized, String content of the document Indexed, Not Stored document_type stored Document type (HTML, MSWORD) Table 3 The document index stores document tag information that is created typically during the data set preprocessing stage as well the meta-data tags and (an indicator to) the full document content. The type of the document is also maintained.
Figure 31 is an example conceptual block diagram of sample contents of a document index of a Syntactic Query Engine. The doc_id field 3101 contains a document identifier; the title filed 3102 contains a string representing the title, the creationDate field 3103 indicates the date the document was created if known. The metadata field 3104 includes a series of meta data tags, each with the metadata name followed by its value. The content field 3105 contains an indicator to the string content of the document.
The document_type field 3106 is an indicator of the format of document (such as an HTML file) determined typically during the data set preprocessing stage.
Specific embodiments of, and examples for, methods and systems of the present invention are described herein for illustrative purposes. The scope of the claims should not be limited by the embodiments set forth in the examples, but should be given the broadest interpretation consistent with the description as a whole.
Aspects of the invention can be modified, if necessary, to employ methods, systems and concepts of these various patents, applications and publications to provide yet further embodiments of the invention. In addition, those skilled in the art will understand how to make changes and modifications to the methods and systems described to meet their specific requirements or conditions. For example, the methods and systems described herein can be applied to any type of search tool or indexing of a data set, and not just the SQE described. In addition, the techniques described may be applied to other types of methods and systems where large data sets must be efficiently reviewed. For example, these techniques may be applied to Internet search tools implemented on a FDA, web-enabled cellular phones, or embedded in other devices. Furthermore, the data sets may comprise data in any language or in any combination of languages. In addition, the user interface and API
components described may be implemented to effectively support wireless and handheld devices, for example, PDAs, and other similar devices, with limited screen real estate. These and other changes may be made to the invention in light of the above-detailed description. Accordingly, the invention is not limited by the disclosure.

APPENDIX A

In Fact - Help 17- Search I Corpus I
Preferences I History I Help c7, oe oe Welcome to the web-based User's Guide for the InFacto system.
=
Search 0 o Overview (5) o Example Queries 0 Query Syntax o What is a Source?

o What is a Target? 0 o gperators o Handling Special Characters (5) o Using Noun Phrases and Modifiers o Actions and ActionTypes o Using Offsets in Document Keyword Search o Filtering Using Metadata o Using Ontology Paths o Entity Link Searches o Sample Queries 1-d o Displaying Results o Nested Search = Using the Query Generator = Ontology Support c.;11 o InFact Standard Ontology -a = Exporting Reports =
Corpus Page oe = Preferences Page = History Pages InFact -Help Overview Unlike most search engines, the In Fact system is much more than just keyword search.
oe The InFacts system uses sophisticated natural language parsing capabilities to provide a way for you to search for and return specific information within a given corpus or body of documents. Unlike other search engines based on keywords, InFact allows users to construct powerful search queries that search within document text to find relationships between entities. Using our query syntax, you can define precisely what you are looking for and get the specific results you seek, embedded in its paragraph or sentence-level context.
In order to understand and use this system effectively, users should expect to spend some time familiarizing themselves with this type of query-based search framework, rather than simply attempting to execute keyword searches that will not exploit the full capabilities of the the system.
Time invested in learning the query syntax will save many hours that you would otherwise spend reviewing unwanted results.
Queries are based on a relationship between two entities and an action that links them. Syntactically CO
we represent this like Source > Action > Target, with may be followed by one or more optional oe constraints. But before examining the query syntax in detail, lets start by briefly looking at some simple examples to familiarize ourselves with the concept. (The examples in this document presume co that queries are being executed against a relevant corpus. If you have a different corpus, you may substitute different terms.) Note the use of '*1 as a wild card character, meaning, "any action performed"
or "any entity found":
Example Query - what you type in Interpretation You can search for any actions Returns all relationships in that a given entity has Bush > * > * which Bush has done something 1-d performed. to another entity.

Returns all relationships in You can search for any actions * > * > Bush which things have been done to performed on a given entity.
Bush by another entity.
You can combine these first two Returns all relationships in queries to search for any which Bush has performed an oe Page of 24 In Fact - Help actions performed on or by a action, or an action has been given entity by making the Bush <>* <> performed on Bush by another arrows go both ways. entity.
You can specify two entities and Returns all relationships found search for a specific action that Clinton > visit > China in the corpus where Clinton might link these entities. visited China.
You can specify two entities and Returns a list of all relationships search for all the actions that Clinton > * > China in which Clinton was involved in link these entities. an action involving China.
Returns any instances in the corpus In which somebody paid You can specify an entity and an bin Laden for something, or he action, and search for any other Bin Laden <> pay >
paid for something. By default, entities that fit that relationship.

similar Nierbs like "purchase" are o searched as well.
You can search for relationships between a given entity and a type of entity, where the type is Returns all relationships In defined as an OntologyPath in which Clinton visited a country.

an associated ontology file. Clinton > visit > [country]
You would see all the countries (InFact provides a standard he visited listed in the results.
default ontolo ay with common entity types, like 'country' or 'name'.) You can search for connections between entities that are based on a whole class or type of Returns all relationships actions, rather than a specific Bush <> [communicate] <>
involving communication action. These ActionTypes may Blair between Bush and Blair.
be customized for your corpus by a InFact System Administrator.
-a Returns any instances in the You can use boolean operators corpus in which somebody such as AND, NOT, and OR to Al Qaeda AND NOT Bin Laden > associated with Al Qaeda, but Page 3 of 24 InFact - Help n.) restrict searches. travel > Saudi Arabia not Bin Laden, traveled to Saudi o o Arabia.
cr cr You can search for connections Returns all relationships In oe oe between an entity and other Bush <> {[person]} <> Blair which another person is linked -n.) entitles. with both Bush and Blair.
You can search for documents Returns all documents in the that contain references to constitution corpus that contain references information you need. to the constitution.
You can search for documents Returns all documents dated that contain references to constitution METADATA after the year 2000 in the Information you need, and filter CONTAINS Date>2000 corpus that contain references n them by metadata. to the constitution.

Returns all relationships iv 0, You can filter queries by israei<>*<>arafat DOCUMENT
between Israel and arafat in co u.) document searches: CONTAINS shamir documents that also contain o, in o references to shamir. co iv Returns all relationships You can filter queries by srael<>*<>arafat PREP between Israel and arafat where co i Information found in CONTAINS J Jordan is referenced within a ordan 0, prepositional phrases:
prepositional phrase, like "in 1 Jordan" or "to Jordan".
H
CA
israel<>*<>arafat PREP Returns the intersection of the You can combine all these CONTAINS Jordan DOCUMENT results of the previous three queries: CONTAINS shamir METADATA queries.
Note that the order of CONTAINS Date>2000 the clauses cannot be changed.
In all cases, if you were to type in a search query and execute it against an appropriate corpus of Iv n documents, it would return a list of relationships where you can easily identify the source entity, the 1-3 action, and the target entity, along with links to the sentences in the document where these relationships appear. The result display can be customized using the preferences page, as discussed in cp n.) the Preferences section later in this document. Results can also be sorted In various ways for easier o o vi viewing using the tabs along the top of the result table. Clicking on an individual result will take you to the sentences in the document where the relationship between the entities was found. .6.
.6.
vD
oe .6.
Page 4 of 24 InFact - Help Query Syntax Let's take a more detailed look at the query syntax.
cr oe oe This search product allows you to have complete control over searches, provided that you conform to the query syntax. Searches are created by means of a single query entered into the search textbox.
A query is made up of Entities and Actions that are linked via a series of operators.
= Entity - An Entity is a noun or noun phrase in the search query or result. An Entity can be the source (initiator of an action), the target (receiver of an action), or the complement of a prepositional phrase. Entities can be multiple words. If they are quoted, the exact phrase must be matched by a phrase in a document being searched. Either double quotes or single quotes may be used; if double quotes are used, then synonyms of the quoted expression will not be 0 Included in a search. If single quotes are used, synonyms of the quoted expression will be included. (Note that entities cannot cannot start or end with a dash ('-') unless quoted, and entities that contain apostrophes must be double quoted.) ui co o Source - The initiator of an action is referred to as the source. For example, in the query co [Country] > threaten > USA
"Country" is the source. Here we are interested in all countries that threaten the US, but not all countries that the US threatens.
o Target - The receiver of an action is referred to as the target. For example, in the query USA > investigates > [organization]
"organization" is the target of the action. Here we are interested in all political organizations that are the target of an investigation, but not those that are initiating an investigation.
o Prepositional Complement - An action is often performed with a prepositional complement. For example, in the query Maya > visit > grandmother PREP CONTAINS Tuesday oe InFact - Help "Tuesday" is the prepositional complement of the sentence. We are only interested in visits that happened on Tuesdays.
cr oe oe = Action - All relationships are based on an action, or verb. For example, in the query Maya > visit > grandmother "visit" is the action.
Operators:
= Action directionality for events: <, >, <>
= Boolean: AND, OR, NOT. The default operation for omitted boolean operators is OR. Booleans do not have to be uppercase, although they are presented that way in this document for clarity. 0 =
Prepositional constraint: PREP
CONTAINS (upper or lowercase), or 'A' 61 = Document keyword constraint: DOCUMENT CONTAINS (upper or lowercase), or ';' = Metadata constraint: METADATA CONTAINS (upper or lowercase), or '#' 1.) co = Wildcards (not within quotes): *
=
Offset indicators: ¨ 0 co = Curly braces -0- are used for indirect link searches, to search for entities that link other entities together (see below) = Brackets 0 are used to denote types, either an OntologyPath, or if used with a verb, an ActionType.
All reserved terms above are case insensitive, however no mixed case is allowed. Also, white space is ignored unless contained in a quoted term string, double quotes are required.
Parenthesis can be used to nest portions of the query.
Query Format:
The relationship query is considered to have the general format Source Entity > Action > Target Entity. However, it is not necessary to specify all three, nor do the arrows need to point to the right.
For example, = Bush < *
= Bush < * < *
= * > Bush oe = Page 6 of 24 In Fact - Help = * > * > Bush are all correct, and there is no difference between the first two or the last tWo. Although both actions and entities can be represented by a wildcard, the position of the wildcard in the query makes it clear cr what it represents. Entitites cannot point to each other directly. For example, "Clinton > Bush" would cr oe not be correct, as there is no action (or wildcard '*' character) specified.
Optional Clauses:
In addition to the basic relationship component of the query, there are three optional clauses that can be added to filter results:
= any prepositional constraints, to filter results by information found in a prepositional phrase;
= any document keyword constraints, to restrict search to documents that have certain keyword (s);
= any metadata constraints, to restrict search to documents tagged with 'specific metadata values 0 or ranges.
Note that these clauses, if combined, must appear in this order, and must be separated by at least one co white space.

co These clauses can be expressed in either a long or abbreviated format. In the long format, the clauses are separated by the self-explanatory terms "PREP CONTAINS", "DOCUMENT
CONTAINS" and "METADATA CONTAINS". For example, look at this example, broken up into several lines for easier reading:
Bush > visit > [Country] AND NOT China PREP CONTAINS plane DOCUMENT CONTAINS "foreign service" OR diplomat METADATA CONTAINS Date>04/2002 Here we see a relationship query that specifies a search for "visit"
relationships between the entity "Bush" and any country except China. The relationship query is constrained by the preposition "plane", meaning that the word plane must be included in a prepositional phrase within this relationship, c.;=
indicating travel by plane. The search is further constrained by the document keywords / keyphrases "foreign service" and "diplomat", meaning that only relationships from documents containing these words should be returned. Finally, the search is constrained by a date range, where we are only oe Page 7 of 24 InFact - Help interested in searching documents written after April 2002. (This assumes that date metadata has been associated with the documents at time of ingestion.) Date and numeric metadata ranges are specified with "=", ">", ">=", and "<=".
Put together, this represents a powerful query that will search specifically for diplomatic trips that oe Bush took by plane since April 2002 to foreign countries with the exception of China. Note that although the query is separated into four lines here for clarity, it is interpreted as a single string by the InFacto system. Of course, queries need not be so specific or constrained; the simpler queries shown above that do not contain document or metadata constraints will simply return more results.
Additional clauses will increase the time needed for results to display to screen.
Here we have specified two expressions for the document filter: "foreign service" and "diplomat". What if a document contained the word "diplomatic" in it's adjective form? It's included. The search system automatically extracts the stem of the word and searches for other forms.
Sometimes when you perform searches, you will see that your query has been "stemmed" or truncated to remove a final 's', 0 led', or other non-essential parts of the word. Such changes to your query are presented in green text (5) so that this will be clear to you.
CO
Document search queries allowed by simply specifying a keyword or phrase. For example:

germany france AND england (5) For users more familiar with the system, filter clauses can also be entered in in a more abbreviated form, in which the terms "PREP CONTAINS", "DOCUMENT CONTAINS", and "METADATA
CONTAINS"
are replaced by a 'A', ';' and a '4#1 character respectively, as in:
Relationship query ^ Prep constraints; Document keyword constraints # Metadata constraints 1-d In our example, this would look like:
Bush > visit > [Country] AND NOT China A plane; "foreign service" OR diplomat # Date>04/2002 Remember that constraint clauses must be white space separated. Also note that multiple Metadata -a constraints can be used with complete boolean expressions, as in:
hamas > act* >* METADATA CONTAINS Author="Andrew Jackson" OR price=300 Page 8 of 24 InFact - Help Also, note that the booleans can be nested, as in:
england AND NOT (aerospace OR airways) >abandon > *
The query can also be restated as:
england AND NOT aerospace AND NOT airways >abandon > *
Additionally, "NOT" is to be used only in a query with multiple terms, in conjunction with "AND". The following queries are not valid:
Bush OR NOT Clinton > > *
[Person] NOT Clinton > be > president Handling Special Characters cr, Certain special characters may not be interpreted by the system correctly, and should be avoided if co possible. The current list of special characters is the following:

co If your search query term contains an apostrophe (`), you will need to put the term inside double quotes.
Using Noun Phrases and Modifiers Within the relationship query, the sources or targets of an action can be either nouns or noun phrases, like "United States of America". However, if the noun phrase has a number of modifiers, the InFact system may have separated them out during ingestion and you may not get many results If the whole phrase is included in the query. Consider the following sentence:
"The recent definition of a consensus DNA binding sequence for the ..."
-a Here the query "DNA binding sequence" > *" would probably not return this sentence as a result, because 'DNA' and 'binding' are modifiers that are not considered part of the source of any actions. Therefore, usinl lin,. rµcl InFact - Help noun phrases in searches may not be your best course of action. Here, you are better off using any of the terms in the noun phrase on its own, such as either of these two queries:
sequence > > *
(or) binding > > *
The InFact o system's modifier handling is one of the product's powerful features and it is worth understanding. Let's consider another example. Suppose that you are looking for a list of people who drive a black vehicle or wear red clothes, and you do not have (or trust) an ontology (e.g. the vehicle could be a tractor or a snowmobile). An effective query can simply include the modifier information, and need not refer to the vehicle or clothes at all. Look at these two queries:

[personl>*>red UJ
(or) UJ
[person]>drive>black co In this second example, notice that we don't even specify that black is a modifier. That's because InFact can 0 co search modifiers as well as nouns in a normal query. On the preferences page you can specify whether 0 modifiers should be included as normal search terms or not.
UJ
Sometimes many different noun phrases describe the same things, like "prostate cancer" and "cancer of the prostate". Because modifiers of key nouns are also searched by the system, you should be able to find all results you are looking for even if they are expressed in different ways.
Similarly, you could find all actions involving an organi72tion, like the National Transportation Safety Board, regardless of whether it is referenced by its full name or simply as the "National Transportation Board".
(Synonyms or acronyms are also searchable, but must be defined before ingestion in an ontology file.) 1-d Actions and ActionTypes Actions are defined by verbs or groups of verbs. When verbs are specified in queries in present tense, by default all forms and tenses of the verbs will be included in searches. For example, if the query includes the verb "talk", results will also include relationships that contain the forms "talked" or "talking". Additionally, similar verbs like the various forms of the word "speak" will also be searched. The InFact system Page 10 of 24 InFact - Help maintains a list of similar verbs that are included in relationship searches (but not document searches) by default.
If users specifically wish to search only on the verb in the query and no other synonyms, verbs can be quoted, as in:
Clinton > "talk" > Bush In this case only the verb "talk" will be searched on. Note that if the verb in the query is not in present tense, it is normalized to it's present tense form. If it is quoted, it will not be normalized, and it is unlikely that any results will be returned.

The InFact system also supports the definition of specific ActionTypes, or categories of actions that can be used to filter or expand your search. This can be very helpful when dealing with a corpus in which there are UJ
UJ
sets of actions that are related, although the verbs may not be considered synonyms in normal English co usage. If ActionTypes are defined, instead of searching on a particular verb users can search on the ActionType. In queries, ActionTypes are denoted by brackets [ ], and any verb found within brackets is 0 interpreted as an ActionType. For example, in the query below, "communication"
is an ActionType defined co by the system that includes a number of actions that are similar, but not synonyms, of the verb 0 communicate:
UJ
Clinton > [communication] > Bush This query would be equivalent to a combined search on all the verbs included within the ActionType "communication".
1-d The InFact system defines a number of default standard ActionTypes, and can be additionally custorni7ed to include additional corpus-specific ActionTypes. ActionTypes are generally created by a InFact System Administrator at the time when documents are ingested. Usually corpus-specific ActionTypes will be much more effective than the ones provided by default. The following is a table of definitions for the standard ActionTypes:
I.
Pane 11 of 24 InFact - Help Body Verbs of grooming, dressing, and bodily care, etc.
Change Verbs of change In size, temperature, intensity, etc.
Cognition Verbs of thinking, judging, analyzing, doubting, etc.
Communication Verbs of telling, asking, ordering, singing, etc.
Competition Verbs of fighting, athletic activities, etc.
Consumption Verbs of eating, drinking, using, etc.
Contact Verbs of touching, hitting, tying, digging, etc.
Creation Verbs of making, building, painting, writing, etc.
Emotion Verbs of feeling, etc.

Motion Verbs of walking, flying, movement, etc.
(5) Perception Verbs of seeing, hearing, observing, etc.
Possession Verbs of buying, selling, owning, transferring, etc.
Social Verbs of political and social activities and events. 0 Stative Verbs of being, having, and spatial relationships. 0 (5) Weather Verbs of raining, snowing, thawing, thundering, etc.
Term Offsets When using a document keyword search query or clause, identified by "DOCUMENT
CONTAINS" or ';', any words or quoted phrases may be included with booleans AND, OR, and NOT.
The search will be restricted to documents in which the terms are found. Additionally you can optionally specify a term offset. 1-d For example, the following query:
"malignant cancer",10 would return all instances in the corpus in which the words "malignant" and "canner" are found within 10 words of each other. This allows users to search for specific terms that may be separated in the documents by several other words, or several lines of text. Note that besides being a valid query by itself, this would Page 12 of 24 InFact - Help also be valid if appended to a Source > Action > Target relationship query (and prefixed by the ';').
MetaData filtering The third optional filter clause of a query contains MetaData constraints.
MetaData filtering allows you to constrain your search based on document level MetaData constraints. For example, let's say the corpus you are searching through has a text MetaData type callediluthor assigned to every document. If you wish to search for data with a certain author, you can specify that in a MetaData clause.
Metadata has either text values, numeric values, or date values. Numeric and date values can be specified as a range, i.e., "date > 04/2000" or "date <= 1998". The following date formats are supported:
Format: Example:

dd/mm/yyyy 12/23/2002 us:
mm/yyyy 04/2002 us:
co co String values are typed in the form tag=value, such as # Author="John Wayne".
You can also use wildcard characters, as in # Author¨Jo*Wayne . However, wildcards cannot be used inside quotes, and cannot be used in phrases with more than one word.
The MetaData fields and values associated with a corpus can be viewed on the Corpus Page.
1-d Ontology Searches:
Ontology paths are enclosed in brackets, as in [person] or [country]. If a bracketed term is found in a search query, we search the ontology for all paths matching the term. If there are multiple matches, all matches are included in the search and results are combined. For example, in a search query containing the type [person], InFact will substitute with [IF/Entity/Person]. (All InFact standard ontology paths begin with the root "IF"). If another path existed in a custom ontology such as "MyOntology/People/Person", this path would also be included in the query and results would be combined. InFact includes a default standard Pane 13 of 24 InFact - Help ontology with terms such as [person] in it. Custom ontologies can also be associated with corpus data during ingestion. For more information about ontologies, see the QtAckgy section below.
Entity Link Searches:
Entity Link searches can be used to discover entities that link two other entities together. For example, imagine that you are searching a political database and attempting to discover any links between Al Qaeda and Saddam Hussein. A search for direct relationships between them, such as "Hussein > * > Al Qaeda", returns nothing interesting. But while the system didn't find any sentences in the corpus that mention an explicit relationship between them, perhaps there exist relationships between a third entity and both of these entities, that would indicate an indirect link between the two. For example, the system might find that there are sentences that indicate a relationship between Al Qaeda and Mohammad Atta, and also other sentences that indicate a relationship between Saddam Hussein and Mohammad Ana. This could indicate an indirect link between Saddam Hussein and Al Qaeda, although you would have to read the document (or multiple 0 documents) to be sure. Note that Entity Link searches can be very slow.
UJ
UJ
Entity Link searches can help you find these third entities. In our example above, you could perform the co search:

co Saddam Hussein <> {Mohammad Atta} <> Al Qaeda UJ
which would return any instances in which Atta was found linking Al Qaeda to Hussein, or Saddam Hussein <> frpersonll <> AI Qaeda which would return any people found linking Al Qaeda to Hussein, or 1-d Saddam Hussein <> {[Name]} <> Al Qaeda which would return any named individuals that are found to link the two.
Note that if you use common words or a wildcard '*' here, many of the entities that are returned may not be particularly useful due to the commonality of the linking word. For instance, if Hussein and Al Qaeda were both linked with the term "country", you would probably find that in the documents the sentences referrc Pam. 11 elf ^).1 InFact - Help to entirely different countries. For this reason, results may be most helpful when named entities are specified. These types of searches also tend to take a long time.
Clicking on a result will display a list of the relationships that link the selected entity with either of the two provided entities. In each relationship listed you should see a reference to the selected entity and one of the provided entities. It may take more effort to establish an actual link between the two entities in your query, because the sentences that establish the individual relationships with each entity and the linking entity may be separated by several sentences, or found in different documents. Using this search may require more reading and attention at the document level.
Sample Queries Here are some more example queries that express some of what has been discussed so far: 0 =
Clinton > visit > [Country] ; War #
Date>2001 UJ
UJ
= [PERSON] AND NOT Clinton <> visit <> [PERSON] AND NOT Clinton; "White House"
co = (Bush OR Clinton) > travel > [Country] ; meeting OR (war AND report) #
Author="John Smith"
=
Bush > * > Putin A Iraq # Date > 2000 0 =
* > visit > "Hillary Clinton" AND NOT
B*Clinton co =
Clinton > {'} > Putin 0 UJ
Displaying results Results are displayed in a relationship table that is intended to present concise and abbreviated representations of the relationships found. Initially it may seem confusing, but once the eye becomes accustomed to the structure of the format the results are easy to scan quickly. The power of the InFact system lies in our ability to summarize these relationships effectively. The display looks like this: 1-d The results table contains three columns: a Source column, an Action column, and a Target column in that order. The first column contains the sources of each relationship, or the entities that are performing some action. The second column contains the actions that define the relationship, and the third column contains the targets in the relationships, or the receivers of the actions. These elements are displayed in blue text, a Pacre 15 of 24 InFact - Help represent the essential core of the relationship. You might also see additional information in black in any of the columns, consisting of adjectives or adverbs that modify the source, target, or action. The action, displayed in blue in the center column, is a link to the document where the relationship is found, with the appropriate sentence highlighted. (In some cases, the action may have a number after it in brackets. In this case, the same relationship was found in multiple places, and clicking on the link will take you to a list of these relationships.) In the above example, the United States is the source and there are a number of relationships based on different actions and targets. Each of the actions is a link to the document, with the relevant sentence highlighted.
In order to display numerous results in a timely fashion, and yet search the entire corpus, results are presented in chunks of data. As you page through results, the system will retrieve the next chunks on demand. The chunk size is limited by hardware constraints and is set by the InFact System Administrator.
You can sort the results across each batch or chunk by setting the appropriate default sort scheme as described in the Preferences section. Note that results are not sorted across all chunks, only the chunk most 0 recently returned. Generally, a chunk will contain multiple pages, although the maximum number of results UJ
UJ
in a given page is also set by the InFact System Administrator. By default the results are sorted by action = co similarity, where the actions at the top of the list are equal to or most similar to the specified action in the query. If the sort scheme is set to "Unsorted", search results will be returned more quickly. 0 co The number of results displayed on a page is set in the preferences and can be changed by the user. The 0 results in a given page can be resorted for easier viewing (apart from the chunk-specific sort setting) by UJ
source, target, action, similarity, frequency, or any available metadata fields by selecting one of the tabs at the top of the table. Again, note that these controls only sort the current page, while the sort settings on the Preferences page pertain to the whole chunk. When paging through a result set, the sort scheme will default to the setting on the Preferences page.
The "Export to text" and "Export to HTML" features are page specific. For more information see the 1-d Exporting Results section below.
Nested Search Setting a nested search allows users to search the results of a given search, that is, a "nested search". lithe results returned from a search are numerous and you wish to "drill down"
further within your result set, you can set this feature to restrict future searches to the set of documents associated with the currently displayed result set.
Paae 16 of 24 InFact - Help Nested search is initiated when the user presses the "Set" button in the main search Ul. There is a hard limit on the number of documents that can be specified, configured by the InFact System Administrator, the default being 10,000. If the current set of documents returned by a query is less than this limit, the controls for this feature are displayed on the page in the top right-hand corner of the result display table.
co New queries submitted will only be run against the most recent result document set at the time this feature is set When set, the fact that searches are constrained is indicated to you on the screen as each result page will say "Nested Search: ON" at the top. Note that the document filter does not continually and automatically reset itself to the set of documents returned after each subsequent search, but remains associated with the document set returned at the time when the feature was last set. Also, note that when using nested search the number of documents returned cannot be accurately determined and is therefore not displayed. The number of pages presented if the results exceed a single page is an estimate and may not correspond exactly to the number of result pages; the estimate is refined as users page through the result set. 0 UJ
Using the Query Generator UJ
(44 CO
The Query Generator is a user interface component designed to help new users write syntactically correct queries. This component can be displayed by clicking on the "Show Query Generator" link at the top right- 0 hand corner of the search input field. This component allows users to construct queries by simply entering 0 in the search terms that they know. When the "Build Query" button is pressed, an appropriate query string is generated in the search input field.
UJ
The only purpose of this component is to help users generate a valid query for submitting to the system.
When users press the Build Query button the query is not submitted, only displayed in the search input field. To submit the query, users must press the "Search" button. As users become more familiar with the query syntax, it is likely that the Query Generator will eventually become more cumbersome to use than simply typing the query in manually. Additionally, manually typing queries allows greater flexibility and specificity. If not desired, The Query Generator can be hidden by clicking on the "Hide Query Generator"
link.
To use the Query Generator, simply enter in any source or target entities, or an action in the top input fields.
If any of these terms is an OntologyPath or an ActionType, select the appropriate checkbox. The remaining lines and inputs allow additional clauses to be specified. Feel free to experiment and test out various co combinations to see what the queries look like.
Paae 17 of 24 InFact - Help Ontology Support Ontologies express type or class information that can be used to allow users to search for specific types of entities, like 'cities' or 'people'. During ingestion one or more ontology files may be submitted that function as a dictionary, mapping terms or phrases found in the corpus with terms or phrases within a hierarchy of oe classes that entities fall into. The InFact system uses a standard ontology by default. For example, during ingestion "Seattle" is mapped to the ontology term "city" by the InFact system. Therefore, if you are interested in searching for cities that Clinton visited in 1998, you could specify in your search that you wish to constrain results to only cities by referencing the ontology term "city" in your earch query:
Clinton > visit > [city]

The InFact system recognizes "city" as within the hierarchical path "IF/Entity/Location/City". This means that if a term in the corpus is mapped to "city", it is also understood to be a "location". Therefore if locations are searched for, any cities and other subpaths are returned along with terms that are specifically mapped to CO
"location".

Ontology paths can be used to dramatically improve search performance.
Customized ontology paths can be co included with any corpus. For more information, see your InFact System Administrator. For help using 0 ontology paths, see also the section Using Ontology Paths.
The following is a table of definitions for the standard ontologies. These and any custom ontologies included in the system at time of ingestion can be viewed and navigated on the Corpus Page.
IF/Entity/Location A geographical place.
1-d IF/Entity/Location/Address An address that denotes a location.
IF/Entity/Location/City City, as in a populated urban area.
IF/Entity/Location/Country Includes past and present nations.
IF/Entity/Location/Geoentity Any geographical Entity.
IF/Entity/Location/Island A body of land completely surrounded by water.
IF/Entity/Location/Province A province or state as in British Columbia, or Virginia.
Page 18 of 24 InFact - Help IF/Entity/Location/Region A geographical region of any size.
IF/Entity/Location/Sea A large body of water.
IF/Entity/Numeric Superset for all Numeric ontology paths.
IF/Entity/Numeric/Amount A quantity of something.
IF/Entity/Numeric/Fiscal Fiscal information.
IF/Entity/Numeric/Fiscal/Money Any references to currency.
IF/Entity/Numeric/Number Numbers such as 1, 44, 55.
IF/Entity/Numeric/Percent Percent values such as 22%, 22 percent, 35 0/0.
/F/Entity/Numeric/Phone Phone numbers including American and International.
IF/Entity/Numeric/Price The amount of money per unit.

IF/Entity/Organization Superset for all Organization ontology paths.
IF/Entity/Organization/Government A government entity as in State Department, Pentagon, Ul or Parliament.
c0 IF/Entity/Organization/Military As in Pentagon, Air Force, or NATO.

CO
The name of an organization as in Insightful, Microsoft, or IF/Entity/Organization/Name VVTO.
IF/Entity/Organization/Political Includes political parties.
IF/Entity/Organization/Trade Examples include: IMF, EU, OPEC, 1NTO.
IF/Entity/Person Superset for all Person ontology paths.
IF/Entity/Person/Name A person's name.
IF/Entity/Person/Designation A superset for all Designation ontology paths.
IF/Entity/Person/Designation/Post As in a post held, like: CEO, chairman, or president.
IF/Entity/Person/Designation/Rote Role a person may have like: sister, brother, father.
IF/Entity/Person/Designation/Title Same as post, but also Includes things like: Mr., Dr., or Mrs.
-a Includes female first name or title, as in Jane, Ms., IF/Entity/Person/Female Chairwoman.
Pane 19 of 24 InFact - Help Includes IF/Entity/Person/Male male first name or title, as In Bob, Mr., or Father.
c:, IF/Entity/Temporal Superset for all Temporal ontology paths.
co IF/Entity/Temporal/Date A date, as in 1945, or Sept. 11, 2001.
/F/Entity/Temporal/Event Historical or calendar event, such as Mardi Gras, Second World War.
IF/Entity/Temporal/Time Time in a day, as In 3PM, 4 AM, morning, or 5:00PM.
IF/Entity/Temporal/Time_Period Amount of time, historical or calendar period.
Synonyms (5) The ontology file associated with a given corpus can also define synonyms of doinain specific entities. Any synonyms of entities typed into the search query are also searched on and included in search results. The co InFacte system maintains a list of synonyms of commonly used terms, for more information see the Corpus Page section below.

co Exporting Reports You can export results to a simple tab delimited text file for easy import into external applications like MS
Excel spreadsheets or 12's entity relationship viewer. In addition, you can export results, along with relationship context information, as printer friendly HTML reports. The report will contain the detailed sentence information from each relationship currently displayed in addition to a number of surrounding sentences. The number of surrounding sentences may be set in the Preferences page.
1-d From the results of any search you can select "Export to Text" or "Export to HTML" from the list box. You can either review the report online, or you may dump the report using the browser's File -> Save As functionality. (When using some versions of Mozilla/Netscape, some of the formatting may be lost when exporting to text. You may have to take the additional step of displaying the page source, and saving it explicitly.) Corpus Page Pave 20 of 24 InFact - Help The Corpus Page provides corpus specific information that may be valuable to you in your search efforts.
This page can be used to search or view the ontology paths associated with this corpus, search for synonyms of terms found in the data, view ActionTypes available for searching, or view the metadata fields associated with data in this corpus. You can also get basic information about the corpus such as when it was ingested, by whom, and general information about what it contains.
= Searching the Ontology: If an ontology is associated with a corpus, it may be searched using the Ontology Search feature in the first tab. If a term entered in the text field exists within the ontology, any matching fully qualified paths are returned. The same term may be found in multiple ontologies.
By default the InFact Standard Ontology is available, identified by the root term "I=F".
If a unique path within the associated ontologies is found matching the search term, the resulting path will be displayed along with a list of its immediate children. The ontology may be navigated by 0 clicking on the children, or any of the elements within the ontology path displayed. If the search UJ
term is found within multiple ontologies, all matching paths will be displayed. Clicking on one path UJ
will display the children of the selected ontology node. Also, the terms contained by the selected path with also be displayed.
co co Ontology path terms or elements are separated by the '1 character. If desired, partially or fully qualified paths can be entered into the search text field, such as "entity/person" or "IF/Entity/Person". (Searches are not case sensitive, and any lowercase or uppercase results will be UJ
returned.) Ontology searches cannot use the wildcard character. Note that if search or ontology terms contain a '/' within the term, this creates an ambiguity that cannot be resolved by the InFactaD
system. If such terms exist in the corpus, they may still be viewed and processed, but they are "escaped" by the system by putting the standard escape character 'V in front of it. For example, the Genia/Medline ontology contains an ontology term "DNA_N/A". This term is represented within the InFacto system as "DNA NVA". Typing the latter into the search text field will return the correct 1-d ontology path. Typing the former in will return nothing, as the system will interpret this term as a path with two terms, 'DNA _N and 'A'.
For more information about Ontologies, see the Ontology help section above.
= Finding Synonyms: If synonyms exist for terms in the corpus and are defined in the ontology file, these synonyms can also be searched on, and are returned in search results by default. This can be Page 21 of 24 InFact - Help confusing in the result display, where terms that were not specifically seardhed upon may appear. To see if synonyms are defined for a given term, click the "Find Synonym" tab.
Users can enter a term in the search field and view all synonyms defined for that term in the corpus.
The InFact system provides a default set of synonyms for things such as country/state references, British/American spellings, and common adjectives (see InFact System Administrator for a detailed list). Synonyms are co automatically searched on and included in search results.
= Viewing Metadata, ActionTypes, and General information: General information about the corpus being searched is displayed along with metadata associated with this corpus and ActionType information. Note that metadata is only displayed if it is included in the IDML file submitted during ingestion. For more information about using metadata in searches, see the Metadata Filtering help section above. ActionTypes define certain classes of actions that are similar within the context of a given corpus, and must be defined by a InFact System Administrator before ingestion.

For more information about ActionTypes, see the ActionTypes help section above.
UJ
UJ

CO
Preferences Page co There are a number of preference settings associated with a given search that may be customized to 0 constrain your search results or improve result display.
UJ
The following options are available on the Preferences page:
= Include negated actions: when this option is enabled, relationships matching both the positive and negative sense of a verb are displayed. If you performed a search like "Clinton > visit > Russia", the sentence "Due to heath reasons Clinton did not visit Russia." would only be returned if this setting was set to true. By default Show Negated Actions is disabled, and only positive actions are displayed.
= Search modifiers along with entities: This option specifies whether modifiers should be searched along with sources and/or targets. In the above example sentence "Bill visits beautiful, green pastures outside Seattle", if this property is set to true, then a search like "Bill >
visit > Seattle" will return the above relationship. If this property is false, then it will not, and only the query "Bill > visit > pasture"
would still yield this result.
= Display modifiers: In the sentence "Bill visits beautiful, green pastures outside Seattle.", "beautiful, co green" is the prefix modifier for pastures, and "outside Seattle" is the postfix modifier. In a searcl=
Page 22 of 24 inFact - Help "Bill > visit > *, with this property set to true you will see the modifiers displayed along with pastures in the target. If this property is set to false, only the word 'pastures' will be displayed as the target in the tabular display.
= Enforce strict bi-directionality: When doing searches with bi-directional arrows, like your search can be interpreted in two different ways. For example, with the search query "Clinton <> *
Bush", one might wish only to view results in which Bush did something to Clinton OR Clinton did something to Bush. Enforcing strict bi-directionality does this. However, you might also wish to see instances in which Bush and Clinton both did something to some other target together. These results are also displayed if strict bi-directionality is not enforced.
= Search ontology path name as term: If a user includes an ontology path like "[city]" in a search query, then results with cities are returned. However, the word "city" is not a city, and is not associated with the ontology path. Therefore, you would not see results that contain the word "city".
If you wanted results with the term "city" in them as well as any terms defined by the ontology path "city", you would set this preference to true.

= Number of relationships per page: The user can set the number of relationships to display on a UJ
UJ
single page of relationship results. The smaller this value, the faster results will be returned.
=
Number of documents per page: The user can set the number of documents to display on a single co page of document results. The smaller this value, the faster results will be returned.

=
Sort scheme: This setting allows users to sort results in a given chunk or batch of results according 0 co to one of several sorting schemes, and to set the default sort scheme for all future searches. Note that 0 an individual result set can also be sorted in the result display. If results are sorted using the drop-down selection box on the results page, the setting does not persist for subsequent searches. For more UJ
information, see the section Displaying results above.
= Surrounding sentences to export: This option allows the user to vary hoW
much contextual information from the document is included along with the sentences returned when the user exports a result set to HTML. Each sentence contains a relationship.
1-d History Page The history page provides you with a history of all searches performed in this browser session. If your browser dies, if you use another browser, or if you press the Clear button, the history will be reset.
When you click on the link for any query in the Query Specification column, you will be taken to the results page for that query. If you click the link in the Documents column, you will be taken to the set of documents that contain the results of your query.
.13 (IP 11 (f-,1 InFact - Help The "Depends On" column indicates whether a given query depends on a previous query. This happens when a user performs a search (A) and then sets the nested search feature, and then performs a second search (B), or subsequent search (C). In these cases, the queries B and C will appear in the history page with their IDs reflecting a dependency on query A. Note that if you click on the link for a query that was co dependent upon a previous query, the nested search feature will remain off.
co Summary The current web page layout, search functions, and result set displays are examples of what can be done with InFact@ search. If you would like to see an InFact search interface tailored to your knowledge domain, contact your Insightful sales representative or Insightful's Professional Services Group at:
consulting@insightful.coni Thank you for using InFact . For further assistance, please contact us at support@infact.com.
UJ
UJ
Go Back and perform a search!
co co UJ
Pq fn.
M.' 1.1 APPENDIX B

QUICK TOUR OF
INFACT 3.0 Introduction 2 Note 2 Acknowledgment 3 The InFact Interface 4 Event Search 6 Overview 6 Entity Search 19 Relationship Search 21 Direct Relationships 21 Indirect Relationships 22 Corpus page 24 Ontology Search 24 Synonym Search 25 Corpus Information 25 Preferences 26 Include Negated Actions 27 Search Modifiers 27 Display Modifiers 27 Enforce Strict Bi-directionality 27 Search Ontology Path Name as Term 27 Number of Relationships per Page 27 Number of Documents per Page 27 Number of Documents per Page 28 Sort Scheme 28 Surrounding Sentences to Export 28 For Content Publishers 29 Quick Tour of InFace 3.0 INTRODUCTION
InFact 3.0 is designed for the dedicated knowledge worker whose mission is the analysis and production of intelligence from human language or linguistically based information. InFact is not a keyword search, and it is not for casual consumers of information.
InFact's mission is to drive the knowledge worker from keyword search to event discovery. InFact extracts events and relationships from documents, not Just entities. By capturing relationships, InFact empowers the analyst with the ability to discover and track activities.
InFact can produce concept maps summarizing vast amounts of information across many documents related to a given person, place, or entity, so you can quickly zero in on what you are looking for. For the intelligence analyst who is typically overwhelmed by volumes of heterogeneous and noisy information sources, this is a new and efficient way of navigating the information sources. At a high level, InFact can provide a bird's eye view of all activities involving one or more entities. The user can narrow the search for a particular type of activity, obtain cross-document sentence summaries of particular events, and hypernavigate from the sentence summaries to the context of the document(s) in which the action is originally described.
No other commercial product provides this level of accuracy, performance and capabilities. InFact features include the following:
= Discover relationships among entities.
= Discover actions involving an entity.
= Search by keywords and concepts.
= Highlight answers within paragraphs.
= Create or modify searches based on sentence structure.
For help or more information, contact search@infact,com.
Note The following pages are meant to illustrate the functionality of InFact . This document is not a tutorial. Search engine results may vary from those presented in the screen shots contained in this document.

Quick Tour of InFace 3.0 InFact is currently certified for use with Microsoft Internet Explorer 5.5 (or higher) and Netscape Communicator 7.0 (or higher).
Acknowledg- The data source for all the document text is the Reuters Corpus, ment Volume 1, English language, 1996-08-20 to 1997-08-19 (Release date 2000-11-03, Format version 1, correction level 0).

Quick Tour of InFact 3.0 THE INFACT INTERFACE
The InFact user interface is easy to use. The different search interfaces supported by the system are all linked to the main page, as shown in Figure 1.1.
InFact - Search I Corpus, PreferencesI
Fjjg_e_ot I Litt dookimpowo--Show (Nary Clentratoc Iw Figure 1.1: User interface for InFact 3.0 Copyright 2001-2004. All rights reserved.
The InFact system supports document keyword searches, as well as more powerful and flexible searches based on specific queries. Users can enter in a keyword or key phrase for document results, or they can enter in a relationship query. that conforms to InFact Query Language (IQL) syntax, and press the Search button to view results.
Both types of searches are discussed in this document.
In addition, four other links are available to help you refine and improve your searches, or manage your result display. The Corpus and Preferences pages are explained in detail later in this document:
= Corpus The corpus page provides you a way to view information about the corpus of documents being searched, including any custom ontologies submitted, metadata associated with the documents, corpus-specific synonyms, and any ActionTypes defined.
= Preferences Setting preferences provides you a way to customize the user interface, and constrain your search results.
= History The history page provides you a way to view and navigate back to the results of previous queries .from the same browser session.
= Help Allows you to access the help system.

Quick Tour of InFact 3.0 When using the interface for the first time, you can get help constructing relationship queries by clicking the Show Query Generator link. This will bring up a Query Generator component that will help you build a valid query. Once you are familiar with the syntax, you will probably find it easier to type in the queries directly.
However, while getting started the Query Generator makes it easier to see how queries are constructed and how the system works. In this document we will not discuss the query syntax in detail; for more information consult the online help which contains more in depth explanations and examples.
InFeir,tP Search i giu.L...is 'Preferences I Lts_t_ca 1 i-j212 ifid., <1..1, Genvator -Query Generator - Specify Relationship:
n-gow:-....041 ki ____________________ -.04-vismimmu_..441:-....:-.
õ41:2111.110.410 3.
yJ -,,,,,/,Wr...:',2' . ,..*57..
r11:11'1 TV'. r., eq.e 4 "-;,'''.-P, 7,..:_W:,,,4.SV,' Po j).7- -42.,..:::FrUiEZ":4-7.44:7,e7,--WH,-Sier--17.24, ', = %
e 4:44. -.-- ,i -44,&.,... A: : .
,-V--:,-, '''" '= - ¨
.00.1-i.ft.

-;,, ;..'a. t704Auttm¨r ' ,--is,--e-..--. ej ta '11111111111111111.
.µg ctrilA1/4 Cal _______ ,liVt-, - ,,,,....;=4t,irtt=47. '. "_ _ 4-i' ....= : .= -. ret,i Author 2 I is e .uatto , r'41ekt4rvf-'0-4: 4.,_. ¨ - i ., .- =
,..-7?-f.-t"Pr,-,--v .
, .oe forea r= - = - --ito-.9.,.. . se 4. _ ... r - . ..t,.. . = -, II ¨..1,44,vv.- - : ... - ,...,..-.. ,:.: 44,- 4-..,4 .. ,õ.
¨
eril.***-4-2.,,, .--4 --t'" - IA .'fa...:.,... 3,!:,.... ,-.4t. ::,:vv.74,450.õ.--P".¨N
___________________________________________ -.-wilywait. .-trz- ki2kg,-A, -.."--;! i" -;,,..2. of ' ;' - -',...- - -- = ',:IP'.:'';µ'r.,..--',:=-',;. ',,,,,:-c_'=:,'-`4 `.-...,-Z.:-;', -::,-. 4-;', ,. :'',-', .- <1 r.,:'- ... -_,:::. '-*'" ;",,,.-;. '..
....',:i','''-..:, , -.. - ; ; ...; 2.,,`,..i' " - - - '' ' - ' '"' - -= , ' = = ¨ ' '," - - - ' .--', ._..; izi.-%,-. ':'-':;.-- c.i' ;
' :'.,=,..-...,;':
:. Ent e'r:11601Y4::i _ _ _ _ _ _ ._ i-Ai',-..A's s!,-,1?-1,51..
....i. r,- ': 2.1 ,..:,-- - ..,;:-,;`. , _.,. .?,.-,:"....--.;--,;i4;=-====:,,-.2.- .:,; .;',.,' : -: ,..- ¨ .."-= ; = ''''`.::.--- -,'':::=, ":''= ,:,-;..44." =
Figure 1.2: The Quety Generator component.

Quick Tour of InFact0 3.0 EVENT SEARCH
Overview The InFacte system provides a way for you to find documents based on a keyword search, or information in the form of events or relationships between entities in a given corpus or body of documents. To see an example of how the system works, simply type a keyword into the search input field and press the Search button. For example, let's type in "china buy" and execute a search. You should see the display presented in Figure 1.3: =
InFacto = Search I
Corpus I Preferences I HAtom I Help.
401011"1"1".-Show Chary Gsritrater rrnabuY LI
= __________________________________________________________________ Basic document level search results are displayed. To get relationships, try:
thins Ai<> = <> = - retuins all relationships involving chiaa bor china err > * > = - returns all relationships vhere cti.te boy does something ch(es Iror < = < = = returns all relationships where something is done to chi:., boy Document results 1 - 50 of about 165: Page 1 of 4 Next = ...=;
1) Fels-CHI-94-101 Daily Reoort 1 May 1994 Date=1971.01.10-05:17:14; Author...Jimmy Carter; Price=285; 0-eationDate.-2003.04,13-18;5709 ".., In particular, they know that in order to have a share in the highly competitive global satellite-launching market, China has to win launch contracts at a low price ..."
2) Economy; Consumer Confidence Hits 17-Year Low. Out Retailers Say Fight Is Far From Lost --Buyers Aren't Charging In. Stores Note. but Things Could Be Much Worse -- By Lawrence Ingrassia Staff Reporter of The Wall Street J
Date=1971.01.10-05:17:14; Auttrcv----Jimmy Carter; Price--,265; CreatiorDate--.2003.04.19-13;16;42 ,^ specialty store for china, silver and home accessories, company President Bruce Meyer describes the sales drop as ...=
3) International: Yugoslav 'Tourists' Flood Into China, Pack Their Bags After They Get There - By ¨

James McGregor Staff Reporter of The Wall Street Journal Date=1971.01.06-19:00:27; Autttoreratd R. Ford; Price=284;
CreationDate=2.003.04.16-18:15:16 = BEIJING -- As the centerpiece of the new $450 million World Trade Center, the China World Hotel was envisioned as an elegant refuge for refined tourists, wealthy traders and globe-trotting tycoons...' Figure 1.3: An example of a document keyword search.
=
The results are presented as a list of document titles, with a description of the metadata associated with the documents, and a brief excerpt of text from the document containing your keyword or phrase. If you click on a title, you will link to the document.

Quick Tour of InFact6) 3.0 This provides a straightforward means of determining what Information is contained in your corpus. However, you might get too many documents back to review efficiently. Also, note that the second result here refers to china home accessories, which is not what you want. Another alternative would be to use the InFact Query Language to specify what you want more accurately in terms of events that occurred or relationships between different entities.
To conduct an Event Search, you need to specify a query and submit it to the InFact system. This involves understanding a bit about the InFact Query Language.
The query is based on specifying a relationship between a source entity and a target entity, involving an action. The source entity is the performer of the action, and the target entity is the receiver of the action. For example, in the sentence The United States searched Iraq for weapons of mass destruction.
The United States is the source of the action "search," and Iraq is the target. The core source - action - target relationship expressed in this sentence could be represented in a query form as:
United States > search > Iraq If you enter this query into the main search input field and press the Search button, the system displays a table with all the relationships that match the search query. The relationships are displayed in three columns, with the sources, actions, and targets highlighted in blue text. The action is an active link, which when clicked on takes you to the sentence and the document where the relationship between the source and target was found.
As an example, lets type into the search input field:
China > buy > *

cg) Quick Tour of InFact 3.0 Here the asterisk means that we are not specifying a target; we are interested in all target entities. Now press the Search button. You should see results displayed in the table as in Figure 1.4.
InFacto Search I
Corpus I Preferences I History I Help Show Gum Generator '',4i'4444440china>buY>* [.^
magifiggAi Relationship results 1 - 99:
Si -I = -ffit5finittanOtra;MV.:,511145MatiaitiraN
WIT470343;^=VINM:MfInP.J.Wiat720NR4iNikWik-3,7eMM:Stff 1 china buvf21 u.s, wheat additional I china One million ! metric ton : of wheat people : in china buy : at will various : kind : of food 60,000 : metric ton : of refined china b_L4 Sugar china travel agency of hong Long stock hong kong china travel agency china navigation subsidiary : of british shipping buy at cost of pounds 37m each three : of yard capesize vessel company john swim son china 250,000 : ton : of refined sugar soviet union chinath.ti everyone : waiting buy : in a I few large autumn last china us wheat purchase china potential buy : in world market -chinese airline bav. 757 : boeing growing number: of well-to-do buy : for private use car chinese Figure 1.4: An example of an Event searrh.
As you can see, this search produces a list of relationships found in the documents that involve the country China buying something. In addition to the verb buy, other similar verbs (e.g., acquire) would be included, if you scrolled down the display. Also, contextual Information such as modifiers are displayed. You can choose not to see this information by specifying the appropriate parameter in the Preferences page.

Quick Tour of InFact 3.0 You can sort your results in different ways by selecting one of the following tabs. For example, you can:
= Sort by Action - sort by action verbs in alphabetical order.
= Sort by Frequency - sort by relationship frequency.
= Sort by Similarity (the default) - sort by action verbs based on their similarity to the query action.
= Sort by Source - sort by the Source in alphabetical order.
= Sort by Target - sort by the Target in alphabetical order.
= Sort by Date - sort by any date information associated with documents.
= Sort by other metadata - sort by any other data associated with the documents during ingestion.
These options only sort the current search results. You can set a permanent default sort scheme by setting a similar parameter in the Preferences page.
From the search results in Figure 1.4, clicking the entry in the Action column links to a view of the document where the relationship was found. If we click on one of the document links, the document is displayed with the sentence highlighted. For example, click on the relationship where the target says "757: Boeing". The resulting relationship is displayed in the context of the document, as shown in Figure 1.5.

Quick Tour of InFact(6) 3.0 ut induetry is also laying viiiit'.aPPettra to hen'S41.1d'.foundation for =:
oxpansion.: SP, foe, exomple ,con-criortad an agreement in /Imy. this year ;or a 51 per cent stai.te";in, a: 0P4itara. idcipi:tetle'acic(pettir Pr*ince;.. Coats ViyelLa has invested soe itPollars 100m in a web of Mills, mostly cOnCentrated'near ' east Of Reii tag; Onii*Cr: through its 9elLs ice cream,group as ant layfe,d !JDolLar. 59n to establish a factory and diatribtitiOn network in 0eijitigl,BTR is investing abont. CDollara '90 in a .-=
bottling plant near Guangzhou in southern China i Pilkington which Was involved in the establishment of e'float glass project in Sbnagnia in 1903, is engaged in three other projects.
Companies such as Rolls Royce 'which have been exporting to China for Many Se*cs, are also doing well. :
The OR company recently von a big order to Supply 42 of ita,RB211-525 jet engines for 'Boeing i.9.7.5 bought by Chinese ' airlines. Rolls Royce Says it is Confident of winning further ordera.
But. ns the China market continnee to npen, 'so does competition become more intense, with the American., staking a alniin for a bigger shAre. The 'rigout high profile visit to Beijing of 'hr Tnn Brown, the.115,"CoMoetce Secretary at the head of a delegation of 24 chief executive officers of lending American companies is just one indication of ,an:intenaifying= iT5'iocus on Chinn:, As Sir Nichael Palliser of the CBTG says : main iiviry is not competition from %other iuropeans, but from the thundering herd ,of 'Americans coming in..
IOn a general economic front, China is redoubling it., efforts to contain =
inflation with a new campaign to Curti rises in the priOes of grain, cotton ----- -Figure 1.5: The document where a relationship was found, with the sentence highlighted.
The action you specify can be a Specific action, or a Type of action. Action types encompass several different verbs and can be used to broaden a search. For example, rather than searching on the verb talk, you could search on the ActionType communicate, which would include not only talk but also similar verbs like speak or tell.
Although the InFact system includes similar verbs in searches by default (as we see later), the action type may provide a more powerful means of expanding searches, particularly if you define your own corpus-specific action types. ActionTypes are are put in brackets when used in a query, like:
United States > [communicate] > Iraq Remember that you can only use ActionTypes that have been defined for the system. You can see what ActionTypes have been defined on the Corpus page described later in this document.
Similarly, you can improve your search by a specifying an OntologyPath instead of a specific entity. Ontologies associated with the documents in the system express type or class information that Quick Tour of InFact9 3.0 can be used to allow users to search for specific types of entities, like 'people' or 'cities'. InFact supports a number of standard OntologyPaths. Examples of some of these are listed in Table 1.1.
Table 1.1: InFact StandardOntologyPaths OntologyPath Subsets IF/Entity/Location Address, City, Country, Island, Province, Sea IF/Entity/Organization Organization Name, Military Organization, Political Organization, Trade Organization, Government IF/Entity/Person Name, Female, Male, Designation IF/Entity/Numeric Number, Amount, Phone, Fiscal, Price, Percent IF/Entity/Temporal Date, Time, Time Period, Event Note For a complete list of the standard OntologyPaths, see the Help link in the Web site. A table is provided with explanations of each. Most of the entity types are intuitive. In addition, an InFact System Administrator can submit one or more custom ontologies with corpus-specific OntologyPaths.
OntologyPaths must be specified inside brackets to make a valid query, like this:
United States > search > [country]
For example, imagine you want to see a list of corporate acquisitions.
Specifically, you would like to see information about how much money was spent. To do this, we will specify "[Organization/Namel"
as both the source and target entities. This is an OntologyPath defined Quick Tour of InFact 3.0 by the InFact system that references any organization name. We'll use "buy" for the action, and specify that an amount of money should appear in prepositional phrase near any relationships found:
[Organization/Name] > buy > [Organization/Name] A [money]
Also, we'll sort the display by the dates associated with the documents where each relationship is found. To do this, we'll go to the Preferences page and select "Reuters Date Published" as the sort scheme. If you press the Search button, you would see results as in Figure 1.6. Note that the dates are displayed on the left.
=
InFact- Search 'Corpus I Preferences I i-listory I Hells 400,100w4"--Show Ovary Otoeratot iitaiiiieV.Ifonganization/name]>buy>rorganizatiort/namel^(rnortey]
Relationship results 1 - 500 of about 2237: Page 1 of 5 next dtig httritini JJ
ReutersDite.j'Ubllshed SciUrCg C7) ACtriori;O:. ft'rget , wash, marysville 20/08/1996 crown pacific partners buy sawmill : from garka mill co I.p.
inc for $2.7 million merv griffin television.

surt international buy,: for $1110 million in celebrity : griffin gaming hoiels ltd stock entertainment inc 20/08/199 manufactured home buy : for $307,3 million in rival : chateau properties communities inc cash inc lth of paStech aluminum commonwea 20/08/1996 acouire group, Inc for Share $20.50 aluminum corp =
in cash buy : under revised term for $5.8 million cash 400,000 20/08/1996 cerprobe corp share of cerprobe corp compuroute inc assumption of $1 million in long term debt -kmart corp automotive penske,aUto centres 20/00/1996 octisber last: acquire, service : centre : for:
$112ailliari commonwealth 20/08/1996 buy : for $272,7 million castech aluminum group aluminum corp inc .effective SO percent nteret: in pr.6µ4fort( =
. . . , 'En4tinntinIrlinnerirvl Figure 1.6: Event result example in which companies were acquired.

Quick Tour of InFac? 3.0 In this example, note that we added a clause to specify that we wanted to limit our results to those that mentioned money. In addition to specifying a source, action, and/or target, you can constrain your query by adding one or more clauses. For example, you can specify that a given term should be contained in a prepositional phrase near the relationship. You can also restrict your search to documents that contain a given keyword, or where the metadata contains a given value, such as a known author or date. You can see what metadata is associated with the documents in the corpus by going to the Corpus page. Here are some examples of how constraints are used:
United States > search > Iraq PREP CONTAINS Baghdad United States > search > Iraq DOCUMENT CONTAINS weapons United States > search > Iraq METADATA CONTAINS Date > 1990 These queries could be combined into a single query with three constraints. Also, to save time typing, we can replace PREP
CONTAINS, DOCUMENT CONTAINS, and METADATA
CONTAINS with the abbreviation characters ' A ',';', and V' respectively:
United States > search > Iraq A Baghdad; weapons # Date>1990 or more information about constructing queries, see the online help.) Now in our example, imagine that you were only interested in companies related to oil. You could add a clause that restricts the search to documents containing the keyword "oil". This will only return results from documents mentioning oil. Here is what it would look like:
fOrganization/Namel > buy > [Organization/Namel A [money] ; oil If we re-run the search, we'd see something like Figure 1.7:

Quick Tour of InFact 3.0 InFact- Search I Corpus I Preferences I
History I Ella Show Outrv Cicintrator *.iiiir'.44774,;1(organization/namel>buy>totganization/nameJAImoney); oil l',..:.i,11,"Oig;'*.V114 Relationship results 1 - 114:
Ala%Rae rsDb3 :r :i owned privately : woodward-clyde 19/08/1997 UrS corp. ,aSAire group inc. : for $100 million in stock cash monterey resources inc. ________________ ; for $1C16:
18/08/1997 Mbfartand energy inc monterey Independent oil producer:
18/08/1997 texaco inc.
monterey resources Inc.: for more hgvy.
than $1.1 billion in move califomia monterey tesoUrces inc. buy,: for $106 18/08/1997 mckarland energy Inc.:
monterey 13/08/1997 zapata corp Puy : for share $8 envirodyne industries inc rand,merchant bank 1 12/08/1997 acquire-. about 35 trading firm ; exatrade : for 45 equity investment -percent of million rand vehicle rmb ventures commodity 1 07/08/1997 meteor industries inc r(2-11-TLhfur $5 rni(ii n fleischli oil company inc I 28/07/1997 gulf,tnada resources . stamppder exploration Rdin acquIre friendly stock-swap worth about ltd c$1,0 billion gulf canada resources Figure 1.7: Event result display showing oil-related company acquisitions.
Lees do an example with an ActionType. Motion is one of the InFact system defined ActionTypes, and it defines a number of different actions related to motion:
Pope > [motion] > [location]

Quick Tour of InFact 3.0 Note that OntologyPaths do not need to be capitalized. Figure 1.8 shows the results of this query when executed. Note that in the Action column we now see a number of different verbs defined by the ActionType "motion".
InFact Search I
Corpus I Preferences History 'Help Vvoir Outtv gentratac .:iV,4:figequerAfipope > [motions > (locatiorA flattiar.:
Relationship results 1 - 100 of about 362; Page 1 of 4 is Action Similarity itti Virµpfa, atIMMEIV 14.ttazi4eagsgOltrittirg an;
SM'7,1,3 1 pope 9g supersonic to Zambia entourage leave; : for lesotho in Sphanrtersburg : jan leiuta eirpert POP? convoy of car IPope dc-9 alitalia : plane leave rome pdpe tge vision : of united Western etirtilie to frence Ipope meet cuban envoy pope meet president : of el salvadar pope meet libyan 2 : no, pope meet : in zimbabwe south african : bishop pope send message : to fithuanian during one of pope reetirig Popp on pope way to lesotho to ?yr : flight : over vaticart swaziland Pope send representative : to warsaw PoP-9 Make pope fourth: tour: of africa in september Ipope : john paul fl : to monday french indian ocean : island Figure 1.8: Event search example demonstrating use of Action ljpes.
Another feature of the query syntax is that you can also use Boolean operators like AND, OR, and NOT. (Note that NOT must be used in conjunction with another term; it is not allowable to simply specify "NOT Israel") For example, here we are searching for suicide attacks that killed people anywhere but in Israel:
suicide AND (attack OR bombing) >kill>* ; suicide NOT Israel We'll run the query with a simpler final clause:
suicide AND (attack OR bombing) >kill>* ; Israel Quick Tour of InFac? 3.0 Figure 1.9 shows what the results would look like.
InFact- Search !Corpus I Preferences 'History I LlialA
gficne Chevy <3enerater Suicide ANO attack OR bornbi > > = = israel Relationship results 101 - 200 of about 275: Page 2 of 3 Eau &est t Adon Simil. v0-510. Ø0.400015.44.2sstliatek463101.404,ie --double suicide: bombing killf51 13: people three israeg:
cide : bombing Vioinan :
in tel aviv.on .
stri friday march 21 tel aviv suicide : bombing killt4 three israeli : woman suicide ; attack year last : kill(31 59 ; people : in Israel suicide : bombing killf31 57 : people palestinian suicide : bombing last week : ki11131 three : woman : at tel aviv cafe two suicide : attack ki11521 bomber Iseparate raid following twin suicide : attack day earlier: 19:1112.1 wcist 13 ; people : in market in jewishjerttatem suicide : attack kitlf21 scores : of Israeli islamist suicide bomb : attack kitlf2l three woman : in tel aviv on friday apparent suicide : attack kill(21 1 'suicide : bombing killf21 three : israeli suicide : bombing killf21 15 ; people : in jerusalem Figure 1.9: Event result page demonstrating use of Booleans in query You could also search for events involving people. Imagine that you want to search for people who had met with Gaddafi. This could easily be expressed with the following query:
[Person/Namel>meet>Gaddafi However, Gaddafi has several known spellings. In order to widen our search to incorporate as many of these as possible, lets use the wildcard character instead of specifying his name. This tells the system to search for any person whose name starts with 'g' and ends in 'It':
[Person/Namel>meet>g*fi Quick Tour of InFact 3.0 If we execute this search, we'd see this:
InFact- Search I Corpus I Preferences I History' 5hor Liam Gineator etr4,I#AV:E/4.16ft:
'..csonfnamekvisit =fi Z MININtrk4, = = 4, 4Fmte=r= if' = 1-41.]
Relationship results 1- 13:
WWII Re uter6- - '!de 11::"n31-31AIRRatil KWAVITSWAO,WAMIMP-4Sral:q.
=
excellent : talk : with 19/03/1997 mobutu sese seko muammar gaddafi official _ .
09/0,54997 klOal meet.: on arriµial inoamMar .gacecrdi) .sadvabacila senior official 09/05/1997 sani abacfia wet ; at airport muammar gaddafi local : people Of3/05/1993 hen ati rn,eit.:: during 03ya ofibuyaniarimleal.adgaerd:d.
13/04/1997 russias zhirinovsky meetf 21 : in Rwa gaddafi 13/04/1997 viadirnir zhidnavsky meet muammar gaddefi controversial : u.s. nation : of meet 08/01/1997muammar gaddafi Islam louis farrakhan leader 07/01/1997 farrakhan malt Muammar gacklafi fibyan leader :
07/01/1997 vladimir zhirinovsky meet muammar gaddafi excellent : talk : with 27/12/1996 mobutu sese seko pall muarnmar gaddeft 26/12/1996 alpha oumar konare twice : meet muammar gaddafi cot 6 : meet : Oil iStallliSA prime minister fitryan leader :
06/10/1996 controversial visit to necrnettio eft:taken, muarnmar gadclafi FbYa cr 20014004 asigetthd Corporatiort. All riotts resd-vol =
Figure 1.10: Example of a event search involving a person.
The InFact system also allows you to export search results to a report.
From the relationship display a drop-down menu at the top of the results table supports two options. Export to HTML exports results, Quick Tour of InFact 3;0 along with relationship context information, as printer friendly HTML reports. Figure 1.11 shows what these reports look like for the query "Boeing > buy > McDonnell Douglas".
Source;
Addeo:
Targ(it; cr..60:04410.0(P.
USA: PikESs'DIGEST - New Vert< times business -Mast 13, Fred -t4i,ayer tno has agi'eeittõ to acquire Smiths rood pr.uictirte!s. Inc for about ,M1,11,t if400 110Cgtnqieitliftl'#võIlthAa.-.40:4TittK*905.4kpgr:vo*100.f.cgf,p1***.ftl*A:(n:
'octootilayfii to.:*ftt roti*iveopfccar fprie_birso,,ow*pgruv zoccqtat The itoltar fell against the yeti...reaching itsleviest'kniel in nearly four months.
Source: 119E041 fon Action: boje. =
Torget:, ifiCidantieltidoticias USA: PAESS - Street Journal - Dec 19, The SECS top mutual fund offici4unjed-vigi!atice en4Morittio,asemt,Speach_.p.F9tyliqudirtiqfy despite its biggest year:ever. 1:1=11Apgki;fithfktil4 r4r4Otrit's,W4tiktfac.I.P*T.4110 Technoloily stocks- suilled, piisring'major Criddraii'tiigii4r:: = "
Figure 1.11: Example of a tut export to HTML, Export to Text exports the results to a simple tab delimited text file for easy import into external applications like MS Excel spreadsheets.
You can specify the amount of context you wish to see around the sentence that contains the matching relationship by selecting the number of Surrounding Sentences to Export in the Preferences page.
= 18 Quick TOW' of InFace 3.0 ENTITY SEARCH
The InFact system can also be used to search for entities in documents. There are two different ways to accomplish this. If you want to see all the relationships involving a given entity, you can specify a relationship query where only a single entity is specified, like this:
Bin Laden <> *
The asterisk indicates that you are interested in any relationships, based upon any action, where Bin Laden is either the source or target entity. If you execute this query, you would see a list of relationships as shown in Figure 1.12.
InFact- Search I
Corpus I Preferences I History I Help Pow, Outry aentratof 'Y,f0i.li;--,n laden<>.
ilEMWEERP:e.;:ill,' -Relationship results 1 - 161:
taiir-farg et - --'' ;:j1,12413#4-ESIMIONTiinf-)Z-(2-.elkartreartaMY ,1 ia,:i4Na!,:Afig41:024:-IC:;'1::R-A--...:'-fat:!ZAI:kAW,741;L'. g:0,!, involvement osania bin laden deny : in afghanistan in two two : indian : in bombing : in saudi arabia osama bin laden wound more than 400 .: peoples in afghanistan : as base for action bin laden tag against cceuntry american i of renewed onslaught bin laden Yteril against force in saudi arabia bin laden 543Ctre p Suportaden : of thousand of pakistani for bin l campaign mullah Mohammad omar talebart islarnic purist first public :
:
call be disapproval: of idea of bin laden clear: reprimand some middle east : expert estimate 1 at $100 million bin laden : fortune .
Figure 1.12: An example of an Entfty searrh.

0 . = .
=
Quick Tour of ItiFact 3.0 Alternatively, you can perform a document search on a given entity, as discussed in the beginning of the document. Any term can simply be entered in the search input field, and the Search button pressed.
The results are different for this kind of search; as shown in Figure 1.13, where we are doing a document search for references to Bin Laden. .
In this display we see a list of documents with references to Bin Laden. The metadata fields and values are included with each document, as well as the first sentence where Bin Laden was found.
The title of the document is a link to the actual document.
In Fact - Search I Corpus I Preferences I
liLtmat I 1111111 pow Outer Goner/Jur :'?4,*"74:1.57.fti^itQt'-:'ft:Vq:,4t-$.-P:'$'',;::µPrY::::f;I::;7...7:e,M71 instl)...;,*4===11'illiMtain laden li'..:;:"a=!..-',n7;;.e..I'....:
Basic document level search results are displayed. To get relationships, try!
Pet /*deur <> = <> = - returns all ft14014nships involving tgo fed.
Or itihk= > = > = - returns ail relationships where 144 ladca does something Per INA." < * < = - return; all rotations-hips where something is done to MI h=dert Oocument results I - 20 of about 72: Page 1 of 4 as unl d44 -.-----MaiffriarriAl;tXt...:AMMCJMETr 'Li 2-:,,"*X.,:lati i 1 i1x):.aAF2tAp1STAt:TIleba-noto pressure bin aienZleave A hr n, sh/03a5"42n%ws4L; Lyiine7msReuters Ovyrightqc) Reuters Limited 1, '1997;Reuters_Creator Country-AR-4-i. AN/STAN;
Reuters_Creator_Location=MBUE; Reuters_Datekno--KABUL
11997-03-05; Reutersikadline-Taleban not to pressure bin Laden to kave Afghanistan, 2) DUBAI: Saudi dissident Bin Laden moved to Kandahar - paper, f Doo-^ath-1997104081495530news14.->=1; Reuters_Byirhe--UNKNOWN;
Reuters_Copyright--(c) Reuters Limited 1997; Reuters_Creator Country-DU8,41; Reuters_Creator Location-DUBAI;
Reuters_Datelino-OUBA11997-04-05;
Re Headline-Saudi dissident Bin Laden moved to Kandahar -paper 3) AFGHANISTAN: Afahan Taleban resist pressure to excel Saudi, DocPath-1997/0327/472678newsM1.4cm1; ReuWrs_Byline-.Tim Johnston; f:euters Copyright-4c) Reuters Limited 1997; Reuters_Creator Country=4FarrINISTAN; Reuters_Creator Location-KABUL.;
Reuters_Datetine-KABUL
1997-03-27; Reuttvs_Headline-A(ghan Takhan resist pressure to cape! Saudi.
(Reuters) -A senior Taleban official said on Thursday that they will not bow to any pressure to expel Saudi dissident Osaina bin Laden from Afghan territory they control 4) AFGHANISTAN: Saudi dissident moves to Afahanistans Kandahar, PocPath-1997/0410/500397newsMI-xml; Reuters_Byline.,Tim Johnston;
Reuters_Copyright(c) Reuters Limited 1997; Reuters_Creator Country-AFCPHANISTAN; Reuters_Creator Locadon-KABUL;
Reuters...Dateline-KABUL .
1997-04-10; Reuters...Headline-Saudi dissident moves to Afghanistan's Kandahar.
>KABUL, AFGHANISTAN, Apr 10 (Reuters) -The Taleban administration confirmed on Thursday that Saudi Arabian dissident Osama bin Laden, who is fiving in Afghanistan, had moved to the southern city of Kandahar, the Taleban's decision-making centre Figure 1.13: The results of a document keyword searth.

Quick Tour of InFace 3.0 RELATIONSHIP SEARCH
The InFact system supports a number of means of extracting relationship information from documents.
Direct A direct Relationship Search is when you are searching for a Relationships relationship between two known entities. In some cases, you may wish to specify a verb, and in other cases, we may not want to specify any action at all. For example, let's search for any relationships of any kind between Bin Laden and the Taleban. Since we want to return all relationships between these two parties, we want to make sure to specify that either entity could be the source of an action or the target of an action. To do this, we'll make sure our arrows point both ways:
Taleban<> * <> Bin Laden Now when we build the query and execute it, we'll see the display in Figure 1.14.
InFact - Search I Corous I Preferences I
History Help Shaw Owen, Oena,ator iileban<>*.c>b1n laded Relationship results 1-10: =
"wad L4. 1144,00M-VR-Mfe. ELTFA...:Mtigiannent,Orig taleban islamic : movement tell osama bin laden mutlah : mohamrped rabbani Icing : fahd : of sauch arable on apol 14 691d. subject : of bin laden head : of taleban interim ruling council taleban islarnic osarna bin laden osama bin laden fiy_e : in kandaher house under islaroictaleban protection still pn3bably : : in osama bin laden islamic taleban afghanistan osama bin laden : supporter : on thursday to unknown area islarnic taleban tell . osama bin laden .
arab businessman osamapresent : in dose region to jalalabad in eastern afghanistan bin laden under protection of taleban movement Figure 1.14: Example of bl-direcdonal Hlationshlp searth =

Quick Tour of InFact 3.0 Indirect A second type of Relationship Search is possible if you know two Relationships entities: a search for Indirect Relationships, in which two known entities are both linked by a third (unknown) entity. InFacte supports these searches as well, with a slightly different query syntax. Here in place of the action, we put the entity we want to link inside curly braces:
Bush > (Person/Name]) > Thatcher This query indicates that we are interested in any relationships in which any entity serves as a link between Bush and Thatcher. If we execute this query, we'll see a slightly different display:
InFatte seiticfi 'carpus I Preferences History I fietp 400.
0101/ QUCV Genenator ::;J:A:.'"n=P:M.sr^":;3;*e4:µ;.W5S:;:14'k'l,R::":":!.;43t7:n1=WIZta.VA,31,+.V.4 :int> {Person/Name)} > Thatcher IinI resutts 1-7:

Bush> (bush) > Thatcher Bush> (ronakt ttagan).> Thatcher Bush> fitagan) > Thatcher Bush> (george bush) > Thatcher Bush> (tosinici kaifu) > Thatcher Bush> (=mitt thatcher) > Thatcher Bush> (nefort =Meta) >Thatcher _ Figure 1.15: An example of an Indirect Relationship search.
Each of the links presented represents a different relationship in which some entity links these two individuals. If we click on:
Bush > (nelson mandelal > Thatcher Quick Tour of InFactck) 3.0 then we would see relationships in which Nelson Mandela is linked to both Bush and Margaret Thatcher:
Ih FaCtv tearcfi I Cornus., I Preferences I History I iletp 4110/11"
Aliox...5butok.osistst Bush> (nelson Mandela), >Thatcher :.4177.:.i7Pi7f : zit 4451,:t3.
Relationshipietults 1 - IL
r7PT;:.4.%tIg' tjas fle.d __________________________________________ ' ' '4 tla. "7;2; TF;i:Zatialker; CE33 ilaigal,aftic-A:riVitet5:14;:-:1*.'g@
I AtOrir-rar;17.:437;i::14;g4i07.:7-MigWi Ifgq,''''Vitil7Pr.VM:041-7474,71 , õr.. trOleiihO'ne : in sortie i-tnited. stateS Of harsh Pres-dÃ4`t ' b',"'" sanOfion of of south africa major trading partner P.91-506761*014 1 I ',president:: (fr,,CSfi ' 00grattatk:'Ortrielson mandelarelease *.nY-Wil.9:14_ II

ne1-06:ola,r.idoi : support : for anti- 1 4101 next week : brush'eside,: on negotiation i:adininistratiOn aitiericarOgader I '0414e.3.11iii.**.00::;.0*:ii:; aftfAr.7 i in prison.
! iTO#10.1c. :-:sr9q. I
!
resident (WO
ne*4 Mandela : on moriday about 1 p :
I
toad MO for negotiation adMi*-tration aA
rofficial as.sistant ii government f $.:aitrl IP( f'IC. i nelSen:M4ndfit,a , leaebh.rine : H.." .
nelson' mandela fleet .91'7.:::,',0i11#5..6,,Q,4'Afgt*.;=:at.30.9t after Firs : Margaret thatcher I
Figure 1.16: An example of an indirect link searrh result set.

Quick Tour of InFac? 3,0 CORPUS PAGE
The Corpus page provides users with information about the corpus of documents being searched and how the InFact Search Service has been set up. There are three main components to the page, presented in a tab-based display.
In Fact - Search I Corpus I Preferences illigsn I fael2 131**1 tirCk '';'";V4:'rVA:4E*fiirZn'r:rcrg'firglrill aVa S'earcfrron IF/Entity ________________________________________ 1 F.:755.:=E?Pli.:7--17.72747(Wir=FYIT;
Pith Wrnfrtv tocallorv L.
Orainititiog Ettl.911 Temoord Figure 1.17: Example of the corpus page, with the first tab expanded.
Ontology The Ontology Search feature allows users to view what ontologies Search have been submitted to the InFact system during indexing. The ontologies are identified by their root nodes, which are presented as links on left. If you click on a root, all the subpaths are displayed below. The subpaths are also links. In Figure 1.17, the InFact standard OntologyPath "IF/Entity" is displayed with all of its subpaths as links below it. Any time you click on a link, you will see the paths found below a given path, and any terms defined by that path if it is a custom OntologyPath. (The terms associated with standard OntologyPaths are not displayed as they would be too numerous.) By clicking on the path links provided users can navigate through any of the ontologies associated with the system. Alternatively, users can enter a term in the search input field, and any matching paths will be displayed.

Quick Tour of InFace 3.0 Synonym When searching for a given entity, it is useful to know any synonyms Search that entity might have. Synonyms are automatically included in search results. To see what synonyms exist for a given word, click on the "Find Synonyms" tab, and then enter the term in the synonym search field and press the search button.
Corpus Inc corpus page contains intorrnation about the corpus, including Information when it was ingested and any comments that might be included by a system administrator. Users can also view a list of the ActionTypes that are available for use in searches, and any metadata associated with the documents in the corpus. (Additional ActionTypes can be added by a system administrator.) To see this, click on the "Corpus Info and Metadata" tab.

Quick Tour of InFace 3.0 PREFERENCES
Setting preferences provides you a way to constrain your search results. InFact allows you to constrain your search in a number of different ways. To enable any of these filtering options, you must set the appropriate filter and click the Set Preferences button. Figure 1.18 shows an example of the Preferences page of the Web reference user interface running against the Reuters corpus. As you can see, there are several different options you can set to optimize and focus your search. In the screenshot, the drop-down list for Sort Scheme is displayed; you can see that there is a large amount of metadata associated with this corpus of documents that you can sort by. For additional information about these options please consult the online help.
InFact" Search I Corpus Preferences I
History I
4W4:Pit' Inudeneatedactrqns 6 :FP=le C 61q!J) s.0**krii06g.t1.03^9 Y40.1' enttie5 C Tnie 0 False, 5riiie 0 :False 0.04Cce.*t t971.1*,i*1,hk True C False :(V
Sarcti 66094i OfKoia!he Fake Number of reteironsiii0s iierittaga:
Number of documents ije4 qe _______________________________________ (1) for PSq9: Unsorted gl (t) . Sod Oct Reuters_Date = .shed ;el, =
SurrouncN ..ntences.1.0 :%Port = - p A ;OP
Meta.* Sort On: iletite4:beteOne = 4 FggittartiMIA t_yi,etto s,40õ:Reuters_i-feuitine m , etelog Sett On: Reuters 'tang Metatog Sod On: Saute rsitern(0 Metetog Sort On: Retsters_ttemlO_Oede Metete.g Sod On: Re etelvJterniO_Oote Mettstag Sod On: Re ute(s_Pubris her Metetng Sod an: Reuters_Source Metateiq Sort On: Reuters Title Figure 118: The Preferences page.

Quick Tour of InFacto 3.0 Include If you ask the question, "When did Clinton visit China?", you get a Negated list of results in which Clinton is found to have visited China.
However, you might also be interested in instances in which Clinton Actions did not visit China. If you select this option, both affirmative and negative aspects of the sentences will be returned.
Search This option determines whether to search modifying clauses in Modifiers addition to the sources and targets. This is true by default, and can be set to false to make the result set smaller and more precise.
Display By default, modifier information is displayed along with the source, Modifiers action, and target in the relationship display. If you wish to only see the core relationship, you can set this to false.
Enforce Strict When using bi-directional arrows that indicate your interest in Bi- relationships in which the source and target can be interchanged, there is a looser interpretation by the system. If this is set to false, then directionality results will include any instances in which the entities specified as source and target both perform the specified action on a third entity.
This option is only relevant when both a source and target are specified in the query.
Search In some cases you may want to search on the term that defines an Ontology Path ontology path as well as the ontology path itself. For example, the word 'location" is a standard ontology path provided by the InFact Name as Term system. If you wanted to search on this ontology path and also include references to the word location, then you would set this to true.
Number of This parameter specifies the number of relationships you wish to Relationships display on a given relationship result page.
per Page Number of This preference specifies the number of documents you wish to Documents per display on a given document result page.
Page Quick Tour of InFace 3.0 Number of This preference specifies the number of documents you wish to Documents per display on a given document result page.
Page Sort Scheme You can sort the result display by Source, Target, Action, Action Frequency, Action Similarity, or by any metadata content available such as Publication_Date. Available content is displayed in the drop down selection box. If this preference is set, any future results will be sorted by this criteria. (You can also sort individual pages using the controls on any given page.) Surrounding Any result set can be exported to either a summary html display or to Sentences to a text display that can be opened in a spreadsheet application. This preference specifies how much context to include around the Export sentences that contain the relationships you searched for in the results.

Quick Tour of InFact 3.0 FOR CONTENT PUBLISHERS
InFact provides huge productivity gains to your users/subscribers, saving time and money while increasing satisfaction. InFact learns the semantics of any text database, including custoiner or sales support information, news, financial data, legal information, scientific abstracts or journals. You can search virtually any document base and retrieve maps, graphs, charts or images containing the search words.
Indexing is fast and requires no time-consuming training, meta-tagging or expert input. Plus, new information can be indexed Incrementally, which is critical when searching any large and growing base of content. So not only does InFact save time and money for your users, it also saves time and money for content publishers.
InFact does not scan pages. It reads, understands and remembers them. Using a process of inductive reasoning, statistical data mining and artificial intelligence. InFact learns word meanings from the context and understands their syntactic relationships.
The InFact user interface and search strategies are rapidly evolving.
Please send us your comments, ideas and suggestions to search@infacicom We would be delighted to tailor InFact to meet your requirements for appearance and behavior.

Claims (165)

What is claimed is:

1. A
method in a computer system for preparing a corpus of documents for performing electronic searches, each document having at least one sentence, each sentence having a plurality of terms, comprising:
for each sentence of each document, parsing the sentence under the control of the computer system to generate a parse structure having a plurality of syntactic elements that correspond to the terms of the sentence;
determining from the structure of the parse structure and the plurality of syntactic elements a corresponding grammatical role for each of a plurality of the terms of the sentence, each grammatical role being at least one of a subject, an object, a governing verb, a modifier, or a part of a prepositional phrase;
normalizing the plurality of terms of the sentence having corresponding grammatical roles to a plurality of tagged terms, each tagged term indicating an association between the term of the sentence that corresponds to the grammatical role and an associated tag type that specifies the corresponding grammatical role, wherein at least one of the tagged terms has an associated tag type that specifies that the associated term of the sentence is a subject or an object of the sentence, wherein at least one of the tagged terms has an associated tag type that specifies that the associated term of the sentence is a modifier of another term of the sentence that has an associated tag type that specifies that the another term is a subject, object, or verb of the sentence, and wherein at least one of the tagged terms has an associated tag type that additionally specifies semantic information that refers to an entity type that identifies the associated term of the sentence as a type of person, location, or thing; and transforming each sentence to an enhanced data structure of terms stored as one or more inverted indexes of terms annotated with relationship information, wherein the plurality of the tagged terms are stored therein and indexed as additional terms of the sentence, each additional term including the term of the sentence and the associated tag type, thereby enabling a search engine to perform relationship searches by determining from the enhanced data structure whether a designated search term having an associated tag type that specifies a grammatical role or an entity type is present in the sentence in a same role, in a manner similar to the manner the search engine uses to determine whether a designated term is present in the sentence, at least one of the relationship searches capable of returning a plurality of relationships between at least two entities as a result of a single specification.

2. The method of claim 1 wherein the search engine is at least one of a keyword search engine or a Boolean search engine.

3. The method of claim 1 or 2 wherein the search engine performs string matching to determine whether the designated term having the associated tag type is present in the sentence in the same role.

4. The method of any one of claims 1 to 3 wherein the search engine performs pattern matching to determine whether the designated term having the associated tag type is present in the sentence in the same role.

5. The method of any one of claims 1 to 4 wherein the one or more inverted indexes of terms is an augmented term-document matrix configured to be searched using pattern matching or string matching to determine sentences that are similar to a designated relationship query.

6. The method of any one of claims 1 to 5 wherein the transforming each sentence to the enhanced data structure is performed for each clause of each sentence such that the tagged terms are treated as additional terms of each clause of the sentence and the search engine determines whether the designated syntactic term having the associated tag type is present in each clause.

7. The method of any one of claims 1 to 6 wherein the at least one of the tagged terms that has the associated tag type that specifies semantic information specifies an action attribute tag.

8. The method of claim 1 wherein the at least one of the tagged terms that has the associated tag type that specifies semantic information specifies an ontology path.

9. The method of any one of claims 1 to 8 wherein the normalizing the plurality of terms of the sentence having corresponding grammatical roles to the plurality of tagged items comprises applying linguistic normalization techniques to the plurality of terms of the sentence having corresponding grammatical roles to generate the plurality of tagged terms.

10. The method of claim 9 wherein the linguistic normalization techniques include applying at least one of a transformational grammar rule, a coreference resolution rule, a verbalization rule, or a verb sense rule.

11. The method of claim 10 wherein the verbalization rule is at least one of a noun verbalization rule, an adjective verbalization rule, or an adverb verbalization rule.

12. The method of claim 10 or 11 wherein the verbalization rule performs verb phrase simplification.

13. The method of any one of claims 10 to 12 wherein the coreference resolution rule is applied to at least one of a noun, a pronoun, a noun phrase, a pronoun phrase, alias, abbreviation, or acronym.

14. The method of any one of claims 9 to 13 wherein the linguistic normalization techniques include applying at least one rule that normalizes a set of synonyms or acronyms to a standard term or phrase.

15. The method of any one of claims 9 to 14 wherein the linguistic normalization techniques comprise identifying and generating tagged terms that include hypernyms and hyponyms.

16. The method of any one of claims 9 to 15 wherein the linguistic normalization techniques comprise identifying and generating tagged terms that include action attributes.

17. The method of claim 16 wherein the action attributes comprise identification of a verb tense.

18. The method of claim 16 or 17 wherein the action attributes comprise a verb mood or modality indication that specifies that the verb indicates a possibility, subjunctive, irrealis, negation, conditional, or causal relationship.

19. The method of any one of claims 16 to 18 wherein the action attributes comprise similar verbs.

20. The method of any one of claims 16 to 18 wherein the action attributes comprise troponyms, verb entailments, or hypernyms.

21. The method of any one of claims 1 to 20, further comprising:
receiving a query that specifies a relationship search that designates at least one of a term and an indication of a grammatical relationship associated with the designated term or a tag type that specifies a type of grammatical role, person, location or thing;
translating the query to a set of Boolean expressions;
executing a search engine that evaluates the Boolean expressions against the enhanced data structures of the sentences to determine a set of sentence clauses that match the query; and returning indications to the set of matching sentence clauses.

22. The method of claim 21 wherein the received query specifies the relationship search by means of a natural language query that is transformed to the designated at least one of the term or the tag type.

23. The method of claim 21 or 22 wherein the received query specifies a relationship search in combination with a document level Boolean search for at least one keyword to be located in each matching document.

24. The method of any one of claims 21 to 23 wherein the received query specifies a relationship search that is constrained by an expression that indicates a keyword search of the documents for at least one search term.

25. The method of any one of claims 21 to 24 wherein the received query specifies a relationship search that is constrained by a meta-data tag expression.

26. The method of any one of claims 21 to 25 wherein the received query specifies a relationship search that is constrained by an expression that indicates a value of a prepositional phrase.

27. The method of any one of claims 21 to 26 wherein the tag type specifies at least one of a an entity specification, or a path in an ontology.

28. The method of any one of claims 21 to 27 wherein the tag type specifies a subject, an object, or a verb.

29. The method of any one of claims 21 to 28 wherein the relationship search specifies a search term using a wildcard.

30. The method of claim 29 wherein the wildcard indicates a single character, range of characters, whole word, range of words, or a specific occurrence of a word.

31. The method of any one of claims 21 to 27 and 30 wherein the relationship search designates a value of at least one of a subject, an object, or a verb and the search engine determines all clauses in the corpus of documents where a grammatical relationship exists that satisfies the designated value of the at least one subject, object, or verb.

32. The method of claim 31 wherein the relationship search designates a value of a subject and the search engine determines a corresponding object and a corresponding verb of all clauses that contain a subject having the designated value.

33. The method of claim 31 or 32 wherein the relationship search designates a value of an object and the search engine determines a corresponding subject and a corresponding verb of all clauses that contain an object having the designated value.

34. The method of any one of claims 31 to 33 wherein the relationship search designates a value of a verb and the search engine determines a corresponding subject and a corresponding object of all clauses that contain a verb having the designated value or a similar verb to the designated value.

35. The method of any one of claims 31 to 34 wherein the relationship search designates a wildcard for at least one of the values of the designated at least one subject, object, or verb.

36. The method of any one of claims 31 to 35 wherein the search engine considers the presence in a corpus sentence clause of a term used in a modifier grammatical role as a modifier of the designated value of the subject, object, or verb as a match to the designated value of the subject, object, or verb.

37. The method of any one of claims 21 to 36 wherein the search engine is an off-the-shelf keyword search engine.

38. The method of claim 1 further comprising:
receiving a script that specifies a plurality of queries in a script language, each query specifying a relationship search that designates at least one of a term and an indication of a grammatical relationship that encompasses the term or a tag type that specifics a type of grammatical role, person, location, or thing;
translating the plurality of queries to a set of Boolean expressions;
executing a search engine that evaluates the Boolean expressions against the enhanced data structures of the sentences to determine a set of sentence clauses that match the Boolean expressions according to the script.

39. The method of claim 38 wherein the script comprises at least one of control flow instructions, group constructs, query order, or functions.

40. The method of claim 1, further comprising:
receiving a search query that designates a desired grammatical relationship between a first entity and at least one of a second entity or an action;
transforming the search query into a Boolean expression;
determining a set of objects that match the Boolean expression by performing a keyword-style search of the enhanced data structures of the sentences of the documents in the corpus, the enhanced data structures including grammatical relationship information as additional terms; and returning an indication of each matching object in the corpus that encompasses the desired relationship.

41. The method of claim 40 wherein the determining the set of objects determines objects are at least one of clauses, sentences, paragraphs, or documents.

42. The method of claim 40 wherein the designated at least one second entity or the action indicates a desire to match any second entity.

43. The method of claim 42, each sentence of each document comprising at least one clause, wherein the any second entity is any term used as a subject of a clause of a sentence.

44. The method of claim 42, each sentence of each document comprising at least one clause, wherein the any second entity is any term used as an object of a clause of a sentence.

45. The method of claim 40 wherein the designated at least one second entity or the action indicates a desire to match any action.

46. The method of claim 40 wherein the designated at least one second entity or the action is a verb.

47. The method of claim 46 wherein the returning the indication of each matching object that encompasses the desired relationship returns indications to objects that contain similar verbs to the designated verb.

48. The method of claim 46 or 47 wherein the returning the indication of each matching object that encompasses the desired relationship returns indications to objects that contain the same verb as the designated verb.

49. The method of any one of claims 46 to 48 wherein the returning the indication of each matching object that encompasses the desired relationship returns indications to objects that contain verbs of a similar classification to the designated verb.

50. The method of claim 40 wherein the designated at least one second entity or the action indicates a desire to match any action and a desire to match any second entity.

51. The method of any one of claims 40 to 50 wherein the first entity is any term that matches a specified entity type or ontology path specification.

52. The method of any one of claims 40 to 51 wherein the receiving the search query that designates the desired grammatical relationship between a first entity and at least one of a second entity or an action specifies at least one of a prepositional constraint, a document keyword constraint, or a document metadata constraint.

53. The method of any one of claims 40 to 52 wherein the search query includes a Boolean operation.

54. The method of claim 53 wherein the Boolean operation includes an AND, OR, or NOT operation.

55. The method of any one of claims 40 to 54 wherein the search query includes an operator that specifies at least one of a proximity, a range, a wildcard, a weighted search based upon frequency, or a weighted keyword search operation.

56. The method of any one of claims 50 to 55 wherein the search query includes a designation of at least one entity type.

57. The method of claim 56 wherein the at least one entity type is a path specification in a classification system.

58. The method of claim 56 or 57 wherein the at least one entity type is a path specification in a taxonomy that is specific to the corpus.

59. The method of any one of claims 40 to 58 wherein the search query includes a wildcard specification in the designation of the desired grammatical relationship.

60. The method of claim 59 wherein the wildcard specification is one of a single character wildcard operator, a multi-character wildcard operator, or a word wildcard operator.

61. The method of any one of claims 40 to 60, wherein the search query designates a desired grammatical relationship between the first entity and the second entity, the search query further designating a link entity specification that used to link the first entity and the second entity.

62. The method of claim 61 wherein the link entity specification is an entity type.

63. The method of claim 61 or 62 wherein the link entity specification is a path specification in a classification system.

64. The method of any one of claims 40 to 63 wherein the transforming the search query to generate a Boolean expression incorporates transformational grammar rules to generate related grammatical relationships to search for.

65. The method of any one of claims 40 to 64 wherein the generated Boolean expression includes an expression that causes a search for the desired grammatical relationship using at least one modifier.

66. The method of claim 65 wherein the at least one modifier is at least one of a subject modifier, an object modifier, a verb modifier, or an argument of preposition.

67. The method of claim 65 or 66 wherein the expression that causes a search for the desired grammatical relationship using the at least one modifier specifies an expression in which the modifier acts as a part of the first entity or the second entity.

68. The method of any one of claims 65 to 66 wherein the expression that causes a search for the desired grammatical relationship using the at least one modifier specifies an expression in which the modifier acts as a part of the action.

69. The method of claim 1 wherein the terms of the sentence and the additional terms are indexed in a reverse index of terms that indexes at least one of documents, sentences, or clauses.

70. The method of claim 69 wherein the reverse index of terms comprises a plurality of reverse indices of terms.

71. The method of claim 1 wherein the one or more inverted indexes of terms is a matrix that tracks occurrences of the terms across the corpus of documents..

72. The method of claim 71 wherein the enhanced data structure is at least one of a term-document matrix, a term-sentence matrix, or a term-clause matrix.

73. The method of claim 71 or 72 wherein the enhanced data structure is a plurality of term-clause matrices, each corresponding to a different grammatical role.

74. The method of claim 73 wherein the plurality of term-clause matrices comprise a subject index, an object index, and a verb index.

75. The method of claim 1 wherein the tagged terms each denote a grammatical role associated with a corresponding term.

76. The method of claim 75 wherein the associated grammatical roles are at least one of subject, object, verb, or modifier.

77. The method of claim 1 wherein the tagged terms each denote a semantic tag associated with a corresponding term.

78. The method of claim 77 wherein the associated semantic tags are path specifications in a classification system.

79. The method of claim 1 wherein each tagged term and additional term is associated with a location that corresponds to a particular clause, sentence, and document.

80. The method of claim 40 wherein the determining the set of sentences that match the Boolean expression performs pattern matching to determine the desired grammatical relationship.

81. The method of claim 40, the returning the indication of each matching object in the corpus that encompasses the desired relationship comprising:
returning an indication of at least one of each matching clause, each matching sentence, or each matching document in the corpus that encompasses the desired relationship.

82. The method of claim 40, the returning the indication of each matching object in the corpus that encompasses the desired relationship comprising:
in response to receiving a search query that designates a desired grammatical relationship between a first entity and any action, returning an indication of each matching object in the corpus that encompasses the first entity along with an indication of a corresponding action encompassed in the matching object.

83. The method of claim 1, the enhanced data structures that index terms of the documents including grammatical relationship information that is stored across a plurality of storage repositories, wherein the determining the set of objects that match the Boolean expression using a keyword-style search of the enhanced data structures further comprises:

performing a keyword-style search of the enhanced data structures against each storage repository that contains a portion of the index; and merging the results of the search to return the indication of each matching object in the corpus that encompasses the desired relationship.

84. The method of claim 83 wherein the keyword-style searches against each storage repository that contains the portion of the index are performed using parallel processing techniques.

85. A computer-readable memory medium containing instructions that control a computer processor to electronically index a corpus of documents and to electronically search the index according to any one of claims 1 to 84.

86. A computing system that is configured to index a corpus of documents for electronic searching, each document having at least one sentence, each sentence having a plurality of terms, comprising:
a parser that is configured, when executed, to parse each sentence of each document to generate a dependency structure that specifies a plurality of syntactic elements that correspond to a plurality of the terms of the sentence and their grammatical relationship to each other;
a post processing module that is configured, when executed, to normalize the dependency structure to a plurality of tagged terms, each tagged term indicating an association between the term that corresponds to the syntactic element and an associated tag type, the associated tag type specifying a grammatical role of the corresponding term as used in the sentence, the grammatical role designating at least one of a subject, an object, a governing verb, a modifier, or a part of a prepositional phrase, wherein at least one of the tagged terms has an associated tag type that that specifies that the corresponding term is a subject or an object of the sentence, wherein at least one of the tagged terms has an associated tag type that specifies that the associated term of the sentence is a modifier of another term of the sentence that has an associated tag type that specifies that the another term is a subject, object, or verb of the sentence, and wherein at least one of the tagged terms has an associated tag type that additionally refers to an entity type that identifies the corresponding term as a type of person, place, or thing; and a sentence transformation module that is configured, when executed, to transform the plurality of tagged terms to an enhanced data structure that stores and treats each tagged term as an encoded additional term of the sentence in one or more inverted indexes of terms annotated with relationship information, thereby enabling a search engine, to perform relationship searches by determining from the enhanced data structure whether a designated term having an associated tag type that specifies a desired grammatical role and/or a desired entity type is present in the sentence in a same role, in a manner similar to the manner the search engine uses to determine whether a designated term is present in the sentence, at least one of the relationship searches capable of returning a plurality of relationships between at least two entities as a result of a single search specification.

87. The system of claim 86 wherein the search engine is at least one of a keyword search or a Boolean search engine.

88. The system of claim 86 or 87 wherein the search engine performs string matching to determine whether the designated term having the associated tag type is present in the sentence in the same role.

89. The system of any one of claims 86 to 88 wherein the search engine performs pattern matching to determine whether the designated term having the associated tag type is present in the sentence in the same role.

90. The system of any one of claims 86 to 89 wherein the one or more inverted indexes of terms is an augmented term-document matrix configured to be searched using pattern matching or string matching to determine sentences that match a specified relationship query.

91. The system of any one of claims 86 to 90 wherein the transformation of each sentence to the enhanced data structure is performed for each clause of each sentence such that the tagged terms are treated as additional terms of each clause of the sentence and the search engine determines, for each clause, whether the designated syntactic term having the associated tag type is present.

92. The system of any one of claims 86 to 91 wherein the at least one of the tagged terms that has the associated tag type that additionally refers to the entity type specifies an ontology path.

93. The system of any one of claims 86 to 92 wherein the post processing module is configured, when executed, to normalize the dependency structure by applying linguistic normalization techniques to the plurality of syntactic elements to generate the plurality of tagged terms.

94. The system of claim 93 wherein the linguistic normalization techniques include applying at least one of a one transformational grammar rule, a coreference resolution rule, a verbalization rule, or a verb sense rule.

95. The system of claim 93 or 94 wherein the linguistic normalization techniques include applying at least one rule that normalizes a set of synonyms or acronyms to a standard term or phrase.

96. The system of any one of claims 93 to 95 wherein the linguistic normalization techniques comprise identifying and generating tagged terms that include hypernyms and hyponyms.

97. The system of any one of claims 86 to 96, further comprising:
a query interface module that is configured, when executed, to receive a query that specifies a relationship search that designates at least one of a tern-i and a grammatical relationship associated with the designated term or a tag type that specifies a type of grammatical role, person, place, or thing;
translate the query to at least one Boolean expression;
execute a search engine that evaluates the at least one Boolean expression against the enhanced data structures of the sentences to determine a set of objects that match the query; and return indications to the set of matching objects in the corpus.

98. The system of claim 97 wherein the search engine is an off-the-shelf keyword search engine.

99. The system of claim 97 or 98 wherein the enhanced data structures store relationship information in the stored additional terms of the documents.

100. The system of any one of claims 97 to 99 wherein the relationship search is indicative of at least one syntactically or semantically annotated term.

101. The system of claim 100 wherein the search engine determines the set of objects that match the query by pattern matching the at least one annotated term indicated by the query to the data structure, such that each matching object encompasses a relationship specified by the relationship search.

102. The any one of claims 97 to 101 wherein the returned indications indicate at least one of clauses, sentences, paragraphs, or documents.

103. The system of any one of claims 97 to 102 wherein the received query specifies the relationship search by means of a natural language query that is transformed to the designated at least one of the term or the tag type.

104. The system of any one of claims 97 to 103 wherein the received query specifies a relationship search in combination with a document level Boolean search for at least one keyword to be located in each matching document.

105. The system of any one of claims 97 to 104 wherein the received query specifies a relationship search that indicates a prepositional constraint, a document keyword constraint, or a document metadata constraint.

106. The system of any one of claims 97 to 105 wherein the relationship search includes a wildcard specification.

107. The system of any one of claims 97 to 106 wherein the relationship search includes a Boolean operation.

108. The system of any one of claims 97 to 107 wherein the relationship search includes an operator that specifies at least one of a proximity, a range, a weighted search based upon frequency, or a weighted keyword search operation.

109. The system of any one of claims 97 to 108 wherein the relationship search specifies at least one entity type or path specification in a classification system.

110. The system of any one of claims 97 to 109 wherein the relationship search designates a value of at least one of a subject, an object, or a verb and the search engine determines all clauses in the corpus of documents where a grammatical relationship exists that satisfies the designated value of the at least one subject, object, or verb.

111. The system of any one of claims 97 to 110 wherein the search engine considers the presence in a sentence clause of a term used in a modifier grammatical role as a modifier of the designated value of the subject, object, or verb as a match to the designated value of the subject, object, or verb.

112. The system of any one of claims 97 to 111 wherein the relationship search specifies a desired grammatical relationship between a first entity and at least one of a second entity or an action.

113. The system of claim 112 wherein the specified at least one second entity or the action indicates a desire to match any second entity.

114. The system of claim 112 or 113 wherein the first entity is any term that matches a specified entity type or ontology path specification.

115.
The system of any claims 112 to 114 wherein the specified at least one second entity or the action indicates a desire to match any action.

116. The system of any claims 112 to 115 wherein the specified at least one second entity or the action is a verb.

117. The system of any claims 112 to 116 wherein the specified at least one second entity or the action indicates a desire to match any action and a desire to match any second entity.

118. The system of any claims 112 to 117, the relationship search specifying a desired action, wherein the returned indications of each matching object of the set of matching objects returns indications to objects that contain similar verbs to a verb indicated by the desired action, the same verb as the verb indicated by the desired action, or a verb of a classification related to the verb indicated by the desired action.

119. The system of any claims 97 to 118, wherein the relationship search includes a link entity specification.

120. The system of any claims 97 to 119 wherein the transformed query incorporates transformational grammar rules.

121. The system of any claims 97 to 120 wherein the transformed query includes an expression that causes a search using at least one modifier.

122. The system of any one of claims 86 to 96, further comprising:
a query interface module that is configured, when executed, to receive a script that specifies a plurality of queries in a script language, each query specifying a relationship search that designates at least one of a term and an indication of a grammatical relationship that encompasses the term or a tag type that specifies a type of grammatical role, person, place, or thing;
translate the plurality of queries to a set of Boolean expressions;
execute a search engine that evaluates the Boolean expressions against the enhanced data structures of the sentences to determine a set of sentence clauses that match the Boolean expressions according to the script.

123. The system of claim 122 wherein the script comprises at least one of control flow instructions, group constructs, query order, or functions.

124. The system of any one of claims 86 to 123 wherein the one or more inverted indexes of terms is a matrix that tracks occurrences of the terms across the corpus of documents.

125. The system of claim 124 wherein the matrix is at least one of a term-document matrix, a term-sentence matrix, or a term-clause matrix.

126. The system of claim 124 wherein the matrix is a plurality of term-clause matrices, each corresponding to a different grammatical role.

127. The system of any one of claims 86 to 126 wherein each indexed term and additional term is associated with a location that corresponds to a particular clause, sentence, and document.

128. The system of any one of claims 86 to 126 wherein the enhanced data structures comprise a reverse index of terms that indexes at least one of documents, sentences, or clauses.

129. The system of claim 128 wherein the reverse index of terms comprises a plurality of reverse indices of terms.

130. The system of claim 128 or 129 wherein the enhanced data structures comprise at least one of a term-document matrix, a term-sentence matrix, or a term-clause matrix.

131. The system of any one of claims 86 to 130, the enhanced data structures that indexes and stores terms of the documents storing and indexing the additional terms across a plurality of storage repositories, and wherein the search engine performs pattern match searches against each storage repository that contains a portion of the index and merges the results of the pattern match searches to return the indication of each matching object in the corpus.

132. The system of claim 131 wherein the pattern match searches against each storage repository that contains the portion of the index are performed using parallel processing techniques.

133. A computer-readable memory medium containing structured data that stores a syntactic query, the query executed by a computer processor under the control of a search engine to search a corpus of objects for objects that match the query, comprising:
a base component that specifies values for desired relationship parameters, the relationship parameters comprising one or more entity parameters, at least one action parameter, and at least one directional operator parameter that specifies a direction of relationship between one of the one or more entity parameters and the at least one action parameter;
a prepositional constraint component that specifies a desired value for a prepositional phrase;
a keyword constraint component that specifies desired keyword values separately from and in addition to the values for the desired relationship parameters; and a metadata constraint component that specifies desired values of metadata associated with each matching object, whereby, when the search engine causes the search to be executed, objects that match the constraints specified by the base component, the prepositional constraint component, the keyword constraint component, and the metadata constraint component are determined to satisfy the query.

134. The memory medium of claim 133 wherein one or more of the components of the syntactic query are optional.

135. The memory medium of claim 133 or 134 wherein at least one of the components of the syntactic query is specified.

136. The memory medium of any one of claims 133 to 135 wherein at least one of the components of the syntactic query contains a Boolean expression.

137. The memory medium of any one of claims 133 to 135 wherein the base component specifies the desired relationship parameters in a general syntactic form:
Entity1 Directional-operator1 Action Directional-operator2 Entity2 wherein at least one of Entity1, Entity2, and Action parameters contains a non null value that indicates a search term, the Directional-operator 1 parameter specifies the direction of the relationship between the Entity1 and the Action parameters, and the Directional-operator2 parameter specifies the direction of the relationship between the Entity2 and the Action parameters.

138. The memory medium of claim 137 wherein a value of the Directional-operator parameter is one of a greater-than symbol (">"), a right arrow symbol ("->"), a less-than symbol ("<"),a left arrow symbol ("<-") or a combination indicating a bi-directional relationship ("<>" or "<->").

139. The memory medium of claim 137 or 138 wherein a specification of a value of ">" or "->" for the Directional-operator 1 parameter indicates that the value indicated by the Entity1 parameter is a subject of the value indicated by the Action parameter.

140. The memory medium of any one of claims 137 to 139 wherein a specification of a value of "<" or "<-" for Directional-operator 1 parameter indicates that the value indicated by the Entity1 parameter is an object of the value indicated by the Action parameter.

141. The memory medium of any one of claims 137 to 140 wherein a specification of a value of ">" or "->" for Directional-operator2 parameter indicates that the value indicated by the Entity2 parameter is an object of the value indicated by the Action parameter.

142. The memory medium of any one of claims 137 to 141 wherein a specification of a value of "<" or "<-" for Directional-operator2 parameter indicates that that the value indicated by the Entity2 parameter is a subject of the value indicated by the Action parameter.

143. The memory medium of any one of claims 137 to 142 wherein a value for the Action parameter indicates a search term that represents at least one of a particular verb, similar verbs, or an action type.

144. The memory medium of any one of claims 137 to 143 wherein a value for the Action parameter that is in the form of a quoted verb indicates a particular verb; a value for the Action parameter that in the form of an unquoted verb indicates similar verbs to that which is specified; and a value for the Action parameter that is in the form of a bracketed verb indicates an action type.

145. The memory medium of any one of claims 137 to 144 wherein a value for the Entity1 or the Entity2 parameter is a noun or noun phrase.

146. The memory medium of any one of claims 137 to 145 wherein a value for the Entity1 or the Entity2 parameter is a modifier.

147. The memory medium of any one of claims 137 to 146 wherein the prepositional constraint component comprises the phrase "PREP CONTAINS" or the character "^" followed by at least one search term.

148. The memory medium of claim any one of claims 144 to 147 wherein the keyword constraint component comprises the phrase "DOCUMENT CONTAINS" or the character ";" followed by at least one search term.

149. The memory medium of claim any one of claims 137 to 148 wherein the metadata constraint component comprises the phrase "METADATA CONTAINS" or the character "#" followed by at least one expression that specifies a desired value for a metadata variable.

150. The memory medium of any one of claims 137 to 149 wherein a wildcard can be specified as the value of a search term or a parameter of the base component.

151. The memory medium of claim 150 wherein the wildcard is at least one of the characters "*" or "?".

152. The memory medium of any one of claims 137 to 151 wherein curly braces are used to indicate indirect link searches.

153. The memory medium of any one of claims 137 to 152 wherein square brackets are used to indicate an action type or an entity classification.

154. A computer readable memory medium containing instructions programmed to control a computer processor to process syntactic queries that are stored in the structured data contained in a computer readable memory medium and structured according to any one of claims 137 to 153.

155. A method in a computer system comprising storing syntactic queries in the structured data contained in the computer readable memory medium and structured according to any one of claims 137 to 153.

156. A computer-readable memory medium that contains a reverse index for storing a corpus of documents according to terms present in the documents, the index configured to be accessed by a computer processor that is controlled by search engine to match a relationship query against the corpus of documents using pattern or string matching, the index comprising:
a plurality of terms, each term of the plurality of terms indicating at least one sentence in which the term occurs; and a plurality of tagged terms, each tagged term specifying a grammatical role that indicates a grammatical relationship of an associated term in the at least one sentence to other terms in the at least one sentence, each tagged term indicating the at least one sentence in which the associated term occurs, at least one of the tagged terms specifying a grammatical role that indicates that the associated term is a subject or an object, at least one of the tagged terms having an associated tag type that specifies that the associated term of the sentence is a modifier of another term of the sentence that has an associated tag type that specifies that the another term is a subject, object, or verb of the sentence, and at least one of the tagged terms additionally specifying a semantic tag that specifies that the associated term is a type of person, location, or thing;
such that the search engine can determine, by pattern matching query terms against the terms and tagged terms of the reverse index, a set of sentences that match a relationship indicated by the query.

157. The memory medium of claim 156 wherein the search engine is a keyword-style search engine.

158. The memory medium of claim 156 or 157 wherein the grammatical relationship indicated by each tagged term is at least one of a subject, an object, a governing verb, a modifier, or a part of a prepositional phrase.

159. The memory medium of any one of claims 156 to 158 wherein the semantic tag is at least one of an entity tag or a path specification in a classification structure.

160. The memory medium of any one of claims 156 to 159 wherein the reverse index is a term-clause index wherein each term indicates a clause within the indicated at least one sentence in which the term occurs.

161. The memory medium of any one of claims 156 to 160 wherein the reverse index is a term-clause index wherein each tagged term indicates a clause within the indicated at least one sentence in which the associated term occurs.

162. A search engine configured, when executed, to process queries against a corpus of documents that are stored in the reverse index contained in the computer readable memory medium and structured according to any one of claims 156 to 161.

163. The search engine of claim 162 wherein keyword searching techniques are performed to process queries.

164. A method in a computing system comprising storing a corpus of documents in the reverse index contained in the computer readable memory medium and structured according to any one of claims 156 to 161.

165. The method of claim 164 wherein the reverse index is at least one of a term-clause index, a term-sentence index, or a term-document index.