US20110202550A1 - Functional-form queries - Google Patents
Functional-form queries Download PDFInfo
- Publication number
- US20110202550A1 US20110202550A1 US12/706,154 US70615410A US2011202550A1 US 20110202550 A1 US20110202550 A1 US 20110202550A1 US 70615410 A US70615410 A US 70615410A US 2011202550 A1 US2011202550 A1 US 2011202550A1
- Authority
- US
- United States
- Prior art keywords
- query
- function
- functional
- relation
- queries
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000006870 function Effects 0.000 claims abstract description 185
- 238000004590 computer program Methods 0.000 claims description 24
- 238000000034 method Methods 0.000 claims description 16
- 238000011156 evaluation Methods 0.000 claims description 14
- 238000010276 construction Methods 0.000 claims description 13
- 238000013507 mapping Methods 0.000 claims description 3
- 238000012545 processing Methods 0.000 description 16
- 230000006855 networking Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 6
- 230000007246 mechanism Effects 0.000 description 4
- 230000010354 integration Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 239000000470 constituent Substances 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000004321 preservation Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24553—Query execution of query operations
- G06F16/24554—Unary operations; Data partitioning operations
- G06F16/24556—Aggregation; Duplicate elimination
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2453—Query optimisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2453—Query optimisation
- G06F16/24534—Query rewriting; Transformation
- G06F16/2454—Optimisation of common expressions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/901—Indexing; Data structures therefor; Storage structures
- G06F16/9024—Graphs; Linked lists
Definitions
- FIG. 1A is a diagram of a computing system, according to the prior art.
- FIG. 1B is a diagram of a computing system, according to an embodiment of the present disclosure.
- FIG. 2 is a diagram of an example dataflow graph to explain relation valued functions, according to an embodiment of the present disclosure.
- FIG. 3 is a diagram of a functional-form query framework, according to an embodiment of the present disclosure.
- FIGS. 4A and 4B are diagrams of example dataflow graphs that are expressible as functional-form queries within a functional-form query framework, according to different embodiments of the present disclosure.
- FIG. 5 is a diagram of middleware integrated within a database engine, according to an embodiment of the present disclosure.
- FIG. 6 is a flowchart of a method, according to an embodiment of the present disclosure.
- application computer programs such as business intelligence application computer programs, formulate queries that are executed against databases to return relevant data, which the application computer programs then use for analysis purposes.
- application computer programs logically speaking are external to database engines that run the queries against the database. That is, there is a database engine that executes queries against a database, and there is an application computer program that interfaces with the database engine to provide the queries to the database engine.
- an application computer program may realize a dataflow process, which is a series of queries in graph form, and thus which is referred to as a dataflow graph. Some of the queries may be duplicated within the dataflow graph. Because the application computer program is external to the database engine, the results of duplicate queries cannot be shared across the dataflow graph within the database engine. Rather, the database engine has to execute each query separately, even if the identical query has already been executed. Such repeated execution of duplicate queries hampers performance of the dataflow graph processing.
- Embodiments of the present disclosure overcome this problem by integrating middleware within a database engine.
- the dataflow graph that would otherwise be processed by an application computer program external to the database engine is instead processed by the middleware. Because the middleware is integrated within the database engine, duplicate queries within the dataflow graph are evaluated just once by the database engine. The results of such duplicate queries are therefore shared across the dataflow graph within the database engine. As a result, duplicate queries are not repeatedly executed, and the performance of dataflow graph processing is improved.
- a functional-form query is a query that includes an expression that declaratively denotes a function based on other functions, which are considered parameters within the expression; this is explained in detail below.
- the functional-form query represents the dataflow graph, and are a mechanism by which the middleware is able to be integrated within the database engine to improve the performance of dataflow graph processing.
- the dataflow graph represented by a functional-form query can include, in graph form, queries as well as what are referred to herein as relation valued functions, to aid in the integration of the middleware within the database engine.
- a relation valued function is a user-defined function, and returns a relation as output in response to receiving a number of relations as input; this is explained in detail below.
- the combination of relation valued functions and functional-form queries, the latter which are expressed within a functional-form query framework that is also described in detail below, thus permits the integration of middleware within a database engine in a way that improves the performance of dataflow graph processing.
- FIG. 1A shows a computing system 100 , according to the prior art
- FIG. 1B shows a computing system 150 , according to an embodiment of the disclosure.
- the computing system 100 includes hardware 102 , a database engine 104 , an application computer program 106 , and a database 108 .
- the computing system 150 includes hardware 152 , a database engine 154 , middleware 156 , and a database 158 .
- the hardware 102 and 152 can be or include one or more processors, memory, storage devices like hard disk drives, and so on.
- the computing systems 100 and 150 may be implemented over one or more computing devices.
- the database engine 104 and the application computer program 106 are executed by or using the hardware 102 , and thus are said to be implemented by the hardware 102 .
- the database engine 104 and the middleware 156 are executed by or using the hardware 152 , and thus are said to be implemented by the hardware 152 .
- the application computer program 106 processes a dataflow graph by issuing queries of the dataflow graph to the database engine 104 , which evaluates the queries against the database 108 .
- duplicate queries are evaluated or executed individually by the database engine 104 , which hampers performance.
- the bandwidth between the application computer program 106 and the database engine 104 may be limited, such that a limited amount of data may be able to be transferred between the program 106 and the engine 104 at any given time, which also hampers performance.
- the middleware 156 is integrated within the database engine 154 , and interacts with the database engine 154 to cause the database engine to execute or evaluate the queries of a dataflow graph. Duplicate queries are not evaluated or executed multiple times; rather, if a query is identical to a previously evaluated query, the results can be reused to improve performance. Furthermore, there are lesser bandwidth limitations between the middleware 156 and the database engine 154 , since the former is integrated within the latter, which also improves performance.
- Dataflow graph processing by the application computer program 106 in FIG. 1A is thus replaced by dataflow graph processing by the middleware 156 in FIG. 1B .
- integrating the functionality of dataflow graph processing is not as simple as just replacing the application computer program 106 with the middleware 156 . Rather, certain modifications are made.
- the dataflow graph is represented as a functional-form query expressed within a functional-form query framework, and the dataflow graph includes relation valued functions as well as queries, as is now described in detail in relation to FIGS. 2 and 3 .
- FIG. 2 is presented to explain what relation valued functions are
- FIG. 3 is presented to explain an exemplary functional-form query framework.
- FIG. 2 shows an example of such a dataflow graph 200 , according to an embodiment of the disclosure.
- the example dataflow graph 200 includes a number of relation valued functions 202 A and 202 B, collectively referred to as the relation valued functions 202 , and a number of queries 204 A, 204 B, and 204 C, collectively referred to as the queries 204 .
- the example dataflow graph 200 is specifically in the form of a tree.
- the relation valued functions 202 are each user-defined functions that have been extended in the following way. To handle operations applied to a set of tuples (i.e., a set of relations) rather than a single tuple, conventional scalar, aggregate, and table user-defined functions are extended to permit relation input and output. A relation valued function thus receives as input a list of relations, and returns as output another relation.
- a relation can be considered as a list of attribute name-type pairs, as well as any optional constraints, indices, and so on that are needed to define the attribute names and the types of the pairs.
- An attribute in this context is a parameter or a variable, which has a particular name, and a particular type, which may be string, integer, real number, or another type.
- a relation valued function can be defined as follows:
- a relation valued function has at least one input relation, but does not update its input relations. Rather, a relation valued function generates a new relation as its output, and in this way is treated as a relational operator or a data source. Extending user-defined functions as relation valued functions thus has certain advantages.
- a relation valued function can model general operations; because a relation valued function can operate on a set of tuples (i.e., a set of relations), it can accommodate more general operations that cannot be defined on a single tuple (i.e., a single relation).
- a relation valued function can compose operations with queries and integrate the invocation of the queries during processing.
- the relation valued function 202 A is composed of queries 204 A and 204 B.
- the query 204 B itself is dependent upon the results of the query 204 C and upon the results of the relation valued function 202 B.
- Relation valued functions can thus be naturally composed with other relational operators within an SQL query.
- the dataflow graph 200 of FIG. 2 represents the SELECT*FROM RFV 1 (Q 1 , Q 2 (Q 3 , RVF 2 )).
- a given relation valued function f applied to a given relation R can itself be expressed as the query SELECT*FROM f(“SELECT*FROM R”).
- relation valued functions provides for performance benefits, because applying a function a set of tuples, instead of tuple-by-tuple, can avoid repeatedly loading the data needed for processing each tuple. Furthermore, this permits the functions to be processes on a batch or parallel basis using multiple-core processors, or massively parallel processing accelerators. Fundamentally, relation valued functions provide the mechanism by which complex operations can be wrapped and integrated within SQL and other types of queries.
- FIG. 3 shows a functional-form query framework 300 , according to an embodiment of the disclosure.
- the functional-form query framework 300 is utilized to formulate functional-form queries that can be evaluated by the middleware 156 integrated within the database engine 154 of FIG. 1B .
- Functional-form queries are a mechanism that permits the middleware 156 to be integrated within the database engine 154 in FIG. 1B , as opposed to having an application computer program 106 external to the database engine 104 , as in the prior art of FIG. 1A .
- a functional-form query framework is a framework of queries that employs a functional form, which means a function-combining form. That is, a functional form is an expression that denotes the function; the function depends on the functions which are the parameters of the expression.
- a functional-form query framework uses a fixed set of functional forms for combining query functions, where a query can invoke relation valued functions. These functional forms, plus additional definitions, provide the mechanism by which new functions can be constructed from existing ones. Such new functions do not use variables or substitution rules, and become the operations of an associated algebra of queries.
- the functions of the functional-form query framework are of the same type, mapping relations into relations, and are the operands of the functional forms.
- the functional-form query framework 300 includes a set O of objects 302 , a set F of functions 304 , an apply meta-operator 305 , a set C of functional form primitives 306 , and a set D of definitions 308 .
- Each object of the set O of objects 302 is a relation 310 , a query variable 312 , or a sequence of elements 314 .
- Each function of the set F of functions 304 is a query function 326 , a relation valued function 328 , or a construction primitive 330 .
- a function of the set F of functions 304 may be a combination 331 of one or more of a query function 326 , a relation valued function 328 , and a construction primitive 330 as well.
- the set O of objects 302 has a relation 310 .
- a query variable 312 is a query that is bound to one or more specific relations. For example, the query
- a query function 326 is a function that is applied to a sequence of relations. For example, the query
- a relation valued function 328 is a user-defined function that has been extended to permit relation input and output, in that a list of relation is received as input, and another relation is returned as output, as has been described above.
- the query function 326 and the relation valued function 328 are both schema aware. That is, insofar as the query function 326 and the relation valued function 328 are applied to a relation, they are defined according to the schema of the relation, such that they are schema aware.
- a construction primitive 330 defines how a function is applied to one or more variables, including in which order the function is applied to the variables.
- Examples of construction primitives 330 include selector, identity, constant, and union primitives.
- the selector primitive selects an element from a sequence of elements, and can be expressed as $i: ⁇ r 1 , . . . , rn>, such that the i-th element ri is selected from the sequence of elements ⁇ r 1 , . . . , rn>.
- the union primitive constructs the union of two elements, and can be expressed as ⁇ r 1 , r 2 >, which means r 1 ⁇ r 2 .
- a combination 331 is a combination of one or more of a query function 326 , a relation valued function 328 , and a construction primitive 330 .
- the apply meta-operator 305 applies a function f to an object r. This is expressed as f:r.
- the apply meta-operator 305 may also apply a function f to a sequence of objects ⁇ r 1 , . . . , rn>. This is expressed as f: ⁇ r 1 , . . . , rn>.
- the set C of functional form primitives 306 combine existing functions to create new functions, whereas the definitions 308 define these existing functions; that is, the definitions 308 define the base preexisting set F of functions 304 , and can assign a name to each for convenience.
- a function form is an expression denoting a function.
- a functional form can be used to combine queries with relation valued functions into higher-level functions for expressing dataflow graphs, which conventional SQL cannot do.
- Examples of functional form primitives include composition, construction, condition, map, and reduce primitives.
- the composition primitive composes a new function from two existing functions.
- the construction primitive composes a new function as a series of existing functions.
- the condition primitive indicates one of two functions being performed based on a given condition.
- p:r is true, then f:r is performed, and otherwise g:r is performed (i.e., if p:r is false).
- the map primitive can also be considered an apply-to-all primitive, which applies a given function to a set of relations.
- the reduce primitive (recursively) converts a sequence of values to a single value.
- the reduce primitive can be generally used for stepwise merging and aggregation purposes. It is also noted that functional form primitives have certain algebraic properties, such as:
- functions can be defined level by level from query functions and relation valued functions in terms of functional form primitives.
- Application of a functional form primitive to a number of functions denotes a new function, and application of the resulting new function to a number of relations denotes a functional-form structure SQL query.
- a functional form SQL query can specify a dataflow graph, which a conventional SQL query cannot do. That is, conventional SQL queries represent tree-structure dataflows, whereas functional form SQL queries can also represent graph-structured dataflows.
- FIG. 4A shows an example dataflow graph in which a query result uses a fork, according to an embodiment of the disclosure.
- FIG. 4B shows an example dataflow graph in which query results are shared, according to an embodiment of the disclosure.
- a table 402 that stores shopping transaction data is retrieved and filtered by a query Q 404 .
- the result of query Q 404 is delivered to two relation valued functions: CF_rvf 406 , which may generate a collaborative filtering matrix as one example, and AR_rvf 408 , which may generate association rules as one example.
- CF_rvf 406 which may generate a collaborative filtering matrix as one example
- AR_rvf 408 which may generate association rules as one example.
- a table 452 stores networking traffic by networking address
- a table 454 stores which users by their networking addresses.
- a query Q 1 456 measures networking traffic on a networking address-to-networking address basis
- a query Q 2 458 determines which networking addresses are associated with which users.
- the results of the queries 456 and 458 are input into a query function Qf 3 460 , which measures networking traffic on a user-to-user basis.
- the relation valued function F 462 receives as input the results of the query Q 1 456 and the results of the query function Qf 3 460 , to analyze user behavior within the networking traffic.
- Using a functional-form query framework permits the dataflow of FIG. 4B to reuse the results of the query Q 1 456 without duplicating evaluation of this query.
- queries Q 1 456 and Q 2 458 can be expressed as query variables:
- middleware that is integrated within a database engine has been functionally described insofar as the middleware evaluates functional-form queries that include relation valued functions and that are expressed within a functional-form query framework.
- functional-form queries that are expressed within a functional-form query framework have been presented.
- additional details as to how such middleware can be implemented in conjunction with an existing database engine are described. Thereafter, the detailed description is concluded with the presentation of a method.
- FIG. 5 shows in more detail the middleware 156 as integrated within the database engine 154 , according to an embodiment of the disclosure.
- the middleware 156 includes a relation valued function component 502 and a functional-form component 504 .
- the database engine 154 is the PostgreSQL database engine, which is available on the Internet at www.postgresql.org.
- the relation valued function component 502 extends the functionality of the database engine 154 to support relation valued functions, so that business intelligence and other types of applications that are expressible via one or more relation valued functions within a dataflow graph can be evaluated by the database engine 154 .
- a relation valued function is invoked as a query.
- a relation valued function such as f(R)
- f(R) can be represented as the query SELECT*FROM f(r). Therefore, the relation valued function component 502 is able to translate relation valued functions into queries that the database engine 154 can evaluate.
- the functional-form component 504 extends the functionality of the database engine 154 to support functional-form queries that express dataflow graphs.
- a functional-form query invokes queries during its execution.
- the functional-form component 504 therefore passes such queries to the database engine 154 for evaluation.
- the functional-form component 504 also passes relation valued functions to the relation valued function component 502 for translation into queries that are then evaluated by the database engine 154 for evaluation.
- the result of an execution (i.e., evaluation) of a query is stored within a section of memory for the query that is referred to as a memory context.
- the memory context of a query is accessible to other invoked queries, which permits sharing of the results of the query. For example, if a first query is duplicative a second query, just the first query is evaluated, and not the second query. The results of the first query are not copied to the memory context of the second query, but rather the second query uses the results of the first query by reference to the first memory context.
- the functional-form component 504 is said to be able to evaluate queries in parallel insofar as a given functional-form query (i.e., representing a corresponding dataflow graph) encompassing these queries permits these queries to be evaluated in parallel. Evaluation in this context means that the functional-form component 504 is able to pass multiple queries to the database engine 154 for evaluation in a parallel manner.
- a functional-form query permits its queries to be evaluated in parallel based on the control implicit within the functional-form query.
- the query functions f and g are execute in a right-to-left order, such that the query function g is first evaluated against the parameter x, and then the query function f is evaluated against the results of the query function g as has been evaluated against the parameter x.
- the query functions f 1 . . . fn can be executed in parallel against the parameter x.
- the query function x can be evaluated against the parameters x . . . xn in parallel.
- the functional-form component 504 is also able to provide for dataflow during evaluation of a functional-form query by passing references to objects of the functional-form query, without having to duplicate the objects, as follows.
- a data object R flowing through a query or a relation valued function means that the query or the relation valued function generates an output relation from the data object R.
- the data object R flows through a construction primitive G, such as id or $1 this means that the data object R is to remain referenced through evaluation of the query or relation valued function in question. Maintaining reference to the data object R during evaluation therefore ensures that the data object R is not copied.
- a database engine 154 can have its functionality extended via the middleware 156 to support relation valued functions and functional-form queries, where the latter represent dataflow graphs.
- the functional-form component 504 of the middleware 156 supports functional-form queries, by parsing such functional-form queries into their constituent relation valued functions and non-functional-form queries.
- the non-functional-form queries are passed by the functional-form component 504 to the database engine 154 for evaluation.
- the relation valued functions are passed by the functional-form component 504 to the relation valued function component 502 , for parsing into (non-functional-form) queries that are then passed to the database engine 154 for evaluation.
- FIG. 6 shows a method 600 , according to an embodiment of the disclosure.
- the method 600 may be implemented as one or more computer programs stored on a computer-readable storage medium.
- Examples of computer-readable storage media include volatile media, such as dynamic random access memory, and non-volatile media, such as hard disk drives and solid-state drives.
- the computer programs are executable by one or more processors of one or more computing devices to cause the method 600 to be performed.
- middleware 156 is integrated within a database engine 154 ( 602 ), as has been described.
- the middleware 156 then receives a functional-form query representing a dataflow graph include queries and relation valued functions ( 604 ), which the middleware 156 evaluates by interacting with the database engine 154 ( 606 ), as has been described.
- Evaluation of the functional-form query results in output, which is provided to the user ( 608 ). For example, the output may be displayed on a display device to a user, communicated over a network, stored on a storage device, and so on.
- the dataflow graphs represented by functional-form queries in embodiments of the present disclosure differ from dataflow graphs that are represented by conventional SQL queries.
- the dataflow graph that is represented by a conventional SQL query can only be a tree, which is typically referred to as a query tree.
- the processing results of a given node of a query tree can thus be passed back to just one parent or root node, since a given node within a tree has just one parent or root node. That is, a tree dataflow graph is such that each node can have at most one parent or root node, and not multiple parent or root nodes.
- the dataflow graph that is represented by a functional-form query in accordance with an embodiment of the present disclosure can be a graph other than a tree.
- processing results of a given node of a dataflow graph can be passed back to more than one more parent or root node, since a given node within a dataflow graph other than a tree can have more than one parent or root node.
- functional-form queries are more powerful than conventional SQL queries, because they can represent dataflow graphs other than trees.
- the functional-form queries can support unlimited recursion within the dataflow graphs that they represent.
- the recursion that is supported by functional-form queries is inter-operator recursion, in which the results of one type of operator are recursively provided to another type of operator.
- conventional SQL queries support only a limited kind of recursion, and not inter-operator recursion in particular.
- the kind of recursion that can be expressed within conventional SQL queries is the application of the same operator (not different operators) to the earlier processing result of this operator. For example, a conventional SQL query may recursively locate an employee's manager, then the manager of the employee's manager, then the manager of the manager of the employee's manager, and so on.
Abstract
Description
- Large organizations, such as enterprises, typically collect a large amount of data that is stored in one or more databases. To analyze this data, such as to discern trends represented within the data, and so on, application computer programs known as business intelligence application computer programs are used. The application computer programs in general formulate queries that are executed against the databases to return relevant data, which the application computer programs then use for analysis purposes.
-
FIG. 1A is a diagram of a computing system, according to the prior art. -
FIG. 1B is a diagram of a computing system, according to an embodiment of the present disclosure. -
FIG. 2 is a diagram of an example dataflow graph to explain relation valued functions, according to an embodiment of the present disclosure. -
FIG. 3 is a diagram of a functional-form query framework, according to an embodiment of the present disclosure. -
FIGS. 4A and 4B are diagrams of example dataflow graphs that are expressible as functional-form queries within a functional-form query framework, according to different embodiments of the present disclosure. -
FIG. 5 is a diagram of middleware integrated within a database engine, according to an embodiment of the present disclosure. -
FIG. 6 is a flowchart of a method, according to an embodiment of the present disclosure. - As noted in the background section above, application computer programs, such as business intelligence application computer programs, formulate queries that are executed against databases to return relevant data, which the application computer programs then use for analysis purposes. These application computer programs logically speaking are external to database engines that run the queries against the database. That is, there is a database engine that executes queries against a database, and there is an application computer program that interfaces with the database engine to provide the queries to the database engine.
- This architecture of a database engine and an application computer program separate from and external to the database engine can be problematic from a performance perspective. For example, an application computer program may realize a dataflow process, which is a series of queries in graph form, and thus which is referred to as a dataflow graph. Some of the queries may be duplicated within the dataflow graph. Because the application computer program is external to the database engine, the results of duplicate queries cannot be shared across the dataflow graph within the database engine. Rather, the database engine has to execute each query separately, even if the identical query has already been executed. Such repeated execution of duplicate queries hampers performance of the dataflow graph processing.
- Embodiments of the present disclosure overcome this problem by integrating middleware within a database engine. The dataflow graph that would otherwise be processed by an application computer program external to the database engine is instead processed by the middleware. Because the middleware is integrated within the database engine, duplicate queries within the dataflow graph are evaluated just once by the database engine. The results of such duplicate queries are therefore shared across the dataflow graph within the database engine. As a result, duplicate queries are not repeatedly executed, and the performance of dataflow graph processing is improved.
- To implement integration of middleware within a database engine for improved performance in processing a dataflow graph, embodiments of the present disclosure employ what is referred to herein as a functional-form query. A functional-form query is a query that includes an expression that declaratively denotes a function based on other functions, which are considered parameters within the expression; this is explained in detail below. The functional-form query represents the dataflow graph, and are a mechanism by which the middleware is able to be integrated within the database engine to improve the performance of dataflow graph processing.
- The dataflow graph represented by a functional-form query can include, in graph form, queries as well as what are referred to herein as relation valued functions, to aid in the integration of the middleware within the database engine. A relation valued function is a user-defined function, and returns a relation as output in response to receiving a number of relations as input; this is explained in detail below. The combination of relation valued functions and functional-form queries, the latter which are expressed within a functional-form query framework that is also described in detail below, thus permits the integration of middleware within a database engine in a way that improves the performance of dataflow graph processing.
-
FIG. 1A shows acomputing system 100, according to the prior art, whereasFIG. 1B shows a computing system 150, according to an embodiment of the disclosure. InFIG. 1A , thecomputing system 100 includeshardware 102, adatabase engine 104, anapplication computer program 106, and adatabase 108. InFIG. 1B , the computing system 150 includeshardware 152, adatabase engine 154,middleware 156, and adatabase 158. - The
hardware computing systems 100 and 150 may be implemented over one or more computing devices. InFIG. 1A , thedatabase engine 104 and theapplication computer program 106 are executed by or using thehardware 102, and thus are said to be implemented by thehardware 102. Likewise, inFIG. 1B , thedatabase engine 104 and themiddleware 156 are executed by or using thehardware 152, and thus are said to be implemented by thehardware 152. - In the
computing system 100 ofFIG. 1A , theapplication computer program 106 processes a dataflow graph by issuing queries of the dataflow graph to thedatabase engine 104, which evaluates the queries against thedatabase 108. As noted above, duplicate queries are evaluated or executed individually by thedatabase engine 104, which hampers performance. Furthermore, the bandwidth between theapplication computer program 106 and thedatabase engine 104 may be limited, such that a limited amount of data may be able to be transferred between theprogram 106 and theengine 104 at any given time, which also hampers performance. - By comparison, in the computing system 150 of
FIG. 1B , themiddleware 156 is integrated within thedatabase engine 154, and interacts with thedatabase engine 154 to cause the database engine to execute or evaluate the queries of a dataflow graph. Duplicate queries are not evaluated or executed multiple times; rather, if a query is identical to a previously evaluated query, the results can be reused to improve performance. Furthermore, there are lesser bandwidth limitations between themiddleware 156 and thedatabase engine 154, since the former is integrated within the latter, which also improves performance. - Dataflow graph processing by the
application computer program 106 inFIG. 1A is thus replaced by dataflow graph processing by themiddleware 156 inFIG. 1B . However, integrating the functionality of dataflow graph processing is not as simple as just replacing theapplication computer program 106 with themiddleware 156. Rather, certain modifications are made. In particular, as noted above, the dataflow graph is represented as a functional-form query expressed within a functional-form query framework, and the dataflow graph includes relation valued functions as well as queries, as is now described in detail in relation toFIGS. 2 and 3 .FIG. 2 is presented to explain what relation valued functions are, whereasFIG. 3 is presented to explain an exemplary functional-form query framework. -
FIG. 2 shows an example of such adataflow graph 200, according to an embodiment of the disclosure. Theexample dataflow graph 200 includes a number of relation valuedfunctions 202A and 202B, collectively referred to as the relation valued functions 202, and a number ofqueries example dataflow graph 200 is specifically in the form of a tree. - The relation valued functions 202 are each user-defined functions that have been extended in the following way. To handle operations applied to a set of tuples (i.e., a set of relations) rather than a single tuple, conventional scalar, aggregate, and table user-defined functions are extended to permit relation input and output. A relation valued function thus receives as input a list of relations, and returns as output another relation. In one embodiment, a relation can be considered as a list of attribute name-type pairs, as well as any optional constraints, indices, and so on that are needed to define the attribute names and the types of the pairs. An attribute in this context is a parameter or a variable, which has a particular name, and a particular type, which may be string, integer, real number, or another type.
- For example, a relation valued function can be defined as follows:
-
DEFINE RVF f(x, y, R1, R2) RETURN R3 { Float x, y; Relation R1 (/* schema1 */); Relation R2 (/* schema2 */); Relation R3 (/* schema3 */); PROCEDURE fn(/*function name */); RETURN MODE SET_MODE; }
In this example, the relations R1, R2, and R3 are defined by schemas schema1, schema2, and schema3, respectively, and denote the scheme of the relation valued function f. Actual relation instances or query results compliant to these schemas can be bound to f as parameters. The input variables x and y can be bound to the attribute values of a tuple in tuple-wise query processing. However, the relations R1 and R2 are passed during the first call of the function f in processing the query that invokes this relation valued function. The returned set of tuples can be returned all at once, or tuple by tuple. - A relation valued function has at least one input relation, but does not update its input relations. Rather, a relation valued function generates a new relation as its output, and in this way is treated as a relational operator or a data source. Extending user-defined functions as relation valued functions thus has certain advantages. First, a relation valued function can model general operations; because a relation valued function can operate on a set of tuples (i.e., a set of relations), it can accommodate more general operations that cannot be defined on a single tuple (i.e., a single relation).
- Second, a relation valued function can compose operations with queries and integrate the invocation of the queries during processing. For example, in the
example dataflow graph 200 ofFIG. 2 , the relation valuedfunction 202A is composed ofqueries query 204B itself is dependent upon the results of thequery 204C and upon the results of the relation valued function 202B. Relation valued functions can thus be naturally composed with other relational operators within an SQL query. More specifically, thedataflow graph 200 ofFIG. 2 represents the SELECT*FROM RFV1(Q1, Q2(Q3, RVF2)). In general, a given relation valued function f applied to a given relation R can itself be expressed as the query SELECT*FROM f(“SELECT*FROM R”). - Using relation valued functions provides for performance benefits, because applying a function a set of tuples, instead of tuple-by-tuple, can avoid repeatedly loading the data needed for processing each tuple. Furthermore, this permits the functions to be processes on a batch or parallel basis using multiple-core processors, or massively parallel processing accelerators. Fundamentally, relation valued functions provide the mechanism by which complex operations can be wrapped and integrated within SQL and other types of queries.
-
FIG. 3 shows a functional-form query framework 300, according to an embodiment of the disclosure. The functional-form query framework 300 is utilized to formulate functional-form queries that can be evaluated by themiddleware 156 integrated within thedatabase engine 154 ofFIG. 1B . Functional-form queries are a mechanism that permits themiddleware 156 to be integrated within thedatabase engine 154 inFIG. 1B , as opposed to having anapplication computer program 106 external to thedatabase engine 104, as in the prior art ofFIG. 1A . - A functional-form query framework is a framework of queries that employs a functional form, which means a function-combining form. That is, a functional form is an expression that denotes the function; the function depends on the functions which are the parameters of the expression. A functional form query is thus a query that includes an expression denoting a function based on other functions that are parameters within the expression. For example, if f and g are relation valued functions, then f·g is a functional form that denotes a new function, such that, for a relation r, f(f·g):r=f:(g:r), provided that the relation r matches the input schema of the function g, and g:r matches the input schema of the function f.
- A functional-form query framework uses a fixed set of functional forms for combining query functions, where a query can invoke relation valued functions. These functional forms, plus additional definitions, provide the mechanism by which new functions can be constructed from existing ones. Such new functions do not use variables or substitution rules, and become the operations of an associated algebra of queries. The functions of the functional-form query framework are of the same type, mapping relations into relations, and are the operands of the functional forms.
- The functional-
form query framework 300 includes a set O ofobjects 302, a set F offunctions 304, an apply meta-operator 305, a set C offunctional form primitives 306, and a set D ofdefinitions 308. Each object of the set O ofobjects 302 is arelation 310, aquery variable 312, or a sequence ofelements 314. Each function of the set F offunctions 304 is aquery function 326, a relation valuedfunction 328, or a construction primitive 330. In one embodiment, a function of the set F offunctions 304 may be acombination 331 of one or more of aquery function 326, a relation valuedfunction 328, and a construction primitive 330 as well. - The set O of
objects 302 has arelation 310. Aquery variable 312 is a query that is bound to one or more specific relations. For example, the query -
- SELECT*FROM Sales, Customers
- WHERE Sales.customer_id=Customers.id;
is bound to the actual relations Sales and Customers. A query variable can be viewed as a relation data object that denotes the result of the query. With respect to a sequence ofelements 314, each element of the sequence is itself anobject 316; undefined, as indicated byreference number 318; a logical value of true, as indicated byreference number 320; a logical value of false, as indicated byreference number 322; or anempty element 324.
- With respect to the set F of
functions 304, aquery function 326 is a function that is applied to a sequence of relations. For example, the query -
- SELECT*FROM Sales, Customers
- WHERE Sales.customer_id=Customers.id;
has the corresponding query function - fQ:=SELECT*FROM $1, $2 WHERE $1.customer_id=$2.id;
In this query function, $1 represents the first element of a sequence, $2 represents the second element of the sequence, and $k, for instance, represents the k-th element of the sequence. Applying the query function fQ to a sequence of relations <Sales, Customers> with associated schemas is expressed as: - fQ:<Sales, Customers>→vQ
One constraint of query functions is schema preservation, which is that the schemas of the relations match the query function. As such, a query function may not be applicable to arbitrary relations.
- A relation valued
function 328 is a user-defined function that has been extended to permit relation input and output, in that a list of relation is received as input, and another relation is returned as output, as has been described above. Thequery function 326 and the relation valuedfunction 328 are both schema aware. That is, insofar as thequery function 326 and the relation valuedfunction 328 are applied to a relation, they are defined according to the schema of the relation, such that they are schema aware. - A construction primitive 330 defines how a function is applied to one or more variables, including in which order the function is applied to the variables. Examples of
construction primitives 330 include selector, identity, constant, and union primitives. The selector primitive selects an element from a sequence of elements, and can be expressed as $i:<r1, . . . , rn>, such that the i-th element ri is selected from the sequence of elements <r1, . . . , rn>. The identity primitive specifies that an element is equal to itself, and can be expressed as id:r=r. The constant primitive defines the name of one element for another element, and can be expressed as !y:x=y, such that x can be specified as y in addition to being specified as x. The union primitive constructs the union of two elements, and can be expressed as <r1, r2>, which means r1∪r2. Acombination 331, as noted above, is a combination of one or more of aquery function 326, a relation valuedfunction 328, and a construction primitive 330. - The apply meta-
operator 305 applies a function f to an object r. This is expressed as f:r. The apply meta-operator 305 may also apply a function f to a sequence of objects <r1, . . . , rn>. This is expressed as f:<r1, . . . , rn>. - The set C of
functional form primitives 306 combine existing functions to create new functions, whereas thedefinitions 308 define these existing functions; that is, thedefinitions 308 define the base preexisting set F offunctions 304, and can assign a name to each for convenience. As noted above, a function form is an expression denoting a function. A functional form can be used to combine queries with relation valued functions into higher-level functions for expressing dataflow graphs, which conventional SQL cannot do. Examples of functional form primitives include composition, construction, condition, map, and reduce primitives. - The composition primitive composes a new function from two existing functions. The composition primitive can be expressed as f·g:r=f:(g:r), for two functions f and g and a relation r. The construction primitive composes a new function as a series of existing functions. The construction primitive can be expressed as [f1 . . . fn]:r=<f1:r . . . fn:r>, for n functions f and a relation r. The condition primitive indicates one of two functions being performed based on a given condition. The condition primitive can be expressed as (p→f, g):r=((p:r)=T→f:r; g:r), for two functions f and, and a relation r, where p denotes the primitive itself. Thus, if p:r is true, then f:r is performed, and otherwise g:r is performed (i.e., if p:r is false).
- The map primitive can also be considered an apply-to-all primitive, which applies a given function to a set of relations. The map primitive can be expressed as αf: <r1 . . . rn>=<f:r1 . . . f:rn>, which maps, or applies, the function f to n relations r. The reduce primitive (recursively) converts a sequence of values to a single value. The reduce primitive can be expressed as /f:r=r==<r1>→r1; r==<r1 . . . rn> & n>=2→f:r1, /f: <r2 . . . rn>>, for a function f and n relations r. For example, /+:<1,2,3> returns the sum 6, since 1+2+3=6—the addition operator “+” is performed on all the values 1, 2, and 3.
- The reduce primitive can be generally used for stepwise merging and aggregation purposes. It is also noted that functional form primitives have certain algebraic properties, such as:
-
- [f·Qf,g·Qf]=[f,g]·Qf
where f and g are functions, and Qf is a query function.
- [f·Qf,g·Qf]=[f,g]·Qf
- In the functional-
form query framework 300, functions can be defined level by level from query functions and relation valued functions in terms of functional form primitives. Application of a functional form primitive to a number of functions denotes a new function, and application of the resulting new function to a number of relations denotes a functional-form structure SQL query. A functional form SQL query can specify a dataflow graph, which a conventional SQL query cannot do. That is, conventional SQL queries represent tree-structure dataflows, whereas functional form SQL queries can also represent graph-structured dataflows. - Two examples of utilization of a functional-form query framework to formulate a functional-form query so that the
middleware 156 integrated within thedatabase engine 154 inFIG. 1B is able to evaluate the query are now described in relation toFIGS. 4A and 4B .FIG. 4A shows an example dataflow graph in which a query result uses a fork, according to an embodiment of the disclosure.FIG. 4B shows an example dataflow graph in which query results are shared, according to an embodiment of the disclosure. - In
FIG. 4A , a table 402 that stores shopping transaction data is retrieved and filtered by aquery Q 404. The result ofquery Q 404 is delivered to two relation valued functions:CF_rvf 406, which may generate a collaborative filtering matrix as one example, andAR_rvf 408, which may generate association rules as one example. Using a functional-form query framework permits the dataflow ofFIG. 4A to be forked without duplicate database retrieval. That is, forking the dataflow inFIG. 4A is expressible using a functional-form query framework, as follows. -
- First, the
query Q 404 can be expressed as a query variable: - Q=SELECT tx-id, product, customer, month, store
- FROM shopping_transactions;
Next, the relation valuedfunctions
- FROM shopping_transactions;
- AR_rvf(Q)—Cross_selling_association_rules;
- CR_rvf(Q)—Collaborative_filter_customer_behavior;
Therefore, the functional-form SQL query representing the dataflow ofFIG. 4A is expressed simply as [AR_rvf, CF_rvf]: Q.
- First, the
- In
FIG. 4B , a table 452 stores networking traffic by networking address, and a table 454 stores which users by their networking addresses. Aquery Q1 456 measures networking traffic on a networking address-to-networking address basis, and aquery Q2 458 determines which networking addresses are associated with which users. The results of thequeries query function Qf3 460, which measures networking traffic on a user-to-user basis. Finally, the relation valuedfunction F 462 receives as input the results of thequery Q1 456 and the results of thequery function Qf3 460, to analyze user behavior within the networking traffic. Using a functional-form query framework permits the dataflow ofFIG. 4B to reuse the results of thequery Q1 456 without duplicating evaluation of this query. - First, the
queries Q1 456 andQ2 458 can be expressed as query variables: -
- Q1=SELECT from-ip, to-ip, SUM(bytes) AS bytes
- FROM Traffic_Table WHERE . . . /*time range*/
- GROUP by from-ip, to-ip;
- Q2=SELECT id, name, organization, category, security_level
- FROM Users Table;
Thequery function Qf3 460 is then expressed as follows:
- FROM Users Table;
- Qf3:=SELECT u1.user-id AS from-user, u2.user-id
- AS to-user, t.bytes FROM $1 t, $2 u1, $2 u2
- WHERE u1.ip=t.from-ip AND u2.ip=t.to-ip
Therefore, the functional-form SQL query representing the dataflow ofFIG. 4B is expressed simply as F·[$1,Qf3]:<Q1, Q2>.
- Q1=SELECT from-ip, to-ip, SUM(bytes) AS bytes
- Thus far in the detailed description, middleware that is integrated within a database engine has been functionally described insofar as the middleware evaluates functional-form queries that include relation valued functions and that are expressed within a functional-form query framework. Two examples of functional-form queries that are expressed within a functional-form query framework have been presented. Next, additional details as to how such middleware can be implemented in conjunction with an existing database engine are described. Thereafter, the detailed description is concluded with the presentation of a method.
-
FIG. 5 shows in more detail themiddleware 156 as integrated within thedatabase engine 154, according to an embodiment of the disclosure. Themiddleware 156 includes a relation valuedfunction component 502 and a functional-form component 504. In one embodiment, thedatabase engine 154 is the PostgreSQL database engine, which is available on the Internet at www.postgresql.org. - The relation valued
function component 502 extends the functionality of thedatabase engine 154 to support relation valued functions, so that business intelligence and other types of applications that are expressible via one or more relation valued functions within a dataflow graph can be evaluated by thedatabase engine 154. A relation valued function is invoked as a query. For example, a relation valued function, such as f(R), can be represented as the query SELECT*FROM f(r). Therefore, the relation valuedfunction component 502 is able to translate relation valued functions into queries that thedatabase engine 154 can evaluate. - The functional-
form component 504 extends the functionality of thedatabase engine 154 to support functional-form queries that express dataflow graphs. A functional-form query invokes queries during its execution. The functional-form component 504 therefore passes such queries to thedatabase engine 154 for evaluation. The functional-form component 504 also passes relation valued functions to the relation valuedfunction component 502 for translation into queries that are then evaluated by thedatabase engine 154 for evaluation. - In some types of database engines, such as the PostgreSQL database engine, the result of an execution (i.e., evaluation) of a query is stored within a section of memory for the query that is referred to as a memory context. The memory context of a query is accessible to other invoked queries, which permits sharing of the results of the query. For example, if a first query is duplicative a second query, just the first query is evaluated, and not the second query. The results of the first query are not copied to the memory context of the second query, but rather the second query uses the results of the first query by reference to the first memory context.
- The functional-
form component 504 is said to be able to evaluate queries in parallel insofar as a given functional-form query (i.e., representing a corresponding dataflow graph) encompassing these queries permits these queries to be evaluated in parallel. Evaluation in this context means that the functional-form component 504 is able to pass multiple queries to thedatabase engine 154 for evaluation in a parallel manner. A functional-form query permits its queries to be evaluated in parallel based on the control implicit within the functional-form query. - For example, in the functional-form query f·g:x=f(g:x), the query functions f and g are execute in a right-to-left order, such that the query function g is first evaluated against the parameter x, and then the query function f is evaluated against the results of the query function g as has been evaluated against the parameter x. By comparison, in the functional-form query [f1 . . . fn]: x=<f1:x . . . fn:x>, the query functions f1 . . . fn can be executed in parallel against the parameter x. In the functional-form query p→f, g:x=((p:x)=T→f:x; g:x), first the query function p is evaluated against the parameter x, and thereafter the query functions f and g can be evaluated in parallel against the parameter x. As a final example, in the functional-form query αf:<x1 . . . xn>=<f:x1 . . . f:xn>, the query function x can be evaluated against the parameters x . . . xn in parallel.
- The functional-
form component 504 is also able to provide for dataflow during evaluation of a functional-form query by passing references to objects of the functional-form query, without having to duplicate the objects, as follows. A data object R flowing through a query or a relation valued function means that the query or the relation valued function generates an output relation from the data object R. Where the data object R flows through a construction primitive G, such as id or $1, this means that the data object R is to remain referenced through evaluation of the query or relation valued function in question. Maintaining reference to the data object R during evaluation therefore ensures that the data object R is not copied. - In these ways, then, a
database engine 154 can have its functionality extended via themiddleware 156 to support relation valued functions and functional-form queries, where the latter represent dataflow graphs. The functional-form component 504 of themiddleware 156 supports functional-form queries, by parsing such functional-form queries into their constituent relation valued functions and non-functional-form queries. The non-functional-form queries are passed by the functional-form component 504 to thedatabase engine 154 for evaluation. The relation valued functions are passed by the functional-form component 504 to the relation valuedfunction component 502, for parsing into (non-functional-form) queries that are then passed to thedatabase engine 154 for evaluation. - In conclusion,
FIG. 6 shows amethod 600, according to an embodiment of the disclosure. Themethod 600 may be implemented as one or more computer programs stored on a computer-readable storage medium. Examples of computer-readable storage media include volatile media, such as dynamic random access memory, and non-volatile media, such as hard disk drives and solid-state drives. The computer programs are executable by one or more processors of one or more computing devices to cause themethod 600 to be performed. - First,
middleware 156 is integrated within a database engine 154 (602), as has been described. Themiddleware 156 then receives a functional-form query representing a dataflow graph include queries and relation valued functions (604), which themiddleware 156 evaluates by interacting with the database engine 154 (606), as has been described. Evaluation of the functional-form query results in output, which is provided to the user (608). For example, the output may be displayed on a display device to a user, communicated over a network, stored on a storage device, and so on. - It is noted that the dataflow graphs represented by functional-form queries in embodiments of the present disclosure differ from dataflow graphs that are represented by conventional SQL queries. The dataflow graph that is represented by a conventional SQL query can only be a tree, which is typically referred to as a query tree. The processing results of a given node of a query tree can thus be passed back to just one parent or root node, since a given node within a tree has just one parent or root node. That is, a tree dataflow graph is such that each node can have at most one parent or root node, and not multiple parent or root nodes. By comparison, the dataflow graph that is represented by a functional-form query in accordance with an embodiment of the present disclosure can be a graph other than a tree. As such, the processing results of a given node of a dataflow graph can be passed back to more than one more parent or root node, since a given node within a dataflow graph other than a tree can have more than one parent or root node. In this way, functional-form queries are more powerful than conventional SQL queries, because they can represent dataflow graphs other than trees.
- Furthermore, the functional-form queries can support unlimited recursion within the dataflow graphs that they represent. In particular, the recursion that is supported by functional-form queries is inter-operator recursion, in which the results of one type of operator are recursively provided to another type of operator. By comparison, conventional SQL queries support only a limited kind of recursion, and not inter-operator recursion in particular. More specifically, the kind of recursion that can be expressed within conventional SQL queries is the application of the same operator (not different operators) to the earlier processing result of this operator. For example, a conventional SQL query may recursively locate an employee's manager, then the manager of the employee's manager, then the manager of the manager of the employee's manager, and so on. In this type of recursion, the same operator—locate an individual's manager—is utilized. By comparison, for example, a functional-form query may recursively locate an employee's manager, then the manager's administrator, then this administrator's backup administrator, then the manager of this backup administrator, and so on. In this type of recursion, two operators—locate an individual's manager and locate an individual's backup—are utilized. In this way, functional-form queries are more powerful than conventional SQL queries, because they can provide for inter-operator recursion.
Claims (20)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US12/706,154 US9361341B2 (en) | 2010-02-16 | 2010-02-16 | Functional-form queries |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US12/706,154 US9361341B2 (en) | 2010-02-16 | 2010-02-16 | Functional-form queries |
Publications (2)
Publication Number | Publication Date |
---|---|
US20110202550A1 true US20110202550A1 (en) | 2011-08-18 |
US9361341B2 US9361341B2 (en) | 2016-06-07 |
Family
ID=44370368
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/706,154 Active 2034-10-06 US9361341B2 (en) | 2010-02-16 | 2010-02-16 | Functional-form queries |
Country Status (1)
Country | Link |
---|---|
US (1) | US9361341B2 (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20130138686A1 (en) * | 2011-11-30 | 2013-05-30 | Fujitsu Limited | Device and method for arranging query |
US20190287017A1 (en) * | 2018-03-15 | 2019-09-19 | Georgia Tech Research Corporation | Methods and systems for integrating machine learning/analytics accelerators and relational database systems |
Citations (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20040015809A1 (en) * | 2001-05-18 | 2004-01-22 | Doreen Yining Cheng | Code generation for integrating devices into a middleware framework |
US20040073539A1 (en) * | 2002-10-10 | 2004-04-15 | International Business Machines Corporation | Query abstraction high level parameters for reuse and trend analysis |
US20040117359A1 (en) * | 2001-03-01 | 2004-06-17 | Snodgrass Richard Thomas | Adaptable query optimization and evaluation in temporal middleware |
US20040243595A1 (en) * | 2001-09-28 | 2004-12-02 | Zhan Cui | Database management system |
US7089266B2 (en) * | 2003-06-02 | 2006-08-08 | The Board Of Trustees Of The Leland Stanford Jr. University | Computer systems and methods for the query and visualization of multidimensional databases |
US20070250330A1 (en) * | 2006-03-31 | 2007-10-25 | Sap Ag | Sourcing controller |
US20080177720A1 (en) * | 2007-01-19 | 2008-07-24 | Vladimir Mordvinov | System and method for query planning for execution using conditional operators |
US20090024500A1 (en) * | 1999-07-30 | 2009-01-22 | Alan Kay | System and Method of Transaction Settlement Using Trade Credit |
US20090112792A1 (en) * | 2007-10-25 | 2009-04-30 | Eric Lawrence Barsness | Generating Statistics for Optimizing Database Queries Containing User-Defined Functions |
US20100049733A1 (en) * | 2008-08-25 | 2010-02-25 | Sap Ag | Systems and methods for assigning hosts in response to a data query |
-
2010
- 2010-02-16 US US12/706,154 patent/US9361341B2/en active Active
Patent Citations (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20090024500A1 (en) * | 1999-07-30 | 2009-01-22 | Alan Kay | System and Method of Transaction Settlement Using Trade Credit |
US20040117359A1 (en) * | 2001-03-01 | 2004-06-17 | Snodgrass Richard Thomas | Adaptable query optimization and evaluation in temporal middleware |
US20040015809A1 (en) * | 2001-05-18 | 2004-01-22 | Doreen Yining Cheng | Code generation for integrating devices into a middleware framework |
US20040243595A1 (en) * | 2001-09-28 | 2004-12-02 | Zhan Cui | Database management system |
US20040073539A1 (en) * | 2002-10-10 | 2004-04-15 | International Business Machines Corporation | Query abstraction high level parameters for reuse and trend analysis |
US7089266B2 (en) * | 2003-06-02 | 2006-08-08 | The Board Of Trustees Of The Leland Stanford Jr. University | Computer systems and methods for the query and visualization of multidimensional databases |
US20070250330A1 (en) * | 2006-03-31 | 2007-10-25 | Sap Ag | Sourcing controller |
US20080177720A1 (en) * | 2007-01-19 | 2008-07-24 | Vladimir Mordvinov | System and method for query planning for execution using conditional operators |
US20090112792A1 (en) * | 2007-10-25 | 2009-04-30 | Eric Lawrence Barsness | Generating Statistics for Optimizing Database Queries Containing User-Defined Functions |
US20100049733A1 (en) * | 2008-08-25 | 2010-02-25 | Sap Ag | Systems and methods for assigning hosts in response to a data query |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20130138686A1 (en) * | 2011-11-30 | 2013-05-30 | Fujitsu Limited | Device and method for arranging query |
CN103136322A (en) * | 2011-11-30 | 2013-06-05 | 富士通株式会社 | Device and method for arranging query |
EP2600259A3 (en) * | 2011-11-30 | 2015-01-28 | Fujitsu Limited | Device, method and program for arranging query in a plurality of devices |
US9141677B2 (en) * | 2011-11-30 | 2015-09-22 | Fujitsu Limited | Apparatus and method for arranging query |
US20190287017A1 (en) * | 2018-03-15 | 2019-09-19 | Georgia Tech Research Corporation | Methods and systems for integrating machine learning/analytics accelerators and relational database systems |
US11521112B2 (en) * | 2018-03-15 | 2022-12-06 | Georgia Tech Research Corporation | Methods and systems for integrating machine learning/analytics accelerators and relational database systems |
Also Published As
Publication number | Publication date |
---|---|
US9361341B2 (en) | 2016-06-07 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US9842137B2 (en) | Performing complex operations in a database using a semantic layer | |
US10769165B2 (en) | Computing data lineage across a network of heterogeneous systems | |
EP2877943B1 (en) | Mapping entities in data models | |
US8375041B2 (en) | Processing queries against combinations of data sources | |
Simitsis et al. | Optimizing ETL processes in data warehouses | |
US9996592B2 (en) | Query relationship management | |
US7383234B2 (en) | Extensible data mining framework | |
US10191943B2 (en) | Decorrelation of user-defined function invocations in queries | |
US20080294596A1 (en) | System and method for processing queries for combined hierarchical dimensions | |
US20110295870A1 (en) | Systems and methods for providing custom or calculated data members in queries of a business intelligence server | |
US6847957B1 (en) | Dynamically extensible rule-based expert-system shell for database-computing environments | |
US20010037228A1 (en) | System and method for using metadata to flexibly analyze data | |
US9069837B2 (en) | Encapsulation of multiplicity and sparsity in multidimensional query execution systems | |
Haag | Managing variants of a personalized product: Practical compression and fast evaluation of variant tables | |
EP3293645B1 (en) | Iterative evaluation of data through simd processor registers | |
US9361341B2 (en) | Functional-form queries | |
Spyratos et al. | HIFUN-a high level functional query language for big data analytics | |
EP3293644B1 (en) | Loading data for iterative evaluation through simd registers | |
Šestak et al. | Extended Property-level k-vertex Cardinality Constraints Model for Graph Databases | |
US11442934B2 (en) | Database calculation engine with dynamic top operator | |
US10949410B2 (en) | Multi-threaded data analytics | |
Deutch et al. | Towards Hypothetical Reasoning Using Distributed Provenance. | |
Simitsis et al. | Logical optimization of ETL workflows | |
US20240095243A1 (en) | Column-based union pruning | |
US20230281339A1 (en) | Centralized data transformation in a multi-tenant computing environment |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: HEWLETT-PACKARD DEVELOPMENT COMPANY, L.P., TEXAS Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:CHEN, QIMING;HSU, MEICHUN;SIGNING DATES FROM 20100210 TO 20100211;REEL/FRAME:023945/0118 |
|
AS | Assignment |
Owner name: HEWLETT PACKARD ENTERPRISE DEVELOPMENT LP, TEXAS Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:HEWLETT-PACKARD DEVELOPMENT COMPANY, L.P.;REEL/FRAME:037079/0001 Effective date: 20151027 |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
AS | Assignment |
Owner name: ENTIT SOFTWARE LLC, CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:HEWLETT PACKARD ENTERPRISE DEVELOPMENT LP;REEL/FRAME:042746/0130 Effective date: 20170405 |
|
AS | Assignment |
Owner name: JPMORGAN CHASE BANK, N.A., DELAWARE Free format text: SECURITY INTEREST;ASSIGNORS:ENTIT SOFTWARE LLC;ARCSIGHT, LLC;REEL/FRAME:044183/0577 Effective date: 20170901 Owner name: JPMORGAN CHASE BANK, N.A., DELAWARE Free format text: SECURITY INTEREST;ASSIGNORS:ATTACHMATE CORPORATION;BORLAND SOFTWARE CORPORATION;NETIQ CORPORATION;AND OTHERS;REEL/FRAME:044183/0718 Effective date: 20170901 |
|
AS | Assignment |
Owner name: MICRO FOCUS LLC, CALIFORNIA Free format text: CHANGE OF NAME;ASSIGNOR:ENTIT SOFTWARE LLC;REEL/FRAME:050004/0001 Effective date: 20190523 |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 4 |
|
AS | Assignment |
Owner name: MICRO FOCUS LLC (F/K/A ENTIT SOFTWARE LLC), CALIFORNIA Free format text: RELEASE OF SECURITY INTEREST REEL/FRAME 044183/0577;ASSIGNOR:JPMORGAN CHASE BANK, N.A.;REEL/FRAME:063560/0001 Effective date: 20230131 Owner name: NETIQ CORPORATION, WASHINGTON Free format text: RELEASE OF SECURITY INTEREST REEL/FRAME 044183/0718;ASSIGNOR:JPMORGAN CHASE BANK, N.A.;REEL/FRAME:062746/0399 Effective date: 20230131 Owner name: MICRO FOCUS SOFTWARE INC. (F/K/A NOVELL, INC.), WASHINGTON Free format text: RELEASE OF SECURITY INTEREST REEL/FRAME 044183/0718;ASSIGNOR:JPMORGAN CHASE BANK, N.A.;REEL/FRAME:062746/0399 Effective date: 20230131 Owner name: ATTACHMATE CORPORATION, WASHINGTON Free format text: RELEASE OF SECURITY INTEREST REEL/FRAME 044183/0718;ASSIGNOR:JPMORGAN CHASE BANK, N.A.;REEL/FRAME:062746/0399 Effective date: 20230131 Owner name: SERENA SOFTWARE, INC, CALIFORNIA Free format text: RELEASE OF SECURITY INTEREST REEL/FRAME 044183/0718;ASSIGNOR:JPMORGAN CHASE BANK, N.A.;REEL/FRAME:062746/0399 Effective date: 20230131 Owner name: MICRO FOCUS (US), INC., MARYLAND Free format text: RELEASE OF SECURITY INTEREST REEL/FRAME 044183/0718;ASSIGNOR:JPMORGAN CHASE BANK, N.A.;REEL/FRAME:062746/0399 Effective date: 20230131 Owner name: BORLAND SOFTWARE CORPORATION, MARYLAND Free format text: RELEASE OF SECURITY INTEREST REEL/FRAME 044183/0718;ASSIGNOR:JPMORGAN CHASE BANK, N.A.;REEL/FRAME:062746/0399 Effective date: 20230131 Owner name: MICRO FOCUS LLC (F/K/A ENTIT SOFTWARE LLC), CALIFORNIA Free format text: RELEASE OF SECURITY INTEREST REEL/FRAME 044183/0718;ASSIGNOR:JPMORGAN CHASE BANK, N.A.;REEL/FRAME:062746/0399 Effective date: 20230131 |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 8 |