US20090210439A1 - Method for efficient data transformation - Google Patents
Method for efficient data transformation Download PDFInfo
- Publication number
- US20090210439A1 US20090210439A1 US12/033,024 US3302408A US2009210439A1 US 20090210439 A1 US20090210439 A1 US 20090210439A1 US 3302408 A US3302408 A US 3302408A US 2009210439 A1 US2009210439 A1 US 2009210439A1
- Authority
- US
- United States
- Prior art keywords
- node
- nodes
- arcs
- cost
- outgoing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/258—Data format conversion from or to a database
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/80—Information retrieval; Database structures therefor; File system structures therefor of semi-structured data, e.g. markup language structured data such as SGML, XML or HTML
- G06F16/84—Mapping; Conversion
Definitions
- the present invention relates to data transformation.
- healthcare providers such as physicians
- create large volumes of patient information at healthcare facilities such as hospitals, clinics, laboratories, and medical offices.
- a patient may be treated by more than one healthcare provider, necessitating that the patient's records at one healthcare provider be readily available to other healthcare providers, as this information might be critical to the healthcare provider when treating the patient.
- the wide variety of formats in which information is stored might impede the healthcare provider's ability to assimilate the information.
- medical data may be converted from one format to another to facilitate data interchange and thereby potentially improve patient care, doing so efficiently and at a minimum cost is vital in light of spiraling medical costs.
- transformation of data from a source format into a target format is carried out as a series of transformations to one or more intermediate data formats. While transformations might also be required that unify multiple data formats (i.e., many-to-one cardinality), that result in several target formats (i.e., one-to-many cardinality), or both (i.e., many-to-many cardinality), techniques for determining the most efficient paths for transformations of various cardinalities do not currently exist.
- the present invention discloses a method for efficient data transformation, particularly where multiple transformation paths are available and where transformations may be one-to-one, one-to-many, many-to-one, or many-to-many.
- the present invention facilitates efficient data transformation and interchange in fields such as, but not limited to, medical records management, multimedia production, and business data warehousing.
- a table of data transformations and their associated costs are expressed in a single digraph, where source and target data formats are represented as nodes connected by cost-labeled arcs. Each arc connecting the nodes has a nonnegative cost, where the cost may be expressed in terms of transformation execution time, labor costs, or any other costs.
- the most efficient transformation paths from the sources to the targets are then determined as those transformation paths that incur the lowest accumulated cost.
- a digraph including a plurality of ordinary nodes, at least one of a composition node and, a decomposition node, and a plurality of arcs interconnecting any of the nodes.
- composition node is connected to at least two of the nodes via arcs incoming to the composition node and to one other of the nodes via an arc outgoing to the other node.
- the decomposition node is connected to one of the nodes via an arc incoming to the composition node and to at least two other of the nodes via arcs outgoing to the other nodes.
- the ordinary nodes represent data formats.
- a first one of the ordinary nodes connected via an outgoing one of the arcs to a second one of the ordinary nodes represents a transformation of one data format into another.
- any of the arcs has an associated non-negative cost.
- a method for constructing a digraph from a plurality of source-to-target traversals including representing the sources and targets as a plurality of ordinary nodes, representing any of the traversals having a one-to-one cardinality by connecting the source node of the traversal to the target node of the traversal by an arc outgoing from the source node, and performing any of the following representing any of the traversals having a many-to-one cardinality by connecting the source nodes of the traversal to a composition node by arcs outgoing from the source nodes, and by connecting the composition node to the target node of the traversal by an arc outgoing from the composition node, representing any of the traversals having a one-to-many cardinality by connecting the source node of the traversal to a decomposition node by an arc outgoing from the source node, and by connecting the decomposition node to the target nodes of the traversal by arcs outgoing from the decomposition
- the method further includes associating a non-negative cost with any of the arcs.
- a method is provided of efficient path discovery in a digraph including a plurality of ordinary nodes, at least one of a composition node and a decomposition node, and a plurality of arcs interconnecting any of the nodes, the method including providing a source node s connected to a set S of source nodes in the digraph via outgoing arcs of zero cost, initializing to zero a cumulative cost of the path to s, providing a composition node t′ connected to a set T of target nodes in the digraph via incoming arcs of zero cost, providing a target node t connected to composition node t′ via an incoming arc of zero cost, defining a set W of nodes in the digraph initially including only node s, defining a set V of all nodes in the digraph, determining the cumulative costs of the paths to all nodes y in V that are connected to node s by an arc, while W ⁇ >V selecting a node x in V
- the method further includes determining the most efficient path from node s to a destination node selected from any of the nodes as including the arcs whose cost was added to the final cumulative cost of the destination node.
- the method further includes determining the most efficient path from S to T as including the arcs whose cost was added to the final cumulative costs of the nodes of T.
- the step of determining the most efficient path includes a) traversing each incoming arc of each node in T whose cost was added to the final cumulative cost of each current node, to arrive at one or more next nodes in the path, b) traversing each incoming arc of each node arrived at in the previous step whose cost was added to the final cumulative cost of each node arrived at in the previous step, to arrive at one or more next nodes in the path, and c) repeating step b) until the currently-arrived-at nodes are the nodes of S, where the traversed arcs together form the most efficient path from S to T.
- a computer program is provided embodied on a computer-readable medium, the computer program including a first code segment operative to provide a source node s connected to a set S of source nodes in a digraph via outgoing arcs of zero cost, the digraph including a plurality of ordinary nodes, at least one of a composition node and a decomposition node, and a plurality of arcs interconnecting any of the nodes, a second code segment operative to initialize to zero a cumulative cost of the path to s, a third code segment operative to provide a composition node t connected to a set T of target nodes in the digraph via incoming arcs of zero cost, a fourth code segment operative to provide a target node t connected to composition node t′]via an incoming arc of zero cost, a fifth code segment operative to define a set W of nodes in the digraph initially including only node s, a sixth code segment operative to define a
- FIGS. 1A , 1 B, 1 C, and 1 D are simplified illustrations of digraph elements, constructed and operative in accordance with a preferred embodiment of the present invention
- FIG. 2 which is a simplified illustration of a digraph, constructed and operative in accordance with a preferred embodiment of the present invention
- FIGS. 3A and 3B taken together, is a simplified flowchart illustration of a method of efficient path discovery in a digraph, operative in accordance with a preferred embodiment of the present invention
- FIGS. 4A , 4 B, 4 C, and 4 D are simplified illustrations of the digraph of FIG. 2 reflecting the application of the method of FIGS. 3A and 3B .
- FIGS. 1A , 1 B, 1 C, and 1 D are simplified illustrations of digraph elements, constructed and operative in accordance with a preferred embodiment of the present invention.
- the many-to-one cardinality is represented by a plus-shaped composition node 16 .
- an ordinary node a node that is neither a composition node nor a decomposition node is herein referred to as an ordinary node.
- FIG. 2 is a simplified illustration of a digraph, constructed and operative in accordance with a preferred embodiment of the present invention.
- the digraph of FIG. 2 is an exemplary construction using the sub-graphs of FIGS. 1A 1 D which models a number of transformations, shown in Table A, from source data formats, such as flat file and database (db), into target data formats, such as rtfXm 1 and dbXm 1 respectively.
- Each transformation is associated with a cost, which may represent any non-negative relevant cost, such as processing time, memory used, number of computer operations, etc.
- transformations may require multiple sources, such as the transformation of rtfXm 1 and dbXm 1 into performanceML, and that some transformations split a single source into multiple target formats, such as the transformation of html into text and gif.
- each source and target of Table A may be expressed as numbered nodes, such as is shown in Table B as follows:
- the digraph of FIG. 2 is constructed to represent the collection of source nodes and target nodes, where each transformation is depicted, using the sub-graphs of FIGS. 1A 1 D, as a set of arcs and nodes, where each arc is shown together with its associated cost.
- Source node 104 is shown connected both to node 118 and to node 120 .
- This type of digraph notation may be used to represent, for example, that a single source data format may be transformed into either of two different data formats, represented by node 118 and node 120 respectively.
- the arc from node 120 to node 118 may also be used to show that the source data format represented by source node 104 may be transformed into the data format represented by node 118 , first via transformation into the data format represented by node 120 , and then via transformation into the data format represented by node 118 .
- Many-to-one cardinality such as where data in two or more data formats are to be combined into a single data format, is shown represented by plus-shaped composition nodes 124 , 126 , and 128 , whose incoming arcs are assigned a zero cost.
- One-to-many cardinality such as where one data format is to be split into or otherwise transformed into two or more different data formats, is shown represented by a square-shaped decomposition node 130 . The outgoing arcs from decomposition nodes are also assigned a zero cost.
- the constructed digraph of the present invention may be used as the basis for formulating a query whose purpose is to determine the most efficient path between any two nodes or between any two groups of nodes, where efficiency is defined as the lowest cumulative cost of the arcs along a given path.
- the decision whether to traverse a particular arc may be made by considering the cumulative cost of the arcs traversed and selecting the path having the lowest cumulative cost.
- FIGS. 3A and 3B which, taken together, is a simplified flowchart illustration of a method of efficient path discovery in a digraph, operative in accordance with a preferred embodiment of the present invention.
- a single source node s is introduced into the digraph and connected via outgoing arcs of zero cost to each of the source nodes S.
- a cumulative cost of the path to s is typically initialized to zero, since s is the node of origin for all source nodes.
- a single composition node t′ is also defined and connected via incoming arcs of zero cost to each of the target nodes T.
- a single target node t is likewise defined and connected via an incoming arc of zero cost to the composition node t′.
- a set W of nodes is defined and initially includes only node s.
- a set V is likewise defined including all nodes in the digraph.
- the cumulative cost of the path from s to any given node in V is initially unknown, and is, therefore, typically considered to be infinite. The following steps are then performed to find the cumulative cost from s to any node in V:
- a node x in V is selected among all nodes in V that are not in W whose cumulative cost is minimal. If there is more than one minimal node, then any node x may be selected.
- the cumulative cost of the path to y is the lesser of a) the current cumulative cost of the path to y, if known, and b) the cumulative cost of the path to x plus the cost of the arc connecting x to y;
- the cumulative cost of the path to y is the lesser of a) the current cumulative cost of the path to y, if known, and b) the sum of the cumulative costs of the paths to all nodes that have outgoing arcs toy.
- the most efficient path from node s to any other destination node, and ultimately to node t, is comprised of the arcs whose cost was added to the final cumulative cost of the destination node. From this, it may be seen that the most efficient path from S to T may be derived as follows:
- step b) repeat step b) until the currently-arrived-at nodes are the nodes of S.
- FIGS. 3A and 3B may be used as a method of efficient data transformation when applied to a data transformation table such as Table A hereinabove.
- FIGS. 3A and 3B may be alternatively understood using the following pseudocode:
- a set of sources nodes S such that each node in S belongs to V.
- Begin: Construct a new node, s, and add arcs with cost 0 from s to every node in S.
- Construct a new node, t, and a new composition node t′. Add arcs with cost 0 from all the nodes in T to the composition node, t′, and from the composition node to t.
- the most efficient path from s to t is determined as those arcs whose cost was added to the final cumulative cost of t. From this, the most efficient path from S to T may be derived as described hereinabove.
- FIGS. 4A , 4 B, 4 C, and 4 D show the digraph of FIG. 2 reflecting the application of the method of FIGS. 3A and 3B to an exemplary query in which the most efficient path is determined for the transformation of the set of source nodes 102 , 104 , and 106 into the set of target nodes 108 and 110 .
- FIG. 4A shows the digraph of FIG. 2 for which s, t, and t have been defined.
- the arcs shown in dashed lines represent those arcs that lay along the most efficient paths from s to any node in V, and in particular to t.
- FIG. 4A shows the digraph of FIG. 2 for which s, t, and t have been defined.
- the arcs shown in dashed lines represent those arcs that lay along the most efficient paths from s to any node in V, and in particular to t.
- the arcs shown in dashed lines represent those arcs that do not lay along any efficient path and are the complementary arcs to those in FIG. 4B .
- the arcs shown in dashed lines represent those arcs that lay along the most efficient paths from source nodes 102 , 104 , and 106 to target nodes 108 and 110 , being a subset of the arcs shown in FIG. 4B .
Abstract
A digraph including a plurality of ordinary nodes, at least one of a composition node and a decomposition node, and a plurality of arcs interconnecting any of said nodes.
Description
- The present invention relates to data transformation.
- In today's business environment, many applications and solutions need to use data that are expressed in different formats and languages. Effective use of data often requires that data be transformed from one data format into another.
- For example, healthcare providers, such as physicians, create large volumes of patient information at healthcare facilities, such as hospitals, clinics, laboratories, and medical offices. Often, a patient may be treated by more than one healthcare provider, necessitating that the patient's records at one healthcare provider be readily available to other healthcare providers, as this information might be critical to the healthcare provider when treating the patient. Unfortunately, the wide variety of formats in which information is stored might impede the healthcare provider's ability to assimilate the information. Although medical data may be converted from one format to another to facilitate data interchange and thereby potentially improve patient care, doing so efficiently and at a minimum cost is vital in light of spiraling medical costs.
- In many cases, transformation of data from a source format into a target format is carried out as a series of transformations to one or more intermediate data formats. While transformations might also be required that unify multiple data formats (i.e., many-to-one cardinality), that result in several target formats (i.e., one-to-many cardinality), or both (i.e., many-to-many cardinality), techniques for determining the most efficient paths for transformations of various cardinalities do not currently exist.
- The present invention discloses a method for efficient data transformation, particularly where multiple transformation paths are available and where transformations may be one-to-one, one-to-many, many-to-one, or many-to-many. The present invention facilitates efficient data transformation and interchange in fields such as, but not limited to, medical records management, multimedia production, and business data warehousing.
- In the present invention, a table of data transformations and their associated costs are expressed in a single digraph, where source and target data formats are represented as nodes connected by cost-labeled arcs. Each arc connecting the nodes has a nonnegative cost, where the cost may be expressed in terms of transformation execution time, labor costs, or any other costs. The most efficient transformation paths from the sources to the targets are then determined as those transformation paths that incur the lowest accumulated cost.
- In one aspect of the present invention a digraph is provided including a plurality of ordinary nodes, at least one of a composition node and, a decomposition node, and a plurality of arcs interconnecting any of the nodes.
- In another aspect of the present invention the composition node is connected to at least two of the nodes via arcs incoming to the composition node and to one other of the nodes via an arc outgoing to the other node.
- In another aspect of the present invention the decomposition node is connected to one of the nodes via an arc incoming to the composition node and to at least two other of the nodes via arcs outgoing to the other nodes.
- In another aspect of the present invention the ordinary nodes represent data formats.
- In another aspect of the present invention a first one of the ordinary nodes connected via an outgoing one of the arcs to a second one of the ordinary nodes represents a transformation of one data format into another.
- In another aspect of the present invention any of the arcs has an associated non-negative cost.
- In another aspect of the present invention a method is provided for constructing a digraph from a plurality of source-to-target traversals, the method including representing the sources and targets as a plurality of ordinary nodes, representing any of the traversals having a one-to-one cardinality by connecting the source node of the traversal to the target node of the traversal by an arc outgoing from the source node, and performing any of the following representing any of the traversals having a many-to-one cardinality by connecting the source nodes of the traversal to a composition node by arcs outgoing from the source nodes, and by connecting the composition node to the target node of the traversal by an arc outgoing from the composition node, representing any of the traversals having a one-to-many cardinality by connecting the source node of the traversal to a decomposition node by an arc outgoing from the source node, and by connecting the decomposition node to the target nodes of the traversal by arcs outgoing from the decomposition node, and representing any of the traversals having a many-to-many cardinality by connecting the source nodes of the traversal to a composition node by arcs outgoing from the source nodes, by connecting the composition node to a decomposition node by an arc outgoing from the composition node, and by connecting the decomposition node to the target nodes of the traversal by arcs outgoing from the decomposition node.
- In another aspect of the present invention the method further includes associating a non-negative cost with any of the arcs.
- In another aspect of the present invention a method is provided of efficient path discovery in a digraph including a plurality of ordinary nodes, at least one of a composition node and a decomposition node, and a plurality of arcs interconnecting any of the nodes, the method including providing a source node s connected to a set S of source nodes in the digraph via outgoing arcs of zero cost, initializing to zero a cumulative cost of the path to s, providing a composition node t′ connected to a set T of target nodes in the digraph via incoming arcs of zero cost, providing a target node t connected to composition node t′ via an incoming arc of zero cost, defining a set W of nodes in the digraph initially including only node s, defining a set V of all nodes in the digraph, determining the cumulative costs of the paths to all nodes y in V that are connected to node s by an arc, while W<>V selecting a node x in V from all nodes in V that are not in W whose cumulative cost is minimal, adding node x to W, and for each node y in V to which x has an outgoing arc if y is not a composition node, determining the cumulative cost of the path to y as the lesser of a) the current known cumulative cost of the path to y, and b) the cumulative cost of the path to x plus the cost of the arc connecting x to y, if y is a composition node, and all nodes that have outgoing arcs to y are in W, determining the cumulative cost of the path to y as the lesser of a) the current known cumulative cost of the path to y, and b) the sum of the cumulative costs of the paths to all nodes that have outgoing arcs to y.
- In another aspect of the present invention the method further includes determining the most efficient path from node s to a destination node selected from any of the nodes as including the arcs whose cost was added to the final cumulative cost of the destination node.
- In another aspect of the present invention the method further includes determining the most efficient path from S to T as including the arcs whose cost was added to the final cumulative costs of the nodes of T.
- In another aspect of the present invention the step of determining the most efficient path includes a) traversing each incoming arc of each node in T whose cost was added to the final cumulative cost of each current node, to arrive at one or more next nodes in the path, b) traversing each incoming arc of each node arrived at in the previous step whose cost was added to the final cumulative cost of each node arrived at in the previous step, to arrive at one or more next nodes in the path, and c) repeating step b) until the currently-arrived-at nodes are the nodes of S, where the traversed arcs together form the most efficient path from S to T.
- In another aspect of the present invention a computer program is provided embodied on a computer-readable medium, the computer program including a first code segment operative to provide a source node s connected to a set S of source nodes in a digraph via outgoing arcs of zero cost, the digraph including a plurality of ordinary nodes, at least one of a composition node and a decomposition node, and a plurality of arcs interconnecting any of the nodes, a second code segment operative to initialize to zero a cumulative cost of the path to s, a third code segment operative to provide a composition node t connected to a set T of target nodes in the digraph via incoming arcs of zero cost, a fourth code segment operative to provide a target node t connected to composition node t′]via an incoming arc of zero cost, a fifth code segment operative to define a set W of nodes in the digraph initially including only node s, a sixth code segment operative to define a set V of all nodes in the digraph, a seventh code segment operative to determine the cumulative costs of the paths to all nodes y in V that are connected to node s by an arc, an eighth code segment operative, while W<>V, to select a node x in V from all nodes in V that are not in W whose cumulative cost is minimal, add node x to W, and for each node y in V to which x has an outgoing arc if y is not a composition node, determine the cumulative cost of the path to y as the lesser of a) the current known cumulative cost of the path to y, and b) the cumulative cost of the path to x plus the cost of the arc connecting x to y, if y is a composition node, and all nodes that have outgoing arcs to y are in W, determine the cumulative cost of the path to y as the lesser of a) the current known cumulative cost of the path to y, and b) the sum of the cumulative costs of the paths to all nodes that have outgoing arcs to y.
- The present invention will be understood and appreciated more fully from the following detailed description taken in conjunction with the appended drawings in which:
-
FIGS. 1A , 1B, 1C, and 1D are simplified illustrations of digraph elements, constructed and operative in accordance with a preferred embodiment of the present invention; -
FIG. 2 , which is a simplified illustration of a digraph, constructed and operative in accordance with a preferred embodiment of the present invention; -
FIGS. 3A and 3B , taken together, is a simplified flowchart illustration of a method of efficient path discovery in a digraph, operative in accordance with a preferred embodiment of the present invention; -
FIGS. 4A , 4B, 4C, and 4D are simplified illustrations of the digraph ofFIG. 2 reflecting the application of the method ofFIGS. 3A and 3B . - Reference is now made to
FIGS. 1A , 1B, 1C, and 1D, which are simplified illustrations of digraph elements, constructed and operative in accordance with a preferred embodiment of the present invention. In accordance with the present invention, a digraph is constructed to represent one or more source-to-target traversals, such as a transformation of data from a source format into a target format, and their attendant costs. If a transformation t has a cost=1, a cardinality of one-to-one, and transforms data from format {a1} to format {b1}, the sub-graph ofFIG. 1A is created, where {a1} and {b1} are represented bynodes 10 interconnected by anarc 12 whose direction indicates the logical direction of the transformation. If t has a cost=1, a cardinality of one-to-many, and transforms data from format {a1} to formats {b1,b2}, the sub-graph ofFIG. 1B is created. The one-to-many cardinality is represented by a square-shaped decomposition node 14. If t has a cost=1, a cardinality of many-to-one, and transforms data from formats {a1,a2} to format {b1}, the sub-graph ofFIG. 1C is created. The many-to-one cardinality is represented by a plus-shaped composition node 16. The composition node of the present invention requires that transformation t may be executed only after all data in source formats {a1,a2} exist. If t has a cost=1, a cardinality of many-to-many, and transforms data from formats {a1,a2,a3} to formats {b1,b2}, the sub-graph ofFIG. 1D is created, combining the sub-graphs ofFIGS. 1B and 1C . For the sake of clarity, a node that is neither a composition node nor a decomposition node is herein referred to as an ordinary node. - Reference is now made to
FIG. 2 , which is a simplified illustration of a digraph, constructed and operative in accordance with a preferred embodiment of the present invention. The digraph ofFIG. 2 is an exemplary construction using the sub-graphs ofFIGS. 1A 1D which models a number of transformations, shown in Table A, from source data formats, such as flat file and database (db), into target data formats, such as rtfXm1 and dbXm1 respectively. Each transformation is associated with a cost, which may represent any non-negative relevant cost, such as processing time, memory used, number of computer operations, etc. -
TABLE A Sources Targets Cost flatFile rtfXml 1 db dbXml 3 db 123 1 wordpro html 2 123 html 0.5 123 dbXml 1 rtfXml, dbXml performanceML 1 html, dbXml performanceML 1 rtfXml, dbXml XHTML 2 html text, gif 2 - It will be seen in Table A that some transformations may require multiple sources, such as the transformation of rtfXm1 and dbXm1 into performanceML, and that some transformations split a single source into multiple target formats, such as the transformation of html into text and gif.
- For both diagrammatic clarity and generality of description, each source and target of Table A may be expressed as numbered nodes, such as is shown in Table B as follows:
-
TABLE B Source Nodes Target Nodes Cost 102 116 1 104 118 3 104 120 1 106 122 2 120 122 0.5 120 118 1 116, 118 110 1 122, 118 110 1 116, 118 108 2 122 112, 114 2 - The digraph of
FIG. 2 is constructed to represent the collection of source nodes and target nodes, where each transformation is depicted, using the sub-graphs ofFIGS. 1A 1D, as a set of arcs and nodes, where each arc is shown together with its associated cost. -
Source node 104 is shown connected both tonode 118 and tonode 120. This type of digraph notation may be used to represent, for example, that a single source data format may be transformed into either of two different data formats, represented bynode 118 andnode 120 respectively. The arc fromnode 120 tonode 118 may also be used to show that the source data format represented bysource node 104 may be transformed into the data format represented bynode 118, first via transformation into the data format represented bynode 120, and then via transformation into the data format represented bynode 118. - Many-to-one cardinality, such as where data in two or more data formats are to be combined into a single data format, is shown represented by plus-shaped
composition nodes decomposition node 130. The outgoing arcs from decomposition nodes are also assigned a zero cost. - The constructed digraph of the present invention may be used as the basis for formulating a query whose purpose is to determine the most efficient path between any two nodes or between any two groups of nodes, where efficiency is defined as the lowest cumulative cost of the arcs along a given path. The decision whether to traverse a particular arc may be made by considering the cumulative cost of the arcs traversed and selecting the path having the lowest cumulative cost. A preferred method of efficient path discovery in the digraph of the present invention is now described.
- Reference is now made to
FIGS. 3A and 3B , which, taken together, is a simplified flowchart illustration of a method of efficient path discovery in a digraph, operative in accordance with a preferred embodiment of the present invention. In the method ofFIGS. 3A and 3B , given a set S of source nodes, a set T of target nodes, and various paths therebetween, a single source node s is introduced into the digraph and connected via outgoing arcs of zero cost to each of the source nodes S. A cumulative cost of the path to s is typically initialized to zero, since s is the node of origin for all source nodes. A single composition node t′ is also defined and connected via incoming arcs of zero cost to each of the target nodes T. A single target node t is likewise defined and connected via an incoming arc of zero cost to the composition node t′. - A set W of nodes is defined and initially includes only node s. A set V is likewise defined including all nodes in the digraph. The cumulative cost of the path from s to any given node in V is initially unknown, and is, therefore, typically considered to be infinite. The following steps are then performed to find the cumulative cost from s to any node in V:
- 1) The cumulative costs of the paths to all nodes y in V that are connected to node s by an arc are determined by the cost of the arc.
- 2) While W<>W:
- 3) A node x in V is selected among all nodes in V that are not in W whose cumulative cost is minimal. If there is more than one minimal node, then any node x may be selected.
- 4) Node x is added to W.
- 5) For each node y in V to which x has an outgoing arc:
- 6) If y is not a composition node, then the cumulative cost of the path to y is the lesser of a) the current cumulative cost of the path to y, if known, and b) the cumulative cost of the path to x plus the cost of the arc connecting x to y;
- 7) If y is a composition node, and all nodes that have outgoing arcs to y are in W, then the cumulative cost of the path to y is the lesser of a) the current cumulative cost of the path to y, if known, and b) the sum of the cumulative costs of the paths to all nodes that have outgoing arcs toy.
- The most efficient path from node s to any other destination node, and ultimately to node t, is comprised of the arcs whose cost was added to the final cumulative cost of the destination node. From this, it may be seen that the most efficient path from S to T may be derived as follows:
- a) traverse each incoming arc of each node in T whose cost was added to the final cumulative cost of each current node, to arrive at one or more next nodes in the path;
- b) traverse each incoming arc of each node arrived at in the previous step whose cost was added to the final cumulative cost of each node arrived at in the previous step, to arrive at one or more next nodes in the path;
- c) repeat step b) until the currently-arrived-at nodes are the nodes of S. The traversed arcs together form the most efficient path from S to T
- It may thus be seen that the method of
FIGS. 3A and 3B may be used as a method of efficient data transformation when applied to a data transformation table such as Table A hereinabove. - The method of
FIGS. 3A and 3B may be alternatively understood using the following pseudocode: -
Input: A digraph D = (V, A), with costs Cuv >= 0 on its arcs, and V having >= 0 composition nodes and >= 0 decomposition nodes. A set of sources nodes S such that each node in S belongs to V. Output: The lowest-cost paths from S to all nodes in V in an array p. Begin: Construct a new node, s, and add arcs with cost = 0 from s to every node in S. Construct a new node, t, and a new composition node t′. Add arcs with cost = 0 from all the nodes in T to the composition node, t′, and from the composition node to t. Add all the new nodes and arcs to the digraph D=(V,A). Set W := {s}; p[s] := 0 ; for all y such that y is a node in V with an incoming arc from s do p[y] := Csy ; while W <> V do begin find min {p[y] : y is not in W} , say p[x]; set W := W union {x} ; for all y in V such that there is an arc from x to y do begin if y is not a composition node then p[y] := min {p[y] , p[x] + Cxy} else if all nodes that have outgoing arcs to y are in W then p[y] := min {p[y] ,sum of all p[z] where z has an outgoing arc to y} end end end - Thus, by applying the method of
FIGS. 3A and 3B , the most efficient path from s to t is determined as those arcs whose cost was added to the final cumulative cost of t. From this, the most efficient path from S to T may be derived as described hereinabove. - Reference is now made to
FIGS. 4A , 4B, 4C, and 4D which show the digraph ofFIG. 2 reflecting the application of the method ofFIGS. 3A and 3B to an exemplary query in which the most efficient path is determined for the transformation of the set ofsource nodes target nodes FIG. 4A shows the digraph ofFIG. 2 for which s, t, and t have been defined. InFIG. 4B the arcs shown in dashed lines represent those arcs that lay along the most efficient paths from s to any node in V, and in particular to t. InFIG. 4C the arcs shown in dashed lines represent those arcs that do not lay along any efficient path and are the complementary arcs to those inFIG. 4B . Finally, inFIG. 4D the arcs shown in dashed lines represent those arcs that lay along the most efficient paths fromsource nodes nodes FIG. 4B . - It is appreciated that one or more of the steps of any of the methods described herein may be omitted or carried out in a different order than that shown, without departing from the true spirit and scope of the invention.
- While the methods and apparatus disclosed herein may or may not have been described with reference to specific hardware or software, it is appreciated that the methods and apparatus described herein may be readily implemented in hardware or software using conventional techniques.
- While the present invention has been described with reference to one or more specific embodiments, the description is intended to be illustrative of the invention as a whole and is not to be construed as limiting the invention to the embodiments shown. It is appreciated that various modifications may occur to those skilled in the art that, while not specifically shown herein, are nevertheless within the true spirit and scope of the invention.
Claims (10)
1. A digraph comprising:
a plurality of ordinary nodes;
at least one of:
a composition node and,
a decomposition node; and
a plurality of arcs interconnecting any of said nodes.
2. A digraph according to claim 1 wherein said composition node is connected to at least two of said nodes via arcs incoming to said composition node and to one other of said nodes via an arc outgoing to said other node.
3. A digraph according to claim 1 wherein said decomposition node is connected to one of said nodes via an arc incoming to said composition node and to at least two other of said nodes via arcs outgoing to said other nodes.
4. A digraph according to claim 1 wherein said ordinary nodes represent data formats.
5. A digraph according to claim 4 wherein a first one of said ordinary nodes connected via an outgoing one of said arcs to a second one of said ordinary nodes represents a transformation of one data format into another.
6. A digraph according to claim 1 wherein any of said arcs has an associated non-negative cost.
7. A method for constructing a digraph from a plurality of source-to-target traversals, the method comprising:
representing said sources and targets as a plurality of ordinary nodes;
representing any of said traversals having a one-to-one cardinality by connecting the source node of said traversal to the target node of said traversal by an arc outgoing from said source node; and
performing any of the following:
representing any of said traversals having a many-to-one cardinality by connecting the source nodes of said traversal to a composition node by arcs outgoing from said source nodes, and by connecting said composition node to the target node of said traversal by an arc outgoing from said composition node;
representing any of said traversals having a one-to-many cardinality by connecting the source node of said traversal to a decomposition node by an arc outgoing from said source node, and by connecting said decomposition node to the target nodes of said traversal by arcs outgoing from said decomposition node; and
representing any of said traversals having a many-to-many cardinality by connecting the source nodes of said traversal to a composition node by arcs outgoing from said source nodes, by connecting said composition node to a decomposition node by an arc outgoing from said composition node, and by connecting said decomposition node to the target nodes of said traversal by arcs outgoing from said decomposition node.
8. A method according to claim 7 and further comprising associating a non-negative cost with any of said arcs.
9-10. (canceled)
11. A method of efficient path discovery in a digraph including a plurality of ordinary nodes, at least one of a composition node and a decomposition node, and a plurality of arcs interconnecting any of said nodes, the method comprising:
providing a source node s connected to a set S of source nodes in said digraph via outgoing arcs of zero cost;
initializing to zero a cumulative cost of the path to s;
providing a composition node t connected to a set T of target nodes in said digraph via incoming arcs of zero cost;
providing a target node t connected to composition node t via an incoming arc of zero cost;
defining a set W of nodes in said digraph initially including only node s;
defining a set V of all nodes in said digraph;
determining the cumulative costs of the paths to all nodes y in V that are connected to node s by an arc;
while W<>V
selecting a node x in V from all nodes in V that are not in W whose cumulative cost is minimal;
adding node x to W; and
for each node y in V to which x has an outgoing arc:
if y is not a composition node, determining the cumulative cost of the path to y as the lesser of a) the current known cumulative cost of the path to y, and b) the cumulative cost of the path to x plus the cost of the arc connecting x to y;
if y is a composition node, and all nodes that have outgoing arcs to y are in W, determining the cumulative cost of the path to y as the lesser of a) the current known cumulative cost of the path to y, and b) the sum of the cumulative costs of the paths to all nodes that have outgoing arcs to y;
further comprising determining the most efficient path from S to T as comprising the arcs whose cost was added to the final cumulative costs of the nodes of T.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US12/033,024 US8180809B2 (en) | 2002-08-01 | 2008-02-19 | Method for efficient data transformation |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US10/211,171 US7356480B2 (en) | 2002-08-01 | 2002-08-01 | Method of data transformation via efficient path discovery using a digraph |
US12/033,024 US8180809B2 (en) | 2002-08-01 | 2008-02-19 | Method for efficient data transformation |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/211,171 Division US7356480B2 (en) | 2002-08-01 | 2002-08-01 | Method of data transformation via efficient path discovery using a digraph |
Publications (3)
Publication Number | Publication Date |
---|---|
US20090210439A1 true US20090210439A1 (en) | 2009-08-20 |
US20110072053A9 US20110072053A9 (en) | 2011-03-24 |
US8180809B2 US8180809B2 (en) | 2012-05-15 |
Family
ID=31187521
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/211,171 Active 2025-10-19 US7356480B2 (en) | 2002-08-01 | 2002-08-01 | Method of data transformation via efficient path discovery using a digraph |
US12/033,024 Expired - Fee Related US8180809B2 (en) | 2002-08-01 | 2008-02-19 | Method for efficient data transformation |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/211,171 Active 2025-10-19 US7356480B2 (en) | 2002-08-01 | 2002-08-01 | Method of data transformation via efficient path discovery using a digraph |
Country Status (1)
Country | Link |
---|---|
US (2) | US7356480B2 (en) |
Families Citing this family (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE10161111A1 (en) * | 2001-12-12 | 2003-07-03 | Siemens Ag | System and method for projecting transformations of object trees |
US7356480B2 (en) * | 2002-08-01 | 2008-04-08 | International Business Machines Corporation | Method of data transformation via efficient path discovery using a digraph |
US8194655B2 (en) * | 2004-08-05 | 2012-06-05 | Dust Networks, Inc. | Digraph based mesh communication network |
US8635248B2 (en) | 2008-06-23 | 2014-01-21 | Microsoft Corporation | Providing localized individually customized updates from a social network site to a desktop application |
US8655811B2 (en) * | 2009-01-29 | 2014-02-18 | Raytheon Company | Method and system for data stream identification by evaluation of the most efficient path through a transformation tree |
US8681635B2 (en) * | 2009-12-31 | 2014-03-25 | Mapquest, Inc. | Computer-implemented systems and methods for planning a route |
US10387666B2 (en) * | 2016-01-15 | 2019-08-20 | Acronis International Gmbh | System and method for synchronization of large amounts of data while maintaining control over access rights to such data |
Citations (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5261080A (en) * | 1987-08-21 | 1993-11-09 | Wang Laboratories, Inc. | Matchmaker for assisting and executing the providing and conversion of data between objects in a data processing system storing data in typed objects having different data formats |
US5418717A (en) * | 1990-08-27 | 1995-05-23 | Su; Keh-Yih | Multiple score language processing system |
US5694578A (en) * | 1992-12-18 | 1997-12-02 | Silicon Graphics, Inc. | Computer-implemented method and apparatus for converting data according to a selected data transformation |
US5884310A (en) * | 1996-06-14 | 1999-03-16 | Electronic Data Systems Corporation | Distributed data integration method and system |
US20010009547A1 (en) * | 2000-01-25 | 2001-07-26 | Akira Jinzaki | Data communications system |
US20030187864A1 (en) * | 2002-04-02 | 2003-10-02 | Mcgoveran David O. | Accessing and updating views and relations in a relational database |
US20040024625A1 (en) * | 2002-08-01 | 2004-02-05 | International Business Machines Corporation | Method for efficient dat transformation |
US6877150B1 (en) * | 2002-12-04 | 2005-04-05 | Xilinx, Inc. | Method of transforming software language constructs to functional hardware equivalents |
US20050108265A1 (en) * | 2001-12-12 | 2005-05-19 | Dirk Langkafel | System and method for projecting transformations of object trees |
US20050154979A1 (en) * | 2004-01-14 | 2005-07-14 | Xerox Corporation | Systems and methods for converting legacy and proprietary documents into extended mark-up language format |
US20060056338A1 (en) * | 2004-08-31 | 2006-03-16 | Ntt Docomo, Inc. | Communication system and method using a relay node |
US20070266347A1 (en) * | 2006-05-10 | 2007-11-15 | Tatung Company | Method of automatic synthesis of sequential quantum Boolean circuits |
US20080021912A1 (en) * | 2006-07-24 | 2008-01-24 | The Mitre Corporation | Tools and methods for semi-automatic schema matching |
US7616984B2 (en) * | 2002-04-06 | 2009-11-10 | National Institutes Of Health (Nih) | Modification of the normalized difference method for real-time optical tomography |
US8032828B2 (en) * | 2002-03-04 | 2011-10-04 | Hewlett-Packard Development Company, L.P. | Method and system of document transformation between a source extensible markup language (XML) schema and a target XML schema |
Family Cites Families (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5428554A (en) * | 1992-09-03 | 1995-06-27 | International Business Machines Corporation | Hierarchical graph analysis method and apparatus |
US6778531B1 (en) * | 1999-11-04 | 2004-08-17 | Lucent Technologies Inc. | Multicast routing with service-level guarantees between ingress egress-points in a packet network |
CA2401444C (en) * | 2000-03-01 | 2014-08-12 | Celltrex Ltd. | System and method for rapid document conversion |
US7362974B2 (en) * | 2001-11-30 | 2008-04-22 | Pirelli & C. S.P.A. | Method for planning or provisioning data transport networks |
-
2002
- 2002-08-01 US US10/211,171 patent/US7356480B2/en active Active
-
2008
- 2008-02-19 US US12/033,024 patent/US8180809B2/en not_active Expired - Fee Related
Patent Citations (20)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5261080A (en) * | 1987-08-21 | 1993-11-09 | Wang Laboratories, Inc. | Matchmaker for assisting and executing the providing and conversion of data between objects in a data processing system storing data in typed objects having different data formats |
US5418717A (en) * | 1990-08-27 | 1995-05-23 | Su; Keh-Yih | Multiple score language processing system |
US5694578A (en) * | 1992-12-18 | 1997-12-02 | Silicon Graphics, Inc. | Computer-implemented method and apparatus for converting data according to a selected data transformation |
US5884310A (en) * | 1996-06-14 | 1999-03-16 | Electronic Data Systems Corporation | Distributed data integration method and system |
US7133407B2 (en) * | 2000-01-25 | 2006-11-07 | Fujitsu Limited | Data communications system |
US20010009547A1 (en) * | 2000-01-25 | 2001-07-26 | Akira Jinzaki | Data communications system |
US20050108265A1 (en) * | 2001-12-12 | 2005-05-19 | Dirk Langkafel | System and method for projecting transformations of object trees |
US8032828B2 (en) * | 2002-03-04 | 2011-10-04 | Hewlett-Packard Development Company, L.P. | Method and system of document transformation between a source extensible markup language (XML) schema and a target XML schema |
US20030187864A1 (en) * | 2002-04-02 | 2003-10-02 | Mcgoveran David O. | Accessing and updating views and relations in a relational database |
US7263512B2 (en) * | 2002-04-02 | 2007-08-28 | Mcgoveran David O | Accessing and updating views and relations in a relational database |
US7616984B2 (en) * | 2002-04-06 | 2009-11-10 | National Institutes Of Health (Nih) | Modification of the normalized difference method for real-time optical tomography |
US7356480B2 (en) * | 2002-08-01 | 2008-04-08 | International Business Machines Corporation | Method of data transformation via efficient path discovery using a digraph |
US20040024625A1 (en) * | 2002-08-01 | 2004-02-05 | International Business Machines Corporation | Method for efficient dat transformation |
US6877150B1 (en) * | 2002-12-04 | 2005-04-05 | Xilinx, Inc. | Method of transforming software language constructs to functional hardware equivalents |
US20050154979A1 (en) * | 2004-01-14 | 2005-07-14 | Xerox Corporation | Systems and methods for converting legacy and proprietary documents into extended mark-up language format |
US20060056338A1 (en) * | 2004-08-31 | 2006-03-16 | Ntt Docomo, Inc. | Communication system and method using a relay node |
US7525939B2 (en) * | 2004-08-31 | 2009-04-28 | Ntt Docomo, Inc. | Communication system and method using a relay node |
US20070266347A1 (en) * | 2006-05-10 | 2007-11-15 | Tatung Company | Method of automatic synthesis of sequential quantum Boolean circuits |
US7398507B2 (en) * | 2006-05-10 | 2008-07-08 | Tatung Company | Method of automatic synthesis of sequential quantum Boolean circuits |
US20080021912A1 (en) * | 2006-07-24 | 2008-01-24 | The Mitre Corporation | Tools and methods for semi-automatic schema matching |
Also Published As
Publication number | Publication date |
---|---|
US7356480B2 (en) | 2008-04-08 |
US20040024625A1 (en) | 2004-02-05 |
US20110072053A9 (en) | 2011-03-24 |
US8180809B2 (en) | 2012-05-15 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US8180809B2 (en) | Method for efficient data transformation | |
US10606834B2 (en) | Methods and apparatus of shared expression evaluation across RDBMS and storage layer | |
US7599948B2 (en) | Object relational mapping layer | |
Gao et al. | Relational approach for shortest path discovery over large graphs | |
US8346812B2 (en) | Indexing in a resource description framework environment | |
US7707230B1 (en) | Methods and structure for use of an auxiliary database for importation of data into a target database | |
Su et al. | Smartsupply: smart contract based validation for supply chain blockchain | |
US7162478B2 (en) | System and method for correlated fragmentations in databases | |
Wang et al. | Distributed Pregel-based provenance-aware regular path query processing on RDF knowledge graphs | |
Zhang et al. | Reducing uncertainty of schema matching via crowdsourcing with accuracy rates | |
Shen et al. | Approximate covering detection among content-based subscriptions using space filling curves | |
US7472130B2 (en) | Select indexing in merged inverse query evaluations | |
US20080294673A1 (en) | Data transfer and storage based on meta-data | |
Elmeleegy et al. | Leveraging query logs for schema mapping generation in U-MAP | |
Cicalese et al. | On the tree search problem with non-uniform costs | |
Zhang et al. | Selecting the optimal groups: Efficiently computing skyline k-cliques | |
Micale et al. | TemporalRI: subgraph isomorphism in temporal networks with multiple contacts | |
US6839749B1 (en) | Network representation and manipulation thereof | |
Montz | Safety and optimization transformations for data flow programs | |
Qin et al. | Efficient XML query and update processing using a novel prime-based middle fraction labeling scheme | |
Zaitsev et al. | Spatial specification of hypertorus interconnect by infinite and reenterable coloured Petri nets | |
Regardt et al. | Anchor Modeling: An Agile Modeling Technique Using the Sixth Normal Form for Structurally and Temporally Evolving Data | |
CN111259082A (en) | Method for realizing full data synchronization in big data environment | |
WO2017176144A1 (en) | Data handling device, database system and method for operating a database system with efficient communication | |
Schmid | Towards Storing 3D Model Graphs in Relational Databases |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
FEPP | Fee payment procedure |
Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
REMI | Maintenance fee reminder mailed | ||
LAPS | Lapse for failure to pay maintenance fees | ||
STCH | Information on status: patent discontinuation |
Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362 |
|
FP | Lapsed due to failure to pay maintenance fee |
Effective date: 20160515 |