WO2015074466A1 - Data search method and apparatus - Google Patents

Abstract

Disclosed are a data search method and apparatus. In the data search method, multiple historical search request and timestamps thereof, as well as historical search results corresponding to the historical search requests one by one are prestored. The search method comprises: receiving a data search request; performing matching on the data search request and the multiple historical search requests; dividing the data search request into at least one sub-search request according to a matching result; performing a data search according to the sub-search request; and combining search results of the at least one sub-search request, and using the combined search results as results of the data search request. In the method, a materialized view does not need to be established and maintained, and repeated searches do not need to be performed for historical result data and data on which a search has been performed. In this way, data amounts of scanning and computation can be significantly reduced, thereby improving search efficiency, enhancing search performance, and reducing a search cost.

Description

Data query method and device Technical field

The present invention relates to the field of data processing technologies, and in particular, to a data query method and apparatus.

Background technique

With the rapid development of IT technology and the Internet, many applications are accompanied by a large amount of data, such as stock trading, e-commerce, social networking and so on. These applications gradually use NoSQL to save data. The data storage method is characterized by the fact that most of the data is saved in an additional way. The data is updated and deleted less frequently. The deletion is usually a new record with a delete tag, and each record is There is a record time, the timestamp.

Based on the foregoing data saving method, in the prior art, in order to improve query efficiency when performing data query for big data in a cloud environment, it is necessary to define a query request in advance, and calculate and materialize according to a predefined query request and original data in the database. The view, when receiving the query request, if the query request is a predefined query request, the materialized view is calculated according to the pre-rule, and the query result is directly output, and for the undefined query request, the full scan is performed on the original data. And then output the query results. However, in this method, if the query efficiency is to be guaranteed, the materialized view needs to be maintained in real time. For example, whenever new data is inserted, the materialized view needs to be recalculated, and the maintenance process of the materialized view is complicated and costly.

Summary of the invention

In the embodiment of the present invention, a data query method and device are provided, which can ensure data query efficiency without maintaining a materialized view.

In order to solve the above technical problem, the embodiment of the present invention discloses the following technical solutions:

The first aspect provides a data query method, which pre-stores a plurality of historical query requests and their timestamps, and a historical query result corresponding to the historical query request, the query method includes:

Receiving a data query request;

Matching the data query request with the plurality of historical query requests;

Dividing the data query request into at least one sub-query request according to the matching result;

Performing a data query according to the subquery request;

Merging the query result of the at least one subquery request and using the merged query result as a result of the data query request.

With reference to the foregoing first aspect, in a first possible implementation, the decomposing the data query request into the at least one subquery request according to the matching result includes:

When the data query request completely matches the first historical query request in the historical query request, the data query request is decomposed into a historical data sub-query request and an incremental data sub-query request;

The performing data query according to the subquery request includes:

Querying, according to the historical data sub-query request, the first historical query result corresponding to the first historical query request in the historical query result;

And querying, according to the incremental data sub-query request, data of the requested query of the data query request in the original data after the timestamp is located in the timestamp of the first historical query request.

In combination with the foregoing first aspect, and/or the first possible implementation manner, in a second possible implementation manner, the method further includes:

Updating the first historical query result to the merged query result, and updating the timestamp of the first historical query request to the latest recording time of the queryed data according to the incremental data subquery request.

With reference to the foregoing first aspect, and/or the first possible implementation manner, and/or the second possible implementation manner, in a third possible implementation manner, the decomposing the data query request according to the matching result Request for at least one subquery, including:

When the data query request does not completely match the plurality of historical query requests, the data query request is decomposed into a new sub-query request;

The performing data query according to the subquery request includes:

The data requested by the data query request is queried in all the original data according to the brand new subquery request.

In combination with the first aspect above, and/or the first possible implementation, and/or the second The implementation of the energy, and/or the third possible implementation manner, in the fourth possible implementation manner, the method further includes:

Adding the data query request as the second historical query request to the plurality of historical query requests, and using the latest record time of the queryed data according to the brand new subquery request as the time stamp of the second historical query request. And the merged query result is used as a historical query result corresponding to the second historical query request.

In combination with the first aspect above, and/or the first possible implementation, and/or the second possible implementation, and/or the third possible implementation, and/or the fourth possible implementation, In a fifth possible implementation, the decomposing the data query request into the at least one subquery request according to the matching result includes:

When the data query request partially matches the first historical query request of the plurality of historical query requests, the data query request is decomposed into a historical data subquery request, an incremental data subquery request, and a brand new subquery request. ;

The performing data query according to the subquery request includes:

Querying, according to the historical data sub-query request, the first historical query result corresponding to the first historical query request in the historical query result;

And querying, according to the incremental data sub-query request, the data of the requested query by the first historical query request in the original data after the timestamp of the first historical query request;

And querying data of the requested query in the data query request that is not matched with the first historical query request in all the original data according to the brand new subquery request.

In combination with the first aspect above, and/or the first possible implementation, and/or the second possible implementation, and/or the third possible implementation, and/or the fourth possible implementation, And/or the fifth possible implementation manner. In the sixth possible implementation manner, the method further includes:

Updating the first historical query result to a merge result according to the result of the historical data sub-query request and the incremental data sub-query request, and updating the timestamp of the first historical query request to The time at which the incremental data subquery requests the most recent record of the queried data;

Adding the data query request as a second history query request to the plurality of historical query requests, and using the latest record time of the query data according to each of the subquery requests as a Determining a timestamp of the second historical query request, and using the merged query result as a historical query result corresponding to the second historical query request;

Adding the new subquery request as the third historical query request to the plurality of historical query requests, and using the latest record time of the queryed data according to the new subquery request as the time stamp of the third historical query request And the query result of the new subquery request is used as a historical query result corresponding to the third historical query request.

In a second aspect, a data query apparatus is provided, including:

a storage unit, configured to store a plurality of historical query requests and their timestamps, and a historical query result corresponding to the historical query request;

a receiving unit, configured to receive a data query request;

a matching unit, configured to match the data query request with multiple historical query requests stored in the storage unit;

a decomposition unit, configured to divide the data query request into at least one sub-query request according to the matching result;

a query unit, configured to perform data query according to the subquery request;

a result output unit, configured to merge the query result of the at least one subquery request, and use the merged query result as a result of the data query request.

With reference to the foregoing second aspect, in a first possible implementation, the decomposing unit is specifically configured to: when the data query request completely matches the first historical query request in the historical query request, The data query request is decomposed into a historical data subquery request and an incremental data subquery request;

The query unit includes:

a historical query unit, configured to query, in the historical query result, a first historical query result corresponding to the first historical query request according to the historical data sub-query request;

And an incremental query unit, configured to query, according to the incremental data subquery request, the data of the requested query of the data query request in the original data after the timestamp of the first historical query request.

With reference to the foregoing second aspect, and/or the first possible implementation manner, in a second possible implementation manner, the method further includes:

An update unit, configured to update the first historical query result to the merged query As a result, the timestamp of the first historical query request is updated to the latest recording time of the data queried according to the incremental data subquery request.

With reference to the foregoing second aspect, and/or the first possible implementation manner, and/or the second possible implementation manner, in a third possible implementation manner, the decomposing unit is specifically configured to use the data When the query request does not completely match the plurality of historical query requests, the data query request is decomposed into a new sub-query request;

The query unit includes a new query unit, and the new query unit is configured to query the data requested by the data query request in all the original data according to the brand new sub-query request.

In combination with the second aspect above, and/or the first possible implementation manner, and/or the second possible implementation manner, and/or the third possible implementation manner, in the fourth possible implementation manner, include:

An update unit, configured to add the data query request to the plurality of historical query requests as a second historical query request, and use the latest record time of the data according to the new sub-query as the second historical query request Time stamp, and the merged query result is used as a historical query result corresponding to the second historical query request.

In combination with the second aspect above, and/or the first possible implementation, and/or the second possible implementation, and/or the third possible implementation, and/or the fourth possible implementation, In a fifth possible implementation, the decomposing unit is specifically configured to: when the data query request partially matches a first historical query request of the plurality of historical query requests, decompose the data query request Subquery request for historical data, incremental data subquery request, and new subquery request;

The query unit includes:

a historical query unit, configured to query, in the historical query result, a first historical query result corresponding to the first historical query request according to the historical data sub-query request;

An incremental query unit, configured to query, according to the incremental data subquery request, data of the requested query of the first historical query request in the original data after the timestamp of the first historical query request ;

The new query unit is configured to query, in the original data, the data of the requested query in the data query request that is not matched with the first historical query request according to the new subquery request.

In combination with the second aspect above, and/or the first possible implementation, and/or the second possible implementation, and/or the third possible implementation, and/or the fourth possible implementation, And/or the fifth possible implementation manner. In the sixth possible implementation manner, the method further includes:

And an updating unit, configured to update the first historical query result to a combined result according to the result of the historical data subquery request and the incremental data subquery request, and the first historical query request Updating the timestamp to the latest recording time of the queried data according to the incremental data subquery request; adding the data query request as a second historical query request to the plurality of historical query requests, according to each of the sub-questions Querying the latest record time of the data requested by the query as the time stamp of the second historical query request, and using the merged query result as a historical query result corresponding to the second historical query request; The query request is added to the plurality of historical query requests as a third history query request, and the latest record time of the query data according to the brand new subquery request is used as a time stamp of the third history query request, and the The query result of the new subquery request is the result of the historical query corresponding to the third historical query request.

The embodiment of the present invention can store the historical query result and the historical query result in advance, and can reuse the historical query result when receiving the data query request, and obtain the final data query result by combining the historical query result and the incremental query result, and the method does not need to be established. And maintain the materialized view, and do not need to repeat the query for the historical result data and the query, which can greatly reduce the amount of data scanned and calculated, improve the query efficiency, improve the query performance, and reduce the query cost.

DRAWINGS

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the description of the prior art will be briefly described below, and it will be apparent to those skilled in the art that In other words, other drawings can be obtained based on these drawings without paying for creative labor.

1 is a flowchart of a data query method according to an embodiment of the present invention;

2 is a flowchart of another data query method according to an embodiment of the present invention;

3 is a schematic diagram of a data query framework according to an embodiment of the present invention;

4 is a flowchart of another data query method according to an embodiment of the present invention;

FIG. 5 is a flowchart of another data query method according to an embodiment of the present invention;

6 is a schematic diagram of waiting for execution when multiple identical data query requests exist simultaneously in an embodiment of the present invention;

FIG. 7 is a schematic structural diagram of a data query apparatus according to an embodiment of the present invention;

FIG. 8 is a schematic structural diagram of another data query apparatus according to an embodiment of the present invention.

detailed description

The above-mentioned objects, features, and advantages of the embodiments of the present invention will become more apparent and understood. Give further details.

FIG. 1 is a flowchart of a data query method according to an embodiment of the present invention.

In the embodiment of the present invention, two databases may be pre-established for storing a historical query request (and its timestamp) and a historical query result corresponding to the historical query request. In the initial stage, the two databases are empty, and the user is in the initial stage. After the data query is performed, the data query request is added as a historical query request to the database for storing the historical query request, and the query result corresponding to the data query request is stored as a historical query result in a database for storing the historical query result. When the data query is performed later, the query may be performed according to the method of the embodiment of the present invention. Each historical query request records a timestamp, which indicates both the time of receiving the historical query request and the time when the original data from the historical query result corresponding to the historical query request is recorded before the timestamp. Raw data.

The data query method can include:

Step 101: Receive a data query request.

Step 102: Match the data query request with the plurality of historical query requests.

After receiving the data query request, the historical query request stored in the database is searched and matched with the received data query request one by one. The result of the matching may be a historical query request or a plurality of historical query requests in the database that exactly match the received data query request. For example, the received data query request is query data A+B, and the database exists. Query the historical query request of the data A+B, or the historical query request of the query data A and the historical query request of the query data B, which is also an exact match; or the historical query request stored in the database and the received data The query request does not match at all. For example, the received data query request is the query data A+B, the database does not have any historical query request containing the query data A, and there is no historical query request containing the query data B; there may be a presence and receipt in the database. The data query request partially matches the historical query request. For example, the received data query request is the query data A+B, and the historical query request of the query data A exists in the database, but there is no historical query request of the query data B.

The above matching process may adopt keyword matching or other matching methods, which is not limited herein.

Step 103: Divide the data query request into at least one sub-query request according to the matching result.

After obtaining the above matching result, or not matching at all, or not matching the matching result, the data query request is decomposed according to the matching result in this step.

If the matching result is an exact match or a complete mismatch, the data query request is divided into subquery requests for the same data in the specified data storage area; if the matching result is not completely matched, the data query request is split into multiple A subquery request for different data in multiple specified data storage areas. For details, please refer to the description of the subsequent embodiments.

Step 104: Perform a data query according to the subquery request.

Step 105: Combine the query result of the at least one subquery request, and use the merged query result as a result of the data query request.

After obtaining the query result of each subquery request, the results of each query are combined, and the combined result is the result of the data query request.

The embodiment of the present invention can store the historical query result and the historical query result in advance, and can reuse the historical query result when receiving the data query request, and obtain the final data query result by combining the historical query result and the incremental query result, and the method does not need to be established. And maintain the materialized view, and do not need to repeat the query for the historical result data and the query, which can greatly reduce the amount of data scanned and calculated, improve the query efficiency, improve the query performance, and reduce the query cost.

2 is a flowchart of another data query method according to an embodiment of the present invention.

FIG. 3 is a schematic diagram of a data query framework according to an embodiment of the present invention, wherein three databases, a historical query request database, and a cloud database 1 and a cloud database 2 are pre-established, and the historical query request database includes all historical query request data and Timestamp, the cloud database 1 stores all the original data and its timestamp, and the cloud database 2 stores the historical query. The historical query request in the request database has a one-to-one correspondence with the historical query result. The query method can include:

Step 201: Receive a data query request.

Assume that the received data query request is the query data A+B.

Step 202: Match the data query request with multiple historical query requests.

Match the data query request with the historical query request in the historical query request database. In this embodiment, it is assumed that the matching result is that the historical query request database has a first historical query request that completely matches the data query request. For example, the historical query request of the query data A+B exists in the database.

After determining the matching result, step 203 is performed.

In step 203, the data query request is decomposed into a historical data subquery request and an incremental data subquery request.

After determining the exact match, the data query request can be decomposed into a historical data sub-query request for querying the historical query result in the cloud database 2, and an incremental data sub-query request queried in the newly added original data of the cloud database 1.

After the query request is decomposed, the process proceeds to steps 204-205, and the data query is performed according to the sub-query request.

Step 204: Query the first historical query result corresponding to the first historical query request in the historical query result according to the historical data sub-query request.

The first historical query result corresponding to the first historical query request is directly found according to the historical data sub-query request, and is recorded as R1.

Step 205: Query, according to the incremental data sub-query request, the data requested by the data query request in the original data after the timestamp is located in the timestamp of the first historical query request.

Before performing an incremental query, first determine the new raw data in the cloud database 1 that needs to be queried.

If the first historical query request is the request query data A+B, and the timestamp of the first historical query request is T1 according to the historical query request, the newly added original data in the cloud database 1 to be queried is a timestamp. The original data after T1, according to the incremental data subquery request, find the data A+B in the newly added original data, obtain the result R4, and record the current The latest record timestamp T1' of the original data in the cloud database 1 being queried.

Step 206: Combine the query result of the historical data subquery request and the incremental data subquery request, and use the merged query result as a result of the data query request.

If the first historical query request is the request query data A+B, the results R1 and R4 are combined as a result of the data query request.

This embodiment may further include:

Step 207: Update the first historical query result to the merged query result, and update the timestamp of the first historical query request to the latest record of the data queried according to the incremental data subquery request. time

Specifically, if the first historical query request is the request query data A+B, the merged results R1 and R4 are updated to the new first historical query result corresponding to the first historical query request, and the first historical query request is obtained. The timestamp is updated to T1'.

In another embodiment, taking the received data query request as the query data A+B as an example, when the data query request is matched with multiple historical query requests, the matching result is the presence and the data query request. A fully matched historical query request, however, the fully matched historical query result is a collection of multiple historical query requests, such as a historical query request for query data A and a historical query request for query data B.

The data query request is decomposed into a historical data subquery request and an incremental data subquery request. Then, an operation similar to steps 204-205 is performed for each historical query request in the set: if the fully matched historical query request is a historical query request of the query data A and a set of historical query requests of the query data B, then The historical data subquery request finds the historical query result corresponding to the historical query request of the query data A and the historical query result corresponding to the historical query request of the query data B in the historical query result, respectively, and records them as R2 and R3, respectively. The timestamp of the historical query request of the data A is T2, and the timestamp of the historical query request of the data B is T3, and the data A is searched in the original data after the time T2 is recorded in the cloud database 1 according to the incremental data subquery request. The result R5 is obtained, and the latest time stamp T2' of the found original data is recorded, the data B is searched for in the original data after the recording time T3, the result R6 is obtained, and the latest time stamp T3' of the found original data is recorded.

Operations similar to step 206 are then performed for the results of all queries, combining the results R2, R3 and R5, R6 as a result of the data query request.

Then perform a similar operation to step 207. Specifically, the data may be first queried. The request is added as a new history query request in the database of the historical query request, and the latest time in T2' and T3' is recorded as a timestamp, and the historical query result corresponding to the newly added historical query request is R2, R3. And the result of the combination of R5 and R6 is added to the cloud database 2, and then the historical query result corresponding to the historical query request of the query data A is updated to R2+R5, the update timestamp is T2', and the historical query request of the query data B is corresponding. The historical query result is updated to R3+R6 and the update timestamp is T3'.

In another embodiment, assuming that the received data query request is the query data A, when the data query request is matched with the plurality of historical query requests, the result of the matching is that there is a historical query request that exactly matches the data query request. , but the matching historical query request includes the received data query request, such as the query data A or B when the found complete matching historical query request is found.

The data query request is decomposed into a historical data subquery request and an incremental data subquery request. Then, operations similar to steps 204-205 are performed:

First, the historical query result corresponding to the historical query request of the query data A or B is searched in the historical query result according to the historical data sub-query request, and then the query result corresponding to the query data A is filtered out from the historical query result, that is, Rx, And obtain the timestamp Tx of the historical query request A or B. The data A is searched in the original data after the time Tx is recorded in the cloud database 1 according to the incremental data subquery request, the result Ry is obtained, and the latest time stamp Tx' of the found original data is recorded.

Operations similar to step 206 are then performed for the results of all queries, combining the results Rx, Ry as a result of the data query request.

Then, the operation similar to step 207 is performed. Specifically, the data query request may be first added as a new history query request in the database of the historical query request, and the record timestamp is Tx', and the newly added history query request is added. The result of the corresponding historical query, that is, the combined result of Rx and Ry is added to the cloud database 2.

FIG. 4 is a flowchart of another data query method according to an embodiment of the present invention.

For example, the data query framework shown in FIG. 3 is taken as an example, and the query method may include:

Step 401: Receive a data query request.

Step 402: Match the data query request with multiple historical query requests.

Match the data query request with the historical query request in the historical query request database. In this embodiment, it is assumed that the matching result is that the historical query request database does not exist and the data If the query request matches the historical query request, step 403 is performed.

In step 403, the data query request is decomposed into a new subquery request.

After the complete mismatch is determined, it can be determined that the query is not performed in the cloud database 2, and the step 404 is directly executed to query the data of the requested query in the cloud database 1.

Step 404: Query the data requested by the data query request in all the original data according to the new sub-query request.

In step 405, the query result of the new subquery request is taken as the result of the data query request.

The embodiment may further include:

Step 406: Add the data query request to the plurality of historical query requests as a second historical query request, and use the latest record time of the query data according to the new subquery request as the second historical query request. a timestamp, and the merged query result is used as a historical query result corresponding to the second historical query request.

FIG. 5 is a flowchart of another data query method according to an embodiment of the present invention.

For example, the data query framework shown in FIG. 3 is taken as an example, and the query method may include:

Step 501: Receive a data query request.

The query data A+B is still taken as an example for the received data query request.

Step 502: Match the data query request with multiple historical query requests.

Match the data query request with the historical query request in the historical query request database. In this embodiment, it is assumed that the matching result is that the data query request partially matches the first historical query request in the historical query request database. For example, only the historical query request of the query data A exists in the historical query request database, and the query data A+ does not exist. B or data B history query request.

After determining that the matching result is not completely matched, step 503 is performed.

Step 503: Decompose the data query request into a historical data sub-query request, an incremental data sub-query request, and a brand new sub-query request.

After the data query request is decomposed, steps 504-506 are performed to perform the query.

Step 504: Query, according to the historical data sub-query request, the first historical query result corresponding to the first historical query request in the historical query result.

The first historical query result corresponding to the first historical query request that is partially matched with the data query request is queried in the cloud database 2. The query process is the same as step 204 in the foregoing embodiment. like.

It is assumed that only the first historical query request of the query data A exists in the historical query request database, the time stamp of the first historical query request is t1, and the first historical query result found is X1.

Step 505: Query, according to the incremental data sub-query request, the data requested by the first historical query request in the original data after the timestamp is located in the timestamp of the first historical query request.

For the original data in the cloud database 1 after the timestamp t1 of the first historical query request is recorded, the data A is queried according to the incremental data subquery request, and the query result is X2, and the original query of the cloud database 1 is recorded. The latest record time of the data is t2.

Step 506: Query, according to the new sub-query request, data in the data query request that is requested by the first historical query request mismatch portion in all the original data.

For the query of the data B, since there is no relevant historical query request, the data B is directly searched for all the original data in the cloud database 1, and the query result is X3, and the latest recording time t3 of the original data in the current cloud database 1 is recorded.

Step 507: merge the historical data subquery request, the incremental data subquery request, and the query result of the brand new subquery request, and use the merged query result as a result of the data query request.

The query result of the data query request is the combined result of X1, X2 and X3.

This embodiment may further include:

Step 508, updating the historical query request database and the cloud database 1 and the cloud database 2.

These include:

Updating the first historical query result to a merge result according to the result of the historical data sub-query request and the incremental data sub-query request, and updating the timestamp of the first historical query request to The incremental data subquery requests the latest recording time of the queried data; specifically, the first historical query result corresponding to the first historical query request of the query data A is updated to X1+X2, and the time of the first historical query request is The stamp is updated to the latest time in t1, t2.

Adding the data query request as a second historical query request to the plurality of historical query requests, according to the latest recording time of the data that is queried by the subquery request a timestamp of the second historical query request, and the merged query result is used as a historical query result corresponding to the second historical query request; specifically, the data query request of the query data A+B is used as the second history The query request is added to the historical query request database, the timestamp is the latest time in t1, t2, t3, and the historical query result X1+X2+X3 corresponding to the second historical query request is added to the cloud database 2.

Adding the new subquery request as the third historical query request to the plurality of historical query results, and using the latest record time of the queryed data according to the brand new subquery request as the time of the third historical query request Stamping, and using the query result of the new subquery request as a historical query result corresponding to the third historical query request; specifically, adding a new subquery request of the query data B as a third historical query request to the historical query In the requested database, the timestamp is t3, and the query result X3 corresponding to the third historical query request is added to the cloud database 2.

In another embodiment of the present invention, when a plurality of identical data query requests exist at the same time, data query may be performed by using a method such as waiting for execution or concurrent execution.

The manner of waiting for execution is as shown in FIG. 6. When performing task 1 to query data A, it is first determined whether the corresponding branch in the historical query request database is in progress, and the corresponding branch is also querying whether the history query request database exists. The query request of the data A, if yes, waits for the execution of the corresponding branch to succeed, and if the corresponding branch is successfully executed, executes the next data query task; if the corresponding branch execution is unsuccessful or no corresponding branch is in progress, the task 1 is executed, and the task is executed. The corresponding branch annotation of the task in the historical query request database is in progress. If the execution of the task 1 is successful, the historical query request database is updated, and the new historical query result is written into the cloud database 2, and then the next data query task is executed, if the task 1 is performed If the execution is unsuccessful, the annotations created on the corresponding branches in the historical query request database are cleared and then exited.

In the parallel execution mode, when the task 1 performs the query on the data A, the existing historical query request and the historical query result in the cloud database 2 are directly applied, and the same task currently being performed is not detected, indicating that the task is in the task. 1 After the execution is completed, when updating the historical query request and the historical query result, it is first determined whether the time stamp of the currently matched historical query request is newer than the current task 1, and if so, the timestamp of the historical query request is not updated, nor updated. The corresponding historical query result in the cloud database 2, if not, the timestamp of the historical query request is updated to the timestamp of the current task 1, and the new historical query result is updated to the cloud database 2.

In the embodiment of the present invention, the cloud database 2 may be periodically cleaned according to the historical query request database, that is, the invalid historical query result in the cloud database 2 is deleted according to the historical query request, and the historical query request database may be periodically merged or deleted. And periodically merge or delete historical query results for cloud database 2.

The embodiment of the present invention decomposes the data query request into one or more of the new query, the historical query, and the incremental query according to the data query request matching the historical query request, and does not need to repeat the query request and the data that has been queried. Query, which can greatly reduce the amount of data scanned and calculated, improve query efficiency, and reduce query cost, and the present invention uses historical query database and cloud database 2 to store historical query requests and corresponding historical query result metadata, each time After the query is completed, the database can be updated with individual data, which is much less expensive than the prior art materialized view maintenance method.

The above is a description of the embodiment of the method of the present invention, and the apparatus for implementing the above method will be described below.

FIG. 7 is a schematic structural diagram of a data query apparatus according to an embodiment of the present invention.

The device can include:

The storage unit 701 is configured to store a plurality of historical query requests and their timestamps, and a historical query result corresponding to the historical query request;

The receiving unit 702 is configured to receive a data query request.

The matching unit 703 is configured to match the data query request with multiple historical query requests stored in the storage unit 701.

The decomposing unit 704 is configured to divide the data query request into at least one subquery request according to the matching result;

The query unit 705 is configured to perform data query according to the subquery request;

The result output unit 706 is configured to merge the query result of the at least one subquery request, and use the merged query result as a result of the data query request.

After receiving the data query request, the receiving unit 702 matches the data query request with the plurality of historical query requests stored in the storage unit 701, and the decomposing unit 704 divides the data query request into at least one according to the matching result. The subquery request may be one or more of a historical data subquery request, an incremental data subquery request, and a brand new subquery request, and then the query unit 705 performs data query according to the decomposed subquery request, and finally outputs the result. Unit 706 merge query unit 705 according to respective subquery request The result of the query is the result of the data query request.

The embodiment of the present invention pre-stores the historical query request and the historical query result by using the foregoing unit, and can reuse the historical query result when receiving the data query request, and obtain the final data query result by combining the historical query result and the incremental query result, the device There is no need to establish and maintain a materialized view, and there is no need to repeat the query for historical result data and queries. This can greatly reduce the amount of data scanned and calculated, improve query efficiency, improve query performance, and reduce query costs.

FIG. 8 is a schematic structural diagram of another data query apparatus according to an embodiment of the present invention.

The device may include an update unit 807 in addition to the storage unit 801, the receiving unit 802, the matching unit 803, the decomposition unit 804, the query unit 805, and the result output unit 806.

The storage unit 801 stores three databases, including a historical query request database, a cloud database 1 and a cloud database 2. The historical query request database includes a plurality of historical query requests and their timestamps, and the cloud database 1 includes all the original data and The time stamp, the cloud database 2 includes a historical query result corresponding to the historical query request.

In this embodiment, the decomposing unit 804 may be specifically configured to: when the data query request completely matches the first historical query request in the historical query request, decompose the data query request into a historical data subquery request and An incremental data subquery request; when the data query request does not completely match the plurality of historical query requests, decomposing the data query request into a new subquery request; and when the data query request is related to the plurality of When the first historical query request in the historical query request partially matches, the data query request is decomposed into a historical data sub-query request, an incremental data sub-query request, and a brand new sub-query request.

The query unit 805 can include:

The history query unit 8051 is configured to query, according to the historical data sub-query request, the first historical query result corresponding to the first historical query request in the historical query result;

The incremental query unit 8052 is configured to query, according to the incremental data sub-query request, the data requested by the data query request in the original data after the timestamp is located in the timestamp of the first historical query request;

a new query unit 8053, configured to query, in the original data, the data query request and the data requested by the unmatched portion of the plurality of historical query requests in all the original data according to the new subquery request, that is, when Data query request and the multiple historical queries please When the complete mismatch is found, the data requested by the data query request is queried in all the original data according to the new subquery request, when the data query request and the first of the plurality of historical query requests are When the historical query request is partially matched, the data requested by the part of the data query request that is not matched with the first historical query request is queried.

The result output unit 806 is configured to merge and output the query result of the history query unit 8051, the delta query unit 8052, and the new query unit 8053 according to the subquery request decomposed by the decomposition unit 804.

The updating unit 807 is configured to update the first historical query result to a result that is queried according to the historical data subquery request and the incremental data subquery request when the matching result is a perfect match or an incomplete match. Merging the result, and updating the timestamp of the first historical query request to the latest recording time of the data queried according to the incremental data subquery request; when the matching result is not completely matched or completely mismatched, The data query request is added to the plurality of historical query requests as a second historical query request, and the latest record time of the query data according to each of the sub-query requests is used as a time stamp of the second historical query request, and The merged query result is used as a historical query result corresponding to the second historical query request; when the matching result is not completely matched, the new subquery request is added as the third historical query request to the plurality of history In the query request, the latest record time of the query data according to the new subquery request is used as the third history query. Time stamp, and the new sub-query request as a query result historical query results corresponding to the query request third history.

Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the various examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the solution. A person skilled in the art can use different methods for implementing the described functions for each particular application, but such implementation should not be considered to be beyond the scope of the present invention.

A person skilled in the art can clearly understand that for the convenience and brevity of the description, the specific working process of the system, the device and the unit described above can refer to the corresponding process in the foregoing method embodiment, and details are not described herein again.

In the several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the device embodiment described above For example, the division of the unit is only a logical function division, and the actual implementation may have another division manner, for example, multiple units or components may be combined or may be integrated into another system, or some Features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in an electrical, mechanical or other form.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

The functions may be stored in a computer readable storage medium if implemented in the form of a software functional unit and sold or used as a standalone product. Based on such understanding, the technical solution of the present invention, which is essential or contributes to the prior art, or a part of the technical solution, may be embodied in the form of a software product, which is stored in a storage medium, including The instructions are used to cause a computer device (which may be a personal computer, server, or network device, etc.) or a processor to perform all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like. .

The above is only a specific embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily think of changes or substitutions within the technical scope of the present invention. It should be covered by the scope of the present invention. Therefore, the scope of the invention should be determined by the scope of the claims.

Claims (14)

1. A data query method, characterized in that a plurality of historical query requests and their time stamps are stored in advance, and historical query results corresponding to the historical query requests are one-to-one, the query method includes:

   Receiving a data query request;

   Matching the data query request with the plurality of historical query requests;

   Dividing the data query request into at least one sub-query request according to the matching result;

   Performing a data query according to the subquery request;

   Merging the query result of the at least one subquery request and using the merged query result as a result of the data query request.
2. The method according to claim 1, wherein the decomposing the data query request into at least one sub-query request according to the matching result comprises:

   When the data query request completely matches the first historical query request in the historical query request, the data query request is decomposed into a historical data sub-query request and an incremental data sub-query request;

   The performing data query according to the subquery request includes:

   Querying, according to the historical data sub-query request, the first historical query result corresponding to the first historical query request in the historical query result;

   And querying, according to the incremental data sub-query request, data of the requested query of the data query request in the original data after the timestamp is located in the timestamp of the first historical query request.
3. The method of claim 2, further comprising:

   Updating the first historical query result to the merged query result, and updating the timestamp of the first historical query request to the latest recording time of the queryed data according to the incremental data subquery request.
4. The method according to claim 1, wherein the decomposing the data query request into at least one sub-query request according to the matching result comprises:

   When the data query request does not completely match the plurality of historical query requests, the data query request is decomposed into a new sub-query request;

   The performing data query according to the subquery request includes:

   The data requested by the data query request is queried in all the original data according to the brand new subquery request.
5. The method of claim 4, further comprising:

   Adding the data query request as the second historical query request to the plurality of historical query requests, and using the latest record time of the queryed data according to the brand new subquery request as the time stamp of the second historical query request. And the merged query result is used as a historical query result corresponding to the second historical query request.
6. The method according to claim 1, wherein the decomposing the data query request into at least one sub-query request according to the matching result comprises:

   When the data query request partially matches the first historical query request of the plurality of historical query requests, the data query request is decomposed into a historical data subquery request, an incremental data subquery request, and a brand new subquery request. ;

   The performing data query according to the subquery request includes:

   Querying, according to the historical data sub-query request, the first historical query result corresponding to the first historical query request in the historical query result;

   And querying, according to the incremental data sub-query request, the data of the requested query by the first historical query request in the original data after the timestamp of the first historical query request;

   And querying data of the requested query in the data query request that is not matched with the first historical query request in all the original data according to the brand new subquery request.
7. The method of claim 6 further comprising:

   Updating the first historical query result to a merge result according to the result of the historical data sub-query request and the incremental data sub-query request, and updating the timestamp of the first historical query request to The time at which the incremental data subquery requests the most recent record of the queried data;

   Adding the data query request as the second historical query request to the plurality of historical query requests, and using the latest record time of the query data according to each of the subquery requests as the time stamp of the second historical query request. And the combined query result is used as the same Determining a historical query result corresponding to the second historical query request;

   Adding the new subquery request as the third historical query request to the plurality of historical query requests, and using the latest record time of the queryed data according to the new subquery request as the time stamp of the third historical query request And the query result of the new subquery request is used as a historical query result corresponding to the third historical query request.
8. A data query device, comprising:

   a storage unit, configured to store a plurality of historical query requests and their timestamps, and a historical query result corresponding to the historical query request;

   a receiving unit, configured to receive a data query request;

   a matching unit, configured to match the data query request with multiple historical query requests stored in the storage unit;

   a decomposition unit, configured to divide the data query request into at least one sub-query request according to the matching result;

   a query unit, configured to perform data query according to the subquery request;

   a result output unit, configured to merge the query result of the at least one subquery request, and use the merged query result as a result of the data query request.
9. The device of claim 8 wherein:

   The decomposing unit is specifically configured to decompose the data query request into a historical data sub-query request and an incremental data sub-query when the data query request completely matches the first historical query request in the historical query request. request;

   The query unit includes:

   a historical query unit, configured to query, in the historical query result, a first historical query result corresponding to the first historical query request according to the historical data sub-query request;

   And an incremental query unit, configured to query, according to the incremental data subquery request, the data of the requested query of the data query request in the original data after the timestamp of the first historical query request.
10. The device according to claim 9, further comprising:

    An update unit, configured to update the first historical query result to the merged query result, and update a timestamp of the first historical query request according to the incremental data sub Query the latest recording time of the data requested by the query.
11. The device of claim 8 wherein:

    The decomposing unit is configured to: when the data query request completely does not match the plurality of historical query requests, decompose the data query request into a new subquery request;

    The query unit includes a new query unit, and the new query unit is configured to query the data requested by the data query request in all the original data according to the brand new sub-query request.
12. The device according to claim 11, further comprising:

    An update unit, configured to add the data query request to the plurality of historical query requests as a second historical query request, and use the latest record time of the data according to the new sub-query as the second historical query request Time stamp, and the merged query result is used as a historical query result corresponding to the second historical query request.
13. The device of claim 8 wherein:

    The decomposing unit is configured to: when the data query request partially matches the first historical query request of the plurality of historical query requests, decompose the data query request into a historical data subquery request, incremental data Subquery request and new subquery request;

    The query unit includes:

    a historical query unit, configured to query, in the historical query result, a first historical query result corresponding to the first historical query request according to the historical data sub-query request;

    An incremental query unit, configured to query, according to the incremental data subquery request, data of the requested query of the first historical query request in the original data after the timestamp of the first historical query request ;

    The new query unit is configured to query, in the original data, the data of the requested query in the data query request that is not matched with the first historical query request according to the new subquery request.
14. The device according to claim 13, further comprising:

    And an updating unit, configured to update the first historical query result to a combined result according to the historical data subquery request and the result of the incremental data subquery request query, and Updating the timestamp of the first historical query request to the latest recording time of the data queried according to the incremental data subquery request; adding the data query request as the second historical query request to the plurality of historical query requests The latest record time of the query data according to each of the subquery requests is used as the time stamp of the second history query request, and the merged query result is used as the history corresponding to the second history query request. Querying the result; adding the new subquery request as the third historical query request to the plurality of historical query requests, and using the latest record time of the queried data according to the new subquery request as the third historical query request a timestamp, and the query result of the brand new subquery request is a historical query result corresponding to the third historical query request.

Images