CN102890679B - The processing method and system of a kind of versions of data - Google Patents
The processing method and system of a kind of versions of data Download PDFInfo
- Publication number
- CN102890679B CN102890679B CN201110203404.7A CN201110203404A CN102890679B CN 102890679 B CN102890679 B CN 102890679B CN 201110203404 A CN201110203404 A CN 201110203404A CN 102890679 B CN102890679 B CN 102890679B
- Authority
- CN
- China
- Prior art keywords
- time
- data
- balance
- module
- unfavourable balance
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Abstract
The invention discloses a kind of processing method of versions of data, the method includes：Start to recover data, read redolog files, obtain data time and data file timestamp in redolog files；If data file timestamp is time unfavourable balance with the data time in redolog files, and the unfavourable balance time is not less than tolerance parameter, or, data file timestamp is time favorable balance with the data time in redolog files, recovers the data in redolog files；If data file timestamp is time unfavourable balance with the data time in redolog files, and the unfavourable balance time, more than tolerance parameter, refusal recovers data.The invention also discloses a kind of processing system of versions of data, using method and system of the present invention, system update data, inquiry data, the accuracy of the versions of data of loading data file are improve.
Description
Technical field
The present invention relates to field of data storage, the processing method and system of a kind of versions of data are particularly related to.
Background technology
As 3G network is gradually ripe, cloud computing and the development of Internet of Things, telecommunication system are also faced with transition and more
Big challenge.Current telecommunication system more pays close attention to the performance of the aspects such as high availability, expansibility and mass memory.To High Availabitity
Property requirement be in infrastructure, storage with server and deposit, once server fail, user can normally access storage
Data, that is, require that server ensures that the data in storage system can use in case of a failure；Furthermore, it desired to system operation
With very strong fault-tolerant ability, to keep high reliability.The requirement of expansibility is being capable of automatic or simple manual configuration
Additions and deletions memory node, reduces system pressure, improves the process performance of system, reduce risk.Mass memory is to be required to quickly
Ground structure data storage, can intactly safeguard mass data, and can rapidly inquire about data.Just because of telecommunications system
The characteristics of requirement in terms of uniting to three above so that in telecommunication system, database, is also more and clearer and more definite, for example, distributed, negative
Carry balanced, failover and the management of small table etc..
Distributed data base must possess the functions such as storage and data query, to the version treatment side recorded in data file
Method is also varied, there is the modes such as release label, timestamp.The record can also be referred to as data.Each time to data
Treatment, correspond to a new version of data, the treatment includes that data update, delete, increasing etc..In distributed storage
In system, multiple memory nodes are distributed on different machines, the version of each memory node data in processing data file
When, mostly it is to use the time as version number, but because the time of each memory node does not ensure that absolute synchronization, so version
This number not very accurate.If common data file version number is inaccurate, the data for inquiring can be caused wrong, but, if
The version number of metadata is inaccurate, can badly influence systemic-function, and the system of may result in cannot normal work.
, it is necessary to safeguard the data of magnanimity in distributed memory system, and the data manipulation language (DML) of high intensity is grasped
Make, in addition it is also necessary to realize the functions such as load balancing, failover, this is directed to data, therefore, handle well with timestamp as data
The version problem of file record, there is very important meaning.
The content of the invention
In view of this, it is a primary object of the present invention to provide the processing method and system of a kind of versions of data, improve
System update data, inquiry data, the accuracy of the versions of data of loading data file.
To reach above-mentioned purpose, the technical proposal of the invention is realized in this way：
A kind of processing method of versions of data, including：Start to recover data, read weight Operation Log redolog files, obtain
Take the data time and data file timestamp in redolog files；If the number in data file timestamp and redolog files
It is time unfavourable balance according to the time, and the unfavourable balance time is not less than tolerance parameter, or, in data file timestamp and redolog files
Data time be time favorable balance, recover redolog files in data；If in data file timestamp and redolog files
Data time be time unfavourable balance, and the unfavourable balance time exceed tolerance parameter, refusal recover data.
It is described start recover data before, the method is further included：It is stopped by hosting node, control node is assigned
Hosting node carries out failover, hosting node according to by the nodename of hosting node, obtain in metadata table with the quilt
The final updating time of the corresponding small table of nodename of hosting node, compared with the current time of itself, if current time with
The final updating time of small table is time favorable balance, then normal adapter；Otherwise, refusal adapter.
A kind of processing method of versions of data, including：Memory node reads the data file of itself, when obtaining data file
Between stab, compare with current time, if current time and data file timestamp are time favorable balance, load data file；If working as
The preceding time is time unfavourable balance with data file timestamp, and the unfavourable balance time is more than tolerance parameter, then refuse loading data file；If
Current time is time unfavourable balance with data file timestamp, and the unfavourable balance time, not less than tolerance parameter, the mode of wait eliminates inverse
Difference, loads data file.
A kind of processing method of versions of data, including：Control node determines the corresponding memory node of the data to be updated, and deposits
Storage node obtains the final updating time of small table belonging to the data to be updated, and compares with current time, if current time and small table
The final updating time be time favorable balance, or, the final updating time of current time and small table is time unfavourable balance, and during unfavourable balance
Between not less than tolerance parameter, normally update the data；If the final updating time of current time and small table is time unfavourable balance, and unfavourable balance
Time exceedes tolerance parameter, then refusal is updated the data.
A kind of processing method of versions of data, including：Control node creates big table and notifies that memory node creates small table；Deposit
Storage node compares the current time of itself with the creation time of big table in control node, if time favorable balance, creates small table, if
Time unfavourable balance, and the unfavourable balance time, not less than tolerance parameter, the mode of wait eliminates unfavourable balance, creates small table；If time unfavourable balance, and it is inverse
The difference time exceedes tolerance parameter, and refusal creates small table.
A kind of processing system of versions of data, including：Data recovery module and contrast module；
Data recovery module, for reading redolog files, obtains data time and the data text in redolog files
Part timestamp, is sent to contrast module；If data file timestamp is time favorable balance with the data time in redolog files,
Or, data file timestamp is time unfavourable balance with the data time in redolog files, and the unfavourable balance time is not less than tolerance ginseng
Number, recovers data；If data file timestamp is time unfavourable balance with the data time in redolog files, and the unfavourable balance time is super
Tolerance parameter is crossed, refusal recovers data；
Contrast module, for comparing the data time in data file timestamp and redolog files, determines that the time is suitable
Difference, notifies data recovery module；Time unfavourable balance is determined, further by the unfavourable balance time compared with parameter is tolerated, if exceed tolerance joining
Number, notifies data recovery module；Not less than tolerance parameter, data recovery module is notified.
The system is further included：Update notification module and metadata updates module；
Update notification module, is stopped by hosting node, for assigning hosting node to carry out result, and notifies metadata
Update module；
Metadata updates module, for reading metadata table in, with it is described by the corresponding small table of hosting node title most
After update the time, be sent to contrast module；If current time is time unfavourable balance with the final updating time of small table, refuse more Singapore dollar
Tables of data；If current time is time favorable balance with the final updating time of small table, metadata table is updated；
Accordingly, contrast module, for smaller table the final updating time with itself where hosting node it is current when
Between, it is time favorable balance if current time is later than the final updating time of small table, notify metadata updates module；Otherwise, when being
Between unfavourable balance, notify metadata updates module.
A kind of processing system of versions of data, including：Data file load-on module and contrast module；
Data file load-on module, for reading the corresponding data file of memory node where itself, obtains data file
Timestamp, is sent to contrast module；If data file timestamp is time favorable balance with current time, data file is loaded；If number
It is time unfavourable balance according to document time stamp and current time, and the unfavourable balance time, not less than tolerance parameter, the mode of wait eliminates unfavourable balance,
Loading data file；If data file timestamp is time unfavourable balance with current time, and the unfavourable balance time is more than tolerance parameter, refusal
Loading data file；
Contrast module, for by current time and data file timestamp comparation, however, it is determined that current time and data file
Timestamp is time favorable balance, notifies data file load-on module；Otherwise, it is time unfavourable balance, if further determining that, the unfavourable balance time surpasses
Tolerance parameter is crossed, data file load-on module is notified；If it is determined that the unfavourable balance time, not less than tolerance parameter, notifies data file loading
Module.
A kind of processing system of versions of data, including：Data update notification module, data update module and contrast mould
Block；
Data update notification module, for determining the memory node corresponding to the data to be updated, notifies the storage section
The data update module of point；
Data update module, the final updating time for obtaining small table belonging to the data to be updated, is sent to contrast mould
Block；If the final updating time of current time and small table is time favorable balance, or time unfavourable balance and unfavourable balance time are not less than tolerance
Parameter, normally updates the data；If time unfavourable balance and unfavourable balance time exceed tolerance parameter, refusal is updated the data；
Contrast module, for the current time of memory node where itself to be compared with the final updating time of small table, if
It is time favorable balance, notifies data update module；If time unfavourable balance, and the unfavourable balance time is more than tolerance parameter, notifies that data update
Module；If time unfavourable balance, and the unfavourable balance time is not less than tolerance parameter, notifies data update module.
A kind of processing system of versions of data, including：Big table creation module, small table creation module and contrast module；
Big table creation module, for creating big table, big table notifies small table creation module after creating；
Small table creation module, for reading the big table creation time in big table creation module, is sent to contrast module；If working as
The preceding time is time favorable balance with big table creation time, normally creates small table；If time unfavourable balance, and the unfavourable balance time is more than tolerance ginseng
Number, refusal creates small table；If time unfavourable balance, and the unfavourable balance time, not less than tolerance parameter, the mode of wait eliminates unfavourable balance, creates small
Table；
Contrast module, for the current time of memory node where itself to be compared with big table creation time, if the time is suitable
Difference, notifies small table creation module；If time unfavourable balance, and the unfavourable balance time is more than tolerance parameter, notifies small table creation module；If the time
Unfavourable balance, and the unfavourable balance time is not less than tolerance parameter, notifies small table creation module.
As can be seen here, using method and system of the present invention, when recovering the data in redolog files, in data
Data time is time favorable balance in document time stamp and redolog files, or time unfavourable balance and unfavourable balance time are not above holding
When bearing parameter, just the data in redolog files are recovered, while guaranteeing data integrity, to time unfavourable balance, and
The data in redolog files when the unfavourable balance time exceedes tolerance parameter are not recovered, it is to avoid system recovers a large amount of repeat numbers
According to；Meanwhile, when data are taken over, the final updating time of the data to wanting hosting node, that is, small table final updating when
Between compared with current time, time favorable balance takes over, after improve adapter, the correctness of query metadata version so that
Ensure that systemic-function is normal；During loading data file, by current time and data file timestamp comparation, to time favorable balance or when
Between unfavourable balance and unfavourable balance time loaded not less than the data file of tolerance parameter, improve the accuracy of loading data file；Number
During according to updating, current time is compared with the final updating time of small table, time favorable balance or time unfavourable balance and unfavourable balance time not
During more than tolerance parameter, just update the data, improve the correctness of versions of data after updating；When building table, current time with it is big
The creation time of table is time favorable balance, or, when time unfavourable balance and unfavourable balance time are not less than tolerance parameter, small table is just set up, really
The correct of small table setup time is protected, is easy to subsequent user that the correctness of data query is carried out by big table.
Brief description of the drawings
Fig. 1 realizes the method flow schematic diagram of data recovery for the embodiment of the present invention；
Fig. 2 realizes the method flow schematic diagram of data file loading for the embodiment of the present invention；
Fig. 3 realizes the method flow schematic diagram that data update for the embodiment of the present invention；
Fig. 4 is the method flow schematic diagram that the embodiment of the present invention realizes building table；
Fig. 5 realizes the system composition schematic diagram of data recovery for the embodiment of the present invention；
Fig. 6 realizes the system composition schematic diagram of data file loading for the embodiment of the present invention；
Fig. 7 realizes the system composition schematic diagram that data update for the embodiment of the present invention；
Fig. 8 is the system composition schematic diagram that the embodiment of the present invention realizes building table.
Specific embodiment
Basic thought of the invention is：Start to recover data, read weight Operation Log (redolog) file, obtain
Data time and data file timestamp in redolog files；Compare the number in data file timestamp and redolog files
According to the time, if both are time unfavourable balance, and the unfavourable balance time is not less than tolerance parameter, or, both are time favorable balance, recover number
According to.
Below by specific embodiment and accompanying drawing, the present invention is described in detail.
A kind of processing method of versions of data, as shown in figure 1, comprising the following steps that：
Step 101, the data recovery that starts, reading redolog files obtain the data time and number in redolog files
According to document time stamp；The redolog files are saved in memory space by memory node, including：The number of distributed memory system
According to, versions of data, that is, time during to data manipulation is data time；Distributed memory system is first stored data into
Redolog files, and record version, in then storing data into data file, and record version；
Distributed memory system exception, memory node is restarted or carried out during failover, and system needs to recover data,
At this moment, the record in data file is probably incomplete, and memory node will load the data in redolog files, for extensive
Complex data.Memory node reads redolog files, obtains the versions of data in redolog files, chooses nearest with current time
Versions of data, that is, data time.
Memory node reads data file in data file according to the path of itself corresponding data file in metadata table
Creation time, that is, data file timestamp.The metadata table includes：Memory node title and the memory node
The scope of the data in the corresponding small table data area of title, that is, the data file that preserves of memory node belonging to small table,
When data file path, the final updating time of small table, the corresponding big table identification number (ID) of small table, the establishment of big table
Between.Wherein, the final updating time of the small table is that the data that the memory node as belonging to small table is preserved carry out final updating, or
Person, small table self information carries out the time of final updating；The small table self information includes memory node, the small table described in small table
Data area etc.；All of memory node and control node one metadata table of correspondence；Memory node is stored data into be deposited
In the data file in storage space, and by metadata table recording data files path；The data file is created by memory node,
Creation time, that is, data file timestamp, data and versions of data are wherein included, the data in data file are according to big
Small order is preserved, the memory space of setting data file, newly-built if the data in data file exceed the memory space of setting
Data file preserves data.The small table is in internal memory, including memory node title is corresponding small with the memory node title
During the final updating of the corresponding big table id of table data area, data file path, small table, the creation time of big table and small table
Between.The big epitope for being opened to user, is easy to user to inquire about, comprising creation time, self ID in certain node；The ID
Generated by the memory node for creating big table.Wherein, when big table is selected to create by user, memory node is created；After creating big table,
Memory node can create small table, and set the data area of small table, and big table id is input into small table, determine small table with big table
Corresponding relation, small table at this moment is the small table of blank；The corresponding big table id of the data area of small table and small table is updated simultaneously is arrived
In metadata table.
Step 102, the data time compared in data file timestamp and redolog files, if between the two for when
Between unfavourable balance, and the unfavourable balance time exceed tolerance parameter, then the versions of data in redolog files is likely to occur mistake, performs step
103；If the unfavourable balance time between the two, it was useful to have partial data in redolog files not less than tolerance parameter, it should
Data are recovered, step 104 is performed；If being between the two time favorable balance, data in data file and redolog files
Preservation order is normal, and data corresponding with the data file are to need to recover in redolog files, perform step
104；
Data time phase nearest apart from current time in data time by data file timestamp with redolog files
Than being between the two time unfavourable balance if data file timestamp is earlier than the data time of redolog files；Otherwise, both it
Between be time unfavourable balance, time interval be the unfavourable balance time.The big I of the tolerance parameter is arranged as required to.
Step 103, refusal recover the data in redolog files；
Step 104, recovery data；
The data of recovering are the data read in redolog files, are loaded into internal memory.
The flow for recovering data in redolog files above can be triggered by various situations, and failover is one of them,
Particular content includes：Memory node is stopped, and memory node here can be referred to as by hosting node, and control node is specified
Another memory node, i.e. hosting node are taken over by hosting node, that is, will be sent to adapter by the nodename of hosting node
Node.Hosting node is according to corresponding by hosting node title with described in the nodename reading metadata table by hosting node
The final updating time of small table, compared with the current time of itself, if the current time of hosting node earlier than small table it is last more
The new time, then the current time of hosting node and the final updating time of small table are time unfavourable balance, are not inconsistent with logic, if now
Take over, still being the newest time by the final updating time of hosting node in metadata table, that is, as small table most
After update the time, now, to inquire about the corresponding memory node of small table because the time is more early than the final updating time of small table, because
This, can only be in query metadata table, and the record before the final updating time of small table, the memory node belonging to small table is still to be connect
Pipe node, inquiry error influences systemic-function；Hosting node refusal adapter；If the current time of hosting node is later than and is taken over
The final updating time of small table in node, then both are time favorable balance, and normal adapter updates metadata table.The normal adapter,
Updating metadata table includes：The nodename of hosting node will be revised as in metadata table by the nodename of hosting node.Connect
After pipe, redolog file data recoveries are carried out.
In addition, versions of data is also related to during data file is loaded, various situations in distributed memory system
Capital trigger data file loading, including memory node is restarted, failover and load balancing etc., wherein, memory node weight
Qi Shi, advanced row data file loading, then carry out data recovery, that is, flow described in Fig. 1；When being loaded for data, logarithm
According to the processing procedure of version as shown in Fig. 2 comprising the following steps that：
Step 201, memory node read itself corresponding data file, data file timestamp are obtained, by current time
With data file timestamp comparation, if time favorable balance, step 202 is performed；If time unfavourable balance, and the unfavourable balance time is more than tolerance ginseng
Number, performs step 203；If the unfavourable balance time, not less than tolerance parameter, performs step 204；
Memory node reads the corresponding data file of memory node title described in metadata table according to own node title
Path, data file is read according to data file path, obtains the data file timestamp in data file, the data that will be obtained
Document time stamp compares with the memory node current time, if current time is later than data file timestamp, time favorable balance；It is no
Then, time unfavourable balance, time interval between the two is the unfavourable balance time.
Step 202, loading data file.
It is described loading data file be：Memory node is read in metadata table, the corresponding small table of itself memory node, is read
Final updating time and data file path in small table, according to the data area of small table, by the final updating time of small table
And data file path, it is added in internal memory in small table corresponding with the data area.
Step 203, refusal loading data file, and by alarm notification user.
Step 204, the mode for waiting eliminate unfavourable balance, load data file.
The mode of the wait eliminates unfavourable balance, and loading data file includes：Memory node is waited and tolerance parameter identical
Time, when node current time to be stored is identical with data file timestamp, load data file.Here, data file is loaded
During time of occurrence unfavourable balance, it is unreasonable in logic, but, because data file is possible to be created by different memory node,
The nonsynchronous possibility of existence time between memory node, if all refusing loading to the data file of all time of occurrence unfavourable balances,
Reduce system availability, although the process that the mode of wait eliminates unfavourable balance reduces the response speed of system, but improve and be
The availability of system.
In distributed memory system, when being updated to the data in data file, versions of data is also related to, it is multiple
Process updates comprising data, such as load balancing, wherein, during load balancing, advanced row data update, then carry out data
Recover；The data update includes that data increase, delete and modification etc., during data renewal, to the handling process of versions of data such as
Shown in Fig. 3, idiographic flow is as follows：
Memory node corresponding to step 301, the determination data to be updated；
Control node receives user's data to be updated, according to the data area of data query metadata table, that is, unit
The data area of small table in tables of data, it is determined that the corresponding small table of the data to be updated, the memory node according to belonging to small table, it is determined that
The corresponding memory node of the data to be updated.Control node notifies that the memory node carries out data renewal, is wrapped in the notice
Containing the data to be updated, i.e., the data that add or delete, or the data and amended data to be changed.
Step 302, memory node obtain the final updating time of the small table belonging to the data to be updated, by memory node
Current time compares with the final updating time of small table, if time favorable balance, performs step 303；If time unfavourable balance, and unfavourable balance time
More than tolerance parameter, step 304 is performed；If the unfavourable balance time, not less than tolerance parameter, performs step 303；
Memory node is read in metadata table, the final updating time of the corresponding small table of the data to be updated, and storage is saved
Point current time compared with the final updating time of small table, if the current time of memory node is later than the final updating of small table
Between, then it is time favorable balance；Otherwise, it is time unfavourable balance, time interval between the two is the unfavourable balance time.
Step 303, normally update the data.
The renewal includes：During data storage, data file path of the memory node in the corresponding small table of data is read
Data file is taken, data order is saved in data file, and record the time, that is, the version after updating is the data file
Latest edition；When modification or deletion data, according to the data file path in the corresponding small table of data, data file is read, looked into
Data corresponding with the data that change or delete in data file are ask, the data are modified or deleted.
Step 304, refusal are updated the data, and by alarm notification user.
Distributed memory system also relates to versions of data problem during table is built, when table is built, to versions of data
Handling process, as shown in figure 4, comprising the following steps that：
Step 401, control node create big table and notify that memory node creates small table；
Control node indicates to create big table according to user, and the big table of establishment includes the big table id of generation, and records big table wound
Build the time；The big table id Jia 1 by control node in existing big table id, such as existing big table id is 1, then the new big table for creating
ID is 2, and the initial value of big table id can set as needed；The notice memory node creates small table to be included：Control node according to
The load balancing of memory node, small table is created by the minimum memory node of load.
Step 402, memory node by the current time of itself compared with the creation time of big table in control node, if the time
Favorable balance, performs step 403；If time unfavourable balance and unfavourable balance time, not less than tolerance parameter, perform step 404；If time unfavourable balance, and
The unfavourable balance time exceedes tolerance parameter, performs step 405；According to normally surface low journey is built, control node first sets up big table, then by depositing
Storage node sets up small table, if the setup time of big table is later than small table, when carrying out data renewal subsequently through small table, what data updated
Time is possible to the setup time earlier than big table, is in logic irrational, it is also possible to cause the functional fault of system.
Step 403, memory node create small table；
The small table of establishment includes：Memory node sets up small table, sets the data area of small table, and by small table and big table id
Correspondence.
Step 404, the mode for waiting eliminate unfavourable balance, create small table.
Memory node is waited with after the tolerance parameter identical time, and the current time of memory node is later than the wound of big table
Build the time, memory node normally creates small table.
Step 405, memory node refusal create small table, and by alarm notification user.
Present invention also offers a kind of processing system of versions of data, as shown in figure 5, the system includes：Data recovery mould
Block 501 and contrast module 502, wherein data recovery module 501 are located at memory node with contrast module 502.
Data recovery module 501, for reading redolog files；Obtain the data time and data in redolog files
Document time stamp, is sent to contrast module 502；If data file timestamp is the time with the data time in redolog files
Unfavourable balance, and the unfavourable balance time, more than tolerance parameter, refusal recovers data；If the unfavourable balance time, not less than tolerance parameter, recovers data；If
Data file timestamp is time favorable balance with the data time in redolog files, recovers data；
Contrast module 502, for comparing the data time in data file timestamp and redolog files, determines the time
Favorable balance, notifies data recovery module 501；Determine time unfavourable balance, further by time interval between the two, i.e., the unfavourable balance time with
Tolerance parameter is compared, and the unfavourable balance time, not less than tolerance parameter, notifies data recovery module 501；The unfavourable balance time exceedes tolerance parameter,
Notify data recovery module 501.
The system is further included：Update notification module 503, positioned at control node；Metadata updates module 504, is located at
Memory node；
Update notification module 503, is stopped by hosting node, for specifying hosting node to take over, and notifies unit
Data update module 504；Included by hosting node title in the notice；
Metadata updates module 504, for reading in metadata table with described by the corresponding small table of hosting node title
The final updating time, it is sent to contrast module 502；If current time is time unfavourable balance, refusal with the final updating time of small table
Update metadata table；If current time is time favorable balance with the final updating time of small table, metadata table is normally updated；
Accordingly, contrast module 502, the final updating time for smaller table is current with memory node where itself
Time, is between the two time unfavourable balance if the current time of hosting node is earlier than the final updating time of small table, notifies first number
According to update module 504；Otherwise, it is time favorable balance, notifies metadata updates module 504.
A kind of processing system of versions of data, as shown in fig. 6, the system includes：Data file load-on module 601 and right
Than module 602；Wherein, data file load-on module 601 is located at memory node with contrast module 602；
Data file load-on module 601, for reading the corresponding data file of memory node where itself, obtains data text
Part timestamp, is sent to contrast module 602；If data file timestamp is time favorable balance with current time, data file is loaded；
If data file timestamp is time unfavourable balance with current time, and the unfavourable balance time is more than tolerance parameter, refusal loading data file；
If data file timestamp is time unfavourable balance with current time, and the unfavourable balance time, not less than tolerance parameter, the mode of wait is eliminated
Unfavourable balance, loads data file；
Contrast module 602, for by current time compared with data file timestamp, when current time is later than data file
Between stab, it is determined that current time and data file timestamp time favorable balance, notify data file load-on module 601；Otherwise, both
Time unfavourable balance, determines that the unfavourable balance time exceedes tolerance parameter, notifies data file load-on module 601；Determine the unfavourable balance time not less than
Tolerance parameter, notifies data file load-on module 601；The determination current time and data file timestamp time favorable balance bag
Include：If current time is later than data file timestamp, time favorable balance, otherwise, time unfavourable balance；The determination unfavourable balance time exceedes
Tolerance parameter includes：The time interval of current time and data file timestamp is obtained, as unfavourable balance time and tolerance parameter phase
Than determining that the unfavourable balance time exceedes tolerance parameter.
A kind of processing system of versions of data, as shown in fig. 7, the system includes：Data update notification module 701, data
Update module 702 and contrast module 703；Wherein, data update notification module 701 is located at control node, data update module
702 and contrast module 703 be located at memory node；
Data update notification module 701, for receiving user's data to be updated, it is determined that corresponding to the data to be updated
Memory node, notifies the data update module 702 of the memory node；Storage section corresponding to the determination data to be updated
Point includes：According to the data area of data query metadata table, that is, in metadata table small table data area, it is determined that will be more
The corresponding small table of new data, the memory node belonging to small table, the corresponding memory node of the data as to be updated；The notice
In include the data to be updated, i.e., data that add or delete, or the data and amended data to be changed；
Data update module 702, the final updating time for obtaining the small table belonging to the data to be updated, it is right to be sent to
Than module 703；If the final updating time of current time and small table is time favorable balance, or current time and small table it is last more
The new time is for time unfavourable balance and the unfavourable balance time is not less than tolerance parameter, normally updates the data；If time unfavourable balance and unfavourable balance time are super
Tolerance parameter is crossed, refusal is updated the data；
Contrast module 703, for the current time of memory node where itself to be compared with the final updating time of small table,
It is determined that then time favorable balance, notifies data update module 702；Otherwise, time unfavourable balance, determines that the unfavourable balance time exceedes tolerance parameter, leads to
Primary data update module 702；Determine time unfavourable balance and the unfavourable balance time is not less than tolerance parameter, notify data update module 702；Institute
Stating determination time favorable balance includes：Current time is later than the final updating time of small table, it is determined that time favorable balance；The determination unfavourable balance
Time includes more than tolerance parameter：By the time interval of current time and the final updating time of small table, as the unfavourable balance time, with
Tolerance parameter is compared, if the unfavourable balance time is more than tolerance parameter, it is determined that the unfavourable balance time exceedes tolerance parameter, otherwise, unfavourable balance time
Not less than tolerance parameter.
A kind of processing system of versions of data, as shown in figure 8, the system includes：Big table creation module 801, small table is created
Module 802 and contrast module 803；Wherein, big table creation module 801 is located at control node, small table creation module 802 and contrast
Module 803 is located at memory node；
Big table creation module 801, for creating big table according to the order of user, and notifies memory node after the establishment of big table
Small table creation module 802 set up small table；The big table of establishment includes：Big table id is generated, and records big table creation time；
Small table creation module 802, for reading the big table creation time in big table creation module 801, is sent to contrast mould
Block 803；If time favorable balance, small table is normally created；If time unfavourable balance and unfavourable balance time exceed tolerance parameter, refusal creates small table；
If time unfavourable balance and unfavourable balance time eliminate unfavourable balance, normally create small table not less than tolerance parameter, the mode of wait；
Contrast module 803, for the current time of memory node where itself to be compared with big table creation time, when determining
Between favorable balance, notify small table creation module 802；Determine time unfavourable balance, and the unfavourable balance time exceedes tolerance parameter, notifies that small table creates mould
Block 802；Determine that time unfavourable balance, and unfavourable balance time, not less than tolerance parameter, notify small table creation module 802；The determination time
Favorable balance includes：If current time is later than big table creation time, time favorable balance；The determination time unfavourable balance, and the unfavourable balance time is super
Crossing tolerance parameter includes：If current time is earlier than big table creation time, time unfavourable balance, by current time and big table creation time
Time interval as the unfavourable balance time, compared with tolerance parameter, it may be determined that the unfavourable balance time exceedes tolerance parameter, or unfavourable balance time
Not less than tolerance parameter.
The above, only presently preferred embodiments of the present invention is not intended to limit the scope of the present invention.
Claims (10)
1. a kind of processing method of versions of data, it is characterised in that be applied to distributed memory system, the method includes：
Start to recover data, read weight Operation Log redolog files, obtain data time and data in redolog files
Document time stamp；If the data time in data file timestamp and redolog files is time unfavourable balance, and the unfavourable balance time does not surpass
Tolerance parameter is crossed, or, data file timestamp is time favorable balance with the data time in redolog files, recovers redolog
Data in file；If data file timestamp is time unfavourable balance with the data time in redolog files, and the unfavourable balance time is super
Tolerance parameter is crossed, refusal recovers data.
2. method according to claim 1, it is characterised in that it is described start to recover data before, the method is further wrapped
Include：
It is stopped by hosting node, control node appointment hosting node carries out failover, and hosting node is saved according to by adapter
The nodename of point, when obtaining the final updating of small table corresponding with the nodename by hosting node in metadata table
Between, compared with the current time of itself, if current time is time favorable balance with the final updating time of small table, normal adapter；
Otherwise, refusal adapter.
3. a kind of processing method of versions of data, it is characterised in that the method includes：
Memory node reads the data file of itself, obtains data file timestamp, compares with current time, if current time with
Data file timestamp is time favorable balance, then load data file；If current time is time unfavourable balance with data file timestamp,
And the unfavourable balance time exceedes tolerance parameter, then refusal loads data file；If current time is that the time is inverse with data file timestamp
Difference, and the unfavourable balance time is not less than tolerance parameter, and unfavourable balance is eliminated by the way of waiting, and loads data file.
4. a kind of processing method of versions of data, it is characterised in that the method includes：
Control node determines the corresponding memory node of the data to be updated, and memory node obtains small table belonging to the data to be updated
The final updating time, compare with current time, if current time is time favorable balance with the final updating time of small table, or, when
The preceding time is time unfavourable balance with the final updating time of small table, and the unfavourable balance time is not less than tolerance parameter, normally updates the data；If
Current time is time unfavourable balance with the final updating time of small table, and the unfavourable balance time is more than tolerance parameter, then refuse to update the data.
5. a kind of processing method of versions of data, it is characterised in that the method includes：
Control node creates big table and notifies that memory node creates small table；Memory node is by itself current time and control node
In the creation time of big table compare, if time favorable balance, create small table, if time unfavourable balance, and the unfavourable balance time join not less than tolerance
Number, eliminates unfavourable balance by the way of waiting, and creates small table；If time unfavourable balance, and the unfavourable balance time, more than tolerance parameter, refusal is created
Small table.
6. a kind of processing system of versions of data, it is characterised in that be applied to distributed memory system, the system includes：Data
Recovery module and contrast module；
Data recovery module, for reading redolog files, when obtaining the data time and data file in redolog files
Between stab, be sent to contrast module；If data file timestamp is time favorable balance with the data time in redolog files, or,
Data file timestamp is time unfavourable balance with the data time in redolog files, and the unfavourable balance time is not less than tolerance parameter, extensive
Complex data；If data file timestamp is time unfavourable balance with the data time in redolog files, and the unfavourable balance time is more than tolerance
Parameter, refusal recovers data；
Contrast module, for comparing the data time in data file timestamp and redolog files, determines time favorable balance, leads to
Primary data recovery module；Determine time unfavourable balance, further by the unfavourable balance time compared with parameter is tolerated, if exceeding tolerance parameter, lead to
Primary data recovery module；Not less than tolerance parameter, data recovery module is notified.
7. system according to claim 6, it is characterised in that the system is further included：Update notification module and unit
Data update module；
Update notification module, is stopped by hosting node, for assigning hosting node to carry out result, and notifies metadata updates
Module；
Metadata updates module, for reading metadata table in, with it is described by the corresponding small table of hosting node title it is last more
The new time, it is sent to contrast module；If current time is time unfavourable balance with the final updating time of small table, refuse more new metadata
Table；If current time is time favorable balance with the final updating time of small table, metadata table is updated；
Accordingly, the current time of contrast module, the final updating time for smaller table and hosting node where itself, if
Current time is later than the final updating time of small table, then be time favorable balance, notifies metadata updates module；Otherwise, it is the inverse time
Difference, notifies metadata updates module.
8. a kind of processing system of versions of data, it is characterised in that the system includes：Data file load-on module and contrast mould
Block；
Data file load-on module, for reading the corresponding data file of memory node where itself, obtains the data file time
Stamp, is sent to contrast module；If data file timestamp is time favorable balance with current time, data file is loaded；If data are literary
Part timestamp is time unfavourable balance with current time, and the unfavourable balance time is not less than tolerance parameter, and unfavourable balance is eliminated by the way of waiting,
Loading data file；If data file timestamp is time unfavourable balance with current time, and the unfavourable balance time is more than tolerance parameter, refusal
Loading data file；
Contrast module, for by current time and data file timestamp comparation, however, it is determined that current time and data file time
It is time favorable balance to stab, and notifies data file load-on module；Otherwise, it is time unfavourable balance, if further determine that the unfavourable balance time exceedes holding
Bear parameter, notify data file load-on module；If it is determined that the unfavourable balance time not less than tolerance parameter, notifies data file loading mould
Block.
9. a kind of processing system of versions of data, it is characterised in that the system includes：Data update notification module, data update
Module and contrast module；
Data update notification module, for determining the memory node corresponding to the data to be updated, notifies the memory node
Data update module；
Data update module, the final updating time for obtaining small table belonging to the data to be updated, is sent to contrast module；If
The final updating time of current time and small table is time favorable balance, or time unfavourable balance and unfavourable balance time are not less than tolerance parameter,
Normally update the data；If time unfavourable balance and unfavourable balance time exceed tolerance parameter, refusal is updated the data；
Contrast module, for the current time of memory node where itself to be compared with the final updating time of small table, if when
Between favorable balance, notify data update module；If time unfavourable balance, and the unfavourable balance time is more than tolerance parameter, notifies data update module；
If time unfavourable balance, and the unfavourable balance time is not less than tolerance parameter, notifies data update module.
10. a kind of processing system of versions of data, it is characterised in that the system includes：Big table creation module, small table creation module
And contrast module；
Big table creation module, for creating big table, big table notifies small table creation module after creating；
Small table creation module, for reading the big table creation time in big table creation module, is sent to contrast module；If when current
Between with big table creation time be time favorable balance, normally create small table；If time unfavourable balance, and the unfavourable balance time is more than tolerance parameter, refuses
Small table is created absolutely；If time unfavourable balance, and the unfavourable balance time is not less than tolerance parameter, and unfavourable balance is eliminated by the way of waiting, and creates small
Table；
Contrast module, for the current time of memory node where itself to be compared with big table creation time, if time favorable balance, leads to
Know small table creation module；If time unfavourable balance, and the unfavourable balance time is more than tolerance parameter, notifies small table creation module；If time unfavourable balance,
And the unfavourable balance time is not less than tolerance parameter, small table creation module is notified.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110203404.7A CN102890679B (en) | 2011-07-20 | 2011-07-20 | The processing method and system of a kind of versions of data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110203404.7A CN102890679B (en) | 2011-07-20 | 2011-07-20 | The processing method and system of a kind of versions of data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102890679A CN102890679A (en) | 2013-01-23 |
| CN102890679B true CN102890679B (en) | 2017-05-31 |
Family
ID=47534186
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201110203404.7A Active CN102890679B (en) | 2011-07-20 | 2011-07-20 | The processing method and system of a kind of versions of data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102890679B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106708825B (en) * | 2015-07-29 | 2019-09-27 | 腾讯科技（深圳）有限公司 | A kind of data file processing method and system |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5261069A (en) * | 1990-08-13 | 1993-11-09 | Hewlett-Packard Company | Method of maintaining consistency of cached data in a database system |
| CN1483164A (en) * | 2000-12-21 | 2004-03-17 | ���ܿ���ϵͳ����˾ | Restoring large capacity memory to previous state responding to processing information |
-
2011
- 2011-07-20 CN CN201110203404.7A patent/CN102890679B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5261069A (en) * | 1990-08-13 | 1993-11-09 | Hewlett-Packard Company | Method of maintaining consistency of cached data in a database system |
| CN1483164A (en) * | 2000-12-21 | 2004-03-17 | ���ܿ���ϵͳ����˾ | Restoring large capacity memory to previous state responding to processing information |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102890679A (en) | 2013-01-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110532247B (en) | Data migration method and data migration system | |
| KR102437664B1 (en) | System and method for transaction recovery in a multitenant application server environment | |
| US20200175035A1 (en) | System and method for maintaining a master replica for reads and writes in a data store | |
| US9201742B2 (en) | Method and system of self-managing nodes of a distributed database cluster with a consensus algorithm | |
| US7702698B1 (en) | Database replication across different database platforms | |
| US7464097B2 (en) | Managing data integrity using a filter condition | |
| CN107391758B (en) | Database switching method, device and equipment | |
| EP1594071A2 (en) | Method and system for synchronizing data between electronic devices | |
| US10248709B2 (en) | Promoted properties in relational structured data | |
| US20040034668A1 (en) | Managing data integrity | |
| EP3391244B1 (en) | Replication control among redundant data centers | |
| CN108845867A (en) | A kind of distributed transaction management method, apparatus, system and storage medium | |
| KR102038529B1 (en) | System for processing real-time data modification of in-memory database | |
| US20170169091A1 (en) | Replication of structured data records among partitioned data storage spaces | |
| US11003550B2 (en) | Methods and systems of operating a database management system DBMS in a strong consistency mode | |
| US11537570B2 (en) | Systems and/or methods for migrating live database schemas to support zero downtime deployments with zero data losses | |
| CN108431808A (en) | Prompting processing to the structured data in the data space of subregion | |
| JP6196389B2 (en) | Distributed disaster recovery file synchronization server system | |
| CN108153804A (en) | A kind of metadata daily record update method of symmetric distributed file system | |
| CN106571940A (en) | Method and device of fusing network management data and resource data | |
| US10664361B1 (en) | Transactionally consistent backup of partitioned storage | |
| CN105550342B (en) | A kind of data processing method of the distributed data base of all-transparent | |
| CN102890679B (en) | The processing method and system of a kind of versions of data | |
| CN105956046A (en) | Dictionary update method and system, and server | |
| CN107180034A (en) | The group system of MySQL database |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |