CN105260261A - Email recovery method - Google Patents
Email recovery method Download PDFInfo
- Publication number
- CN105260261A CN105260261A CN201510801232.1A CN201510801232A CN105260261A CN 105260261 A CN105260261 A CN 105260261A CN 201510801232 A CN201510801232 A CN 201510801232A CN 105260261 A CN105260261 A CN 105260261A
- Authority
- CN
- China
- Prior art keywords
- data
- write
- group
- logical
- error checking
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Memory System Of A Hierarchy Structure (AREA)
Abstract
The invention provides an email recovery method. The email recovery method comprises the steps that one of multiple storage areas in a storage array of an email storage system serves as a logic storage area, newly-written data and data existing in the logic storage area form new error checking data, the data existing in the logic storage area are read, and email data recovery is achieved by combining the logic storage area and a source storage space during data recovery. The email recovery method keeps low-energy-consumption parallel access, also solves the local-write problem caused by parallel access and has outstanding write performance and energy-saving efficiency.
Description
Technical field
The present invention relates to email storage, particularly a kind of mail restoration methods.
Background technology
The quick growth of mail data, the energy consumption of mail storage systems is sharply increased, store in application at mail data, meeting under performance requirement and the fault-tolerant condition of single-deck, the concurrent access data ordering of storage array can cause storage array substantially to perform read-write operation: when writing new data, need first to read corresponding legacy data, old error check data, write new error check data again generate new error check data together with new data after, had a strong impact on performance.In order to provide specified write performance, storage array must run more disks to make up the performance loss locally writing and bring, thus can consume more multi-energy.Therefore, the energy-saving efficiency of storage array is in urgent need to be improved.
Summary of the invention
For solving the problem existing for above-mentioned prior art, the present invention proposes a kind of mail restoration methods, comprising:
Using in the multiple memory blocks in the storage array of mail storage systems as logical storage area, the data newly write are generated new error check data together with logical storage area data with existing, read described logical storage area data with existing, realize mail data when date restoring in conjunction with logical storage area and source storage space and recover.
Preferably, described mail storage systems is made up of N block disk, each disk is divided into N+1 memory block, the memory block of N number of same offset forms a storehouse, form N+1 storehouse altogether, select one of them as logical base, all the other are physical library, and each physical library comprises 1 error checking district, a N-1 data field; In physical library i, error checking district is designated as PBi, is positioned at disk N-1-i; V data field is designated as DB (i, v), and as i+v<N-1, DB (i, v) is positioned at disk v, otherwise is positioned at disk v+1; Wherein, 0≤i<N, 0≤v<N-1; The value of PBi is tried to achieve by by following computing:
Each memory block comprises M equal-sized piece, in each storehouse, the integrated stick of block of same offset; Described mail storage systems adopts following concurrent access data to arrange: the data field of the N-1 in each physical library is divided into P group, and often group comprises Q block; The block that often in group, side-play amount is identical can by concurrent access, and in each stick, only partial block provides concurrency;
Described mail storage systems only divides into groups to physical library, and logical base does not participate in grouping, and also not participating in addressing, is transparent to the upper layer application of mail storage systems; Divide in group address and mix, mail storage systems adopts following strategy, and in each physical library, the logical address of the group that sequence number is adjacent is adjacent; Namely the logical address organizing m block in the memory block q of p in the i of storehouse is: NumB (MQPi+MQp+Qm+q); Wherein NumB is the data block number that block comprises, 0≤p<P, 0≤i<N, 0≤q<Q, 0≤m<M;
Preferably: for the incremental error checking data of data genaration newly write in circulation at every turn increases an auxiliary cache, incremental error checking data and logical base medial error checking data place disk, generate new error check data with pipeline mode; In each storehouse, arrange 3 groups, auxiliary cache keeps in the incremental error checking data in logical base;
When group 0 to arbitrary source thesaurus writes data, the group 0 of data write logical base, and the error checking of generation group 0, the error checking district of write logical base; Group 0 is write completely, when the group 1 to source thesaurus writes data, and the group 1 of data write logical base, and according to writing data, incremental error verification, generate new error check data, write auxiliary cache; Group 1 is write completely, when the group 2 to source thesaurus writes data, and the group 2 of data write logical base, and according to writing data, incremental error verification, generate new error check data, the error checking district of write logical base; Logical base is write completely, amendment mapping table, make it replace source thesaurus, and source thesaurus is as the logical base in next circulation; If the group number of logical base is odd number, then first write error check data to error checking district; Otherwise, first write error check data to auxiliary cache; When adopting solid state hard disc as auxiliary cache, read incremental error checking data from auxiliary cache, new error check data also writes auxiliary cache, until when writing last group data, incremental error checking data is read from auxiliary cache, and by new error check data write disk.
The present invention compared to existing technology, has the following advantages:
The present invention proposes a kind of mail restoration methods, maintain the low energy consumption of concurrent access, solve again the local write problem that concurrent access brings, there is outstanding write performance and energy-saving efficiency.
Accompanying drawing explanation
Fig. 1 is the process flow diagram of the mail restoration methods according to the embodiment of the present invention.
Embodiment
Detailed description to one or more embodiment of the present invention is hereafter provided together with the accompanying drawing of the diagram principle of the invention.Describe the present invention in conjunction with such embodiment, but the invention is not restricted to any embodiment.Scope of the present invention is only defined by the claims, and the present invention contain many substitute, amendment and equivalent.Set forth many details in the following description to provide thorough understanding of the present invention.These details are provided for exemplary purposes, and also can realize the present invention according to claims without some in these details or all details.
An aspect of of the present present invention provides a kind of mail restoration methods.Fig. 1 is the mail restoration methods process flow diagram according to the embodiment of the present invention.
The present invention adopts the high-effect storage array stored towards mail data, have employed new concurrent access data ordering, and the following strategy of integrated use, realizing high-performance and high energy-saving efficiency.In address maps, random writing is converted into and is sequentially written in; Storage space is divided into multiple equal memory block, one of them is as logical storage area, when upgrading source storage space, and data write logical storage area; Logical storage area is write completely, and amendment mapping table makes its replace source storage space; In next circulation, source storage space is as logical storage area, other memory blocks of buffer memory write data: write data and generate new error check data together with logical storage area data with existing, along with the increase of data in logical storage area, the error checking scope of error check data also expands gradually.Without the need to reading legacy data when generating new error check data, reading logical storage area data with existing when adopting pipeline mode, when writing new error check data, the impact of reader error checking data on performance can be eliminated; Utilize segment data fault-tolerant: associating logical storage area, source storage space realize date restoring.Mail storage systems of the present invention maintains the low energy consumption of concurrent access, solves again the local write problem that concurrent access brings, has outstanding write performance and energy-saving efficiency.
Mail storage systems implementation method mainly comprises data ordering, write operation process, date restoring 3 aspects.Wherein, write operation process comprises again address maps, remote data update, expansion error checking.Mail storage systems is made up of N block disk, and each disk is divided into N+1 memory block.The memory block of N number of same offset forms a storehouse, and form N+1 storehouse altogether, select one of them as logical base, all the other are physical library.Each physical library comprises 1 error checking district, a N-1 data field.In physical library i, error checking district is designated as PBi, is positioned at disk N-1-i; V data field is designated as DB (i, v), and as i+v<N-1, DB (i, v) is positioned at disk v, otherwise is positioned at disk v+1.Wherein, 0≤i<N, 0≤v<N-1.The value of PBi is tried to achieve by by XOR:
Each memory block is made to comprise M equal-sized piece, in each storehouse, the integrated stick of block of same offset.
In order to provide suitable performance, mail storage systems adopts following concurrent access data to arrange: the data field of the N-1 in each physical library is divided into P group, and often group comprises Q block.The block that often in group, side-play amount is identical can by concurrent access, and in each stick, only partial block provides concurrency.
Mail storage systems only divides into groups to physical library, and logical base does not participate in grouping, and also not participating in addressing, is transparent to the upper layer application of mail storage systems.Divide in group address and mix, mail storage systems have employed following strategy: in each physical library, and the logical address of the group that sequence number is adjacent is adjacent.The data block number that if NumB is block to be comprised, then storehouse i, group p, in the q of memory block, the logical address of m block is: NumB (MQPi+MQp+Qm+q);
Here, 0≤p<P, 0≤i<N, 0≤q<Q, 0≤m<M.
The data ordering of mail storage systems and addressing mode can provide enough degree of parallelisms, and application is stored for mail data, can ensure that I/O request concentrates in one or several group in long time, other most disks have sufficiently long stand-by time, schedulable to standby mode to save energy consumption.
The write operation process synthesis of mail storage systems has used the strategy such as address maps, remote data update, expansion error checking, without the need to reading legacy data during generation error check data, reading existing error check data when adopting pipeline mode, when writing new error check data, effectively can solve the local write problem that concurrent access brings.In addition, random writing converts to and is sequentially written in by address maps, improves again the write performance of mail storage systems further.
Mail storage systems, to write new data, lessly carries out rewriting operation, is applicable to adopting block collection to map.Address mapping information is 8/ (1024 × x) of memory capacity, and wherein, 8 byte (64) records block collection address, x is that block collection size is in units of KB.When the memory capacity of mail storage systems be 30TB, block integrate size as 64KB time, address mapping information is only 3.67GB, be applicable to adopt solid state hard disc store, even can call in internal memory completely during operation, with accelerate reading and writing operation in address slewing rate.
Random writing be converted to and be sequentially written in, need in the face of garbage reclamation problem, rubbish storage space is produced by rewriting operation, and in mail data stores, the data volume of rewriting is little, can carry out garbage reclamation in load lower than during predetermined threshold value; If pursuit performance, also can sacrifice a small amount of storage space and ignore garbage reclamation.
Discrete virtual address map is continuous print physical address by address maps, and records mapping relations in the mapping table.Wherein, virtual address is the read-write requests address that application program is submitted to, and physical address is the memory addresss of data in mail storage systems.On this basis, mail storage systems performs Data Update: when writing data to certain physical address, data are not write direct this address, but write its logical address (address identical with its side-play amount in logical base), and revise mapping table at the appropriate time, make logical address replace this physical address.
Suppose that mail storage systems is made up of N block disk, mark off N+1 storehouse, appoint and get one of them as logical base, all the other are physical library, then the remote data update process of mail storage systems is as follows:
(1), when writing data to certain source thesaurus, data are not write direct this storehouse, but write logical base;
(2) according to the error check data having write data in logical base in write data, this circulation, the new error check data in formation logic storehouse;
(3) if logical base is not write full, step (1) is forwarded to;
(4) otherwise, modified address mapping relations, make logical base replace source thesaurus, this end that circulates;
(5) the source thesaurus be substituted, can as logical base in next circulation now without mapping relations.
In above write operation process, owing to having carried out address maps, so be sequentially written in into data to each physical library successively, data can not be write to two physical library simultaneously, also when a physical library is not write full, data can not be write to another one physical library.
The error check data of logical base writes data genaration according in this circulation, is called incremental error checking data.When writing new data, new error check data can be calculated according to new data, incremental error checking data.Along with the increase writing data, the error checking scope also progressive expansion of incremental error checking data, until expand to whole logical base.
Without the need to reading legacy data during expansion error check data, only need to read incremental error checking data, therefore an auxiliary cache can be increased, with logical base medial error checking data place disk, generate new error check data with pipeline mode and (read incremental error checking data for 1, write new error check data for 1), now, the impact of reader error checking data on write performance effectively can be eliminated.
Suppose that each storehouse comprises 3 groups, auxiliary cache keeps in the incremental error checking data in logical base.Concrete implementation is as follows:
(1) when writing data to the group 0 of arbitrary source thesaurus, the group 0 of data write logical base, and the error checking of generation group 0, the error checking district of write logical base;
(2) organize 0 to write completely, when group 1 to source thesaurus writes data, the group 1 of data write logical base, and according to writing data, the incremental error verification (error checking of group 0, error checking district at logical base), generate new error check data (group 0, the error checking of group 1), write auxiliary cache;
(3) organize 1 to write completely, when group 2 to source thesaurus writes data, the group 2 of data write logical base, and according to writing data, incremental error verification (group 0, the error checking of group 1, at auxiliary cache), generate new error check data (group 0, group 1, the error checking of group 2), the error checking district of write logical base;
(4) logical base is write completely, amendment mapping table, make it replace source thesaurus, and source thesaurus is as the logical base in next circulation.
For ensureing the error checking district of the last error check data write logical base generated, need flowing water according to the following rules: if the group number of logical base is odd number, then first write error check data to error checking district; Otherwise, first write error check data to auxiliary cache.
When adopting solid state hard disc as auxiliary cache: read incremental error checking data from auxiliary cache, new error check data also writes auxiliary cache, until when writing last group data, read incremental error checking data from auxiliary cache, and by new error check data write disk.
For making the logical base medial error checking data place disk most of the time also can be standby, performance is not affected again while further raising energy-saving efficiency, can be adopted following pipeline mode: two auxiliary cache auxiliary caches 1 and auxiliary cache 2 are set, incremental error checking data is read in turn from one of them, new error check data is write to another, until the final error checking data in formation logic storehouse, then write disk.
In sum, the present invention proposes a kind of mail immediate processing method, maintain the low energy consumption of concurrent access, solve again the local write problem that concurrent access brings, there is outstanding write performance and energy-saving efficiency.
Obviously, it should be appreciated by those skilled in the art, above-mentioned of the present invention each module or each step can realize with general computing system, they can concentrate on single computing system, or be distributed on network that multiple computing system forms, alternatively, they can realize with the executable program code of computing system, thus, they can be stored and be performed by computing system within the storage system.Like this, the present invention is not restricted to any specific hardware and software combination.
Should be understood that, above-mentioned embodiment of the present invention only for exemplary illustration or explain principle of the present invention, and is not construed as limiting the invention.Therefore, any amendment made when without departing from the spirit and scope of the present invention, equivalent replacement, improvement etc., all should be included within protection scope of the present invention.In addition, claims of the present invention be intended to contain fall into claims scope and border or this scope and border equivalents in whole change and modification.
Claims (3)
1. a mail restoration methods, is characterized in that, comprising:
Using in the multiple memory blocks in the storage array of mail storage systems as logical storage area, the data newly write are generated new error check data together with logical storage area data with existing, read described logical storage area data with existing, realize mail data when date restoring in conjunction with logical storage area and source storage space and recover.
2. method according to claim 1, it is characterized in that, described mail storage systems is made up of N block disk, each disk is divided into N+1 memory block, the memory block of N number of same offset forms a storehouse, forms N+1 storehouse altogether, selects one of them as logical base, all the other are physical library, and each physical library comprises 1 error checking district, a N-1 data field; In physical library i, error checking district is designated as PBi, is positioned at disk N-1-i; V data field is designated as DB (i, v), and as i+v<N-1, DB (i, v) is positioned at disk v, otherwise is positioned at disk v+1; Wherein, 0≤i<N, 0≤v<N-1; The value of PBi is tried to achieve by by following computing:
Each memory block comprises M equal-sized piece, in each storehouse, the integrated stick of block of same offset; Described mail storage systems adopts following concurrent access data to arrange: the data field of the N-1 in each physical library is divided into P group, and often group comprises Q block; The block that often in group, side-play amount is identical can by concurrent access, and in each stick, only partial block provides concurrency;
Described mail storage systems only divides into groups to physical library, and logical base does not participate in grouping, and also not participating in addressing, is transparent to the upper layer application of mail storage systems; Divide in group address and mix, mail storage systems adopts following strategy, and in each physical library, the logical address of the group that sequence number is adjacent is adjacent; Namely the logical address organizing m block in the memory block q of p in the i of storehouse is: NumB (MQPi+MQp+Qm+q); Wherein NumB is the data block number that block comprises, 0≤p<P, 0≤i<N, 0≤q<Q, 0≤m<M;
3. method according to claim 2, it is characterized in that, also comprise: for the incremental error checking data of data genaration newly write in circulation at every turn increases an auxiliary cache, incremental error checking data and logical base medial error checking data place disk, generate new error check data with pipeline mode; In each storehouse, arrange 3 groups, auxiliary cache keeps in the incremental error checking data in logical base;
When group 0 to arbitrary source thesaurus writes data, the group 0 of data write logical base, and the error checking of generation group 0, the error checking district of write logical base; Group 0 is write completely, when the group 1 to source thesaurus writes data, and the group 1 of data write logical base, and according to writing data, incremental error verification, generate new error check data, write auxiliary cache; Group 1 is write completely, when the group 2 to source thesaurus writes data, and the group 2 of data write logical base, and according to writing data, incremental error verification, generate new error check data, the error checking district of write logical base; Logical base is write completely, amendment mapping table, make it replace source thesaurus, and source thesaurus is as the logical base in next circulation; If the group number of logical base is odd number, then first write error check data to error checking district; Otherwise, first write error check data to auxiliary cache; When adopting solid state hard disc as auxiliary cache, read incremental error checking data from auxiliary cache, new error check data also writes auxiliary cache, until when writing last group data, incremental error checking data is read from auxiliary cache, and by new error check data write disk.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510801232.1A CN105260261B (en) | 2015-11-19 | 2015-11-19 | A kind of mail restoration methods |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510801232.1A CN105260261B (en) | 2015-11-19 | 2015-11-19 | A kind of mail restoration methods |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105260261A true CN105260261A (en) | 2016-01-20 |
| CN105260261B CN105260261B (en) | 2018-06-15 |
Family
ID=55099961
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510801232.1A Active CN105260261B (en) | 2015-11-19 | 2015-11-19 | A kind of mail restoration methods |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105260261B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107786419A (en) * | 2016-08-26 | 2018-03-09 | 西安交大捷普网络科技有限公司 | Realize the annex restoring method of webpage mailbox annex |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040128470A1 (en) * | 2002-12-27 | 2004-07-01 | Hetzler Steven Robert | Log-structured write cache for data storage devices and systems |
| US20090070653A1 (en) * | 2007-08-14 | 2009-03-12 | Lg Electronics Inc. | Method of transmitting data |
| CN101752011A (en) * | 2008-12-08 | 2010-06-23 | 忆正存储技术(深圳)有限公司 | Data storage method used for multichannel non-volatile solid storage device |
| CN102023820A (en) * | 2010-12-01 | 2011-04-20 | 北京同有飞骥科技股份有限公司 | Method for constructing double-disk fault-tolerant vertical packet parallel access disk array |
-
2015
- 2015-11-19 CN CN201510801232.1A patent/CN105260261B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040128470A1 (en) * | 2002-12-27 | 2004-07-01 | Hetzler Steven Robert | Log-structured write cache for data storage devices and systems |
| US20090070653A1 (en) * | 2007-08-14 | 2009-03-12 | Lg Electronics Inc. | Method of transmitting data |
| CN101752011A (en) * | 2008-12-08 | 2010-06-23 | 忆正存储技术(深圳)有限公司 | Data storage method used for multichannel non-volatile solid storage device |
| CN102023820A (en) * | 2010-12-01 | 2011-04-20 | 北京同有飞骥科技股份有限公司 | Method for constructing double-disk fault-tolerant vertical packet parallel access disk array |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107786419A (en) * | 2016-08-26 | 2018-03-09 | 西安交大捷普网络科技有限公司 | Realize the annex restoring method of webpage mailbox annex |
| CN107786419B (en) * | 2016-08-26 | 2020-05-26 | 西安交大捷普网络科技有限公司 | Attachment restoring method for realizing attachment of webpage mailbox |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105260261B (en) | 2018-06-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102779096B (en) | Page, block and face-based three-dimensional flash memory address mapping method | |
| CN104794070B (en) | Solid state flash memory write buffer system and method based on dynamic non-covered RAID technique | |
| TWI494761B (en) | Method of partitioning physical block and memory system thereof | |
| CN101425041B (en) | Optimizing method for establishing FAT file systems on NAND FLASH memory | |
| CN100426259C (en) | Virtual access method of storage document data | |
| CN106775476A (en) | Mixing memory system and its management method | |
| CN109582593B (en) | FTL address mapping reading and writing method based on calculation | |
| CN103631536B (en) | A kind of method utilizing the invalid data of SSD to optimize RAID5/6 write performance | |
| CN105930097B (en) | The distribution of small write operation verifies formula disk array in a kind of elimination local parallel | |
| CN102023809A (en) | Storage system, method for reading data from storage system and method for writing data to storage system | |
| CN107391391A (en) | The method, system and solid state hard disc of data copy are realized in the FTL of solid state hard disc | |
| CN110032521A (en) | For enhancing flash translation layer (FTL) mapping flexibility to obtain performance and service life improved method and system | |
| CN101515276A (en) | Method for write operation of file data, and recovery method and recovery system for file data | |
| CN104899117B (en) | Memory database parallel logging method towards Nonvolatile memory | |
| CN108255740A (en) | A kind of FLASH equilibriums erasing-writing method and system | |
| WO2013062562A1 (en) | Shiftable memory supporting in-memory data structures | |
| CN109407985B (en) | Data management method and related device | |
| CN102306124A (en) | Method for implementing hardware driver layer of Nand Flash chip | |
| CN103473258A (en) | Cloud storage file system | |
| CN105930099B (en) | The fault-tolerant disk array of double plate of small write operation in a kind of elimination local parallel | |
| CN102339255B (en) | Nand writing balance processing method | |
| CN113253926A (en) | Memory internal index construction method for improving query and memory performance of novel memory | |
| CN102023845A (en) | Cache concurrent access management method based on state machine | |
| CN105260261A (en) | Email recovery method | |
| CN105426130A (en) | Rapid mail handling method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20230605 Address after: F13, Building 11, Zone D, New Economic Industrial Park, No. 99, West Section of Hupan Road, Xinglong Street, Tianfu New District, Chengdu, Sichuan, 610000 Patentee after: Sichuan Shenhu Technology Co.,Ltd. Address before: No. 5, 1st Floor, Unit 1, Building 19, No. 177, Middle Section of Tianfu Avenue, High tech Zone, Chengdu, Sichuan, 610043 Patentee before: SICHUAN CINGHOO TECHNOLOGY Co.,Ltd. |
|
| TR01 | Transfer of patent right |