WO2013086867A1

WO2013086867A1 - Information processing method and equipment

Info

Publication number: WO2013086867A1
Application number: PCT/CN2012/080274
Authority: WO
Inventors: 陈跃泉
Original assignee: 华为技术有限公司
Priority date: 2011-12-14
Filing date: 2012-08-17
Publication date: 2013-06-20
Also published as: CN102523112A; CN102523112B

Abstract

An information processing method comprises: the network storage equipment receives information; the network storage equipment carries out Hash operation on the information, and obtains at least two Hash values of the information; according to the at least two Hash values, the network storage equipment stores or does not store the information. And an information processing equipment includes: a receiving unit, used to receive information; obtaining unit, used to carry out Hash operation on the information and obtain at least two Hash values of the information; processing unit, used to store or not store the information according to the at least two Hash values. With the invention, the reduction of the utilization rate of the network storage equipment and the efficiency of information maintenance and management are enhanced.

Description

Information processing method and device

The present application claims priority to Chinese Patent Application No. 201110416964.0, entitled "Information Processing Method and Apparatus", filed on Dec. 14, 2011, the entire contents of which is incorporated herein by reference.

Technical field

The present invention relates to information processing technologies, and in particular, to an information processing method and device.

Background technique

With the development of the communication technology, the terminal device or the network management system can periodically back up a large amount of information on the terminal device to the network backup device located on the network side, or the user/network administrator can manually manually use the terminal device as needed. A large amount of information is backed up in a network backup device located on the network side.

However, due to the diversification of information backup methods, the same information may be backed up multiple times in the network storage device, so that a large amount of the same information exists in the network storage device, thereby causing a decrease in the utilization rate of the network storage device, and Reduced efficiency of information maintenance and management.

Summary of the invention

The present invention provides an information processing method and device for improving the utilization of network storage devices and the efficiency of information maintenance and management.

On the one hand, it provides an information processing method, including:

The network storage device receives the information;

The network storage device performs a hash operation on the information to obtain at least two hash values of the information;

The network storage device stores the information or does not store the information according to the at least two hash values.

Optionally, the storing, by the network storage device, the information according to the at least two hash values, or not storing the information, includes:

The network storage device obtains a bitmap corresponding to the at least two hash values; the information does not store the information. Storing the information or not storing the information, comprising: comparing, by a bitmap of the at least one information received by the storage device, if the location corresponding to the bitmap corresponding to the at least two hash values is in the storage The network storage device does not store the information; if the location corresponding to the bitmap corresponding to the at least two hash values is not in the stored bitmap, the network storage device stores The information.

Optionally, after the network storage device stores the information according to the bitmap, the method further includes: storing the bitmap corresponding to the at least two hash values. Before comparing the bitmap of at least one piece of information received by the network storage device, the method further includes:

If the network storage device stores a message, the network storage device uses a bitmap corresponding to the hash value of the information as a reference bitmap, and stores the bitmap;

If the network storage device stores at least two pieces of information, the network storage device performs or operates a bitmap corresponding to the hash value of the at least two pieces of information, and uses the obtained bitmap as a reference bitmap and stores the data; A bitmap of at least one piece of information received by the device is compared, including: Bitmaps for comparison.

Optionally, the method further includes:

The network storage device performs an exclusive OR operation on the bitmap of the information within the selected range and the reference bitmap, and performs or operates with a bitmap of the information outside the selected range to obtain a new bitmap. As the reference bitmap after the update, and store it.

Another aspect provides an information processing apparatus, including:

a receiving unit, configured to receive information;

And an obtaining unit, configured to perform hash operation on the information, to obtain at least two hash values of the information;

And a processing unit, configured to store the information according to the at least two hash values or not to store the information. Optionally, the processing unit is specifically configured to:

Obtaining a bitmap corresponding to the at least two hash values, and storing the information or not storing the information according to the bitmap corresponding to the at least two hash values.

Optionally, the processing unit is specifically configured to:

Comparing the bitmap corresponding to the at least two hash values with a bitmap of the at least one information previously received by the network storage device, if the bitmap corresponding to the at least two hash values The corresponding location is in the stored bitmap, the information is not stored; if the location corresponding to the bitmap corresponding to the at least two hash values is not in the stored bitmap, the information is stored .

Optionally, the processing unit is further configured to:

If the network storage device stores at least two pieces of information, the network storage device performs or operates a bitmap corresponding to the hash value of the at least two pieces of information, and uses the obtained bitmap as a reference bitmap and stores the bitmap;

The processing unit is specifically configured to

Comparing the bitmap corresponding to the at least two hash values with the reference bitmap.

Optionally, the processing unit is further configured to:

Performing an exclusive OR operation on the bitmap of the information within the selected range and the reference bitmap, and performing or operating on the bitmap of the information outside the selected range to obtain a new bitmap as a reference after the update Bitmap, and store it.

According to the foregoing technical solution, after receiving the information, the network storage device obtains at least two hash values of the information by performing a hash operation on the foregoing information, so that the network storage device can store the foregoing according to the at least two hash values. The information does not store the above information, which can avoid the problem that a large amount of the same information exists in the network storage device due to the fact that the same information may be backed up multiple times in the network storage device in the prior art, thereby improving the utilization of the network storage device. Reduced rates, as well as the efficiency of information maintenance and management.

DRAWINGS

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings to be used in the embodiments or the description of the prior art will be briefly introduced below. The drawings in the above are some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without any inventive labor.

1 is a schematic flowchart of an information processing method according to an embodiment of the present invention;

2 is a schematic flowchart of an information processing method according to another embodiment of the present invention;

FIG. 3 is a schematic structural diagram of an information processing device according to another embodiment of the present invention.

detailed description

The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is a partial embodiment of the invention, and not all of the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

FIG. 1 is a schematic flowchart of an information processing method according to an embodiment of the present invention. As shown in FIG. 1, the information processing method in this embodiment may include:

101. The network storage device receives the information.

The foregoing information may include, but is not limited to, a short message, a multimedia message, a mail, or an instant message.

102. The network storage device performs a hash operation on the foregoing information to obtain at least two hash values of the foregoing information.

For example: the network storage device performs a hash operation such as MD5 or CityHash on the information (that is, the content of the information), and uses at least two hash functions to map the binary value of the information into at least two fixed length smaller binary values. Hash value. To further reduce overhead, non-encrypted hash operations can be performed, such as: CityHash and other hash operations.

103. The network storage device stores the foregoing information or does not store the information according to the at least two hash values.

In this embodiment, after receiving the information, the network storage device obtains at least two hash values of the information by performing a hash operation on the foregoing information, so that the network storage device can store the information according to the at least two hash values. Or not storing the foregoing information, which can avoid the problem that a large amount of the same information exists in the network storage device caused by the same information being backed up in the network storage device multiple times in the prior art, thereby improving the utilization rate of the network storage device. The reduction, as well as the efficiency of information maintenance and management. FIG. 2 is a schematic flowchart of an information processing method according to another embodiment of the present invention. This embodiment is a specific embodiment of the embodiment corresponding to FIG. 1. As shown in FIG. 2, the information processing method in this embodiment may include:

201. The network storage device receives the information.

202. The foregoing network storage device performs a hash operation on the foregoing information to obtain at least two hash values of the foregoing information.

203. The network storage device obtains a bitmap corresponding to the at least two hash values. For example, if the network storage device performs a hash operation on the information, the three hash values of the foregoing information are respectively obtained as 2 6, and 14 can be assumed that the bit corresponding to the hash value is 1, and the bitmap corresponding to the hash value can be 010001000000010000.

204. The foregoing network storage device stores the foregoing information or does not store the foregoing information according to the bitmap.

Comparing the bitmaps of the at least one information received by the network storage device, if the location corresponding to the bitmap corresponding to the at least two hash values is in the stored bitmap, the network storage device does not store the information; The location corresponding to the bitmap corresponding to the at least two hash values is not in the stored bitmap, and the network storage device stores the information.

Further, after the network storage device stores the foregoing information, the bitmap corresponding to the at least two hash values may be further stored. In an application scenario of the embodiment of the present invention, the network storage device may store a bitmap of the information after storing one piece of information each time. In another application scenario of the embodiment of the present invention, the network storage device may further store the bitmap of the information into a bitmap of the previously stored information after storing one information each time, for example: a bitmap of the information. Perform or operate with a bitmap of previously stored information, use the resulting bitmap as a reference bitmap, and store the reference bitmap.

Optionally, before the foregoing network storage device receives the bitmap and the network storage device, Before the bitmap of the at least one information is compared, if the network storage device stores a message, the network storage device may further use the bitmap corresponding to the hash value of the above information as the reference bitmap and store the data; if the network storage The device stores at least two pieces of information, and the network storage device may further perform or operate on the bitmap corresponding to the hash value of the at least two pieces of information, and use the obtained bitmap as a reference bitmap and store the bitmap. In this way, the network storage device can compare the bitmap corresponding to the at least two hash values with the reference bitmap, if the location corresponding to the bitmap corresponding to the at least two hash values is stored in the foregoing In the reference bitmap, the network storage device does not store the information; if the location corresponding to the bitmap corresponding to the at least two hash values is not in the stored reference bitmap, the network storage device stores the information.

Optionally, the network storage device in this embodiment may further clear the bits in the expired bitmap or bitmap in an instant. For example: the network storage device may perform an exclusive OR operation on the bitmap of the information within the selected range and the above reference bitmap, and perform or operate with a bitmap of the information outside the selected range to obtain a new bitmap. As the reference bitmap after the update, and stored, it can meet a huge amount of information backup, and some information is time-effective (for example: a short message about a product promotion issued by a certain brand group, the effective time limit of the short message When the promotion is stopped, the short message has a time validity requirement, which saves the memory overhead occupied by the bitmap.

In this embodiment, after receiving the information, the network storage device obtains at least two hash values of the foregoing information by performing a hash operation on the foregoing information, and obtains a bitmap corresponding to the at least two hash values, so that the network is configured. The storage device can store the foregoing information or not store the foregoing information according to the foregoing bitmap, and can avoid a large amount of the same information in the network storage device caused by the same information being backed up in the network storage device multiple times in the prior art. The problem is to improve the utilization of network storage devices and the efficiency of information maintenance and management.

In order to make the method provided by the embodiment of the present invention clearer, an 18-bit bitmap will be exemplified below. Suppose the network storage device does not currently store any information.

The first cycle is within T1

The network storage device stores information A, and obtains the bitmap corresponding to at least two hash values corresponding to A. The bitmap is A bitmap =010001000000010000, and is stored in the memory;

The network storage device receives the information B, and the bitmap corresponding to the at least two hash values corresponding to the information B is B bitmap=000010000001000010. After comparison, the B bitmap is different from the A bitmap, and the network storage The device stores information B;

At this time, the network storage device performs or operates on the B bitmap and the A bitmap to obtain a reference bitmap T1 bitmap =01001 1000001010010, and stores it in the memory;

When T1 ends, the T1 bitmap is used as a bitmap S1 bitmap of the network storage device in T1 and stored in the disk.

The second cycle is within T2

The network storage device receives the information C, and the bitmap corresponding to the at least two hash values corresponding to the information C is C bitmap = 000101000001000000. After comparison, the C bitmap is different from the T1 bitmap, and the network storage device stores the information C. ;

At this point, the network storage device performs or operates on the C bitmap and the T1 bitmap to obtain the reference bitmap T2 bitmap.

=0101 1 1000001010010, and stored in the memory;

The network storage device receives the information D, and the bitmap corresponding to the at least two hash values corresponding to the information D is D bitmap=000101000001000000. After comparison, the D bitmap is in the T2 bitmap, indicating that the network storage device has been stored. a message identical to the information D, the information D is a duplicate information, and the network storage device no longer stores the information D;

When T2 ends, the C bitmap is used as a bitmap S2 bitmap of the network storage device in T2, and stored in the disk;

The third cycle is within T3

The network storage device receives the information E, and the bitmap corresponding to the at least two hash values corresponding to the information E is the E bitmap 00001 1000001000000. After comparison, the E bitmap is in the T2 bitmap, indicating that the network storage device has been stored. After the same information as the information E, the information E is a duplicate information, and the network storage device no longer stores the information E;

The network storage device receives the information F, and the bitmap corresponding to the at least two hash values corresponding to the information F is F bitmap=100000000000010001. After comparison, the F bitmap is different from the T2 bitmap, and the network storage device stores the information F. ;

At this time, the network storage device performs or operates on the F bitmap and the T2 bitmap to obtain a reference bitmap T3 bitmap =1 101 1 100000101001 1 and stores it in the memory;

When T3 ends, the F bitmap is taken as a bitmap S3 bitmap of the network storage device in T3 and stored in the disk. If it is necessary to clear the bitmap of the information in T1, the network storage device may perform an exclusive OR operation on the bitmap of the information in Τ1, that is, the S1 bitmap and the bitmap of the at least one information, that is, the 位3 bitmap, and the S2 bitmap. Performing or operating with the S3 bitmap to obtain a new bitmap as a bitmap of the at least one piece of information Τ update=1 00101000001010001.

It should be noted that, for each of the foregoing method embodiments, for the description of the package, it is expressed as a series of action combinations, but those skilled in the art should understand that the present invention is not limited by the described action sequence. Because certain steps may be performed in other sequences or concurrently in accordance with the present invention. In addition, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

In the above embodiments, the descriptions of the various embodiments are different, and the details are not described in the specific embodiments. For details, refer to related descriptions of other embodiments.

FIG. 3 is a schematic structural diagram of an information processing device according to another embodiment of the present invention. As shown in FIG. 3, the information processing device of this embodiment may include a receiving unit 31, an obtaining unit 32, and a processing unit 33. The receiving unit 31 is configured to receive information, and the obtaining unit 32 is configured to perform hash operation on the information received by the receiving unit 31 to obtain at least two hash values of the foregoing information. The processing unit 33 is configured to obtain the information according to the obtaining unit 32. The at least two hash values described above store the above information or do not store the above information.

The functions of the network storage device in the embodiments corresponding to the foregoing FIG. 1 and FIG. 2 can be implemented by the information processing device provided in this embodiment. The bitmap should be stored, and the above information is stored or not stored according to the above bitmap. For example, the processing unit 33 compares the bitmap with the bitmap of the at least one information previously received by the network storage device, and if the location corresponding to the bitmap is in the stored bitmap, the information is not stored; The corresponding location is not in the bitmap stored above, and the above information is stored.

Further, if the network storage device stores an information, the processing unit 33 in this embodiment may further use a bitmap corresponding to the hash value of the one information as a reference bitmap, and store the data storage device; For at least two pieces of information, the processing unit 33 in this embodiment may further perform or operate on the bitmap corresponding to the hash value of the at least two pieces of information, and use the obtained bitmap as a reference bitmap and store the bitmap. The corresponding bitmap is compared to the reference bitmap.

Optionally, the information processing device in this embodiment may further clear the bits in the expired bitmap or the bitmap in an instant. For example, the processing unit 33 may be further configured to perform an exclusive OR operation on the bitmap of the information within the selected range and the reference, and perform or operate with a bitmap of the information outside the selected range to obtain a new bit. The map is used as a reference bitmap after the update, and is stored, so as to satisfy a huge amount of information backup, and some information is time-effective (for example: a short message about a product promotion issued by a certain brand group, the effective time of the short message) When the promotion is stopped, it indicates that the short message has time validity), which saves the memory overhead occupied by the bitmap.

In this embodiment, after the information processing device receives the information by the receiving unit, the obtaining unit performs a hash operation on the information received by the receiving unit to obtain at least two hash values of the information, so that the processing unit can obtain the information according to the obtaining unit. The foregoing at least two hash values store the foregoing information or do not store the foregoing information, which can avoid the existence of a large amount of the same information in the network storage device caused by the same information being backed up in the network storage device multiple times in the prior art. The problem is to improve the utilization of network storage devices and the efficiency of information maintenance and management.

A person skilled in the art can clearly understand that, for the convenience and the cleaning of the description, the specific working processes of the system, the device and the unit described above can be referred to the corresponding processes in the foregoing method embodiments, and details are not described herein again.

In the several embodiments provided herein, it should be understood that the disclosed systems, devices, and methods may be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the unit is only a logical function division. In actual implementation, there may be another division manner, for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be electrical, mechanical or otherwise.

The units described as separate components may or may not be physically separate, and the components displayed as the units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solution of the embodiment.

In addition, each functional unit in various embodiments of the present invention may be integrated in one processing unit. It is also possible that each unit physically exists alone, or two or more units may be integrated in one unit. The above integrated unit can be implemented in the form of hardware or in the form of hardware plus software functional units.

The above-described integrated unit implemented in the form of a software functional unit can be stored in a computer readable storage medium. The above software functional units are stored in a storage medium and include a number of instructions for causing a computer device (which may be a personal computer, server, or network device, etc.) to perform part of the steps of the method of the various embodiments of the present invention. The foregoing storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, and the like. The medium of the program code.

Finally, it should be noted that the above embodiments are only for explaining the technical solutions of the present invention, and are not intended to be limiting; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art will understand that The technical solutions described in the foregoing embodiments are modified, or some of the technical features are equivalently replaced. The modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

Rights request

An information processing method, comprising:

The network storage device receives the information;

The method according to claim 1, wherein the storing, by the network storage device, the information or not storing the information according to the at least two hash values comprises:

The network storage device obtains a bitmap corresponding to the at least two hash values; the information does not store the information.

The method according to claim 2, wherein the storing, by the network storage device, the information or not storing the information according to the bitmap corresponding to the at least two hash values comprises: Comparing the bitmaps of the at least one information received by the storage device, if the location corresponding to the bitmap corresponding to the at least two hash values is in the stored bitmap, the network storage device does not store the location The network storage device stores the information if the location corresponding to the bitmap corresponding to the at least two hash values is not in the stored bitmap.

The method of claim 3, wherein the network storage device, after storing the information according to the bitmap, further includes:

And storing the bitmap corresponding to the at least two hash values.

The method according to claim 3 or 4, wherein the network storage device pairs the bitmap corresponding to the at least two hash values with at least one information previously received by the network storage device Before the bitmap is compared, it also includes:

If the network storage device stores at least two pieces of information, the network storage device performs or operates on a bitmap corresponding to the hash value of the at least two pieces of information, and uses the obtained bitmap as a reference bitmap, and performs Storage; comparing bitmaps of at least one piece of information received by the storage device, including: Bitmaps for comparison.

The method according to claim 5, wherein the method further comprises: the network storage device performing an exclusive-OR operation on the bitmap of the information within the selected range and the reference bitmap, and A bitmap of the information outside the selected range is ORed to obtain a new bitmap as a reference bitmap after the update and stored.

7. An information processing device, comprising:

a receiving unit, configured to receive information;

And a processing unit, configured to store the information according to the at least two hash values or not to store the information.

The device according to claim 7, wherein the processing unit is specifically configured to obtain a bitmap corresponding to the at least two hash values, and according to the at least two hash values The corresponding bitmap stores the information or does not store the information.

The device according to claim 8, wherein the processing unit is configured to: use the bitmap corresponding to the at least two hash values and at least one information previously received by the network storage device The bitmap is compared, if the location corresponding to the bitmap corresponding to the at least two hash values is in the stored bitmap, the information is not stored; if the at least two are The location corresponding to the bitmap corresponding to the hash value is not in the stored bitmap, and the information is stored.

The device according to claim 9, wherein the processing unit is further configured to: if the network storage device stores a piece of information, the network storage device uses a bitmap corresponding to a hash value of the one piece of information as Reference bitmap, and store it;

If the network storage device stores at least two pieces of information, the network storage device performs or operates a bitmap corresponding to the hash value of the at least two pieces of information, and uses the obtained bitmap as a reference bitmap and stores the bitmap; The processing unit is specifically configured to

1 1. The device according to any one of claims 9 or 10, wherein the processing unit is further used for