CN103559319A

CN103559319A - Cache synchronization method and equipment for distributed cluster file system

Info

Publication number: CN103559319A
Application number: CN201310596276.6A
Authority: CN
Inventors: 曾红丽
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2013-11-21
Filing date: 2013-11-21
Publication date: 2014-02-05
Anticipated expiration: 2033-11-21
Also published as: CN103559319B

Abstract

The embodiment of the invention provides a cache synchronization method and equipment for a distributed cluster file system. According to the method, a global cache table is defined for an object file and includes the record that each node modifies the object file; when a first node performs write addressing on the object file, a latest global cache table is obtained according to the information of a second node which modifies the object file lastly, wherein the information of the second node is included in a writing lock, and the data that the second node modifies the object file is obtained according to the record that the second node modifies the object file lastly, wherein the record is included in the latest global cache table; when the two nodes write different data of the object file interactively at the same time, the data in caches of the two nodes are valid, and follow-up reading can be obtained from the caches; when the two nodes write same data interactively, modification data is needed to be obtained from another node. The efficiency that data is transmitted in a network is far higher than that of once disk writing and disk reading, so that the systematicness of cluster files is greatly improved when compared with that of a conventional manner.

Description

Buffer memory synchronous method and the equipment of distributed cluster file system

Technical field

The embodiment of the present invention relates to data communication technology, relates in particular to a kind of buffer memory synchronous method and equipment of distributed cluster file system.

Background technology

Distributed cluster file system can solve the problem of mass data storage, in cluster, the multiple devices service of joining together externally to provide, replaces a node to finish the work with a plurality of nodes (server), both improve processing power, can also guarantee high reliability.The cluster file system of symmetric distributed has the unified NameSpace of the overall situation, storage space is all visible and identical for each node, metadata and data can be distributed on any node of cluster, certainly, from arbitrary node, also can have access to all data of storing system.

Current storage system is all at least divided into 2 grades of storages: storage physical medium and the buffer memory hewing out from main memory.In order to promote the access performance of distributed cluster file system, at each node, have the high-speed cache of data, when client read access partial data, whether the node that receives this request first data query is stored in the high-speed cache of this node, if, directly return, if not, to disk, send I/O request, data are read buffer memory and temporarily saved from disk, while accessing in order to next time, need to not read from disk, from buffer memory, directly read, to promote, read performance.Equally, when client write access partial data, the node that receives this request first writes buffer memory by data, rather than directly writes dish, in node or in the buffer memory of whole system, accumulated and a certain amount ofly together data have been write to disk again while writing data, to promote write performance.When in system, a plurality of nodes carry out read and write access to identical file alternately, the consistance that how to guarantee efficiently data is the problem that system needs emphasis to solve.

In prior art, for distributed cluster file system, in order to guarantee that the consistance of data must use distributed lock, before visit data, first need the distributed lock of applying for that these data are corresponding.Distributed lock has read lock and writes lock, and read lock can be concurrent, writes lock for monopolizing, be that a plurality of nodes can concurrently be read same partial data, but if certain node is being revised this partial data, other nodes could be accessed after must by the time having revised, and the data that have access to comprise the data of up-to-date modification.Below will illustrate by an object lesson, Fig. 1 is the read-write schematic flow sheet of file in prior art, as shown in Figure 1, first node 1 has revised the 0th data block of file 1, and owing to applying for before the lock of writing of file 1 correspondence without any node, lock is in init state, spot lock management device directly licenses to node 1, node 1 is revised the 0th data block after obtaining and writing lock, and amended data are kept in buffer memory.2 pairs of file 1 write accesss of node afterwards, the 4th data block that needs revised file 1, distributed lock discovery node 1 applied for writing lock, so lock of writing to node 1 recovery fiel 1, after node 1 receives the order of writing lock recovery, first by the data cached lower wall of file 1, due to node 2 applications is to write lock, mean that node 2 wants the data of revised file 1, so on node 1 relevant to file 1 data cached may not can be up-to-date, need to it is invalid, afterwards lock authority is given back to spot lock management device, spot lock management device will be locked permission grant to node 2, node 2 is revised corresponding data to buffer memory after obtaining and writing lock, after revising, node 1 carrys out again the 0th data block of read access file 1, reclaim and write lock to node 2 with the same spot lock management device before, in removal process, the data of node 2 are write to dish, due to node 1 application is read lock, reading not can Update Table, so buffer memory of node 2 or effective, need not invalid corresponding buffer memory, spot lock management device licenses to node 1 by read lock afterwards, node 1 finds corresponding data from buffer memory, but find that these data are invalid, illustrate that these data may not be up-to-date, from disk, read.

By above-mentioned example, can find out, by latest data being write to dish, guarantee the consistance of the data that each node visit arrives, can there is following shortcomings: if each node carries out read-write operation to same file alternately in cluster, can cause the data of this file frequently to read dish and write dish, buffer memory is without any accelerating effect, increase on the contrary the complexity of system, and read frequently to coil, write dish by causing the delay of system to increase, will greatly affect performance.

Summary of the invention

The embodiment of the present invention provides a kind of buffer memory synchronous method and equipment of distributed cluster file system, can avoid reading frequently dish, write dish by causing the delay of system to increase, and improves system performance.

The present invention's the first invention provides a kind of buffer memory synchronous method of distributed cluster file system, comprising:

When client is carried out write access to the file destination on first node, described first node to file destination described in spot lock management device application corresponding write lock, described in write lock and comprise the information of described file destination being carried out to the Section Point of up-to-date modification;

The up-to-date global buffer table that described first node is corresponding according to file destination described in the acquisition of information of described Section Point, preserves the up-to-date amendment record of described Section Point to described file destination in described up-to-date global buffer table;

Described first node is modified to described file destination to the up-to-date amendment record of described file destination according to described Section Point, amended data are kept in the buffer memory of described first node, and described first node is updated to described up-to-date global buffer table to the amendment record of described file destination.

In the possible implementation of the first of first aspect present invention, described first node is modified to described file destination to the up-to-date amendment record of described file destination according to described Section Point, comprising:

Described first node determines whether to obtain the Update Table of described Section Point to described file destination according to described Section Point to the up-to-date amendment record of described file destination;

If, described first node obtains the Update Table of described Section Point to described file destination, the Update Table of the described file destination getting is stored in the buffer memory of described first node, the data of the described file destination stored on described first node and described Section Point are consistent, from the buffer memory of described first node, read described data, described file destination is modified;

If not, described first node reading out data from the buffer memory of described first node, modifies to described file destination.

In the possible implementation of the second of first aspect present invention, described file destination comprises at least two data blocks, and described Section Point comprises the up-to-date amendment record of described file destination: described Section Point carried out the information of the data block of up-to-date modification to described file destination;

Described first node determines whether to obtain the Update Table of described Section Point to described file destination according to described Section Point to the up-to-date amendment record of described file destination, comprising:

Described first node carried out the information of the data block of up-to-date modification to described file destination according to described Section Point, determine that described first node carries out the data block of write access to described file destination and whether data block that described Section Point carried out up-to-date modification to described file destination is same data block;

If so, described first node obtains the Update Table of described Section Point to described file destination.

In the third possible implementation of first aspect present invention, described first node, according to the up-to-date global buffer table of the acquisition of information of described Section Point, comprising:

Described first node sends global buffer table synchronization request message to described Section Point;

Described first node receives the global buffer table synchronous response message that described Section Point sends, in described global buffer table synchronous response message, comprise described up-to-date global buffer table, in described up-to-date global buffer table, preserve the up-to-date amendment record of described Section Point to described file destination.

In the 4th kind of possible implementation of first aspect present invention, described first node obtains the Update Table of described Section Point to described file destination according to described Section Point to the up-to-date amendment record of described file destination, comprising:

Described first node, according to the up-to-date amendment record of Section Point to described file destination described in described up-to-date global buffer table, sends data synchronization request message to described Section Point;

Described first node receives the data sync response message that described Section Point sends, and comprises the Update Table of described Section Point to described file destination in described data sync response message.

In the 5th kind of possible implementation of first aspect present invention, described first node carries out write operation to described file destination, amended data are kept in the buffer memory of described first node, and after described first node is updated to described up-to-date global buffer table to the amendment record of described file destination, also comprise:

When described first node carries out read access to described file destination, to read lock corresponding to file destination described in described spot lock management device application, in described read lock, comprise the information of described file destination being carried out to the described first node of up-to-date modification;

Described first node reads described file destination from the buffer memory of described first node.

In the 6th kind of possible implementation of first aspect present invention, described in described first node, file destination carries out write operation, amended data are kept in the buffer memory of described first node, and after described first node is updated to described up-to-date global buffer table to the amendment record of described file destination, also comprise:

Described first node receives the global buffer table synchronization request message that described Section Point sends, and described global buffer table synchronization request message is that described Section Point sends to described first node need to carry out read access or write access to described file destination time;

Described first node, according to described global buffer table synchronization request message, returns to global buffer table synchronous response message to described Section Point, comprises the up-to-date global buffer table after described renewal in described global buffer table synchronous response message.

In the 7th kind of possible implementation of first aspect present invention, described first node is according to described global buffer table synchronization request message, after described Section Point or described the 3rd node return to global buffer table synchronous response message, also comprises:

Described first node receives the data synchronization request that described Section Point sends, and described data synchronization request message is that described Section Point sends the amendment record of described file destination according to first node described in the up-to-date global buffer table after described renewal;

Described first node sends data sync response to described Section Point, described data sync response comprises the Update Table of described first node to described file destination, so that described Section Point is kept at described first node in the buffer memory of described Section Point to the Update Table of described file destination, when carrying out read access or write access to described data, described Section Point can from the buffer memory of described Section Point, read described file destination.

In the 8th kind of possible implementation of first aspect present invention, described first node to file destination described in spot lock management device application corresponding write lock, comprising:

Described first node sends to described spot lock management device the lock application request of writing that described file destination is corresponding, so that described spot lock management device is write lock described in described Section Point recovery;

Described first node receives that described spot lock management device sends writes lock authorization response.

Second aspect present invention provides a kind of buffer memory synchronous method of distributed type assemblies target file system, comprising:

Spot lock management device receives the lock application request of writing corresponding to file destination that described first node sends, described in to write lock application request be client when described file destination is carried out to write access, described first node sends to described spot lock management device;

Described spot lock management device determine Section Point application cross described file destination corresponding write lock, described file destination was carried out to up-to-date modification, described spot lock management device sends to described Section Point the lock of writing that described file destination is corresponding and reclaims order, to described Section Point reclaim described file destination corresponding write lock, described spot lock management device to described Section Point, reclaim described file destination corresponding write when lock, the data of the described file destination in the buffer memory of described Section Point are not write to dish and invalid;

Described spot lock management device sends and writes lock authorization response to described first node, described in write the information that lock comprises described Section Point.

In the possible implementation of the first of second aspect present invention, described spot lock management device sends and writes lock authorization response to described first node, described in write the information that lock comprises described Section Point after, also comprise:

Described spot lock management device receives read lock application request corresponding to described file destination that described first node sends;

Described spot lock management device determine described first node application cross described file destination corresponding write lock, described file destination was carried out to up-to-date modification, described spot lock management device sends to described first node the lock of writing that described file destination is corresponding and reclaims order, to described first node reclaim described file destination corresponding write lock, described spot lock management device to described first node, reclaim described file destination corresponding write when lock, the data of the described file destination in the buffer memory of described first node are not write to dish and invalid;

Described spot lock management device sends read lock authorization response to described first node, and described read lock comprises the information of described first node, so that described first node reads described file destination from the buffer memory of described first node.

In the possible implementation of the second of second aspect present invention, described spot lock management device sends and writes lock authorization response to described first node, described in write the information that lock comprises described Section Point after, also comprise:

Described spot lock management device receives the read lock that described file destination that described Section Point sends is corresponding or writes lock application request;

Described spot lock management device sends read lock or writes lock authorization response to described Section Point, and described read lock or write the information that lock comprises described first node, so that described Section Point is according to the up-to-date global buffer table of the acquisition of information of described first node.

Third aspect present invention provides a kind of buffer memory synchronous method of distributed type assemblies target file system, comprising:

Described Section Point receives the global buffer table synchronization request message that described first node sends;

Described Section Point is according to described global buffer table synchronization request message, to described first node, return to global buffer table synchronous response message, in described global buffer table synchronous response message, comprise described up-to-date global buffer table, in described up-to-date global buffer table, preserve the up-to-date amendment record of described Section Point to described file destination.

In the possible implementation of the first of third aspect present invention, described Section Point, according to described global buffer table synchronization request message, after described first node returns to global buffer table synchronous response message, also comprises:

Described Section Point receives the data synchronization request message that described first node sends, and described data synchronization request message is that described first node sends the up-to-date amendment record of described file destination according to Section Point described in described up-to-date global buffer table;

Described Section Point, according to described data synchronization request message, to described first node return data synchronous response message, comprises the Update Table of described Section Point to described file destination in described data sync response message.

Fourth aspect present invention provides a kind of node, and described node is first node, comprising:

Lock application module, for when client is carried out write access to the file destination on first node, to file destination described in spot lock management device application corresponding write lock, described in write lock and comprise the information of described file destination being carried out to the Section Point of up-to-date modification;

Cache table acquisition module, for the up-to-date global buffer table corresponding according to file destination described in the acquisition of information of described Section Point, preserves the up-to-date amendment record of described Section Point to described file destination in described up-to-date global buffer table;

File modification module, for the up-to-date amendment record of described file destination being modified to described file destination according to described Section Point, amended data are kept in the buffer memory of described first node, and described first node is updated to described up-to-date global buffer table to the amendment record of described file destination.

In the possible implementation of the first of fourth aspect present invention, described file modification module comprises judging unit, data capture unit and file modification unit;

Described judging unit, for determining whether to obtain the Update Table of described Section Point to described file destination according to described Section Point to the up-to-date amendment record of described file destination;

If, described acquiring unit obtains the Update Table of described Section Point to described file destination, the Update Table of the described file destination getting is stored in the buffer memory of described first node, the data of the described file destination stored on described first node and described Section Point are consistent, described file modification unit reads described data from the buffer memory of described first node, and described file destination is modified;

If not, described file modification unit reading out data from the buffer memory of described first node, modifies to described file destination.

In the possible implementation of the second of fourth aspect present invention, described file destination comprises at least two data blocks, and described Section Point comprises the up-to-date amendment record of described file destination: described Section Point carried out the information of the data block of up-to-date modification to described file destination;

Described judging unit specifically for:

According to described Section Point, described file destination was carried out the information of the data block of up-to-date modification, determined that described first node carried out the data block of write access to described file destination and whether data block that described Section Point carried out up-to-date modification to described file destination is same data block;

If so, described data capture unit obtains the Update Table of described Section Point to described file destination.

In the third possible implementation of fourth aspect present invention, described cache table acquisition module specifically for:

To described Section Point, send global buffer table synchronization request message;

Receive the global buffer table synchronous response message that described Section Point sends, in described global buffer table synchronous response message, comprise described up-to-date global buffer table, in described up-to-date global buffer table, preserve the up-to-date amendment record of described Section Point to described file destination.

In the 4th kind of possible implementation of fourth aspect present invention, described data capture unit specifically for:

According to the up-to-date amendment record of Section Point to described file destination described in described up-to-date global buffer table, to described Section Point, send data synchronization request message;

Receive the data sync response message that described Section Point sends, in described data sync response message, comprise the Update Table of described Section Point to described file destination.

In the 5th kind of possible implementation of fourth aspect present invention, also comprise: file read module;

Described lock application module also for: when described first node carries out read access to described file destination, to read lock corresponding to file destination described in described spot lock management device application, in described read lock, comprise the information of described file destination being carried out to the described first node of up-to-date modification;

Described file read module, reads described file destination for the buffer memory from described first node.

In the 6th kind of possible implementation of fourth aspect present invention, also comprise:

Receiver module, the global buffer table synchronization request message sending for receiving described Section Point, described global buffer table synchronization request message is that described Section Point sends to described first node need to carry out read access or write access to described file destination time;

Sending module, for according to described global buffer table synchronization request message, returns to global buffer table synchronous response message to described Section Point, comprises the up-to-date global buffer table after described renewal in described global buffer table synchronous response message.

In the 7th kind of possible implementation of fourth aspect present invention, described receiver module also for:

Receive the data synchronization request that described Section Point sends, described data synchronization request message is that described Section Point sends the amendment record of described file destination according to first node described in the up-to-date global buffer table after described renewal;

Described sending module also for: to described Section Point, send data sync response, described data sync response comprises the Update Table of described first node to described file destination, so that described Section Point is kept at described first node in the buffer memory of described Section Point to the Update Table of described file destination, when carrying out read access or write access to described data, described Section Point can from the buffer memory of described Section Point, read described file destination.

In the 8th kind of possible implementation of fourth aspect present invention, described lock application module specifically for:

To described spot lock management device, send the lock application request of writing that described file destination is corresponding, so that described spot lock management device is write lock described in described Section Point recovery;

What receive that described spot lock management device sends writes lock authorization response.

A kind of spot lock management device of fifth aspect present invention, comprising:

Receiver module, the lock application request of writing corresponding to file destination sending for receiving described first node, described in to write lock application request be client when described file destination is carried out to write access, described first node sends to described spot lock management device;

Determination module, for determine Section Point application cross described file destination corresponding write lock, described Section Point carried out up-to-date modification to described file destination;

Sending module, for send the lock of writing that described file destination is corresponding to described Section Point, reclaim order, to described Section Point reclaim described file destination corresponding write lock, described spot lock management device to described Section Point, reclaim described file destination corresponding write when lock, the data of the described file destination in the buffer memory of described Section Point are not write to dish and invalid;

Described sending module also for: to described first node, send and write lock authorization response, described in write the information that lock comprises described Section Point.

In the possible implementation of the first of fifth aspect present invention, described receiver module also for:

Receive read lock application request corresponding to described file destination that described first node sends;

Described determination module also for: determine described first node application cross described file destination corresponding write lock, described first node carried out up-to-date modification to described file destination;

Described sending module also for: to described first node, send the lock of writing that described file destination is corresponding and reclaim order, to described first node reclaim described file destination corresponding write lock, described spot lock management device to described first node, reclaim described file destination corresponding write when lock, the data of the described file destination in the buffer memory of described first node are not write to dish and invalid;

Described sending module also for: to described first node, send read lock authorization response, described read lock comprises the information of described first node, so that described first node reads described file destination from the buffer memory of described first node.

In the possible implementation of the second of fifth aspect present invention, described receiver module also for:

Receive the read lock that described file destination that described Section Point sends is corresponding or write lock application request;

Described sending module also for: to described Section Point, send read lock or write lock authorization response, described read lock or write the information that lock comprises described first node, so that described Section Point is according to the up-to-date global buffer table of the acquisition of information of described first node.

Sixth aspect present invention provides a kind of node, and described node is Section Point, comprising:

Receiver module, the global buffer table synchronization request message sending for receiving described first node;

Sending module, be used for according to described global buffer table synchronization request message, to described first node, return to global buffer table synchronous response message, in described global buffer table synchronous response message, comprise described up-to-date global buffer table, in described up-to-date global buffer table, preserve the up-to-date amendment record of described Section Point to described file destination.

In the possible implementation of the first of sixth aspect present invention, described receiver module also for:

Receive the data synchronization request message that described first node sends, described data synchronization request message is that described first node sends the up-to-date amendment record of described file destination according to Section Point described in described up-to-date global buffer table;

Described sending module also for: according to described data synchronization request message, to described first node return data synchronous response message, in described data sync response message, comprise the Update Table of described Section Point to described file destination.

The embodiment of the present invention provides a kind of buffer memory synchronous method and equipment of distributed cluster file system, by being a global buffer table of file destination definition, in global buffer table, include the amendment record of each node to file destination, when first node carries out write access to file destination, according to write in lock, comprise file destination was carried out to up-to-date modification the acquisition of information of Section Point to up-to-date global buffer table, according to Section Point in up-to-date global buffer table, the up-to-date amendment record of file destination is got to the Update Table of Section Point to file destination, the data of writing alternately in the situation of different pieces of information of file destination in buffer memory separately when two nodes are all effective simultaneously, follow-up reading can directly be obtained from buffer memory, when two nodes are write in the situation of identical data alternately, need to obtain Update Table from another node, data are more a lot of than once writing dish and reading to coil efficiency height in internetwork transmission, so intersect to revise identical data or different pieces of information no matter be, cluster file system performance is compared existing mode and is all had a distinct increment.

Accompanying drawing explanation

In order to be illustrated more clearly in the embodiment of the present invention or technical scheme of the prior art, to the accompanying drawing of required use in embodiment or description of the Prior Art be briefly described below, apparently, accompanying drawing in the following describes is some embodiments of the present invention, for those of ordinary skills, do not paying under the prerequisite of creative work, can also obtain according to these accompanying drawings other accompanying drawing.

Fig. 1 is the read-write schematic flow sheet of file in prior art;

Fig. 2 is the physical structure schematic diagram of distributed cluster file system;

Fig. 3 is the process flow diagram of the buffer memory synchronous method embodiment mono-of distributed cluster file system of the present invention;

Fig. 4 is the process flow diagram of the buffer memory synchronous method embodiment bis-of distributed cluster file system of the present invention;

Fig. 5 is the process flow diagram of the buffer memory synchronous method embodiment tri-of distributed cluster file system of the present invention;

Fig. 6 is the process flow diagram of the buffer memory synchronous method embodiment tetra-of distributed cluster file system of the present invention;

Fig. 7 is the structural representation of first node of the present invention;

Fig. 8 is the structural representation of the embodiment bis-of first node of the present invention;

Fig. 9 is the structural representation of the embodiment mono-of spot lock management device of the present invention;

Figure 10 is the structural representation of Section Point embodiment mono-of the present invention;

Figure 11 is the structural representation of first node embodiment tri-of the present invention;

Figure 12 is the structural representation of spot lock management device embodiment bis-of the present invention;

Figure 13 is the structural representation of Section Point embodiment bis-of the present invention.

Embodiment

For making object, technical scheme and the advantage of the embodiment of the present invention clearer, below in conjunction with the accompanying drawing in the embodiment of the present invention, technical scheme in the embodiment of the present invention is clearly and completely described, obviously, described embodiment is the present invention's part embodiment, rather than whole embodiment.Embodiment based in the present invention, those of ordinary skills, not making the every other embodiment obtaining under creative work prerequisite, belong to the scope of protection of the invention.

Before introducing various embodiments of the present invention, first simply introduce distributed cluster file system, Fig. 2 is the physical structure schematic diagram of distributed cluster file system, as shown in Figure 2, distributed cluster file system has six servers, be respectively node 1, node 2, node 3, memory node 1, memory node 2, memory node 3, node 1, node 2 and node 3 are mainly used to process the operations such as data read request that client sends and write request, memory node 1, memory node 2, memory node 3 is mainly used to store data, node 1, node 2, the data of node 3 storages can be stored in memory node 1, memory node 2, on memory node 3, the physical structure of this distributed cluster file system shown in Fig. 2 is just used for illustrating, in actual application process, can also there is more node, and the method that various embodiments of the present invention provide is applicable to all distributed cluster file systems, be not subject to the impact of the hardware configuration of distributed cluster file system.

Distributed cluster file system has the unified NameSpace of the overall situation, storage space is all visible and identical for each node, metadata and data can be distributed in cluster on any node, certainly, from arbitrary node, also can have access to all data of storing system, take Fig. 2 as example, and memory node 1, memory node 2, memory node 3 have formed storage space jointly, and node 1, node 2, node 3 can be accessed all data on these three memory nodes.For distributed cluster file system, the consistance that guarantees data must be used distributed lock, distributed lock is hereinafter to be referred as lock, before visit data, first node needs to the lock of file corresponding to these data of spot lock management device application, spot lock management device, for the correspondence lock of management system All Files, is responsible for each node and distributes lock, and reclaim lock to each node.Each file has a read lock and writes lock, in fact the read lock of a file and write lock and can think a lock, spot lock management device is in node license lock, in lock, be provided with rank, write the rank that is superior to read lock of lock, if node application is to write lock, the rank of lock is set to write the rank that lock is corresponding, in lock, also comprise the object that lock is authorized, if lock will license to first node, spot lock management device writes the information of first node in lock in authorization object.Wherein, read lock can be concurrent, writes lock for monopolizing, and a plurality of nodes can concurrently be read same partial data, if but certain node is being revised this partial data, and other nodes could be accessed this partial data after must by the time having revised.

Fig. 3 is the process flow diagram of the buffer memory synchronous method embodiment mono-of distributed cluster file system of the present invention, and as shown in Figure 3, the method that the present embodiment provides can comprise the following steps:

Step 101, when client is carried out write access to the file destination on first node, first node to spot lock management device application file destination corresponding write lock.

When client is carried out write access to the file destination on first node, first node first to spot lock management device application file destination corresponding write lock, in the present embodiment, spot lock management device is to first node mandate, this writes when lock, at this, writes and in lock, comprises the information of file destination being carried out to the Section Point of up-to-date modification.

Wherein, first node to spot lock management device application file destination corresponding write lock, be specially: first node sends to spot lock management device the lock application request of writing that file destination is corresponding, so that spot lock management device reclaims the lock of writing of this file destination to Section Point.Spot lock management device receives corresponding the writing after lock application request of file destination that first node sends, determine whether other node applications cross file destination corresponding write lock, spot lock management device find Section Point application cross file destination corresponding write lock, Section Point carried out up-to-date modification to file destination, spot lock management device sends to Section Point the lock of writing that file destination is corresponding and reclaims order, Section Point reclaims after order at the lock of writing that receives spot lock management device transmission, preparing lock gives back, different with prior art is, Section Point is being locked while giving back, not by the data lower wall in the buffer memory of Section Point and invalid, data in Section Point buffer memory are still effectively.Section Point will be write lock and return to spot lock management device, and spot lock management device sends and writes lock authorization response to first node, writes the information that lock comprises Section Point, and first node receives that spot lock management device sends writes lock authorization response.

Step 102, the up-to-date global buffer table that first node is corresponding according to the acquisition of information file destination of Section Point.

First, introduce global buffer table, it is global buffer table of each file set up (Global Cache Table that various embodiments of the present invention are passed through in cluster, be called for short GCT), be the buffer memory distribution table of latest data in cluster, the modification situation of this table record this document in cluster, along with read lock is updated on each node with writing to lock, when a certain node is revised file, this node is updated in global buffer table the amendment record of file, follow-up when other node visit this document, first obtain global buffer table corresponding to this document, this up-to-date global buffer table is kept at file was carried out on the node of up-to-date modification.In the present embodiment, file destination refers to the current file that will access of client, first node according to the information of Section Point obtain the up-to-date global buffer table that file destination is corresponding, in this global buffer table, record the up-to-date amendment record of Section Point to file destination, for example Section Point carried out modification to which data block in file destination.

First node is according to the up-to-date global buffer table of the acquisition of information of Section Point, be specially: first node sends global buffer table synchronization request message to Section Point, Section Point receives after the global buffer table synchronization request message of first node transmission, to first node, return to the response of global table table synchronization request, first node receives the global buffer table synchronous response message that Section Point sends, in global buffer table synchronous response message, comprise up-to-date global buffer table, in up-to-date global buffer table, preserve the up-to-date amendment record of Section Point to file destination.

Step 103, first node are modified to file destination to the up-to-date amendment record of file destination according to Section Point, amended data are kept in the buffer memory of first node, and first node is updated to up-to-date global buffer table to the amendment record of file destination.

First node gets after up-to-date global buffer table, according to the Section Point recording in this up-to-date global buffer table, the up-to-date amendment record of file destination is modified to file destination, be specially, first node determines whether to obtain the Update Table of Section Point to file destination according to Section Point to the up-to-date amendment record of file destination, if, be that first node is determined and need to be obtained the Update Table of Section Point to file destination, first node obtains the Update Table of Section Point to file destination, the Update Table of the file destination getting is stored in the buffer memory of first node, the data of the file destination stored on first node and Section Point are consistent, then, first node is reading out data from the buffer memory of first node, file destination is modified.If not, i.e. the data consistent of the file destination of the upper storage of first node and Section Point, the direct reading out data from the buffer memory of first node of first node, modifies to file destination.

Generally, file destination comprises at least two data blocks, when first node and Section Point are alternately modified to file destination, may be that the same data block in file destination is modified, also may be that different data blocks is modified, if same data block is modified, after Section Point is revised, when first node is revised again, the data of this same data block of now preserving on first node are not up-to-date, up-to-date data are kept on Section Point, in this case, first node need to obtain the Update Table of Section Point to this same data block from Section Point, obtain the up-to-date Update Table of this same data block, if the data that the data that first node will be revised and Section Point will be revised are not same data blocks, need to not obtain the Update Table of Section Point to file destination from Section Point.

In the present embodiment, Section Point is to comprising in the up-to-date amendment record of file destination: Section Point carried out the information of the data block of up-to-date modification to file destination.First node determines whether to obtain the Update Table of Section Point to file destination according to Section Point to the up-to-date amendment record of file destination, be specially: first node carried out the information of the data block of up-to-date modification to file destination according to Section Point, determine that first node carries out the data block of write access to file destination and whether data block that Section Point carried out up-to-date modification to file destination is same data block; If so, being first node carries out the data block of write access and data block that Section Point carried out up-to-date modification to file destination is same data block to file destination, and first node obtains the Update Table of Section Point to file destination.If file destination is carried out to the data block of write access to first node and data block that Section Point carried out up-to-date modification to file destination is not same data block, first node does not need to obtain the Update Table of Section Point to file destination, and directly from the buffer memory of first node, reading out data is modified.By the method, internodal data interaction is transmitted as required, if different data blocks is carried out to write or read operation, between node, only mutual up-to-date global buffer table is just passable, has avoided unnecessary data transmission, and the network overhead taking is few.

In the present embodiment, first node obtains the Update Table of Section Point to file destination according to Section Point to the up-to-date amendment record of file destination, be specially: first node is according to the up-to-date amendment record of Section Point to file destination in up-to-date global buffer table, to Section Point, send data synchronization request message, Section Point receives after data synchronization request message, to first node return data synchronous response message, first node receives the data sync response message that Section Point sends, in data sync response message, comprise the Update Table of Section Point to file destination.Here, data synchronization request message is for obtaining Section Point to the amended data of file destination to Section Point, in a kind of mode, first node can only obtain the data that Section Point is revised file destination, as Section Point has only been revised the data in the first data block of file destination, first node only obtains the corresponding amended data of the first several piece, does not need to obtain all data of file destination, can reduce like this transmission of data between first node and Section Point.Certainly, also can obtain the data of the whole file destination of storing on Section Point, the present invention is not as limit.First node is by obtaining the Update Table of Section Point to the first data block from Section Point, making the data of the file destination stored in the buffer memory on first node is up-to-date data, thereby make the data between first node and Section Point can be synchronous, guarantee that the data of this file destination of storing on each node are all consistent.

Step 104, first node carry out write operation to file destination, amended data are kept in the buffer memory of first node, and first node is updated to up-to-date global buffer table to the amendment record of file destination.

When first node determines that the data of the global buffer table preserved on first node and file destination are all up-to-date, from the buffer memory of first node, read file destination, file destination is carried out to write operation, amended data are kept in the buffer memory of first node, after successfully modified to file destination, first node is updated to up-to-date global buffer table by first node to the amendment record of file destination, due to write access, can modify to the data of file destination, so each node after file destination is modified, all will upgrade the global buffer table that self stores.

The method that the present embodiment provides, by being a global buffer table of file destination definition, in global buffer table, include the amendment record of each node to file destination, when first node carries out write access to file destination, according to write in lock, comprise file destination was carried out to up-to-date modification the acquisition of information of Section Point to up-to-date global buffer table, according to Section Point in up-to-date global buffer table, the up-to-date amendment record of file destination is got to the Update Table of Section Point to file destination, the data of writing alternately in the situation of different pieces of information of file destination in buffer memory separately when two nodes are all effective simultaneously, follow-up reading can directly be obtained from buffer memory, when two nodes are write in the situation of identical data alternately, need to obtain Update Table from another node, data are more a lot of than once writing dish and reading to coil efficiency height in internetwork transmission, so intersect to revise identical data or different pieces of information no matter be, cluster file system performance is compared existing mode and is all had a distinct increment.

On the basis of embodiment mono-, in another embodiment of the present invention, after step 104, can also comprise the following steps: when first node carries out read access to file destination, to read lock corresponding to spot lock management device application file destination, in read lock, comprise the information of file destination being carried out to the first node of up-to-date modification; First node reads file destination from the buffer memory of first node.After first node is modified to file destination, if first node applies for again file destination to carry out read access, first node is first to read lock corresponding to spot lock management device application file destination, spot lock management device is after receiving the read lock application request of first node transmission, the buffer memory on first node is not write to dish, data in first node buffer memory are still effectively, spot lock management device licenses to first node by read lock, in this read lock, comprise the information of file destination being carried out to the first node of up-to-date modification, first node finds it oneself is file destination to be carried out to the node of up-to-date modification according to read lock, therefore, the data of the file destination of storing in the buffer memory of first node are effective, first node reads file destination from buffer memory.

On the basis of embodiment mono-, in further embodiment of this invention, after step 104, can also comprise the following steps: first node receives the global buffer table synchronization request message that Section Point sends, this global buffer table synchronization request message is that Section Point sends to first node need to carry out read access or write access to file destination time.Concrete, after first node is modified to file destination, when if Section Point need to carry out read access or write access to file destination, Section Point is first to read lock corresponding to spot lock management device application file destination or write lock, spot lock management device is receiving the read lock of Section Point transmission or is writing after lock application request, the buffer memory on first node is not write to dish, data in first node buffer memory are still effectively, spot lock management device is by read lock or write lock and license to Section Point, this read lock or write and comprise the information of file destination being carried out to the first node of up-to-date modification in lock, Section Point sends global buffer table synchronization request message according to the information of first node to first node, first node receives after the slow table of overall situation synchronization request message, to Section Point, return to global buffer table synchronous response message, in global buffer table synchronous response message, comprise up-to-date global buffer table after renewal according to the up-to-date global buffer table after upgrading.

After the up-to-date global buffer table of Section Point after getting this renewal, up-to-date global buffer table after this upgrades is as up-to-date global buffer table, according to the amendment record of first node to file destination in up-to-date global buffer table, determine that whether first node is identical with the data of the file destination that oneself will access to the Update Table of file destination, if identical, illustrate that the data of preserving on first node are up-to-date, Section Point sends data synchronization request to first node, first node receives the data synchronization request that Section Point sends, data synchronization request message is that Section Point sends the amendment record of file destination according to first node in the up-to-date global buffer table after upgrading.

First node sends data sync response to Section Point, data sync response comprises the Update Table of first node to file destination, so that Section Point is kept at first node in the buffer memory of Section Point to the Update Table of file destination, when carrying out read access or write access to data, Section Point can from the buffer memory of Section Point, read file destination.

Fig. 4 is the process flow diagram of the buffer memory synchronous method embodiment bis-of distributed cluster file system of the present invention, and as shown in Figure 4, the method that the present embodiment provides comprises the following steps:

Step 201, spot lock management device receive the lock application request of writing corresponding to file destination that first node sends, and when writing lock application request and be client file destination being carried out to write access, first node sends to spot lock management device.

When client is carried out write access to file destination, first node first to spot lock management device application file destination corresponding write lock.

Step 202, spot lock management device determine Section Point application cross file destination corresponding write lock, file destination was carried out to up-to-date modification, spot lock management device sends to Section Point the lock of writing that file destination is corresponding and reclaims order, to Section Point reclaim file destination corresponding write lock, spot lock management device, when locking to corresponding the writing of Section Point recovery file destination, is not write dish and invalid to the data of the file destination in the buffer memory of Section Point.

Spot lock management device is in corresponding the writing after lock application request of the file destination that receives first node transmission, determine whether other node applications cross this file destination corresponding write lock, if spot lock management device determine Section Point application cross file destination corresponding write lock, file destination was carried out to up-to-date modification, spot lock management device to Section Point reclaim this file destination corresponding write lock.With different in prior art, spot lock management device is when locking to corresponding the writing of Section Point recovery file destination, the data of the file destination in the buffer memory of Section Point are not write to dish and invalid, therefore, the data of the file destination of storing in the buffer memory of Section Point are still effectively, if when follow-up Section Point is accessed in this file destination again, need to not read, directly from buffer memory, read the data that need in file destination from disk.

Step 203, spot lock management device send and write lock authorization response to first node, write the information that lock comprises Section Point.

Spot lock management device reclaims corresponding the writing lock of file destination from Section Point, to first node, send and write lock authorization response, the lock of writing that file destination is corresponding licenses to first node, and comprises the information of file destination being carried out to the Section Point of up-to-date modification writing lock.First node is write after lock in acquisition, find that Section Point carried out writing lock to file destination, to determine the data of the file destination of preserving in self buffer memory be not up-to-date to first node, therefore, first node need to obtain up-to-date global buffer table from Section Point, preserves the up-to-date amendment record of Section Point to file destination in up-to-date global buffer table.First node determines whether to obtain the Update Table of Section Point to file destination according to Section Point to the up-to-date amendment record of file destination.

The method that the present embodiment provides, when first node is locked to corresponding the writing of spot lock management device application file destination, spot lock management device determine Section Point also applied for file destination corresponding write lock, spot lock management device is when locking to corresponding the writing of Section Point recovery file destination, the data in Section Point buffer memory are not write to dish and invalid, data in Section Point buffer memory are still effectively, thereby when Section Point is modified to file destination again, can be from the buffer memory of Section Point reading out data, even if first node and Section Point are alternately modified to file destination, data in the buffer memory of two nodes are all effective, avoided reading frequently dish and write dish.

The present embodiment spot lock management device is after lock authorization response is write in first node transmission, if first node need to carry out read access to file destination, the method that the present embodiment provides can also comprise the following steps: spot lock management device receives read lock application request corresponding to file destination that first node sends; Spot lock management device determine first node application cross file destination corresponding write lock, file destination was carried out to up-to-date modification, spot lock management device sends to first node the lock of writing that file destination is corresponding and reclaims order, to first node reclaim file destination corresponding write lock, spot lock management device is when locking to corresponding the writing of first node recovery file destination, the data of the file destination in the buffer memory of first node are not write to dish and invalid, therefore, the data in first node buffer memory are still effectively.Spot lock management device reclaims corresponding the writing after lock of file destination to first node, to first node, sends read lock authorization response, and read lock comprises the information of first node, so that first node reads file destination from the buffer memory of first node.Here, because spot lock management device is when locking to corresponding the writing of first node recovery file destination, the data in first node buffer memory are not write to dish and invalid, data in first node buffer memory are still effectively, first node is reading out data from buffer memory directly, when if follow-up Section Point also needs to read the data of file destination, can be according to first node the amendment record to file destination, obtain the Update Table of first node to file destination, first node sends to Section Point by self to the Update Table of file destination, make the data consistent of two nodes, thereby avoided Section Point reading out data from disk, because data are more a lot of than once writing dish and reading to coil efficiency height in internetwork transmission, improved the utilization factor of resource.

The present embodiment spot lock management device is after lock authorization response is write in first node transmission, if Section Point need to carry out read access or write access to file destination, the method that the present embodiment provides can also comprise the following steps: spot lock management device receives the read lock that file destination that Section Point sends is corresponding or writes lock application request, spot lock management device is receiving the read lock that file destination that Section Point sends is corresponding or is writing after lock application request, determine whether other node applications cross this file destination corresponding write lock or read lock, spot lock management device determine first node application cross file destination corresponding write lock, file destination was carried out to up-to-date modification, spot lock management device sends to first node the lock of writing that file destination is corresponding and reclaims order, to first node reclaim file destination corresponding write lock, spot lock management device is when locking to corresponding the writing of first node recovery file destination, the data of the file destination in the buffer memory of first node are not write to dish and invalid, data in first node buffer memory are still effectively, spot lock management device is reclaiming corresponding the writing after lock of file destination to first node, to Section Point, send read lock or write lock authorization response, read lock or write the information that lock comprises first node, so that Section Point is according to the up-to-date global buffer table of the acquisition of information of first node.After Section Point obtains read lock or writes lock authorizing, know that first node carried out modification to file destination, read lock or write the information that lock comprises first node, from first node, obtain up-to-date global buffer table, in this up-to-date global buffer table, record the amendment record of first node to file destination.And according to first node, the amendment record of file destination is determined whether need to be from first node synchrodata, particularly, first node determines that first node carries out the data block of write access to file destination and whether data block that Section Point will be accessed is same data block, for instance, if Section Point is wanted the 0th data block in access destination file, first node carried out modification to the 2nd data block of file destination, but first node was not revised the 0th data block, the data of the 0th data block of the file destination of preserving on Section Point are still up-to-date, Section Point does not need to obtain the Update Table of first node to file destination, reading out data from the buffer memory of Section Point, if do not had in buffer memory, from the disk of Section Point, read.If what Section Point will be accessed is the 2nd data, Section Point need to obtain the Update Table of first node to file destination, makes two data on node synchronous.Section Point passes over data from first node by network, thereby has avoided Section Point reading out data from disk, because data are more a lot of than once writing dish and reading to coil efficiency height in internetwork transmission, has improved the utilization factor of resource.

Fig. 5 is the process flow diagram of the buffer memory synchronous method embodiment tri-of distributed cluster file system of the present invention, and as shown in Figure 5, the method that the present embodiment provides comprises the following steps:

Step 301, Section Point receive the global buffer table synchronization request message that first node sends.

First node, receiving writing after lock or read lock authorization response that spot lock management device sends, sends global buffer table synchronization request message to Section Point, for obtaining up-to-date global buffer table.

Step 302, Section Point are according to global buffer table synchronization request message, to first node, return to global buffer table synchronous response message, in global buffer table synchronous response message, comprise up-to-date global buffer table, in up-to-date global buffer table, preserve the up-to-date amendment record of Section Point to file destination.

After Section Point sends to first node by up-to-date global buffer table, in up-to-date global buffer table, preserve the up-to-date amendment record of Section Point to file destination, first node is the up-to-date amendment record to file destination according to Section Point, determine whether to obtain the Update Table of first node to file destination, if desired, first node sends data synchronization request message to Section Point, and Section Point receives the data synchronization request message that first node sends; Section Point, according to data synchronization request message, to first node return data synchronous response message, comprises the Update Table of Section Point to file destination in data sync response message.Thereby make the data on first node and Section Point synchronous.

Fig. 6 is the process flow diagram of the buffer memory synchronous method embodiment tetra-of distributed cluster file system of the present invention, the present embodiment will be described in detail the scheme shown in embodiment mono-by a concrete example, as shown in Figure 6, the method that the present embodiment provides comprises the following steps:

Step 401, when Section Point is wanted the 4th data block of written document 1, Section Point is to the lock application request write of spot lock management device Transmit message 1 correspondence.

In the present embodiment, file 1 is file destination, and file 1 comprises six data blocks, the 0th data block, the 1st data block, the 2nd data block, the 3rd data block, the 4th data block, the 5th data block.When first node and Section Point all do not carry out write operation to file 1, the global buffer table of file 1 correspondence is empty, when being empty, global buffer table carries out read access, elder generation's reading out data from the buffer memory of node, if the data that need to read that are cached with of node are just returned from buffer memory, if there is no the data that will read in the buffer memory of node, read from disk.Section Point carries out write access to file 1 afterwards, the 4th data block of written document 1, and Section Point is first to the lock of writing of spot lock management device application documents 1 correspondence.

Step 402, spot lock management device respond to the lock application of writing of Section Point backspace file 1 correspondence.

Spot lock management device is applied for, after request, determining and not having other node applications to cross the lock of writing of file 1 correspondence at the lock of writing that receives Section Point transmission, writing of file 1 correspondence is locked and licensed to Section Point.

Step 403, Section Point are modified to file 1 the 4th data block, and the global buffer table of updating file 1 correspondence.

Section Point hold file 1 write lock after, the 4th data block to file 1 is revised, and amended data are kept in buffer memory, the global buffer table of while updating file 1 correspondence, global buffer table in file 1 correspondence adds new entry, the corresponding global buffer table of each file, in global buffer table, there are a plurality of entries, the corresponding data block of each entry, if there are 5 data blocks in file, for each data block is set up an entry, after a certain node is modified to a certain data block, upgrade entry corresponding to this data block in global buffer table.In this step, owing to before global buffer table being sky, need to add new entry, this entry is entry corresponding to the 4th data block, here Section Point is updated in entry corresponding to the 4th data block the up-to-date amendment record of the 4th data block of file 1.

Step 404, when first node is wanted the 4th data block of written document 1, first node is to the lock of writing of spot lock management device application documents 1 correspondence.

Step 405, spot lock management device reclaim request to the lock of writing of Section Point Transmit message 1 correspondence.

Step 406, Section Point reclaim response to the lock of writing of spot lock management device backspace file 1 correspondence.

Section Point reclaims after request at the lock of writing that receives file 1 correspondence of spot lock management device transmission, prepares to lock and returns, and when lock is returned, the data in Section Point buffer memory is not write to dish and invalid, and the data in Section Point buffer memory are still effectively.Step 407, spot lock management device respond to the lock application of writing of first node backspace file 1 correspondence.

Spot lock management device licenses to first node by the lock of writing of file 1 correspondence, writes and in lock, comprises the information of file 1 being carried out to the Section Point of up-to-date modification.

Step 408, first node send global buffer table synchronization request message to Section Point.

First node sends global buffer table synchronization request message to Section Point, for obtain up-to-date global buffer table to Section Point.

The global buffer table synchronous response message that step 409, Section Point send to first node.

In global buffer table synchronous response message, comprise up-to-date global buffer table, in up-to-date global buffer table, preserve the up-to-date amendment record of Section Point to file 1, be i.e. the amendment record of Section Point to the 4th data block of file 1.

Step 410, first node, according to the up-to-date amendment record of Section Point to file destination in up-to-date global buffer table, send the data synchronization request message of the 4th data block to Section Point.

First node receives after the global buffer table synchronous response message of Section Point transmission, according to the up-to-date global buffer table of file 1, determine that Section Point revised the 4th data block of file 1, first node sends data synchronization request message to Section Point, for obtaining the Update Table of Section Point to the 4th data block of file 1.

Step 411, Section Point send the data sync response message of the 4th data block to first node.

Wherein, in the data sync response message of the 4th data block, comprise the Update Table of Section Point to the 4th data block of file destination.

Step 412, first node are revised the 4th data block, amended data are saved in the buffer memory of first node to the global buffer table of updating file 1 correspondence.

First node receives after the Update Table of the 4th data block that Section Point sends, the Update Table of the 4th data block is inserted in the buffer memory of allocating in advance, the data that complete two nodes are synchronous, carry out afterwards real modification, will be kept in buffer memory the amended data of the 4th data block.Due to when Section Point is revised the 4th data block, in global buffer table, added the amendment record item of the 4th data block, so, here first node upgrades the entry of global buffer table after the 4th data block has been revised, and is updated to: the 4th data block of file 1 is revised by first node.

Step 413, when first node will be revised the 3rd data block of file 1, inquiry global buffer table, determines and does not have other nodes to revise the 3rd data block, and the 3rd data block is modified.

If first node carrys out the 3rd data block of revised file 1 again, inquiry global buffer table, discovery does not have other nodes to carry out modification to the 3rd data block, without synchronously, directly revise, the 3rd data block does not have relevant entry in global buffer table, need to add the entry of the 3rd data block, and the 3rd data block of file 1 was revised by first node.

Step 414, to read the 4th data block of file 1 when Section Point, to the read lock application request of spot lock management device Transmit message 1 correspondence.

If follow-up Section Point will carry out read access to the 4th data block of file 1, the read lock of application documents 1 correspondence first,

Step 415, spot lock management device respond to the read lock application of Section Point backspace file 1 correspondence.

Step 416, Section Point send global buffer table synchronization request to first node.

Section Point comprises the information of the 4th data block of file 1 being carried out to the first node of up-to-date modification after obtaining read lock in this read lock, before first node, applied for writing lock, so up-to-date during global buffer table on first node.Section Point is initiated global buffer table synchronization request to first node.

Step 417, first node return to global buffer table sync response to Section Point.

In this global buffer table sync response, comprise up-to-date global buffer table, after Section Point gets up-to-date global buffer table, determine that the 4th data block first node revised, so initiate data synchronization request to first node.

Step 418, Section Point send the data synchronization request of the 4th data block to first node.

Step 419, first node return to the data sync response of the 4th data block to Section Point.

In the data sync response of the 4th data block, comprise the Update Table of first node to the 4th data block, Section Point obtains the Update Table of first node to the 4th data block, make two data on node synchronous, and Update Table is stored into the buffer memory of Section Point.

Step 420, Section Point read the data of the 4th data block from the buffer memory of Section Point.

In the present embodiment, first node and Section Point also may be write or get file 1 afterwards, here will not enumerate, for example Section Point also will read the 2nd data block, when reading, find that the 2nd data block revised by Section Point, so be up-to-date data certainly in the buffer memory of Section Point, from buffer memory, return.If find when Section Point reads the 5th data block, the 5th data block do not revised by any node, walked normally to read flow process: if having in buffer memory, from buffer memory, return, if do not have, from disk, read.

It should be noted that, in various embodiments of the present invention, can be concurrent for read access, Section Point has carried out write access, all the other nodes all carry out read access to it, if all the other node visits have arrived the data that Section Point is revised, all from the corresponding data of Section Point request.

The method that the present embodiment provides, by set up the global buffer table of file in cluster, this table is current along with lock mandate moves to will be on the node of file access, in the time of access, pass through inquiry global buffer table, to determine the data of access, whether by other nodes, revised, and determine whether to need synchrodata, by less expense, realize " as required " synchrodata, add after GCT, not be used in the latest data of each node cache file, data that buffer memory need to be used, the consumption that greatly reduces buffer memory is avoided to frequent read-write discs simultaneously, to reach, promote cluster file system performance.

The method that the present embodiment provides, in the time of can avoiding concurrent alternate access identical file between clustered node, write frequently moving without buffer memory that dish reads that dish causes, the method that the present embodiment provides has simultaneously realized the scheme of a kind of " as required " interaction data, internodal data interaction is transmitted as required, in most situations, between node, only mutual global buffer table is just passable, avoided unnecessary data transmission, the network overhead taking is few, data are coiled efficiency height much in internetwork transmission than once writing to coil → read, it is all effective writing alternately in the situation of different pieces of information buffer memory separately simultaneously, follow-up reading can directly be obtained from buffer memory, so intersect to revise identical data or different pieces of information no matter be, cluster file system performance is compared existing mode and is all had a distinct increment.

For convenience of description, two nodes of only take in various embodiments of the present invention describe as example, but the present invention can be applied in the cluster of n node (n>=2), the present invention does not limit the number of node, when a plurality of nodes alternately carry out write access to file destination simultaneously, the scheme that various embodiments of the present invention provide stands good.

Fig. 7 is the structural representation of first node of the present invention, comprising: lock application module 51, cache table acquisition module 52, file modification module 53.

Lock application module 51, for when client is carried out write access to the file destination on first node, to spot lock management device application file destination corresponding write lock, write lock and comprise the information of file destination being carried out to the Section Point of up-to-date modification;

Cache table acquisition module 52, for the up-to-date global buffer table corresponding according to the acquisition of information file destination of Section Point, preserves the up-to-date amendment record of Section Point to file destination in up-to-date global buffer table;

File modification module 53, for the up-to-date amendment record of file destination being modified to file destination according to Section Point, amended data are kept in the buffer memory of first node, and first node is updated to up-to-date global buffer table to the amendment record of file destination.

Lock application module 51 specifically for: to spot lock management device, send the lock application request of writing that file destination is corresponding, so that spot lock management device reclaims and writes lock to Section Point; What receive that spot lock management device sends writes lock authorization response.

File modification module 53 comprises judging unit, data capture unit and file modification unit; Judging unit, for determining whether to obtain the Update Table of Section Point to file destination according to Section Point to the up-to-date amendment record of file destination; If, acquiring unit obtains the Update Table of Section Point to file destination, the Update Table of the file destination getting is stored in the buffer memory of first node, the data of the file destination stored on first node and Section Point are consistent, file modification unit is reading out data from the buffer memory of first node, and file destination is modified; If not, file modification unit reading out data from the buffer memory of first node, modifies to file destination.

In the present embodiment, file destination comprises at least two data blocks, and Section Point comprises the up-to-date amendment record of file destination: Section Point carried out the information of the data block of up-to-date modification to file destination; Judging unit specifically for: according to Section Point, file destination was carried out the information of the data block of up-to-date modification, determined that first node carried out the data block of write access to file destination and whether data block that Section Point carried out up-to-date modification to file destination is same data block; If so, data capture unit obtains the Update Table of Section Point to file destination.

Cache table acquisition module 52 specifically obtains up-to-date global buffer table in the following manner: to Section Point, send global buffer table synchronization request message; Receive the global buffer table synchronous response message that Section Point sends, in global buffer table synchronous response message, comprise up-to-date global buffer table, in up-to-date global buffer table, preserve the up-to-date amendment record of Section Point to file destination.

Data capture unit is specifically for obtaining in the following manner Update Table: according to the up-to-date amendment record of Section Point to file destination in up-to-date global buffer table, to Section Point, send data synchronization request message; Receive the data sync response message that Section Point sends, in data sync response message, comprise the Update Table of Section Point to file destination.

The first node that the present embodiment provides, can be used for the technical scheme of manner of execution embodiment mono-and embodiment tetra-, and specific implementation and technique effect are similar, repeat no more here.

Fig. 8 is the structural representation of the embodiment bis-of first node of the present invention, as shown in Figure 8, the first node of the present embodiment, on the basis of the structure of first node shown in Fig. 7, can also comprise further: file read module 54, receiver module 55, sending module 56.

Wherein, lock application module 51 also for: when first node carries out read access to file destination, to read lock corresponding to spot lock management device application file destination, in read lock, comprise the information of file destination being carried out to the first node of up-to-date modification;

File read module 54, reads file destination for the buffer memory from first node.

After 53 pairs of file destinations of file modification module are modified, if receiver module 55, receive the global buffer table synchronization request message that Section Point sends, global buffer table synchronization request message is that Section Point sends to first node need to carry out read access or write access to file destination time, sending module 56, be used for according to global buffer table synchronization request message, to Section Point, return to global buffer table synchronous response message, in global buffer table synchronous response message, comprise the up-to-date global buffer table after renewal.

Receiver module 55 also for: receive the data synchronization request that Section Point sends, data synchronization request message is that Section Point sends the amendment record of file destination according to first node in the up-to-date global buffer table after upgrading; Sending module 56 also for: to Section Point, send data sync response, data sync response comprises the Update Table of first node to file destination, so that Section Point is kept at first node in the buffer memory of Section Point to the Update Table of file destination, when carrying out read access or write access to data, Section Point can from the buffer memory of Section Point, read file destination.

Fig. 9 is the structural representation of the embodiment mono-of spot lock management device of the present invention, and the spot lock management device of the present embodiment, comprising: receiver module 61, determination module 62, sending module 63.

Wherein, receiver module 61, the lock application request of writing corresponding to file destination sending for receiving first node, when writing lock application request and be client file destination being carried out to write access, first node sends to spot lock management device;

Determination module 62, for determine Section Point application cross file destination corresponding write lock, Section Point carried out up-to-date modification to file destination;

Sending module 63, for send the lock of writing that file destination is corresponding to Section Point, reclaim order, to Section Point reclaim file destination corresponding write lock, spot lock management device, when locking to corresponding the writing of Section Point recovery file destination, is not write dish and invalid to the data of the file destination in the buffer memory of Section Point;

Sending module 63 also for: when spot lock management device is being recovered to corresponding the writing after lock of file destination to Section Point, to first node, send and to write lock authorization response, write the information that lock comprises Section Point.

Receiver module 61 also for: receive read lock application request corresponding to file destination that first node sends; Determination module 62 also for: determine first node application cross file destination corresponding write lock, first node carried out up-to-date modification to file destination; Sending module 63 also for: to first node, send the lock of writing that file destination is corresponding and reclaim order, to first node reclaim file destination corresponding write lock, spot lock management device, when locking to corresponding the writing of first node recovery file destination, is not write dish and invalid to the data of the file destination in the buffer memory of first node; Sending module 63 also for: to first node, send read lock authorization response, read lock comprises the information of first node, so that first node reads file destination from the buffer memory of first node.

In the present embodiment, receiver module 61 also for: receive the read lock that file destination that Section Point sends is corresponding or write lock application request; Determination module 62 also for: determine first node application cross file destination corresponding write lock, first node carried out up-to-date modification to file destination; Sending module 63 also for: to first node, send the lock of writing that file destination is corresponding and reclaim order, to first node reclaim file destination corresponding write lock, spot lock management device, when locking to corresponding the writing of first node recovery file destination, is not write dish and invalid to the data of the file destination in the buffer memory of first node; Sending module 63 also for: to Section Point, send read lock or write lock authorization response, read lock or write the information that lock comprises first node, so that Section Point is according to the up-to-date global buffer table of the acquisition of information of first node.

The spot lock management device that the present embodiment provides, can be used for the technical scheme that manner of execution embodiment bis-provides, and specific implementation and technique effect are similar, repeat no more here.

Figure 10 is the structural representation of Section Point embodiment mono-of the present invention, and the Section Point that the present embodiment provides comprises: receiver module 71, sending module 72.

Wherein, receiver module 71, the global buffer table synchronization request message sending for receiving first node;

Sending module 72, be used for according to global buffer table synchronization request message, to first node, return to global buffer table synchronous response message, in global buffer table synchronous response message, comprise up-to-date global buffer table, in up-to-date global buffer table, preserve the up-to-date amendment record of Section Point to file destination.

Receiver module 71 also for: receive the data synchronization request message that first node sends, data synchronization request message is that first node sends the up-to-date amendment record of file destination according to Section Point in up-to-date global buffer table; Sending module 72 also for: according to data synchronization request message, to first node return data synchronous response message, in data sync response message, comprise the Update Table of Section Point to file destination.

The Section Point that the present embodiment provides, can be used for the technical scheme that manner of execution embodiment tri-provides, and specific implementation and technique effect are similar, repeat no more here.

Figure 11 is the structural representation of first node embodiment tri-of the present invention, as shown in figure 11, the first node 800 of the present embodiment comprises: processor 81, storer 82, transmitter 83 and receiver 84, storer 82, transmitter 83 is connected by bus with processor 81 with receiver 84, instruction is carried out in storer 82 storages, when first node 800 operation, between processor 81 and storer 82, communicate by letter, processor 81 is carried out instruction and is made first node 800 carry out the buffer memory synchronous method of distributed cluster file system provided by the invention, processor 81 is specifically for carrying out following operation:

When client is carried out write access to the file destination on first node, to spot lock management device application file destination corresponding write lock, write lock and comprise the information of file destination being carried out to the Section Point of up-to-date modification;

The up-to-date global buffer table corresponding according to the acquisition of information file destination of Section Point, preserves the up-to-date amendment record of Section Point to file destination in up-to-date global buffer table;

According to Section Point, the up-to-date amendment record of file destination is modified to file destination, amended data are kept in the buffer memory of first node, and first node is updated to up-to-date global buffer table to the amendment record of file destination.

Processor 81 to spot lock management device application file destination corresponding write lock, be specially: processor 81 sends to spot lock management device the lock application request of writing that file destination is corresponding by transmitter 83, so that spot lock management device reclaims and to write lock to Section Point, and by receiver 84, receive that spot lock management devices send write lock authorization response.

Processor 81 is modified to file destination to the up-to-date amendment record of file destination according to Section Point, is specially: according to Section Point, the up-to-date amendment record of file destination is determined whether to obtain the Update Table of Section Point to file destination; If, obtain the Update Table of Section Point to file destination, the Update Table of the file destination getting is stored in the buffer memory of first node, the data of the file destination stored on first node and Section Point are consistent, reading out data from the buffer memory of first node, modifies to file destination; If not, reading out data from the buffer memory of first node, modifies to file destination.

In a kind of possible implementation of the present invention, file destination comprises at least two data blocks, and Section Point comprises the up-to-date amendment record of file destination: Section Point carried out the information of the data block of up-to-date modification to file destination;

Processor 81 determines whether to obtain the Update Table of Section Point to file destination according to Section Point to the up-to-date amendment record of file destination, be specially: according to Section Point, file destination was carried out the information of the data block of up-to-date modification, and determined that first node carried out the data block of write access to file destination and whether data block that Section Point carried out up-to-date modification to file destination is same data block; If so, obtain the Update Table of Section Point to file destination.

First node, according to the up-to-date global buffer table of the acquisition of information of Section Point, is specially:

Transmitter 83 sends global buffer table synchronization request message to Section Point, then, receiver 84 receives the global buffer table synchronous response message that Section Point sends, in global buffer table synchronous response message, comprise up-to-date global buffer table, in up-to-date global buffer table, preserve the up-to-date amendment record of Section Point to file destination.

Processor 81 obtains the Update Table of Section Point to file destination according to Section Point to the up-to-date amendment record of file destination, is specially:

Processor 81, according to the up-to-date amendment record of Section Point to file destination in up-to-date global buffer table, sends data synchronization request message by transmitter 83 to Section Point; Then, receiver 84 receives the data sync response message that Section Point sends, and comprises the Update Table of Section Point to file destination in data sync response message.

In the present embodiment, when first node carries out write operation to file destination, amended data are kept in the buffer memory of first node, and after first node is updated to up-to-date global buffer table to the amendment record of file destination, processor 81 also for: when first node carries out read access to file destination, to read lock corresponding to spot lock management device application file destination, in read lock, comprise the information of file destination being carried out to the first node of up-to-date modification, after obtaining writing lock and authorizing, from the buffer memory of first node, read file destination.

In the present embodiment, first node file destination carries out write operation, amended data are kept in the buffer memory of first node, and after first node is updated to up-to-date global buffer table to the amendment record of file destination, processor 81 also for:

By receiver 84, receive the global buffer table synchronization request message that Section Point sends, global buffer table synchronization request message is that Section Point sends to first node need to carry out read access or write access to file destination time, then, processor 81 is according to global buffer table synchronization request message, by transmitter 83, to Section Point, return to global buffer table synchronous response message, in global buffer table synchronous response message, comprise the up-to-date global buffer table after renewal.

In the present embodiment, first node is according to global buffer table synchronization request message, after Section Point or the 3rd node return to global buffer table synchronous response message, also comprises:

Receiver 84 receives the data synchronization request that Section Point sends, and data synchronization request message is that Section Point sends the amendment record of file destination according to first node in the up-to-date global buffer table after upgrading; Transmitter 83 sends data sync response to Section Point, data sync response comprises the Update Table of first node to file destination, so that Section Point is kept at first node in the buffer memory of Section Point to the Update Table of file destination, when carrying out read access or write access to data, Section Point can from the buffer memory of Section Point, read file destination.

The first node that the present embodiment provides, can be used for the technical scheme that manner of execution embodiment mono-and embodiment tetra-provide, and specific implementation and technique effect are similar, repeat no more here.

Figure 12 is the structural representation of spot lock management device embodiment bis-of the present invention, as shown in figure 12, the spot lock management device 900 of the present embodiment comprises: processor 91, storer 92, transmitter 93 and receiver 94, storer 92, transmitter 93 and receiver 94 are connected by bus with processor 91, instruction is carried out in storer 92 storages, when spot lock management device 900 operation, between processor 91 and storer 92, communicate by letter, processor 91 execution instructions.

Wherein, receiver 94, the lock application request of writing corresponding to file destination sending for receiving first node, when writing lock application request and be client file destination being carried out to write access, first node sends to spot lock management device;

Processor 91, for determine Section Point application cross file destination corresponding write lock, file destination was carried out to up-to-date modification, spot lock management device sends to Section Point the lock of writing that file destination is corresponding and reclaims order, to Section Point reclaim file destination corresponding write lock, spot lock management device, when locking to corresponding the writing of Section Point recovery file destination, is not write dish and invalid to the data of the file destination in the buffer memory of Section Point;

Transmitter 93, for sending and write lock authorization response to first node, writes the information that lock comprises Section Point.

Transmitter 93 is after lock authorization response is write in first node transmission, and receiver 94 is also for receiving the read lock application request corresponding to file destination of first node transmission; Processor 91 also for determine first node application cross file destination corresponding write lock, file destination was carried out to up-to-date modification, spot lock management device sends to first node the lock of writing that file destination is corresponding and reclaims order, to first node reclaim file destination corresponding write lock, spot lock management device, when locking to corresponding the writing of first node recovery file destination, is not write dish and invalid to the data of the file destination in the buffer memory of first node; Transmitter 93 spot lock management devices send read lock authorization response to first node, and read lock comprises the information of first node, so that first node reads file destination from the buffer memory of first node.

Figure 13 is the structural representation of Section Point embodiment bis-of the present invention, as shown in figure 13, the Section Point 100 of the present embodiment comprises: processor 11, storer 12, transmitter 13 and receiver 14, storer 12, transmitter 13 and receiver 14 are connected by bus with processor 11, instruction is carried out in storer 12 storages, when Section Point 100 operation, between processor 11 and storer 12, communicate by letter, processor 11 execution instructions.

In this enforcement, receiver 14, the global buffer table synchronization request message sending for receiving first node;

Processor 11, be used for according to global buffer table synchronization request message, by transmitter 13, to first node, return to global buffer table synchronous response message, in global buffer table synchronous response message, comprise up-to-date global buffer table, in up-to-date global buffer table, preserve the up-to-date amendment record of Section Point to file destination.

After returning to global buffer table synchronous response message to first node at transmitter 13, receiver 14 also for: receive the data synchronization request message that first node sends, data synchronization request message is that first node sends the up-to-date amendment record of file destination according to Section Point in up-to-date global buffer table; Processor 11 according to data synchronization request message, to first node return data synchronous response message, comprises the Update Table of Section Point to file destination by transmitter 13 according to Section Point in data sync response message.

One of ordinary skill in the art will appreciate that: all or part of step that realizes above-mentioned each embodiment of the method can complete by the relevant hardware of programmed instruction.Aforesaid program can be stored in a computer read/write memory medium.This program, when carrying out, is carried out the step that comprises above-mentioned each embodiment of the method; And aforesaid storage medium comprises: various media that can be program code stored such as ROM, RAM, magnetic disc or CDs.

Finally it should be noted that: each embodiment, only in order to technical scheme of the present invention to be described, is not intended to limit above; Although the present invention is had been described in detail with reference to aforementioned each embodiment, those of ordinary skill in the art is to be understood that: its technical scheme that still can record aforementioned each embodiment is modified, or some or all of technical characterictic is wherein equal to replacement; And these modifications or replacement do not make the essence of appropriate technical solution depart from the scope of various embodiments of the present invention technical scheme.

Claims

1. a buffer memory synchronous method for distributed cluster file system, is characterized in that, comprising:

2. method according to claim 1, is characterized in that, described first node is modified to described file destination to the up-to-date amendment record of described file destination according to described Section Point, comprising:

3. method according to claim 2, it is characterized in that, described file destination comprises at least two data blocks, and described Section Point comprises the up-to-date amendment record of described file destination: described Section Point carried out the information of the data block of up-to-date modification to described file destination;

4. method according to claim 1, is characterized in that, described first node, according to the up-to-date global buffer table of the acquisition of information of described Section Point, comprising:

5. method according to claim 3, is characterized in that, described first node obtains the Update Table of described Section Point to described file destination according to described Section Point to the up-to-date amendment record of described file destination, comprising:

6. method according to claim 1, it is characterized in that, described first node carries out write operation to described file destination, amended data are kept in the buffer memory of described first node, and after described first node is updated to described up-to-date global buffer table to the amendment record of described file destination, also comprise:

7. method according to claim 1, it is characterized in that, described in described first node, file destination carries out write operation, amended data are kept in the buffer memory of described first node, and after described first node is updated to described up-to-date global buffer table to the amendment record of described file destination, also comprise:

8. method according to claim 7, is characterized in that, described first node is according to described global buffer table synchronization request message, after described Section Point or described the 3rd node return to global buffer table synchronous response message, also comprises:

9. method according to claim 1, is characterized in that, described first node to file destination described in spot lock management device application corresponding write lock, comprising:

10. a buffer memory synchronous method for distributed type assemblies target file system, is characterized in that, comprising:

11. methods according to claim 10, is characterized in that, described spot lock management device sends and writes lock authorization response to described first node, described in write the information that lock comprises described Section Point after, also comprise:

12. methods according to claim 10, is characterized in that, described spot lock management device sends and writes lock authorization response to described first node, described in write the information that lock comprises described Section Point after, also comprise:

The buffer memory synchronous method of 13. 1 kinds of distributed type assemblies target file system, is characterized in that, comprising:

14. methods according to claim 13, is characterized in that, described Section Point, according to described global buffer table synchronization request message, after described first node returns to global buffer table synchronous response message, also comprises:

15. 1 kinds of nodes, described node is first node, it is characterized in that, comprising:

16. nodes according to claim 15, is characterized in that, described file modification module comprises judging unit, data capture unit and file modification unit;

17. nodes according to claim 16, it is characterized in that, described file destination comprises at least two data blocks, and described Section Point comprises the up-to-date amendment record of described file destination: described Section Point carried out the information of the data block of up-to-date modification to described file destination;

Described judging unit specifically for:

18. nodes according to claim 15, is characterized in that, described cache table acquisition module specifically for:

19. nodes according to claim 17, is characterized in that, described data capture unit specifically for:

20. nodes according to claim 15, is characterized in that, also comprise: file read module;

21. nodes according to claim 15, is characterized in that, also comprise:

22. nodes according to claim 21, is characterized in that, described receiver module also for:

23. nodes according to claim 15, is characterized in that, described lock application module specifically for:

24. 1 kinds of spot lock management devices, is characterized in that, comprising:

25. spot lock management devices according to claim 24, is characterized in that, described receiver module also for:

26. spot lock management devices according to claim 24, is characterized in that, described receiver module also for:

27. 1 kinds of nodes, described node is Section Point, it is characterized in that, comprising:

28. nodes according to claim 27, is characterized in that, described receiver module also for: