EP0405859B1 - Method and apparatus for managing a shadow set of storage media - Google Patents

Method and apparatus for managing a shadow set of storage media Download PDF

Info

Publication number
EP0405859B1
EP0405859B1 EP90306856A EP90306856A EP0405859B1 EP 0405859 B1 EP0405859 B1 EP 0405859B1 EP 90306856 A EP90306856 A EP 90306856A EP 90306856 A EP90306856 A EP 90306856A EP 0405859 B1 EP0405859 B1 EP 0405859B1
Authority
EP
European Patent Office
Prior art keywords
data
log
storage medium
write
host
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Lifetime
Application number
EP90306856A
Other languages
German (de)
French (fr)
Other versions
EP0405859A3 (en
EP0405859A2 (en
Inventor
Scott H. Davis
William L. Goleman
David W. Thiel
Robert G. Bean
James A. Zahrobsky
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Digital Equipment Corp
Original Assignee
Digital Equipment Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Digital Equipment Corp filed Critical Digital Equipment Corp
Publication of EP0405859A2 publication Critical patent/EP0405859A2/en
Publication of EP0405859A3 publication Critical patent/EP0405859A3/en
Application granted granted Critical
Publication of EP0405859B1 publication Critical patent/EP0405859B1/en
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/16Error detection or correction of the data by redundancy in hardware
    • G06F11/20Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements
    • G06F11/2053Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where persistent mass storage functionality or persistent mass storage control functionality is redundant
    • G06F11/2056Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where persistent mass storage functionality or persistent mass storage control functionality is redundant by mirroring
    • G06F11/2082Data synchronisation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/16Error detection or correction of the data by redundancy in hardware
    • G06F11/20Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements
    • G06F11/2053Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where persistent mass storage functionality or persistent mass storage control functionality is redundant
    • G06F11/2056Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where persistent mass storage functionality or persistent mass storage control functionality is redundant by mirroring
    • G06F11/2064Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where persistent mass storage functionality or persistent mass storage control functionality is redundant by mirroring while ensuring consistency
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/16Error detection or correction of the data by redundancy in hardware
    • G06F11/20Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements
    • G06F11/2053Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where persistent mass storage functionality or persistent mass storage control functionality is redundant
    • G06F11/2056Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where persistent mass storage functionality or persistent mass storage control functionality is redundant by mirroring
    • G06F11/2071Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where persistent mass storage functionality or persistent mass storage control functionality is redundant by mirroring using a plurality of controllers

Definitions

  • This invention relates to a device for storing digital data.
  • the preferred embodiment is described in connection with a system for establishing and maintaining one or more duplicate or "shadow" copies of stored data to thereby improve the availability of the stored data.
  • a typical digital computer system includes one or more mass storage subsystems for storing data (which may include program instructions) to be processed.
  • data which may include program instructions
  • the data is actually stored on disks.
  • Disks are divided into a plurality of tracks, at selected radial distances from the center, and sectors, defining particular angular regions across each track, with each track and set of one or more sectors comprising a block, in which data is stored.
  • shadow set typically data that is stored in particular blocks on one member of the shadow set is the same as data stored in corresponding blocks on the other members of the shadow set. It is usually desirable to permit multiple host processors to simultaneously, access (i.e., in parallel) the shadow set for read and write type requests ("I/O" requests).
  • Shadow set members have data that is inconsistent if they have corresponding blocks whose data contents are different. For example, if one of the hosts malfunctions (e.g., fails), it may have had outstanding writes that completed to some shadow set members but not to others, resulting in data that is inconsistent.
  • a merge operation ensures that the data stored on corresponding blocks of the shadow set members are consistent but does not determine the integrity (i.e., accuracy) of the data stored in the blocks which were inconsistent.
  • the integrity of the data is verified by higher level techniques (e.g., by an applications program).
  • EP-0303856 there is described a technique for managing identical pairs of storage devices wherein data is written to records on a primary storage device. Changed records are then retrieved and duplicated in a secondary storage device as a result of the write operation to the primary storage device.
  • US-4686620 discloses a method of generating a back-up copy of a database system in which changes to "pages" in the database are denoted by a corresponding bit in a bit map. From the generation of one back-up copy to the next, only those pages which have changed, ie. whose bit in the bit map has changed, are copied.
  • the invention is defined in claims 1 and 10, and generally features a write history log that is used in the preferred embodiment to perform a management operation, such as a merge operation, on a shadow set.
  • the preferred embodiment features a method of managing a shadow set of storage media accessible by one or more data sources, comprising the steps of: A. maintaining a log of information relating to locations within the shadow set storage media to which data is written from at least one of the data sources, and B. carrying out a management operation by: a. accessing the log to determine information relating to one or more of the locations to which data has been written, and b. carrying out the management operation with respect to the one or more locations, based on the results of the step of accessing the log.
  • the information comprises the addresses of the locations to which the data was written and an identification of the source or sources from which data was written.
  • the management operation is carried out with respect to locations selected on the basis of the source from which data was written.
  • the information is stored in the log for data written to the shadow set over a predetermined period of time.
  • the information in the log relates to locations within the storage media to which data was written from a plurality of sources.
  • the preferred embodiment of the invention allows a system to determine which sections of a shadow set storage medium have been the subject of a write command without needing to access the source of the write command (e.g., the source may be unavailable or unknown). This information is used to substantially reduce the time required to perform a merge operation. Because the only sections of the shadow set members that are usually inconsistent are those that have been the subject of a write operation, the system does not waste time merging every block of each member of the shadow set to correct inconsistencies, but merges only those blocks where data has been recently written. The result is a significantly more efficient merge operation that allows minimal impact on I/O requests.
  • Fig. 1 is a system according to the present invention using a shadow set.
  • Figs. 2-6 illustrate data structures used with the invention.
  • Fig. 7 is a flow chart illustrating a method of merging two members of a shadow set according to a preferred embodiment of the invention.
  • a computer system including the invention includes a plurality of hosts 9, each of which includes a processor 10, memory 12 (including buffer storage) and a communications interface 14.
  • the hosts 9 are each directly connected through a communications medium 16 (e.g., by a virtual circuit) to two or more storage subsystems illustrated generally identified by reference numeral 17 (two are shown).
  • Each storage subsystem 17 includes a disk controller 18 that controls one or more disks 20, which form the members of the shadow set.
  • Disk controller 18 includes a buffer 22, a processor 24 and memory 26 (e.g., volatile memory).
  • Processor 24 receives I/O requests from hosts 9 and controls reads from and writes to disk 20.
  • Buffer 22 temporarily stores data received in connection with a write command before the data is written to a disk 20.
  • Buffer 22 also stores data read from a disk 20 before the data is transmitted to the host in response to a read command.
  • Processor 24 stores various types of information in memory 26, described more fully below.
  • Each host 9 will store, in its memory 12, a table that includes information about the system that the hosts 9 need to perform many operations. For example, hosts 9 will perform I/O operations to storage subsystems 17 and must know which storage subsystems are available for use, what disks are stored in the subsystems, etc. As will be described in greater detail below, the hosts 9 will slightly alter the procedure for I/O operations if a merge operation is being carried out in the system by a particular host 9. Therefore, the table will store status information regarding any ongoing merge operation (as well as other operations). The table also contains other standard information.
  • each storage subsystem may include multiple disks 20, the members of the shadow set are chosen to include disks in different storage subsystems 17. Therefore, a host may directly access each member of the shadow set, through its interface 14 and over communication medium 16, without requiring it to access two shadow set members through the same disk controller 18. This will avoid a "single point of failure" in the event of a failure of one of the disk controllers 18. In other words, if members of a shadow set have a common disk controller 18, and if that controller 18 malfunctions, the hosts will not be able to successfully perform any I/O operations. In the preferred system, the shadow set members are "distributed", so that the failure of one device (e.g., one disk controller 18) will not inhibit I/O operations because they can be performed using another shadow set member accessed through another disk controller.
  • a host 9 initiates a merge operation in which it makes data on two disks 20 comprising members of a shadow set consistent.
  • the host 9 initiates by means of write and read commands, read and write operations. Before proceeding further, it will be helpful to describe these commands in greater detail.
  • a host 9 When a host 9 wishes to write data to a disk 20, which may comprise a member of a shadow set, the host issues a command whose format is illustrated in Fig. 2A.
  • the command includes a "command reference number” field that uniquely identifies the command, and a "unit number” field that identifies the unit (e.g. the disk 20) to which data is to be written.
  • the host issues a separate write command to each disk 20 with the proper unit number that identifies the disk 20.
  • the "opcode" field identifies that the operation is a write.
  • the "byte count” field contains a value that identifies the total number of bytes comprising the data to be written and the "logical block number" identifies the starting storage location on the disk at which the data is to be written.
  • the "buffer descriptor” identifies the location in host memory 12 that contains the data to be written.
  • the "host reference number,” "entry locator” and “entry id” are used in connection with a “write log” feature, described in detail below.
  • a read command is illustrated in Fig. 2B, and includes fields that are similar to the write command fields.
  • the buffer descriptor contains the location in host memory 12 where the data read from the disk is to be stored.
  • the read command does not include the fields in the write command that are associated with the write log feature, namely, the host reference number field, entry locator field, and entry ID field (Fig. 2A).
  • a host transmits a read or write command, it is received by the disk controller 18 that serves the disk 20 identified in the "unit number" field.
  • the disk controller 18 will perform the write operation in connection with the identified disk 20 and return an "end message" to the originating host 9.
  • the format of the write command end message is illustrated in Fig. 3A.
  • the end message includes a number of fields, including a command reference number field whose contents correspond to the contents of the command reference number field of the write command which initiated storage operation, and status field that informs the host whether or not the command was completed successfully. If the disk 20 was unable to complete the write operation, the status field can include an error code identifying the nature of the failure.
  • the disk controller 18 In response to a read command, the disk controller 18 will read the requested data from its disk 20 and transmit the data to memory 12 of the originating host. After the data has been transmitted, an end message is generated by the disk controller and sent to the originating host, the format of the read command end message being illustrated in Fig. 3B.
  • the read command end message is similar to the end message for the write command, with the exception that the read command end message does not include an entry locator or entry ID field associated with the write history log feature, described below.
  • the disk controller 18 also maintains a write history log that includes a number of "write history entries" (Fig. 4), each of which stores information regarding a recent write operation.
  • each of the storage subsystems that form each shadow set member includes a processor 24 and an associated memory 26.
  • its disk controller 18 stores, in a write history log in its memory 26, information in a write history entry indicating the data blocks of the shadow set member to which data has been written.
  • the write history entry also stores information identifying the source of the write command message (e.g., the originating host) which initiated the write operation.
  • a host 9 managing the merge operation accesses the write history log for each shadow set member engaged in the merge operation and determines from the log entries which data blocks may be inconsistent. For example, if one of hosts 9 in Fig. 1 should fail while initiating write operation to members of the shadow set, one shadow set member may have completed the write operation but not another shadow set member, leaving the data on the shadow set inconsistent to the extent of the one write operation. Therefore, a merge operation may be performed by a properly functioning host, but it need only be performed for the data blocks that the failed host has recently enabled to be written because other data blocks will be consistent.
  • the host performing the merge operation will access the write history log associated with each member of the shadow set, determine which blocks have been written by the host that failed, and perform a merge on corresponding blocks in the shadow set. Since only the blocks that were written in the shadow set are merged, the operation is completed much more quickly than if a host managing a merge operation enabled the contents of an entire shadow set member to be copied to another shadow set member.
  • a write history log is created and maintained and then will describe how this information is utilized in a merge operation.
  • the disk controller prepares and stores a write history entry in its memory 26.
  • the format of a write history entry is shown in Fig. 4.
  • the "entry flags” indicate the state of a write history entry.
  • An “Allocated” flag is set if the write history entry is currently allocated (i.e., being used to store information about a write operation).
  • the contents of a "unit number” field identify the specific disk to which the write operation associated with the write history entry was addressed.
  • a "command identifier/status” field is used to identify and give the status of a current command (examples of the information stored in this field are described below).
  • the "starting logical block number” gives the address (position) on the disk volume at which the associated write operation begins and the "transfer length” specifies the number of bytes of data written to the member's disk. I.e., these fields specify what part of the shadow set member has been potentially modified.
  • the "host reference number” field identifies the host from which the write operation originated.
  • the "entry id" field contains a value assigned by a host to uniquely identify a write history entry.
  • the "entry locator" field contains a value assigned by the shadow set member that uniquely identifies the internal location of the write history entry, i.e., the location within memory 26.
  • a shadow set disk controller 18 When a shadow set disk controller 18 receives a write command from a host, the controller performs the following operations. First, the controller validates the command message fields and checks the state of the disk 20 to perform the write operation in a standard manner for the protocall being used. If the field validation or state checks fail, the shadow set member rejects the command issuing a write end message (Fig. 3A) whose status field contains the appropriate status.
  • the controller then checks a flag in the write command that indicates the contents of whether a new write history entry for the command is to be allocated or whether a previously allocated write history entry will be reused. If a new write history entry is to be allocated, the "Host Reference Number/entry locator" field will contain the Host reference number. The controller will search the set of write history entries in memory 26 for an entry that is not currently allocated -- i.e., a write history entry with a clear "allocated entry" flag.
  • the controller 18 completes the command and sends an end message to the host whose "status" field indicates that the write history log is invalid with respect to that host.
  • the controller will also invalidate all write history entries for the host that issued the write command in order to prevent another host from relying on these entries in a merge operation.
  • the entries can be invalidated by using a "history log" modifier contained in each entry that indicates whether that entry is valid or invalid.
  • the controller 18 performs the following operations in connection with the write history entry:
  • the "Host Reference number/ entry locator” field will contain the "entry locator” that defines the location of the write history entry to be used. This may occur if, for example, a host 9 is re-transmitting a previous write command message, which will have the same value in the "host reference number” field. To accomplish this, the controller 18 will first determine if the value contained in the "entry locator” field identifies one of the set of write history entries in its write log. If the contents of the "entry locator” field does not identify an entry in the set, the controller 18 rejects the command as an invalid command.
  • the disk controller 18 uses the value in the "entry locator” field as an index into the set of write history entries to find the write history entry to be reused. The controller then checks the setting of the allocate entry flag of the found write history entry. If that flag is clear, indicating that the entry was not actually already allocated, the controller 18 rejects the command with a status of Write History Entry Access Error.
  • the controller then checks the "command identifier/status" field of the identified write history entry to see if the entry is currently associated with an in progress command such as a write command that is being carried out -- i.e., the controller determines if an "encode” flag within the "opcode” field is clear. The encode flag is set when an operation begins and is cleared when the operation completes. If the entry is associated with an in progress command, the controller rejects the command and sends an end message to the host whose status field identifies a Write History Entry Access Error.
  • the controller performs the following operations:
  • the disk controller copies the "encode,” “flags,” and “status" end message fields into the appropriate fields of the write history entry associated with the command and then continues standard processing.
  • the disk controller sets the "host reference number,” "entry id,” and "entry locator” end message fields equal to the values contained in the corresponding fields of the write history entry associated with the command. Note that with one exception the requirement just described can also be met by copying those fields directly from the command message to the end message. The only exception is that when a new write history entry has been allocated, the controller must set the "entry locator” end message field equal to the value contained in the "entry locator" field of the associated write history entry.
  • the system utilizes a "Compare Host” operation in performing a merge of two shadow set members.
  • the command message format for the Compare Host operation is shown in Fig. 6A.
  • the Compare Host operation instructs the disk controller supporting the disk identified in the "unit number” field to compare the data stored in a section of host memory identified in the "buffer descriptor” field, to the data stored on the disk in the location identified by the "logical block number” and "byte count” fields.
  • the disk controller receiving the Compare Host command will execute the requested operation by reading the identified data from host memory, reading the data from the identified section of the disk, and comparing the data read from the host to the data read from the disk.
  • the disk controller then issues an end message, the format of which is shown in Fig. 6B, to the host that issued the Compare Host command.
  • the status field of the end message will indicate whether the compared data was found to be identical.
  • One of the system's hosts will control operations for merging two storage devices.
  • the system can select any host to execute the operation. For example, the host with the best transmission path (e.g., the shortest path) to the shadow set may be chosen.
  • a write History Management command i.e., the message sent by the host to the shadow set
  • Fig. 5A a Write History Management command
  • Fig. 5B illustrates the end message format (i.e., the message returned to the host by a shadow set member's disk controller).
  • the host selects a particular operation as the operations are needed during the merge, and specifies the operation in the "operation" field of the command message.
  • the other fields contain other information, explained more fully below as each operation is explained.
  • the "DEALLOCATE ALL” operation is used to deallocate all of the write history entries for the disk identified in the "unit number” field.
  • the DEALLOCATE ALL operation makes all of the write log spaces available for new entries. For example, after a merge operation is performed, the entries stored in the write history log are no longer needed because the members are assumed to be consistent immediately following a merge.
  • the DEALLOCATE ALL operation will deallocate (i.e. free up) all of the write history entries to make them available for new information as new writes are made to the shadow set.
  • a host can deallocate only those write log entries associated with a particular host using a "DEALLOCATE BY HOST REFERENCE NUMBER" operation. This may be desirable if a merge was performed as a result of a particular host having failed. Such a merge, as described more fully below, would involve merging only those blocks that were written with information from the failed host. Once that merge is completed, the write history entries associated with that host will no longer be needed and may be deallocated. All write history entries are not deallocated because if a different host fails, its entries would be needed to perform a similar merge. To execute this operation, each disk controller deallocates all of the write history entries that are associated with the host identified in the "host reference number" field for the disk identified in the "unit number” field.
  • a specific write history entry can be deallocated using a "DEALLOCATE BY ENTRY LOCATOR" operation.
  • the disk controller deallocates the specific write history entry that is located within the write log at the location specified in the "entry locator" field. If the contents of the entry locator field does not specify an entry within the limits of the write history log (i.e., if there is no write history entry at the identified location), the command is rejected and an end message is returned as an invalid command.
  • the command is rejected as an invalid command.
  • the controller rejects the command as an invalid command.
  • the "READ ALL" operation is used to read information from the write log of a shadow set member supporting the disk identified in the unit number field. If a host wishes to determine the total number of write history entries stored in the write log, the "count” field is set to zero, and the disk controller sets the command's end message "count” field (see Fig. 6B) equal to the number of write history entries that are associated with the identified unit.
  • the READ ALL operation is also used to read all of the write log entries from an identified shadow set member.
  • the "count" field is nonzero, and the disk controller transfers the number of the write history entries specified in the "count” field to the host memory 12 (specified in the "write history buffer descriptor" field) beginning with the first write history entry. Note that only those write history entries that are associated with the unit identified in the "unit number” field are included in the transfer.
  • the "READ BY HOST REFERENCE NUMBER” operation is used to read information from the write logs associated with a specific host. If the "count" field message is zero, the disk controller sets the command's end message "count” field equal to the number of write history entries that are associated with both the host identified in the "host reference number” field and the unit identified in the "unit” field. Therefore, the controller counts only those entries that resulted from writes to the identified unit from the identified host.
  • the controller transfers the number of write history entries specified in the "count” field to the location in host memory 12 specified in the "write history buffer descriptor" field, beginning with the first write history entry that is associated with both the host identified in the "host reference number” field and the unit identified in the "unit number” field.
  • the "unit alloc” field will contain the total number of write history entries currently allocated and associated with the unit identified in the command message "unit number” field.
  • the "server alloc” field will contain the total number of write history entries currently allocated across every disk served by the particular disk controller.
  • the “server unalloc” field contains the total number of write history entries currently available.
  • step 1 the host issues one of the Write History Management commands described above to each shadow set member in order to obtain information from the write logs stored in memory 26 of each disk controller 18.
  • the specific commands used will depend on the circumstances that created the need for the merge operation. For example, if a host fails, and a merge is performed by one of the properly functioning hosts, the host performing the merge will obtain, from the write log associated with each shadow set member, a complete list of all data blocks on that member disk to which the failed host had performed a write operation. As discussed above, the only data blocks that can possibly be inconsistent are those to which data was written by the failed host.
  • the host performing the merge will first issue a Write History Management command to perform a "Read Host Reference Number” operation to each disk controller that supports a shadow set member, with the "count” field set to zero.
  • the command will identify the failed host in the "host reference number” field.
  • each disk controller will receive its command and will send an end message to the host with the count field set to the total number of write history entries in its write log that were created due to writes from the failed host (step 2).
  • the host will then determine whether it needs to issue more Write History Management commands (step 3).
  • the host has received end messages specifying the number of write history entries contained in each write history log for the failed host.
  • the host controlling the merge will therefore need to issue another command to each shadow set member's disk controller that indicated it had write history entries for the failed host.
  • a disk controller that returned an end message with the "count" field set to zero has no write history entries for the failed host and the host does not send a second command to these disk controllers. (Note that in the rare case where every shadow set controller returns a valid end message with the "count" field set to zero, indicating that no shadow set member has been written with data from the failed host, no merge operation is necessary and the process terminates.)
  • a second Write History Management command is sent to each disk controller having needed write history entries (step 1), the command again specifying the "Read by Host Reference Number” operation and identifying the failed host, but this time setting the "count” field to the number of write history entries that the particular write history log has for the failed host.
  • each disk controller reads the write history entries from memory 26, sends them to the controlling host's memory 12, and issues an end message (step 2).
  • the host receives the end messages and will determine, in this case, that it does not need to issue another Read By Host Reference Number command since it will now have all of the needed write history information.
  • the host (step 4) proceeds to establish a Logical Cluster Number table in its memory 12.
  • the table contains numbers that identify the "clusters" (i.e., groups of data blocks) in the shadow set that are to be merged.
  • the controlling host sets up the table by listing logical cluster numbers that identify all of the clusters that are identified by the write history entries transmitted by the shadow set members.
  • the host prepares a new entry, each of which is identified by a logical cluster counter (which starts at 1 and increases sequentially).
  • the entry contains a number identifying the cluster to which the data had been written, and a disk controller ID that identifies the disk controller that sent the write history entry.
  • the host may discount logical cluster number table entries if they are the result of write history entries where corresponding entries were received from every disk controller supporting a shadow set member, because if a write has been performed in every shadow set member, the shadow set will not be inconsistent due to that write. In other words, since shadow set inconsistencies occur due to a write succeeding on some members but failing on other members, if a write history entry identifying a specific write command can be found in the write log associated with every shadow set member, then we know that the write operation succeeded on every member, and logical cluster numbers formed from these write history entries need not be used.
  • step 5 the host sets a logical cluster counter equal to the first logical cluster count number in the logical cluster number table.
  • the logical cluster counter is used to access a logical cluster number.
  • the first entry in the logical cluster number table is identified.
  • the host selects one of the members as a "source” and the other member as the “target” (step 6).
  • the host issues a read command to the disk controller serving the source to read the data stored in the section of the disk identified by the current logical cluster counter (step 7).
  • the read command issued by the host is described above and shown in Fig. 2B.
  • the "unit number” is set to describe the source disk 20 with the "Logical Block Number” and “Byte Count” being set according to the cluster currently identified by the logical cluster counter.
  • the source receives the read command and, after reading the identified data from the disk 20 to buffer 22, will transmit the data to the host memory 12 in the section identified by the "Buffer Descriptor" field in the read command.
  • the source will then transmit an end message of the type illustrated in Fig. 4B, which informs the host that the read operation has been performed (step 8).
  • the host After the host receives the end message (step 9), the host will issue a Compare Host command to the target controller to compare the data read from the source to the data in the corresponding cluster in the target to determine if the data is identical (step 10).
  • step 13 the host will check to see if the logical cluster counter identifies the last logical cluster number in the logical cluster number table (step 13). If the result of step 13 is "yes”, the merge operation is finished. Otherwise, the logical cluster counter is incremented (step 14) and the method returns to step 7 to process the next cluster.
  • the host will first establish cross-system synchronization by transmitting a message over communications medium 16 to all other hosts in the system (step 15). Each host will receive the transmitted message, will complete all outstanding I/O requests to the shadow set and will stall new I/O requests. Once this has been accomplished, each host sends an "acknowledge" message to the host controlling the merge indicating that it has complied with the synchronization request (step 16).
  • the host After receiving all of the acknowledge messages, the host will issue another read command to the source for the same cluster read in step 7, using an identical command message (step 17).
  • the data is read again because another host may have modified the data since it was read in step 7.
  • the source receives the read command message and executes it in the same manner as described in connection with step 8 above (step 18).
  • the host will once again receive the end message from the source as in step 9 above (step 19).
  • the host will now issue a write command to the target using the write command message format shown in Fig. 2A, and described above (step 20).
  • the write command will instruct the target to write the data read from the source in step 18, to the cluster in the target identified by the logical cluster counter.
  • the target receives the write command, executes it and sends an end message to the host (step 21).
  • These steps will result in the data in the two corresponding clusters in the source and target being consistent. Note that, because no I/O requests are being processed during these steps (i.e., the system is synchronized) there is no danger of the data in the source being modified after it was read in step 18, so the data written to the target's cluster in step 21 will be identical to the data now stored at the corresponding cluster in the source.
  • the host After the host receives the end message indicating the write has been completed (step 22), the host transmits a message to all other hosts to end the cross-system synchronization, the message informing all other hosts that normal I/O operations may resume (step 23). The host will then return to step 13 and continue as described above.
  • the host selects a shadow set member as the source and sequentially compares the data stored in each data block (a cluster at a time) to data in the corresponding data blocks in the target shadow set member. If the host finds an inconsistency, the system is temporarily synchronized while the host resolves the inconsistency by reading the inconsistent data from the source and writing the data to the corresponding cluster in the target. Because the shadow set members being merged normally have only a very small amount of data that is inconsistent, cross-system synchronization will be necessary for only a short time, resulting in only minimal disruption to normal I/O operations.
  • a host in the system needs to perform a read operation while another host is performing a merge as described above, the host must first ensure that the specific location to which the read is directed is consistent. To accomplish this, the host will effectively perform a merge operation on that specific section by first issuing the read to the source and then issuing a Compare Host command to the target to determine if the data is consistent. If it is consistent, then the host continues with its processing of the read data. If it is not consistent, then the host will synchronize the system, reread the data from the source and write the data to the target as described above.
  • each disk controller memory 26 Since there is limited space in each disk controller memory 26 for the write history log, the hosts will try to keep as many write history entries as possible available for use. Therefore, if a host issues a write command to each member of the shadow set, and receives end messages from each disk controller indicating that the write request was successfully performed, the host will reuse the write history entries that were allocated for the write requests sent to each disk controller. These entries may be reused because, if a write has succeeded to all members of the shadow set, then the shadow set will not be inconsistent due to that write. As discussed above, a host that performs a merge operation will deallocate those entries that were used in the merge operation once the operation is completed.
  • the illustrative embodiment describes a merge operation performed on a single target, but several targets may be used to thereby merge three or more storage media. Similarly, the system need not use the same member disk as the source throughout the merge operation, but may use other disks as the source.
  • the write history log feature has been described above in connection with a merge operation in a shadow set, and specifically is used to improve the efficiency of a merge operation by determining which data blocks may contain inconsistent data (i.e., data blocks that have not been written with data should not contain inconsistent data and need not be merged).
  • inconsistent data i.e., data blocks that have not been written with data should not contain inconsistent data and need not be merged.
  • Fig. 1 While the system of Fig. 1 is the preferred embodiment, the invention can be used with other shadowing systems such as ones that use a "central controller.” In these systems, the hosts access the shadow set members through a central controller.

Abstract

The invention generally features a write history log that is used in the preferred embodiment to perform a management operation, such as a merge operation, on a shadow set. The preferred embodiment features a method of managing a shadow set of storage media accessible by one or more data sources, comprising the steps of: A. maintaining a log of information relating to locations within the shadow set storage media to which data is written from at least one of the data sources, and B. carrying out a management operation by: a. accessing the log to determine information relating to one or more of the locations to which data has been written, and b. carrying out the management operation with respect to the one or more locations, based on the results of the step of accessing the log. In the preferred embodiment, the information comprises the addresses of the locations to which the data was written and an identification of the source or sources from which data was written. The management operation is carried out with respect to locations selected on the basis of the source from which data was written. The information is stored in the log for data written to the shadow set over a predetermined period of time. The information in the log relates to locations within the storage media to which data was written from a plurality of sources.

Description

  • This invention relates to a device for storing digital data. The preferred embodiment is described in connection with a system for establishing and maintaining one or more duplicate or "shadow" copies of stored data to thereby improve the availability of the stored data.
  • A typical digital computer system includes one or more mass storage subsystems for storing data (which may include program instructions) to be processed. In typical mass storage subsystems, the data is actually stored on disks. Disks are divided into a plurality of tracks, at selected radial distances from the center, and sectors, defining particular angular regions across each track, with each track and set of one or more sectors comprising a block, in which data is stored.
  • Since stored data may be unintentionally corrupted or destroyed, systems have been developed that create multiple copies of stored data, usually on separate storage devices, so that if the data on one of the devices or disks is damaged, it can be recovered from one or more of the remaining copies. Such multiple copies are known as the shadow set. In a shadow set, typically data that is stored in particular blocks on one member of the shadow set is the same as data stored in corresponding blocks on the other members of the shadow set. It is usually desirable to permit multiple host processors to simultaneously, access (i.e., in parallel) the shadow set for read and write type requests ("I/O" requests).
  • It is sometimes necessary to "merge" two (or more) storage devices to reassemble a complete shadow set, where the devices were previously members of the same shadow set, but currently contain data that is valid, although possibly inconsistent. Data in a particular block is valid if it is not erroneous, that is, if it is correct, as determined by an error correction technique, or, if it is incorrect but correctable with use of the error correction technique. Shadow set members have data that is inconsistent if they have corresponding blocks whose data contents are different. For example, if one of the hosts malfunctions (e.g., fails), it may have had outstanding writes that completed to some shadow set members but not to others, resulting in data that is inconsistent. A merge operation ensures that the data stored on corresponding blocks of the shadow set members are consistent but does not determine the integrity (i.e., accuracy) of the data stored in the blocks which were inconsistent. The integrity of the data is verified by higher level techniques (e.g., by an applications program).
  • In EP-0303856 there is described a technique for managing identical pairs of storage devices wherein data is written to records on a primary storage device. Changed records are then retrieved and duplicated in a secondary storage device as a result of the write operation to the primary storage device.
  • In US-4686620 discloses a method of generating a back-up copy of a database system in which changes to "pages" in the database are denoted by a corresponding bit in a bit map. From the generation of one back-up copy to the next, only those pages which have changed, ie. whose bit in the bit map has changed, are copied.
  • SUMMARY OF THE INVENTION
  • The invention is defined in claims 1 and 10, and generally features a write history log that is used in the preferred embodiment to perform a management operation, such as a merge operation, on a shadow set. The preferred embodiment features a method of managing a shadow set of storage media accessible by one or more data sources, comprising the steps of: A. maintaining a log of information relating to locations within the shadow set storage media to which data is written from at least one of the data sources, and B. carrying out a management operation by: a. accessing the log to determine information relating to one or more of the locations to which data has been written, and b. carrying out the management operation with respect to the one or more locations, based on the results of the step of accessing the log. In the preferred embodiment, the information comprises the addresses of the locations to which the data was written and an identification of the source or sources from which data was written. The management operation is carried out with respect to locations selected on the basis of the source from which data was written. The information is stored in the log for data written to the shadow set over a predetermined period of time. The information in the log relates to locations within the storage media to which data was written from a plurality of sources.
  • The preferred embodiment of the invention allows a system to determine which sections of a shadow set storage medium have been the subject of a write command without needing to access the source of the write command (e.g., the source may be unavailable or unknown). This information is used to substantially reduce the time required to perform a merge operation. Because the only sections of the shadow set members that are usually inconsistent are those that have been the subject of a write operation, the system does not waste time merging every block of each member of the shadow set to correct inconsistencies, but merges only those blocks where data has been recently written. The result is a significantly more efficient merge operation that allows minimal impact on I/O requests.
  • Other advantages and features of the invention will be apparent from the following detailed description of the invention and the appended claims.
  • DESCRIPTION OF THE PREFERRED EMBODIMENTS Drawings
  • We first briefly describe the drawings.
  • Fig. 1 is a system according to the present invention using a shadow set.
  • Figs. 2-6 illustrate data structures used with the invention.
  • Fig. 7 is a flow chart illustrating a method of merging two members of a shadow set according to a preferred embodiment of the invention.
  • Structure and Operation
  • Referring to Fig. 1, a computer system including the invention includes a plurality of hosts 9, each of which includes a processor 10, memory 12 (including buffer storage) and a communications interface 14. The hosts 9 are each directly connected through a communications medium 16 (e.g., by a virtual circuit) to two or more storage subsystems illustrated generally identified by reference numeral 17 (two are shown).
  • Each storage subsystem 17 includes a disk controller 18 that controls one or more disks 20, which form the members of the shadow set. Disk controller 18 includes a buffer 22, a processor 24 and memory 26 (e.g., volatile memory). Processor 24 receives I/O requests from hosts 9 and controls reads from and writes to disk 20. Buffer 22 temporarily stores data received in connection with a write command before the data is written to a disk 20. Buffer 22 also stores data read from a disk 20 before the data is transmitted to the host in response to a read command. Processor 24 stores various types of information in memory 26, described more fully below.
  • Each host 9 will store, in its memory 12, a table that includes information about the system that the hosts 9 need to perform many operations. For example, hosts 9 will perform I/O operations to storage subsystems 17 and must know which storage subsystems are available for use, what disks are stored in the subsystems, etc. As will be described in greater detail below, the hosts 9 will slightly alter the procedure for I/O operations if a merge operation is being carried out in the system by a particular host 9. Therefore, the table will store status information regarding any ongoing merge operation (as well as other operations). The table also contains other standard information.
  • While each storage subsystem may include multiple disks 20, the members of the shadow set are chosen to include disks in different storage subsystems 17. Therefore, a host may directly access each member of the shadow set, through its interface 14 and over communication medium 16, without requiring it to access two shadow set members through the same disk controller 18. This will avoid a "single point of failure" in the event of a failure of one of the disk controllers 18. In other words, if members of a shadow set have a common disk controller 18, and if that controller 18 malfunctions, the hosts will not be able to successfully perform any I/O operations. In the preferred system, the shadow set members are "distributed", so that the failure of one device (e.g., one disk controller 18) will not inhibit I/O operations because they can be performed using another shadow set member accessed through another disk controller.
  • In some cases a host 9 initiates a merge operation in which it makes data on two disks 20 comprising members of a shadow set consistent. In a merge operation, the host 9 initiates by means of write and read commands, read and write operations. Before proceeding further, it will be helpful to describe these commands in greater detail.
  • When a host 9 wishes to write data to a disk 20, which may comprise a member of a shadow set, the host issues a command whose format is illustrated in Fig. 2A. The command includes a "command reference number" field that uniquely identifies the command, and a "unit number" field that identifies the unit (e.g. the disk 20) to which data is to be written. To accomplish write operations for each disk 20 comprising a member of the shadow set, the host issues a separate write command to each disk 20 with the proper unit number that identifies the disk 20. The "opcode" field identifies that the operation is a write. The "byte count" field contains a value that identifies the total number of bytes comprising the data to be written and the "logical block number" identifies the starting storage location on the disk at which the data is to be written. The "buffer descriptor" identifies the location in host memory 12 that contains the data to be written.
  • The "host reference number," "entry locator" and "entry id" are used in connection with a "write log" feature, described in detail below.
  • The format of a read command is illustrated in Fig. 2B, and includes fields that are similar to the write command fields. For a read command, the buffer descriptor contains the location in host memory 12 where the data read from the disk is to be stored. The read command does not include the fields in the write command that are associated with the write log feature, namely, the host reference number field, entry locator field, and entry ID field (Fig. 2A).
  • Once a host transmits a read or write command, it is received by the disk controller 18 that serves the disk 20 identified in the "unit number" field. For a write command, the disk controller 18 will perform the write operation in connection with the identified disk 20 and return an "end message" to the originating host 9. The format of the write command end message is illustrated in Fig. 3A. The end message includes a number of fields, including a command reference number field whose contents correspond to the contents of the command reference number field of the write command which initiated storage operation, and status field that informs the host whether or not the command was completed successfully. If the disk 20 was unable to complete the write operation, the status field can include an error code identifying the nature of the failure.
  • In response to a read command, the disk controller 18 will read the requested data from its disk 20 and transmit the data to memory 12 of the originating host. After the data has been transmitted, an end message is generated by the disk controller and sent to the originating host, the format of the read command end message being illustrated in Fig. 3B. The read command end message is similar to the end message for the write command, with the exception that the read command end message does not include an entry locator or entry ID field associated with the write history log feature, described below.
  • The disk controller 18 also maintains a write history log that includes a number of "write history entries" (Fig. 4), each of which stores information regarding a recent write operation. As described above, each of the storage subsystems that form each shadow set member includes a processor 24 and an associated memory 26. When a write operation is performed to a shadow set member, its disk controller 18 stores, in a write history log in its memory 26, information in a write history entry indicating the data blocks of the shadow set member to which data has been written. The write history entry also stores information identifying the source of the write command message (e.g., the originating host) which initiated the write operation.
  • Thereafter, if a merge operation becomes necessary, a host 9 managing the merge operation accesses the write history log for each shadow set member engaged in the merge operation and determines from the log entries which data blocks may be inconsistent. For example, if one of hosts 9 in Fig. 1 should fail while initiating write operation to members of the shadow set, one shadow set member may have completed the write operation but not another shadow set member, leaving the data on the shadow set inconsistent to the extent of the one write operation. Therefore, a merge operation may be performed by a properly functioning host, but it need only be performed for the data blocks that the failed host has recently enabled to be written because other data blocks will be consistent. Therefore, the host performing the merge operation will access the write history log associated with each member of the shadow set, determine which blocks have been written by the host that failed, and perform a merge on corresponding blocks in the shadow set. Since only the blocks that were written in the shadow set are merged, the operation is completed much more quickly than if a host managing a merge operation enabled the contents of an entire shadow set member to be copied to another shadow set member.
  • We will first describe how the write history log is created and maintained and then will describe how this information is utilized in a merge operation. When a write command is received by one of the disk controllers 18, the disk controller prepares and stores a write history entry in its memory 26. The format of a write history entry is shown in Fig. 4. The "entry flags" indicate the state of a write history entry. An "Allocated" flag is set if the write history entry is currently allocated (i.e., being used to store information about a write operation). The contents of a "unit number" field identify the specific disk to which the write operation associated with the write history entry was addressed. A "command identifier/status" field is used to identify and give the status of a current command (examples of the information stored in this field are described below).
  • The "starting logical block number" gives the address (position) on the disk volume at which the associated write operation begins and the "transfer length" specifies the number of bytes of data written to the member's disk. I.e., these fields specify what part of the shadow set member has been potentially modified. The "host reference number" field identifies the host from which the write operation originated.
  • The "entry id" field contains a value assigned by a host to uniquely identify a write history entry.
  • The "entry locator" field contains a value assigned by the shadow set member that uniquely identifies the internal location of the write history entry, i.e., the location within memory 26.
  • When a shadow set disk controller 18 receives a write command from a host, the controller performs the following operations. First, the controller validates the command message fields and checks the state of the disk 20 to perform the write operation in a standard manner for the protocall being used. If the field validation or state checks fail, the shadow set member rejects the command issuing a write end message (Fig. 3A) whose status field contains the appropriate status.
  • The controller then checks a flag in the write command that indicates the contents of whether a new write history entry for the command is to be allocated or whether a previously allocated write history entry will be reused. If a new write history entry is to be allocated, the "Host Reference Number/entry locator" field will contain the Host reference number. The controller will search the set of write history entries in memory 26 for an entry that is not currently allocated -- i.e., a write history entry with a clear "allocated entry" flag.
  • If an unallocated write history entry cannot be found, the controller 18 completes the command and sends an end message to the host whose "status" field indicates that the write history log is invalid with respect to that host. The controller will also invalidate all write history entries for the host that issued the write command in order to prevent another host from relying on these entries in a merge operation. The entries can be invalidated by using a "history log" modifier contained in each entry that indicates whether that entry is valid or invalid.
  • If an unallocated write history entry is found, the controller 18 performs the following operations in connection with the write history entry:
    • a. Sets the "allocated entry flag".
    • b. Copies the contents of the command message's "unit number" field to the entry's "unit number" field.
    • c. Copies the contents of the command message's "opcode" and "modifiers" fields to the entry's "command id//status" field.
    • d. Copies the contents of the command message's "byte count" (or "logical block count") field to the entry's "transfer length" field.
    • e. Copies the contents of the command message's "logical block number" (or "destination lbn") field to the entry's "starting logical block number" field.
    • f. Copies the contents of the command message 's"host reference number" field to the entry's "host reference number" field.
    • g. Copies the contents of the command message's "entry id" field to the entry's "entry id" field.
    • h. Continues normal processing of the command.
  • If a previously used write history entry is to be reused, then the "Host Reference number/ entry locator" field will contain the "entry locator" that defines the location of the write history entry to be used. This may occur if, for example, a host 9 is re-transmitting a previous write command message, which will have the same value in the "host reference number" field. To accomplish this, the controller 18 will first determine if the value contained in the "entry locator" field identifies one of the set of write history entries in its write log. If the contents of the "entry locator" field does not identify an entry in the set, the controller 18 rejects the command as an invalid command. If the contents of the "entry locator" identifies one of the entries in the set of write history entries, the disk controller 18 uses the value in the "entry locator" field as an index into the set of write history entries to find the write history entry to be reused. The controller then checks the setting of the allocate entry flag of the found write history entry. If that flag is clear, indicating that the entry was not actually already allocated, the controller 18 rejects the command with a status of Write History Entry Access Error.
  • The controller then checks the "command identifier/status" field of the identified write history entry to see if the entry is currently associated with an in progress command such as a write command that is being carried out -- i.e., the controller determines if an "encode" flag within the "opcode" field is clear. The encode flag is set when an operation begins and is cleared when the operation completes. If the entry is associated with an in progress command, the controller rejects the command and sends an end message to the host whose status field identifies a Write History Entry Access Error.
  • Finally, if the entry is not associated with an in progress command, the controller performs the following operations:
    • a. Copies the contents of the command message's "unit number" field to the entry's "unit number" field.
    • b. Copies the contents of the command message's "opcode" and "modifiers" fields to the entry's "command id//status' field.
    • c. Copies the contents of the command message's "byte count" (or "logical block count") field to the entry's "transfer length" field.
    • d. Copies the contents of the command message's "logical block number" (or "destination lbn") field to the entry's "starting logical block number" field.
    • e. Continues normal processing of the command.
  • After a write command has aborted, terminated, or completed, the disk controller copies the "encode," "flags," and "status" end message fields into the appropriate fields of the write history entry associated with the command and then continues standard processing.
  • In addition, prior to returning the end message of a write command, the disk controller sets the "host reference number," "entry id," and "entry locator" end message fields equal to the values contained in the corresponding fields of the write history entry associated with the command. Note that with one exception the requirement just described can also be met by copying those fields directly from the command message to the end message. The only exception is that when a new write history entry has been allocated, the controller must set the "entry locator" end message field equal to the value contained in the "entry locator" field of the associated write history entry.
  • As will be explained below, the system utilizes a "Compare Host" operation in performing a merge of two shadow set members. The command message format for the Compare Host operation is shown in Fig. 6A. The Compare Host operation instructs the disk controller supporting the disk identified in the "unit number" field to compare the data stored in a section of host memory identified in the "buffer descriptor" field, to the data stored on the disk in the location identified by the "logical block number" and "byte count" fields.
  • The disk controller receiving the Compare Host command will execute the requested operation by reading the identified data from host memory, reading the data from the identified section of the disk, and comparing the data read from the host to the data read from the disk. The disk controller then issues an end message, the format of which is shown in Fig. 6B, to the host that issued the Compare Host command. The status field of the end message will indicate whether the compared data was found to be identical.
  • One of the system's hosts will control operations for merging two storage devices. The system can select any host to execute the operation. For example, the host with the best transmission path (e.g., the shortest path) to the shadow set may be chosen.
  • During a merge operation, the host controlling the merge will utilize a "Write History Management" command to perform a number of operations. The command message format (i.e., the message sent by the host to the shadow set) of a Write History Management command is shown in Fig. 5A. Fig. 5B illustrates the end message format (i.e., the message returned to the host by a shadow set member's disk controller). The host selects a particular operation as the operations are needed during the merge, and specifies the operation in the "operation" field of the command message. The other fields contain other information, explained more fully below as each operation is explained.
  • The "DEALLOCATE ALL" operation is used to deallocate all of the write history entries for the disk identified in the "unit number" field. The DEALLOCATE ALL operation makes all of the write log spaces available for new entries. For example, after a merge operation is performed, the entries stored in the write history log are no longer needed because the members are assumed to be consistent immediately following a merge. The DEALLOCATE ALL operation will deallocate (i.e. free up) all of the write history entries to make them available for new information as new writes are made to the shadow set.
  • A host can deallocate only those write log entries associated with a particular host using a "DEALLOCATE BY HOST REFERENCE NUMBER" operation. This may be desirable if a merge was performed as a result of a particular host having failed. Such a merge, as described more fully below, would involve merging only those blocks that were written with information from the failed host. Once that merge is completed, the write history entries associated with that host will no longer be needed and may be deallocated. All write history entries are not deallocated because if a different host fails, its entries would be needed to perform a similar merge. To execute this operation, each disk controller deallocates all of the write history entries that are associated with the host identified in the "host reference number" field for the disk identified in the "unit number" field.
  • A specific write history entry can be deallocated using a "DEALLOCATE BY ENTRY LOCATOR" operation. The disk controller deallocates the specific write history entry that is located within the write log at the location specified in the "entry locator" field. If the contents of the entry locator field does not specify an entry within the limits of the write history log (i.e., if there is no write history entry at the identified location), the command is rejected and an end message is returned as an invalid command. If the value contained in the "host reference number" field of the write history entry identified by the entry locator does not equal the value contained in the command message "host reference number" field (i.e., if the host identified in the command is also not the same as the host identified in the write history entry), the command is rejected as an invalid command. Similarly, if the value contained in the "entry id" field of the write history entry located via the "entry locator" does not equal the value contained in the command message "entry id" field, the controller rejects the command as an invalid command.
  • The "READ ALL" operation is used to read information from the write log of a shadow set member supporting the disk identified in the unit number field. If a host wishes to determine the total number of write history entries stored in the write log, the "count" field is set to zero, and the disk controller sets the command's end message "count" field (see Fig. 6B) equal to the number of write history entries that are associated with the identified unit.
  • The READ ALL operation is also used to read all of the write log entries from an identified shadow set member. In this case, the "count" field is nonzero, and the disk controller transfers the number of the write history entries specified in the "count" field to the host memory 12 (specified in the "write history buffer descriptor" field) beginning with the first write history entry. Note that only those write history entries that are associated with the unit identified in the "unit number" field are included in the transfer.
  • The "READ BY HOST REFERENCE NUMBER" operation is used to read information from the write logs associated with a specific host. If the "count" field message is zero, the disk controller sets the command's end message "count" field equal to the number of write history entries that are associated with both the host identified in the "host reference number" field and the unit identified in the "unit" field. Therefore, the controller counts only those entries that resulted from writes to the identified unit from the identified host.
  • If the "count" field in a "READ BY HOST REFERENCE NUMBER" operation is nonzero, the controller transfers the number of write history entries specified in the "count" field to the location in host memory 12 specified in the "write history buffer descriptor" field, beginning with the first write history entry that is associated with both the host identified in the "host reference number" field and the unit identified in the "unit number" field.
  • As noted above, the end message format for the Write History Management command is shown in Fig. 5B. The "unit alloc" field will contain the total number of write history entries currently allocated and associated with the unit identified in the command message "unit number" field. The "server alloc" field will contain the total number of write history entries currently allocated across every disk served by the particular disk controller. The "server unalloc" field contains the total number of write history entries currently available.
  • Now that the Write History Management command, and its associated operations, have been described in detail, a merge operation utilizing the write log feature will be described with reference to the flowchart of Figs. 7A-C.
  • In step 1, the host issues one of the Write History Management commands described above to each shadow set member in order to obtain information from the write logs stored in memory 26 of each disk controller 18. The specific commands used will depend on the circumstances that created the need for the merge operation. For example, if a host fails, and a merge is performed by one of the properly functioning hosts, the host performing the merge will obtain, from the write log associated with each shadow set member, a complete list of all data blocks on that member disk to which the failed host had performed a write operation. As discussed above, the only data blocks that can possibly be inconsistent are those to which data was written by the failed host.
  • To obtain this information, the host performing the merge will first issue a Write History Management command to perform a "Read Host Reference Number" operation to each disk controller that supports a shadow set member, with the "count" field set to zero. The command will identify the failed host in the "host reference number" field. As described above, each disk controller will receive its command and will send an end message to the host with the count field set to the total number of write history entries in its write log that were created due to writes from the failed host (step 2).
  • The host will then determine whether it needs to issue more Write History Management commands (step 3). In this example, the host has received end messages specifying the number of write history entries contained in each write history log for the failed host. The host controlling the merge will therefore need to issue another command to each shadow set member's disk controller that indicated it had write history entries for the failed host. A disk controller that returned an end message with the "count" field set to zero has no write history entries for the failed host and the host does not send a second command to these disk controllers. (Note that in the rare case where every shadow set controller returns a valid end message with the "count" field set to zero, indicating that no shadow set member has been written with data from the failed host, no merge operation is necessary and the process terminates.)
  • Therefore, a second Write History Management command is sent to each disk controller having needed write history entries (step 1), the command again specifying the "Read by Host Reference Number" operation and identifying the failed host, but this time setting the "count" field to the number of write history entries that the particular write history log has for the failed host. This time each disk controller reads the write history entries from memory 26, sends them to the controlling host's memory 12, and issues an end message (step 2). The host receives the end messages and will determine, in this case, that it does not need to issue another Read By Host Reference Number command since it will now have all of the needed write history information.
  • The host (step 4) proceeds to establish a Logical Cluster Number table in its memory 12. The table contains numbers that identify the "clusters" (i.e., groups of data blocks) in the shadow set that are to be merged.
  • The controlling host sets up the table by listing logical cluster numbers that identify all of the clusters that are identified by the write history entries transmitted by the shadow set members.
  • As each write history entry is received, the host prepares a new entry, each of which is identified by a logical cluster counter (which starts at 1 and increases sequentially). The entry contains a number identifying the cluster to which the data had been written, and a disk controller ID that identifies the disk controller that sent the write history entry. By sequentially going through the logical cluster number table, the host will be able to identify every cluster to which data has been written by the failed host.
  • The host may discount logical cluster number table entries if they are the result of write history entries where corresponding entries were received from every disk controller supporting a shadow set member, because if a write has been performed in every shadow set member, the shadow set will not be inconsistent due to that write. In other words, since shadow set inconsistencies occur due to a write succeeding on some members but failing on other members, if a write history entry identifying a specific write command can be found in the write log associated with every shadow set member, then we know that the write operation succeeded on every member, and logical cluster numbers formed from these write history entries need not be used.
  • Next, in step 5, the host sets a logical cluster counter equal to the first logical cluster count number in the logical cluster number table. The logical cluster counter is used to access a logical cluster number. When initialized to one in step 5, the first entry in the logical cluster number table is identified.
  • The host then selects one of the members as a "source" and the other member as the "target" (step 6). The host issues a read command to the disk controller serving the source to read the data stored in the section of the disk identified by the current logical cluster counter (step 7). The read command issued by the host is described above and shown in Fig. 2B. The "unit number" is set to describe the source disk 20 with the "Logical Block Number" and "Byte Count" being set according to the cluster currently identified by the logical cluster counter.
  • The source receives the read command and, after reading the identified data from the disk 20 to buffer 22, will transmit the data to the host memory 12 in the section identified by the "Buffer Descriptor" field in the read command. The source will then transmit an end message of the type illustrated in Fig. 4B, which informs the host that the read operation has been performed (step 8).
  • After the host receives the end message (step 9), the host will issue a Compare Host command to the target controller to compare the data read from the source to the data in the corresponding cluster in the target to determine if the data is identical (step 10).
  • If a "yes" result is obtained, the host will check to see if the logical cluster counter identifies the last logical cluster number in the logical cluster number table (step 13). If the result of step 13 is "yes", the merge operation is finished. Otherwise, the logical cluster counter is incremented (step 14) and the method returns to step 7 to process the next cluster.
  • If a no result was returned, indicating that the data read from corresponding clusters on the two merge members are not identical, the host will implement the following steps to make the data consistent.
  • The host will first establish cross-system synchronization by transmitting a message over communications medium 16 to all other hosts in the system (step 15). Each host will receive the transmitted message, will complete all outstanding I/O requests to the shadow set and will stall new I/O requests. Once this has been accomplished, each host sends an "acknowledge" message to the host controlling the merge indicating that it has complied with the synchronization request (step 16).
  • After receiving all of the acknowledge messages, the host will issue another read command to the source for the same cluster read in step 7, using an identical command message (step 17). The data is read again because another host may have modified the data since it was read in step 7. The source receives the read command message and executes it in the same manner as described in connection with step 8 above (step 18). The host will once again receive the end message from the source as in step 9 above (step 19).
  • The host will now issue a write command to the target using the write command message format shown in Fig. 2A, and described above (step 20). The write command will instruct the target to write the data read from the source in step 18, to the cluster in the target identified by the logical cluster counter. The target receives the write command, executes it and sends an end message to the host (step 21). These steps will result in the data in the two corresponding clusters in the source and target being consistent. Note that, because no I/O requests are being processed during these steps (i.e., the system is synchronized) there is no danger of the data in the source being modified after it was read in step 18, so the data written to the target's cluster in step 21 will be identical to the data now stored at the corresponding cluster in the source.
  • After the host receives the end message indicating the write has been completed (step 22), the host transmits a message to all other hosts to end the cross-system synchronization, the message informing all other hosts that normal I/O operations may resume (step 23). The host will then return to step 13 and continue as described above.
  • Therefore, during a merge operation, the host selects a shadow set member as the source and sequentially compares the data stored in each data block (a cluster at a time) to data in the corresponding data blocks in the target shadow set member. If the host finds an inconsistency, the system is temporarily synchronized while the host resolves the inconsistency by reading the inconsistent data from the source and writing the data to the corresponding cluster in the target. Because the shadow set members being merged normally have only a very small amount of data that is inconsistent, cross-system synchronization will be necessary for only a short time, resulting in only minimal disruption to normal I/O operations.
  • Because the data on both shadow set members is equally valid, if a cluster on the shadow set member selected as the source is corrupted (e.g., cannot be read), the corresponding cluster on the target is utilized. I.e., the target acts as the source for that cluster. Therefore, data transfer in a merge operation can occur in both directions.
  • If a host in the system needs to perform a read operation while another host is performing a merge as described above, the host must first ensure that the specific location to which the read is directed is consistent. To accomplish this, the host will effectively perform a merge operation on that specific section by first issuing the read to the source and then issuing a Compare Host command to the target to determine if the data is consistent. If it is consistent, then the host continues with its processing of the read data. If it is not consistent, then the host will synchronize the system, reread the data from the source and write the data to the target as described above.
  • Since there is limited space in each disk controller memory 26 for the write history log, the hosts will try to keep as many write history entries as possible available for use. Therefore, if a host issues a write command to each member of the shadow set, and receives end messages from each disk controller indicating that the write request was successfully performed, the host will reuse the write history entries that were allocated for the write requests sent to each disk controller. These entries may be reused because, if a write has succeeded to all members of the shadow set, then the shadow set will not be inconsistent due to that write. As discussed above, a host that performs a merge operation will deallocate those entries that were used in the merge operation once the operation is completed.
  • The illustrative embodiment describes a merge operation performed on a single target, but several targets may be used to thereby merge three or more storage media. Similarly, the system need not use the same member disk as the source throughout the merge operation, but may use other disks as the source.
  • The write history log feature has been described above in connection with a merge operation in a shadow set, and specifically is used to improve the efficiency of a merge operation by determining which data blocks may contain inconsistent data (i.e., data blocks that have not been written with data should not contain inconsistent data and need not be merged). However, there are other possible applications of a storage device that includes the write log feature.
  • Similarly, while the system of Fig. 1 is the preferred embodiment, the invention can be used with other shadowing systems such as ones that use a "central controller." In these systems, the hosts access the shadow set members through a central controller.
  • Accordingly, the invention shall not be limited by the specific illustrative embodiment described above, but shall be limited only by the scope of the appended claims.

Claims (18)

  1. A method of managing data stored in a shadow set of storage media comprising a first storage medium (20) and a second storage medium (20) each accessible by a plurality of data processing devices (10), the data in a plurality of locations of said first storage medium being identical to data in a corresponding plurality of locations of said second storage medium, said method comprising the steps of:
    A. maintaining a log of information associated with write commands received from the data processing devices, comprising the steps of:
    (i) receiving a first write command from one of the data processing devices (10) directed to said first storage medium, said first write command including first data and first address information;
    (ii) writing said first data to a first location in said first storage medium determined by said first address information;
    (iii) issuing a first end message indicative of whether the step of writing to said first storage medium was completed successfully;
    (iv) writing said first address information to said log;
    (v) receiving a second write command from said one of the data processing devices (10) directed to said second storage medium, said second write command including said first data and second address information;
    (vi) writing said first data to a second location in said second storage medium determined by said second address information;
    (vii) issuing a second end message indicating whether the step of writing to said second storage medium was completed successfully; and
    (viii) writing said second address information to said log; and
    B. implementing a management operation on only locations in the shadow set of said storage media indicated by address information in said log to make data in said first storage medium identical to data stored in corresponding locations in said second storage medium when first or second end message is issued indicating a write operation to one of the first and second storage media was not completed successfully.
  2. The method of claim 1 wherein the log is stored in a location (26) independent from said shadow set of storage media.
  3. The method of claim 2 wherein said independent location (26) is independent of each of said data processing devices (10).
  4. The method of claim 1 wherein at least one of said first and second write commands comprises information identifying the data processing device from which the write command was received, and wherein said step of writing information to said log further comprises writing said identifying information into said log.
  5. The method of claim 4 wherein said management operation is carried out on locations selected on the basis of the data processing device from which data was written.
  6. The method of claim 1 wherein said first and second address information is stored in said log for data written to said first and second storage media over a predetermined period of time.
  7. The method of claim 6 wherein successive said management operations are carried out, said predetermined period of time being equal to the time between two consecutive management operations.
  8. The method of claim 1, wherein said log is cleared of address information after said management operation is completed.
  9. The method of claim 1, wherein said step of maintaining a log of information comprises maintaining a first log of information associated with first write commands to said first storage medium and a second log of information associated with second write commands to said second storage medium .
  10. An apparatus for managing data stored in a shadow set of storage media comprising a first storage medium (20) and a second storage medium (20) each accessible by a plurality of data processing devices (10), the data in a plurality of locations of said first storage medium being identical to data in a corresponding plurality of locations of said second storage medium, said apparatus comprising:
       means for maintaining a log of information associated with write commands received from the data processing devices, said means for maintaining comprising:
    means for receiving a first and second write command from one of the data processing devices (10), said first write command including first data and first address information, said second write command including said first data and second address information;
    means for writing said first data to a first location in said first storage medium determined by said first address information and for writing said first data to a second location in said second storage medium determined by said second address information;
    means for issuing a first and second end message indicating whether write operation to said first and second storage media respectively was completed successfully; and
    means for writing said first and second address information to said log; and
    means for implementing a management operation on only locations in the shadow set of said storage media indicated by address information in said log to make data in said first storage medium identical to data stored in corresponding locations in said second storage medium when first or second end message is issued indicating a write operation to one of the first and second storage media was not completed successfully.
  11. The apparatus of claim 10 wherein the shadow set of storage media comprises:
    a first storage subsystem (17) including a first disk controller (18) and said first storage medium (20); and
    a second storage subsystem (17) comprising a second disk controller (18) and said second storage medium (20).
  12. The apparatus of claim 10 wherein at least one of said first and second write commands comprises information identifying the data processing device (10) from which the write command was received, said apparatus further comprising means for writing said identifying information in said log.
  13. The apparatus of claim 12 wherein said management operation is carried out with respect to locations selected on the basis of the data processing device from which data was written.
  14. The apparatus of claim 10 wherein said address information is stored in said log for data written to said shadow set of storage media over a predetermined period of time.
  15. The apparatus of claim 14 wherein successive said management operations are carried out, said predetermined period of time being equal to the time between two of said management operations.
  16. The apparatus of claim 10 wherein said means for maintaining a log of information maintains a first log of information for said first address information and a second log of information for said second address information .
  17. The apparatus of claim 16, wherein the shadow set of storage media comprises:
    a first storage subsystem (17) including a first disk controller (18), a first local memory (26), and said first storage medium (20); and
    a second storage subsystem (17) including a second disk controller (18), a second local memory (26), and said second storage medium (20);
    wherein said first log of information is stored in said first local memory and said second log of information is stored in said second local memory.
  18. The apparatus of claim 10 further comprising means for clearing said log of address information after the management operation is completed.
EP90306856A 1989-06-30 1990-06-22 Method and apparatus for managing a shadow set of storage media Expired - Lifetime EP0405859B1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US37425189A 1989-06-30 1989-06-30
US374251 1989-06-30

Publications (3)

Publication Number Publication Date
EP0405859A2 EP0405859A2 (en) 1991-01-02
EP0405859A3 EP0405859A3 (en) 1991-12-04
EP0405859B1 true EP0405859B1 (en) 1997-09-17

Family

ID=23475942

Family Applications (1)

Application Number Title Priority Date Filing Date
EP90306856A Expired - Lifetime EP0405859B1 (en) 1989-06-30 1990-06-22 Method and apparatus for managing a shadow set of storage media

Country Status (8)

Country Link
US (1) US5546536A (en)
EP (1) EP0405859B1 (en)
JP (1) JPH07111689B2 (en)
AT (1) ATE158424T1 (en)
AU (1) AU622333B2 (en)
CA (1) CA2020269A1 (en)
DE (1) DE69031443T2 (en)
TW (1) TW285727B (en)

Families Citing this family (71)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2778786B2 (en) * 1990-03-02 1998-07-23 富士通株式会社 Data update / restoration processing method
US5446904A (en) * 1991-05-17 1995-08-29 Zenith Data Systems Corporation Suspend/resume capability for a protected mode microprocessor
JP3243919B2 (en) * 1994-02-23 2002-01-07 三菱電機株式会社 Disk device and FA controller using this disk device
DE69510251T2 (en) * 1994-02-28 1999-11-25 Canon Kk Information processing method and device
JP3136258B2 (en) * 1995-09-27 2001-02-19 三菱電機株式会社 Disk update log recording method
US5862318A (en) * 1995-10-26 1999-01-19 Microsoft Corporation System for generating a gapless series of identity values
US5953742A (en) * 1996-07-01 1999-09-14 Sun Microsystems, Inc. Memory management in fault tolerant computer systems utilizing a first and second recording mechanism and a reintegration mechanism
JP3671595B2 (en) * 1997-04-01 2005-07-13 株式会社日立製作所 Compound computer system and compound I / O system
US5951695A (en) * 1997-07-25 1999-09-14 Hewlett-Packard Company Fast database failover
US6192460B1 (en) 1997-12-16 2001-02-20 Compaq Computer Corporation Method and apparatus for accessing data in a shadow set after a failed data operation
US6170063B1 (en) * 1998-03-07 2001-01-02 Hewlett-Packard Company Method for performing atomic, concurrent read and write operations on multiple storage devices
JP4689137B2 (en) * 2001-08-08 2011-05-25 株式会社日立製作所 Remote copy control method and storage system
US6128699A (en) * 1998-10-27 2000-10-03 Hewlett-Packard Company Method for providing read/write access while copying data between shared storage devices
US6341317B1 (en) * 1998-12-30 2002-01-22 Emc Corporation Method and apparatus for managing a log of information in a computer system including an intelligent storage system
US6415300B1 (en) 1999-07-06 2002-07-02 Syncsort Incorporated Method of performing a high-performance backup which gains efficiency by reading input file blocks sequentially
US6408314B1 (en) 1999-07-06 2002-06-18 Synscort Incorporated Method of performing a high-performance sort which gains efficiency by reading input file blocks sequentially
US6671705B1 (en) * 1999-08-17 2003-12-30 Emc Corporation Remote mirroring system, device, and method
US6684294B1 (en) * 2000-03-31 2004-01-27 Intel Corporation Using an access log for disk drive transactions
DE10053016A1 (en) * 2000-10-17 2002-04-25 Libelle Informatik Gmbh Device for data mirroring has mirror system with older data version, temporary memory for data related to actions leading from older version to current one, monitoring/changeover program
IL139628A0 (en) * 2000-11-12 2002-02-10 Eci Telecom Ltd Data mirroring restoration in a distributed system
US6823349B1 (en) * 2001-09-21 2004-11-23 Emc Corporation Method and system for establishing, maintaining, and using a persistent fracture log
US6587536B1 (en) * 2002-03-18 2003-07-01 Holtec International, Inc. Method and apparatus for maximizing radiation shielding during cask transfer procedures
US6928513B2 (en) * 2002-03-26 2005-08-09 Hewlett-Packard Development Company, L.P. System and method for managing data logging memory in a storage area network
US6993635B1 (en) * 2002-03-29 2006-01-31 Intransa, Inc. Synchronizing a distributed mirror
US7171396B2 (en) * 2002-04-04 2007-01-30 Hewlett-Packard Development Company, L.P. Method and program product for specifying the different data access route for the first data set includes storing an indication of the different access for the first data set providing alternative data access routes to a data storage
GB0227786D0 (en) 2002-11-29 2003-01-08 Ibm Improved remote copy synchronization in disaster recovery computer systems
US7809762B1 (en) * 2003-07-11 2010-10-05 Bmc Software, Inc. Outage-less database change operation
WO2005048085A2 (en) 2003-11-13 2005-05-26 Commvault Systems, Inc. System and method for performing an image level snapshot and for restoring partial volume data
US7137033B2 (en) * 2003-11-20 2006-11-14 International Business Machines Corporation Method, system, and program for synchronizing subtasks using sequence numbers
US7412577B2 (en) * 2004-02-05 2008-08-12 International Business Machines Corporation Shared data mirroring apparatus, method, and system
US7130971B2 (en) * 2004-03-30 2006-10-31 Hitachi, Ltd. Assuring genuineness of data stored on a storage device
US7809898B1 (en) * 2004-05-18 2010-10-05 Symantec Operating Corporation Detecting and repairing inconsistencies in storage mirrors
US7472129B2 (en) * 2004-06-29 2008-12-30 Microsoft Corporation Lossless recovery for computer systems with map assisted state transfer
US7360111B2 (en) * 2004-06-29 2008-04-15 Microsoft Corporation Lossless recovery for computer systems with remotely dependent data recovery
CA2632935C (en) 2005-12-19 2014-02-04 Commvault Systems, Inc. Systems and methods for performing data replication
US7606844B2 (en) 2005-12-19 2009-10-20 Commvault Systems, Inc. System and method for performing replication copy storage operations
US7651593B2 (en) 2005-12-19 2010-01-26 Commvault Systems, Inc. Systems and methods for performing data replication
US7617262B2 (en) 2005-12-19 2009-11-10 Commvault Systems, Inc. Systems and methods for monitoring application data in a data replication system
US8655850B2 (en) 2005-12-19 2014-02-18 Commvault Systems, Inc. Systems and methods for resynchronizing information
US8726242B2 (en) * 2006-07-27 2014-05-13 Commvault Systems, Inc. Systems and methods for continuous data replication
US8290808B2 (en) 2007-03-09 2012-10-16 Commvault Systems, Inc. System and method for automating customer-validated statement of work for a data storage environment
US9495382B2 (en) 2008-12-10 2016-11-15 Commvault Systems, Inc. Systems and methods for performing discrete data replication
US8204859B2 (en) 2008-12-10 2012-06-19 Commvault Systems, Inc. Systems and methods for managing replicated database data
US8095828B1 (en) * 2009-08-31 2012-01-10 Symantec Corporation Using a data storage system for cluster I/O failure determination
US8504517B2 (en) 2010-03-29 2013-08-06 Commvault Systems, Inc. Systems and methods for selective data replication
US8352422B2 (en) 2010-03-30 2013-01-08 Commvault Systems, Inc. Data restore systems and methods in a replication environment
US8725698B2 (en) 2010-03-30 2014-05-13 Commvault Systems, Inc. Stub file prioritization in a data replication system
US8504515B2 (en) 2010-03-30 2013-08-06 Commvault Systems, Inc. Stubbing systems and methods in a data replication environment
WO2011150391A1 (en) 2010-05-28 2011-12-01 Commvault Systems, Inc. Systems and methods for performing data replication
US9298715B2 (en) 2012-03-07 2016-03-29 Commvault Systems, Inc. Data storage system utilizing proxy device for storage operations
US9471578B2 (en) 2012-03-07 2016-10-18 Commvault Systems, Inc. Data storage system utilizing proxy device for storage operations
US9342537B2 (en) 2012-04-23 2016-05-17 Commvault Systems, Inc. Integrated snapshot interface for a data storage system
US8862642B1 (en) 2012-06-29 2014-10-14 Emc Corporation Endurant cache
US9058326B1 (en) * 2012-06-29 2015-06-16 Emc Corporation Recovery and flush of endurant cache
US9886346B2 (en) 2013-01-11 2018-02-06 Commvault Systems, Inc. Single snapshot for multiple agents
US20140201140A1 (en) 2013-01-11 2014-07-17 Commvault Systems, Inc. Data synchronization management
US9495251B2 (en) 2014-01-24 2016-11-15 Commvault Systems, Inc. Snapshot readiness checking and reporting
US9639426B2 (en) 2014-01-24 2017-05-02 Commvault Systems, Inc. Single snapshot for multiple applications
US9632874B2 (en) 2014-01-24 2017-04-25 Commvault Systems, Inc. Database application backup in single snapshot for multiple applications
US9753812B2 (en) 2014-01-24 2017-09-05 Commvault Systems, Inc. Generating mapping information for single snapshot for multiple applications
US9774672B2 (en) 2014-09-03 2017-09-26 Commvault Systems, Inc. Consolidated processing of storage-array commands by a snapshot-control media agent
US10042716B2 (en) 2014-09-03 2018-08-07 Commvault Systems, Inc. Consolidated processing of storage-array commands using a forwarder media agent in conjunction with a snapshot-control media agent
US9448731B2 (en) 2014-11-14 2016-09-20 Commvault Systems, Inc. Unified snapshot storage management
US9648105B2 (en) 2014-11-14 2017-05-09 Commvault Systems, Inc. Unified snapshot storage management, using an enhanced storage manager and enhanced media agents
US10503753B2 (en) 2016-03-10 2019-12-10 Commvault Systems, Inc. Snapshot replication operations based on incremental block change tracking
GB2549737B (en) * 2016-04-26 2019-05-08 Advanced Risc Mach Ltd An apparatus and method for managing address collisions when performing vector operations
US10732885B2 (en) 2018-02-14 2020-08-04 Commvault Systems, Inc. Block-level live browsing and private writable snapshots using an ISCSI server
US11055184B2 (en) 2018-12-19 2021-07-06 Vmware, Inc. In-place garbage collection of a sharded, replicated distributed state machine based on supersedable operations
US10877881B2 (en) * 2019-01-11 2020-12-29 Vmware, Inc. In-place garbage collection of a sharded, replicated distributed state machine based on mergeable operations
US11042318B2 (en) 2019-07-29 2021-06-22 Commvault Systems, Inc. Block-level data replication
US11809285B2 (en) 2022-02-09 2023-11-07 Commvault Systems, Inc. Protecting a management database of a data storage management system to meet a recovery point objective (RPO)

Family Cites Families (45)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US3544777A (en) * 1967-11-06 1970-12-01 Trw Inc Two memory self-correcting system
US3668644A (en) * 1970-02-09 1972-06-06 Burroughs Corp Failsafe memory system
US4199810A (en) * 1977-01-07 1980-04-22 Rockwell International Corporation Radiation hardened register file
US4189781A (en) * 1977-01-25 1980-02-19 International Business Machines Corporation Segmented storage logging and controlling
JPS5637883A (en) * 1979-09-04 1981-04-11 Fanuc Ltd Information rewrite system
US4467421A (en) * 1979-10-18 1984-08-21 Storage Technology Corporation Virtual storage system and method
JPS607818B2 (en) * 1980-04-28 1985-02-27 横河電機株式会社 How to back up files
US4432057A (en) * 1981-11-27 1984-02-14 International Business Machines Corporation Method for the dynamic replication of data under distributed system control to control utilization of resources in a multiprocessing, distributed data base system
US4476526A (en) * 1981-11-27 1984-10-09 Storage Technology Corporation Cache buffered memory subsystem
US4450559A (en) * 1981-12-24 1984-05-22 International Business Machines Corporation Memory system with selective assignment of spare locations
US4636946A (en) * 1982-02-24 1987-01-13 International Business Machines Corporation Method and apparatus for grouping asynchronous recording operations
DE3208573C2 (en) * 1982-03-10 1985-06-27 Standard Elektrik Lorenz Ag, 7000 Stuttgart 2 out of 3 selection device for a 3 computer system
US4503534A (en) * 1982-06-30 1985-03-05 Intel Corporation Apparatus for redundant operation of modules in a multiprocessing system
WO1984002409A1 (en) * 1982-12-09 1984-06-21 Sequoia Systems Inc Memory backup system
US4819154A (en) * 1982-12-09 1989-04-04 Sequoia Systems, Inc. Memory back up system with one cache memory and two physically separated main memories
JPS59142799A (en) * 1983-02-04 1984-08-16 Hitachi Ltd Doubled storage device with electricity storage device for backup
JPS59165162A (en) * 1983-03-11 1984-09-18 インタ−ナシヨナル ビジネス マシ−ンズ コ−ポレ−シヨン Volume restoration system
US4602368A (en) * 1983-04-15 1986-07-22 Honeywell Information Systems Inc. Dual validity bit arrays
JPS59201297A (en) * 1983-04-28 1984-11-14 Nec Corp Information inspecting system
US4600990A (en) * 1983-05-16 1986-07-15 Data General Corporation Apparatus for suspending a reserve operation in a disk drive
US4584681A (en) * 1983-09-02 1986-04-22 International Business Machines Corporation Memory correction scheme using spare arrays
US4608687A (en) * 1983-09-13 1986-08-26 International Business Machines Corporation Bit steering apparatus and method for correcting errors in stored data, storing the address of the corrected data and using the address to maintain a correct data condition
US4608688A (en) * 1983-12-27 1986-08-26 At&T Bell Laboratories Processing system tolerant of loss of access to secondary storage
US4638424A (en) * 1984-01-12 1987-01-20 International Business Machines Corporation Managing data storage devices connected to a digital computer
JPS60156150A (en) * 1984-01-25 1985-08-16 Hitachi Ltd File sharing system between computers
US4755928A (en) * 1984-03-05 1988-07-05 Storage Technology Corporation Outboard back-up and recovery system with transfer of randomly accessible data sets between cache and host and cache and tape simultaneously
US4916605A (en) * 1984-03-27 1990-04-10 International Business Machines Corporation Fast write operations
US4617475A (en) * 1984-03-30 1986-10-14 Trilogy Computer Development Partners, Ltd. Wired logic voting circuit
US4959774A (en) * 1984-07-06 1990-09-25 Ampex Corporation Shadow memory system for storing variable backup blocks in consecutive time periods
US4686620A (en) * 1984-07-26 1987-08-11 American Telephone And Telegraph Company, At&T Bell Laboratories Database backup method
JPS6150293A (en) * 1984-08-17 1986-03-12 Fujitsu Ltd Semiconductor memory
US4747038A (en) * 1984-10-04 1988-05-24 Honeywell Bull Inc. Disk controller memory address register
JPS61148546A (en) * 1984-12-24 1986-07-07 Nec Corp File control device
JPH0622021B2 (en) * 1985-01-18 1994-03-23 富士通株式会社 Dynamic reconfiguration system for multiplexed shared file memory
JPS61264599A (en) * 1985-05-16 1986-11-22 Fujitsu Ltd Semiconductor memory device
US4751639A (en) * 1985-06-24 1988-06-14 Ncr Corporation Virtual command rollback in a fault tolerant data processing system
US4710870A (en) * 1985-07-10 1987-12-01 Bell Communications Research, Inc. Central computer backup system utilizing localized data bases
US4814971A (en) * 1985-09-11 1989-03-21 Texas Instruments Incorporated Virtual memory recovery system using persistent roots for selective garbage collection and sibling page timestamping for defining checkpoint state
US4805095A (en) * 1985-12-23 1989-02-14 Ncr Corporation Circuit and a method for the selection of original data from a register log containing original and modified data
JPS62197858A (en) * 1986-02-26 1987-09-01 Hitachi Ltd Inter-system data base sharing system
SE454730B (en) * 1986-09-19 1988-05-24 Asea Ab PROCEDURE AND COMPUTER EQUIPMENT FOR SHORT-FREE REPLACEMENT OF THE ACTIVITY FROM ACTIVE DEVICES TO EMERGENCY UNITS IN A CENTRAL UNIT
DE3853503T2 (en) * 1987-08-20 1995-09-28 Ibm Method and device for the maintenance of duplex pair distributed devices by means of a double copy function.
JP2924905B2 (en) * 1988-03-25 1999-07-26 エヌシーアール インターナショナル インコーポレイテッド File backup system
KR900005300A (en) * 1988-09-13 1990-04-13 윌슨 디.파아고 Hot backup file server machine
US5089958A (en) * 1989-01-23 1992-02-18 Vortex Systems, Inc. Fault tolerant computer backup system

Also Published As

Publication number Publication date
JPH03135638A (en) 1991-06-10
US5546536A (en) 1996-08-13
CA2020269A1 (en) 1990-12-31
JPH07111689B2 (en) 1995-11-29
EP0405859A3 (en) 1991-12-04
ATE158424T1 (en) 1997-10-15
TW285727B (en) 1996-09-11
AU622333B2 (en) 1992-04-02
DE69031443T2 (en) 1998-04-23
DE69031443D1 (en) 1997-10-23
EP0405859A2 (en) 1991-01-02
AU5795490A (en) 1991-01-03

Similar Documents

Publication Publication Date Title
EP0405859B1 (en) Method and apparatus for managing a shadow set of storage media
EP0405925B1 (en) Method and apparatus for managing a shadow set of storage media
EP0405926B1 (en) Method and apparatus for managing a shadow set of storage media
US6950915B2 (en) Data storage subsystem
US5555371A (en) Data backup copying with delayed directory updating and reduced numbers of DASD accesses at a back up site using a log structured array data storage
US6732171B2 (en) Distributed network storage system with virtualization
US5568628A (en) Storage control method and apparatus for highly reliable storage controller with multiple cache memories
US7596659B2 (en) Method and system for balanced striping of objects
US6862632B1 (en) Dynamic RDF system for transferring initial data between source and destination volume wherein data maybe restored to either volume at same time other data is written
US7043610B2 (en) System and method for maintaining cache coherency without external controller intervention
US6438661B1 (en) Method, system, and program for managing meta data in a storage system and rebuilding lost meta data in cache
US7373470B2 (en) Remote copy control in a storage system
US5210865A (en) Transferring data between storage media while maintaining host processor access for I/O operations
US6101574A (en) Disk control unit for holding track data in non-volatile cache memory
US5719885A (en) Storage reliability method and apparatus
US6363457B1 (en) Method and system for non-disruptive addition and deletion of logical devices
US6715036B1 (en) Method, system, and data structures for transferring blocks of data from a storage device to a requesting application
US6304941B1 (en) Method and apparatus for reducing processor operations when adding a new drive to a raid-6 drive group
US20050223180A1 (en) Accelerating the execution of I/O operations in a storage system
EP0405861A2 (en) Transferring data in a digital data processing system
JP2002288014A (en) File control system and file data writing method
JPS6027953A (en) Check point processing system
JP2000132917A (en) Method of managing defect in information recording medium, apparatus therefor and information recording medium

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 19900710

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AT BE CH DE DK ES FR GB GR IT LI LU NL SE

PUAL Search report despatched

Free format text: ORIGINAL CODE: 0009013

RHK1 Main classification (correction)

Ipc: G06F 11/14

AK Designated contracting states

Kind code of ref document: A3

Designated state(s): AT BE CH DE DK ES FR GB GR IT LI LU NL SE

17Q First examination report despatched

Effective date: 19931125

GRAG Despatch of communication of intention to grant

Free format text: ORIGINAL CODE: EPIDOS AGRA

GRAH Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOS IGRA

GRAH Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOS IGRA

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE CH DE DK ES FR GB GR IT LI LU NL SE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Effective date: 19970917

Ref country code: LI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 19970917

Ref country code: AT

Effective date: 19970917

Ref country code: CH

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 19970917

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 19970917

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 19970917

Ref country code: ES

Free format text: THE PATENT HAS BEEN ANNULLED BY A DECISION OF A NATIONAL AUTHORITY

Effective date: 19970917

Ref country code: DK

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 19970917

REF Corresponds to:

Ref document number: 158424

Country of ref document: AT

Date of ref document: 19971015

Kind code of ref document: T

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REF Corresponds to:

Ref document number: 69031443

Country of ref document: DE

Date of ref document: 19971023

ITF It: translation for a ep patent filed

Owner name: STUDIO TORTA S.R.L.

ET Fr: translation filed
PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Effective date: 19971217

NLV1 Nl: lapsed or annulled due to failure to fulfill the requirements of art. 29p and 29m of the patents act
REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: LU

Payment date: 19980819

Year of fee payment: 9

26N No opposition filed
PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 19990622

REG Reference to a national code

Ref country code: GB

Ref legal event code: 732E

REG Reference to a national code

Ref country code: GB

Ref legal event code: IF02

REG Reference to a national code

Ref country code: GB

Ref legal event code: 732E

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20050622

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20070731

Year of fee payment: 18

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20070628

Year of fee payment: 18

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20070618

Year of fee payment: 18

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20080622

REG Reference to a national code

Ref country code: FR

Ref legal event code: ST

Effective date: 20090228

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20090101

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20080622

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20080630