US20020042818A1 - Geographic data replication system and method for a network - Google Patents
Geographic data replication system and method for a network Download PDFInfo
- Publication number
- US20020042818A1 US20020042818A1 US09/160,389 US16038998A US2002042818A1 US 20020042818 A1 US20020042818 A1 US 20020042818A1 US 16038998 A US16038998 A US 16038998A US 2002042818 A1 US2002042818 A1 US 2002042818A1
- Authority
- US
- United States
- Prior art keywords
- server
- data
- active file
- temporary data
- file server
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 38
- 230000010076 replication Effects 0.000 title claims description 62
- 230000003362 replicative effect Effects 0.000 claims abstract description 10
- 230000004044 response Effects 0.000 claims description 8
- 230000000977 initiatory effect Effects 0.000 claims description 2
- 238000012545 processing Methods 0.000 abstract description 21
- 230000008569 process Effects 0.000 abstract description 12
- 230000008901 benefit Effects 0.000 abstract description 3
- 230000003252 repetitive effect Effects 0.000 abstract description 2
- 238000012546 transfer Methods 0.000 description 12
- 238000010586 diagram Methods 0.000 description 6
- 230000004913 activation Effects 0.000 description 5
- 238000004891 communication Methods 0.000 description 5
- 230000008859 change Effects 0.000 description 4
- 238000013500 data storage Methods 0.000 description 4
- 238000012217 deletion Methods 0.000 description 4
- 230000037430 deletion Effects 0.000 description 4
- 238000007726 management method Methods 0.000 description 3
- 230000001360 synchronised effect Effects 0.000 description 3
- 230000007613 environmental effect Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 238000007792 addition Methods 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 239000003054 catalyst Substances 0.000 description 1
- 230000001934 delay Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000011664 signaling Effects 0.000 description 1
- 230000005641 tunneling Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/40—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass for recovering from a failure of a protocol instance or entity, e.g. service redundancy protocols, protocol state redundancy or protocol service redirection
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99951—File or database maintenance
- Y10S707/99952—Coherency, e.g. same view to multiple users
- Y10S707/99953—Recoverability
Definitions
- the Internet is an international network of interconnected government, education, and business computer networks.
- a person at a computer terminal or personal computer with the proper software communicates through the Internet by placing data in an Internet protocol (IP) packet with an address for a particular destination on the Internet.
- IP Internet protocol
- TCP Transmission Control Protocol
- IP Internet protocol
- Communication software on processors or routers on the intervening networks between the source and the destination network read the address assigned to the packet and forward the packets towards their destination.
- Intranets are typically corporate networks that use the same networking or transport protocols as the Internet.
- dedicated servers provide access to corporate information in a cohesive fashion.
- documents are stored in a HyperText Markup Language (HTML), and users of the intranet network use web browser software to obtain access to the documents.
- HTML HyperText Markup Language
- users of the intranet network use web browser software to obtain access to the documents.
- both the Internet and intranets may be IP networks.
- the IP network allows a user to gain access to various applications and related data. For example, a user with web browser software enters an address of a desired application. The address and the user's request for access is then routed through the IP network or a plurality of IP networks to an application server at the requested address. The user may then request further information such as an HTML document. Using HyperText Transport Protocol (HTTP), HTML documents are communicated to the user.
- the documents or other application information comprise content, such as a Intranet web page, graphics or information representing various text and graphics associated with a company and its services.
- Various groups provide the servers and associated applications. For example, a corporation provides a content server for access to information about that corporation by users through the IP network. Multiple servers, each geographically remote to the other, may run the same applications for more efficient data retrieval. However, these servers operate independently.
- the temporary data may include user selections, other user input information, status information associated with the application processing, or application generated information used for further processing. However, if the application server or associated memory fails, the temporary data may be lost.
- the present invention is directed to a geographic data replication system and method.
- temporary data for a local server is replicated periodically to a remote server.
- the temporary data for the remote server is replicated to the local server.
- Replication includes copying temporary data to file servers associated with the local and remote servers.
- an architecture according to the present invention allows for about 100% service availability. If a server fails, such as the local server, the remote server begins processing user requests based on the temporary data it received from the local server. Failure of a server may not result in failure of services or loss of previously generated temporary data.
- a method and associated system provide geographic data replication. Temporary data is stored on a first active file server.
- the first active file server is geographically remote from a second active file server.
- the temporary data associated with the first active file server is replicated to the second active file server.
- a system and associated method provide geographic data replication.
- the same content is provided with first and second application servers.
- the first application server is geographically remote from the second application server.
- First and second sets of temporary data associated with the content and with the first and second application servers, respectively, are replicated to the second and first application servers, respectively.
- FIG. 1 is a schematic block diagram of one embodiment of a processing and data replication architecture in accordance with the present invention.
- FIG. 2 is a flow chart diagram of one embodiment of a data replication method for use on the architecture of FIG. 1.
- FIG. 3 is a flow chart diagram of one embodiment for implementing the data replication method of FIG. 2.
- FIGS. 1 - 3 embodiments of a system and associated methods for replicating temporary data are shown.
- the temporary data associated with at least two geographically remote servers is replicated between the servers. If a server generates temporary data associated with a user, such as identifying selected shopping items, the temporary data is replicated to a remote server. If the local server fails, the user request is routed to the remote server. The remote server processes the request with the benefit of the previously generated temporary data. Temporary data, such as data identifying the selected shopping items, is applied by the remote server without repetitive user input or processing. Therefore, about 100% service availability is provided.
- Network 10 comprises data centers 14 A and 14 B and lines 12 A and 12 B. Each line 12 A and 12 B connects to a data center 14 A and 14 B, respectively.
- Data center 14 A is geographically remote from data center 14 B. Geographically remote components include components connected to a Wide Area Network (WAN), through two or more switches, or components housed in structures at different geographic locations. For example, data center can be located in San Jose, Calif. and data center 14 B can be located in Relay, Md.
- Data centers 14 A and 14 B may comprise a WAN or a portion of a WAN.
- WAN Wide Area Network
- each data center 14 A and 14 B While only two data centers 14 A and 14 B are shown, more data centers may be provided, such as data centers 14 N.
- Various components within each data center 14 A and 14 B are preferably located proximate to each other, such as in Local Area Networks (LANs), but may be dispersed in various geographical regions.
- LANs Local Area Networks
- Each data center 14 A and 14 B preferably includes at least one file server 16 A and 16 B, one utility server 18 A and 18 B, and one application server 20 A and 20 B operatively connected to at least one switch 22 A and 22 B. Switches 22 A and 22 B also operatively connect to lines 12 A and 12 B, respectively, and firewalls 24 A and 24 B, respectively.
- Data center 14 A and 14 B may comprise more or fewer components, such as only an application server and associated memory. Furthermore, redundant components and associated connections and interfaces may be provided within each data center 14 A and 14 B.
- each data center 14 A and 14 B comprises the data centers disclosed in U.S. application Ser. No. 09/021,466, filed Feb. 10, 1998, for a Survivable Scalable Data System And Method For Computer Networks, the disclosure of which is herein incorporated by reference.
- a user request is routed to architecture 10 on one of lines 12 A or 12 B.
- a TCP/IP or other protocol request is routed to line 12 A.
- one or more requests are routed lines 12 A and 12 B.
- the routing may be performed in accordance with the system and method disclosed in U.S. patent application Ser. No. 09/021,091 for a Method And System For Network Routing, filed Feb. 10, 1998, the disclosure of which is herein incorporated by reference.
- Architecture 10 generates responses to the requests and places the responses on one of lines 12 A or 12 B for routing back to the user.
- the requests are provided on lines 12 A and 12 B to switches 22 A and 22 B, respectively.
- the corresponding components of each data center are the same type of device, but different types of devices may be used in each data center 14 A and 14 B.
- Switch 22 A comprises a network switch operating under the control of any of various software applications, such as a Catalyst 5000 Ethernet switch by CISCO (Dual Power Supply, 2 Supervisor Modules, 8 Fast Ethernet Modules).
- Switch 22 A transfers data among the various components of data center 14 A.
- Various transport formats such as asynchronous transfer mode (ATM), Fiber Distributed Data Interface, or fast switched Ethernet formats, may be used.
- ATM asynchronous transfer mode
- Fiber Distributed Data Interface or fast switched Ethernet formats
- a fast Ethernet transport format is used.
- the connections between the various components may be duplicative Ethernet connections and may be encrypted in a Level 2 Forwarding (L2F) format, other formats, or not encrypted.
- L2F Level 2 Forwarding
- Other connections and formats such as Level 2 Transfer Protocol or Point-to-Point Tunneling Protocol, may be used.
- Other switches 22 A or other means for connecting various components within data center 14 A may be used, such as direct connections.
- Switch 22 A transfers information from or to firewall 24 A.
- Firewall 24 A comprises a server with firewall software, such as a Sun Ultra 2 server (300 Mhz, 256M Ram, two 2 GB drives and two Fast Ethernet Adapters) running Raptor Eagle Software. Other servers from other manufacturers with the same or different software may be used.
- firewall 24 A is provided at data center 14 A and secures write access into data center 14 A.
- firewalls 24 A and 24 B connect data centers 14 A and 14 B through a Virtual Private Network (VPN).
- VPN Virtual Private Network
- Other network or communications connections may be used, such as ISDN, TCP/IP networks, or modem links.
- the VPN may comprise a back-end or private network for communications between various data centers.
- Lines 12 A and 12 B provide communications for public access.
- Application server 20 A preferably comprises servers as known in the art, such as SUN SPARC (300 Mhz, 256M Ram, two 2 GB drives and two Fast Ethernet Adapters) computer processors each associated with a unique network address. Other servers or combinations of different servers may be used.
- Application server 20 A represents one or more shared or dedicated servers for providing content information. Content is provided in response to requests, such as requests input on line 12 A.
- Content is associated with an application run on application server 20 A and includes any information provided without alteration.
- the application may include any known service, such as, for example, mail, news, directory, groupware, CyberCash/DNS or other content providing applications.
- Groupware applications include any type of collaborative tool, such as video conferencing and whiteboard software.
- CyberCash requires communication with various financial institutions on the Internet. Therefore, queries from application server 20 A for CyberCash information are preferably routed through a back-end interface and are proxied by a firewall, such as firewall 24 A.
- the content associated with the applications may include web pages, lists, documents or other information.
- Each application server 20 A and 20 B runs at least one of the same applications as the other.
- both application servers 20 A and 20 B run a same news application.
- the news application may include processing associated with bulletin boards of current events and chat rooms, as known in the art.
- Application server 20 A receives one or more requests related to customer content and application server 20 B also receives one or more requests related to the same customer content.
- File server 16 A comprises one or more hard disk drives and associated processors or other data storage systems.
- file sever 16 A may include two devices configured for the Network File System (NFS) standard, such as Auspex NS7000 servers.
- file server 16 A includes three Fast Ethernet interfaces with two connected to different interface cards within switch 22 A. The other Fast Ethernet interface preferably connects to a standalone Fast Ethernet hub connected to switch 22 A. A system active signal is transmitted on the interfaces connected to switch 22 A as well as the interface connected to the Fast Ethernet hub.
- NFS Network File System
- data stored on file server 16 A is mirrored.
- the data is mirrored or copied to different locations on a drive, different drives or different cabinets. Any changes to the data are made to the mirrored data.
- identical or substantially the same data is provided in two locations within file server 16 A.
- data is striped across multiple local drives. If one of the drives fail, the remaining drives continue to service requests for data that existed on the failed drive.
- File server 16 A is active.
- active servers include servers operated to respond to requests or retrieve data for use by other components. For example, in the active mode, data for storage (i.e., writes) is transferred to file servers 16 A or 16 B and data from storage (i.e., reads) are serviced by file server 16 A or 16 B.
- Other servers may be in sleep or standby modes of operation.
- multiple sources of mirrored data within data center 14 A are active.
- both or more than one source of the same mirrored data is used to respond to queries.
- the other source of the same mirrored data is available to respond to queries.
- the two mirrored sources of the same data are synchronized.
- the sources of the same data are synchronized to verify that the data is mirrored.
- Data from the recently failed file server 16 A is verified against the source of mirrored data and updated as necessary.
- the recently failed and now operational file server 14 A or source of data is then used to respond to queries, along with the other source of the same data. If more than one query associated with the same data is generated, then the queries are transferred to the same or different mirrored sources of the same data.
- File server 16 A is mounted by utility server 18 A.
- Utility server 18 A comprises a processor for controlling operation of data center 14 A.
- utility server 18 A comprises a Sun Ultra 2 running Unix or Solaris 2.5.1, but other servers and software may be used. More than one utility server 18 A may be used in data center 14 A.
- Utility server 18 A may provide data center 14 A monitoring.
- a separate remote management point or connection is provided.
- the various components of data center 14 A may be monitored for operational status over T-1 connections.
- Information associated with any hardware additions, deletions and reconfigurations may be provided to the appropriate components.
- Hardware management and/or administration of data center 14 A may be through a remote or local access point.
- Utility server 18 A also controls adding, removing and changing various application processing and data storage. For example, storage and application processing for new or changed content is controlled. File server 16 A and the appropriate application server 20 A is provided data storage address information. Other controls, such as control over the switches, may also be provided through utility server 18 A.
- Utility server 18 A controls replication of temporary data.
- temporary data includes data generated by application server 20 A, file server 16 A, user input and any other data created during and associated with hosting network services.
- html or data files created by CGI or other programs for processing user requests comprise temporary data files.
- temporary data includes log files generated after initiating data center 14 A.
- user input information such as associated with shopping selections or other user data, comprises temporary data for use hosting or providing content.
- application server 20 A generates queries in response to user requests.
- the queries comprise temporary data.
- the requests correspond to a recent news article and application server 20 A generates a query to obtain the news article.
- the queries are in any of various formats, such as Simple Mail Transfer Protocol (SMTP), Network News Transfer Protocol (NNTP), HyperText Transfer Protocol (HTTP), Calendar Access Protocol (CAP), File Transfer Protocol (FTP), and Lightweight Directory Access Protocol (LDAP).
- SMTP Simple Mail Transfer Protocol
- NTP Network News Transfer Protocol
- HTTP HyperText Transfer Protocol
- CAP Calendar Access Protocol
- FTP File Transfer Protocol
- LDAP Lightweight Directory Access Protocol
- Other TCP/IP formats or other protocols may be used.
- the query is routed to file server 18 A to obtain the news article.
- the query may be addressed for another server in application server 20 A.
- the temporary data is stored on file server 16 A.
- the temporary data is identified, in part, by application server 20 A and customer or content, such as by storage in a customer or content directory.
- application server 20 A provides content associated with company XYZ.
- Temporary data associated with processing the content or other XYZ application is stored as XYZ data from application server 20 A.
- data center 14 B and other data centers comprise the same or similar components for generating, transferring and storing temporary data. For maximum service availability, the temporary data is replicated between data center 14 A, 14 B and any other data centers.
- Replication is controlled by software on utility servers 18 A and/or 18 B.
- the software preferably comprises a Perl script daemon, but other languages may be used.
- the software executes at one data center 14 A (i.e., the local data center) to control replication of data generated by data centers 14 A and 14 B to other data centers 14 B and 14 A, respectively.
- the software described or other replication software may be implemented on both utility servers 18 A and 18 B or other servers, such as application servers 20 A or 20 B or file servers 16 A or 16 B.
- the temporary data is stored at the data center 14 A or 14 B that generated the temporary data as discussed above.
- the software checks for a replicate command.
- the check and/or the command are preferably periodically performed or provided, respectively.
- the software activates (i.e., the software wakes) as a function of a time or event trigger. Preferably, the software activates every 15 seconds or some other predetermined amount of time.
- the software causes replication in response to activation.
- the temporary data is replicated between data centers 14 A and 14 B (see FIG. 1).
- Replication includes execution of a rdist on each customer's or application server's 20 A and/or 20 B temporary data directory. While the Unix system's rdist is used in one preferred embodiment, systems other than Unix using other utilities may be used.
- rdist is executed at each data center 14 A and 14 B to copy temporary data. Preferably, only temporary data created or changed since the previous activation of the software is copied. In alternative embodiments, temporary data identified in other groupings or associated with different times is copied.
- the temporary data is synchronized between data centers 14 A, 14 B.
- a change in the temporary data in either data center 14 A, 14 B results in a change in the temporary data of the other data center 14 B, 14 A.
- Any given temporary data may be originally changed in either data center 14 A, 14 B.
- the copied temporary data is transferred to other data centers 14 A, 14 B or 14 N.
- the temporary data is preferably transferred to and copied in all other data centers 14 A, 14 B through 14 N, but may be transferred to one or other subset of data centers 14 A, 14 B through 14 N.
- the temporary data is transferred through firewalls 24 A and 24 B over the VPN.
- the temporary data is transferred pursuant to the TCP/IP protocols, but other formats may be used. Reference will be made below to local data center 14 A and remote data center 14 B, but the discussion applies to additional remote data centers, such as through remote data center 14 N.
- the replication software (i.e., the replication daemon) is executed in one data center 14 A or 14 B, such as the local data center 14 A.
- the software generates a client process for running at a remote site for local operations and communication of results and other data to the replication software in the other remote data centers, such as remote data center 14 B. Therefore, local data center 14 A checks for and activates replication.
- the client processes perform rdist or other replication utilities at the remote data center 14 B, and rdist is also performed at the local data center 14 A.
- the temporary data is copied and transferred between data centers 14 A and 14 B.
- the temporary data associated with local data center 14 A is copied to data center 14 B. If data center 14 A becomes unavailable, little or no temporary data is lost. Data center 14 B begins processing user requests previously processed by data center 14 A with little or no interruption. For example, data center 14 A accepts user selections of various shopping items. The corresponding temporary data is replicated to data center 14 B. When data center 14 A is not available, data center 14 B continues processing with the benefit of the temporary data showing the user selections. Since remote utility server 18 B already mounts remote file server 16 B, little delay is caused, such as delay to transfer data and for other priority processing. The user may not be required to reenter the selections.
- one data center 14 A or 14 B fails, user requests processed by the failed data center are routed to the other data center. Any routing technique may be used, including the routing disclosed in U.S. patent application Ser. No. 09/021,091 for a Method And System For Network Routing, filed Feb. 10, 1998. Therefore, the requests and the associated temporary data are available at the various data centers.
- a back-up utility server may be used to replicate temporary data if the primary utility server fails.
- FIG. 3 a flow chart diagram of one embodiment for implementing the temporary data replication is shown. This embodiment preferably runs on a Unix system, but other systems may be used.
- the replication daemon executes various functions to enable replication. These enabling steps are performed once each time the system is turned on.
- the replication daemon configures various environmental variables, such as replication period or frequency, and labels identifying one of a plurality of configurations of directories and appropriate actions.
- the replication daemon also reads any command line arguments, such as variables overriding the default environmental variables, transferred to the replication daemon.
- the replication daemon detaches from the Unix file descriptors.
- a new process is created as a copy for running the replication in the background
- the new process is disassociated from the existing process group and the standard Unix file descriptors STDIN, STDOUT, and STDERR to allow undisturbed background processing. Disassociation creates multiple instances of replication capabilities at each data center 14 A and 14 B in a peer-to-peer relationship. In alternative embodiments, a master-slave relationship is used.
- the replication daemon writes the Process Identification (PID) number of the replication daemon to a designated file. Using this PID, the replication daemon is controlled and monitored with various signaling or other management processing.
- PID Process Identification
- Data center information is retrieved in step 56 .
- the host name i.e., the address
- the replication daemon obtains a copy of the file.
- the process enters a loop. For each activation, the loop is processed starting at step 58 .
- the replication daemon identifies and copies the appropriate temporary data for application servers 20 A within local data center 14 A. Preferably, new, newer than previous copies, or newer than temporary data stored at remote data center 14 B is copied. The copy of the temporary data is stored (i.e., the data is pushed or transferred) to one, all, or a sub-set of remote data centers, such as data center 14 B.
- UNIX utility rdist version 6.1.3 from University of Southern California transfers the temporary data, but other software for identifying and transferring data may be used.
- the temporary data may be substantially simultaneously or sequentially transferred or broadcast to multiple data centers.
- acknowledgment of complete transfer of data is provided.
- the replication process continues while waiting for acknowledgment from each remote data center 14 A or 14 B. Without waiting for acknowledgment, delays associated with broadcast transfers over a WAN are avoided.
- the temporary data from one, all, or a sub-set of remote data centers 14 B is replicated in step 60 .
- the replication daemon at local data center 14 A instructs utility server 18 B at remote data center 14 B to replicate temporary data using the client process.
- Remote utility server 18 B execute rdist or other utilities to identify and copy the temporary data.
- new, newer than previous copies, or newer than temporary data stored at another data center is copied.
- the temporary data is replicated to local data center 14 A, another remote data center and combinations thereof.
- the replication daemon checks for UNIX signals in step 62 . If no UNIX signals are received, the replication daemon returns to step 58 and awaits activation.
- the replication daemon If a USR 1 or other signal designating a change in processing state is received, the replication daemon enters a sleep mode (i.e., the daemon waits for another signal) in step 64 . In the sleep mode, the replication daemon continues to run but does not perform any replication. The sleep mode allows system maintenance and temporary data deletion as described below. If another USR 1 or other signal designating a change in processing state is received, the replication daemon returns to step 58 and awaits activation.
- the replication daemon ceases operation in step 66 .
- the replication daemon finishes any unfinished replication, deletes the PID file and exits from operation.
- the sleep mode discussed above may be used for deleting expired or no longer used temporary data.
- a Perl script or other language software for deletion is preferably separate from the replication daemon and is run on utility servers 18 A and 18 B for each data center 14 A and 14 B, respectively. Every hour, other time period, or at a triggering event, temporary files originally generated in the associated data center 14 A or 14 B and ready for deletion are removed. For example, temporary data created more than twenty four hours or another time period prior to the current time is deleted. Only temporary data with the appropriate prefix designating data center 14 A or 14 B that created the temporary data is deleted. Once deleted from the creating data center 14 A or 14 B, rdist as implemented with the replication daemon identifies deleted temporary data as part of the replication process. The temporary data corresponding to the deleted temporary data is removed from other data centers 14 A or 14 B.
- temporary and content data are backed up periodically, such as every night with a nightly rotation of partial, full (i.e., complete backup of part of the data), and incremental back ups. Due to the large amount of data, a seven day rotation for full back-up of portions of the data is preferably used. Preferably, seven groups of file systems are identified and each group is fully backed-up once a week on alternating nights. On any given night, the remaining groups are incrementally backed up. The back-up data is archived.
Abstract
Description
- Not applicable
- Not applicable
- Not applicable
- 1. Field of the Invention
- This invention generally relates to computer networks. In particular, the invention relates to Internet and intranet computer systems.
- 2. Description of the Prior Art
- The Internet is an international network of interconnected government, education, and business computer networks. A person at a computer terminal or personal computer with the proper software communicates through the Internet by placing data in an Internet protocol (IP) packet with an address for a particular destination on the Internet. Using Transmission Control Protocol (TCP), transport of the packet is managed. Communication software on processors or routers on the intervening networks between the source and the destination network read the address assigned to the packet and forward the packets towards their destination.
- Intranets are typically corporate networks that use the same networking or transport protocols as the Internet. Typically, dedicated servers provide access to corporate information in a cohesive fashion. For example, documents are stored in a HyperText Markup Language (HTML), and users of the intranet network use web browser software to obtain access to the documents. Thus, both the Internet and intranets may be IP networks.
- The IP network allows a user to gain access to various applications and related data. For example, a user with web browser software enters an address of a desired application. The address and the user's request for access is then routed through the IP network or a plurality of IP networks to an application server at the requested address. The user may then request further information such as an HTML document. Using HyperText Transport Protocol (HTTP), HTML documents are communicated to the user. The documents or other application information comprise content, such as a Intranet web page, graphics or information representing various text and graphics associated with a company and its services.
- Various groups provide the servers and associated applications. For example, a corporation provides a content server for access to information about that corporation by users through the IP network. Multiple servers, each geographically remote to the other, may run the same applications for more efficient data retrieval. However, these servers operate independently.
- As an application server responds to user requests, various temporary data is generated. The temporary data may include user selections, other user input information, status information associated with the application processing, or application generated information used for further processing. However, if the application server or associated memory fails, the temporary data may be lost.
- The present invention is directed to a geographic data replication system and method. According to one feature of the invention, temporary data for a local server is replicated periodically to a remote server. According to a second feature, the temporary data for the remote server is replicated to the local server. Replication includes copying temporary data to file servers associated with the local and remote servers. Advantageously, an architecture according to the present invention allows for about 100% service availability. If a server fails, such as the local server, the remote server begins processing user requests based on the temporary data it received from the local server. Failure of a server may not result in failure of services or loss of previously generated temporary data.
- In a particular first aspect of the invention, a method and associated system provide geographic data replication. Temporary data is stored on a first active file server. The first active file server is geographically remote from a second active file server. The temporary data associated with the first active file server is replicated to the second active file server.
- In a second aspect of the invention, a system and associated method provide geographic data replication. The same content is provided with first and second application servers. The first application server is geographically remote from the second application server. First and second sets of temporary data associated with the content and with the first and second application servers, respectively, are replicated to the second and first application servers, respectively.
- FIG. 1 is a schematic block diagram of one embodiment of a processing and data replication architecture in accordance with the present invention.
- FIG. 2 is a flow chart diagram of one embodiment of a data replication method for use on the architecture of FIG. 1.
- FIG. 3 is a flow chart diagram of one embodiment for implementing the data replication method of FIG. 2.
- Referring to FIGS.1-3, embodiments of a system and associated methods for replicating temporary data are shown. The temporary data associated with at least two geographically remote servers is replicated between the servers. If a server generates temporary data associated with a user, such as identifying selected shopping items, the temporary data is replicated to a remote server. If the local server fails, the user request is routed to the remote server. The remote server processes the request with the benefit of the previously generated temporary data. Temporary data, such as data identifying the selected shopping items, is applied by the remote server without repetitive user input or processing. Therefore, about 100% service availability is provided.
- Referring now to FIG. 1, a schematic block diagram of one embodiment of a processing and data replication architecture is shown generally at10.
Network 10 comprisesdata centers 14A and 14B and lines 12A and 12B. Each line 12A and 12B connects to adata center 14A and 14B, respectively.Data center 14A is geographically remote from data center 14B. Geographically remote components include components connected to a Wide Area Network (WAN), through two or more switches, or components housed in structures at different geographic locations. For example, data center can be located in San Jose, Calif. and data center 14B can be located in Relay, Md.Data centers 14A and 14B may comprise a WAN or a portion of a WAN. While only twodata centers 14A and 14B are shown, more data centers may be provided, such as data centers 14N. Various components within eachdata center 14A and 14B are preferably located proximate to each other, such as in Local Area Networks (LANs), but may be dispersed in various geographical regions. - Each
data center 14A and 14B preferably includes at least onefile server utility server application server 20A and 20B operatively connected to at least one switch 22A and 22B. Switches 22A and 22B also operatively connect to lines 12A and 12B, respectively, and firewalls 24A and 24B, respectively.Data center 14A and 14B may comprise more or fewer components, such as only an application server and associated memory. Furthermore, redundant components and associated connections and interfaces may be provided within eachdata center 14A and 14B. In one preferred embodiment, eachdata center 14A and 14B comprises the data centers disclosed in U.S. application Ser. No. 09/021,466, filed Feb. 10, 1998, for a Survivable Scalable Data System And Method For Computer Networks, the disclosure of which is herein incorporated by reference. - Through an intranet or Internet or other network connection, a user request is routed to
architecture 10 on one of lines 12A or 12B. For example, a TCP/IP or other protocol request is routed to line 12A. At any given time, one or more requests are routed lines 12A and 12B. For example, the routing may be performed in accordance with the system and method disclosed in U.S. patent application Ser. No. 09/021,091 for a Method And System For Network Routing, filed Feb. 10, 1998, the disclosure of which is herein incorporated by reference.Architecture 10 generates responses to the requests and places the responses on one of lines 12A or 12B for routing back to the user. - The requests are provided on lines12A and 12B to switches 22A and 22B, respectively. Reference will be made below to components of
data center 14A, but the discussion applies to the same or similar components of data center 14B. Preferably, the corresponding components of each data center are the same type of device, but different types of devices may be used in eachdata center 14A and 14B. - Switch22A comprises a network switch operating under the control of any of various software applications, such as a Catalyst 5000 Ethernet switch by CISCO (Dual Power Supply, 2 Supervisor Modules, 8 Fast Ethernet Modules). Switch 22A transfers data among the various components of
data center 14A. Various transport formats, such as asynchronous transfer mode (ATM), Fiber Distributed Data Interface, or fast switched Ethernet formats, may be used. Preferably, a fast Ethernet transport format is used. The connections between the various components may be duplicative Ethernet connections and may be encrypted in a Level 2 Forwarding (L2F) format, other formats, or not encrypted. For example, each connection is a 100 Mb Fast Ethernet connection. Other connections and formats, such as Level 2 Transfer Protocol or Point-to-Point Tunneling Protocol, may be used. Other switches 22A or other means for connecting various components withindata center 14A may be used, such as direct connections. - Switch22A transfers information from or to
firewall 24A.Firewall 24A comprises a server with firewall software, such as a Sun Ultra 2 server (300 Mhz, 256M Ram, two 2 GB drives and two Fast Ethernet Adapters) running Raptor Eagle Software. Other servers from other manufacturers with the same or different software may be used. In one embodiment,firewall 24A is provided atdata center 14A and secures write access intodata center 14A. - There may be various connections to networks or components outside
data center 14A proxied byfirewall 24A. Preferably, firewalls 24A and 24B connectdata centers 14A and 14B through a Virtual Private Network (VPN). Other network or communications connections may be used, such as ISDN, TCP/IP networks, or modem links. The VPN may comprise a back-end or private network for communications between various data centers. Lines 12A and 12B provide communications for public access. - Public access is provided to
application server 20A.Application server 20A preferably comprises servers as known in the art, such as SUN SPARC (300 Mhz, 256M Ram, two 2 GB drives and two Fast Ethernet Adapters) computer processors each associated with a unique network address. Other servers or combinations of different servers may be used.Application server 20A represents one or more shared or dedicated servers for providing content information. Content is provided in response to requests, such as requests input on line 12A. - Content is associated with an application run on
application server 20A and includes any information provided without alteration. The application may include any known service, such as, for example, mail, news, directory, groupware, CyberCash/DNS or other content providing applications. Groupware applications include any type of collaborative tool, such as video conferencing and whiteboard software. CyberCash requires communication with various financial institutions on the Internet. Therefore, queries fromapplication server 20A for CyberCash information are preferably routed through a back-end interface and are proxied by a firewall, such asfirewall 24A. The content associated with the applications may include web pages, lists, documents or other information. - Each
application server 20A and 20B runs at least one of the same applications as the other. For example, bothapplication servers 20A and 20B run a same news application. The news application may include processing associated with bulletin boards of current events and chat rooms, as known in the art.Application server 20A receives one or more requests related to customer content and application server 20B also receives one or more requests related to the same customer content. - Content and other data associated with each application is stored on
file server 16A.File server 16A comprises one or more hard disk drives and associated processors or other data storage systems. For example, file sever 16A may include two devices configured for the Network File System (NFS) standard, such as Auspex NS7000 servers. In one preferred embodiment,file server 16A includes three Fast Ethernet interfaces with two connected to different interface cards within switch 22A. The other Fast Ethernet interface preferably connects to a standalone Fast Ethernet hub connected to switch 22A. A system active signal is transmitted on the interfaces connected to switch 22A as well as the interface connected to the Fast Ethernet hub. - Preferably, data stored on
file server 16A is mirrored. The data is mirrored or copied to different locations on a drive, different drives or different cabinets. Any changes to the data are made to the mirrored data. Thus, identical or substantially the same data is provided in two locations withinfile server 16A. Additionally or alternatively, data is striped across multiple local drives. If one of the drives fail, the remaining drives continue to service requests for data that existed on the failed drive. -
File server 16A is active. As used herein, active servers include servers operated to respond to requests or retrieve data for use by other components. For example, in the active mode, data for storage (i.e., writes) is transferred tofile servers file server - In one embodiment, multiple sources of mirrored data within
data center 14A are active. To further increase efficiency of storage space, both or more than one source of the same mirrored data is used to respond to queries. Upon any failure of one of the sources of mirrored data, such as no response within 5 milliseconds, the other source of the same mirrored data is available to respond to queries. Once the failed source of data is operational, the two mirrored sources of the same data are synchronized. After synchronizing to the same clock, the sources of the same data are synchronized to verify that the data is mirrored. Data from the recently failedfile server 16A is verified against the source of mirrored data and updated as necessary. The recently failed and nowoperational file server 14A or source of data is then used to respond to queries, along with the other source of the same data. If more than one query associated with the same data is generated, then the queries are transferred to the same or different mirrored sources of the same data. -
File server 16A is mounted byutility server 18A.Utility server 18A comprises a processor for controlling operation ofdata center 14A. In one preferred embodiment,utility server 18A comprises a Sun Ultra 2 running Unix or Solaris 2.5.1, but other servers and software may be used. More than oneutility server 18A may be used indata center 14A. -
Utility server 18A may providedata center 14A monitoring. Preferably, a separate remote management point or connection is provided. For example, the various components ofdata center 14A may be monitored for operational status over T-1 connections. Information associated with any hardware additions, deletions and reconfigurations may be provided to the appropriate components. Hardware management and/or administration ofdata center 14A may be through a remote or local access point. -
Utility server 18A also controls adding, removing and changing various application processing and data storage. For example, storage and application processing for new or changed content is controlled.File server 16A and theappropriate application server 20A is provided data storage address information. Other controls, such as control over the switches, may also be provided throughutility server 18A. -
Utility server 18A controls replication of temporary data. As used herein, temporary data includes data generated byapplication server 20A,file server 16A, user input and any other data created during and associated with hosting network services. For example, html or data files created by CGI or other programs for processing user requests comprise temporary data files. For another example, temporary data includes log files generated after initiatingdata center 14A. As another example, user input information, such as associated with shopping selections or other user data, comprises temporary data for use hosting or providing content. - As yet another example,
application server 20A generates queries in response to user requests. The queries comprise temporary data. For example, the requests correspond to a recent news article andapplication server 20A generates a query to obtain the news article. The queries are in any of various formats, such as Simple Mail Transfer Protocol (SMTP), Network News Transfer Protocol (NNTP), HyperText Transfer Protocol (HTTP), Calendar Access Protocol (CAP), File Transfer Protocol (FTP), and Lightweight Directory Access Protocol (LDAP). Other TCP/IP formats or other protocols may be used. The query is routed tofile server 18A to obtain the news article. The query may be addressed for another server inapplication server 20A. - The temporary data is stored on
file server 16A. Preferably, the temporary data is identified, in part, byapplication server 20A and customer or content, such as by storage in a customer or content directory. For example,application server 20A provides content associated with company XYZ. Temporary data associated with processing the content or other XYZ application is stored as XYZ data fromapplication server 20A. As discussed above, data center 14B and other data centers comprise the same or similar components for generating, transferring and storing temporary data. For maximum service availability, the temporary data is replicated betweendata center 14A, 14B and any other data centers. - Replication is controlled by software on
utility servers 18A and/or 18B. The software preferably comprises a Perl script daemon, but other languages may be used. In one embodiment, the software executes at onedata center 14A (i.e., the local data center) to control replication of data generated bydata centers 14A and 14B toother data centers 14B and 14A, respectively. The software described or other replication software may be implemented on bothutility servers application servers 20A or 20B orfile servers - Referring to FIG. 2, a representative flow diagram of one embodiment of a data replication method is shown. At
step 40, the temporary data is stored at thedata center 14A or 14B that generated the temporary data as discussed above. - At
step 42, the software checks for a replicate command. The check and/or the command are preferably periodically performed or provided, respectively. The software activates (i.e., the software wakes) as a function of a time or event trigger. Preferably, the software activates every 15 seconds or some other predetermined amount of time. The software causes replication in response to activation. - At
step 44, the temporary data is replicated betweendata centers 14A and 14B (see FIG. 1). Replication includes execution of a rdist on each customer's or application server's 20A and/or 20B temporary data directory. While the Unix system's rdist is used in one preferred embodiment, systems other than Unix using other utilities may be used. In this embodiment, rdist is executed at eachdata center 14A and 14B to copy temporary data. Preferably, only temporary data created or changed since the previous activation of the software is copied. In alternative embodiments, temporary data identified in other groupings or associated with different times is copied. - In one preferred embodiment, the temporary data is synchronized between
data centers 14A, 14B. A change in the temporary data in eitherdata center 14A, 14B results in a change in the temporary data of theother data center 14B, 14A. Any given temporary data may be originally changed in eitherdata center 14A, 14B. - The copied temporary data is transferred to
other data centers 14A, 14B or 14N. The temporary data is preferably transferred to and copied in allother data centers 14A, 14B through 14N, but may be transferred to one or other subset ofdata centers 14A, 14B through 14N. The temporary data is transferred throughfirewalls local data center 14A and remote data center 14B, but the discussion applies to additional remote data centers, such as through remote data center 14N. - In one embodiment, the replication software (i.e., the replication daemon) is executed in one
data center 14A or 14B, such as thelocal data center 14A. The software generates a client process for running at a remote site for local operations and communication of results and other data to the replication software in the other remote data centers, such as remote data center 14B. Therefore,local data center 14A checks for and activates replication. In response, the client processes perform rdist or other replication utilities at the remote data center 14B, and rdist is also performed at thelocal data center 14A. The temporary data is copied and transferred betweendata centers 14A and 14B. - The temporary data associated with
local data center 14A is copied to data center 14B. Ifdata center 14A becomes unavailable, little or no temporary data is lost. Data center 14B begins processing user requests previously processed bydata center 14A with little or no interruption. For example,data center 14A accepts user selections of various shopping items. The corresponding temporary data is replicated to data center 14B. Whendata center 14A is not available, data center 14B continues processing with the benefit of the temporary data showing the user selections. Sinceremote utility server 18B already mountsremote file server 16B, little delay is caused, such as delay to transfer data and for other priority processing. The user may not be required to reenter the selections. - If one
data center 14A or 14B fails, user requests processed by the failed data center are routed to the other data center. Any routing technique may be used, including the routing disclosed in U.S. patent application Ser. No. 09/021,091 for a Method And System For Network Routing, filed Feb. 10, 1998. Therefore, the requests and the associated temporary data are available at the various data centers. A back-up utility server may be used to replicate temporary data if the primary utility server fails. - Referring to FIG. 3, a flow chart diagram of one embodiment for implementing the temporary data replication is shown. This embodiment preferably runs on a Unix system, but other systems may be used.
- Prior to replication, the replication daemon executes various functions to enable replication. These enabling steps are performed once each time the system is turned on.
- At
step 50, the replication daemon configures various environmental variables, such as replication period or frequency, and labels identifying one of a plurality of configurations of directories and appropriate actions. The replication daemon also reads any command line arguments, such as variables overriding the default environmental variables, transferred to the replication daemon. - The replication daemon detaches from the Unix file descriptors. At
step 52, a new process is created as a copy for running the replication in the background The new process is disassociated from the existing process group and the standard Unix file descriptors STDIN, STDOUT, and STDERR to allow undisturbed background processing. Disassociation creates multiple instances of replication capabilities at eachdata center 14A and 14B in a peer-to-peer relationship. In alternative embodiments, a master-slave relationship is used. - At
step 54, the replication daemon writes the Process Identification (PID) number of the replication daemon to a designated file. Using this PID, the replication daemon is controlled and monitored with various signaling or other management processing. - Data center information is retrieved in
step 56. The host name (i.e., the address) of eachutility server data center 14A and 14B for receiving replicated temporary data is listed in a file. The replication daemon obtains a copy of the file. - Once the replication daemon enables replication, the process enters a loop. For each activation, the loop is processed starting at
step 58. Atstep 58, the replication daemon identifies and copies the appropriate temporary data forapplication servers 20A withinlocal data center 14A. Preferably, new, newer than previous copies, or newer than temporary data stored at remote data center 14B is copied. The copy of the temporary data is stored (i.e., the data is pushed or transferred) to one, all, or a sub-set of remote data centers, such as data center 14B. - In one embodiment, UNIX utility rdist version 6.1.3 from University of Southern California transfers the temporary data, but other software for identifying and transferring data may be used. The temporary data may be substantially simultaneously or sequentially transferred or broadcast to multiple data centers. In one embodiment, acknowledgment of complete transfer of data is provided. Preferably, the replication process continues while waiting for acknowledgment from each
remote data center 14A or 14B. Without waiting for acknowledgment, delays associated with broadcast transfers over a WAN are avoided. - Once the temporary data from
local data center 14A is transferred to remote data center 14B, the temporary data from one, all, or a sub-set of remote data centers 14B is replicated instep 60. The replication daemon atlocal data center 14A instructsutility server 18B at remote data center 14B to replicate temporary data using the client process.Remote utility server 18B execute rdist or other utilities to identify and copy the temporary data. Preferably, new, newer than previous copies, or newer than temporary data stored at another data center is copied. The temporary data is replicated tolocal data center 14A, another remote data center and combinations thereof. - Once the temporary data from
remote data center 14A or another remote data center is transferred to other data centers, the replication daemon checks for UNIX signals instep 62. If no UNIX signals are received, the replication daemon returns to step 58 and awaits activation. - If a
USR 1 or other signal designating a change in processing state is received, the replication daemon enters a sleep mode (i.e., the daemon waits for another signal) instep 64. In the sleep mode, the replication daemon continues to run but does not perform any replication. The sleep mode allows system maintenance and temporary data deletion as described below. If anotherUSR 1 or other signal designating a change in processing state is received, the replication daemon returns to step 58 and awaits activation. - If a TERM signal or other signal designating termination of the replication daemon is received in
steps step 66. For example, the replication daemon finishes any unfinished replication, deletes the PID file and exits from operation. - The sleep mode discussed above may be used for deleting expired or no longer used temporary data. A Perl script or other language software for deletion is preferably separate from the replication daemon and is run on
utility servers data center 14A and 14B, respectively. Every hour, other time period, or at a triggering event, temporary files originally generated in the associateddata center 14A or 14B and ready for deletion are removed. For example, temporary data created more than twenty four hours or another time period prior to the current time is deleted. Only temporary data with the appropriate prefix designatingdata center 14A or 14B that created the temporary data is deleted. Once deleted from the creatingdata center 14A or 14B, rdist as implemented with the replication daemon identifies deleted temporary data as part of the replication process. The temporary data corresponding to the deleted temporary data is removed fromother data centers 14A or 14B. - In addition to replication, temporary and content data are backed up periodically, such as every night with a nightly rotation of partial, full (i.e., complete backup of part of the data), and incremental back ups. Due to the large amount of data, a seven day rotation for full back-up of portions of the data is preferably used. Preferably, seven groups of file systems are identified and each group is fully backed-up once a week on alternating nights. On any given night, the remaining groups are incrementally backed up. The back-up data is archived.
- It should be understood that many changes and modifications can be made to the embodiments described above. For example, different numbers of application servers or data storage devices may be used to provide different processing and data. It is therefore intended that the foregoing detailed description be understood as an illustration of the presently preferred embodiments of the invention, and not as a definition of the invention. It is only the following claims, including all equivalents, that are intended to define the scope of the invention.
Claims (16)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US09/160,389 US6411991B1 (en) | 1998-09-25 | 1998-09-25 | Geographic data replication system and method for a network |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US09/160,389 US6411991B1 (en) | 1998-09-25 | 1998-09-25 | Geographic data replication system and method for a network |
Publications (2)
Publication Number | Publication Date |
---|---|
US20020042818A1 true US20020042818A1 (en) | 2002-04-11 |
US6411991B1 US6411991B1 (en) | 2002-06-25 |
Family
ID=22576690
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/160,389 Expired - Lifetime US6411991B1 (en) | 1998-09-25 | 1998-09-25 | Geographic data replication system and method for a network |
Country Status (1)
Country | Link |
---|---|
US (1) | US6411991B1 (en) |
Cited By (14)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20030159083A1 (en) * | 2000-09-29 | 2003-08-21 | Fukuhara Keith T. | System, method and apparatus for data processing and storage to provide continuous operations independent of device failure or disaster |
US20040199609A1 (en) * | 2003-04-07 | 2004-10-07 | Microsoft Corporation | System and method for web server migration |
US6816980B1 (en) | 2000-09-15 | 2004-11-09 | Zeronines Technology, Inc. | Fault tolerant, state-compatible computer system and method |
US7197519B2 (en) | 2002-11-14 | 2007-03-27 | Hitachi, Ltd. | Database system including center server and local servers |
US20080130812A1 (en) * | 2006-11-30 | 2008-06-05 | Seoul National University Industry Foundation | Data synchronization system |
US7600148B1 (en) * | 2006-09-19 | 2009-10-06 | United Services Automobile Association (Usaa) | High-availability data center |
US7685465B1 (en) * | 2006-09-19 | 2010-03-23 | United Services Automobile Association (Usaa) | High-availability data center |
US20100131545A1 (en) * | 2008-11-26 | 2010-05-27 | Yahoo! Inc. | Distribution Data Items Within Geographically Distributed Databases |
US7747898B1 (en) * | 2006-09-19 | 2010-06-29 | United Services Automobile Association (Usaa) | High-availability data center |
US8171111B1 (en) * | 2008-08-07 | 2012-05-01 | United Services Automobile Association (Usaa) | Systems and methods for non-specific address routing |
US20130036092A1 (en) * | 2011-08-03 | 2013-02-07 | Amadeus S.A.S. | Method and System to Maintain Strong Consistency of Distributed Replicated Contents in a Client/Server System |
US8688787B1 (en) | 2002-04-26 | 2014-04-01 | Zeronines Technology, Inc. | System, method and apparatus for data processing and storage to provide continuous e-mail operations independent of device failure or disaster |
US8850035B1 (en) * | 2007-05-16 | 2014-09-30 | Yahoo! Inc. | Geographically distributed real time communications platform |
US11044195B1 (en) | 2008-08-21 | 2021-06-22 | United Services Automobile Association (Usaa) | Preferential loading in data centers |
Families Citing this family (34)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6687874B2 (en) * | 1998-10-05 | 2004-02-03 | Nexpress Solutions Llc | System for generating and maintaining field service publications |
US7110973B1 (en) * | 1999-09-29 | 2006-09-19 | Charles Schwab & Co., Inc. | Method of processing customer transactions |
DE60010853T2 (en) * | 2000-03-10 | 2005-07-14 | Alcatel | Method and device for data backup |
US6928481B1 (en) * | 2000-05-05 | 2005-08-09 | International Business Machines Corporation | Method, apparatus and program to optimize the network distribution of digital information based on hierarchical grouping of server topology and code distribution |
US7802010B1 (en) | 2000-06-14 | 2010-09-21 | Sprint Communications Company L.P. | Computer network method and system for guaranteed messaging service |
US7203755B2 (en) * | 2000-12-29 | 2007-04-10 | Webex—Communications, Inc. | System and method for application sharing in collaborative setting |
WO2002054264A1 (en) | 2000-12-29 | 2002-07-11 | Webex Communications, Inc. | Distributed network system architecture for collaborative computing |
US20030167304A1 (en) * | 2000-12-29 | 2003-09-04 | Min Zhu | Distributed meeting management |
US20030167302A1 (en) * | 2000-12-29 | 2003-09-04 | Min Zhu | Scalable distributed network system for collaborative computing |
US7069298B2 (en) * | 2000-12-29 | 2006-06-27 | Webex Communications, Inc. | Fault-tolerant distributed system for collaborative computing |
US20020091881A1 (en) * | 2001-01-10 | 2002-07-11 | International Business Machines Corporation | Managing a failure to access a database in a computer system |
US6944133B2 (en) * | 2001-05-01 | 2005-09-13 | Ge Financial Assurance Holdings, Inc. | System and method for providing access to resources using a fabric switch |
US6957251B2 (en) | 2001-05-07 | 2005-10-18 | Genworth Financial, Inc. | System and method for providing network services using redundant resources |
JP2003141006A (en) * | 2001-07-17 | 2003-05-16 | Canon Inc | Communication system, communication device, communication method, storage medium and program |
US7243137B2 (en) * | 2001-09-28 | 2007-07-10 | Sun Microsystems, Inc. | Remote system controller and data center and methods for implementing the same |
US6912282B2 (en) * | 2003-02-06 | 2005-06-28 | Cisco Tehnology, Inc. | Enabling Cisco legacy power to support IEEE 802.3 AF standard power |
JP2004355083A (en) * | 2003-05-27 | 2004-12-16 | Nec Corp | Backup system and backup program |
US7454502B2 (en) * | 2003-12-04 | 2008-11-18 | International Business Machines Corporation | System for transferring standby resource entitlement |
US7543017B1 (en) * | 2004-05-28 | 2009-06-02 | Sun Microsystems, Inc. | Cluster file system node failure file recovery by reconstructing file state |
US7321906B2 (en) * | 2004-07-23 | 2008-01-22 | Omx Technology Ab | Method of improving replica server performance and a replica server system |
US7664080B2 (en) * | 2004-10-27 | 2010-02-16 | Honeywell International Inc. | Discreet event operators for event management in a wireless sensor network |
US7561544B2 (en) * | 2004-10-27 | 2009-07-14 | Honeywell International Inc. | Machine architecture for event management in a wireless sensor network |
US7630336B2 (en) * | 2004-10-27 | 2009-12-08 | Honeywell International Inc. | Event-based formalism for data management in a wireless sensor network |
US7590098B2 (en) * | 2004-10-27 | 2009-09-15 | Honeywell International Inc. | Publish/subscribe model in a wireless sensor network |
US8027280B2 (en) * | 2004-10-27 | 2011-09-27 | Honeywell International Inc. | Layered architecture for data management in a wireless sensor network |
US7715308B2 (en) * | 2004-12-09 | 2010-05-11 | Honeywell International Inc. | Fault tolerance in a wireless network |
US8874477B2 (en) | 2005-10-04 | 2014-10-28 | Steven Mark Hoffberg | Multifactorial optimization system and method |
US7596598B2 (en) * | 2005-10-21 | 2009-09-29 | Birthday Alarm, Llc | Multi-media tool for creating and transmitting artistic works |
US7480817B2 (en) * | 2006-03-31 | 2009-01-20 | International Business Machines Corporation | Method for replicating data based on probability of concurrent failure |
US7778183B2 (en) * | 2006-03-31 | 2010-08-17 | International Business Machines Corporation | Data replica selector |
US9201745B2 (en) * | 2008-01-23 | 2015-12-01 | Omx Technology Ab | Method of improving replica server performance and a replica server system |
US8966027B1 (en) * | 2010-05-24 | 2015-02-24 | Amazon Technologies, Inc. | Managing replication of computing nodes for provided computer networks |
US10681113B2 (en) | 2015-10-14 | 2020-06-09 | International Business Machines Corporation | Geographically distributed highly available mailbox |
US10168948B2 (en) * | 2017-02-20 | 2019-01-01 | International Business Machines Corporation | Replicating data in a data storage system |
Family Cites Families (32)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4914570A (en) | 1986-09-15 | 1990-04-03 | Counterpoint Computers, Inc. | Process distribution and sharing system for multiple processor computer system |
US5613106A (en) * | 1989-09-15 | 1997-03-18 | Motorola, Inc. | Method for processing and storing a transaction in a distributed database system |
US5555371A (en) * | 1992-12-17 | 1996-09-10 | International Business Machines Corporation | Data backup copying with delayed directory updating and reduced numbers of DASD accesses at a back up site using a log structured array data storage |
US6233702B1 (en) | 1992-12-17 | 2001-05-15 | Compaq Computer Corporation | Self-checked, lock step processor pairs |
US5689706A (en) * | 1993-06-18 | 1997-11-18 | Lucent Technologies Inc. | Distributed systems with replicated files |
US5694546A (en) * | 1994-05-31 | 1997-12-02 | Reisman; Richard R. | System for automatic unattended electronic information transport between a server and a client by a vendor provided transport software with a manifest list |
US5745753A (en) * | 1995-01-24 | 1998-04-28 | Tandem Computers, Inc. | Remote duplicate database facility with database replication support for online DDL operations |
CN100452071C (en) | 1995-02-13 | 2009-01-14 | 英特特拉斯特技术公司 | Systems and methods for secure transaction management and electronic rights protection |
JP2708009B2 (en) * | 1995-03-17 | 1998-02-04 | 日本電気株式会社 | LAN connection device and connection method |
US5870552A (en) | 1995-03-28 | 1999-02-09 | America Online, Inc. | Method and apparatus for publishing hypermedia documents over wide area networks |
US5713017A (en) * | 1995-06-07 | 1998-01-27 | International Business Machines Corporation | Dual counter consistency control for fault tolerant network file servers |
US6199082B1 (en) | 1995-07-17 | 2001-03-06 | Microsoft Corporation | Method for delivering separate design and content in a multimedia publishing system |
US5819020A (en) * | 1995-10-16 | 1998-10-06 | Network Specialists, Inc. | Real time backup system |
US5765171A (en) * | 1995-12-29 | 1998-06-09 | Lucent Technologies Inc. | Maintaining consistency of database replicas |
US6055522A (en) | 1996-01-29 | 2000-04-25 | Futuretense, Inc. | Automatic page converter for dynamic content distributed publishing system |
US5781550A (en) | 1996-02-02 | 1998-07-14 | Digital Equipment Corporation | Transparent and secure network gateway |
US5894554A (en) | 1996-04-23 | 1999-04-13 | Infospinner, Inc. | System for managing dynamic web page generation requests by intercepting request at web server and routing to page server thereby releasing web server to process other requests |
US5870558A (en) | 1996-06-25 | 1999-02-09 | Mciworldcom, Inc. | Intranet graphical user interface for SONET network management |
US5794253A (en) * | 1996-07-12 | 1998-08-11 | Microsoft Corporation | Time based expiration of data objects in a store and forward replication enterprise |
US5781912A (en) * | 1996-12-19 | 1998-07-14 | Oracle Corporation | Recoverable data replication between source site and destination site without distributed transactions |
US6026433A (en) | 1997-03-17 | 2000-02-15 | Silicon Graphics, Inc. | Method of creating and editing a web site in a client-server environment using customizable web site templates |
US5950198A (en) * | 1997-03-24 | 1999-09-07 | Novell, Inc. | Processes and apparatuses for generating file correspondency through replication and synchronization between target and source computers |
US5867667A (en) | 1997-03-24 | 1999-02-02 | Pfn, Inc. | Publication network control system using domain and client side communications resource locator lists for managing information communications between the domain server and publication servers |
US5948108A (en) | 1997-06-12 | 1999-09-07 | Tandem Computers, Incorporated | Method and system for providing fault tolerant access between clients and a server |
US6134584A (en) | 1997-11-21 | 2000-10-17 | International Business Machines Corporation | Method for accessing and retrieving information from a source maintained by a network server |
US6208986B1 (en) | 1997-12-15 | 2001-03-27 | International Business Machines Corporation | Web interface and method for accessing and displaying directory information |
US6065018A (en) * | 1998-03-04 | 2000-05-16 | International Business Machines Corporation | Synchronizing recovery log having time stamp to a remote site for disaster recovery of a primary database having related hierarchial and relational databases |
US6240105B1 (en) * | 1998-03-30 | 2001-05-29 | International Business Machines Corporation | Video server streaming synchronization |
US6035307A (en) * | 1998-03-30 | 2000-03-07 | Bmc Software | Enterprise data movement system and method including opportunistic performance of utilities and data move operations for improved efficiency |
US6092086A (en) * | 1998-03-31 | 2000-07-18 | Bmc Software | System and method for handling backout processing during capture of changed data in an enterprise computer system |
US6144999A (en) * | 1998-05-29 | 2000-11-07 | Sun Microsystems, Incorporated | Method and apparatus for file system disaster recovery |
GB2345158A (en) | 1998-12-23 | 2000-06-28 | Ibm | Publish and subscribe data processing with ability to specify a local publication/subscription |
-
1998
- 1998-09-25 US US09/160,389 patent/US6411991B1/en not_active Expired - Lifetime
Cited By (27)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6816980B1 (en) | 2000-09-15 | 2004-11-09 | Zeronines Technology, Inc. | Fault tolerant, state-compatible computer system and method |
US6760861B2 (en) * | 2000-09-29 | 2004-07-06 | Zeronines Technology, Inc. | System, method and apparatus for data processing and storage to provide continuous operations independent of device failure or disaster |
US20040215596A1 (en) * | 2000-09-29 | 2004-10-28 | Fukuhara Keith T | System, method and apparatus for data processing and storage to provide continuous operations independent of device failure or disaster |
US20030159083A1 (en) * | 2000-09-29 | 2003-08-21 | Fukuhara Keith T. | System, method and apparatus for data processing and storage to provide continuous operations independent of device failure or disaster |
US8688787B1 (en) | 2002-04-26 | 2014-04-01 | Zeronines Technology, Inc. | System, method and apparatus for data processing and storage to provide continuous e-mail operations independent of device failure or disaster |
US7693879B2 (en) | 2002-11-14 | 2010-04-06 | Hitachi, Ltd. | Database system including center server and local servers |
US7197519B2 (en) | 2002-11-14 | 2007-03-27 | Hitachi, Ltd. | Database system including center server and local servers |
US20070143362A1 (en) * | 2002-11-14 | 2007-06-21 | Norifumi Nishikawa | Database system including center server and local servers |
US20040199609A1 (en) * | 2003-04-07 | 2004-10-07 | Microsoft Corporation | System and method for web server migration |
US7379996B2 (en) * | 2003-04-07 | 2008-05-27 | Microsoft Corporation | System and method for web server migration |
US7685465B1 (en) * | 2006-09-19 | 2010-03-23 | United Services Automobile Association (Usaa) | High-availability data center |
US7600148B1 (en) * | 2006-09-19 | 2009-10-06 | United Services Automobile Association (Usaa) | High-availability data center |
US7747898B1 (en) * | 2006-09-19 | 2010-06-29 | United Services Automobile Association (Usaa) | High-availability data center |
US8010831B1 (en) * | 2006-09-19 | 2011-08-30 | United Services Automobile Association (Usaa) | High availability data center |
US8402304B1 (en) | 2006-09-19 | 2013-03-19 | United Services Automobile Association (Usaa) | High-availability data center |
US9612923B1 (en) * | 2006-09-19 | 2017-04-04 | United Services Automobile Association | High-availability data center |
US8812896B1 (en) | 2006-09-19 | 2014-08-19 | United Services Automobile Association | High-availability data center |
US8874795B2 (en) * | 2006-11-30 | 2014-10-28 | Seoul National University Industry Foundation | Data synchronization system |
US20080130812A1 (en) * | 2006-11-30 | 2008-06-05 | Seoul National University Industry Foundation | Data synchronization system |
US8850035B1 (en) * | 2007-05-16 | 2014-09-30 | Yahoo! Inc. | Geographically distributed real time communications platform |
US8171111B1 (en) * | 2008-08-07 | 2012-05-01 | United Services Automobile Association (Usaa) | Systems and methods for non-specific address routing |
US11683263B1 (en) | 2008-08-21 | 2023-06-20 | United Services Automobile Association (Usaa) | Preferential loading in data centers |
US11044195B1 (en) | 2008-08-21 | 2021-06-22 | United Services Automobile Association (Usaa) | Preferential loading in data centers |
US20100131545A1 (en) * | 2008-11-26 | 2010-05-27 | Yahoo! Inc. | Distribution Data Items Within Geographically Distributed Databases |
US8495021B2 (en) * | 2008-11-26 | 2013-07-23 | Yahoo! Inc. | Distribution data items within geographically distributed databases |
US8495017B2 (en) * | 2011-08-03 | 2013-07-23 | Amadeus S.A.S. | Method and system to maintain strong consistency of distributed replicated contents in a client/server system |
US20130036092A1 (en) * | 2011-08-03 | 2013-02-07 | Amadeus S.A.S. | Method and System to Maintain Strong Consistency of Distributed Replicated Contents in a Client/Server System |
Also Published As
Publication number | Publication date |
---|---|
US6411991B1 (en) | 2002-06-25 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US6411991B1 (en) | Geographic data replication system and method for a network | |
US7139809B2 (en) | System and method for providing virtual network attached storage using excess distributed storage capacity | |
US7370336B2 (en) | Distributed computing infrastructure including small peer-to-peer applications | |
US6892221B2 (en) | Data backup | |
CA2270462C (en) | Regeneration agent for back-up software | |
US7047300B1 (en) | Survivable and scalable data system and method for computer networks | |
US7254636B1 (en) | Method and apparatus for transparent distributed network-attached storage with web cache communication protocol/anycast and file handle redundancy | |
US5917997A (en) | Host identity takeover using virtual internet protocol (IP) addressing | |
US6202085B1 (en) | System and method for incremental change synchronization between multiple copies of data | |
US5774660A (en) | World-wide-web server with delayed resource-binding for resource-based load balancing on a distributed resource multi-node network | |
US6543004B1 (en) | Method and apparatus for archiving and restoring data | |
US7047377B2 (en) | System and method for conducting an auction-based ranking of search results on a computer network | |
US7356589B2 (en) | Content collection | |
US20020169889A1 (en) | Zero-loss web service system and method | |
WO2002052381A2 (en) | System and method for intelligently distributing content over a communications network | |
EP1399836A2 (en) | Continuous availability updating of multiple memories | |
JP2004536406A (en) | Method and apparatus for session replication and failover | |
JP4958951B2 (en) | Content collection | |
JP2004252938A (en) | Wide area storage localization system | |
US6219799B1 (en) | Technique to support pseudo-names | |
US8010627B1 (en) | Virtual content publishing system | |
Burkard | Herodotus: A peer-to-peer web archival system | |
Litiu et al. | Stateful group communication services | |
Stanek | Microsoft Exchange Server 2010 Administrator's Pocket Consultant | |
Li | Keeping TCP connections intact across server failures |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
CC | Certificate of correction | ||
FPAY | Fee payment |
Year of fee payment: 4 |
|
FPAY | Fee payment |
Year of fee payment: 8 |
|
FPAY | Fee payment |
Year of fee payment: 12 |
|
AS | Assignment |
Owner name: DEUTSCHE BANK TRUST COMPANY AMERICAS, NEW YORK Free format text: GRANT OF FIRST PRIORITY AND JUNIOR PRIORITY SECURITY INTEREST IN PATENT RIGHTS;ASSIGNOR:SPRINT COMMUNICATIONS COMPANY L.P.;REEL/FRAME:041895/0210 Effective date: 20170203 |
|
AS | Assignment |
Owner name: SPRINT COMMUNICATIONS COMPANY L.P., KANSAS Free format text: TERMINATION AND RELEASE OF FIRST PRIORITY AND JUNIOR PRIORITY SECURITY INTEREST IN PATENT RIGHTS;ASSIGNOR:DEUTSCHE BANK TRUST COMPANY AMERICAS;REEL/FRAME:052969/0475 Effective date: 20200401 Owner name: DEUTSCHE BANK TRUST COMPANY AMERICAS, NEW YORK Free format text: SECURITY AGREEMENT;ASSIGNORS:T-MOBILE USA, INC.;ISBV LLC;T-MOBILE CENTRAL LLC;AND OTHERS;REEL/FRAME:053182/0001 Effective date: 20200401 |
|
AS | Assignment |
Owner name: SPRINT SPECTRUM LLC, KANSAS Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:DEUTSCHE BANK TRUST COMPANY AMERICAS;REEL/FRAME:062595/0001 Effective date: 20220822 Owner name: SPRINT INTERNATIONAL INCORPORATED, KANSAS Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:DEUTSCHE BANK TRUST COMPANY AMERICAS;REEL/FRAME:062595/0001 Effective date: 20220822 Owner name: SPRINT COMMUNICATIONS COMPANY L.P., KANSAS Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:DEUTSCHE BANK TRUST COMPANY AMERICAS;REEL/FRAME:062595/0001 Effective date: 20220822 Owner name: SPRINTCOM LLC, KANSAS Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:DEUTSCHE BANK TRUST COMPANY AMERICAS;REEL/FRAME:062595/0001 Effective date: 20220822 Owner name: CLEARWIRE IP HOLDINGS LLC, KANSAS Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:DEUTSCHE BANK TRUST COMPANY AMERICAS;REEL/FRAME:062595/0001 Effective date: 20220822 Owner name: CLEARWIRE COMMUNICATIONS LLC, KANSAS Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:DEUTSCHE BANK TRUST COMPANY AMERICAS;REEL/FRAME:062595/0001 Effective date: 20220822 Owner name: BOOST WORLDWIDE, LLC, KANSAS Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:DEUTSCHE BANK TRUST COMPANY AMERICAS;REEL/FRAME:062595/0001 Effective date: 20220822 Owner name: ASSURANCE WIRELESS USA, L.P., KANSAS Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:DEUTSCHE BANK TRUST COMPANY AMERICAS;REEL/FRAME:062595/0001 Effective date: 20220822 Owner name: T-MOBILE USA, INC., WASHINGTON Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:DEUTSCHE BANK TRUST COMPANY AMERICAS;REEL/FRAME:062595/0001 Effective date: 20220822 Owner name: T-MOBILE CENTRAL LLC, WASHINGTON Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:DEUTSCHE BANK TRUST COMPANY AMERICAS;REEL/FRAME:062595/0001 Effective date: 20220822 Owner name: PUSHSPRING, LLC, WASHINGTON Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:DEUTSCHE BANK TRUST COMPANY AMERICAS;REEL/FRAME:062595/0001 Effective date: 20220822 Owner name: LAYER3 TV, LLC, WASHINGTON Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:DEUTSCHE BANK TRUST COMPANY AMERICAS;REEL/FRAME:062595/0001 Effective date: 20220822 Owner name: IBSV LLC, WASHINGTON Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:DEUTSCHE BANK TRUST COMPANY AMERICAS;REEL/FRAME:062595/0001 Effective date: 20220822 |