WO1997040441A1

WO1997040441A1 - Network acceleration system

Info

Publication number: WO1997040441A1
Application number: PCT/US1997/006569
Authority: WO
Inventors: Eugene M. Walden
Original assignee: Asante Technologies, Inc.
Priority date: 1996-04-23
Filing date: 1997-04-23
Publication date: 1997-10-30
Also published as: AU2735997A

Abstract

The present invention, generally speaking, provides a method and apparatus for accelerating data transfer across a network. In one embodiment the invention consists of one or more modules which install on client (111) and server (112) computers and which deliver significant performance enhancements to users transferring files over a network (17), between clients or between clients and a server. The network may be an Ethernet or Fast Ethernet network that uses the AppleTalk protocol. One method involves intercepting selected calls to a first handler employing a first data transfer protocol and handling those calls by a second handler in accordance with a second data transfer protocol capable of a higher data transfer rate. Other calls to the first handler are not intercepted but are passed through. The first data transfer protocol maybe the AppleTalk protocol, and second data transfer protocol may be TCP/IP, for example, in which case the data transfer rate may be increased by as much as an order or magnitude. By intercepting only selected calls and passing through remaining calls, transparency of operation is achieved.

Description

NETWORK ACCELERATION SYSTEM

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to data transfer between networked computers.

2. State of the Art

Computer networks are becoming widespread in the workplace. Networks may employ various physical links, such as Ethernet, token ring, etc. Networks may also use various protocol architectures. Preferably, a network protocol architecture should be link-independent in order to be able to use further technologies without the major cost of redesigning the protocol architecture.

An example of a link-independent protocol architecture is the AppleTalk™ protocol architecture. The AppleTalk protocol architecture provides virtually unsurpassed ease of use. An AppleTalk network is "plug-and-play," meaning that the user can plug a computing device into the network and use it immediately without any of the complications of configuration. Also in an AppleTalk network, the network system is transparent, forming a seamless extension of the user's computer. Despite these considerable advantages, the transfer of large files in accordance with the AppleTalk protocol architecture is often slower than other protocols. The AppleTalk protocol architecture includes the Apple Transfer Protocol (ATP), used for transferring files across the network. ATP is a "stop- and wait" protocol, meaning that after some discrete amount of data has be sent, the sender stops and waits for an acknowledgment from the receiver.

Furthermore, the packet size used in ATP is small (only 512 bytes). To transfer a large file therefore, the file must be broken up into a very large number of packets, each of which incurs some transmission overhead. Referring more particularly to Figures 1 and 2, physical and logical views of a portion of a conventional computer network are shown. A computer 111 is connected over a physical link 17 to a computer 112. In the example shown, the computer 111 is assumed to be operating as a client and the computer 112 is assumed to be operating as a server to transfer a large file to the client. The client may be a Macintosh client, for example. The server may be based on the Macintosh platform or on some other platform, such as UNIX, Windows NT, etc.

Both the client 111 and the server 112 are assumed to be running the AppleShare protocol. AppleShare is a protocol that allows a computer to act as a file server on an AppleTalk network. Just as Novell's NetWare has Internet Packet Exchange (IPX) as its underlying transport protocol and Sun's Network File System (NFS) has TCP/IP as its underlying protocol, likewise AppleShare has AppleTalk as its underlying transport protocol. AppleShare is built upon and refers primarily to the Apple Filing Protocol (AFP).

More particularly, AppleTalk itself simply has the notion of two computers exchanging data. Layered on top of AppleTalk is the Apple Session Protocol, which maintains the notion of a connection between two computers that must be opened and closed. The Apple Transaction Protocol allows two computers that have an open session to issue transactions from the client application to the server application. There are many applications that use the Apple Transaction Protocol. For instance, a database application might run on a Macintosh server. Clients might change records in the database by issuing Apple Transaction Protocol transactions.

AppleShare is simply another application that is layered on top of the Apple Transaction Protocol. Once a client and server have established an open session using the Apple Session Protocol, the AppleShare client sends AFP transactions to the AppleShare server using Apple Transaction Protocol. The following is a partial list of the AFP transactions that comprise AppleShare:

1. Login to server

2. Mount volume on server 3. List directory on volume

4. Open file on volume

5. Read from open file

6. Write to open file

7. Close open file 8. Unmount volume

9. Logout from server

Referring now to Figure 2, a sequence of operations performed in accomplishing the transfer of a file from the server to the client of Figure 1 will be described. First, on the client side, an application 201 running on the client computer detects a user event calling for transfer of a file from the server to the client, e.g., an Open command. The application 201 might be a graphics application such as Adobe Photoshop, whose files are typically many megabytes in size. The application calls an operating system File Manager 203 . The File Manager in turn passes the request to an AppleShare External File System (XFS) extension 205. This extension allows the computer to operate as a client of an AppleShare server. The AppleShare XFS is layered on top of the AppleTalk protocol stack 207. For file transfers, the AppleShare XFS relies in particular on two protocols within the AppleTalk protocol stack, Apple Filing Protocol (AFP) and Apple Transaction Protocol (ATP). ATP is comparatively slow in terms of the realizable data transfer rate. Although AppleTalk includes Apple Data Streaming Protocol (ADSP), this protocol is not used by the AppleShare XFS. Further information concerning the AppleTalk protocol stack may be found in Sidhu et al., Inside AppleTalk, Second Edition, Addison Wesley, 1990, incoφorated herein by reference. The AppleTalk stack 207 calls driver software 209, which performs actual transactions over an expansion bus 211 of the computer, such as NuBus • or PCI, to cause a network card 213 to transfer packets over the wire 214. Information is transferred in 512-byte packets in accordance with a stop-and- wait protocol, as described previously.

On the server side, the network card 215 receives the packets, communicates them across the computer bus 217 to the card driver 219, from whence the packets pass through the AppleTalk protocol stack 221 to the AppleShare server software 223. The AppleShare server software 223 invokes the File Manager 225, which invokes a further manager such as SCSI Manager 227 for performing a disk read of the requested file. Because of the small packet size, the result of the request is a large number of small disk reads (or, in the case of a write request, a large number of small writes).

Currently, the average size of files on the network is growing with the increasing use of multimedia objects. As files grow larger, the inadequacy of the current version of AppleShare to handle large files becomes increasingly apparent.

What is needed then is a mechanism for accelerating the transfer of large files across a network such as an AppleShare network transparently without compromising other advantageous features of that network.

SUMMARY OF THE INVENTION

The present invention, generally speaking, provides a method and apparatus for accelerating data transfer across a network. In one embodiment, the invention is provided in software and consists of one or more modules which install on client and server computers and which deliver significant performance enhancements to users transferring files over a network, between clients or between clients and a server. The network may be an Ethernet or Fast Ethernet network that uses the AppleTalk protocol. One method in accordance with the invention involves intercepting selected calls to a first handler employing a first data transfer protocol and handling those calls by a second handler in accordance with a second data transfer protocol capable of a higher sustained data transfer rate. Other calls to the first handler are not intercepted but are passed through. The first data transfer protocol may be the AppleTalk protocol, for example, and the second data transfer protocol may be TCP/IP, for example, in which case the data transfer rate may be increased by as much as an order of magnitude. By intercepting only selected calls and passing through remaining calls, transparency of operation is achieved.

BRIEF DESCRIPTION OF THE DRAWING

The present invention may be further understood from the following description in conjunction with the appended drawing. In the drawing:

Figure 1 is a physical view of a conventional network computer system; Figure 2 is a logical view of the network computer system of Figure 1 ; Figure 3 is a logical view of a network computer system in accordance with the present invention, wherein a file request is being communicated from a network accelerated client to a network accelerated server;

Figure 4 is a logical view of a network computer system in accordance with the present invention, wherein a requested file is being communicated from the network accelerated client to the network accelerated server;

Figure 5 is a block diagram of a file handler chain including the network accelerated client software of the present invention;

Figure 6 is a diagram of an packet format used to identify whether a target server is a network accelerated server;

Figure 7 and Figure 8 are flowcharts of operations performed by the network accelerated client software and server software, respectively; and Figure 9 is a diagram of the format of a UDP packet sent from the network accelerated client to the network accelerated server.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Referring now to Figure 3, there is shown a logical view of a network computer system in accordance with the present invention. On the client side, an application 301 running on the client computer detects a user event calling for transfer of a file from the server to the client, e.g., an Open command, as previously described. The application calls an operating system File Manager 303. The File Manager in turn passes the request to network accelerated client software 305 (referred to hereinafter as an "N.D client," where N.D. stands for NetDoubler™). If the target file resides on a server not provided with network accelerated server software (an "N.D. server"), then the request is passed through the N.D. client on to the AppleShare XFS 307 and handled by the AppleShare XFS in the conventional manner. If the target file resides on a server provided with N.D. server software, however, then the request is handled by the N.D. client software.

Unlike the AppleShare XFS, which relies on ATP, a transaction protocol, the N.D. client software relies on a data streaming protocol. That protocol may be ADSP, which is part of AppleTalk. Because ADSP is a data streaming protocol instead of a stop-and-wait protocol like ATP, data transfer rates may be improved several-fold. ADSP is still limited to a small packet size of 512 bytes, however. Therefore, instead of using the AppleTalk protocol stack, the N.D. client preferably uses a separate protocol stack 306 capable of achieving high data transfer speeds. Although the particular protocol chosen is not critical, suitable protocols include Transmission Control Protocol (TCP) and Asynchronous Transfer Mode (ATM). In general, the protocol should be a data streaming protocol that allows for high speed data transfer.

The N.D. client software calls driver software 311, which performs actual transactions over an expansion bus 313 of the computer, such as NuBus or PCI, to cause a network card 315 to transfer packets over the wire 317. Information is transferred, not in 512-byte packets in accordance with a stop- and-wait protocol as described previously, but rather in full MTU-size (Maximum Transfer Unit) packets in accordance with an appropriate data streaming protocol. MTU refers to the largest packet size that can be sent on a particular network medium.

On the server side, the network card 319 receives the packets, communicates them across the computer bus 321 to the card driver 323, from whence the packets pass either 1) through the AppleTalk protocol stack 325 to the AppleShare server software 327 in the case of a conventional AppleShare server, or 2) through an N.D. server 329 based on appropriate protocol stack such as TCP. The AppleShare server software 325 or the N.D. server software 329 invokes the File Manager 331, which invokes a further manager such as SCSI Manager 331 for performing a disk read of the requested file. In the case of an N.D. server, the result of the request is a relatively small number of large disk reads (or, in the case of a write request, a small number of large writes). In a preferred embodiment, accelerated disk reads and writes are performed in blocks of 128KB.

One important aspect of the present network acceleration software involves intercepting selected calls to handler based on the default protocol and rerouting those calls through the handler based on the alternative protocol. Another important aspect involves establishing a mapping between AppleTalk addresses, for example, on the one hand, and IP addresses, for example, on the other hand. More generally, a mapping must be established between the addresses for the "default" protocol and the "alternative" protocol, where the alternative protocol offers higher file transfer performance.

Referring to Figure 5, the N.D. client "traps" selected calls to the AppleShare XFS by inserting itself ahead of the AppleShare XFS in a chain of file handlers maintained by the operating system of the client machine. In the case of the Macintosh, for example, an external file system hook, "FSHOOK," is provided that points to the head of a file handler chain. The file handler chain might include, for example, the N.D. client (501), AppleShare XFS (503) and DOS XFS (505). When the File Manager makes a call to open a file, it is passed to the first file handler in the file handler chain. The call includes a volume and a file system ID of the file. Each file handler receives the call in turn and determines whether or not, based on the volume and file system ID, the call pertains to that handler. If so, the handler takes the call. If not, the handler passes the call on to the next handler in the chain.

Each handler, when it is loaded, inserts itself at the front of the file handler chain. So long as the N.D. client is loaded after AppleShare XFS, the N.D. client will precede AppleShare XFS in the file handler chain and will be able to intercept selected File Manager calls. In particular, the N.D. client intercepts File Manager calls pertaining to open files, including the following: OPEN, CLOSE, READ and WRITE. Although the N.D. client may intercept certain other calls as well, the foregoing calls are of principal interest. The bulk of File Manager calls are not intercepted by the N.D. client but are instead passed on to the AppleShare XFS to be handled in the normal manner.

Furthermore, even in the case of OPEN, CLOSE, READ and WRITE, these calls are intercepted only if the server on which the volume resides is a network accelerated server, i.e. , has N.D. software installed. When the AppleShare XFS is called, it is passed a volume control block (VCB). The AppleTalk address of the server is maintained in a special section of the VCB. Given the server's AppleTalk address, the AppleTalk Name Binding Protocol (NBP) provides a convenient way of determining whether the target server is an N.D. server. Equally important, if the target server is an N.D. server, the AppleTalk Name Binding Protocol (NBP) provides a convenient way of establishing a mapping between the AppleTalk address of that server and an IP address of that server. These functions are performed in the following manner. N.D. servers in effect "advertise" themselves on the network through NBP by making an entry in a name table maintained by NBP within the server's node. Entries are of a form shown in Figure 6. The entity address 610 is the AppleTalk address of the node. The entity name includes an Object name 621, a Type name 623 and a Zone name 625. The Zone name 625 is node- specific and is determined in accordance with AppleTalk' s Zone Information Protocol (ZIP). The Object name 621 and the Type name 623 are determined by the particular entity registering with NBP. When an N.D. server enters itself in the name table, it sets the Type name 623 to a string (e.g. "NETDOUBLER") identifying the server as an N.D. server. The Object name 621 is set so as to identify an IP address and UDP (User Datagram Protocol) port number of the node, as follows: where is the IP address and UUUU is the UDP port. The UDP port is the port used by the network accelerated server software to receive requests from the N.D. client software.

Comparing Figure 3 and Figure 4, Figure 3 illustrates sending an N.D. request to the UDP port of the N.D. server 329. In Figure 4, the N.D. server responds by "blasting" back file transfer blocks from a TCP port of the server. Because the file transfer rate is much greater using TCP than using ATP, the TCP path is shown as being thicker than the conventional ATP path, which continues to be used for calls not requiring file transfer and for calls to non- N.D. servers.

The overall process followed on the client side is illustrated in Figure 7. When a call is received from the File Manager, the call includes the AppleTalk node address of the server to which the call pertains. The N.D. client software checks to see if the call is one dealing with open files, namely OPEN, CLOSE, READ or WRITE (S701). If the request is an OPEN request (i.e., the file has not already been opened, as determined in Step S703), then the N.D. client performs an NBP lookup for the "NETDOUBLER" service on the specified node (S705). If the NETDOUBLER service is not found on that node (S707), then the call is passed through from the N.D. client software to the normal AppleShare XFS (S709) to be handled in a conventional manner. The client only checks to see if a server has NetDoubler running on it when the user on the client machine mounts a volume (logs into the server). If no NetDoubler service is found on the server, then the client doesn't check again. If the user subsequently unmounts the volume and remounts it, then the check will happen again upon re- mounting the volume.

If the NETDOUBLER service is found on that node, then the name of the NETDOUBLER service on that node, which includes the IP/UDP address of the network accelerated server software on that node, is returned to the network accelerated client software. The N.D. client then sends a file request to the N.D. server (Step S711). Calls from the File Manager are translated into one or more of the following requests: OPEN FILE, CLOSE FILE, FILL CACHEBLOCK, FLUSH CACHEBLOCK, and WRITE. The request is sent as a UDP packet to the UDP port of the network accelerated server. The UDP packet may have a format as shown in Figure 9. The packet includes an IP header 901, followed by a UDP header 903, followed by a Remote Procedure Call (RPC) header 905. A Remote Procedure Call (RPC) is a well-known mechanism used by a client to request a server to perform a particular service on behalf of the client. The RPC header includes various information such as a transaction ID, the client address, etc., followed by a procedure number corresponding to one of the foregoing procedures.

When the NetDoubler server sends an "Open" reply, the open reply returns a TCP port number for this file. When the client gets this port number from the open RPC reply, it then establishes a TCP connection to this port on the server. The connection is maintained as long as the file is opened, and the connection is torn down when the file is closed.

After the N.D. client sends an RPC to the N.D. server, it processes any reply (S713) and returns.

Referring to Figure 8, if the request is an OPEN FILE request, then the N.D. server, in response to the RPC, interacts with the File Manager portion of the operating system of the server platform to open the requested file. The File Manager assigns a file reference number, or REFNUM, to the file (S801), and the N.D. server maintains a table of REFNUMS of files being handled by it. The REFNUM is returned to the network accelerated client software, which also maintains a table of REFNUMS being handled by the network accelerated client, and returns the REFNUM to the application making the original request (S803). Note that other requests pertaining to the same file but not requiring file transfer continue to be handled through AppleTalk. Because AppleTalk and the network accelerated server software both invoke the services of the File Manager of the server platform, they effectively share the same state information, enabling transparent operation of the network accelerated client and server software.

Referring again to Figure 7, if in Step S703 the request pertains to a file that has already been opened, the N.D. client checks in Step 704 whether the file REFNUM matches the REFNUM of any of the files being handled by the N.D. client. If a match is found, then an RPC is sent to the N.D. server as previously described. If no match is found, then the call is passed to AppleShare XFS to be handle in the conventional manner.

As a result of the RPC, of Step S711 , a TCP flow of data is then established, either from the server to the client or from the client to the server. Because TCP is a streaming protocol and does not use stop-and-wait flow control, and because full MTU-size packets may be used, data transfer speeds may be achieved many times the speed that may be obtained using AppleTalk.

It will be appreciated by those of ordinary skill in the art that the invention can be embodied in other specific forms without departing from the spirit or essential character thereof. The foregoing description is therefore considered in all respects to be illustrative and not restrictive. The scope of the invention is indicated by the appended claims, and all changes which come within the meaning and range of equivalents thereof are intended to be embraced therein.

Claims

What is claimed is:

1. A method of accelerating file transfer across a computer network including a computer operating as a client computer and a computer operating as a server computer the method comprising the steps of: intercepting selected calls to a file handler employing a first data transfer protocol; and handling those calls by a file handler employing a second data transfer protocol capable of a higher sustained data transfer rate.

2. The method of Claim 1, wherein said selected calls include all calls pertaining to open files.

3. The method of Claim 2, wherein said calls include the following calls: OPEN, CLOSE, READ and WRITE.

4. The method of Claim 3, wherein the file handler employing the second data transfer protocol translates READ and WRITE calls into FILL CACHEBLOCK and FLUSH CACHEBLOCK Remote Procedure Calls.

5. The method of Claim 4, comprising the further steps of: the server computer, in response to said Remote Procedure Calls, exchanging data with the client computer using the second data transfer protocol.

6. The method of Claim 1 , wherein the file handler employing a first data transfer protocol is a preceding file handler in a file handler chain of the client computer, and the file handler employing a second data transfer protocol is a succeeding file handler in the file handler chain, wherein calls besides said selected calls are passed through from the preceding file handler to the succeeding file handler.

7. The method of Claim 1 , wherein the first data transfer protocol is a data transfer protocol within the AppleTalk protocol stack.

8. The method of Claim 7, wherein the second data transfer protocol is TCP/IP.

9. The method of Claim 1, wherein the computer network includes software for converting entity names into network addresses, the method comprising the further steps of: receiving as part of one of the selected calls a target network address; causing a lookup operation to be performed for a predetermined service name and receiving in response one or more packets each of which includes a network address of a computer provided with a service corresponding to the predetermined service name; and comparing the target network address with network addresses received in response to the lookup operation.

10. The method of Claim 9, comprising the further steps of: including as part of each of said packets an alternative network address in accordance with the second protocol; and if the target address matches a network address received in response to the lookup operation, sending a request to a corresponding alternative network address.

11. Client software for accelerating file transfers across a computer network, comprising program instructions stored on a machine-readable medium for: intercepting selected calls to a file handler employing a first data transfer protocol; and handling those calls by a file handler employing a second data transfer protocol capable of a higher sustained data transfer rate.

12. An accelerated computer network, comprising: a client computer including means for intercepting selected calls to a file handler employing a first data transfer protocol and handling those calls by a file handler employing a second data transfer protocol capable of a higher sustained data transfer rate; and a server computer including means for exchanging data with the client computer using the second data transfer protocol.