CA2030349C - Dynamic window sizing in a data network - Google Patents

Abstract

A method for use in a high-speed virtual circuit digital network for resizing windows of virtual circuits in nodes of the network. The resizing of a virtual circuit's window is initiated by an input router at an edge of the digital network. When the input router determines that resizing is necessary, it sends a first congestion control message to the nodes through which the virtual circuit passes. If the message indicates a larger window, the node receiving the message determines what size window it can provide and sends the message with that window size on to the nextnode. An output router at the other edge of the digital network receives the message, sets the window size based on the message as altered by the nodes, and returns asecond message with the final window size via the nodes. On receipt of the second message, the nodes alter their windows and the input router sends cells as permitted by the new window. Included in the method are novel techniques for determining the ideal window size for a virtual circuit, for determining at the input router when a change in window size is necessary, and for determining the size of the window in the nodes.

Description

20303'~9 DYNAMIC WINI)OW SIZING IN A DATA NETWORK

The present patent application contains the complete Detailed Description and Figures of copending C~n~ Application No. 2,029,054. The additional disclosure begins at the section titled "Overview of Window Sizing" and includes5 two new Figures 13 and 14.

Technical Field The present invention relates to data networks in general and more particularly to protocols, methods and apparatus that improve the flow of information within such networks.

10 Bacl~round of the Invention Packet-switched networks for the transport of digital data are well known in the prior art. Typically, data are transmitted from a host connecting to a network through a series of network links and switches to a receiving host. Messages from the transmitting host are divided into packets that are transmitted through the 15 network and reassembled at the receiving host. In virtual circuit networks, which are the subject of the present invention, all data packets transmitted during a single session between two hosts follow the same physical network path.
Owing to the random nature of data traffic, data may arrive at a switching node of the network at an instantaneous rate greater than the transmission speed of the 20 outgoing link, and data from some virtual circuits may have to be buffered until they can be transmitted. Various queuing disciplines are known in the prior art. Early data networks typically used some form of first-in-first-out (FIFO) queuing service. In FIFO service, data packets arriving from different virtual circuits are put into a single buffer and transmitted over the output link in the same order in which they arrived at 25 the buffer. More recently, some data networks have used queuing disciplines of round robin type. Such a network is described in a paper by A.G. Fraser entitled, "TOWARDS A UNIVERSAL DATA TRANSPORT SYSTEM", and printed in the IEEE Journal on Selected Areas in Communications, November 1983. Round robin service involves keeping the arriving data on each virtual circuit in a separate30 per-circuit buffer and transmitting a small amount of data in turn from each buffer that contains any data, until all the buffers are empty. U.S. Patent No. 4,583,219 to Riddle describes a particular round robin embodiment that *

203034q gives low delay to messages consisting of a small amount of data. Many other variations also fall within the spirit of round robin service.
First-in-first-out queuing disciplines are somewhat easier to implement than round robin disciplines. However, under heavy-traffic conditions first-in-first-out 5 disciplines can be unfair. This is explained in a paper by S. P. Morgan entitled, "QUEUEING DISCIPLINES AND PASSIVE CONGESTION CONTROL IN BYTE-STREAM NETWORKS," printed in the Proceedings of IEEE INFOCOM, April 1989.
When many users are contending for limited transmission resources, first-in-first-out queuing gives essentially all of the bandwidth of congested links to users who submit long 10 messages, to the exclusion of users who are attempting to transmit short messages. When there is not enough bandwidth to go around, round robin disciplines divide the available bandwidth equally among all users, so that light users are not locked out by heavy users.
On any data connection it is necessary to keep the transmitter from overrunning the receiver. For highly bursty data traffic, such as is characteristic of digital 15 data networks, this is commonly done by means of a sliding-window protocol, as described by A. S. Tanenbaum in the book COMPUTER NETWORKS, 2nd ed., published by Prentice Hall (1988), pp. 223-239. The transmitter sends data in units called frames, each of which carries a sequence number. When the receiver has received a frame, it returns the sequence number to the transmitter. The transmitter is permitted to have only a 2 0 limited number of sequence numbers outstanding at once; that is, it may transmit up to a specified amount of data and then it must wait until it receives the appropriate sequential acknowledgment before transmitting any new data. If an expected acknowledgment does not arrive within a specified time interval, the transmitter retransmits one or more frames.
The maximum number of bits that the transmitter is allowed to have in transit at any given 25 time is called the window size and will be denoted here by W. The maximum number of outstanding sequence numbers is also sometimes called the window size, but that usage will not be followed here.
Suppose that the transmitter and receiver are connected by a circuit of speed S bits per second with a round-trip propagation time To seconds, and that they 3 0 are able to generate or absorb data at a rate not less than S. Let W be the window size. Then, to maintain continuous transmission on an otherwise idle path, W must be at least as large as the round-trip window W0, where W0 is given by Wo=STo. W0 is sometimes called the delay-bandwidth product. If the circuit passes through anumber of links whose speeds are different, then S represents the speed of the 2~3~ 9 slowest link. If the window is less than the round-trip window, then the averagefraction of the network bandwidth that the circuit gets cannot exceed W/W0.
In principle, if a circuit has a window of a given size, buffer space adequate to store the entire window must be available at every queueing point to5 prevent packet loss in all cases, since fol ~vald progress can mofnentarily come to a halt at the beginning of any link. This is explained in more detail below. On a lightly loaded network, signifi~nt delays are unlikely and there can generally be sharing of buffer space between circuits. However, the situation is different when the network is congested. Congestion means that too much traffic has entered the10 network, even though individual circuits may all be flow controlled. Uncontrolled congestion can lead to data loss due to buffer overflow, or to long delays that the sender interprets as losses. The losses trigger retr~ncmicsions, which lead to an unstable situation in which network throughput declines as offered load increases.
Congestion instability comes about because whenever data has to be retransmitted, 15 the fraction of the network's capacity that was used to transmit the original data has been lost. In e~ R cases, a congested network can deadlock and have to be restarted.
Congestion control mPtho~1c are surveyed by T~nenb~nm, op. cit., pp.
287-88 and 309-320. Many cong~stion control methofls involve the statistical 20 sharing of buffer space in conjunction with trying to sense the onset of network congestion. When the onset of congestion is detecl~d, au~ll,pts are made to request or require hosts to slow down their input of data into the network. These techniques are particularly the ones that are subject to congestio~ instability. Abusive hosts may continue to submit data and cause buffer overflow. Buffer overflow causes 25 packet losses not only of a host submitting the packets that cause the overflow, but also of other host~s. Such packet loss then gives rise to retr~ncmicsion requests &om all users losing packets and it is this effect that pushes the network toward instability and ~e~ Alternatively, as m.qntir n~A above, it has been recognized for a long time that congestion inct~biliry due to data loss does not occur in a virtual-circuit 30 network, provided that a full window of memory is ~ cated to each virtual circuit at each queueing node, and provided that if a sender times out, it does not retransmit autom~ti~lly but first issues an inquiry message to determine the last &ame correctly received. If full per-circuit buffer ~lloc~tion is combined with an intrincic~lly fair queueing discipline, that is, some variant of round robin, the 35 network is stable and as fair as it can be under the given load.

20303~q The DATAKIT (Registered trademark) network is a virtual circuit network marketed by AT&T that operates at a relatively low transmission rate and provides full window buffering for every virtual circuit as just described. This network uses technology similar to that disclosed in U.S. patent Re 31,319, which reissued on July 19, 1983 from A. G. Fraser's patent 3,749,845 of July 31, 1973, and operates over relatively low-speed T1 channels at approximately 1.5 megabits per second. The DATAKIT network is notsubject to network instability because of full-window buffering for each virtual circuit and because data loss of one host does not cause data loss of other users. Dedicated full-window buffering is reasonable for such low-speed channels; however, the size of a data lo window increases dramatically at speeds higher than 1.5 megabits per second, such as might be used in fiber-optic transmission. If N denotes the maximum number of simultaneously active virtual circuits at a node, the total buffer space that is required to provide a round-trip window for each circuit is NSTo. It may be practicable to supply this amount of memory at each node of a low-speed network of limited geographical extent.
However, at higher speeds and network sizes, it ultimately ceases to be feasible to dedicate a full round-trip window of memory for every virtual circuit. For example, assuming a nominal transcontinental packet round-trip propagation time of 60 ms, a buffer memory of 11 kilobytes is required for every circuit at every switching node for a 1.5 megabits per second transmission rate. This increases to 338 kilobytes at a 45 megabits per second rate.
2 0 A need exists for solutions to the problem of avoiding congestion instability, while at the same avoiding the burgeoning buffer memory requirements of known techniques. It is therefore an overall object of the present invention to retain the advantages of full-window buffering while substantially reducing the total amount of memory requlred.
2 5 It is another object of the invention to reduce the amount of buffering required for each circuit by the sharing of buffer memory between circuits and by dynamic adjustment of window sizes for circuits.
U.S. Pat. No. 4,736,369 to Barzilai et al. addresses some aspects of the problem of adjusting window sizes dynamically during the course of a user session, 3 0 in response to changes in traffic patterns and buffer availability. However, this patent assumes a network in which flow control and window adjustment are done on a link-by-link basis, that is, as a result of separate negotiations between every pair of adjacent nodes on the path between transmitter and receiver. For high-speed networks, link-by-link flow control is generally considered to be less suitable than end-to-end control, because of the additional computing load that link-by-link control puts on the network nodes.
Thus, it is an another object of the invention to perform flow control on an end-to-end basis with dynamically adjustable windows.

S Summa~y of the Invention In accordance with one aspect of the invention there is provided in a virtual circuit data network having one or more nodes, a method of adjusting the size of a window Wk, in the node for the kth virtual circuit using a given link from the node, the method comprising the steps of: deterrnining the average number of active virtual circuits Jk l using the link prior to the addition of the kth virtual circuit; and adjusting the size of the window Wk l for the kth virtual circuit from the equation Wk-l =
Wo ( Jk~

where W0 is a full-speed round trip window.
In accordance with another aspect of the invention there is provided in a virtual circuit data network having one or more nodes, a method of adjusting the size of a window Wk l in the node for the kth virtual circuit using a given link from the node, the method comprising the steps of: determinin~ that there is to be a fixed number M of window sizes; dividing the number N of virtual circuits which can exist simultaneously in the network into a set of M+1 break points such that 0=No<NI< . . . <NM=N; associating M window sizes WO>W,> . . . >WM I with the break points No through NM-I such that Nj is associated with Wj; determining theaverage number of active virtual circuits JNj using the link prior to the addition of the kth virtual circuit; and adjusting the size of the window Wj associated with virtual circuit Nj from the equation Wi - 1 2 5 WO JNi where Nj<k<Nj+, and W0 is a full-speed round trip window.

20303'~9 - 5a-In accordance with yet another aspect of the invention there is provided a method employed at an edge of a virtual circuit data network in which data is transported in cells to determine the size of the window to be requested in the network for a given virtual circuit origin~ting at the edge, the method comprising the S steps of: keeping track of the total number of cells TC that the given virtual circuit has at the edge and in the network; keeping track of the cumulative throughput CT
of cells of the given virtual circuit during a most recent renegotiation interval; and requesting a window for the virtual circuit in the network which is large enough to accommodate a number of cells which is the larger of TC and the product of CT
10 times the quotient of the round-trip time divided by the length of the most recent renegotiation interval.
More specifically, the invention is a method for controlling the congestion of data cells in a network which has one or more nodes and in which the data cells are transferred across the network via virtual circuits. The steps of the 15 method include: when a virtual circuit is established in the network, assigning a default cell buffer for the virtual circuit at each node; and for the duration of the virtual circuit, performing steps including: determining from the behaviour of the virtual circuit whether a different-sized cell buffer is needed; if a different-sized cell buffer is needed, providing a first resizing signal indicating the different size to the 20 nodes in the virtual circuit; in each node of the virtual circuit, responding to the first resizing signal by indicating the buffer size which the node can provide if that is less than the different size; and in each node of the virtual circuit, setting the cell buffer for the virtual circuit to a size no larger than the smallest buffer size indicated by any of the nodes. Other aspects of the invention include a method for deterrninin~
25 the cell buffer size, a method for deterrnining the size of the cell buffer to be requested, and a method for deterrnining the size of cell buffer which the node can provide. The methods for determining the cell buffer size and determining the size of the cell buffer which the node can provide involve a constant h, which is derived from the equation renegotiation interval=hT, where T iS the longest round trip time in the network and the renegotiation interval is an interval of time after which the resizing of the virtual circuit's cell buffer is automatically reconsidered.

.. , 20303~
- 5b-Brief Description of the Drawin~
In the drawing, FIG. I discloses the architecture of a typical data switching network having a plurality of switching nodes connected to user packet host sources and 5 destinations;

2 0 ~ 9 FIG. 2 discloses illustrative details of a data receiving and queuing arrangement at a node for an incoming channel having a plurality of multiplexed time slots coll~s~nding to individual virtual circuits;
FIG. 3 discloses illustrative details of a controller of FIG. 2 that 5 ,s~Amini~ters the buffer space allocation and data queuing of virtual circuits on an incoming ch-s-nn~l;
FIG. 4 discloses illustrative details of a router that converts between variable-length data packets from a host and constant-length data cells and further aAminist~rs the buffer space allocation and data queuing at the router, and FIG. S shows an illustrative method of dct~ lining buffer lengths of data for a virtual circuit at a router or switching node;
FIG. 6 and 7 show illustrative flowcharts depicting the protocols and method steps performed at the routers and nodes of dyn~mic~lly ~ll~s~ting buffer space for a virtual circuit at routers and nodes for an emboAimçn~ in which buffer 15 lengths at input routers are used as decision criteria for dynamic buffer allocation;
FIG. 8 through 12 disclose flowch6l~ depicting the protocols and method steps pclrol~d at the routers and nodes for sllocating buffer space in blocks of fixed or valying sizcs to virtual circuits in an emb~l;~ in which buffer lengths at nodes are used in conjunction with buffer lcngths at routers as decision criteria;
FIG. 13 shows a congçstion control field; and FIG. 14 is a diagram showing how cell buffer sizes are adjusted in response to a request.

Detailed Description FIG. 1 shows a block diagram of an illustrative packet-switching 25 nCIWC~I~. It is sccvm~ that thc r,clwo,k i~t~;onnects many packet sources andAej,l;ns~ nc by mcans of virtual circuits among a nu,llbcr of routers and switching nodcs. P. cket sources . nd destin-s-~iQnc arc attachcd to local area nelwo,ks that are on user sites. For example, a source 102 is co~ln~ct~ to a local nc.wu,k 106, which is connected to a router 110. One of the functions of the router is to convert between 30 the variable-length data packets issued by the source . nd the constant-length data cells trsncmit~e~ and swi~hcd by the cell nelwo,k 100. While cells are considered to be of fL~ced length, this is not a limitstion of the invention. Other fun.;lions of the router relevant to the invention will be described below.

The router attaches the local network 106 to the cell network 100 via the access line 108. Data cells belonging to a particular virtual circuit are transmitted through a sequence of switching nodes 114 and data links 116 to an access line 118 that is connected to a router 120. The router 120 reassembles the data cells into data packets 5 addressed to a particular destination, and transmits the packets to the local network 124, from whence they are taken by the destination 128.
It is assumed for purposes of disclosure that the network 100 is similar to the DATAKIT (R) virtual circuit network marketed by AT&T, except that the network 100 operates at a considerably higher transmission rate. That is, it is assumed that network 100 10 establishes a virtual circuit path between a source router and a destination router via selected ones of the switching nodes 114 when a connection is first initiated. Packets passing from a source to a destination are routed via the virtual circuit for the duration of the connection, although the actual transmission lines and bandwidth on the transmission lines in the path are not dedicated to the connection in question, but might be time-shared 15 among many such connections.
In accordance with the invention, FIG. 2 shows an illustrative embodiment of a cell buffering arrangement at a node. This buffering arrangement is able to handle many virtual circuits. Buffer space is allocated per-virtual-circuit and the allocation for a virtual circuit can be changed dynamically, under control of the monitor 200. The monitor is a 20 conventional microprocessor system that is used to implement congestion control mech~ni~m~ to be described later. The receiver 202 and transmitter 204 in the figure are conventional, and the transmitter may implement round robin service among the virtual circuits using established techniques.
When a cell arrives, the receiver 202 determines whether the cell is a 2 5 congestion message as indicated by a bit in the header. Congestion messages are stored in a separate FIFO queue 206 for the monitor. If an arriving cell is not a congestion message, the receiver 202 produces a virtual circuit number on bus WVC and a write request on lead WREQ. The receiver places the cell on its output bus 208 where it is buffered in the cell queue 210 under the control of the controller 212. The cell queue 210 3 0 is a memory array of some suitable size, which for the purposes of exposition is organized in words which are one cell wide.
The receiver 202 and the transmitter 204 are autonomous circuits. Each operates independently of the other to enter cells to and remove cells from the cell queue 210, respectively. When the transmitter 204 is ready to send a cell, it ~031D349 produces a virtual circuit number on bus RVC and a read request on lead RREQ. Ifthe allocated buffer in queue 210 associated with virtual circuit RVC is empty, the controller 212 willin-lirate this condition by setting signal EM~ to a value of TRUE and the trqncminer can try another virtual circuit. Otherwise, the next cell in S the buffer associated with RVC will appear on the output bus to be read by thetrqncmitter204. The controller 212 controls the cell queue via signals on bus MADDR and leads MW and MR. MADDR is the address in the cell queue 210 at which the next cell is to be written or read. MW and MR signify a queue write orread operation, ~;.~ /ely. Co~gestir,n mçssqvges ge.lelatcd by the monitor 200 are 10 stored in a separate outgoing FIFO 214. These mçss~ges are multiplexed with outgoing cells onto the trqncmic~ion line 216 by the l~ns~
To implement COI gestion control schemes, the ,l,onilor 200 has access to data structures internal to the controller 212 over the buses ADDR, R, W, andDATA. These data structures include the incl~n-Anço-lc buffer length for each 15 virtual circuit and the overall number of cells in the cell queue. Averaging operations ç~uhcd to implement congestion control, accolding to the protocols described below, are pe.rolllled by the Illol~itor 200.
FIG. 3 shows illustrative details of the controller 212 of FIG. 2. The major funrtionc of the controller are to keep track of the buffer qllocation for each 20 virtual circuit, to keep track of the in~l~. IAnrouc buffer use (buffer length) for each virtual circuit, to manage the qllrxation of memory in the cell queue such that data can be buffered for each virtual circuit in a ~lellirqte~l buffer of dynqmil~qlly varying length, and to control the writing and reading of data in the cell queue as it is received and ~ c~ ed For the ~ oses of exposition) memory is partitioned in the queue in units of one cell. This section first describes the basic elements of the controller, and then describes the operations of these elçmentc in detail.
An arbiter 300 receives signals WREQ and RREQ, which are requests to write a cell to a buffer qccociqted with a particular virtual circuit or to read a cell from the buffer assc~ ated with a particular virtual circuit, ~ ~c~ ely. The arbiter 30 insures that read and write operations occur in a non-int~.r~,ling manner, and that the select input to the multiplexer (W OR R) is set such that input RVC is present on bus VC during read operations and input WVC is present on bus VC during write operations. The remqinder of this ~i~cuccion will concide~ read and write operations separately.

2 ~ 9 A table COUNT_TABLE 304 is provided for storing the buffer allocation and buffer use for each virtual circuit. The table is addressed with a virtual circuit number on bus VC from the multiplexer 302. Each virtual circuit has two entries in COUNT_TABLE. One entry, L~MIT[VC], contains the maximum 5 number of cells of data that virtual circuit VC is presently allowed to buffer. This, in turn, determines the window size allocated to the virtual circuit. The second entry, COUNT[VC], contains the number of cells that are pl~,senlly used in the cell queue 210 by virtual circuit VC. The contents of COUNT_TABLE can be read or written by the ll~onilol 200 at any time before or during the operation of the controller 212.
A table QUEUE POINTERS 306 contains the read and write pointers for the buffer associated with each virtual circuit. Read pointer RP[VC] references the location cont~ining the next cell to be read from the buffer associated withvirtual circuit VC; write pointer WP[VC] references the next location to be written in the buffer associ~te~ with virtual circuit VC.
Buffers of dyn~m~ lly varying length are m~int~ine~ by keeping a linked list of cells for each virtual circuit. The linked lists are m~int~ine~d by the LIST_MANAGER 308, which also m~int~inc a linked list of unused cells that make up the free buffer space. Ope~ration of the LIST MANAGER is described below.
A GLOBAL_COUNT register 310 keeps track of the total number of 20 cells in all virtual circuit buffers. If each virtual circuit is initi~li7ed with one (unused) cell in its buffer, the initial value of the GLOBAL COUNT register is equal to the nul..ber of virtual circuits. The GLOBAL COUNT register can be written or read by the mt .~it~r. The TI~fING+CONTROL circuit 312 supplies all of the control signals needed to operate the controller.
Prior to the start of read request or write request operations, the contrQller iS initi~li7~d by the ll~nitor. For each virtual circuit, WP[VC] and RP[VC] are initi~li7~A with a unique cell number and COUNT[VC] is initi~li7~1 with a value of 1, l~,pl~nting an empty buffer with one (unused) cell present for receipt of incoming data. The initial value of LIMIT[VC] is the initial buffer 30 ~ tiQn for that ~irtual circuit, which is equivalent to its initial window size. The LIST MANAGER is initi~li7f~d such that the free list is a linked list cont~ining all cells in the cell queue 210 except those which are initi~li7~i in table QUEUE POINTERS.
When a cell arrives, the receiver asserts a write request on WREQ and 35 the virtual circuit number on WVC. Bus VC is used to address COUNT_TABLE
causing the values in the COUNT[VC] and LIMIT[VC] fields to be sent to a 2~3~3~9 co~ tol 314. If the virtual circuit in question has not consumed all of its te~d space in the cell queue, i.e. if COUNT[VC] is less than LIMIT[VC] in the table, the co"l~dtor will generate a FALSE value on lead LIMITREACHED. Bus VC is also used to address the QUEUE_POINTERS table such that WP[VC] is 5 present on bus MADDR. When LIMITREACHED is FALSE, the timing and control circuit will generate signal MW which causes the cell to be written to the cell queue 210, and will control the LIST_MANAGER to cause a new cell to be ~lloc~ted and linked into the buffer associated with VC. In addition, the buffer use for VC and the overaU cell count values will be ~ t~A To update the buffer use, 10 the present value in COUNT[VC] will be routed via bus COUNT to an up/down counter, which inclenlcnts the present nurnber of ceUs recorded in COUNT[VCl by one. This new value, ap~il-g on bus NCOUNT, is present at the input of COUNT_TABLE, and wiU be written into the table. The overall cell count is inc~ ted in a sirnilar manner using register GLOBAL_COUNT 310 and an 15 up/down counter 316.
If, during a write operation, LIMlTREACHED is TRUE, which means that the virtual circuit in question has con~ d all of its ~11QCate~I space in the cell queue, the T+C circuit 312 will not gcn~ tc signals to write data into the cell queue, to ~llocate a new cell, or to inc.elllcnt the value of COUNT[VC] or 20 GLOBAL_COUNT. Accordingly, any VC exceeding its assigned window size loses the coll~s~ ing cells, but the data for other virtual circuits is not affected.
When the tran~...;tt~ ~ is ready to send a new cell, it asserts a read request on lead RREQ and the v~tual circuit number on bus RVC. COUNT_TABLE is ~ccesse~l causing the value of COUNT[VC] to be sent to a col~ator 318, whose 25 second input is the value zero. If the buffer ~soci~e~l with VC contains no data, the colllp~hat~l 318 will g~,n.,l~tc a TRUE signal on EMErIY, and the operation will be termir~te~ by the TIMING+CONTROL circuit 312. If EMPI'Y is FALSE, the up/down counter 320 willdc~lc~ ,nt the value of COUNT[VC], and the resulting value will be written into COUNT_TABLE 304. In this case, the value of RP[VC]
30 from QUEUE_POINTERS is present on bus MADDR and the MR signal is generated, reading a cell from the cell queue 210. RP[VC] is also input to the LIST_MANAGER 308so that the cell can be ~ oc~ted and re~ -ed to the free store. The address of the next cell in the buffer for VC is present on bus NRP and is written into QUEUE_POINTERS 306. The overall count of cells b.~rre.c;l, which is35 stored in GLOBAL_COUNT 310,is d~lcl.~nted.

203~3~9 The LIST_MANAGER 308 m~int~in~ a linked list of memory locations which make up cell buffers for each virtual circuit. It also m~int~in~ a linked list of memory locations which make up the free list. The LIST_MANAGER 308 contains a link Illellloly LNKMEM 322, which contains one word of information for every 5 cell in the cell queue 210. The width of a word in LNKMEM 322 is the logarithmto base 2 of the number of cells in the cell queue 210. There is a register, FREE
324, which contains a pointer to the first entry in the free list.
Consider the buffer for virtual circuit VC. The read pointer RP[VC]
points to a location in the cell buffer at which the next cell for virtual circuit VC is 10 to be read by the tr~n~mitter. RP[VC] points to a location in LNKMEM 322 which contains a pointer to the next cell to be read from the cell queue 210 and so on.
Procee.1ing in this manner, one arrives at a location in LNKMEM 322 which pointsto the same locatiQn pointed to by WP[VC]. In the cell queue 210 this location is an unused loc~ti~n which is available for the next cell to atTive for VC.
Free space in the cell queue 210 is tracked in LNKMEM 322 by means of a free list. The beginning of the free list is m~inr~ined in a register FREE 324 which points to a location in the cell queue 210 which is not on the buffer for any virtual circuit. FREE points in LNKMEM 322 to a loc~tion which contains a pointer to the next free cell, and so on.
When a w~ite request occurs for a virtual circuit VC, if VC has not exceeded its buffer ~lloc~tion~ a new cell will be ~lloc~ted and linked into the buffer associated with VC. The value in WP[VC] is input to the LIST MANAGER 308 on bus WP at the beginning of thc operation. A new value NWP of the write pointer is output by the LIST MANAGER 308 at the end of the operation. NWP will be 25 written into table QUEUE POINTERS 306. This occurs as follows:

1) The value in rcgister FREE 324, which ~p,esenls an unused cell, will be chained into thc linked list ~csc ci~ted with VC, and will also be output as NWP.
NWP = LNKMEM[WP] = FREE

2) The next free loc~tion in the free list will be writtcn into FREE 324.
FREE = LNKMEM[FREE]
When a read request occurs for a virtual circuit VC, the cell which is cullcl1tly being read, namely RP[VC], will be input to the LIST MANAGER 308 on bus RP to be returned to the free list and the next cell in the buffer associated with VC will be returned as NRP. NRP will be written into table QUEUE POINTERS

306. This occurs as follows:

1) A new read pointer is returned which points to the next cell in the buffer associated with VC.
NRP=LNKMEM[RP]

5 2) The cell which was read in this cycle is deallocated by linking it into the free list.
LNKMEM[RP] =FREE
FREE=RP
FIG. 4 is an illustrative embodiment of a router, such as 110 of FIG. 1.
Variable length packets arriving from the local area network 106 of FIG. 1 are received by 10 the LAN receiver 400 at the upper left of FIG. 4. A global address, present in each packet, is translated to a virtual circuit number by the translation circuit 402. Since the packet will be transported using fixed length cells that may be smaller or larger than the length of the particular packet under consideration, additional header or trailer bytes may need to be added to the packet to facilitate reassembly of the packet from a sequence of 15 cells which arrive at the destination router, to allow a destination router to exert flow control over a source router, or to allow dropped or misdirected cells to be detected. The resulting information must be padded to a length which is an integral multiple of the cell size. These functions are not pertinent to the invention, however, an illustrative embodiment is described to indicate the relationship of these functions to the congestion 2 0 management functions that must be performed by the router.
The LAN packet and the virtual circuit number produced by the translation circuit 402 are passed to segmentation circuit 404, which may add header or trailer bytes to the packet, either for the functions described above or as placeholders for such bytes to be supplied by a second segmentation circuit 408.
2 5 The resulting information is padded to an integral multiple of the cell size and is stored in a cell queue 406, which may be identical in structure to the cell queue 210 described in FIG. 2. In particular, internal data structures in a controller 410 may be accessed by monitor 412 that allow the buffer use (buffer length) to be monitored for each virtual circuit, and that allow the buffer allocation per virtual circuit to be 3 0 adjusted dynamically. Segmentation circuit 408 performs window flow control on each virtual circuit, where the window size for each virtual circuit may be varied dynamically under the control of the protocols described below. To perform window ~`:

flow control, segmentation circuit 408 may fill in the added data bytes as appropriate to complete the reassembly and flow control protocol. As a minimum, segmentation circuit 408 maintains a counter per virtual circuit which keeps track of the amount of outstanding, unacknowledged data that it has sent in order to implement window flow control, and it 5 receives acknowledgments from the remote receiver indicating data that has passed safely out of the flow control window. Techniques for implementing reassembly and window flow control are well known in the art; the unique aspect of the invention is that the window sizes and buffer sizes may change dynamically under the influence of congestion control messages. The transmitter 415 takes cells from segmentation circuit 408, from the 0 local receiver as described below, and from the outgoing congestion FIFO 419 and sends them out on the outgoing cell transmission line 416.
Router 110 also receives cells from network 100 via the access line 112 of FIG. 1. These cells arrive at the receiver 414 at the lower right corner of FIG. 4. Insofar as these cells result from packets originated by the source 102 and intended for the 5 destination 128, they will be either congestion messages or acknowledgments from the remote router 120. The handling of cells that may arrive on access line 112 from other sources, which are ~ llplillg to communicate with destinations attached to local network 106, will be deferred until the discussion of router 120 below.
When a cell of one of the two types under consideration arrives, the receiver 20 414 determines whether the cell is a congestion message as indicated by a bit in the header. Congestion messages are stored in a separate FIFO queue 417 for the monitor 412 and handled according to one of the protocols described below. If the protocol generates a further congestion message, an appropriate cell is sent from the monitor 412 to segmentation circuit 408 and muliplexed onto the outgoing cell transmission line 416. If 2 5 an arriving cell is not a congestion message, the receiver 414 sends the cell to reassembly circuit 418, which determines whether a cell is an acknowledgment from the remote router.
If this is the case, reassembly circuit 418 sends an acknowledgment-received notification to segmentation circuit 408, so that it can update the count of the amount of outstanding data.
A router identical in structure with FIG. 4 may also represent element 120 30 of FIG. 1. In such case, the receiver 414 corresponding to such router takes cells from the outgoing access line 118 of FIG. 1. Insofar as these cells result from packets originated by the source 102 and intended for the destination 128, they will be either data cells or congestion messages from the remote router 110. When a cell arrives, the receiver 414 determines whether the cell is a congestion message as ~, ~6~C349 in~ t A by a bit in the header. Congestion messages are stored in a separate FIFO
qucue 417 for the monitor 412 and handled accordillg to one of the protocols described below. If the protocol gene.ales a further congestion message, an a~ ,pl;ate cell is sent from the monitor 412 to segm~ntation circuit 408 and 5 multiplexed onto the outgoing cell trancmicsion line 416 at the lower left of F~G. 4.
If an arriving cell is not a congestion message, the receiver 414 sends the cell to reacsembly circuit 418, which buffers the arriving cell in a per-virtual circuit buffer in cell queue 420. If the re~ccemhly circuit 418 detects that a complete local area network packet has been accnml~lated. reacsemhly circuit 418 sends a send-10 acknowle~lgment command to the local tr~nsmitt~r 416 on lead 422, which causes anacknowle.1grn~ont message to be sent to the remote router 110. In addition, re~csc-..bly circuit 418 issues multiple-read requests to the buffer controller 422 causing the cells which make up the packet to he sent in sUccescion to reassembly circuit 424. To facilitate the l~cse.l~hly procelu.~, re~csernhly circuit 424 may 15 delete any header or trailer bytes which were added when the packet was converted to cells by router 110. The packet is then sent to the translation circuit 426, where the global address is translated into a local area network specific address hefore the packet is sent onto the local area mtwul~ 124.

Choice of window sizes The operation of the appalalus and protocols described in this invention does not depend on the choice of window sizes. Various practical con~;~e~ations may ~ t~ the window sizes that are used. If there are only two window sizes, the following concidçrations lead to p~fel.~;d relationships among the numhers of virtual circuits and the window sizes.
Suppose that the m~yimllm number of virtual circuits that can be cimlllt~neol)Sly active at a given node is No. Suppose further that it is ~e~iAed to provide some numher Nl less than No of the virtual circuits with full-size windows WO, while providing the rem~ining No -Nl virtual circuits with buffers of some smaller size Bo that is adequate for light traffic. If there are Nl sim~ltaneol)s users 30 each of whom gets an equal fraction of the c~annel~ the fraction of the channel that each gets is l/Nl. The .~ fraction of the channel capacity that can be obtained by a user having a window size Bo is Bo/Wo. Setting 1/N1 equal to the maximum fraction of the trunk that can he had by a user with a small buffer, namely Bo/Wo, gives the following rel~honchip among the ql-~ntities: Wo/Bo = Nl . The 35 total buffer space B allocated to all the virtual circuits is ~3~49 B = (No -Nl)Bo +NlWo = NoBo -W0 +Wo2/Bo -~inimi7ing B with respect to Bo leads to Bo = Wo/(No)l/2, Nl = (No)1/2, 5 B = [2(No)1/2 - l]Wo .

These equations provide p~ef~ d relationships among Bo, N1, No, and W0.
If there are more than two window sizes, various choices are possible. Tt may be convenient to choose the sizes in g~n~e~ic progression, for example, increasing by powers of 2. An ~lt~rnAtive approach that may be ~l~fellc;d in some 10 inctqnces is to have different sizes correspond to round-trip windows at various standard trncmicsiQn speeds. Still other choices may be ~ tqted by other ci~u~ nceS.

Buffer ~llocation Protocols The following iiccucses protocols by means of which sharable buffer 15 space is qllocated and deaUocated and by means of which virtual- circuit nodes, routers, and hosts are so alerted. The reader is dir~ed to FIG. 5 through 12 as required.
Each node controller 212 keeps track of the buffer length of each of its virtual circuits via the entry COUNT[VC] in the table COUNT TABLE that has 20 been descri~eA in co~ l ;on with FIG. 3. Each node controUer also keeps track of the size of its free list, which is the dirr.,~nce ~~ cn the (fixed) number of cells in the cell queue 210 of FIG. 2 and the co,~çnl~ of the register GLOBAL COUNT 310 ~esc~-bed in conl-~';Qn with FIG. 3. All of these qn~ntities are available to be read at any time by the node mr~nitor 200 shown in FIG. 2. In a similar way, each router 25 keeps track of the input buffer length of each of its virtual circuits, in a table that is available to the router ~I itOl 412 shown in FIG. 4. For purposes of ~licclosllre~ it will be as~ e~ that each router mqnages its ceU queue 406, shown on the left side of FIG. 4, in a manner similar to the switching nodes, so that qllqntities analogous to COUNT and GLOBAL COUNT 310 are available to the router's Illonitol.
It is nnn~-ess~ ~, but desirable, for the node controllers and the routers tO mqintqin smoothed averages of buffer lengths A popular sllloothil-g pl~)cclul~ for the time-varying quantity q is given by the easilj i.l.plcl.~nlable recursive algo.ill,l-l, rn = ( 1 ~f)qn + frn-l ~

20303~

where qn represents the value of q at epoch n, rn, represents the moving average at epoch n-l, rn represents the moving average at epoch n, and f is a number bet~,veen 0 and 1 that may be chosen to control the length of the averaging interval. If observations are made at intervals of At seconds, the approximate averaging interval is TAV seconds, where 5 TAV = (1-1/logf)~t-Appropriate averaging intervals for network congestion control may be between 10 and 100 round-trip times.
In various embodiments of the present invention, up to four binary quantities are used with each virtual circuit as indicators of network congestion. These quantities are 10 defined as follows.
BIG_INPUT. A repetitive program at a router is executed periodically (FIG.
5, step 500) to update this parameter. It is set equal to l (step 508) for a virtual circuit if a buffer in a cell queue such as 406 for that virtual circuit at the router 110 has been occupied during more than a certain fraction of the time in the recent past, and it is set 15 equal to 0 (step 510) if the buffer has not been occupied during more than that fraction of time. For the determination of BIG_INPUT, the quantity q in the moving-average algorithm (step 504) may be taken as 1 or 0 depending on whether or not any data is found in the buffer at the given observation. The quantity r (step 506) is then an estimate of the fraction of time that the buffer has been occupied during the past TAV seconds. A
20 representative but by no means exclusive threshold for r would be 0.5.
SOME_BACKLOG. This quantity is set equal to 1 for a given virtual circuit at a given node 114 or output router 120 if the virtual-circuit buffer at that node or router has been occupied during more than a certain fraction of the time in the recent past, and it is set equal to 0 otherwise. For the determination of SOME_BACKLOG, the quantity q in 2 5 the moving-average algorithm may be taken as 1 or 0 depending on whether or not any data is found in the virtual-circuit buffer at the given observation. The quantity r is then an estimate of the fraction of time that the buffer has been occupied during the past TAV
seconds. The flow of control for the monitor program that calculates SOME_BACKLOG
is entirely similar to FIG. 5. A representative but by no means exclusive threshold for r 3 0 would be 0.5. The thresholds for BIG_INPUT and for SOME_BACKLOG need not be the same.
BIG_BACKLOG. This quantity is set equal to 1 for a given virtual circuit at a given node or output router if the virtual circuit has a large buffer length at the node or router, and is set equal to 0 otherwise. Since the lengths of buffers at bottleneck nodes vary slowly, smoothing of the buffer length is probably unnecessary.
The criterion for a large buffer length may depend on the set of window sizes. If the window sizes are related by factors of 2, a representative although not exclusive choice would be to set BIG_BACKLOG equal to I if the instantaneous buffer length exceeds 5 75% of the current window, and equal to 0 otherwise. If the window sizes are equally spaced, a representative choice would be to set BIG_BACKLOG equal to 1 if the instantaneous buffer length exceeds 150% of the spacing between windows, and equal to 0 otherwise.
SPACE_CRUNCH. This quantity is set equal to 1 at a given node or output 0 router if the instantaneous number of occupied cells, namely GLOBAL_COUNT 310, at that node or router is greater than some fraction F of the total number of cells in the cell queue 210 or 406 of FIG. 2 or FIG. 4, respectively, and it is set equal to 0 otherwise. A
representative choice would be F=7/8, although the value of F does not appear to be critical.
Various window management protocols may be embodied using some or all of the congestion indicators defined above. Without limiting the scope of the invention, two embodiments are described below. In each of the two embodiments, each virtual circuit always has a buffer allocation at least as large as the minimum size Bo and it may have other sizes variable up to the limit of a full size window W0. The first embodiment 2 o makes use only of the length of a buffer at a data source (a router) and the availability of free queue space at the nodes to manage changes in window size. The second embodiment makes coordinated use of conditions relating to buffer lengths and free queue space at the data source and at all the nodes of the virtual circuit.
In each of the two embodiments, it is assumed that both directions of the 2 5 virtual circuit traverse exactly the same nodes, and that each node has a single monitor 200 that can read and respond to messages carried by congestion control cells traveling in either direction. If the forward and return paths are logically disjoint, obvious modifications of the protocols can be used. Instead of carrying out some functions on the return trip of a control message, one can make another traverse of the virtual circuit so that 3 0 all changes are effected by control messages traveling in the forward direction.
In the first embodiment, the flow of control in the program that runs in the monitor of the input router 110 is shown schematically in FIG. 6. In FIG. 6, thequantity LIMIT refers to the existing buffer allocation for a particular virtual circuit.
The quantity WINDOW_SIZE refers to a proposed new buffer allocation. The input ~, '1 router 110 monitors the quantity BIG_INPUT for each of its virtual circuits (step 602 of FIG. 6). From time to time, as will be described below, it may request a change in the size of the window assigned to a given virtual circuit. It makes such a request by transmitting a control message over the virtual circuit (steps 608 and 614). In the 5 embodiment described here, the message is carried by a special congestion control cell that is identified by a bit in its header. Alternatively, the congestion control message may be carried by special bits in a congestion field in the header of an ordinary data cell, if such a field has been provided. There is no logical difference between the use of special control cells and the use of header fields.
An input router that wishes to change the size of its window transmits a message containing the quantities 0, WINDOW_SIZE. The initial 0 represents a variable called ORIGIN. Messages that carry requests from input routers are distinguished by the value ORIGIN=0; messages that carry responses from output routers have ORIGIN=1, as will appear below. WINDOW_SIZE is the size of the requested window, coded into as many bits as are necessary to represent the total number of available window sizes. By way of example, if there are only two possible sizes, WINDOW_SIZE requires only a single 0 or 1 bit.
An input router that requests a new window size larger than its present window size (steps 612, 614) does not begin to use the new window size until it has received confirmation at step 616 (as described below). On the other hand, a router does not request a window size smaller than its current allocation until it has already begun to use the smaller window (step 606). Since switch nodes can always reduce buffer allocations that are above the initial window size, confirmation of a request for a smaller window is assured.
When the node controller 212 of a switching node along the forward path receives a control message containing 0,WINDOW_SIZE, it processes the message as follows. If the node controller can make the requested buffer allocation it does so, and passes the message to the next node without change. If there is insufficientunallocated space in the free list to meet the request, the node allocates as large 3 0 a buffer size as it can, the minimum being the current buffer size. In either case, the controller writes the value of WINDOW_SIZE that it can allow into the message before passing it along to the next node. The output router also meets the requested value of WINDOW_SIZE as nearly as it can, sets ORIGIN = 1 to indicate a response message, and transmits the response containing the final value of 3 5 WINDOW_SIZE to the first switching node on the return path. Node controllers on t he return path read ORIGIN = 1 and the WINDOW_SIZE field and adjust their ~03Q3~

~llocations accordingly. The adjustments involve, at most, downward allocations for nodes that met the original request 'oefore some node failed to do so. When the input router receives a control mçss~ge cont~ining 1,WINDOW SIZE, it ~cnows that a setof buffer allocations consistent with the value WINDOW_SIZE exist along the 5 whole path.
A newly opened virtual circuit has a buffer allocation Bo at each node and has a window of size Bo. The input router should request an increase in window size as soon as it observes that BIG_INPUT = 1. After requesting a window changeand receiving a response, the input router may wait for some period of time D, such 10 as 10 to 100 round-trip times, before inspecting BIG_INPUT again. Then if BIG_INPUT = 1, it may ask for another increase in window size, or if BIG_INPUT
= 0, it may ask for a decrease. If a decrease is called for, the input router does not issue the request until the amount of outct~n~ling data on the virtual circuit will fit into the smaller window, and from that time on it observes the new window 15 restriction. The actual ~lloc~tion is not changed until the value of LIMIT is set equal to the value of WINDOW_SIZE (steps 608, 618).
The flow of control in the program that runs in the ,lloniLol of a switching node 114, in response to the arrival of a congçstion control cell from either direction, is depicted in FIG. 7. Step 700 c~n~nges LlMIT to match the requested20 window size as closely as poscible Step 702 writes the new value of LIMIT into the control cell and passes the message along to the next node in the virtual circuit.
The previous c-..ho~ nt has made use only of congestion information at the input router. A second em~imç~t employs a protocol that COOfliind~S
co~ngesti~n information across the entire circuit in order to pick a new window size if 25 one is nelç~ lt uses a two-phase sign~ling ploc~ul~, in which the first phase sets up thc new window and the second phase resolves any discrepancies that may existamong the new wil~dow and the buffer allocations at the various nodes. The logical steps carried out by the input and output routers and by the ~.wilching nodes are illu~ ted schçm~tic~lly in FIG. 8 through 12.
The protocol for the second embodiment uses the qu~ntitiçs ORIGIN, BIG_INPUT, SOME BACKLOG, BM BACKLOG, and SPACE CRUNCH that were defined earlier. Since the protocol uses two phases of ~ign~ling, it f~uires one extra binary quantity, PHASE, which takes .~he value 0 for Phase 1 and I for Phase 2.
In Phase 1, the input router 110 initi~tçs a control m~ss~ge carrying a ~bit field that 35 consists of the qu~ntities ORIGIN = 0, PHASE = 0, BIG INPUT, SPACE CRUNCH = 0, SOME BACKLOG = 0, BIG_BACE~LOG = 0. The flow of 20303~

control for the input router is depicted in FIG. 8.
The flow of control for a node controller is shown in FIG. 9. When a node controller receives a Phase 1 control message, it inspects the values of SPACE_CRUNCH
(step 900), SOME_BACKLOG (step 904), and BIG_BACKLOG (step 910), and if its own value of the given quantity is 0, it passes that field unchanged. If its own value of the quantity is 1, it writes I into the corresponding field, as shown in FIG. 9 (steps 902, 906, 910), before transmitting the control message to the next switching node (step 912).
When the receiving router 120 receives a Phase 1 control message, it first combines its own values of SPACE_CRUNCH, SOME_BACKLOG, and BIG_BACKLOG
with the values in the arriving message, just as the switching nodes have done. The receiving router then inspects the last four bits of the modified message and calculates a proposed value of WINDOW_SIZE according to the four cases below, using the logic flow shown in FIG. 10.
l) If BIG_INPUT=1 and SOME_BACKLOG=0 (step 1000), then increase the window size.
The virtual circuit is nowhere bottlenecked by the round robin scheduler and the virtual circuit would like to send at a faster rate; it is being unnecessarily throttled by its window.
2) If BIG_BACKLOG=1 and SPACE_CRUNCH=1 (steps 1002, 1004), 2 0 then reduce the window size.
Some node is bottlenecked by the round robin scheduler and a big buffer has built up there, so the window is unnecessarily big; and some node is running out of space.
3) If BIG_INPUT=0 and SOME_BACKLOG=0 and SPACE_CRUNCH=1 (step 1006), then reduce the window size.
2 5 The virtual circuit has a light offered load, so it does not need a big window to carry the load; and some node is running out of space.
4) In all other cases (step 1008), the present window size is appropriate.
The receiving router then transmits the Phase 1 control message to the first switching node on the return path (step 1012). The response message contains the fields ORIGIN = 1, PHASE = 0, WINDOW_SIZE where the last field is a binary encoding of the recommended window size. Each node controller 212 on the return path looks at the control message and takes the action shown in FIG. 11. If an increased allocation is requested (step 1100), the node makes the allocation if it can (step 1102). If it cannot make the requested allocation, it makes whatever allocation 'J~

~030~49 it can make, the minim-lm being the present buffer size, and writes the allocation it has made into the WlNDOW SIZE field (step 1104). The node then transmits the control message to the next node on the return path (step 1106). If the request is for a decreased allocation. the node does not make the d,nease yet, but it passes the S WINDOW_SIZE field along unchanged.
When the tr~nc. ~ ~;tl ;ng router receives the Phase 1 response message (step 804), the WINDOW_SI:ZE field inAicatec the window that the virtual circuit is going to have. If there is an increase over the present window size, it is available immçAi~tçly. If there is a decrease, the tr~ncm~ ng router waits for the amount of 10 unacknowledged data in the virtual circuit to drain down to the new window size, as shown in FIG. 8 at step 806. Then it transmits a Phase 2 control message with the fields ORIGIN = 0, PHASE = 1, WINDOW_SIZE (step 810). Node controllers receiving this message take the action shown in FIG. 12. They adjust their buffer allocations dow--~d, if necessary, to the value of W~DOW SIZE (step 1200), 15 and pass the control m~s~ge along unchanged (step 1202). The receiving routerreturns a Phase 2 ,csponse mess~ge with the fields ORIGIN = 1, PHASE = 1, WINDOW S~. The ~witching nodes simply pass this m~ss~ge along, since its only purpose is to notify the tl~ ;ng router that a co~sistçnt set of buffer allocations exists along the entire virtual circuit.
After comrleting Phase 2, the tr~ncm~ ng router waits for a while, as shown at step 816 in FIG. 8, before be~inning Phase 1 again. First it waits until either a wh~dow's worth of data has been ~ns~;ued since the end of Phase 2 or a certain period of time D, such as 10 to 100 round-trip times, has elapsed since the end of Phase 2, whichever comes first. Then, if the present window size is greater than the --in;-------- window size Bo (step 818) or if BIG INPUT = 1 (step 800), Phase 1 begins imm~Ai~tely; otherwise, Phase 1 begins as soon as BIG INPUT = 1.
A newly opcned virtual circuit, whose initial window size and buffer ~llocatiol~c are Bo, should begin Phase 1 as soon as BIG INPUT = 1, if ever.
Over~iew of Window Sizin~: ~IG. 1 and 2 Further work since the filing of the parent of the present application has resulted in improved metho-lc for sizing windows in cell nctwollc l00 of FIG. 1. The me~ho~1c are particularly adapted for use with the first e ~ 1h~nt disclosed in the parent. The following ~i~cussio~ f~rst l~n,scrlts an ovcnriew of these methods and then provides ~t~ile~ ~liccllssir~nc of techniques for determining the ideal window 35 size for a virtual circuit, techniques for d,t~ ining when a resizing is n~cess~ry, and techniques for pc.roluling the resizing.

203034q In network 100, each switching node 114 has a memory buffer 210 (FIG.2).
The buffer is logically divided into a per-circuit queue for each virtual circuit, together with some unassigned memory space. The per-circuit queues contain cells being sent via the virtual circuit to which the queue belongs. The queues are served round robin. The 5 amount of buffer space assigned to each virtual circuit's queue may vary from time to time but is never less than the circuit's current window size. Switching controller 212 maintains a list of the amount of memory allocated to each virtual circuit. If a virtual circuit's queue overflows, cells belonging to that virtual circuit only are discarded.
However, if the per-circuit queues are sized according to the present invention, overflows lo will only occur at router 110, and never within the network.
The routers 110 and 120, which are assumed to belong to the network vendor, enforce edge-to-edge cell windows on the virtual-circuit network. Each virtual circuit is given a default window when it is set up. The default window is at a minimum large enough to hold the largest packet which will be transmitted via the virtual circuit and 15 at least some sign:~lling information. The virtual circuit retains a window of at least the size of the default window for as long as it exits. Input router 110 can issue requests for increases in window size up to a full round-trip window, or decreases down to the default window. Increases may be requested at intervals as short as one round-trip time; decreases are requested at longer intervals determined by a renegotiation timer in router 110. A
2 0 request travels around the virtual circuit and the node monitors 200 confirm or modify it as described below and also make appropriate changes in buffer allocation. When the response returns to input router 110, router 110 knows that cell buffer allocations corresponding to a new window for the virtual circuit exist in all switching nodes 114 in the virtual circuit. Router 110 begins to use a larger window after the increased buffer 2 5 allocations are confirmed. It begins to enforce a smaller window before requesting a decrease in buffer allocations.
Determining the Ideal Window Size for a Virtual Circuit In the following, an algorithm is developed for determining the ideal window size for a virtual circuit. The development of the algorithm begins with a simplified case 3 o in which circuits are turned on one at a time and transmit continuously after they are turned on, then modifies the simple case to take into account intermittent transmission of data by a virtual circuit.

A single user (virtual circuit) which encounters no contention on any link through which it passes needs a window of size W0 in order to use the full network bandwidth. A user who shares a link with other active users does not need a window as large as W0, beeause such a user is not entitled to the entire bandwidth.
S In prineiple, if N users transmit eontinuously via a link, eaeh of them needs a window no larger than Wo/N, and the total window r~uirc.llent is just W0.
As a slightly more realistie example, eonsider the hypothetical case in which circuits ll~n~ ;ng via a link are turned on one at a time and transmit continuously. If we assign a window of size W0 /k to the kth cireuit to be turned on, 10 then when k circuits are tr~ncmitting, the round robin queueing discipline gives each one a fraetion l/k of the trunk bandwidth, and nobody's throughput is ~lnn~eess~rily window-limited. If the kth user gets a window of size Wo/k, the total buffer space B
for N users s~ti~fies B/Wo=~ l/k=logN, ~1 15 so that in practice B would be a few times W0.
In gen~ral data net~vul~s will have many types of users, who may be cl~ccifiçd into three broad catcg~"ies: 1) Inte,àc~ e users, whose tr~ncmicsions are shorter than the default window; 2) bulk users, whose tr~ncmicsions are long co~ ~d tO Wo and are sep~ated by intervals long eû,~pd~ed to the round-trip 20 propag~tio~ time To; and 3) il-~c~ t userc, who exhibit periods of activity marked by lepcdt~l tr~ncmiscions cc,u~ able to W0 at intervals co~ alable to To, separated by periods of inaetivity long cou-l)a~ to To. Inte.luiu~,nl users are the hardest to provide for effieiently, and it is this class of users that we consider in choosing window sizes.
The model that led to Eq. (1) is unrealistic in that it ~csllm~s that the number of users who are aetively tr~ncmitting when a new us,er arrives is equal to the number of users who are holding window alloc~tio~c as large as or larger than the new user. On the average, the number of aetively ~ l;ng users is so,lle~vhat smaller, because users dû not transmit during some part of the time that they hold a 30 window alloeation. We model the effeet of idle time as follows.
~ ss~lming that the number of users with window ~lloeations varies slowly, we co~ider a closed queueing nel~,volk with two nodes, "active" and "idle". Tr~ncmitting users are cor~ red to be queued at the active node, which - 20303~9 has a single processor-sharing server. The average service-time requirement of an inl~l.l.ill~,nt user on each pass through the active node is ~. After each burst of activity, the user visits the idle state, m~eled as an infinite-server node, for an average time h~. In the actual network, a user that has been idle for longer than about S hT would be judged q~ Psçent and have its window reduced to the default window, but this population loss is not included in our closed network model.
We can apply mean-value analysis as described in E.D. Lazowska, J.
Zahorjan, G.S. Graham, and K.C. Sevcik, Quanntative System Performance:
Computer System Analysis Using Queueing Network Models, Prentice-Hall (1984), 10 pp. 112-116, to dete~ e J~c, the average nu.llber of active users in a network with k users. The average time for a user to remain active is (J~C_1+1)~, out of an average cycle time of (J~C_l+l)~+h~; hence J~ + 1 J~
J~_l+l +h k It is trivial to evaluate J~c-l numcrically from the l~,cu-l~;nce (2) for any value of h, 15 starting from J0 =0. However, a l~n.a.l.qbly accurate closed-form expression for J~_l over the p~al~ete~ ranges of inurest can be obtained by replacing the right side of (2) by J~ t(k-1) and solving the res~ ing quadratic equation.
This closed-nelwu.k analysis can be used to size windows for the real nelwol~ as follows. Suppose that the kth a~iving user receives a window allocation 20 W~ when k-1 int,~ ;t~ent users are already present with windows greater than or equal to W~ . The convention of ~lPno~ing the kth user's window by Wlc_1 perrnits us to denote the f~rst user' s window by the symbol W0 that we have used for the full round-trip window. In order that the kth user's throughput not be nnn~essqrily restric~A, W~c_l needs to satisfy W~,_l/Wo = l/(J~_l+l).

Using the closed-form approxim-qtion to Jk-l yields W~c_l 2 W0 k-h+ [(k-h)2 +4h] 1/2 2~:~034~

If h = 0, Eq. (4) gives W,c_l /W0 = 1 /k, as it should. If h > 0, we have Wk_l/W0 = 1-(k-1)/h forkch, so that the window sizes given to the first few users decrease linearly from W0. Near k = h there is a transition. The hth user gets a window S Wk_l /W0 = 1/hl/2 for k = h, and the tail of the window-size distribution is W~_l/WO = l/(k-h) fork:~h, which is a shifted version of (1).
If the kth user is given an ~lloca~iQn according to (4), the total buffer 10 allocation for N users satisfies B _ 2 Wo ~i k-h + [(k-h)2 + 4h~ 1/2 The sum may be app.o~;...-'c~ by an integral, giving Wo 1/' 1, X+(X2+4h)l/2 r2+x2 rl +xl log rl +x where xl=1/2-h, x2=xl+N, and rl 2=(xl2 2+4h)1/2. A little algebra shows that B/Wo = logN+h/2 for N ~ h :~ 1. Tradeoffs among B/Wo, N, and h can be calculated using this equation.
What are realistic values of h? ~re~u~ bly the in~ iU~nl users will have meSs~ge lengths no shorter than about one round-trip window, because if they 20 had much shorter mess~ges they would not have been ~llocatecl m~xim~l windows in the first place. A virtual cirCuit's window can be shut down in about one round-trip time, so in principle h could be near unity. However one may wish to allow intermittent users a few seconds of think time before the network decides that they are quiescent. A transcontinental round trip is about 60 ms. An idle time of 3 s per round-trip window of transmitted data corresponds to h=50; and idle time of 6 s corresponds to h=lO0.
If a constant default window is assigned to every circuit beyond a certain point, at that point the curves of total buffer usage become linear. Suppose that the first K
users are given windows computed as above, and the remaining N - K users are given windows of size ~. Then the total buffer space required by N users is K
B=~Wk l+(N-K)A.
k=l The right-hand side of (5) increases with K so long as WK 12~

In practice one would choose the largest value of K, say K2, for which (8) is satisfied. If the total buffer space calculated by setting K=K2 turns out to be greater than the maximum amount of memory, say Bm~ than one wishes to provide at each switch, then one would choose the largest value of K, say K" for which B<Bm. If K,<K2, then under some conditions when there are between Kl and K2 intermittent users the trunk may not be fully 2 0 utilized, although the users desire to do so. Of course, one cannot even provide default windows to N users unless Bm>N~.
Discrete Ideal Window Sizes Suppose that there are M different discrete window sizes. Suppose that the maximum number of virtual circuits that can exist simultaneously is N. We wish to 25 choose a set of break points, say 0=No<Nl< <NM=N. We also wish to choose a set of window sizes, say WO>Wl>...>WM l, and assign them to circuits in the following way. A
circuit that becomes active is given the smallest window Wj such that Nj<kSNj+l, where k-l is the number of other circuits with windows of size Wj or greater.
For any particular choice of Nj we can get the minimum value of Wj that 3 0 does not restrict the bandwidth of the (Nj+l)st circuit by solving (2) recursively for JN
and then calculating Wj from Wjlwo=ll(JNi+l) However, it is much easier to use the closed-form approximation described in thepreceding section. According to this approximation, the normalized window wi=Wj/WO
satisfies the quadratic equation hW.2+(Ni+ l -h)wj- l =0, 5 whose solution is Wi=
Ni+ 1 -h+[(Ni+ 1 -h)2+4h] /2 For any given set of break points and window sizes, the normalized total 10 buffer usage b=B/Wo is M-l b=~(Nk+, - Nk)Wk-k O
We wish to minimize b with respect to the M-l disposable break points, or alternatively 5 with respect to the M-l disposable window sizes. Differentiating the right side of (l l) with respect to each Wj in turn, and noting that Nj depends only on wi, we obtain (wk-l-wk)dNkldwk+Nk+l-Nk ~

for k=l, 2, ..., M-l . In addition, we have by differentiating Eq.(l l), 2 0 dNk 2hwk+Nk+l-h dwk Wk The case h=O is particularly simple. If there are M different window sizes, it is an elementary exercise to show that the total buffer space B used by N virtual 2 5 circuits is minimi7ed if the values of N~ + l, N2 + l, ..., are chosen as the M - l geometric means between l and N + l. Thus Ni + l = (N+l)i/M i=O, l, ..., M, , . -~

203034q and the minimum value of buffer space satisfies B/Wo=M[(N+l)"M- 1].
In Eq.(17), it can be shown that if N is large and if we choose the number of window sizes M equal to the number of users N, the right side of (17) is approximately log N.
If h is not zero, (14) is a system of M-1 equations that can be solved numerically. The simplest way to observe that N2, , NM-1 can be calculated successively from an arbitrary starting value of N" so that the whole system reduces to f(N,)=N, where N is the maximum number of virtual circuits. The value of N, that satisfies (18) can be determined as accurately as desired by a root-finding technique such as the method of bisection.
Chan~in~ Cell Buffer Sizes: FIG. 13 As already explained, requests for changes in cell buffer sizes for a virtual circuit are initiated by input router 110. The request takes the form of congestion control 1 5 inforrnation. The information may either be incorporated into the header of each cell sent via the virtual circuit, or may be included in a separate congestion control cell. FIG. 13 shows congestion control information 1301 in a preferred embodiment. Field RR 1303 is a request-response bit; field SN 1305 is a sequence bit; the remaining bits make up window size field 1307, which indicates a cell buffer size in nodes 114.
In the preferred embodiment, congestion control information 1301 is part of the cell header and the fields are employed as follows: to make a request to change the size of the cell buffers in nodes 114 for a virtual circuit, router 110 sets congestion control inforrnation 1301 in each outgoing cell belonging to the virtual circuit so that RR 1303 indicates a request, field SN 1305 has a sequence numberother than that of the last request to change size, and field 1307 contains the desired window size. If the request is for a reduction in window size, router 110 first reduces the flow of cells to the virtual circuit to a rate compatible with the new window size and then requests the reduction; if the request is for an increase in window size, router 110 does not increase the flow of cells until it has received congestion controlinformation 1301 from router 120 indicating the size to which the window can be 2Q~349 increased.
The ~ llw~ for each node 114 includes for each virtual circuit passing through the node a copy of the co~gestion control information which was last used to adjust the size of the virtual circuit's window. In the following, this is terrned the 5 current congestion control infoll~aLion for the node. Whenever a cell belonging to a given virtual circuit passes through node 114, controller 212 copies the currentcongestion control infonn~tio~ into the cell's header.
When a node 114 receives a cell, congesdon control inforrnation 1301 is stripped out of the cell and provided to -,onitor 200. If the congestion control 10 info~n~tion 1301 has an M field 1303 which inrlica~es a request and a SN field 1305 which is diff~ , t from that in the current congçstion control inf~...la~ion, monitor 200 d~t~,ll.lines to what extent it can provide a window of the size specified in window size field 1307. Having done so, it sets the current congestion control infc,l...al;on to the values of RR and SN in the incoming request and the value of 15 WS 1307 to the value which it determined. From that time on, the headers of all cells for the virtual circuit leaving node 414 will have copies of the new current congestion control info.,..~l;on Deterrnination of the window size is as follows: If the sizc specified for the virtual circuit in the request is smaller than the present window, lllollitol 200 20 simply copies the request into the current congçstion control information for the virtual circuit. If the size specified is larger than the present window and the node 114 can provide a cell buffer of the ~ccirlcd size, l--olullx 200 again simply copies the request into the current congesti~n control infolll~l.on for the virtual circuit. If the size specifie~l is larger but node 114 cannot provide a window of the size 25 in~1ir~t1 in window size field 1307, l~nitol 200 determines the size of the window it can provide and copies the request into the current congestion control information in con~llcr 212, but c~nges window size 1307 to inrlir~e the size of window which it can provide. The method by which node 114 det~ninçs the size of window it can provide will be explained in more detail below.
As is appdl~nt from the fc,l~going, when a cell with congestion control information 1301 for the request by router 110 reaches router 120, window size 1307 will indi-~te the sm~llest window size which any of the nodes 114 can provide.
Router 120 then de~,..;nes whctl,c~ thc availability of buffer space in router 120 lcqui~s a further reduction in window size. After ~ct~ ....;ning the window size, 35 router 120 sets congestiQ~ control fields 1301 in cells which it is sen~ling via the virtual circuit as follows: RR 1307 is set to in-1icate a ~s~nse, SN 1305 remains 203034~
-the same, and WS 1307is set either to the smallest window size specified by any of nodes 114 or to the still smaller window size required by router 120. When each node 114 receives the response message, it sets the cell buffer size for the virtual circuit as specified in window size field 1307, copies the response message into the current congestion control 5 information, and begins copying the new current control information into cells of the virtual circuit destined for router 1 10. When router 1 10 receives a cell for the virtual circuit whose header contains the response message, it knows that cell buffers of the size specified in window size field 1307 are available for the virtual circuit in all of the nodes and increases the flow of cells to the virtual circuit accordingly. The next request by 10 router l lO for a change in window size is made in the same fashion, except that SN 1305 is set to a value different from that of the last request. As is clear from the foregoing, router 110 must wait at least until it has received a response from router 120 before it can again request an adjustment in window size.
If there are too many window sizes to be coded into the available header 15 space, then congestion control information can be carried as a data field in special congestion control cells each marked by a single bit in its header.
Determining When to Request a Chan~e in Cell Buffer Size While transmitting router 110 may request an increase in window size at any time after the response to the previous request has been received, in a preferred 2 0 embodiment, requests to decrease window size are separated by a renegotiation interval. A
renegotiation interval is roughly h~, where ~ is the longest round-trip time in the network and h is the idle-time parameter previously discussed. As previously indicated, realistic values of h are values such that h~ is a small number of seconds.
To determine the size of the next window request, router 110 keeps track 25 of the total number of cells that the given virtual circuit has in the system, that is, the number of cells router 110 has waiting to be transmitted via the virtual circuit plus the number of cells in nodes 114 for the virtual circuit. The latter number can be estimated by keeping track of the difference between the number of cells sent and the number of cells which have been acknowledged. Router 110 also keeps track of the 3 0 cumulative throughput of the virtual circuit during the last renegotiation interval.
When router 110 requests a window, the size to be requested is determined from the default window size which was set for the virtual circuit when the virtual circuit was established, the total number of cells that the virtual circuit has in the system, and the number of cells which would have passed through the circuit during one round-trip , ~ ~

- 2Q303~9 time if the circuit had opel~t~,d during the round-trip time at a rate corresponding to its average throughput during the renegotiation interval. The latter value, TN, is derived from the e~ ssion TN CI xTo Tr~eg 5 where CT is the cumlllq~ive throughput, To is the round-trip time, and Tren,g is the ~enegoliation interval. The size of the request is dc~te~ulincd from the largest of the default window size, the total nu~l of cells in the system, and TN, up to the full round-trip window. TN is included in the above co,~pulalion to prevent router 110 from closing down a virtual circuit's window b~ause of momentary idleness.
If router 110 cannot aCco~ oA~ a full round-trip window of data in the input packet buffcr of each virtual circuit that it :~Up~ , router 110 should provide an input buffer at least as large as the current cell window for the virtual circuit, and it should keep track of input buffer ovcrflows. An input overflow since the last request for an i~ ase in cell-window size can be treated as if the circuit had 15 a full round-trip's worth of data in the system at the time the next request for an increase in window size goes out.
Determinin~ Cell Buffer Sizes in Nodes 114: FIG. 14 If the ideal window sizes for the windows W0 through W,c_l which result when the window sizes are derived using Eq. 3 or 4 above, are plotted against 20 the circuit nu~ l..k, the result is a curve which appears in FIG. 14 as curve1405. Each ~nito~ 200 has a table in which the sizc of the ideal window for eachpotential virtual circuit l..k as A~te.ln;~-eA. by curve 1405 is associated with the potentiql virtual circuit's number. Monitor 200 further keeps track of the culTent cell buffer ~lloca-~A to each virtual circuit in the node and the rank of each of those 25 virtual circuits, where the rank is the ordinal number of that circuit's cell buffer qlloc.,tion. The circuit with the largest qllqtion has rank 1, the next largest has rank 2, and so on. Thus, if Wc rep~sen~ the cuIrent qllocano~ of the circuit whose rank is k, then Wc must be less than or equal to W~
When a request is made to increase the size of the virtual circuit's 30 window to a re~l.este~l size Wr, the table, the size of the current window, and the rank of the virtual circuit are employed as follows to A-ete~nine the size of the virtual circuit's new window: When cong~,l;on control infr,....q~ion 1301 arrives which requests enlatge~ nt of the cell buffer for a given virtual circuit, monitor 200 first determines what rank the given virtual circuit would have relative to the other virtual circuits if it were given the window size Wr specified in WS field 1307. This is the given virtual circuit's hypothetical rank. Beginning at the current rank of the given virtual circuit, monitor 200 then exchanges rank with each next-higher ranked virtual circuit until 5 the given virtual circuit either attains the hypothetical rank or until the exchange would require giving the next higher virtual circuit a rank which was inconsistent with the present cell buffer size for the next higher virtual circuit. Inconsistency is determined using the table: the new rank for the next higher circuit is used as one of the numbers l..k in the table, and if the cell buffer size specified for that number is less than the present cell 10 buffer size for the next higher virtual circuit, the rank to which the next higher virtual circuit would be demoted is inconsistent.
Let j be the rank which the given circuit has at the end of the series of interchanges of rank. The circuit then receives an actual allocation equal to min(Wr,W~
and the final ranks are consistent with the final allocations. If a decreased allocation is 15 requested, the decrease is given automatically and the circuit's rank is exchanged with circuits having lower ranks until consistency is achieved. Once monitor 200 has determined the new buffer allocation, it sets the current congestion control information as described above, and the next node 114 responds to the congestion control information in the headers of cells for the virtual circuit which have passed through this node 114 in the 2 0 manner just described.
Figure 14 shows an example of an increase. In the figure, each vertical line 1407 A..D represents a virtual circuit. The length of the line represents the virtual circuit's present window size. In panel 1401, circuits A, B, C, and D are in rank order and the cell buffer size for each of the virtual circuits 1407 is consistent with the circuit's position with regard to curve 1405. Then monitor 200 receives congestion control information 1301 for virtual circuit D which requests a full round trip window W0 for the virtual circuit, as shown by the dotted line labelled 1409. As shown by the fact that line 1409 extends above curve 1405, the window size thus requested is larger than curve 1405 permits for a virtual circuit which has rank 4.
3 0 Virtual circuit D is consequently first exchanged with virtual circuit C and then with virtual circuit B. In neither case does the demotion of the exchanged virtual circuit cause the line representing the size of the demoted virtual circuit's cell buffer to intersect curve 1405, i.e., the new rank is consistent with the demoted virtual circuit's current cell buffer size. However, when the exchange with virtual circuit A is 3 5 attempted, the demotion of virtual circuit A results in an inconsistency. Consequently, virtual circuit D is given rank 2 and its cell buffer size is set to the .

2Q~!~3 ~ 9 m~xim-lm which curve 140S permits for that rank. The result is shown in panel 1403. Circuit D's new cell buffer size gives it a fair share of bandwidth in comretition with other circuits. While curve 1405 results in a particularly advantageous and efficient ~ tion of cell buffers in the nodes, the method just described may be employed with any other curve where the window for a virtual circuit i+l is smaller than that for virtual circuit i, l<i<k Conclu: :~r The foregoing Detailed Description has disclosed how one of ordinary skill in the art may control congestion in a virtual circuit ncLwo.~ in which data is tr~ncmi~e~l in 10 cells. In particular, the Detailed Description has disclosed improved techniques for selecting the window size for a cell buffer for a virtual circuit in a node of the netork, improved techniques for determining when a virtual circuit requires a change in cell buffer size, and improved techniques for determining how a node should respond to a request to change the cell buffer size for a virtual circuit. The Detailed Description has further disclosed the best modes p,~,se.,~ly known to the invcntors for imple~ n-;ng the inventions which were first disclosed in this application.
However, as is clear from the Detailed Description, and as is appal~- t to those of ordinary skill in the art, the invention may be hllyk-..~ .~t~ in other fashions without departing from the spirit and prin~iples of the present invention. For example, other imple~ t;o~c may employ other systems for cign~lling a change of cell buffer size, may give the con~estion control infol~ation dirr~nt forms, and curves other than curve 1405 rnay be employed to govern selection of buffer size in a node.
Consequently, the Detailed Dcs~ tion is to be collci~1ered in all ~ s as being illustrative and exe-mrl~ry, but not l~ cli~re~ and the scope of the invention is not to be ~leterrnined from the Detailed Dcsclil,lion, but rather from the claims as read in light of the Detailed Description and i~lt~ll r~t~l according to the Doctrine ofEquivalents.
What is cl~im~d is:

Claims (32)

1. In a virtual circuit data network having one or more nodes, a method of adjusting the size of a window Wk-1 in the node for the kth virtual circuit using a given link from the node, the method comprising the steps of:
determining the average number of active virtual circuits Jk-1 using the link prior to the addition of the kth virtual circuit; and adjusting the size of the window Wk-1 for the kth virtual circuit from the equation where W0 is a full-speed round trip window.

2. The method set forth in claim 1 wherein:
a virtual circuit in intermittent use has an average service time requirement ? in which it is in active use and an average idle time of h? and the step of determining the average number of active virtual circuits Jk-1 is performed using the recurrence starting from J0=0.

3. The method set forth in claim 1 wherein:
a virtual circuit in intermittent use has an average service time requirement ? in which it is in active use and an average idle time of h? and the step of determining the average number of active virtual circuits Jk-1 is performed using the approximation for Jk-1.

4. The method set forth in claim 3 wherein:
the step of determining the average number of active virtual circuits is performed using the longest round trip time of the network as ? and a value of hsuch that h? is a small number of seconds.

5. In a virtual circuit data network having one or more nodes, a method of adjusting the size of a window Wk-1 in the node for the kth virtual circuit using a given link from the node, the method comprising the steps of:
determining that there is to be a fixed number M of window sizes;
dividing the number N of virtual circuits which can exist simultaneously in the network into a set of M+1 break points such that 0=N0<N1<...<NM=N;
associating M window sizes W0>W1>...>WM-1 with the break points N0 through NM-1 such that Ni is associated with Wi;
determining the average number of active virtual circuits JNi using the link prior to the addition of the kth virtual circuit; and adjusting the size of the window Wi associated with virtual circuit Ni from the equation where Nj<kNi+1 and W0 is a full-speed round trip window.

6. The method set forth in claim 5 wherein:
a virtual circuit in intermittent use has an average service time requirement ? in which it is in active use and an average idle time of h? and the step of determining the average number of active virtual circuits JNi is performed using the recurrence starting from J0=0 and retaining the result only for each JNi.

7. The method set forth in claim 5 wherein:
a virtual circuit in intermittent use has an average service time requirement ? in which it is in active use and an average idle time of h? and the step of determining the average number of active virtual circuits JNi is performed using the approximation for JNi.

8. The method set forth in claim 7 wherein:
the step of determining the average number of active virtual circuits is performed using the longest round trip time of the network as ? and a value of hsuch that h? is a small number of seconds.

9. The method set forth in claim 8 wherein:
the normalized total buffer usage b is minimized by solving the system of equations for k=1, 2, . . ., M-1 where

10. A method employed at an edge of a virtual circuit data network in which data is transported in cells to determine the size of the window to be requested in the network for a given virtual circuit originating at the edge, the method comprising the steps of:
keeping track of the total number of cells TC that the given virtual circuit has at the edge and in the network;

keeping track of the cumulative throughput CT of cells of the given virtual circuit during a most recent renegotiation interval; and requesting a window for the virtual circuit in the network which is large enough to accommodate a number of cells which is the larger of TC and the product of CT times the quotient of the round-trip time divided by the length of the most recent renegotiation interval.

11. The method set forth in claim 10 further including the step of:
keeping track of whether a cell buffer for the given virtual circuit at the edge overflows; and if the cell buffer overflows during the renegotiation interval, requesting a round-trip window in the step of requesting a window.

12. The method set forth in claim 10 wherein:
when the requested window is larger than the current window, the step of requesting a window may be performed at any time at which there is no pendingrequest for a window; and when the requested window is smaller than the current window, the step of requesting a window may not be performed until the end of the current renegotiation interval.

13. The method set forth in any of claims 10 through 12 wherein:
the renegotiation interval is a small number of seconds.

14. A method employed in a node of a virtual circuit network for sizing a window in the node for a given one of k virtual circuits using a given link from the node in response to a request for a different-sized window in the node, the method comprising the steps of:
determining the maximum window to which each of the k virtual circuits is entitled according to a function whereby the maximum window for a virtual circuit j is greater than the maximum window for a virtual circuit j+1, 1jk;
determining the window for each virtual circuit in the node and the rank of each virtual circuit with regard to the current size of its window;
determining a potential rank which the given virtual circuit would have if its window were the requested size; and changing the rank of the given virtual circuit in the direction required by the potential rank by exchanging its rank with that of the next ranking virtual circuit in the required direction until the given virtual circuit either attains the potential rank or until further changing the rank of the given virtual circuit would require changing the rank of the next ranking virtual circuit to a rank such that the current size of the window of the next ranked virtual circuit is greater than the maximum window forthe rank which the next ranked virtual circuit would receive as a result of the exchange; and sizing the window for the given virtual circuit such that the window's size is the lesser of the size of the requested window and the size of the maximum window for the final rank attained by the given virtual circuit.

15. The method set forth in claim 14 wherein:
the step of determining the maximum window Wi-1 to which a given virtual circuit i is entitled includes the steps of:
determining the average number of active virtual circuits Ji-1, using the link from the node; and determining the maximum window Wi-1 using the equation where W0 is a full-speed round-trip window.

16. The method set forth in claim 15 wherein:
a virtual circuit in intermittent use has an average service time requirement ? in which it is in active use and an average idle time of h? and the step of determining the average number of active virtual circuits Ji-1 is performed using the recurrence starting from J0=0.

17. The method set forth in claim 15 wherein a virtual circuit in intermittent use has an average service time requirement ? in which it is in active use and an average idle time of h? and the step of determining the average number of active virtual circuits Ji-1 is performed using the approximation for Ji-1.

18. The method set forth in claim 16 or claim 17 wherein:
the step of determining the maximum window size is performed using the longest round trip time of the network as ? and a value of h such that h? is a small number of seconds.

19. A method of controlling the congestion of data cells in nodes of a network for carrying cells of data via virtual circuits, the method comprising the steps of:
when a virtual circuit is established, assigning a default cell buffer for the virtual circuit at each node;
for the duration of the virtual circuit, performing steps including determining from the behaviour of the virtual circuit whether a different-sized cell buffer is needed;
if a different-sized cell buffer is needed, providing a first resizing signal indicating the different size to the nodes in the virtual circuit;
in each node of the virtual circuit, responding to the first resizing signal by indicating the buffer size which the node can provide if that is less than the different size; and in each node of the virtual circuit, setting the cell buffer for the virtual circuit to a buffer size which is no greater than the smallest buffer size indicated by any of the nodes.

20. The method set forth in claim 19 wherein:
the step of determining whether a different-sized cell buffer is needed is performed at a first edge of the network at which the virtual circuit enters thenetwork; and the method includes the further step of receiving the first resizing signal at a second edge of the network at which the virtual circuit leaves the network and selecting the buffer size to which the cell buffers in the nodes are to be set.

21. The method set forth in claim 20 wherein:
the method includes the further step of providing a second resizing signal containing the selected buffer size from the second edge to the nodes; and the step of setting the cell buffer for the virtual circuit is performed using the selected buffer size specified in the second resizing signal.

22. The method set forth in claim 21 wherein:
the step of providing the first resizing signal is performed by serially providing the first resizing signal to every node in turn.

23. The method set forth in claim 22 wherein:
the step of providing the second resizing signal is performed by serially providing the second resizing signal to every node in turn.

24. The method set forth in claim 20 wherein:
the step of determining whether a different-sized cell buffer is needed is performed by doing steps including:
keeping track of the total number of cells TC that the given virtual circuit has at the first edge and in the network;
keeping track of the cumulative throughput CT of cells of the given virtual circuit during a most recent renegotiation interval; and requesting a window for the virtual circuit in the network which is large enough to accommodate a number of cells which is the larger of TC and the product of CT times the quotient of the round-trip time divided by the length of the most recent renegotiation.

25. The method set forth in claim 24 further including the step of:
keeping track of whether a cell buffer for the given virtual circuit at the first edge overflows; and if the cell buffer overflows during the renegotiation interval, requesting a round-trip window in the step of requesting a window.

26. The method set forth in claim 24 wherein:
when the requested window is larger than the current window, the step of requesting a window may be performed at any time at which there is no pendingrequest for a window; and when the requested window is smaller than the current window, the step of requesting a window may not be performed until the end of the current renegotiation interval.

27. The method set forth in claim 26 wherein:
the renegotiation interval is a small number of seconds.

28. The method of claim 19 wherein the step of responding to the first resizing signal includes the steps of:
determining the rank of each virtual circuit relative to the other virtual circuits from the current size of each virtual circuit's window;
determining a potential rank which the given virtual circuit would have if its window were the requested size;
changing the rank of the given virtual circuit in the direction required by the potential rank by exchanging its rank with that of the next ranking virtual circuit in the required direction until the given virtual circuit either attains the potential rank or until further changing the rank of the given virtual circuit would require changing the rank of the next ranking virtual circuit to a rank such that the current size of the window of the next ranked virtual circuit is greater than the maximum window forthe rank which the next ranked virtual circuit would receive as a result of the exchange, the maximum window for a rank being determined according to a functionwhereby the maximum window for a virtual circuit j is greater than the maximum window for a virtual circuit j+1, 1jk; and sizing the window for the given virtual circuit such that the window's size is the lesser of the size of the requested window and the size of the maximum window for the final rank attained by the given virtual circuit.

29. The method set forth in claim 28 wherein:
the maximum window Wi-1 to which a given virtual circuit i is entitled is determined by steps including:
determining the average number of active virtual circuits Jk-1 using the link from the node; and determining the maximum window Wi-1 using the equation where W0 is a full-speed round-trip window.

30. The method set forth in claim 29 wherein:
a virtual circuit in intermittent use has an average service time requirement ? in which it is in active use and an average idle time of h? and the step of determining the average number of active virtual circuits Ji-1 is performed using the recurrence starting from J0=0.

31. The method set forth in claim 29 wherein a virtual circuit in intermittent use has an average service time requirement ? in which it is in active use and an average idle time of h? and the step of determining the average number of active virtual circuits Ji-1 is performed using the approximation for Ji-1.

32. The method set forth in claim 30 or claim 31 wherein:
the step of determining the maximum window size is performed using the longest round trip time of the network as ? and a value of h such that h? is a small number of seconds.