CA2089789C - Method and apparatus for optimum path selection in packet transmission networks - Google Patents

Abstract

A packet communications system utilizes a route determining mechanism by identifying principal paths between the source and the destination in the system. Principal paths are minimum hop count paths with a transmission delay less than a specified threshold. Principal path links are accepted as legs of the optimum path, if feasible, i.e., if the resulting load on the link is less than a specified principal threshold. Secondary links are accepted only if the resulting load on the link is less than a specified secondary threshold, where the secondary threshold is less than the principal threshold. All paths must also have a transmission delay less than a specified threshold. Each request for a route includes the source node, the destination node, the load required, the maximum transmission delay and, if desired, the quality of service parameters which all of the legs of the route must satisfy. A modified Bellman-Ford breadth-first searchalgorithm is used to identify the principal links and, using these principal link identifications, determining the optimum path.

Description

Dock;et Number RA9-91-044 METHODS AND APPA*ATUS FOR OPTIMUM PATH SELECTION
IN PACKET TRANSMISSION NEl~VORKS

Technical Field This invention relates to packet lln,~ ion sy~lellls and, more S particularly, to OplilllUIll path selection for cQnnP~tions l~lwæn two nodes in such ~y~lellls.
B~kpround of the Invention It has become i~l~;leasin~y useful to Ll~l~col~ L a plura1ity of data ~roce-c~ g e1emPnt~ by means of a packet ,wil~ lg nC~WOlk in which data is 10 ll~slllilled as data assemblages called "~acL~." Such l~lwulk~ include a plurality of inter~l,..ectecl l~twul~ ~wil~hing nodes which, in turn, are conn~ctçd to end nodes ~u~lLillg the data proce~ equipment. Such packet nelwu.ks can become quite large with an ~Ale-¢ive geographical distribution. In such a situation, the selection of an efficient path l~lw~n two end nodes which 15 wish to co-l--llul-icate with each other becomes of paramount illlpol~lce.
The major criteria for sele~hng paths L~lweell nodes in packet networks are lllhlilllulll hop count and lllil~illlulll path length. The hop count is the number of links used to construct the path I~Lweell the two end nodes. The path length is a function of the overall tlnl-sl~ ;on delay impased by the path 20 between the two end nodes. In most high speed n~lwu~ks, the delay (path length) is not a major consideration since the worst-case delay through such n~lwolks is nearly always acceptable. The hop count, however, is a direct measure of the amount of resour~es required to implement a given path and hence is of considerable in-~l Lallce in selecting paths.
It is to be noted that a selected path need not be a minimum hop count path since congestion on the n~Lwo.k Iinks may force the choice of a larger hop count path. However, such longer alt~l-lale paths cannot be allowed to grow without limit since inordinal~ amounts of network resol.lces might be committed to the one path, res~ in~ in further cnngestion for other paths and 30 forcing yet longer hop count paths to be selected for yet other connections. The long term network throughput could thereby be ad~ ely affected.

Docket Number RA9-91-044 2~89789 The plo'olf;lll, then, is to select a path l~lw~ll an origin node and a ~lf-~shn~1ion node which has a IllinilllUm hop count, a minimum path length, and which does not utiliæ an i,lor~lhlale ~mollnt of n~two~h l`eSOUl`~S.
S~ ry of the Invention S In accordallce with the ill~ live embo~limfnt of the plesent invention, O~)tilllUIII paths l~lwæll orig-in and df~~ ;on nodes in a packet netw-ork are sele~tf~ by a modification of the so called '~ m~n-Ford algolilhlll," a shortest path on a wf~ightf~l graph algolillllll taught by D. P.~ll~kas in Dynamic Progr~r,~,.ing: Deterrninistic and Stochastic Models, pages 318-322, Prentice-Hall, 1987, Englewood Cliffs, New Jersey, and D. P.
Bertsekas and R. Gallager in Data Networks, pages 315-332, Prentice-Hall, 1987, Englewood Cliffs, New Jersey. More particularly, the algo,iLIllll of the pl~elltinvention defines "plin~il)al paths" l~lw~l~ any given origin-de~ tion pair. A
~lhlcipal path is defined as a feasible nlil~ unl-hop count path and plill~ipal 15 links are definf~l as links in a plillcipal path. All other linl~s are defined as se~n~1~ry links. Secolldal~ paths are paths including at least one secondary link and including more than the millilllulll-hop count.
A plillcipal path is acce~ted as a route if none of its principal links is saturated, i.e., exceeds its pre~ignf~l traffic load. A secondary path, 20 however, is accepted as a route only if none of its principal links, if any, is saturated and if the load levels on its secondary links are below a preselected threshold (typically lower than that for links df-sign~te~l as plincipal). If this load threshold is f'~lf~l on any of the secondary links, the secondary path is rejected as a route.
One advantage of the path selection technique of the pleselll invention is that a maximum path length coll~lldilll can be imposed on the path selection ~roces~. That is, feasible paths can be tested to determine if the path length col~lldilll has been eYc~leA and leje~ed if the CO~lldillt iS eYCee~1PA.
Such ccl~lldinl~ can be used to prohibit Lloldinate resoulce consumption in 30 implementing a route and can also be used to impose specific grade of servicerequirements such as avoidance of low speed links. For this reason, the path length col~lldillls must typically be specified for each connection request, and the plillcipal links dete,lllined se~dtely for each c~nnpction request.

2 0 8 9 7 8 9 Docket Number RA9-91 044 In su~.. ~.y, the route selection technique of the pleselll invenffon h,volves two phases. In the first phase, the p,i.~;ipal links are identified for the requested c4....-o~t;0n. If no maximum length col~ h~l is i."l~l, the plillcipallinhs ~Iw~l~ any two nodes of the nelwu-h can be ~leco.l-pul~] and stored for 5 use in the second phase of the algolill-,ll. If a maximum length c~l~l.aint isil"posed, the p,i,~ipal links must be c~lc~ te~l for each new c~ ;on request, C~...p~ lg each linh length with the c~ illl, or the c~ t decreased by the previously accepted plillci~al link lengthc The route det~.l.i-lation te~hn;ql~e of the ~r~sellt invention has the 10 adv~nt~ges of prodl~cing o~ .lUI-l paths I~GlW~II a~ uy nodes of a packet i~ g system, taking into ~ç~ol~nt not only the hop count and the path length, but also i,n~i-lg a ma~ .,um ceiling on the path length. In addition, the computation of optimum paths is sufficiently rapid that a path co..lpul~tioncan be made for each request for a col...P~;l;on.
15 Brief D~sc-;~tion of the D- a~
A complete undel~ 1ing of the pleselll invention may be gained by considering the following detailed description in conjunction with the o...p~..yillg drawings, in which:
FIG. 1 shows a general block diagram of a packet communications 20 nelwol~ in which the route delelmil,alion system of the present invention might find use;
FIG. 2 shows a graphical r~rese"lation of the header of a data packet which might be llanslllitted on the packet co-.~ u"ications network of FIG. 1;
25FIG. 3 shows a block diagram of a ty-pical decision point at the entry point for p~ s entering the Il~Lwulk of FIG. 1;
FIG. 4 shows in tabular form a portion of the topology data baæ
in each decision point such as that shown in FIG. 3 and which is used to calculate optimum paths;
FIG. S shows a general flow chart of the path co-.. pu~lion procedure of the plesel~l invention;
FIG. 6 shows a more detailed flow chart of Phase I of the procedure of FIG. 4;

2 0 8 9 7 8 9 Docket Number RA9-91-044 FIG. 7 show a more detailed flow chart of Phase II of the proceJure of FIG. 4;
FIG. 8 is a yet more detailed flow chart of Routine A of the Phase I portion of the piocedule of FIG. 5;
S FIG. 9 is a yet more det~il~ flow chart of Routine B of the Phase I portion of the ~locelule of FIG. 5; and FIG. 10 is a detailed flow chart of the search algo.illll.l used in Routine A of FIG. 8.
To facilitate reader undel~ ding, identical reference numerals are 10 used to decign~te elements common to the figures.
Detailed Description Rere~ g more particularly to FIG. 1, there is shown a general block diagram of a packet l~ s...i~cion system 10 comprising eight network nodes 11 numbered 1 through 8. Each of network nodes 11 is linked to others 15 of the network nodes 11 by one or more communication links A through L.
Each such communication link may be either a permanent conlle~lion or a selectively enabled (dial-up) connection. Any or all of network nodes 11 may be attached to end nodes, network node 2 being shown as attached to end nodes 1, 2 and 3, network node 7 being shown as attached to end nodes 4, 5 and 6, and 20 nelwulk node8 being shown as attached to end nodes7, 8 and 9. Network nodes 11 each comprise a data pr~ g system which provides data communications services to all col~e~;led nodes, llelwu k nodes and end nodes, as well as decision points with the node. The ~lelwolk nodes 11 each comprise one or more decision points within the node, at which i..~ g data p~et~ are 25 selectively routed on one or more of the outgoing communication links ~elllli..~ted within that node or at another node. Such routing decisions are made in l~onse to informaffon in the header of the data packet. The network node also provides ~n~ ry services such as the calculation of routes or paths l~lween tellllinal nodes, and providing dir~loly services and m~i..le..~ e of 30 network topology data bases used to support route calculations.
Each of end nodes 12 colnplises either a source of digital data to be l.~lllilled to ~nother end node, a ~ltili7~tion device for c~n~ digital data received from another end node, or both. Users of the packet collllllul ications network 10 of FIG. 1 utilize an end node device 12 Cf.~

Docket Number RA9-91-044 208~789 . s to the local llclwolk node 11 for access to the packet llelwol~ 10. The local nt;lwolh node 11 tr~n~l~tes the user's data into p~c~ots formatted ap~l~liately for Ir~ cion on the packet l~lwolh of FIG. 1 and ge~ tes the header which is used to route the p~L~I~ through the n~w~llh 10. The header has the S general form shown in FIG. 2 and inclufles control fields 20, a routing field æ
and a re~1lm~1~n~y check byte 23. The roulillg field 22 cQ..~ the illlc:lmation l-ec~ to route the packet through the packet llt;lwulh 10 to the (~ ;on end node 12 to which it is addl~ed.
In FIG. 3 there is shown a general block diagram of a typical 10 packet l~lw~lh decision point such as is found in the llclw~Jlh nodes 11 of FIG. 1. The decision point of FIG. 3 colllplises a high speed packet ~wil~llillgfabric 33 onto which p~rl etc arriving at the decision point are entered. Such packets arrive over l,~s--.i~;on links via ll~ ;on ada~)te~ 34, 35, ..., 36, or originate in user applications in end nodes via application adapt~l~ 30, 31, ....
15 32. It should be noted that one or more of the ll~s...i~sion ada~tel~ 34-36 can be connected to intranode ll~..~...i~sion links connected to yet other packet switcl~ing fabrics similar to fabric 33, thereby expanding the ~wil~hillg capacity of the node. The decision point of FIG. 3 thus serves to co..~ the p~etc arriving at the decision point to a local user (for end nodes) or to a Ll~ cion 20 link leaving the decision point (for network nodes and end nodes). The adaptel~ 30-32 and 34-36 may include queuing circuits for queuing p~ets prior to or subsequent to ~wilcl~ g on fabric 33. A route controller 37 is used to calculate o~tilll~nl routes through the l~clwolh for packets orig;.~l;..g at thedecision point of FIG. 3. A topology data base 38 CQ-~ .C information about 25 all of the nodes and l~ on links of the llelw~lh of FIG. 1 which information is used by controller 37 to calculate optimum paths.
The route controller 37 of FIG. 3 may comp-ise disclete digital cil.;uilly or may preferably comprise pro~lly ~rogl~mll.ed digital com~LIter circuits. Such a plogl~llllled colllp~lter can be used to generate headers for 30 p~rl~.tc origin~bng at user applications in the decision point of FIG. 3 or co.~ ct~l dir~lly thereto. The h~llllation in data base38 is IJ~ ted when new links are activated, new nodes are added to the l~lwol~, when links or nodes are dropped from the llelWUlk or when link loads change significantly.
Such il~lol---ation ori~gin~tes at the llelwulk node to which the resoulces are 2 0 8 9 7 8 9 Docket Number RA9-91-044 ~cherl and is eY~nged with all other nodes to assure u~to-date topological information needP,d for route c~ tion. Such data can be carried on packets very similar to the i~.m~lion pa~et~ eYcll~ngp~l betwGGIl end users of the I~GlW~lh.
The in~ ........ i.ccion links to the packet decision point of FIG. 3 may Colll~lisG links from local end nodes such as end nodes 12 of FIG. 1, or links from adjacelll nGlWOIk nodes 11 of FIG. 1. In any case, the decision point of FIG. 3 ope.a~es in the same fashion to rfflive each data packet and folwd~d it on to ~ .e~ local or remote decision point as dictated by the 10 information in the packet header. The packet lletwulk of FIG. 1 thus o~l~tes to enable collllllul~lcation l~twæn any two end nodes of FIG. 1 without dedicating any ll~ ,e;on or node facilities to that c4.~....un;cation path except for the duration of a single packet. In this way, the uhli~Ahon of the collllllul~ication facilities of the packet nclwulh is o~;...;,Pd to carry significantly 15 more traffic than would be possible with dedicated lr~ls~ni~sion links for each comlllunication path.
In FIG. 4 there is shown in tabular form a portion of the information stored in the data base 38 of FIG. 3. As can be seen in FIG. 4, a number of dirrelGIll characteristics of each link in the llelwol~ is stored in the 20 data base. For the l~ul~oses of the ~rcselll invention, only a few of these characteristics will be ~ c~s~3. As might be ~ ~1, one of the critical characteristics of the ~ .c.ni~cion links is the load threshold available on that link. Moreover, it is well known that such ll~ n,i~sion facilities can only be loaded up to a fraction of their theoretical maximum load if reAco..Able 25 ll~ "~i~cion l,ropelLies are to be ,..Ai~ in~. The load threshold of such a llt~..~..li~sion facility can be rcpl~nted by the quantity CL~ the errcc~ c loadc~r~bility of the t~...c..~i~;on link l~lwæll nodes k and l. For r~asolls to be scd helcinarlel~ two dirrelcllt load thresholds are ~lefinpA for each l~A..~.,.i~cion links, ~1epen~1ing on whether the link is se~ectçd as a ~,linci~al link 30 in a route or as a sec~nd~ry link in the route. A plillcipal link is d~-fined as a leg of a plincil)al path where a plill ipal path is a feasible Illinil~ ll hop count path ~tW~ll the origin~ting node and the de~ t;on node. The hop count is simply the number of tl~ ...icsion links in the path. All other links are defineA to besec~ links. Any non-minimum hop count path between the origin~ ng Docket Number RA9-91-044 208978g node and the ~ l;on node is called a seC~ ry path and it always incll~rle at least one sec4n~1~ry link. In accoldal~ce with the present invention, a ~ lcipal path is l,rerelled over a s~ y path in det~~ lg o~t;,~ routes l~lweell nodes. If, hvw~, a ~lincipal path is not available due to its already being 5 fully loaded, a secQnd~ry path can be chosen. In order to dis~il;...;..~te against such a seC~n~l~ry path, a load threshold is der~ for each 5ec4~ Iink which is less than the collc~ lin~ llci~l load threshold for that same link. Thus the table of FIG. 4 includes two ~lirrerellt load thr~ho1~1s for each ll~ i~;on link, one to be used if the link is a plh~ )al link in a route being calc~ tYl and 10 the other to be used in the link if a seCo~ Iink in the route being c~1r~ te 1.
Also shown in the table of FIG. 4 is the Total ~ tefl Load, T(AL), for each link. This va1ue rt;~lesell~ the total load which has already been allocated for this ll~ ion link due to previously c~1cu1~ted routes. If the diL~lellce l~lw~ll this already allocated load and the total available p~ al15 or secondary load of the ctl~nn~-1 (depenrling on the whether the link is a plhlcipal link or a seco~-d~.y Iink) is not sufficient to carry the new connection, then the link cannot be selecte~l. In accordance with the preselll invenffon, inaddition a path can be selected only if the overall path delay does not exoeed amaximum delay defin~l as Pr~ In order to calculate the incremental delay 20 introduced by this link, d," the following formula can be used, using the values show in the table of FIG. 4:

d C (1) where c~ = Total Bandwidth of Tr~nsmi~cion Link from Node k to l, = c~, p (FIG. 4) if the link is a principal link, or = C~ s (FIG- 4) if the link is a secondary link, c (1) = Allocated Bandwidth Before This Connection, and c (2) = Allocated Bandwidth After This Connect on.
To Sup~ll the calculation of equation (1), the topology data base 38 co..~
30 for each link, not only the load threshold (e.g., c~p and c"5), but also the ~;ullelllly ~ te~l load for each ll~ ---icsion link (e.g., c~,). The definiffon of Docket Number RA9-91-044 ~089789 the increment~l delay given in equation (1) is for illustration yul~ses only andmany other formulations can be used. This incremellt~l delay is sulJlla~ from the maAh~lulll delay to yl~uce a new maAilllunl delay available for future linksin the selectetl route.
S In addition to the bandwidth hlrollll&lion .l;~l.~J above, the topological data base 38 of FIG. 3 and illu.,ll~ted in FIG. 4 includes so-called"Quality Of Service (QOS)" p&lanletcl~,. Such QOS p~alll( tClS can be speciffed in a particular request for an oythlluln route and the y~cseill invenffon < ~lates to insure that all of the selected links in the path con~oll~ to the requested QOS
10 parameters. Such QOS parameters may include, for cA~ul~le7 a parffcular sc~;ulily level, a maAhll~ prop~g~tion delay or a l~ in~unl probability of buffer overflow.
In accordallce with the yl~7ent invention, the total amount of delay pcllllillcd for each route has the finite limit Pr. This limit is imposed to 15 avoid (A~7iVc delay and to yrcvelll the dedicaffon of an undue amount of resources to any one con.~1ion. Without such a limit~tion, it is possible to create an unstable condition where the lesoulces of the packet communications system are rapidly used up in ill. rcasingly more complex long paths. That is, the use of c~ces7ivc amounts of l'cSOurCcS for one route leaves less rcsoulccs available 20 for other routes, leqlliling even greater l`cSOul`~S for future routes.
If no limitation is put on the maximum delay ~lnlilled for each route, it is possible to calculate all plillcil)al paths and plillcipal links between any two nodes ahead of time and merely access them from a stored table when the route delelmillalion is done. In order to accommodate a finite limitation on 25 path delay, h~ r, it is ~-e~ to detellllille the principal paths, and hence the ~lhlcipal links, for each new route request.
In accordance with the l,reselll invention, each request for a route delcllllhlation includes the following input parameters:
Origin Node (i) D~stin~t on Node (~) Required Bandwidth (c) Maximum Path T~neth Threshold (PT) Docket Number RA9-91-044 -- - 9 - 2 ~ 8 9 7 8 9 Quality of Service Parameters (QOS), Optional Using these parameters, a path is det~ln;lled by first seal~hing through all links to det~ll,ine lllL~illlulll hop count path that satisfy the Illa~il"un, path length COl~tlaint PT and then l,a~ll~c~i,lg to derive a list of p,;llcipal links. This S search is ~Up~'e~n~nt~t by accumulating the link delays for each path. Once the tin~tion node is re~elle~l, Illil~llluln hop count p.i,~cipal paths are l~a~ll~c~ed to create a list of ~lincipal links. This plillcil,al link list is used in the second phase of the algo.;ll"" to dele.,l,i,le the o~ti",u", path from the source to the ~tp~t;,~t;on node. This specific ploced.llt; for dt:t~,lll,hlil,g paths through a 10 packet co",l"u,lications system will be described with referel~ce to the b~t~nc~ of the ffgures.
In FIG. 5, there is shown a general flow chart of the path co..ll,ut~tion ploced-lre of the pl~nt invention. Starting at start box 40, input box 41 is entered to specify the inputs req~lired to calculate an o~ ulll route in 15 accoldalloe with the presellt invention. As noted above, these input parameters include the origi~ node i, the d~stin~tion node j, the requested colln~tion bandwidth c, the maximum path length PT and, optionally, a group of "quality of service" parameters QOS. With these inputs, box 42 is enteled where, as Phase 0, the list of links in the netwu.k is pruned by removing all links which do not20 meet the QOS parameters. With this de~im~ted list of l,;..~ ;on links, Phase I
box 43 is enteled to identify the p.i~ al links. In del~,l...i..~t;.-~ the lengths of the ll;~ cion links in this plocess, it is ~sum~l that the uhli7~tion of each link is _ero (Ch, = o), i.e., the entire bandwidth of the link is available. Theproc~ used to search the llelwolk for plhlcipal paths is a modification of the 25 so-called Rellm~n-Ford algwiLhlll described at pages 318-322 of Dynamic Prog~am~irlg: Deterrninistic and Stochastic Models, by D. P. Bertsekas, Prentice-Hall, 1987, Englewood Cliffs, New Jersey. This algolilLIll will be described in detail hereillafhr.
Once the plhlcipal paths have been identified in bo~ 43, Phase II
30 box 44 is ent~ed where the o~lilllulll path is identified, using the ~Ihlci~al paths from box 43, and using the ;uilent uhli7~hon data (allocated bandwidth c~,).
As noted in box 45, the output from box 44 is an acceptable Il~illhlluln hop count path from the origin~hn~ node i to the ~ ;on node j which has the minimum possible hop count and which path length is less than the input Dock~t Number RA9-91-044 maximum of Pr. If no such path exists, an failure signal ~ is relullled. The process ends in l~ llnillal box 46.
In FIG. 6 there is shown a more detailed flow chart of Phase I of the flow chart of FIG. S in which the links of the packet cwlllllullications system S of FIG. 1 are lll~hed as L,lhlcipal or Sec4~ for this particular request. As noted, in boxS1 Routine A utilizes a modified R~llm~n-Ford algolitlLll to search all of the links l~lw~n the origin and the ~lestin~tion nodes, keeping track of the hop count and the path length increment for each link. Once the kesl;..;.~;on node is reached, boxS2 is entered where Routine B l,d~l,acks 10 through the links to dis~vel the paths with both a lllil~lnuln hop count and a length that satisfies the m~illllllll path length col~ l PT These paths are hlcipal paths and all of the linhs in these paths are ylhlci~al links. All otherlinks are secondary links. Only the information as to whether the link is principal or secon~l~ry is retained for use in Phase II of the procedure.
In FIG. 7 there is shown a more detailed flow chart of Phase II of the flow chart of FIG. S in which the optimum path is conlpuled. Starting in start box 60, box 61 is enl~led where Routine A, ecsen~lly identical to Routine A in FIG. 6, uses the same modified Re.llm~n-Ford algolillln~ to det~llllh~e thefeasibility and the link length for each possible link in the ~lltim~te path.
20 ~hllal~ links are accepted as feasible providing only that load capacity is available for the new co~ ;on, i.e., c(2) does not exceed the plil-cil)al link load threshold Ck, p. Secondary links are acce~ted as feasible provided that adding the new collne.;lion does not result in ~YC~li~g the secon~l~ry link load threshold c~,5. Links are accepted as feasible only if the increme.nt~l increase in path 25 length caused by the link does not hl-lease the total path length so that it is greater than the specified maximum path length Pr~ In box 62, the list of feasible links are luac~ cked through using the hop count and the path lengths to find the shortest path with the minimum hop count. The pl`~XeSS of FIG. 7 ends in lel~linal box 63.
In FIG. 8 there is shown a more detailed flow chart of Routine A
used in FIGS. 6 and 7. When used in FIG. 6 to identify the plhlcil)al links, theflow chart of FIG. 8 starts in box 70 from which box 71 is entered where a breadth-first eYh~l-stive search is made, starting at the origin node i. That is, each link from each node is followed to the next node where each link is again Docket Number RA9-91-044 20897~9 followed to the next node, and so forth. For each node k ~rc~ ed in this search, the ...i~ -.. hop count (h,~) from the origin node to node k iS saved along with the length (Ih) of the ~holt~t path from the origin to node k only ifthe length is less than Pr. When the ~ ;on node is reached in this search, S in box 72, the hop count h~ iS the ,llini",u,l. hop count from i to j such that the shortest path length is less than PT. If no such path exists, box 73 relu,lls a null value of ~. Routine A then h....~ t~ in box 74.
In FIG. 9 there is shown a more ~etAil~A flow chart of Routine B
used in FIGS. 6 and 7. When used in FIG. 6 to identify plhl~i~al links, the 10 flow chart of FIG. 9 starts in box 80 from which box 81 is enl~ r~l. In box 81, the path is retraced from the le~ l;on node j to the origin node i. Links are marked as ~lhlcipal in this bac~ c~ing if the link belongs to a path with h~illlulll hop count and length less than PT. In box 82, links which are not ~Ihlcil)al are l~lalked as seco..~ . Box 83 tel...i~At~ the ~ocess.
Relulnillg to FIG. 8, when Routine A is used to calculate the optimum path. The actual ~ltili7~tions of the links are used to detGlllline whether the links are feasible and the link length. When routine A is completed, a path has been identified with the IllinllllUIII feasible hop (i.e., a hop count equal to or greater than the Illh~illlulll hop count h~) and with a length less 20 than Pr. In FIG. 9, the hop count and lllinilllunl lengths associated with each node are used to identify the actual optimum path.
In FIG. 10 there is shown a detailed flow chart of the modiffed 12ellm~n-Ford algolill-lll used to ~l~llll the search for an optimum path. In FIG. 10, starting at box 90, box 91 is entered to set the next node to the origin 25 node i. In box 92, the data for the next node is retrieved from the topology data base 38 of FIG 3. In box 93, the data for the next link leaving that node is ob~ çA from the data base. In decision box 94 it is dete~lllilled whether or notthat link is a plincipal link. If the link is a principal link, decision box 95 is entered where the accumulated load c(2) is co...l~rGd to the link plil~ipal 30 threshold c~,p. If the accumulated load is equal to or less than the plillcipal threshold, box 97 is entered to calculate the illclG.nç~.t~l delay in accordallce with equation (1). If the accumulated load is greater than the plhlcipal threshold, as detelll~ ed by decision box 95, the next link is olut~ined in box 100.

Docket Number RA9-91-044 If it is dete~ ed in decision box 94 that the link is a sec~
link, decision box 96 is entered to C~J~.P~Ie the a~umlllated load c(2) to the link secondary threshold Ck~5. If the ~r~u--~ul~te~l load is equal to or less than the secondary lhl~llold, box 97 is enter~d to c~lc.~ te the hlc~...e.-t~l delay in 5 accord~loe with equation (1). If the accwlllllated load is greater than the sec~ threshold, as det~,llunel by decision box 96, the next link is ol7t~hled in box 100.
In decision box 98, the ~c~ ate~l path length up to this point is co-~ -ed to the m~ -,u,~ path length Pr. If the accumulated path length is less 10 than PT, the hop count h,~ and the ac~uln~llated path length dh, are save in a list in box 99. If the accumulated path length is equal to or greater than Pr~ box 99 isl~a~d and the data for this link is not added to the list. In either case, decision box 100 is then elllered to detell.lille if there are any more links exiting from this node. If so, box 93 is entered to get the next link and collthl.le the15 process. If there are no more links exiting from this node, decision box 101 is entered to det~llllhle if the node is the ~lestin~tion node j. If so, the process is complete and Lel---i-~tes in stop box 102. If this node is not the lestin~tion node , box 92is entered to get the next node and ~llhl~le the pl~S.
When the pl~lure of FIG. 10 is col.lplete, a list of the minimum 20 hop counts h~ and the path lengths dk~ is available. As ~ ~l above, Routine B then backtracks through this list to identify the path with the minimum path delay as well as the minimum hop count. This is the optimum path to be used in sending the packets involved in the colln~ion through the system of FIG. 1.
When used to identify the principal paths, the flow chart of 25 FIG. 10 is modified to omit the decision box 94. Using the principal load threshold of each link, the lllil~illl~ l hop count and the length of the path to each node el-co,l.~tered is kept on the list. This list can then be proce~l by backtracking to identify the plill~ipal paths (having both lllillilll~ hop counts and lengths less than PT)- The links of the principal path are the p~ cipal links 30 to be used in computing the optimum tl~ cion route. The entire proce(l~
for collll~uling plhlcipal links and OptilllUlll routes is described in the pseu~loc~e in the ~tt~rh~l Appendix. The colles~...~lence l~lween the pseu-loc~le and the flow charts of FIGS. 5 through 10 is obvious and will not be described in detailhere.

Docket Number RA9-91-044 It should also be clear to those skilled in the art that further embo l~"~.,L~ of the pleselll invention may be made by those skilled in the art vithout departing from the te~rlling~ of the p~ ll invention.

2 0 8 9 7 8 9 Docket Number RA9-91-044 APPENDIX
Minimum Path Algorithm The algo,illLIl to be described for c~,l.pulillg the best possible path with the IllL,ilnul,l possible hop count ~ "~c the availability of a set of all of 5 the p,i,l.ipal links l~wæn the origin node and the f1e~ Alion node. This p~ ipal link list can be preco,,,puled by ~sv~ g that the Illi,~i",u", path length is illril ily (Pr = ~), or can be co~ ulecl, as will be described, by- I)hli~ing the same algo,;lll", in a phase ~re~l:~lg the path co...~ t;1~;on phase.

Definitions:

i is the index of the origin node.
j is the index of destin~tion node.
N iS the total number of nodec in the nelwo,k.
h iS the step number of the algorithm, equal to the hop count.
PT is the ma~i~l,u~l path length ~lwæn origin and ~ ;on nodes.
h~ iS the nlillilllulll hop count of a selected path of length lecs than Pr (if such a path exists).
dL, iS the length of the link (if any) between nodes k and I
D,(l,h) iS the length of the shortest path ~lwæn nodes ~ and j of exactly h hops (D,(l,h) = ~ if no such path exists). D,(l,h) can be r~p~e~lp~cl by a (sparse) two--limen.cional array, with indices l ,h = O, 1, , N-1.
A (k ,1) iS a function that is equal to "1" if the link is acceptable, and "0"otherwise. This function ~lepen-ls on whether the link kl iS plil,cipal or not.

Note that both of the functionc A(k,l) and d,t, llepend upon the 25 c~".~ ;on request.

Docket Number RA9-91-044 20897g9 The following pse~ ococle is used to both identify p~ cipal links and to c~...p.~le the best ac~e~t~ble path. This algolilhlll is first described to colllp.lte the best acce~)t~ble path, ~su~..i.-g that the principal links have already been identified. Ther~ter, it will be shown how this same algolillllll can be 5 used to identify all of the ~li,lcipal links. The inputs to the algolilh,ll are a co~ ~;on r~uc~l, the index i of the origin node, the index J of the de~ t;on node (i 9~j), and the path length ar~ t;.~ ILlc~hold Pr. The algorithm will produce an acceptable ",ini,l,u,ll hop path from i to j with l~ h~u~ length lessthan PT~ if such a path exists. If no such path exists, the algolill,lll retU11IS a 10 value of ~.

Best Path Pseudocode /*----- Initi~1i7~tion-----*/
for (h = O;h <N;h++) {

for (l = o;l <N;l++) {

Di (I ,h )=~;

20 Di(i,O) = 0;
/*----- Detell"ille Shortest Path From Origin to Deshn~t on -----*/
h~ = O; /*----- Initialize Final Hop Count to 0 -----*/
for (h = O;h <N;h++) {

for (I = 0;1 <N;l++) {

/*----- Shortest Path From i to I in (h + 1) Hops -----*/
for (k = O;~ <N;k++) {

if (A (k ,l ) = 1) then {

2 0 8 9 7 8 gD~cet Number RA9-91-044 Dl (t, h + 1) = min [D~(l ,h + 1), DI (k ,h ) + d~d ]
}
}

if (Dl (J ,h + 1) < Pr) then /*----- Found a Path Within the Threshold Pr -----*l hf = h+1;
break;
}

10 }
if (hf = O) then Return ~ /*----- No Path -----*/
/*----- Retrace Selected Shortest Path From Deslin~ion to Origin -----*/
Path = j;
k = j;
for (h = hf--1; h ~ 0; h----) {

for(l =0;1<0;1++) {

if (A (I ,k) = 1 and D~(k, h +1) = Dj(I ,h) + d~) then {
Path= l,Path;
k = I;
break; /*----- See Comment Below -----*/
}

}
}

Path = i,Path;
Return Path;
COMMENT: It is assumed that either a single predec~or exists, or the ffrst 30 predec~ or is picked when more than one exists. A prelelled allt;l,~li~e is to col~lluct a list of all possible p~ o~ and select one prelec~-~or from the list by a random plocess.

2 0 8 9 7 ~ ~cket Number RA9-91-044 Principal Link Identification If the co~ ;on ~u~l specifies a ffnite length of the acce~)tallce threshold Pr, then the first step must be to d~le~ ine the ~ ipal links ~o~te l with this request for a new llelw~,lk co~ 1;on. The above-described 5 algoli~lllll iS used with the following modifications:
A new acce~ ce function A'(k,l) only checks for links that cannot accomodate the new co.~ ~l;on since pli~ al links are not yet known and a special load threshold for non-~li,l~i~l links is me~ ,lP~.
The weight of the link k I (wilhoul considering e ;~ g ne~wolh traffics) is given by d'~,.
It is assumed that A (k,l) ~ A' (k,I) and that 0 5 d',d s d,d for any link kl .
Retracing the path from the d~l;.~l;on to the origin can be omitted.
Colll~ul~lion of d'~d and A'(k,l) ~ss~m~ that the illconlillg col~le~lion is the only one on the linh. This el;...ilu~ links that are either not r~p~ble of Call~ g the co~ e~l;on, or are part of a path whose length can never be lower than Pr-The output of this algolillllll includes not only the hop count h~ and the path length DiC~ ,hf ) of the best feasible path l~lw~en the origin and the ~l~l;lli1t;on~ but also the length of all lower hop count paths to possible transit nodes. This h~follllalion is used to identify ~lhlcipal links.
In general, plil~ipal links are identified by backtracking from the destin~hon node j, breadth-first and by decreasillg the hop count, all computed paths sati~yillg the length col~ l Pr. More particularly, 1. Starting with rlestn~t-on j, inspect all adjacent nodes l that are one hop away to del~ ille if they satisfy D~(l ,k~-1) + d'" < PT-2. For each node I satisfying (1), mark the link lj a principal and define a new length threshold Pr(l) = PT - d'lJ. By coll~lllion, it is assumed that PT(J ) = Pr -3. After i.~ ;ng all adjacent nodes, decrement the hop count by 1 (h = hf--1).

2 0 8 9 7 8 9 Docket Number RA9-91-044 4. Tn~pect all nodes k that are two hops away from ~leshn~hion node j.

5. Mark all links kl as pliucipal if D,~(k,h~--2) + d',~, iS less than PT(l)-6. Define a new length threshold for node k such that Pr(k ) = PT(l ) ~ d'~ .

7. Repeat steps (1) through (6) until the hop count is equal to zero. At this S time, all plhlci~al links have been identified.

Principal Link Identification Pseudocode /*----- Inih~1i7~hon-----*/
Principal= ~
for (l=o; l<N;l++) 10 {
Pr(l) = O
}

PT (1 ) PT
/*----- Decrement Hop Count -----*/
15 for (h = hf; h >O; h - -) {

PrinLinks = {}; /* Initialize the I'lincil,al Link List to an empty list */
for (l = O;l<N;l++) {

for (k =0; k <N; k + +) {

if (A'(k,l) = 1) then {

if (Di(k,h-1) + d~ < PT(l)) then {
Prin~ ;nl~c = PrinLinks U {(k ,l )}
}

}
}
Principal - Principal U PrinLinks;

Docket Number RA9-91-044 __ - 19 -/~ Now update the accumulated delay m~sim~ Pr ~l for (k ,l ) in PrinLinks {

Pr(k) = ma~[PT(k), PT(I) - d'",];
S }
}

return Principal;

Claims (15)

1. A packet communications system comprising a first plurality of packet switching nodes for receiving and sending data packets in said system, a second larger plurality of transmission links interconnecting pairs of said packet switching nodes, means for determining optimum routes through said communications system between a packet originating node in said system and a packet destinationnode in said system, said means for determining optimum routes comprising means for identifying principal paths between said originating node and said utilizing node, said principal paths each including a minimal hop count and, if more than one principal path has the same minimal hop count, the one having a minimal transmission delay, where a hop is a single one of said transmission links in said principal path, means utilizing said principal path identifications for determining an optimal path between said originating and destination nodes, and means for accepting said optimal path only if said optimal path also has a transmission delay below a preselected threshold.

2. The packet communications system according to claim 1 wherein said means for determining optimal paths includes means for utilizing a Bellman-Ford algorithm.

3. The packet transmission system according to claim 1 wherein said means for determining an optimal path comprises means for weighing each said transmission link in accordance with a weighing function dependent on the non-allocated bandwidth available in the corresponding one of said transmission links.

4. The packet transmission system according to claim 3 wherein said means utilizing said principal path identifications comprises means for providing a larger effective bandwidth for links when in said principal paths than for links when not in said principal paths.

5. The packet transmission system according to claim 1 wherein said means for determining principal paths comprises means for weighing each said transmission link according to the equation where dkl is the incremental delay between nodes k and l, Ckl = total bandwidth of said transmission link from Node k to l, ?kl(1) = already allocated bandwidth before this route, and ?kl(2) = allocated bandwidth after this route.

6. A route controller for a packet communications system comprising packet switching nodes interconnected by transmission links comprising means for identifying principal paths between an originating node and a destination node in said system, said principal paths each including a minimal hop count, and, if two or more of said principal paths have the same minimal hop count, the path also having a minimal transmission delay, where a hop is a single one of said transmission links in said principal path, means, responsive to said means for identifying principal paths, for determining an optimal path through said communications system between said originating and destination nodes, and means for accepting said optimal path only if said optimal path has a transmission delay below a preselected threshold.

7. The route controller according to claim 6 wherein said means for identifying principal paths includes means for utilizing a Bellman-Ford algorithm.

8. The route controller according to claim 6 wherein said means for identifying principal paths comprises means for weighing each said transmission link in accordance with a weighing function dependent on the non-allocated bandwidth available in the corresponding one of said transmission links.

9. The route controller according to claim 8 wherein said means responsive to said means for identifying principal paths comprises means for specifying a larger effective bandwidth for links when in said principal paths than for linkswhen not in said principal paths.

10. The route controller according to claim 6 wherein said means for identifying principal paths comprises means for weighing each said transmission link according to the equation where dkl is the incremental delay between nodes k and l, Ckl = total bandwidth of said transmission link from Node k to l, ?kl(1) = already allocated bandwidth before this route, and ?kl(2) = allocated bandwidth after this route.

11. A method for determining routes in a packet communications system comprising the steps of interconnecting a first plurality of packet switching nodes for receiving and sending data packets in said system with a second larger plurality of transmission links, identifying principal paths between an originating node and a destination node in said system, each said principal path including a minimal hop count and, if two or more of said principal paths include the same minimal hop count, the pathalso having a minimal transmission delay, where a hop is a single one of said transmission links in said principal path, utilizing said principal path identifications for determining an optimal path between said originating and destination nodes, and accepting said optimal path only if said optimal path has a transmission delay below a preselected threshold.

12. The method according to claim 11 wherein said step of identifying principal paths includes the step of utilizing a Bellman-Ford algorithm.

13. The method according to claim 11 wherein said step of identifying principal paths comprises the step of weighing each said transmission link in accordance with a weighing function dependent on the non-allocated bandwidth available in the corresponding one of said transmission links.

14. The method according to claim 13 wherein said step of utilizing said principal path identifications comprises the step of specifying a larger effective bandwidth for links when in said principal paths than for links when not in saidprincipal paths.

15. The method according to claim 11 wherein said step of identifying principal paths comprises the step of means for weighing each said transmission link according to the equation where dkl is the incremental delay between nodes k and l, Ckl = total bandwidth of said transmission link from Node k to l, ?kl(1) = already allocated bandwidth before this route, and ?kl(2) = allocated bandwidth after this route.