WO2000074305A2 - Method for routing in hierarchical networks - Google Patents

Abstract

An architecture, system, and method are provided for efficiently transferring packets of data across a structured communication network. The communication network is structured such that there are hierarchical levels of high-speed modules existing throughout the network. Distributed routing of packets is achieved by comparing identification numbers of only select modules with the destination addresses on a field-by-field basis. Not all fields need be compared at all modules. The routing operation between nodes or devices outside the structured network can therefore be achieved using a single mapping operation. An additional protocol or addressing domain specific to the structure is wrapped onto the incoming packet protocol using a series of read cycles to a memory which temporarily receives the incoming protocol. Thus, the structured network may include specific identification numbers assigned to each module separate and distinct from identification numbers of modules, nodes, or termination devices external to the structured network. Each packet may also be assigned a security code and a priority code as it enters the structured network.

Description

TITLE: RELATIVE HIERARCHICAL COMMUNICAΗON NETWORK HAVING DISTRIBUTED ROUTING ACROSS MODULAR SWITCHES WITH PACKETIZED SECURITY CODES. PARITY SWITCHING, AND PRIORITY TRANSMISSION SCHEMES

Field of the Invention:

This invention relates to a communication system ("network") and, more particularly, to a network of interconnected nodes and/or modular switches ("modules") that have a hierarchical identification address based on the topological, structured relationship of one module to another. Each module can have a structured address or identification number and, knowmg an end module address, a data packet targeted for that module need only perform a single mapping translation when traversing the network. Moreover, the entry end module can perform fast read operations to wrap an internet protocol ("IP") address with the structured addresses of modules throughout the structured network. The data can be forwarded across the structured network accordmg to security levels, reject unsecured accesses, authorize transmission from the network, and detect network locations undergomg security violations. Transmissions can be undertaken accordmg to a transmission pπoπty scheme whereby packets of data are sent between nodes and/or modules depending on the hierarchical level of the forwarding modules. Each packet can have one or more paπty bits in order to discern the header, trailer, and data portions of the packet, and to detect and/or coπect eπors contained therein. Even if the network is partially severed, transmission can be maintained.

Descπption of Related Art:

A communication network is generally regarded as an interconnected set of subnetworks or subnets. The network can extend over localized subnets as an intranet, or can extend globally as an internet between one or more intranets. A communication network can therefore forward data within a localized network between termination devices extending to almost anywhere around the world. The termination devices mclude any data entry/retπeval system (e.g., telephone or computer), and a network includes a local and/or global interconnection of termination devices configured on one or more subnets.

The basic underpinnings of network operation is the vaπous protocols used to communicate across the network. A popular foundation for those protocols is the Open System Interconnect ("OSI") model. Usmg that model or a deπvative thereof, protocols can be developed which work m concert with each other. A popular communication protocol includes the Transmission Control Protocol ("TCP") and the Internet Protocol ("IP"). TCP/IP are used in networks that are known as packet-switched networks. The advent of asynchronous transfer mode ("ATM") has brought about a divergence from packet-based standards to one usmg a cell-switched network. Packet-switched and cell-switched networks are in contrast with circuit-switched networks, such as the telephone system. As opposed to maintaining a fixed routing connection for the transmitted message, packet or cell switching evenly allocates or "switches" packet or cell portions of the message across dissimilar routes of the network. The term packet switching henceforth refers geneπcally to switching message portions, regardless of whether that portion is a cell or packet.

Popular protocols such as TCP/IP deliver data across the network in the form of a packet. Each packet of a particular message may be sent across different routes of the network at the same time, and then reassembled at the proper termination device. In order to ensure the packets are properly received, certain layers of the popular Open System Interconnect ("OSI") protocol stack will wrap the data before the packet is sent across the network. For example, TCP can divide data into segments that are then placed into, for example. IP l datagrams having a segment of data interposed between a header and a trailer The combination of header, data segment, and trailer is interchangeably called the IP datagram or "packet " The IP datagram can be further wrapped using a Point-to-Point Protocol ("PPP"), a popular such protocol being that which follows the Ethernet specification at the physical layer Thus, it is not until the data segment has been wrapped possibly numerous times will the packet be forwarded across the network

Coupled between subnets of a network are vaπous classes of devices, including bπdges, routers, and switches Bπdges and switches for the most pan operate pπmaπly in hardware and within the domain of the data link layer Legacy routers, on the other hand, operate within the network layer and can be programmed and reconfigured to intelligently forward packets across the network according to the most optimal routing path Although the distinction has become somewhat bluπed with modem day network layer switches and bπdges, it is still the primary intent that a switch and bπdge be employed mostly in hardware Based on configuration of the switch or bπdge. all data packets received by a particular input port are routinely sent to any particular output port or ports This operation can be quickly repeated each time a packet is received

While switches are relatively fast and can send data from an input port to an output port with very little delay or lag time, switches do not have the capability to adapt to change in traffic conditions as do routers

Because routers can easily adapt to avoid traffic bottlenecks, routers employ look-up tables and fairly complex routing algorithms Typical routers are based on a packet processor The processor can be programmed to determine the most optimal connection between input and output ports of the router Coupled to the processor is a memory bus and associated system memory, whereby routing tables can be stored in the system memory and are peπodically fetched by the processor whenever a routing decision is to be made Thus, when a packet is received on one port of the router, the processor will execute a look-up operation by fetching the appropπate table information from system memory Most routing tables are dynamic, in that they constantly change as network traffic and conditions change, so that the processor can interconnect the ports in a way that is most efficient at that time Dynamic routing tables can be built and re-built using various routing protocols, popular such protocols include Routing Information Protocol ("RIP") and Exteπor Gateway Protocol ("EGP")

Peπodically programming the routing tables may mean that one router on a nerwork will not know the location of the physically closest router or its placement within the network The router will simply route based on its existing routing table and the resulting look-up outcome Such a system can be considered a non-relative or non-deterministic router topography Non-deterministic routers tend to rely on the number of hops (i e , the number of routers or gateways) when determining the most optimal routmg path Thus, while a router may know the number of additional routers on a given path, it does not necessaπly know or have a link to the identity of those routers

In most modern networks, a packet will travel more than 10 or 20 hops before amving at its destination As defined herein, a hop is equivalent to a traversal of a router or address translator More specifically, a hop is defined as an operation that invokes a look-up or translation operation When travehng across a network, a packet may be sent across numerous look-up or translation tables, and also through numerous switches, before amving at its destination It is generally recognized that transmission latency is lessened, and transmission bandwidth increased, if the number of hops is reduced Therefore, it is desirable that the number of hops from one termination device to another termination device be minimal This is due, in part, to the time-consumptive look-up (or address translation modification) procedure performed by each router

Furthermore, since a particular router will not necessarily know the placement of all other routers of a network, each look-up table must include identification numbers or addresses for each of the other routers within the network - even those not w ithin the optimal routing path The destination address is compared against each of those identification numbers for each routmg operation If a network is quite large, the access times needed to perform a look-up from the packet processor to the system memory can be coπespondingly long This problem is compounded with the advent of high speed networks of one gigabits per second (Gb/s) or higher Look-up techniques needed to render forwarding decisions on rather large look-up tables drastically reduces the benefits of using high speed networks Many termination devices use high speed physical media, such as Ethernet Moreover, subnet loops which interconnect localized devices can use even higher speed signals such as, for example, Synchronous Optical Networks ("SONETs") If the IP packets enter the mput port of a router more quickly than that router can process those packets, the packets are sent to an mput queue The router processes packets from the queue m the order m which they are received If the number of packets received exceeds the capacity of the queue, packets may be lost and may need to be re-sent If the companson between the IP destination and the routmg table takes a significant amount of time, then it may be likelv that numerous packets may be lost Frequency of lost data would increase when bursts of data are being sent The time needed to re-send packets deleteπouslv hmders the overall throughput of the network

It would be of benefit to introduce an improved switch (hereinafter refeπed to as a "module" or, alternatively, as a "modular switch") havmg switching speeds of a conventional switch yet also havmg the adaptability of a router The improved module would avoid the throughput constraints of a conventional packet processor and system memory mterface and, more importantly, the detπments of an extensive look-up table. The desired module would be even more beneficial if it could maintain backward compatibility with the OSI model, protocols of that model, and address resolution used by vaπous Address Resolution Protocols ("ARP"). Still further, the improved module could desirably be used in an overall network that is deterministic as to the relative placement of each switch within that network Thus, the module could determine if it should wrap (or stπp) mcommg packets as an end module or should pass-through packets as an mtermediate or pass-through module, both modules advantageously performing those functions with minimal time delay

An important aspect of network management and administration is the need to control accesses to the network infrastructure Accesses can be controlled for numerous reasons, some of which include security and pπoπtization Security deals with restπcting improper accesses, while pπoπtization deals with pπontizing an access to a shared resource

There are vaπous ways in which to secure a communication network, all of which deal with mechanisms which prevent unauthorized access to packets of data, such access is often referred to as "packet sniffing" and or "packet spoofing" Common secuπty mechanisms include use of firewalls implemented m hardware and software (e g , proxy servers, bastion hosts, filtenng routers) and or authentication systems implemented in solely m software (e g , passwords and encryption code) Most firewalls use some form of screening subnet architecture that analyzes the mcommg internet packet to determine if that packet should be placed on the internal, intranet structure Analyzing the packet and, more specifically, the source and destination of that packet, typically adds a lag time or latency at the mterface between the intranet and the internet Placing an encryption code also adds overhead to the packet and mvolves time-consumptive decryption at the receivmg end of the network Use of passwords appears less consumptive of transmission bandwidth Passwords, however, can sometimes be readily broken either through a user's improper choice of password name or through a hacker seπdmg thousands of user names and passwords until a successful combination is achieved

It would be of benefit to secure a network without requiring the overhead of conventional firewalls For example, eliminating routmg tables within either an intenor or exteπor filtenng router, elimmatmg a dedicated network of a bastion host, elimmatmg encryption bits and decryption, and eliminating user-specified passwords would prove beneficial if the same level (or possibly a higher level) of secuπty could be maintained An improved level of secuπty which avoids conventional firewalls is preferablv one based on an analysis of each packet as the packets are switched throughout the network Analysis on each packet could desirably be performed without requiring a circuit-switched path or dedicated, pπvate path such as those found m pπvate lmes and or Virtual Pπvate L es ("VPLs") attπbutable to a Virtual Pπvate Network ("VPN") VPNs and circuit-switched networks impose significant limitations on bandwidth utilization and network throughput and therefore should be avoided as a secuπty solution

Modem day networks are expected not only to be highly secure, but also to support numerous types of applications, some of which may require greater bandwidth than others Other applications may require deterministic response time across the network Network managers mav be required to adapt the network, or at least portions of the network, so that it can guarantee a predetermined amount of bandwidth and/ or propagation time for a requested Quality Of Service ("QOS") The term "QOS" generally subsumes all the vaπous terms used to descnbe "accesses" between a pair of termination devices Access is therefore a term that can be quantified to include the speed and degree of secuπty by which packets travel between termination devices and the fault tolerance measures which ensure the transmitted signal is clear and accurate

The network manager may require, for example, fast, highly secure communication between termination devices residing on a local intranet, whereby the termination devices are sending both voice and video in a time-sensitive manner There may be other intranets, however, that are not sendmg time- sensitive information, each of which may be connected to the intranet allocated to higher speed The network manager may therefore need to give pπoπty to packets sent (or received) to and from some, but not all, termination devices within the network In other words, a need exists for allocating bandwidth through a shared resource such as a network link, switch or router, or arbitrating among competing devices that are requestmg vaπous types of accesses to the shared resource

Conventional QOS, however, cannot provide an accurate determination of a delay (I e . response time) between a request sent by a termination device and a response returned to that device For instance, the transmission delay vanes depending on whether the routmg table portion of mterest resides in the control processor cache or is within the system memory linked to the control processor by a system memory bus Time needed to access the routmg tables will therefore vary across the network leaving uncertainty as to when the requested packets will be returned to the requesting agent Thus, conventional QOS solutions cannot guarantee response times even though a specific class of service has been designated Instead, QOS is limited to determining availability and reliability of the transmission path not, e g , worst case response time

It would be of benefit to be able to specify a QOS and to designate classes of service for certain accesses within the network based on the time-sensitive nature by which the applications must operate The benefit would be deπved in designating certain nodes accordmg to a structure and to guarantee worst case response time between those nodes based on the classes of service and reserved bandwidth unique to those nodes The desired QOS would benefit bv implementing packet-based classes oi services The classes of service might beneficially extend to secuπty codes attributed to those classes so that not only could performance be assured, but also secuπty

The size of the data segment bemg wrapped is often governed by the minimum size of the physical layer protocol. For example, Ethernet specification may require that a minimum data segment size is 64 bytes. The overall size of the packet is governed based on the number of layers specified and the particular PPP used. Generally speakmg, each segment of data being wrapped includes a rather significant header and trailer portion of the overall packet. As the header and trailer portions mcrease relative to the data segment, the overhead increases and transmission efficiency decreases.

Contained within the header is a seπes of bits which mdicate the start of a packet. The start-of-packet sequence is generally fixed with specified bit values that, whenever detected, signify all bits which follow pertaining to a unique packet. Also contained within the header are addressing bits that tell the packet where, within the network, the packet is destined. For example, a termination device may have a unique identification number that is addressed by the destination address within the header.

The trailer may contam a series of bits which mdicate the end of a packet. The end-of-packet designator, similar to the start-of-packet designator, contains a unique seπes of bit values that are fixed and identify the end of a packet when encountered. Also included within the trailer might be eπor detection and correction bits, as well as possible security codes and/or pnoπty codes. The secuπty codes provide a secure transfer while pnoπty codes label each packet transmission priority based on us coπespondmg code.

It is often difficult to discern when a packet begins and when it ends. Thus, the start-of-packet and end-of-packet designators contam a fixed and recognizable bit field. However, the data segment is a variable bit field and, in some mstances, might have bytes or words contammg the same bit values as the start-of-packet or end-of-packet designators. In these circumstances, one or more bytes within the data segment may incorrectly signify a start-of-packet or end-of-packet condition. Therefore, mstead of sendmg the data segment as an entire segment, one or more words within that segment might mdicate that the earlier portion of the segment belongs to one packet and the latter portion of that segment belongs to another packet should, for example, those words have the same bit values as start-of-packet or end-of-packet designator

In an attempt to resolve the problem of usmg vaπable data bits hich might align with the same bit values as fixed, packet designators, methodologies have been implemented for appending fixed bit patterns . These bit patterns can be added to each byte of data within the data segment A popular form of byte encoding mcludes the 8b 10b solution use of 8b 10b involves addmg two additional bits to each byte of data, whereby each byte is replaced with predefined ten bits. Addition of two bits for every byte allows for redundancy to have unique start-of-packet or end-of-packet designator and other control information In this manner, the data can be assured as distinguishable over bit patterns within the header or trailer.

Usmg the 8b 10b, or 4b5b, solution helps ensure the integnty of the data segment is not jeopardized relative to its neighboring header and trailer. However, addmg two bits to every byte within the data segment significantly increases the data transfer overhead. In the 8b 10b example, the addition of two bits would decrease data transfer efficiency by 20%. It would be desirable to implement a mechanism or methodology for preventing mis-identification of data as header or trailer control bits However, an appropπate solution must be one that does not add significant overhead to the packet. Identifying data separate from the header or trailer not only would be beneficial if done with minimal overhead, but also could have additional advantages if performed within the purview of transmission error detection, and/or coπecπon Once the packet has been properly wrapped, it will m most mstances be forwarded unhindered across the network. Unfortunately, however, transmission disruptions can often occur. Such might be the case where a transmission path of the network is severed, or otherwise temporanly or permanently disabled If the disrupted path is m a cntical segment of the network, it may be impossible to reroute a packet between termination devices while avoidmg that path. Thus, even with networks havmg more than several alternative routes between a pair of termination devices, a disrupted cntical route would disable the entire network transmission.

In an effort to obviate temporary or permanent network downtime resultmg from a disturbed (i.e.. corrupted, disabled, or severed) transmission path, many architectures use redundancy. For example, in a ring topology, transmission from one station to another station connected to the πng proceeds in a smgle direction around the πng. If the transmission channel is disturbed, transmissions between those stations can no longer take place if. indeed, those stations are separated by the disruption. Usmg a second, or "redundant," channel, redundant packets can be sent m both directions on both channels around the nng. Thus, if the left transmission channel is cut, the right transmission channel is able to maintain transmission in the πght direction around the πng. In an implementation of dual ring topology, data packets can be sent in both directions simultaneously. One direction can be designated as the primary transmission direction Normally, packets will be received from this path. Packets received from the other path will be dropped at the receiver end. However, if there is a disruption in the primary direction, the receiver will receive the packet via the alternate direction. An alternative example may be a star or mesh topology, whereby transmission can proceed from station A to station B either directly or, if the channel separatmg stations A and B is disturbed, then transmission can proceed through a redundant channel such as from station A to station C and station C to station B.

As used herem below, a transmission path includes two or more alternative transmission channels, where one channel is the primary channel and the other channels can be considered redundant channels. Each channel is deemed either a copper conductor, fiber or wireless transmission medium. In a nng topology, the primary and redundant channels can be formed in a smgle transmission path around the loop. In a mesh topology, transmission channels can exists between each of n-1 number of stations, where n equals the number of stations within the star In a mesh topology, transmission channels exist between each pair of n stations. In the star or mesh topology, the redundant transmission channel can be the alternative pathway between two stations via other stations.

In physical fiber optic implementation, a smgle fiber will consist of multiple strands and each strand consists of multiple Wave Division Multiplex (WDM) channels. A physical cut (or severance) of the fiber will generally result in disruption of services for all channels associated with that fiber Frequently, this will result in disruption of service in both primary and redundant channels at that pomt. In a nng topology, there is a significant likelihood that both the primary and redundant channels will be disrupted, rather than a smgle channel smce both channels are typically located next to one another around the nng. For example, both the primary and redundant channels can be clad m a umtary structure, and whenever one is cut it is most likely that the other channel will also be cut thereby rendering useless the entire redundancy feature. Therefore, even though Synchronous Optical Networks ("SONET") employ redundant optical fibers, a disruption of both the primary and redundant fiber can, at the very least, reduce the overall transmission bandwidth or, in the extreme, prevent transmission altogether Usmg the former mstance and the aforesaid example, if both the left and πght fibers are cut between stations A and B, station A can still send mformation across, for example, the πght fiber to station B, and can send packets across the left fiber to an mtermediate station upstream of the cut. Unfortunately, packets sent m the left and nght fibers are redundant of each other So, even though the overall bandwidth capacity is 5 Gbits the maximum bandwidth achievable usmg the SONET redundant system is 2.5 Gbits This is due to one set of packets being sent on the left channel and the very same (redundant) set of packets bemg sent on the πght channel It would be desirable to introduce a redundant system that does not send the same (l e , redundant) packets across both left and πght channels in a πng topology Instead, the desired system would use two channels to be routed around the nng or between each mesh- or between most star-configured stations, but allow each channel to be utilized for data transmission This would maximize the bandwidth capacity of the network or communication system If each channel is to be utilized, then the desired system should be one which still permits non-redundant transmission to all stations, regardless of where w lthin the network one or both channels are disturbed (I e , severed, disrupted or otherwise disabled) The improved system, method, architecture, and packet protocol must desirablv not only maintain packet transmission speed, but must also note where within the network the severed transmission path exists so as to re-route future packets to avoid the disturbed channel

Summary of the Invention

The problems outlined above are in large part solved by a structured nerwork. an enablmg distπbuted routmg mechanism, an addressing mechanism used within that network, and forwarding/directing devices ("modules") topologically related to one another based on their position within the network The modules, due to an awareness of their position or location with respect to the structured network, allow adaptive fast forwarding of packets across the network The modules represent an improvement over conventional switches and/or routers Instead of statically routing packets in the same manner each time, as m conventional switches, the modules mclude some features of conventional routers, yet without the detπments of routers The modules can forward and/or direct packets of data relatively fast (similar to conventional switches), and can dynamically change the forwarding path based on activity within the network For sake of brevity m its descπption, the modules perform a hybπd of a switch route function and are henceforth mterchangeably refeπed to as switches or modules based on the speed at which thev forward packets of data, and that they perform a selection function applicable to switches, yet are not conventional switches and, therefore, are designated as "switches" or "modules," in general However, it is recognized that the present switches are altogether dissimilar from conventional switches, and are also dissimilar from conventional routers The network can be made up of numerous switches (or modules) extending between termination devices Each switch or module performs fast deterministic switchmg of packets across the network based on conditions of the network The network can be placed within only a portion of a conventional network, a subnet of which is coupled to one or more termination devices Regardless of where it is configured, the present network mcludes one or more topological levels of switches that are structurally related to one another Each sw tch within the network is assigned a unique identification number possiblv coπespondmg to the hierarchical level of the network topography that can be ascertained within a given field attributed to that level For example, the most significant field of bits mav be allocated to the highest hierarchical level of switches, followed by the next most significant field allocated to the next highest level of switches With each level of hierarchy, structure is provided Structure is also provided within each level The switches are organized accordmg to some predetermined structure within a given hierarchical level which is reflected I the field of the identification number coπespondmg to that level Distπbuted routmg can therefore be achieved by comparing a destination address of the wrapped data packet with identification numbers of those hierarchical levels depending on the location of the switch receivmg the mcommg packet Switches direct forward data packets by determining the direction of flow of the mcommg packet based on comparing (or decodmg) the destination address of the packet with the relative position of the switch mdicated by the switch's identification number. As the result of this direction/forwarding operation, traditional routing methods are eliminated. Also, decoding operations eliminate routmg m what was determined, at a higher hierarchical level, to be an unused path. Decodmg withm lower levels can be restπcted only to those switches that the higher levels pomt toward. Moreover, decodmg can be accomplished in a fairly rapid manner and need not occur if a companson at the higher level will direct the packet to another branch of lower level switches. In this manner, each switch is assigned an identification number based on its relative position withm the network Accordmgly, the structured network can be thought of as havmg a relative or deterministic routing topography absent the detπments of conventional routers Hence the deterministic routing acts as a distπbuted router. Instead of performing routing functions at every node in the network, the distnbuted router performs incremental routing functions at every node and the data forwarding function is achieved across the entire network Withm the structured network is at least one switch or module and, preferably, numerous switches or modules interconnected to form an intranet, an internet, or a combmation of both The switches or modules can be classified as an end switch or module, a pass-through switch or module, or an mtermediate switch or module. The end modules are those that are configured adjacent termination devices and are used to perform protocol wrapping functions as packets of data enter the structured network, or stπp protocol used by the structured network when packets exit the network The end modules also perform a routing function similar to mtermediate modules or pass-through modules (mterchangeably refeπed to as switches). The end switches or modules, mtermediate switches or modules, and pass-through switches or modules can be configured in hardware to be substantially the same, with only the programmed function bemg different based on where withm the network the modules are placed. Each module mcludes a minimum of one bi-directional port and a traffic manager which controls the port, or ports. The number of ports is scalable in that there may be one or more mput ports and one or more output ports aπanged m vanous configurations with the traffic manager decidmg which packet received on an mput port will be forwarded to which of the vaπous output ports. The traffic manager mcludes a decoder and vaπous buffers. The decoder performs a companson between the destination address and the identification number of that module based on the position of that module withm the overall structured network The companson and decision are based on the companson represented as an incremental step m the distπbuted routmg function (data forwarding function) of the network.

A shared or distπbuted buffer may be placed withm the data flow path to receive a packet and to forward that packet to the appropnate output port based on the destmation address Instead of forwarding the data mto a buffer, the data can be selectively forwarded across one or more bypass conductors coupled in parallel across the buffer Thus, a module which forms an mtermediate module or end module can be configured with bypass operations applicable to a pass-through module or a pass-through node, and the bypass paths are used to reduce latency in the modules. The buffer can also be contained with an end module and configured with control mformation Through a seπes of read or fetch operations, the buffer may be used to wrap an mcommg packet and associated address with control mformation as well as an oπgmatmg address of the end module. The fetch operations occur m rapid succession from a traffic controller of the traffic manager The structured network is one that is completely compatible with protocols cuπently used m, for example, the OSI model Specifically, a conventional address resolution protocol can be used to determine the location of a destination termination device by comparing an address (e g JP address) broadcast over the structured network Once the destmation termmation device is found, the end module withm the structured network that is linked to the termmation device can also be found The end module nearest the destmation termination device can be thought of as an exit end module whose identification number is known When packets of mformation are to be sent to the destination termmation device, it is known that those packets will be targeted to the exit end module and forwardmg operations are achieved through the hierarchical decode operation usmg a senes of compare operations withm only select modules of certain hierarchical levels. The entry end module adjacent the entry termination device will wrap the physical media packet with an onginaαng identification number of the entry end module The packet can then be forwarded throughout the structured network without havmg to perform any look-up operations

When the packet arrives upon the exit end module, control mformation of the structured network is stnpped from the packet and a relatively small mappmg table may be used to forward the packet to a relatively small number of destination termination devices selectively linked to the exit end module If. however, only one termination device is linked to the exit end module, then mappmg is not needed and consequently, no translation need occur The companson or translation performed at the exit end module occurs to bndge the gap between the structured network and any external protocol or domain, hence the translation is a full companson. For example, a full 32-bit random IP address can be used to look-up a 48-bit Ethernet address of the termmation device Across the structured network, instead of havmg to make a comparison of an mcommg address agamst all module identification numbers, the deterministic network hereof need only compare a field of bits withm a certain level of the network dependmg on where the entry end module is located withm the hierarchical level Further comparisons are performed only on a field-by- field basis, each time lessening the number of fields need g companson In instances where the structured network is quite small, companson may be required only on a single field to determine, for example, whether to forward the packet left or πght around a smgle loop. Only a fraction of the overall number of module identification numbers are compared agamst the destmation address and. more importantly, no look-ups are needed m the forwarding operations In particular, a self-compare operation occurs withm each module's decoder along the forwardmg path A result of the self- compare is a module withm one level pointing to another module m another level based solely on the decode operation Only a small mappmg operation may be needed by the exit module when identifying which of the vanous destmation termmation devices connected to the exit end module will receive data. The number of termmation devices connected to the exit module is significantly smaller than the sum total of modules or nodes withm a network Importantly, the extent of the translation table associated with the exit end module is local to that module's location withm the network node and to the domain of the termination devices known to be connected to it The mappmg table is completely mdependent of the intenor structured network and is based solely on a known relationship between an exit end module and termmation devices connected thereto. This allows the mapping table to be static for the most part Any changes to this table is completely localized and does not require updating of other nodes and/or modules withm the structured network

The addressing mechanism thereby targets the exit end module after performing a conventional address resolution protocol When forwardmg data back from the exit end module via the termmation device connected thereto, the forwarded data can be directed to the entry end module via the entry end module identification number placed on the packet received by the destination termmation device Accordmgly, m a reply to the onginating packet, no look-ups are needed. Only a relatively small mappmg operation may be needed when forwardmg the reply outside of the structured network (i.e., when forwardmg the reply packet back to the onginating termination device) Thus, if more than one termmation device is linked to the end modules, only one mappmg occurs when routmg packets of data from the structured network, either to the destination termmation device or back to the ongination termmation device All routmg functions are achieved by a field- by-field companson, much like a distπbuted routing operation.

The entry and exit end module identification numbers may be configured or re-configured such that they are recognizable to the termination devices connected thereto. For example, if the network is to receive Ethernet-based packets, then the entry and exit end modules are configured with identification numbers that are withm the Ethernet address range. Since the network is structured, with successive nodes and or modules bemg recognizable to each other, all other modules m the network are in the same addressing domain. This allows for replacement of the termmation device address with the identification number of the entry/exit end module proximate to that device. Importantly, the termination device which forwards the destination address can now recognize that the address (e.g., Ethernet address) of the termination device at the opposmg end of the network is at the Ethernet address of the end switch connected to that device.

At the entry end module, the mcommg termination device address (e.g., Ethernet address) can be replaced with the Ethernet address of the entry end module (i.e., the entry end module identification number) m the data packet. The targeted or destmation termmation device recognizes that the data oπgmatmg from the mcommg termmation device is that of the entry end module and now knows to forward all replies to the identification number, or address, of that module When the reply reaches the onginating entry end module (i.e., exit end module for the reply), the next highest level of protocol (e.g., IP address) address is used to identify 1 of possibly N termmation devices attached to the entry end module. This is a fairly simply reply translation, and the only look-up needed during the reply operation. The relatively simple look-up, like the request look-up preferably mvolves a small, static look-up table local only to the end modules. The only entπes withm the table are the next protocol level (e.g., IP withm the protocol stack) relative to the physical protocol level (e.g., Ethernet) address mappmg.

A traditional hop used by conventional routers may involve replacement of a lower level (e.g., Ethernet) source and destmation address of the packet with the next destmation Ethernet address. The next destination Ethernet address (or hop address) is determmed by the routmg table withm the convention router, said table is dynamically managed by the network. The present modules replace the source address at the entry and destination address at the exit end module, but not for purposes of "hop" routmg. Instead, the present modules merely allows for more than one termmation device per entry-exit module. If only one termmation device is coupled to an end module, then address translation, look-up or hop is not needed. Simply a 1-to-N or N-to-1 mappmg occurs. In a conventional sense, a hop is not used. Instead, the exit end module simply performs a mappmg of 1 -to-N when exitmg the structured network, provided of course that N termination devices are present. If only one termmation device is present, or if routmg functions are present in multiple termmation devices, then the end modules need not perform any mappmg whatsoever.

The traffic manager withm a paπicular module can be combmed with other traffic managers of other modules to form a modular aπangement havmg any of numerous mput and output port configurations.

Additionally, each mput port of the modular-connected traffic managers can receive data at differing speeds and with diffeπng formats or protocols The modular connection of traffic managers can form an overall module that is compatible with dissimilar protocols and transmission speeds for use in any desired network connection or multiple network connections. For example, the input port attπbuted with a first traffic manager can receive 1.0 gigabit Ethernet information, whereas another mput port attπbuted with another traffic manager can receive 5.0 gigabit fiber optic mformation. Thus, each traffic manager mcludes buffers which not only receive and temporanly store packets, but also are used to mterface with differing transmission speeds of packets forwarded to the mput port and packets exiting from the output port.

Withm a modularized, interconnected set of traffic managers and associated ports, is a local bus which couples a control processor to each of the scaled traffic managers. A smgle control processor can therefore be used to receive diagnostic, monitonng, and control mformation necessary to perform those functions upon one or more of the mter-linked traffic managers. The diagnostic, monitonng, and control information can be sent across an optical fiber, a copper conductor, or a wireless medium to the control processor which will then forward the appropriate control signals across the local bus. The diagnostic, monitormg and control ("DCM") signals may be transmitted to the control processors and traffic managers usmg the same network, resulting m an m-band signally or more optimally, the processors and traffic managers can use the DMC signal for out-of- band signaling.

For example, the traffic managers can be programmed or configured when they are installed withm the network and reconfigured from a centralized location at a later date via a transceiver and the control processor. Reconfiguration performed in conjunction with self-testing operations can determine histoπcal traffic congestion areas of the nerwork to allow the network to adapt and reroute packets away from the congested areas. Reroutmg can occur either by configuring the traffic managers m those suspected areas or broadcasting a reconfiguration signal from the centralized locations. Importantly, however, the DMC signals sent to the control processor, and the control processor and local bus mterface can remam entirely outside the data flow path. Data flow does not depend on, and therefore is not slowed by, access to and from the control processor. Once the control processor has performed its function, further access thereto is not needed

Both the secunty code and the pπoπty code are placed withm the header of each individual packet so as to assign a QOS on a packet-by-packet basis. The added flexibility of assigning QOS to each packet not only enhances the dynamics of resource allocation over a relatively short peπod of time, but also establishes a packet-switched security network The network is a structured network that is se p and controlled mdependent of the data flow path. By separatmg the data flow path from control and semp mcludmg QOS control, it can be assured that secuπty features (packet sniffing and packet spoofing) are substantially eliminated at each module smce accesses via the network data flow path to control mformation cannot be achieved

Each module can be semp to send or accept packets oπgmatmg from or addressed to specific modules of the structured network Thus, each module can be assigned a security code that is placed on the packet traversing that module, and the downstream module will either grant access or deny access based on secuπty code placed upstream. This allows for pnvate lme type secuπty services with guaranteed pnvate access, but withm a packet-switched environment without havmg to dedicate a circuit or path m times when the circuit or path is not bemg used The modules can be programmed to only pass certain classes of messages based again on the secuπty class of the oπgmatmg, or upstream, module as designated by the identification number of that module and or the secunty code assigned to that module. Along with the header, a trailer may be added to the packet to note if a secunty breach has occurred somewhere withm the secured network, and the location of that breach -- either packet sniffing or packet spoofing at a particular module havmg a unique identification number or secuπty code

In some instances, unique secunty code can be assigned to each module In other instances, there could be a unique identification number for each module and several secunty codes assigned to groupmgs of modules based on the secured status or class of those grouped modules The same applies to the pπoπty code also. The exit end module may contam one or more mappmg tables selectable by the secuπty code transfeπed with the packet that arπves on the exit end module The selected mappmg table contains a grouping of identification numbers attπbuted to the upstream module or the entry end module If the identification number forwarded with the packet matches with an identification number in the selected table, then it is known that the termmation device connected to the exit end module is authorized to receive the packet amving upon the exit end module

The entry end module and the exit end module provide secunty identification and secunty authorization Optionally, the trailer placed upon the packet can include the number of modules traversed and the identification number of each module If the count number of modules traversed does not equal the identification number entries, then a secuntv breach is detected and. based on the user s configuration data, isolation of the breached identification number can be determmed From that identification number, an access violation can be pmpomted to a specified module and therefore a termmation device connected to that module

The entry or mtermediate module may also assign a pπoπty code based on the mcommg identification number of either a termmation device, a user from a termmation device, or an upstream module and the bandwidth allocated to that upstream module or termination device For a specified bandwidth allocation, pπoπty is given depending on whether the mcommg packets of data exceed or are less than the allocated bandwidth The entry or intermediate module will select mcommg packets from dissimilar nodes upon the shared resource (interconnect bus or link) depending on the pπoπty code of the mcommg packet. If one packet pπoπty code is higher than the other, than the module will operate as an arbiter and select the higher pπoπty packet based on one of numerous types of arbitration schemes

The structured network is one which can send a packet aπanged m a format usable withm any communication system employing panty generators and paπtv detectors The improved packet is one that uses at least one panty bit assigned to each word withm the packet Thus, a panty bit is generated for each word withm the header, trailer, and data portions of the packet The panty generator generates either even panty or odd panty, dependmg on whether the correspondmg word is withm the data portion, header portion, or trailer portion Accordmg to one embodiment, the generated panty bit produces even panty when combmed with words withm the header or trailer, and produces odd panty when combmed with words withm the data. Accordmg to another embodiment, the generated panty bit produces odd parity when combmed with words withm the header or trailer, and produces even panty when combmed with words in the data Regardless of the panty chosen, the mtent is that panty switches (from even panty to odd panty or vice-versa) between the header and data, and also between the data and the trailer

As defined herem, "even panty" means that when combining the value of the panty bit with values of the coπespondmg word, the number of 1 's is even Thus, a panty detector is any logic which can receive words withm the packet and coπespondmg panty bits and adds the number of 1 's contained in the word and coπespondmg panty bit to determme if that number is even or odd Similar to even panty, "odd panty" simply determines if the number of 1 's contained withm the word and coπespondmg pantv bit is odd A panty generator is one which generates a paπtv bit so that, when transmitted, either even panty or odd panty is targeted However, after the word and coπespondmg panty bit are transfeπed. it is desirable that the word and coπespondmg panty bit be re-checked to see if the generated (1 e , "targeted") even panty condition remains after transmission In this manner, the panty bit serves two functions First, the generated panty bit, when combmed with the coπespondmg word, produces a panty condition or value which is different for the header (or trailer) than the data Thus, the data can be clearly identified as bemg different from anv designator or control bit pattern withm either the header or trailer Second, the panty bit, when received after transmission, can be compared with the coπespondmg word to determme if the even panty value generated for that word remains as even panty when detected after transmission If the even panty word somehow is modified to odd panty, then it is known that transmission eπor occuπed Conditions which might cause transmission eπor clude, for example, noise imputed into the transmission channel of the communication network or system at select times which corrupt some words but not others

Implementing at least one parity bit to an entire word of the packet, instead of a byte-to-byte basis, will significantly reduce the added overhead and thereby increase transfer efficiencv For example, if the word length is 4 bytes, and a mmimum size data segment is 64 bytes, then 64/4 or 16 bit panty bits are added to the data segment If an 8b 1 Ob scheme is used, a 64 byte Ethernet packet would require 2 bits added for each byte totaling 128 added bits Employing 16 bits rather than 128 substantially reduces the overhead needed to transfer an Ethernet data segment, however, as the packet size increases , the relative decrease m overhead will become even more significant The unproved packet format contemplates use of at least one parity bit on each word of mformation sent with a packet A word is defined as a plurality of bits which are greater in number than eight bits, or an eight-bit byte Preferably, a word is the number of bits which can be transfeπed during a smgle cycle or operation Accordmg to one example, the word length mvolves 32 bits, or 4 bytes The packet format is designed to be sent across at least a portion of a communication network The communication network mcludes an optical waveguide, copper wire, or other transmission media The improved packet format is any packet of information which can be used in the OSI model and mcludes, for example, IP-wrapped datagrams or any wrapped data segment which can be sent over a physical medium usmg various protocols mcludmg IP, ATM, IPX, DEC, NET, Apple Talk, etc The packet generated prior to transmission across at least a portion of the network involves at least one parity bit assigned to each word of the data, header, and trailer portions of the packet. The panty bit, when added to a coπespondmg word of the header or trailer, produces a value that is dissimilar from the panty bit added to the coπespondmg word of the data when different panty type (odd or even) is used for data and header/trailer Accordmgly, the data pattern resultmg from this appendmg of dissimilar panty type is different for header/ trailer and the data Thus, same word (or data pattern) in header or trailer will appear different from that in data The improved packet protocol is one which can detect if an eπor anses from a disturbance withm a specific portion of a communication channel Once the disturbance has been determmed, then stations adjacent to that disturbance will be noted as the receiving module immediately upstream of the disturbance As defined herem, a module is a geneπc term which descπbes a node which can be represented as a station employmg a switch or router, and can forward packets across different physical media The modules are organized according to some predetermined structure withm a given hierarchical level that is reflected in the field of the identification number coπespondmg to that lev el Distπbuted routmg can therefore be achieved by comparing a destination address of a wrapped packet with identification numbers of those erarchical levels dependmg on the location of the module receivmg the incoming packet Modules therefore determine the directional flow of the mcommg packet based on comparing (or decodmg) the destmation address of the packet with the relative position of the module indicated by the module's identification number In nng and mesh topologies, a transmission path between any two modules preferably contains at least two transmission channels A smgle transmission path extends around a nng topology, and one or more paths extend between modules of a star or mesh topology If the entire transmission path is disturbed between a pair of modules coupled in the nng topology, communication to either of those modules can be maintained provided transmission is sent m the proper direction around the πng The location at which the transmission path is disturbed can be arπved at by the last module adjacent the disturbance not detecting a null bit pattern for a peπod of time exceedmg that need to transfer a packet As noted herein, disturbances cause eπors m transmission across the network There are vaπous types of disturbances For example, disturbances may mclude not onlv severance of a channel, but also practices such as a network system administrator setting a node inactive or other eπor detection and reporting methodologies Responsive to eπor/disturbance detection, the receivmg module will return the packet through a loop-back conductor of that module to the onginating module The returned packet preferably mcludes control bits and eπor identification bits m the header and trailer portions of the packet The control bits mdicate that an eπor exists (I e , that the transmission path is disturbed), and thereby informs the oπgmating module to either re-send the packet m an alternative channel m the opposite direction about the loop or, upon unsuccessful retry, drop the packet altogether The ongmatmg module, when receivmg the returned packet containing an eπor value of the control bits, can then forward back an acknowledge bit pattern so as to prevent future "thrashing" that might anse if the ongmatmg device was not made aware of and acknowledged the returned eπor value In addition, an oπgmating module as well as other modules are made aware of the disturbance and the location of the disturbance for future transmission Absent an acknowledgment, the oπgmatmg module would attempt to contmuously re-send the packet across the alternative channel or path deletenously tymg up the network By setting the acknowledge bits to acknowledge a returned eπor, the ongmatmg module can then re-send the packets across one or both channels, but in the opposite direction around the nng, or across another channel across the star or mesh This allows the network to achieve its full bandwidth transmission capability by maintaining the bi-directional status of each channel and avoiding redundantly sendmg the same packets across a dedicated left-directed channel and a dedicate nght- directed channel, as m a conventional πng network

According to one embodiment, a communication network is provided The communication network is a structured nerwork with distπbuted routing function compnsmg a plurality of interconnected modules To enable distnbuted routmg, each module mcludes an identification number bifurcated mto several groupmgs of bits, or fields A first groupmg of bits helps identify a first module for recei \ mg data of the network pπor to a second groupmg of bits identify mg a second module for receivmg data from the first module The first module can be considered as bemg withm a first hierarchical level and the second module can be considered within a second hierarchical level The first and second hierarchical levels are not necessaπly the highest level and second to highest level, but are simply two levels withm the overall hierarchy A decoder withm the first module can compare an address of the forwarded data or packet with the first groupmg of bits if the address oπgmates irom a module withm a hierarchical level dissimilar from the first hierarchical level Thus, the ongmatmg address is either m a hierarchical level above or below the first hierarchical level The same can be said regardmg a decoder withm the second module and its companson function.

The first module, or first set of modules withm the first level, are connected upon or between a first set of loops that are associated with a first set of network nodes. Likewise, the second module, or second set of modules withm the second level, are connected upon or between a second set of loops that are associated with a second set of network nodes. Accordmg to one example, at least one network branch may be terminated at both ends by a module withm the first set of modules and a module withm the second set of modules to enable couplmg of the first set of network nodes to the second set of network nodes to provide a selectable routing path. The network nodes and nerwork branches can be thought of as extendmg at least partially between terminating devices to form a data flow path therebetween.

The structured network can be modularized. The modular network mcludes a set of modules interconnected across at least a portion of the network. A smgle control processor is also provided havmg a local bus extendmg between the control processor and each of the set of modules The local bus is adapted to receive from the control processor diagnostic, monitormg or control signals. The control signals may mclude signals used to, e.g., reconfigure an identification number for at least one of the set of modules to change routing of data or packets through those set of modules. The control processor can receive wireless or wired (optical or solid conductor) communication from a transceiver placed remote from the control processor.

The structured network may mclude more than three nodes and three branches. Only one module within one node may be used to perform a single 1-to-N address mappmg as the data traverses the entire routing network and is forwarded to the selected one of N termination devices with no routmg look-ups m the mterim. The mappmg can be performed by a selection device, such as for example a multiplexer or router, and the mappmg is used to select only between multiple termination devices, and not between ensumg nodes and/or modules withm the network.

Yet further, a method is contemplated for routmg data. The method mcludes compiling a packet compnsing a destmation address of the exit end switch and a source address of the entry termmation device, and thereafter forwardmg the packet from the entry termmation device to the entry end switch. While maintaining the destmation address, updatmg the source address to the address of the entry end switch while forwardmg the packet from the entry end switch to the exit end switch, exclusive of look-up operations m the interim. While mamtammg the source address, updatmg the destination address to the address of the exit termmation device while routing the packet from the exit end switch to the exit termmation device.

Before compiling the packet, an address resolution protocol may be broadcast from the entry termmation device to the entry end module, and thereafter to the exit end module for finding the location of the exit end module relative to the exit termmation device. Withm each module is a traffic manager. The traffic manager may mclude a traffic controller which dispatches a seπes of read operations to memory coupled withm a data flow path between the mput and output ports of the module. Thus, the memory may mclude control information placed therein durmg a configuration cycle. The memory may also temporanly store packets of data dispatched from the mput port. The traffic manager reads the control mformation and the packets of data m a predefined sequence so as to compile a wrapped packet that can then be forwarded to the output port.

Both pπonty codes and secuπty codes can be assigned to modules according to vanous hierarchical levels of the network, each level bemg assigned based on where the nodes or modules are arranged structurally relative to one another withm the network For example, the highest pπonty level may be attnbuted to a smgle loop of interconnected modules needing significant bandwidth resources for use in videoconferencing applications, whereas a lower pπoπry may be assigned to packets destined to one or more modules of another loop where voice transmission is all that is needed, both loops possibly connected to form a multi-loop subnet. The method for wrapping a packet of data may therefore mclude temporanly storing a packet of data m memory located withm the flow path of the packet. A set of read operations can be dispatched to memory for reading vaπous addresses (destmation and oπgmation addresses), control information, and the temporanly stored packets, in order to form at least a portion of a wrapped data packet.

Accordmg to one embodiment, a communication network is provided havmg a plurality of forwardmg modules. The modules can be classified as entry modules, mtermediate modules, or end modules. The entry module may further mclude a buffer contammg a secuπty code and a traffic controller coupled to the buffer. The traffic controller may read the secuπty code from the buffer and place the secuπty code and identification number withm a packet compπsmg the forwarded data. A counter may be used to count the number of accesses upon the entry module by the entry device and block further accesses if a count value exceeds a value stored withm the storage device. The entry module may further mclude a mappmg table coupled to map the mcommg identification number to a coπespondmg secuπty code. The mcommg data is aπanged withm a smgle packet and the secured data is transfeπed to the communication network on a packet-by-packet basis.

An exit module of the communication network may mclude an output poπ adapted to receive a packet of data havmg a header. The header mcludes a secunty code and an identification number. A storage device is contained withm the exit module and compnses a plurality of tables, one of which may be selected by the secuπty code of the packet header The exit module may also include a compare unit coupled between the output port and the storage device for comparing the identification number with entnes withm the selected table to determme if the identification number deπves from a secured path that extends at least partially through the communication network. As such, the tables are used to authorize transmission of a secured packet through a termmation device coupled to the exit module. The table may also be used to assure that the identification number of the entry module, or an intermediate module, has assigned to it a secuπty code of a proper hierarchical level The packet is either secured or not secured and. if secured, the exit module confirms secuπty before sendmg the secured packet to a termmation device that can read the packet

The secured path thereby extends from a secured module withm the communication network. The secured module is one having been assigned the secunty code and the identification number and placed the code withm a header of a packet received by the exit module output port. A decoder withm the exit module may be used to decode a groupmg of bits withm the secuπty code to select one of a plurality of tables. Accordmg to one example, the groupmg of bits may be selected as the most significant bits withm the secuπty code.

A decoder withm mtermediate forwardmg modules forces a secure packet to be forwarded only when there is a secunty code mismatch of the mcommg packet with its security tap enable These forces secure broadcast packets to be visible to only other modules withm the network that are authorized to receive this particular secure broadcast transmission Similarly, if a secure packet is destined to a non-secure module, the packet will be rejected by the exit node decoder to prevent a secuπty breach.

Each identification number is preferably a unique number and the secuπty codes are assigned accordmg to their secunty level or class of service. That is, a groupmg of modules that are related to one another may have the same secunty code, but dissimilar from a secunty code of another groupmg placed withm the same network, or which control routmg to dissimilar locations. The entry module may be coupled to assign and transfer a secuπty code and an identification number to a packet of data. An exit compare unit within the exit module is coupled to receive the packet of data and compare the secuπty code and the identification number before transferring the packet of data from the communication network. Each of the intermediate modules may also include a storage device containing a security code and a compare unit coupled to receive an incoming packet and compare the incoming security code against the security code of the intermediate module to define a secured path through the communication network.

The entry modules, intermediate modules, and exit modules are generically refeπed to as "forwarding modules." Each forwarding module includes an mput port coupled to receive an incoming packet of data. A forwarding module can also include a mapping table which assigns a priority code to the packet depending on the address from which the packet of data was sent. The address is defined as the identification node of the upstream termination device or module. According to one example, the priority code is assigned depending on an amount of bandwidth allocated to a section of the data flow path to which the packet of data is forwarded from the module. The amount of bandwidth may include an allocated transmission speed of that section of data flow path. The prioπty code may be maintained for a plurality of successive packets provided the number of bits forwarded per second by the plurality of successive packets is less than the allocated transmission speed. The priority and /or secunty code may also be maintained for a plurality of forwarding nodes resulting in segment based priority and/or security transmission.

Thus, in addition to providing packet-by-packet security codes and identification numbers, the communication network also provides packet-by-packet priority codes. Similar to security, priority can be assigned based on vaπous factors, such as the overall structure of the network, levels of structure, inteπelationships of various structurally related nodes and/or modules, the destination address, the source address, or both, so as to achieve data link type functionality. Data link type functionality allows us to have high priority data transmission paths across multiple levels of hierarchy without requiring the particular address having high priority across all of these levels of hierarchy. The priority codes are used to provide a guaranteed response time through certain paths of the network, regardless of whether those paths are secured or not. The communication network can therefore be thought of as including a first forwarding module coupled to receive a first packet of data and assign a first prionty code to the first packet of data depending on an address from which the first packet of data was sent. The network may therefore include a second forwarding module coupled to receive a second packet of data and assign a second priority code to the second packet of data depending on an address from which the second packet of data was sent. An arbiter is coupled to forward the first packet of data across a portion of the communication network instead of a second packet of data if the first prionty code is higher in priority than the second prioπty code. The arbiter is therefore said to grant accesses to downstream paths depending on the priority code of the incoming packets. Thus, the arbiter decides how to utilize a shared resource (e.g., the downstream path) when two or more packets arrive upon a network node or module. There are various ways in which the arbiter forwards multiple packets from one source before forwarding multiple packets of another source. Fixed arbitration may be one method, round-robin arbitration may be another.

The first forwarding module may be coupled within a first hierarchical portion of the communication network and have a first storage device containing a first grouping of bits unique to that first level. The second forwarding module may be coupled within a second hierarchical portion and have a second storage device containing a second groupmg of bits unique to the second level. A third forwarding module may be coupled withm a third hierarchical portion and mclude a third storage device contammg a third groupmg of bits unique to the third level, as well as an arbiter coupled to forward a packet of data from either the first forwardmg or the second forwardmg module dependmg on a companson of the first, second, and third groupmgs of bits.

The arbiter may therefore be used to forward the packet of data from the first forwardmg module if the first groupmg of bits yields a first level of pπoπty greater than the second level of pπonty. Alternatively, the arbiter can forward the packet of data from the first forwardmg module if the first groupmg of bits yields a first level of pnoπty the same as the third level of pπoπty. Still further, the arbiter may forward the packet of data from the first forwardmg module if the first groupmg of bits yields a first level of pnonty different from the third level of pnonty. Regardless of the arbitration algoπthm used, the arbiter decides which packet of data is to be forwarded to a shared resource. The first, second, and third groupmgs of bits are read from the first, second, and third storage devices and appended to data forwarded from the first, second, and third forwardmg modules, respectively. Importantly, the first, second, and third groupmgs of bits are set by instructions sent over a bus separate from the data flow path of the communication network In this manner, a hacker cannot sniff or spoof data packets by accessmg secunty registers and or mappmg tables configured withm the forwardmg modules. Absent access to sensitive mappmg mformation contained therein, a hacker cannot tap mto, wnte, or read packets of data sent across the structured, secured nerwork.

The communication system mcludes a transmission channel havmg a panty generator coupled to one end of the transmission channel and a panty detector coupled to the other end of the transmission channel. The panty generator produces a panty bit that, when added to a coπespondmg word of the header or trailer, produces a data pattern (alternatively known as "panty value") dissimilar from a data pattern (panty value Resulting from a dissimilar panty bit added to a coπespondmg word of the data. The data pattern attnbuted to the header or trailer is thereby dissimilar from the data pattern attnbuted to data. The generated panty bit is forwarded, along with its coπespondmg word, across the transmission channel. A panty detector is coupled at the other end of the transmission channel to receive the header, trailer, data, and generated panty. The panty detector is coupled to detect a panty value and determme a coπespondmg demarcation between the header, trailer, and data. The panty detector thereby detects an actual panty value at the output of the transmission channel that may be dissimilar from the panty value generated by the panty generator. Thus, if an eπor occurs when transferring the words of the packet across the transmission channel, the coπespondmg actual panty value received by the detector will yield an unexpected value, indicative of transmission eπor. If a word is discovered to be sent m eπor, then either the word is dropped altogether, the word is coπected, or the word is re-transmitted.

Dependmg on the number of panty bits generated by the panty generator and forwarded across the transmission channel coπespondmg to words of the packet, not only can eπors be detected, but those eπors can be coπected at the parity detector receive end of the transmission channel. A method is preferably used for transfeπing mformation across a communication network, and mcludes assignmg at least one panty bit to each word of a data, header, and trailer portion of a packet of mformation. The panty bit forms a different panty value when added to a coπespondmg word of the data then when added to a coπespondmg word of the header and/or trailer. The parity value assigned to each word of the data, header, and trailer before bemg transfeπed across the network can then be compared against a panty value resulting from transfer across the communication network. If the companson is negative, an eπor can be determmed m the transfer of the coπespondmg word. Furthermore, if the quantity of the panty bits transfeπed with each word is sufficient, not only can eπor be detected, but also that eπor can be coπected The mechanism of detecting eπor and coπecting eπor applies regardless of whether the coπespondmg word is found withm the header, the trailer, or the data portion of the packet

The packet of data transfeπed across the structured network mcludes control bits and eπor identification bits that mdicate where, withm the communication network, a channel dismrbance exists. The control bits and eπor identification bits contam values dependent on which module, withm a senes of modules connected to the communication network, the dismrbance is adjacent. The control bits may mclude an eπor value set by the module adjacent to the disturbance and also may mclude an acknowledge value set by a module withm the senes of modules that oπginated the packet. The module adjacent the disturbance can be deemed a receivmg module, and the module which oπgmated the packet can be deemed an oπgmating module.

Furthermore, the module to which the packet is destined can be deemed a destmation module The receivmg module adjacent the dismrbance is a module immediately upstream of the dismrbance Therefore, the receivmg module is the module closest to the dismrbance, and is the last module which receives the packet destined for the destination module which is downstream of the channel dismrbance (I e . severed, disrupted or error-induced channel) The terms "upstream" and "downstream" reflect the direction at which the packets are transmitted across the network and the relative locations of modules relative to the disturbance withm that network.

The communication system mcludes a transmission path compnsmg at least two transmission channels. The system also mcludes a receivmg module connected to the transmission path and havmg a loop-back conductor which operably connects one channel to the other channel, and returns the packet contammg eπor bits if the transmission channel or path downstream of the receivmg module is dismrbed The eπor bits returned with the packet by the receivmg module may mclude an acknowledge value set by the ongmatmg module whenever the eπor bits remmed from the receivmg module mdicate the transmission path is disturbed. The acknowledge value is used to instruct the ongmatmg module to either drop the packet or re-send the packet m an opposite direction around the loop to which the ongmatmg module and destination module are attached. The eπor bits mclude one or more unique identification number assigned to the receivmg module to note the receivmg module was the last of a plurality of modules that received the packet destined for a destination module Thus, the eπor value set by the eπor bits notes that a particular receivmg module was the last module not to detect a disturbance Each module withm the overall network is noted above as havmg a unique identification number That identification number is assigned based on the hierarchical status of all modules withm the structured, distπbuted routing system hereof

A method is contemplated for forwardmg a packet from a ongmatmg module to a destination module. The packet can then be received by a receivmg module configured between the originating module and the destination module If the transmission channel is severed between the receivmg module and the destination module, the receivmg module returns the packet to the ongmatmg module with the eπor bits appropnately set. Thus, the receivmg module snoops the packet for absence of a null pattern for a duration that exceeds the time needed to transmit a packet of maximum size allowed m the given protocol If a null pattern does not exist for the pre-deter ined time duration, it is noted that the receivmg module was the last module withm the transmission path that received a packet pπor to that packet bemg forwarded to the dismrbed channel. The receivmg module can then perform its appropnate loopmg function and appropπatelv set the eπor bits. The returned packet contains eπor bits which mdicate an eπor or transmission path dismrbance, and further mcludes the unique identification number of the last module (1 e, receiv mg module ) immediately preceding, or upstream, the disturbance.

Bπef Descnption of the Drawings Other objects and advantages of the invention will become apparent upon readmg the following detailed descnption and upon reference to the accompanying drawings m which

Fig. 1 is a diagram of a set of hierarchically aπanged nodes withm a structured network configured between nodes of a non-structured network, or between termination devices;

Fig. 2 are diagrams of a node configured accordmg to a smgle loop, multi-loop, multiple interconnected multi-loops or pomt-to-pomt topography, each configuration havmg a set of modules interconnected with one another between loops or along a loop or backbone;

Fig. 3 is a block diagram of a module employed as either an mtermediate, pass-through or end module, each of which includes a traffic manager;

Fig. 4 is a block diagram of a modular implementation of several traffic managers grouped together m building block fashion accordmg to a particular application and linked by a local bus extendmg from a control processor;

Fig. 5 is a block diagram of a traffic manager compnsmg a traffic controller linked to buffers and decoders placed withm the data transmission path of the network;

Fig. 6 is a block diagram of end modules placed on opposmg ends of the structured, hierarchical nerwork, each end module is shown linked to at least one termmation device and havmg resulting source and destination addresses shown,

Fig. 7 is a block diagram of the vanous buffers of Fig. 6 coupled m a shared aπangement between vanous external ports of the traffic manager, accordmg to one embodiment,

Fig. 8 is a block diagram of the vanous buffers of Fig. 6 distnbuted as dedicated buffers withm each data transmission path between respective external ports of the traffic manager, accordmg to another embodiment;

Fig. 9 is a module identification number and/or module address packet forwarded through a structured hierarchical network to enable fast decode of bit groupmgs (or fields) on a field-by-field basis accordmg to the hierarchical, relative placement of modules withm levels of the network; Fig. 10 is a logical diagram of nodes and associated switches aπanged withm levels, each level indicating a field of bits unique to switches withm that level;

Fig. 11 is a state diagram of a fast decodmg algonthm used to identify a routmg path through the network of hierarchically placed switches,

Fig. 12 is a block diagram of end modules placed on opposmg ends of the structured, hierarchical network, each end module is shown linked to at least one termmation device and havmg resulting source and destination addresses shown;

Fig. 13 is a diagram illustrating resolution of an end module address relative to a targeted termination device, and the routmg of data through the entire structured network without involving a hop or look-up operation, except for a relatively small mappmg to possibly N number of targeted termmation devices; Fig. 14 is a diagram illustrating a buffer of the traffic manager withm an end switch, wherein the buffer is placed withm a data path and contains wrap mformation to be appended to data as it traverses the buffer; Fig 14a is a block diagram of a companson function occumng at the entrv module and, resulting from that companson, placement of security mformation (identification and authorization codes) and pnoπty code at the header of the packet as well as secuπty breach information (auditing codes) at the trailer of the packet,

Fig 15 is a state diagram of a sequence of read operations dispatched to the buffer of Fig 6 in order to compile a packet of mformation stored m the buffer,

Fig 16 is a block diagram of a selector and various multiplexers which form at least a poπion of the decoders shown m Fig 5, wherein the selector and multiplexers route data either directly between ports or between ports via buffers depending upon the availability of the targeted port,

Fig 17 is a block diagram of a packet wrapped m accordance with a sequence of read cycles shown in Fig 15, and forwarded across the structured hierarchical network,

Fig 18 is a flow diagram of a senes of secunty and pnonty assignment operations occumng withm an entry (or mtermediate) module before forwardmg the wrapped packet to the successive module withm the network,

Fig 19 is a block diagram of a mappmg table withm an entry (or mtermediate) module used to assign a secuπty code to that module based on the source address ( identification number) of the mcommg packet,

Fig 20 is a diagram of a node or several nodes aπanged m multi-loop, wherem each loop may have a specified security code to effectuate hierarchical transfer of data from loop-to-loop based on the mter-loop secunty code,

Fig 21 is a flow diagram of a compare operation used to determine whether to forward data to a successive loop withm, for example, the multi-loop structure shown in Fig 20,

Fig 22 is a block diagram of possibly multiple mappmg tables within an exit module, wherem a companson of the ongmatmg identification number withm the entry module can be compared agamst entnes withm the selected mappmg table to determine if a user is authorized to receive the packet coπespondmg with the secuπty code and the ongmatmg identification number, Fig 23 is a flow diagram of the companson routine of Fig 22 for authorizing or not authorizing transmission of the packet to the user at a termination device connected to the exit module.

Fig 24 is a block diagram of a mappmg table withm an entry (or intermediate ) module used to assign a pπonty code to that module based on a bandwidth allocation value and a comparison to the source address (identification number) of the mcommg packet, Fig 25 are two examples of timing diagram used to illustrate dynamic assignment of pnonty to a header of a packet dependmg on whether the packet transmission speed exceeds or is less than the bandwidth allocation value,

Fig 26 is a tuning diagram of an arbitration scheme for granting bandwidth resources among mcommg packets havmg dissimilar pπoπty codes, Fig 27 is a diagram of a node or several nodes aπanged in a multi-loop wherem each module of each loop may have a specified pπontv code assignment for packets which traverse the module, and wherem each module which links two or more modules grants mastership to an incoming packet based on an arbitration scheme,

Fig 28 is a pian diagram of the packet compiled in accordance with Fig 15 wherein the packet preferably contams data wrapped with a header and trailer and each word of mformation contamed therem is preferably assigned one or more paπtv bits which when added to the coπesponding word produces an even or odd parity depending on whether the word is withm the header, trailer or data portion of the packet,

Fig. 29 is schematic diagram of block diagram of the a modulo two adder used as an even panty bit generator,

Fig. 30 is a block diagram of counters that can count a fixed number of words withm the header, trailer or data portion of the packet, and further mcludes panty detectors that can determme if eπors occur withm each word and, if sufficient panty bits are provided, provide coπection to words contammg eπors;

Fig 31 is a plan diagram of the packet compiled by a seπes of fetch operations to the buffers shown m Fig. 15, wherein a control field of bits and an eπor identification field of bits contamed withm the header and trailer identify the location of a dismrbed transmission path, or individual transmission channels which form a path;

Fig. 32 is a plan diagram of the control field of bits used to note and acknowledge an eπor, and maintain efficient transfers even though an eπor (e g , ring, star or mesh transmission path or channel dismrbance) has occuπed,

Fig 33 is a plan diagram of a mesh or star topology of multiple modules interconnected with one another and employed to send and receive eπor control bits and eπor identification bits withm packets forwarded across the topology, and

Fig. 34 is a plan diagram of one type of network employmg a πng, wherem the nng or loop mcludes a set of modules connected to a pair of fibers or connected to a pair of conductors or wireless media, each of which are henceforth refeπed to as transmission channels. While the invention may be modified and have alternative forms, specific embodiments thereof are shown by way of example m the drawings and will herein be descπbed m detail It should be understood, however, that the drawings and detailed descnption thereto are not mtended to limit the invention to the particular form disclosed, but on the contrary, the mtention is to cover all modifications, equivalents and alternatives falling withm the spiπt and scope of the present mvention as defined by the appended claims.

Detailed Descnption of Prefeπed Embodiments

Turning now to the drawings, Fig 1 illustrates a communication network 10 Network 10 mcludes an interconnection of subnets linked by nodes 12 Network 10 can be thought of as one or more intranets interconnected with one another, or interconnected via an internet. Each node 12 may embody a subnet or a plurality of mtercoimected subnets Select nodes 12a and 12b can be used to receive mput data mto the structured network or transmit output data from the structured network Nodes 12a and 12b can be connected to external nodes of a conventional network or termmation devices 14 Fig. 2 illustrates exemplary forms of node 12 and. more specifically, modules 16 In the example shown, an entry module 16a receives packets of data from a termmation device 14 Node 12c illustrates a node compnsmg modules interconnected about a nng topography, refeπed to as a "loop" Along with the entry module 16a are mtermediate modules 16b and 16c

It is recognized that a node or subnet can mclude either a backbone or πng topography, dependmg on the desired application The vaπous nodes 12 can be classified as smgle nodes or multi-nodes, each of which can be an end node, a pass-through, or an mtermediate node dependmg on where withm the network they are arranged The vaπous modules withm each node can also be classified as an end module, a pass-through, or an mtermediate module based on their position withm the network Further illustrated by node 12d is the lnter- relationship between an end module 16a and an intermediate module 16b An end module 16a is classified as either an entry module or an exit module, dependmg on whether it receives packets coming mto the structured network or transmits packets from the structured network. An mtermediate module simply passes the packets from one module to another module, or from one node to another node. Each module is configured m hardware to be somewhat similar to other modules, dependent only on the number of mput and output ports needed by that particular module Variability is imparted m the way m which the modules are programmed. An entry module is programmed to wτap the mcommg packets, an mtermediate module may modify and/or forward the wrapped packets, and an end module strips the wrap mformation. Specificity of each module is imparted during configuration dependent on each module's position withm the overall network It is understood that the modules access physical media and configuration occurs withm registers or each module. A decoder which operates similar to a Media Access Controller ("MAC") discerns the incoming packets based on the destmation MAC address usmg vanous types of physical media protocol, such as Ethernet. Token Ring, FDDI, or ATM.

Node 12e illustrates two multi-loop combmations of modules, the multi-loops are indicated as reference numerals 20a and 20b Each multi-loop 20 is shown interconnected bv an intermediate module 16b. It may be that one or more mtermediate modules of the node are configured off the loop, as shown by reference numeral 18. Modules 16 can have almost any number of mput and output poπs. and can be configured to either wrap an mcommg packet, stπp an outgoing packet, and/or pass-through the original packet or modified packet (with or without buffering), dependmg on where the module is m relation to the network topography or termmation devices 14. Incoming packets can be received over any type of media, mcludmg wired media or wireless media.

Fig. 3 illustrates a traffic manager 22 which serves numerous functions, one of which is to optimally route packets of data between mput and output ports. The number of mput and output ports can vary and the interconnection between certain input ports and certain output ports can also vary dependmg on the configuration of modules 16 Modules 16 can be called upon to mterface with a different physical media than the loop protocol by usmg a Physical Media Interface ("PMI") 24. PMI 24 performs many functions, one of which mcludes the MAC function of moving data over physical hardware conductors based on the physical address on each network mterface card withm the computer termination device, and another of which is to synchronize the packets of data being sent or received. If modules 16 is not used to convert between OSI layers, then the wrapped packet (e.g., a physical media packet wrapped with higher domain protocols) may be directly sent through modules 16 to the appropπate port.

Fig. 4 illustrates the modulaπty by which multiple traffic managers can be interconnected to a control processor 25 via a local bus. The functions of control processor 25 and its associated local bus are completely separate from and outside the data communication path of the forwarded packets Thus, packets are received by mput ports, shown as port A and are disbursed to output ports, shown as port B or C The mcommg packets on port Ai can be forwarded at a dissimilar speed and possibly with a dissimilar protocol from that of packets forwarded across port A₂. As shown, packets speeds for port Ai may be 1.0 Gb/s, whereas speeds across port A₂ may be 2.5 Gb/s. The mcommg packets on port A_. can be forwarded to port B| or port , while mcommg packets on port A₂ can be forwarded to port B₂ or port C₂, etc. Alternatively, packets from ports A, B, or C can be forwarded back to itself Traffic managers 22 can each be formed on a printed circuit board 26 separate from each other. Thus, each traffic manager 22 may be formed on a smgle mtegrated circuit or a set of mtegrated circuits mtercoimected upon a respective printed circuit board. Control processor 25 can be located m a chassis contammg the printed circuit board on which each traffic manager 22 reside, or m a chassis separate from those printed circuit boards. One or more pnnted conductors, cables, or wireless medium may extend from control processor 25 to each of the vanous mtegrated circuits which form traffic managers 22.

It is desired that each traffic manager be configured during assembly of the network. Each traffic manager can mclude programmable logic and. more importantly, non-volatile memory. When programmed, the traffic manager 22 will maintain its programmed status unless it is reconfigured The traffic managers are therefore configurable usmg known "firmware" program techniques. Such techniques mclude programmable read-only memory ("PROM"), programmable registers of non-volatile memory, fuses, anti-fuses, etc.

Reconfiguration of the traffic manager can take place if, for example, traffic congestion is discovered m a select region of the network. Alternatively reconfiguration can occur when the network is changed or expanded. Still further, reconfiguration might occur when a section (e.g., subnet) of the network is taken down or deactivated for vanous reasons In these mstances, one or more traffic managers can be reconfigured m the field or, alternatively, those traffic managers can be selectively reconfigured usmg a broadcast technique. Reconfiguration signals may be sent from control processor 25 to achieve that end It is desired that each traffic manager be configured during assembly of the network That is, each traffic manager can mclude programmable logic and, more importantly, non-volatile memory When programmed, the traffic manager will maintain its programmed status unless it is reconfigured. The traffic managers are therefore configurable usmg known program techniques usable with, for example, firmware, such techniques mclude programmable read only memory ("PROM"), programmable registers from non-volatile memory, fuses, anti-fuses, etc.

The traffic manager configuration will be maintained unless the network is reconfigured. Reconfiguration can take place if, for example, traffic congestion is discovered m a select region of the network. Alternatively, reconfiguration can occur when the network is changed or expanded. Still further, reconfiguration might occur when a section (e.g., subnet) of the network is taken down or deactivated for vanous reasons. In this mstance, one or more traffic managers can be reconfigured m the field or, alternatively, those traffic managers can be selectively reconfigured usmg a broadcast technique. In particular, a transceiver may be used to transmit reconfiguration signals to control processor 24. whereby processor 24 will dispatch the appropπate reconfiguration signals to one or more traffic managers connected to the local bus. In this manner, multiple traffic managers can be scaled to meet a particular application of a switch havmg mput and output ports carrying packets transmitted at dissimilar speeds and or dissimilar protocols. An intermediate switch can be linked between subnets, whereby certain subnets mvolve packets transmitted at different speeds from those of other subnets, and by interconnecting multiple traffic managers withm an overall motherboard 28 configuration, an mtermediate switch formed by multiple traffic managers can accommodate this, as well as other applications. Fig. 5 illustrates one example of module modulanty. Illustrated is a loop 30 havmg respective ports of modules 16 connected to that loop. Loop 30 may consist of one or more optical pathways, or metal conductors Each module 16 may have one or more mput ports as well as one or more output ports. Module 1 (MOD^ illustrates a Y-port configuration, whereas module 2 (MODW₂) illustrates a star-port configuration. Module 3 (MOD ) is shown scaled with multiple traffic managers, each havmg at least two ports, one bemg an mput port and the other bemg an output port Coupled between modules may be a diagnostic-momtonng-control (DMC) network. DMC network mcludes any wired or wireless communication network that can selectively forward diagnostic, monitonng. and/or control mformation to a traffic manager at the vaπous locations of the modules. The DMC network thereby allows modifications to the traffic manager function, wherem the DMC network can extend to all or a select few of traffic managers throughout the network DMC network communication is either separate from the structured network flow path, or can use m-band signaling

Fig. 6 illustrates traffic manager 22, shown m further detail. Traffic manager 22 functions to direct mcommg data to an appropnate output port. It determines the output port for an mcommg data packet based on a simple algoπthm. The traffic manager is aware of its position relative to the network and its neighboring traffic managers or nodes. Based on this knowledge, the traffic manager determines the appropnate output port in accordance with the pre-determined algonthm. The traffic manager 22 also interfaces between ports of different media, speed and direction. After the output port assignment is made, the traffic manager can: (l) store the data in a local buffer if the mcommg data is slower than the speed of the directed-to output port, until there is sufficient data to sustam the speed of the output port, (n) forward the data without an intervening storage if the output port is free or if the mcommg data is at the same speed as the output port, or (in) transfer the data through a FIFO type storage device when the mcommg data is coming at a faster rate than the speed of the output port. The traffic manager 22 manages resource allocation, such as allocation of vanous output ports and buffers, and the algonthms make decisions regardmg allocation to reduce the latency through the traffic manager. A cntical flow path for achievmg extremely low latency across the switch is to forward mcommg data from a port to the output of the same port. To accomplish this, mput processmg logic of the traffic manager recognizes this case and informs the logic withm a couple of cycles to start the data packet. To speed this process, the wrapped header has the necessary mformation at the very beginning of the packet. Control logic enables the flow through path if the output port is free withm a couple of cycles and informs the output logic. Input logic pipelines the data packet and stπps some of the header. Output logic creates the snipped head and starts sendmg the packet out. It then appends the mcommg data and continues the transmission of the packet. In a pipeline configuration mvolvmg multiple cycles, an example can be used to mdicate how the traffic manager minimizes latency when undergomg a flow through condition. The sequence of transfers or cycles received begm with a start of packet (cycle 1), followed by a first header (cycle 2), a second header (cycle 3), data 1 (cycle 4), data 2 (cycle 5), data 3 (cycle 6), data 4 (cycle 7), etc., to complete seven exemplary cycles. During the third cycle (e.g., the first cycle is when a staπ of packet arnves at the input logic), mcommg logic signals the control logic that there is a flow through packet. During the sixth cycle, the control logic notifies the outgomg logic that a flow through packet is on its way and the outgomg logic sets up the start of packet (start of packet mformation remains fixed from packet to packet) and receives header data from the mcommg logic. The seventh cycle mdicates mcommg data bemg passed onto the output port through the output logic.

Traffic controller 22, shown in Fig. 6, mcludes one or more buffers, as well as one or more decoders. In the example shown, the buffer is distπbuted as two buffers 34a and 34b, and the decoder is distπbuted as three decoders 36a, 36b, and 36c. Distπbution of the buffers and decoders depends m part on the configuration of the ports. In the example shown m Fig. 6, a three-port configuration is shown. However, more than three ports and possibly only two ports can be used, wherein the number of decoders and buffers would coπespondmgly change.

Each port is shown as a bi-directional port and thereby accommodates full-duplex dialogs. The physical media of port A may be different from that of port B or port C. especially if module 22 is configured as an end module or an intermediate module. For example, decoder 36a serves to decode addresses of an mcommg packet across port A to ensure that module 22 is to receive the packet Traffic controller 38 can be used to decide whether that packet should be forwarded to output port B or port C via decoder 36b In many instances, the packet can be forwarded directly from port A to port B (or port C), or is forwarded through buffer 34a. Similar to port A, port B and port C incoming packets can be forwarded to port A, port B, or port C via decoder 36c. Traffic controller 38 includes control signals and address lmes extendmg to each of the buffers and decoders for effectuating optimal routmg based on the availability of the destmation ports. Coupled m parallel across buffer 34b is a conductor over which packets can be sent mstead of sendmg those packets through buffer 34b. Thus, incoming packets on port B or port C can be forwarded directly to port A, to port A through buffer 34b, or directly to port B (or port C), as shown m the example of Fig. 6. A bypass conductor could also be shown coupled m parallel across buffer 34a; however, for sake of brevity and clanty m the drawing, only one bypass conductor is shown Fig. 6 illustrates a three-poπ (or Y) configuration. However, a different port configuration is contemplated with additional buffers, decoders, bypass conductors, etc , to effectuate any switch configuration and numerous ports of that module m order to directly or indirectly forward packets from any given input port to any desired output port

Withm decoders 36 and traffic controller 38 are configuration registers The configuration registers may be configured from an In-System Programming ("ISP") tool used for configurmg a Programmable Logic Device ("PLD") or a Programmable Gate Aπay ("PGA") The configuration registers and therefore the traffic controllers and buffers can be thought of as a PLD or a PGA Alternatively, the configuration registers can be programmed from non-volatile memory possibly associated with switch 22, or from the control processor via the local bus. The configuration registers 40 receive configuration mformation, preferably duπng initiation of power to switch 22. Altematively, registers 40b may be formed as an SRAM Still further, registers 40 may mclude a PROM or ROM permanently programmed by the network installer Contamed withm each register are bits used to enable certain ports m order to fix the forwardmg of packets from one port to another, predefined port. Alternatively, each register may be configured to select vaπous ports and paths based on dynamic traffic conditions. Withm register 40a bits may be programmed to enable transfer of all packets from port A to port B, for example. Other bits programmed withm the vanous registers mclude an identification number attnbuted to that particular switch Each module withm the network has a coπespondmg unique identification number. The identification number signifies the relative location of the node with respect to other nodes m the structured network hierarchy. The identification numbers of all modules withm the network contam bits bifurcated mto fields and, as descnbed herem below, each field coπesponds to a particular level A decoder which receives mcommg packets can be configured by its coπespondmg register to only decode a field of bits to which that particular module has been attnbuted. In other words, if module 22 belongs to level 3, then decoder 36a can only decode the destination address of the mcommg packet sent across port A withm field 3 of the multi-field address. This ensures a fast decode operation and significantly enhances the throughput of data. If the decode yields a companson, then decoder 36a will forward the packet to the appropnate decoder 36b or 36c dependmg on which port is configured as the output port established withm register 40a, 40b, 40c, and 40d Alternatively, a decoder may be configured to decode all fields above it m hierarchical level, or one field above it, to determme if the packet belongs to the cuπent level Further details regarding differing functions, configurations, and operations of module 22 will be provided when discussmg whether module 22 operates as an end module (entry end module or exit end module), mtermediate module, or pass-through module

Referring to Figs. 5 and 6 m combination, module 1 (MOD,) can be configured as a pass-through module, whereby ports B and C are coupled to coπesponαing loops, such that mcommg data on one loop can be sent to mput port B and back out to output port B on the same loop, or output port C on another loop. Thus, a pass-through module may mclude only two ports, where the decoder adjacent the mcommg packet will make the determination on forwardmg that packet to the output port An mtermediate module, configured as module 1 , module 2, or module 3 includes coπespondmg decoders of each mput port for determining whether the module is bemg targeted and which output port is to receive the forwarded packets, and further whether that packet will be sent directly or through a buffer based on configuration mformation withm the mcommg port decoder, or the availability of resources

Figs. 7 and 8 illustrate buffer 34 of a traffic manager bemg either a centralized buffer 34' or a distributed buffer 34" A centralized buffer 34' is used to receive and disburse packets of mformation from any particular mput buffer to any particular output buffer Accordmgly, a buffer can be considered as any contiguous memory space, such as that found in semiconductor memory A popular form of semiconductor memory mcludes smgle or multi-port RAM or DRAM (e.g., ZBT RAM or SDRAM) Buffer 34' may mclude dedicated memory portions across which packets from one port to another port are stored. Distπbuted buffer 34" is one havmg buffers dedicated to stormg packets of information sent between port pairs Regardless of the configuration, it is appreciated that data flow packets may be temporanly stored in mam memory ~ either withm a portion of a centralized memory or to a dedicated memory embodied upon one or more mtegrated circuits.

Fig. 9 illustrates the format of bits representing a module identification number 40. The same format is used as a destination address of a wrapped packet forwarded through the structured, hierarchical network hereof. Each module withm the network contams a unique identification number programmed withm the configuration registers the module's decoder and/or traffic controller. All modules associated with a particular level have a unique bit or set of bits programmed withm only one groupmg of bits (field) In the format shown, field N may mclude X number of bits which can therefore accommodate 2 number of modules withm level N of the structured network. Similarly, level N- 1 may have Y bits and there can be 2^X level N-1 groups and each of these groups m level N-1 can have 2^Y number of switches Given that field N coπesponds with the highest hierarchical level, any routmg beginning at that level entails decodmg field N address of the incoming data with field N identification numbers before decodmg the next level (field N-1)

Referring to Figs. 9 and 10 m combination, one example of a distributed routing mechanism usmg structured hierarchical decode mechanism is provided The network 10 mdicates three levels of possibly numerous levels. Modules 16 are each configured with a unique identification number withm a field of bits applicable to that module's level In particular, the identification number is different withm only one field attnbuted to modules X, and the identification number is different with another field attributed to modules Y. All other fields are don't care fields, or are lπelevant for comparing withm that le\ el In the example shown m Figs. 9 and 10, communication from a termmation device 14a to 14d is forwarded to a senes of compare operations. First, the address of a packet sent from termmation device 14a, at lev el N-4, is compared with an identification number and, more specifically, a field of bits in level N-3 and higher If the companson does not yield a match, then it is determined that the packet must be forwarded from module A 1 m level N-4 to the next higher level When a compare of fields N-2 and higher yields a match, the match is then checked against all nodes at that level and forwarded to the appropnate module identification number (e g , module A2) that yielded a match m level N-3 It is known that termmation device 14a is coupled to module A-l and, therefore, decode need not take place at level N-4 Secondly, another decode must occur at the next higher level (level N-2) to determme if the address forwarded from termmation device 14a is to be routed to a module m level N-2 If the companson does not yield a match, then it is known that routing need not occur to the next highest level. The routing algoπthm performs a companson of the next lower level. Knowmg the highest level of routing is currently at modules X withm level N-3, the next companson of the packet address is performed on modules m level N-4 If a match results, then that match is identified as a particular module (e g , module A3). The companson or decode process continues onto the next lower level, level N-5, to determme if further routing is needed. If the address at the next lower level of fields compares with an identification numbers m the next lower level field (level N-5), then it is determmed that the routing must continue to a particular module A4 that yields the companson The term "routmg" is used m the sense of forwardmg data packets across a network, from module-to-module, and should not be construed as mvolvmg a router. Instead, the routmg herem merely refers to an mcremental step m the distπbuted routmg mechanism of the overall network and is used by a module to forward packets among hierarchical levels based on a decode companson

As shown m Figs 9 and 10, and further illustrated m Fig 11 , the forwardmg or mcremental routing algonthm operates solely upon the destmation packet address and identification numbers stored m registers of each module Registers within the input/output port decoders and traffic controllers are called upon by the decoders to perform a comparison with the mcommg addresses without need for performing a look-up operation Importantly, decode need only be performed on one or more number of fields withm a plurality of fields ~ that field coπespondmg with where the pnor companson occuπed withm the overall hierarchical structure. Knowledge of the termmation device whereabouts relative to the entry and exit end switches is a prerequisite to the present fast decodmg algonthm. The mechanism for determmmg termmation devices relative to end switches will be discussed herem below when performing a broadcast operation of the IP addresses across the structured network to all the vanous termination devices coupled to that network.

Fig. 11 illustrates a fast decodmg state diagram 42 of distributed routmg mechanism. Dependmg on which level a termmation device is connected, decodmg occurs at all higher levels, beginning with the next higher level field of bits. An entry end module can mark a bit m the control word that sigmfies which level the module is m. and then other modules m that level can compare only the field attnbuted to that level. The mtermediate modules can perform the same function when forwardmg the packet to the next level — either a higher or lower level that is setting a bit or bits m a control word signifying a match of levels In the example shown m Fig 10, decodmg occurs m level N-3 for an oπgmation termmation device 14a coupled to a switch m level N-4 State 44 is contmued for however many times it takes to yield a non-compare (! COMPARE). Once a non-compare occurs, then it is known that routmg must now proceed, or possibly stop, at the level just below the level that yielded a non-compare. In the example of Fig. 10, companson stopped when level N-3 yielded a non- compare — indicating that all higher levels greater than level N-3 will not receive the forwarded packet Beneficially, all higher levels and denvatives from those higher levels which are outside of switch A2 shown m Fig. 10, need not undergo a companson function Compansons at each level can be compaπsons at all higher levels. Once a higher level compare is true, then the lower level compansons are performed. Entry end modules or nodes can mark a bit in a control word which signifies the level and the remaining modules or nodes m that level may compare only that level of bits Intermediate nodes or modules can do the same when forwardmg the packet to the next level. Once the highest level is identified for forwardmg the packet, decode now continues down the hierarchical levels m succession, as shown by state 46 Compaπsons for those levels are performed until a lower level module identification number favorably concurs with the destmation address That module is identified as an exit end module that is selectably coupled to the target or destmation termmation device 14d shown in Fig. 10

The successive decode of bits withm only a certain field, and progressing field-by-fϊeld across the mcommg address and identification numbers minimizes the number of compaπsons needed and the number of switch identification numbers bemg compared agamst This relatively fast decode operation performs distnbuted routmg function withm the structured, hierarchical network, with the benefit of performing mcremental routing by decode operations only on the field of interest and ignoring all other fields at any given pomt. The concept of bemg aware of all switch whereabouts withm the network is of primary importance when forwardmg the packets This awareness in the structured network is beneficial not only m a hierarchical network but also in smgle level of hierarchy such as local loops For mstance, based on the identification number mdicatmg where the switch is located, and the destination address, the packet can be sent around the local loop among the interconnected switches, either nght of left

Figs. 12 and 13, m combination, show path routmg outside of the structured, hierarchical network. That is, while structured positioning and fast decodmg performs distributed routmg withm the network, identification of a routmg path to and from termmation devices must be established. Coupled to termmation devices 14a and 14b are end modules 16a and 16b, shown m Fig 12 If device 14a is the ongmator of a packet, then end module 16a can be considered an entry end module If the incoming packet mcludes a destination address to termmation device 14b, then device 14b is considered the targeted or destmation termination device, and end module 16b is considered an exit end module From this example, Fig 13 illustrates firstly, a mechanism for identifying the relationship between termmation devices and end modules (particularly when there are multiple termmation devices per end module) and, secondly, how the mcommg packet is routed from the oπgmatmg termination device to the destmation termmation device 14a and 14b, respectively Transfer of a wrapped packet from an ongmatmg termmation device 14a to a destination termination device 14b entails numerous mechanisms Reference numeral 37 illustrates a wrapped packet ensumg from entry module 16a. The wrapped packet mcludes a source address (SA) of an ongmatmg termination device havmg a unique identification number, and also mcludes a destmation address (DA) of the exit module (MOD2 ADDR) 16b. The source address is therefore the address of termmation device 14a which gets translated withm the entry module 16a to an address of module 16a This translation proves beneficial when the destmation termmation device replies to the oπgmating termination device.

Refeπing to Fig 13 m more detail, an Address Resolution Protocol ("ARP") may be broadcast from termmation device 14a The destmation address of the broadcast ARP is to each and every node withm the network, and continues on until all nodes have received the resolution protocol, as shown by block 50. Contamed withm the resolution protocol may be an IP address of destmation termmation device 14b. If so, then device 14b replies, as shown m block 52 That reply represents an ARP answer forwarded back across the network and eventually to the ongmatmg termination device 14a Based on the broadcast of addresses and ensumg replies, the positions of termmation devices relative to end switches can be determmed based on the IP address of those destmation and ongmation devices. Accordmgly, the address at destination device 14b is found based on its IP address bemg broadcast, as shown by block 54 That address and its configured location relative to end switch 16b will thereby resolve their connectivity relationship, as shown by block 56 Blocks 50-56 illustrates a mechanism for determining the relative location of termmation devices, nodes, and switches accordmg to well-known address resolution protocols Accordmgly, the technique of broadcasting an address (e.g , Ethernet address) across the network to resolve locations hereof is backward compatible with existmg legacy location mechanisms known to the skilled artisan. However, it is from the resolved locations that a packet can be quickly transfeπed between locations external to the present structured network (i.e , termination devices or a non-structured, conventional network)

Transfer of a wTapped packet from an oπgmatmg termination device 14a to a destmation termmation device 14b is shown in block 58, and entails mechanism 60 Other protocols or domams outside of IP can be wrapped usmg the present technique, and therefore Fig. 13 is merely an example of one way m which a geneπc protocol (e.g., IP can be wrapped). Referring to the examples of both Figs. 12 and 13, reference numeral 60 illustrates an incoming IP packet wrapped by end switch 16a and simply havmg a source address (SA) of ongmation termination device 14a and a destmation address (DA) of exit end switch 16b The wrapped packet is shown as reference numeral 62 More specifically, entry end switch 16a merely wraps an existing packet with a known identification address of an exit end switch, known m its relationship to a destination termination device by virtue of the address resolution protocol Moreover, the destmation termination device address is contamed withm the IP packet of wrapped packet 62

Before sendmg the wrapped packet onto the structured network, entry end switch 16a converts only the source address from that of termmation device 14a to that of the entry end switch 16a, if multiple termmation devices are attached to the end switch, as shown by the wrapped packet 64 Knowmg the source address as the entry end switch address proves beneficial when the destmation termmation device replies to the ongmatmg termination device, as will be discussed herem below The destmation address and the IP packet remam the same, as illustrated by companson between wrapped packet 62 and wrapped packet 64.

Once the wrapped packet is received upon the exit end switch 16b, only the destmation address changes from exit end switch 16b to destination termmation device 14b especially if there are multiple termmation devices attached to end switch 16b. The source address, however, remams the same as that which was changed when the packet is forwarded mto the structured network Given that the destination address is termmation device 14b, exit end switch 16b performs the smgle, and only look-up, needed by the entirety of the structured network. As defined herem, look-up is a deterministic, static. 1 -to-N mappmg to a localized device from among many such devices Mappmg proceeds from the destmation address placed on the wrapped packet by the exit end switch

16b m order for the IP packet to be directed to the appropnate termination device by comparmg the IP address withm that packet with the appropnate IP identification number of a possible x number of termmation devices, as shown by translation table 68. Mappmg table 68 may be formed withm memory of end switch 16b or, altematively, a selection unit such as multiple multiplexers, or any other form of routmg based on selection signals. Mappmg is performed by comparing the destmation address (IP address) withm wrapped packet 66 with the IP identification number stored m an appropnate termination device T_λ When the companson yields a match, the packet is forwarded to the appropnate termination device (e g , termmation device 14b as opposed to numerous other termination devices connection to end switch 16b).

Any reply to the initial forwarded address can be sent back across the structured network based on the format of wrapped packet 66 More specifically, the source address of the reply remams the address of termmation device 14b, however, the destination address is the previous source address — l e., the address of entry end switch 16a. In this manner, reply mformation is made similar to the initial forwarded information. Regardless of the initial request or a subsequent response, the packets of data are routed to the opposmg end switch. In particular, mformation from termination device 14a is directed to exit end switch 16b, while reply mformation from termmation

ice 14b is directed to entry end switch 16a The source address is therefore said to change whenever a packet enters the network, however, the destination address remams the same. The same applies on a reply Whenever a packet exits the network, only the destmation address changes with the source address remaining the same This ensures that when traversing the entire structured network between termmation devices, only a smgle mappmg or translation occurs when forwarding the packet outside the structured network to one of a plurality of termmation devices. Intermediate switches and/or nodes of the network do not perform conversion of the destmation or source addresses, nor do they perform any look-up. The look-up anses from a relatively small table, known alternatively as a translation table. Importantly, the number of termmation devices linked to the exit end switch on a request operation (or an entry end switch on a reply operation) is relatively small. It will be appreciated that the structured network can form a subnet withm a larger, conventional network In this manner, the hierarchical decodmg and smgle mappmg applies to data traversing the structured network, that data bemg packets which are forwarded across multiple subnets where one subnet may be dedicated to the structured network Accordingly, the structured network can form the entire network of subnets (intranets) as well as the internet between one or more subnets or, alternatively, can form only a portion (one or more subnets) of a conventional network Regardless of which portion of the entire mter- and lntra-network is formed by the structured network, or whether the structured network compnses the entue network, only one look-up (defined as a mappmg function) need occur as packets exit the structured network to beneficially improve overall packet throughput. Absence of traditional routing m the structured network greatly enhances the speed at which packets are forwarded to their appropnate destination.

Fig. 14 illustrates a memory area withm a memory (or buffer) of an end switch Memory area 70 may be bifurcated mto regions, some of which receive certam packets placed m a defined order withm those regions. That is, the first packet (PK1 ) is forwarded mto and placed withm a defined portion of buffer 70 before the second packet (PK2) is placed and stored Buffer 70 therefore serves as a FIFO buffer, for example. The mcommg packets are wrapped as they leave buffer 70 by attnbutmg wrap mformation to a particular packet before forwardmg the wrapped packet from the buffer Wrap mformation is stored m another region of buffer 70 and can mclude but is not limited to the mformation shown m table 72 That mformation may compnse a groupmg of bits mdicatmg a start of packet (SOP) Another group of bits are used to mdicate the ongmatmg identification number of the ongmatmg switch (ORIG ID ADDR) Another set of bits may be dedicated to general control mformation, such as the level of hierarchy, length of packet header (or footer), packet types, etc. Yet another group of bits are dedicated to forwardmg a null pattern used possibly to mdicate spacmg between wrapped packets. Yet another set of bits may be used to mdicate an eπor m the packet transmission. It is recognized that the wrap mformation 72 can mclude numerous other bits dedicated to achievmg many dissimilar functions, only a small groupmg of which is shown by the examples provided. The manner m which the wrap information is wrapped about a packet is further descnbed m reference to Fig 15 More specifically, descnption of Fig 15 is applicable when referring to Fig. 15 m conjunction with Fig 6 Fig 14a illustrates vanous components of an entry or mtermediate module and. more particularly, to a compare unit 50 coupled between an mput port decoder (shown m Fig 6) and a storage device 52. The storage device may be one or more configuration registers or a table Each node mav have logically separate storage devices or address ranges withm device 52 dedicated for different functions such as an entry storage device portion, an mtermediate storage device portion, and an exit storage device portion dependmg on whether the node is attnbuted to an entry, mtermediate or exit function, for example As shown m Fig 6, the decoder receives an mcommg packet and determines whether the mcommg packet is destined for the network based on an identification number forwarded with that packet Fig 14a indicates compare unit 50 hich compares the identification number on the received packet with a set of bits within the entry storage device 52 to determme if the data will be transfeπed as secured data to the communication network The configuration register is set by the administrator during configuration usmg, for example, ISP techniques Entry configuration register 52 can be thought of as part of the entry module decoder configured at the mput port of that decoder or, alternatively, can be separate from the register of the mcommg packet decoder Once compare unit 50 determines whether the mcommg packet is a secured packet, it will place that packet m a particular position withm buffer 34

Buffer 34 is thought of as a contiguous memory location which may be bifurcated mto regions, some of which receive packets placed withm those regions That is, the first packet (PK1) is forwarded mto and placed withm a selected portion of buffer 34 before the second packet (PK2) is placed and stored Buffer 34 therefore serves as a FIFO buffer, for example The incoming packets are wrapped as they leave buffer 34 by attnbuting wrap information to a particular packet before forwardmg the wrapped packet from the buffer Wrap information is stored can be another region of buffer 34 and can mclude, but is not limited to, the information shown m tables 54 and 56 That mformation may compnse a groupmg of bits indicating a start-of packet (SOP) Another group of bits may be used to mdicate the ongmatmg identification number of the entry module (ORIG ID). Another set of bits may be dedicated to assignmg a given pnonty at which the attnbuted packet is to be sent across the network, while yet another set of bits may be used to assign a secunty code to those packets. The pπonty code and the secunty code may be determined based on the identification of the user generating the packet Table 54 can be drawn from buffer 34 and, by virtue of a read pomter, bit fields are applied to a header of the wrapped packet Table 56 illustrates bit fields which are drawn and placed as a trailer upon the wrapped packet A first groupmg of bits withm the trailer may be used in conjunction with a counter to mcrement each time the packet traverses a module The first groupmg, or MOD TRAVERSAL #, represents the number of modules traversed, mcludmg the existmg module, by a packet sent through the communication network. The MOD TRAVERSAL # is dependent on the path taken either around a loop or among loops The MOD TRAVERSAL # can be forced m select loops or paths to limit it vanabihty and number of permutations Additional fields are reserved for indicating the first module identification number, second module identification number, thud module identification number, etc for all modules bemg traversed up to and mcludmg the existing module It is therefore important to note that module 16a need not be merely an entry module Instead, it may be an mtermediate module, whereby the wrap information is modified as it traverses the network In other words, the mcommg packet can be wrapped with an oπginal ID, pπoπty code, secunty code, module traversal #, and earlier (upstream) module identification numbers, and the present module simply updates that information or maintains that information on the header and trailer exitmg the present module

Traffic controller 38 operates basically as a compiler which performs a senes of read operations upon buffer 34 Those read operations occur to fixed regions of that buffer so as to read bits from those regions m a particular sequence so that the resulting, outgomg packet is senally fed from the buffer m a particular order. As such, the compiler part of the traffic controller can be thought of as contammg a state machine of sequential logic and a pointer which perform a first read of possibly a null pattern, indicating the end of the pnor wrapped data packet The read null pattern is contamed withm the wrap mformation of the buffer, as well as other wrap mformation descnbed above The order in which the oπgmatmg identification number, pπoπty code, and secunty code withm the header is not necessanly important, as long as that mformation is somehow contamed upon the header of the packet as it exits the module and is consistent across the network The header mformation, as well as the trailer mformation, can be appended by an entry module or modified by an mtermediate module based on the mformation stored m buffer 34

The compare unit 50 of Fig 14a can either be unique to an entry module or an mtermediate module, and serves merely to determme if the packet should be continued from the module as secured data dependmg on the companson results If the bits withm the entry storage device 52 of an entry module do not compare with the mcommg identification number, then the packet may simply be dropped or tagged as an unsecured packet If the bits withm the intermediate storage device 52 of an intermediate module do not compare with the secunty code of the incoming packet, it will not be allowed to be accessed by any termmation device attached to that module If the bits withm the exit storage device 52 of an exit module do not compare with the secuπty code of the mcommg packet, the mcommg packet will be rejected and not forwarded to any attached terminating devices

Figs 6 and 15 illustrate traffic controller 38 operating as a compiler which performs a seπes of read operations upon a buffer Those read operations occur to fixed regions of that buffer so as to read bits from those regions m a particular sequence so that the resulting wrapped packet is seπally fed from the buffer m a particular order As such, the compiler part of the traffic controller can be thought of as contammg a state machine of sequential logic which performs a first read of a null pattern 74 indicating the end of the pnor wrapped data packet The read null pattern is contamed withm the wrap information of the buffer, as well as other wrap mformation thereafter beginning with the start of packet at state 76 The destination address of the structured network protocol is thereafter read at state 78, and the ongmatmg identification address is read at state 80 Control mformation withm the wrap mformation region of the buffer is then read at step 82, and so forth for the remammg read operations 84, 86, and 88 to complete the wrap of a packet from the buffer Control information may mclude information withm the header and trailer generally wrapped at the beginning and end of the data packet Each read or fetch occurs to a specific location withm memory to routinely drawn whatever information is present m that location Thus, the wrap mformation is configured m specific regions of the memory so that they can be drawn from those regions m the order specified by the state machine of Fig. 15

The module which has a traffic controller performing the various read operations is considered an end module and, more specifically, an entry end module The entry end module wraps the mcommg IP packet with vaπous wrap mformation needed to transfer that IP packet across the strucmred network and, additionally, to a termmation device outside of the structured network Therefore, the strucmred network is backward compatible with all other existing networks, with the wrap information simply subsuming the pre-existing wrap information The entry end module adds the wrap mformation and exit end switch removes or stπps the wrap information, as shown m Fig. 17

Fig 17 further illustrates bits added to and removed from an embedded packet (e g , Ethernet packet) The wrapped packet 63 shown m Fig 17 is but one example of a packet format used by the structured network There may be additional bits or field of bits added to the format, if desned The entry end module thereby adds the wrap mformation and the exit end module removes the wrap mformation to vield only the embedded packet with starting and endmg identifiers forwarded to a termmation device or to other non-structured subnets outside of the present structured network.

Fig. 16 illustrates vanous multiplexers 92 explamed m conjunction with Fig. 6 Multiplexers 92a and 92b can form a part of decoder 36b, while multiplexers 92c and 92d may form a part of decoder 36c. Vanous enable (EN) signals can be sent from the traffic controller which serves as an arbiter B EN serves to signify when the output port B is available, whereas C EN is used to signify when output port C is available. Dependmg on whether select signal B EN or select signal C EN is asserted, packets from mput port A, mput port B, or mput port C are forwarded to respective output ports B or C. In the example of Fig 6, Fig. 16 provides a corollary, m that signals upon mput ports B and C can be directed either to output ports B or C, as well as output port A, dependmg on the status of select signals B EN, C EN, and A EN. It is appreciated that the example shown m Fig 16 is used only for sake of brevity and clanty, as well as for consistency with Fig. 6. There maybe vanous configurations possible as to how these ports can be interconnected There may be numerous other ports beyond port A, B, and C used m practical applications and. therefore, Figs 6 and 16 are only for exemplary purposes only The vaπous enable signals may be fixed withm coπespondmg registers during configuration, those registers include register 40, shown m Fig 6 The enable signals can either be static or dynamically determmed by traffic controller 38. A selector 94 can further be used to forward data through a buffer output port A or bypass the buffer and place packets across a bypass conductor coupled m parallel with that buffer. The vanous enable signals, m combmation with the selector 94, are used to send data either directly between ports or between ports via buffers dependmg on the availability of the targeted port. The desirability of selecting between whether a packet can be sent directly to an output port or through a buffer is beneficial if the status of the output port is known That will be dynamically determined based on resource availability and a resource allocation algoπthm. The resource allocation algoπthm will be determined by the control registers configured through the local bus by the control processor. Fig. 16a mdicates a counter 61 coupled between the decoder (Fig 6) and the storage device 52.

Counter 60 can be a set of counters and is preferably embodied withm an entry module and counts the number of accesses upon the entry module by an entry termmation device This count can be used to control the amount of access generated to and from a secured user. For mstance, to control the access from the secured user, further accesses may be blocked when the count number exceeds the count value stored withm storage device 52. Use of a stored count value is beneficial if several successive, and possibly unauthorized, attempts to access the network are detected. The counter may undergo penodic reset so that access blockmg occurs only if an impermissible number of accesses occur withm a specified period of time Otherwise, the accesses are allowed and packet mformation is transmitted withm the secured network.

The secunty code shown m Fig 14a may contam secunty control bits which are reserved to mdicate whether secunty is on or off More specifically, the entry module storage device 52 contams bits which may match with an mcommg termmation device identification number. If so, then the compare unit 50 will forward one or more secunty control bits to the secuπty code withm buffer 34a. and those secuπty control bits will be transfeπed with the secunty code from one module to the next until they aπive on the exit module. The exit module can then determme if the data is secured based on a companson of those bits In the simplest form, the number of bits simply equals one, to designate a secunty on or off condition Alternatively, this control bit can be part of separate control word that will be transmitted with each packet Fig. 17 illustrates bits added to and removed from an embedded packet (e g., a physical layer packet, such as Ethernet). The wrapped packet 62 is but one example of a packet format used by the structured network. There may be additional bits or fields of bits added to the packet, if desired. The entry end module thereby adds the wrap information and intermediate module simply updates the wrap mformation applicable to the hierarchy of that module. The exit end switch removes the wrap information to yield only the embedded packet with starting and endmg identifiers forwarded to a termmation device or to other non-structured subnets outside of the present strucmred network The embedded packet may or may not include encryption code and can be any of the vaπous packet formats conventionally known m the art. As shown in the secuπty code field, a secunty bit (SB) may be reserved to mdicate whether secunty for that packet is on or off. Fig. 18 illustrates a flow diagram 65 of operations occurring withm an entry module. The operations begm 67 by reading the mcommg address and optionally the destmation address, whether that address is from (or to) a termmation device or from (or to) an upstream module designated with an oπgmating (or destination) identification number 69 The storage device, bemg either a configuration register or table of the module, is also read as shown by block 69. A companson 71 is performed on the mcommg identification number or address and the bits are read from the storage device. If the companson yields a match, then the secunty bit will be set, as shown by block 73 Moreover, the secunty code and pnonty code for that particular module will also be assigned 75. Thereafter, the wrapped packet contammg the secunty code and pπonty code (initially placed or updated) will be forwarded to the next module closest to the exit module, as shown by block 77. If the companson function 71 does not yield a compare, then the secuπty bit will be reset, no secuπty code will be set, and pnonty code will be set based on the assigned pnonty code for that packet at that mstance. The packet m this case can be sent as a non-secured packet with default pnoπty, if the module is configured to send non- secure packets or the packet can be dropped. In a given embodiment, pnonty code and secunty code can use two mdependent compansons. As understood from the foregomg, the destination address of the mcommg packet can also be used to determme the secured transmission of packets. In this case, the storage device will also contam a list of allowed destmation IDs for secure transmission Absent (or m addition to) determination ansmg from the ongmation or destination addresses, secunty or prionty classes of nodes and/or modules can be used to fix secured or pnontized transmission.

According to an alternative embodiment, each time an access to a module (e.g., entry module) occurs, a counter will be mcremented, as shown by block 79. When the count value exceeds a number stored in the storage device or configuration register, as shown by decision block 81, then further accesses are blocked 83. The blocked accesses can anse by executing a time out on the entry module as shown by block 85. If the count value (i.e., between timed resets) does not exceed that which is stored m the storage register, then further accesses can occur.

Fig. 19 illustrates an entry or intermediate module 91 which is configured to receive packets of data from an upstream module or termmation device 93. Module 91 includes a table 95 operably coupled to the mput decode unit. Table 95, similar to a configuration register, can be programmed by signals sent across the local bus 97. Local bus 97 is situated outside of the data flow entering module 91 and, therefore, any mformation programmed by local bus 97 cannot be read or written to by mcommg data packets. Only the system admmistrator can effectuate a change to table 95, or can read the status of table 95. Similarly, only the system administrator will be given access to table 95 - otherwise it is locked such that modification (read/wnte) is prevented. Table 95 compnses a relatively small mappmg table of incoming (termmation device) addresses and/or users and a coπespondmg secunty code There may be numerous links and, therefore, numerous upstream modules and termination devices connected to module 91, each one havmg an unique address or identification number known herem as possiblv an oπgmation identification number Vaπous users may also be linked to the upstream termmation devices, and secuπty codes can be assigned based also on the users which initiates the source address (SA) and users which receive the destmation address (DA) The identification number situated on the mcommg packet is read and compared agamst identification numbers withm the table If a match occurs, then the coπespondmg new or updated secuπty code will be placed on the packet as part of the header information If a match does not exist, then a secuπty bit can be set indicating that the packet shall not be a secured packet or, alternatively, the entire packet may be dropped An unauthorized user would therefore have to know both the secunty code and the identification code of an entry node/module m order to gam access mto the network and would have to know the secuπty code and identification code of m mcommg packet m order to gam access to the packet An additional layer of secuπty might be to employ a user-assigned password, where a user would have to know the security code, identification code and password and match the three indicia before being granted access Use of the password can prevent circumventing the secunty codes and identification codes by unwaπanted access to the programmed module tables

Fig 20 illustrates the hierarchy of secunty codes and identification numbers Accordmg to one embodiment, the number of modules A, B, C, and D can be relatively small, whereby a unique secunty code, e g , 180 - 193, is assigned to the coπespondmg modules accordmg to a predefined hierarchy In the example shown, three loops or rings have a secunty code between 81 and 8F, and one loop has a secunty code between 91 and 9F If the number of modules on one or more loops cannot be represented by a given hierarchical class of secunty codes, then separate identification numbers must be attnbuted to each module The identification numbers are shown in parenthesis beneath the secunty code In one embodiment, withm each loop, the secunty code or identification number monotomcally increases around that loop It is possible that each module withm a class may have the same secuπty code and the identification number is all that changes withm that class In this example, two or more modules may have the same secuπty code number, but with different identification numbers When the network becomes fairly large and complex, it is simpler to merely assign a unique identification number for each module and a secunty code to modules withm a given secunty class In this manner, a secunty code can remain constant with the packet as it traverses the network from module-to-module, and possibly only the initial identification number of the entry module is assigned to the packet and compared with the secuπty code at the exit module to determme secuπty authorization Altematively, m rather simple networks, the identification number can be eliminated and both the identification function and the secuπty function can be incorporated into a unique secuπty code for each module, the most significant bits of the secuπty code are reserved for a class or level of secunty and the least significant bits are reserved for identifying each module withm that class, as shown by the exemplary secuπty code format of reference number 100 In the example shown by reference 100, a secuπty code of a 81 may be assigned to a particular module, with secuπty code 82 assigned to the next successive module withm. for example, a loop The secuπty code 83 can be possibly the next module withm the loop, and so forth for all modules withm the secunty class of 81 to 8F, m the example shown Turning to Fig 21, a flow diagram 102 is shown of operations which define a secured path based on the hierarchical status of secuntv codes withm one or more modules of the present network In particular, determination must be made on w nether to torward a packet of data across a nerwork based on the secuπty code read by the mcommg packets

Descnption of the steps shown m Fig 21 is best illustrated in reference to Fig 20 For example, determination bv module C3 on whether to forward an incoming packet of data from module AI is dependent on the secuπty code of the mcommg packet and the secuπty code read from the buffer of module C3 In the flow diagram 102 the secunty code of the initial loop A, at module A 1 is read 104 Next, the secuπty code programmed mto the buffer bv the user via the local bus of module C3 is read, as shown by step 106 The same can be said of module B3 reading the security code stored withm its buffer via a configuration cycle Modules C3 and B3 perform a companson function by, for example, comparmg the most significant bits of the mcommg secunty code dispatched from module A I and the most significant bits of the assigned secuπty code of modules C3 and B3, as shown bv the compare operation 108 The compare operation can be aπanged so that if the incoming secunty code's most significant digit is greater than or equal to the most significant bit of the downstream, assigned security code, then the incoming packet of data will be forwarded to anv module havmg that assignment In the example of Figs 20 and 21, the mcommg security code is either 180 (if identification numbers are used) or 181 (if identification numbers are not used) The assigned secunty code of module C3 is 186 and the assigned security code of module B3 is 193 Smce the most significant digits "18" of security code 181 is not greater than the most significant digits "19" of security code 193, but is equal to the most significant digits "18" of secuπty code 186. then it is determmed that the secured path will extend from module AI to module C3, and not from module AI to module B3 As noted by the result of compare operation 108, the packet will be forwarded to the next loop C, as shown by step 1 10, and not forwarded to the next loop B, as shown by step 1 12 The process is repeated from loop to loop, and therefore from node to node until a secured path is presented through the entire strucmred network

Similar to the mapping table of an entry or intermediate module, Fig 22 illustrates a plurality of mappmg tables 1 18 withm an exit module 1 16 A plurality of mappmg tables are needed if the packet contams both an identification number and a security code As stated above if a secuπty code suffices both as an identifier and a security classifier then a identification number is not needed However m mstances where both indicia is provided withm the header, then the most significant bits of the security code defines a particular table withm the plurality of tables 1 18 If the most significant bit represents a number N, then the mappmg table N is chosen from the plurality of mappmg tables 1 18 Contamed within mapping table N is a set of identification numbers Those identification numbers are stored withm the exit module durmg configuration via, for example, the local bus or durmg m-svstem programming If the mcommg packet identification number (e g , ongmatmg identification number of the entry module) matches the identification number stored in mappmg table N, then it will be noted that the entry module has sent a secured packet The same can be said of an intermediate module bearing an identification number which matches an identification number ithin a secured packet In either mstance, each of the plurality of mappmg tables bears identification numbers which are configured durmg construction or reconfiguration of the network by commands or instructions mputed upon each module by the system admmistrator Users of packets sent into, across and from the network cannot access the storage devices, configuration registers, or tables ithm the entry, mtermediate or exit modules

Fig 23 is a flow diagram 120 that illustrates a senes of operations performed by the exit module of Fig 22 In particular the decode unit of the exit module reads the secunt\ code and chooses one of the plurality of tables based on. e g , the most significant bits of the security code For example, if the secunty code bears a number 184. then table "18" will be chosen Reading of the most significant bits and choosmg the mappmg table is shown by reference number 122. The mcommg packet mcludes an identification number, preferably the ongmatmg identification number of the entry module, and step 124 illustrates the exit module reading that identification number from the coπesponding packet A comparison of the ongmatmg identification number with entπes m the table is made at comparison block 126 If the companson yields a match, then the packet is authorized to be transmitted from the network to a termmation device, as shown by block 128. Otherwise, the packet is not transmitted, as shown by block 130.

Fig. 24 illustrates an entry or mtermediate module 140 that includes a mapping table 142. Mappmg table 142 is used to assign prioπty to an mcommg packet forwarded from an upstream module or termmation device 144. The pπonty code is allocated based on the mcommg address of module/termmation device 144 and the bandwidth allocated to a section of the data flow path reserved for that particular packet Altematively, the pπonty code can be allocated based on the source address (SA) and/or destination address (DA) of a module or termination device. In the example shown, prioπty can be assigned to a subset of the network nodes based on the source address, destination address, structural classification, etc., based on the bandwidth allocated to the transmission branch. Each termination device can, for example, handle diffeπng pnonty mput based on the user connected to the termmation device. Pπonty can therefore be dependent on the user accessmg a termmation device, as well. Pπoπty can be assigned to any packet aπsmg from a specific source address or, altematively, from any packet ansmg from a particular user. Indeed, a packet is assigned pnonty based on numerous cπteπa, all of which fall within the spint of the packet-by-packet assignment herem. For example, the packet can be denved from one of N modules that is allocated a bandwidth (BW ALLOC.) of 1 0 MB/sec.

Shown m Fig. 25 is the inteπelationship between the allocated bandwidth and the pnonty code. More specifically, the forwardmg module (entry or mtermediate module) shown m Fig. 24 forwards a sequence of packets between a pair of counter resets 146 and 148. As the timing diagram 150 illustrates, a plurality of packets can be sent between resets at a fairly high transmission rate. When the count value of bits or packets achieves, for example, 1 0 Mbits at time 152, all packets pnor to that time and subsequent to reset 146 are transmitted at the prioπty code established withm mappmg table 142. However, all packets sent after time 152 are sent at a lower, default pnonty Tuning diagram 154 illustrates the opposite effect, whereby packets are sent at a relatively slow rate. In particular, the packets are sent at a rate slower than the allocated bandwidth, such that, for example, less than 1 0 Mbits are sent between resets and the remammg 1.0 Mbits sent after reset 148 are still sent at the assigned pnonty of mappmg table 142. The allocated bandwidth is therefore the maximum bandwidth that is allocated to the shared resource for the senes of packets sent through that resource. If the packets arnve at the shared resource faster than the allocated bandwidth, as shown by reference 150, then the remaining packets pnor to the next reset will be forwarded at a lower priority The lower pπoπty is signified as a lower pπoπty code set forth in each packet sent after 1.0 Mbits have been sent and before the next reset 148 Fig. 26 illustrates exemplary arbitration schemes for grantmg bandwidth resources among mcommg packets havmg dissimilar pnoπty codes In particular, timing diagram 160 may send X, number of packets at a relatively high pπonty before sendmg Y, number of packets at a relatively low pnonty In this manner, service is granted to the higher pnoπty packets (X) before that of the lower pnonty packets (Y) However, to prevent undue accumulation of lower pnoπty packets, the forwardmg module will grant transmission of lower pnoπty packets on a fixed or round-robin scheme. Possibly after a seπes by which higher pπonty packets are sent followed by lower pπoπty packets, even lower pπoπty number packets may be sent, those packets indicated as Z|, Z₂, and αi packets Reference 160 illustrates one of possibly numerous examples bv which a forwardmg module decides which packets to be sent across the shared resource dependmg on the pnoπty code assigned to those packets A relatively fair selection process should be used so as to evenly service mcommg packets with greater deference given to higher pπoπty packets If, for example, packets amve at the highest level of pπonty (N+M) as shown by dashed lme 162, then possibly only those packets and no other packets will be serviced by the forwardmg module In this case, the lower prioπty packets will merely accumulate on the forwarding module mput buffer until all of the highest pnonty packets hav e been serviced

A problem mav exit when forwardmg packets of varying lengths A rather long packet may take a longer time to transmit through a module after it has begun its transmission If the next packet is of higher pnonty, it may be necessary to hold of f (I e , suspend) transmission of the existmg, longer yet lower pπonty packet In this manner, stalling the higher priority packet is prevented Added to the trailer mformation of the cuπent transmitted packet may be pπonty information applicable to the following packet An mtermediate node or module can therefore use the trailer information while schedulmg the next packet If a long packet needs to be scheduled at a lower priority and the mcommg packet mdicates that there is a high pnonty packet to follow, the node can hold off sendmg the lower pnonty longer packet, mcludmg the cuπent packet Each mtermediate node can update the next-packet pnonty field of the trailer

Pnoπty code m the present structural network can be assigned on a packet-by-packet basis not only for the existing packet but possibly also for subsequent packets The pπonty code can be assigned based on the oπgmation address, destmation addresses, or on the structural configuration Thus a packet can be assigned a high pnonty withm its own level and a lower pnoπty at other levels This can be achieved by setting a bit m the control word notifymg a level change mtermediate module such as AI of Fig 12 to change the pπonty when traveling outside the loop Alternatively, pnonty code could be assigned with vanous fields, each field defining the pnoπty at a given level

As shown m Fig 27, each module withm a particular level may have a pπonty code unique to that level Also, as shown, prioπty code can be assigned different for differing users Packets denved from user VI may have a higher transmission pnoπty than packets denved for user V2 Similarly, packets sent to user Ul may be assigned a higher pnoπty than packets sent to user U2. etc It is noted that the hierarchical nature of pπonty codes, user dependency of pπoπty codes, and the source and destmation dependency of pπonty codes are also applicable to the security codes as descnbed above Thus, the example of Fig 27 is equally applicable to security codes The modules mdicated with a "F" pnonty designator are those possibly at the highest pπoπty level In the example shown, four bits may be all that is needed to mdicate all the vaπous levels of pπoπty Loop 164 is illustrative of a local loop havmg the highest pπonty level to ensure that any packets sent between modules on that loop will be given pnonty to any mcommg packets onto that loop This may provide for high speed video teleconferencmg among termmation devices connected on, for example, a LAN Loop 166 illustrates mtercoimected modules havmg dissimilar pπonty codes stored m the wrap portion of their buffers Any packets amving on those modules will be forwarded with the pπonty code of that module to the next module withm the loop or to an interconnected loop Loop 168 is shown havmg a high pnoπty module "F" that matches with high pnonty modules m loop 164 This signifies a high pnoπtv data path across the mtercoimected loops from each of those modules if communication is so desired Fig 28 illustrates a compiled packet 260 havmg vanous words aligned m sequence and forwarded from the read buffer across a transmission channel or network, also contammg at least one panty bit coπespondmg to eacn word of the packet Thus, packet 260 preferably includes a parity bit assigned on a word-by-word basis Beginning with the null pattern, the panty bit attributed to the null word produces an even panty. and all other words and associated parity bits, when combmed, produce an even panty value, accordmg to one example. All words within the data poπion, when combmed with the panty bit attnbuted to that data portion, produce an odd panty value, and moreover, all words withm the trailer, when combmed with their associated panty bits, produce an even panty Alternatively, mstead of the header and trailer producmg even panty and the data producmg odd panty, the header and trailer can produce odd panty and the data produce even panty Regardless of the panty chosen, it is prefeπed that the panty attπbuted to the header and trailer be different from the panty associated with the data. In this fashion, the header and trailer mformation transmitted (when combmed with the associated parity bit) is different from the data mformation transfeπed (when combmed with its associated panty bit) Thus, distinctions are readily made as to the transmitted mformation (i.e , whether that information is associated with the header, data, or trailer) This uniquely distinguishes any control pattern in non-data (null, header, trailer, etc ) from the same pattern withm data Smce the data is interposed between the header and trailer, once the data is identified separate from the header, logically speakmg, the trailer must follow the data. In one embodiment, the trailer can compπse of only null pattern

It is noted in reference to Fig 28. that the header, data, and trailer portions are several words long. The null patterns, start-of-packet, and end-of-packet designators can also be one or more words long. In the example shown, a word is greater than one byte, and preferably at least 4 bytes m length. If only a smgle panty bit is added to each word, then the additional overhead is 1/32 (assuming a word length is 32 bits). This is significantly smaller than the overhead mvolved m the 8b 10b format

Fig. 29 illustrates an example of an even (or odd) panty bit generator. The panty bit P is generated by addmg the number of 1 's associated with all bits of the word extendmg from bit 0 to bit N. In an even panty generator, if the summation forwarded through the modulo 2 adder 262 mdicate an even number 1 's, then panty bit P' will show a value of 0. However, if the number of l 's is odd, then panty bit P' will mdicate a 1 value. To formulate an odd panty bit generator, an inverter 64 can be employed Inverter 264 merely inverts the bit value of P' panty bit, as shown Thus, an even parity bit generator produces a 1 panty bit value if the summation is odd, while an odd parity bit generator produces a 1 panty bit value if the summation is even (or all 0's). The coπespondmg parity bit value will then be inserted with the transmitted word mto the transmission medium as designated by the panty bit field shown m Fig. 28. Fig. 30 illustrates various ways m which to detect the number of words associated with the header, data, and trailer portions of a packet and, alternatively, or m addition thereto, detect eπors and coπect those eπors if the parity bit (S) mdicate a transmission eπor. Specifically, Fig. 30 illustrates, m block diagram form, counters and panty detectors coupled to receive the word mformation and associated panty bits received at the output of the transmission channel. Accordmgly, the block diagram of detectors and counters, noted as reference numeral 270 can form a part of the decoder and parity-strip logic 36c, shown m Fig 6. In essence, item 270 can be used to detect the beginning of each portion withm a packet, either by counting the number of fixed words withm the header, data, and trailer portions, or detecting the panty value of the received words and panty bits, and comparmg those values agamst either an even or odd parity scheme. In the former mstance, counters 272a, 272b, and 272c can be used to count a fixed number of words in each respective portion, wherem counter 272a counts the number of words withm the start-of-packet designator, counter 272b counts the number of words withm the header, and counter 272c counts the number of words withm the data, and so forth. Counters 272a, 272b, and 272c can physically be implemented in the same counter with vaπous count values thus logically mimicking three counters Knowing that the start-of-packet designator is defined to be one word m length, then counter 272a will count that smgle word to mdicate that the next word is the beginning of the header after the start-of-packet designator. Counter 272b then counts the number of words withm the remainder of that header to note the initial word of data, wherem counter 272c will then count all the words withm data to signify the beginning of the end-of-packet designator. S ce the header data and trailer portions have a known number of words, counters can adequately be used to demarcate words m one portion from words m another portion so as to prevent the problem of confusion even though data may have the same bit pattern as, for example, the start-of-packet designator. Alternatively, a smgle panty detector 274 can receive the stream of words and panty bits, and after performmg addition operations, determme (based on whether the panty value is even or odd) when a word anses from each of the packet portions. For example, if detector 274 detects even panty and then detects odd panty, it is noted that the coπespondmg word havmg odd panty is the first word of the data portion.

Unfortunately, eπors may occur m the transmission medium, whereby the authenticity of the even or odd detected panty value cannot be relied upon. In those mstances, a combination of counters and panty detectors can be employed For example, counter 272a can count once after the null pattern to note the end-of- packet designator has been transfeπed, and the next word is the beginning word of the header. Detect logic 274 can then be configured simply to confirm that the panty value of each and every ensumg word withm the header is of even panty value If not, then the words which have an odd panty are marked as eπor and possibly re- transmitted. The same can be applied to odd panty detect logic 276 which is used to confirm an odd parity value beginning with the first data word received after counter 272b counts the predefined and fixed number of header words. Detect logic 276 will know that all ensumg data words will have a coπespondmg odd parity, unless an eπor occurs m transmission.

Fig. 30 also illustrates detect logic 274 and 276 as possibly havmg an eπor coπection feature. Provided the number of parity bits is sufficient to not only detect, but coπect, a defined word length, any errors discovered m the transmission of a word can be coπected.

Given that the transmission channel may be an optical fiber, it is important m many mstances to peπodically strobe light mto that fiber m order to maintain synchronization of the timing circuitry at the receive end of the fiber. Accordmgly, it is desirable that the null pattern not contam all 1 's or all 0's Instead, the null pattern can have an mtermittent 1 and 0 pattern m order to strobe the opto-electromc receivers and thereby synchronize the clock recovery circuit connected thereto. The same can be said about the start-of-packet and end-of-packet designators Similar to any pattern of mformation forwarded in the header and trailer, these designators should not contam all 1 's or all 0's and, of course, should not have the same bit value as the null pattern. Instead, the designators should have a relatively small groupmg of 1 's intermixed as part of the overall word length. For example, the null pattern can be 101010 .., while the start-of-packet and end-of-packet designators can be 111000. . and 00011 1..., respectively. Similarly, if the data is usmg even panty, it is guaranteed to have at least a panty bit that has a value of 1 once every word This helps m synchronization when data has consecutive 0 values. In one embodiment, item 270 can have another counter that keeps counting the number of words (or bits) between a start of packet and next null pattern. If it does not recognize a null pattern for over a time penod required to transmit the largest packet allowed, it generates an eπor condition. This counter can be used to detect stuck-at-one eπor for data usmg even panty For data usmg odd panty, the two mechanisms above can be used to detect the reverse condition

A receivmg module can detect severance of a channel When the encoder assigned to the particular mput port of a traffic manager detects such a severance, the encoder will apply an appropnate field of control bits onto the header of a received packet based on what had been programmed mto its configuration register. Thus, if an eπor is to be programmed, then the configuration register of the appropnate mput port encoder may be programmed with a "01" value, as shown by the control bit field 362 of Fig 32 However, there may be instances m which an eπor is not detected (i.e., a downstream disturbance does not exist), then the configuration register will be programmed to impute a "00" bit field onto the returned packet, as shown by the control bit field 362 of Fig. 32. The control bit field 362 can also be reserved to mdicate an acknowledgement of eπor set by the oπgmating module after havmg received back the packet contammg the eπor value "01" from two receivmg modules placed immediately upstream two disturbed or severed paths The acknowledge bit value "10" will note that the ongmatmg module should not attempt to re-send the packet smce all alternative paths are severed and, instead, should simply drop the packet The ongmatmg modules drops the packet upon setting the control

Fig. 31 illustrates an eπor identification bit field contamed withm the trailer portion of the packet 360, accordmg to one example The eπor identification bit field is merely reserved to be programmed based upon the module identification number of that receivmg module immediately upstream of the disturbed (i.e., corrupted, disabled, or severed) transmission path, or upstream of the dismrbed transmission channel. Thus, the configured identification number withm module E, of the example shown m Fig 5, will be placed withm the error identification bit field shown m Fig 31 as the packet is returned to the oπgmating module A In this manner, the ongmatmg module A will know that the packet can be resent to all modules downstream of the receivmg module, but should be resent m a counter-clockwise fashion, rather than the exemplary clockwise route. The ongmatmg module, like all modules m the structured, hierarchical network, knows not only its identification number, but all other module identification numbers withm the network Therefore, the ongmatmg module A will know the relative placement of all other modules connected to that loop, or across any loop withm the network

Similarly, control word of a header shown m Fig. 31 can have one or more bit used to broadcast removal of severance mformation Associated with this bit, trailer of the packet can have eπor removal identification number that notifies other modules to remove specific severance

In the example of Fig 5, a loop is shown However, it is understood that the present packet protocol, system and method can also be applied to a star or mesh topology. Fig 33 illustrates a star topology m solid line, and mesh by a combination of solid and dashed lmes, wherem altemative channels exist between modules to form a transmission path between modules. For example a channel between modules E and C can be disturbed, as shown by reference number 364 The transmission path between modules A and C can therefore be redundantly shown as primary channel A to E to C, or as secondary channel A to C An eπor can be detected by receivmg module E , and the appropnate control and eπor identification bits are set and returned to ongmatmg module A Module A can then re-send the packet to module C via the altemative, direct channel between modules A and C If, for example the direct channel is also disturbed, then ongmatmg module can set the acknowledge bit value and send the packet on any other alternative paths such as A to D to C or A to D to B to C. If this case, the number of eπor bits will be a function of number of alternate paths If all paths are severed only then the packet will be dropped The same principles that apply to a πng topology are therefore equally applicable to the star or mesh topology, except that the pair of channels which for a transmission path are separated m a star or mesh topology, whereas they generally extend adjacent and parallel to one another in a πng topology. In an example by which a πng topography is provided, Fig. 34 is shown Indicated m Fig. 34 are multiple modules 16, labeled "A"-"F" are attached to and communicate with two transmission channels 322a and 322b that form a smgle transmission path 324 The dual transmission channels can be thought of as a redundant system, however, not as m the conventional sense Instead, the secondary (or redundant) transmission channel 322b can be used if path 322a is severed. Importantly, however, both paths can be used to send dissimilar packets m order to mcrease two- fold the overall bandwidth of the network. Thus, even though node 12 is represented as a smgle loop, each loop withm the network can also have redundant transmission channels. Similarly even though node 12 is represented as a loop, it can be configured m star or mesh or any other topologies.

In one embodiment, channel 322a can be assumed to be used for data transmission m a counter- clockwise data transmission direction, while channel 322b is used for clockwise data transmission. Thus, any given module will be receivmg data from both of its neighbors at all times. In the absence of data, null pattern will be present m both transmission channels. If a given module does not detect null pattern m any one of the mcommg transmission channel for a time exceedmg the maximum packet size allowed m a given protocol or a preset maximum time from any direction, the module may detect a severance m the transmission channel associated with that direction. A severance can also be detected when any predefined pattern is not detected for a set time, or upon notification of a severance.

Fig. 34 illustrates one example by which the transmission path 324 can be severed at location 326, yet transmission is maintained. In this example, communication from module A to module C will detect the severance at location 326 by module E employing a loop-back path of a packet sent across transmission channel 322b to the destmation module C In one embodiment, modules are designed to detect severance based on absence of null pattern for a preset time, with ongmatmg module A forwardmg the packet across transmission channel 322b, and the receivmg module E pre-conditioned as not receivmg the requisite null pattern of Is and 0s for a time duration that exceeds the time needed to transfer maximum size packet. A null pattern is defined as a pattern normally sent across the network whenever a packet is not bemg sent. Preferably, the null pattern contams some combination of altematmg Is and 0s m order to sync the clocks m the receiver circuits of the modules. If the pattern forwarded between a parr of modules separated by disturbance contams a pattern not recognizable as the pre-defined null pattern (i.e., the pattern is all Is or all 0s) for a duration that exceeds the maximum length of a packet, then it is known that the pair of modules straddle a channel that has been disrupted or severed. For example, a transmitter will normally send alternating Is and 0s (i.e., transmit a null pattern) whenever a packet is not sent However, if the channel is severed, then the expected alternating Is and 0s might become all Is or all 0s, indicating an eπor in that channel. As such, m the example shown, receivmg module E will return the control bits to the oπgmating module A, indicating a downstream eπor. In yet another embodiment, module E can detect severance based on other preset condition or by introduction of severance from external source such as network admmistrator. The packet received by receivmg module E is then returned via feedback path 328 to the ongmatmg module A. Smce receivmg module E was the last module m the path to have detected a severance (i.e., an error downstream thereto), receivmg module E returns with the packet both control bits and eπor identification bits. The control bits are set to indicate a disruption immediately downstream of receivmg module E The eπor identification bits identify the receivmg module E by the identification number assigned to that module. Therefore, the remmed packet contams not only indication of a disrupted transmission path, but also the module immediately upstream of that severance. Accordmgly, usmg a feedback path and appending control bits and eπor identification bits to the packet, allows the present system, packet protocol, and methodology to identify where, withm the communication network, a severance exists

After receivmg the remmed packet contammg control bits and eπor identification bit values, ongmatmg module A sets an acknowledge value withm the control bit field The acknowledge value with note that the packet can be re-sent, however, the re-sent packet must be forwarded across an alternative transmission path. This will avoid ongmatmg module A from re-sendmg the packet to destmation module C via receivmg module E Instead, the packet can be re-sent along the πght path via modules F, D, and B. Thus, the present methodology tolerates a severance of both the primary and secondary (redundant) transmission channels m a smgle location Any further transmission is merely sent m the direction opposite the severance. Advantageously, smce the underlying network is a structured network, entire network can be made aware of the location of any severance. In the example, when module E detects severance 326, it mcludes its identification number signifying other modules that it is the module immediately downstream of the severance. When a packet is looped back, this information of the severance and its location will be transmitted to other modules m the network. Module A will be made aware of disturbance/severance 326 by the looped back packet from module E Module A will then send the re-routed packet through the other transmission path 322a along with the information of severance and its location Modules F, D, and B will pass this packet through while learning about the severance 326. Thus all modules m the network will be made aware of the severance 326 and its location. All modules use this information during routing of all future packets. For example, after bemg cognizant of the severance, module F will route packets destined to module C via transmission channel 322a and not transmission channel 322b. Packets destined for Module E will be routed through transmission path 322b and not transmission path 322a.

Due to the fact that entire networks can be made aware of any dismrbance or severance and its location in the network, packet re-routing is reduced considerably mcreasmg overall bandwidth utilization and reducing network flooding of rerouted packets Packets need to be rerouted only until modules are made aware of the severance. This will generally be limited to initial few packets immediately following the severance. Packets onginating from a module will be directed to cross a severance (situations when packets are directed to module C or beyond on transmission path 322b or packets are directed to module E or beyond on transmission path 322a) only until the first loop back (or re-routed) packet reaches that module. For mstance, a re-routed packet from module A is bemg directed to module C will inform mtermediate modules F, D, B, and C about severance 326 and its location immediately upstream of module E. These modules will use this mformation for all future routmg purposes.

Upon fixing or removal of the severance, this information will be transmitted to all nodes and will be reflected m their routing algonthms. The module immediately upstream of the severance will recognize removal of severance When severance 326 is removed, module E will recognize it for transmission path 322a and will notify other modules upstream Similarly, module C will recognize and notify all other upstream modules for transmission path 322b. A network with t o transmission paths similar to the one in Figure 34, can tolerate one severance anywhere m the network without service disruption Networks with N complete transmission paths can tolerate up to N-1 severances with no service disruption If a network with two transmission paths has more than one severance at any given time, it can still function partially For instance, if the network m Fig 3 has two severances, severance 326 and severance 330, it can still function partially If module A needs to route a packet to module C and chooses transmission path 322b Module E will loop back the packet due to severance 326. Module A will re-route the packet along transmission path 322a When the packet reaches module D, module D will loop back the packet along transmission path 322b due to severance 330 with the indication that the packet has been re-routed from the second severance along with the identification number of module D as the module downstream from the severance for transmission path 322a When this packet reaches module A, it will be dropped as unde verable packet as it was re-routed on both available transmission paths This dropping operation will enable the system to not flood the network with un-routable packets Durmg the re-routing process modules E, A, F, and D will learn that they are isolated from modules C and B However, transmission between modules E, A, F, and D will continue Similarly, modules C and B can communicate with each other but not with modules E, A, F, and D

Thus with two severances 326 and 330 m a nng topology, the present protocol can detect those locations, but cannot reroute to avoid the severed transmission paths whenever reroutmg requires transmission between modules separated by one severance of the pair of severances. If a severance is not interposed between the ongmation and destination modules, then transmission can occur m its normal course, where non-duphcative packets are sent across both channels to maximize the overall transmission bandwidth.

The same mechanism used to isolate and work around severance of transmission paths can be extended to modules for troubleshooting, maintenance, reconfiguration, and upgrading For instance, module C can be isolated and brought off-line by mducmg severance between modules C and E as well as modules C and B. After the maintenance, module C will be brought onlme by informing other modules of removal of severance. The mechanism by which the feedback path is selectively coupled between redundant channels, and the protocol used to assign module identification numbers to each module is descπbed m reference to the overall module design shown m Fig 6 A localized configuration bus or channel may be coupled to vanous configuration registers contamed withm decoders/encoders of the traffic manager The traffic manager serves numerous functions, one of which is to optimally route packets of information between mput and output ports associated with that traffic manager Each traffic manager can be configured durmg assembly of the network. Reconfiguration of traffic managers can take place if, for example, traffic congestion is discovered m a select region of the network The traffic manager mcludes one or more buffers as wells as one or more decoders, and each port is shown as a bi-directional port which can direct, bypass, or feedback data sent among modules. It will be appreciated to those skilled m the art havmg the benefit of this disclosure that the vanous embodiments herem are believed to be capable of performmg fast and efficient transfers across a stmctured network which either replaces or forms at least a portion of a conventional communication system Vanous modification and changes may be made as would be obvious to a person skilled m the art havmg the benefit of this disclosure. It is mtended that the following claims be interpreted to embrace all such modifications and changes. Accordmgly, the specification and drawings are to be regarded m an illustrative rather than restπctive sense.

Claims

WHAT IS CLAIMED IS:

1. A communication network, compnsmg a plurality of interconnect nodes, each of which perform only a portion of a distributed routmg operation that extends across the communication network.

2. The communication network as recited m claim 1 , wherem each of the plurality of mtercoimected nodes is assigned a level, and said distnbuted routing operation occurs on a level-by-level basis, and each level is assigned based on where, withm the network, each of the respective nodes is topologically located.

3 The communication network as recited m claim 1 , wherem each of the plurality of interconnect nodes compnse a set of interconnected modules that has associated identification numbers bifurcated mto several groupmg of bits, wherem a first groupmg of said several groupmgs of bits identifies a first module for receivmg data of the network pnor to a second groupmg of said several groupings of bits identifying a second module for receivmg the data from the first module

4. The communication network as recited m claim 3, wherem the first module is grouped withm a first hierarchical level with a plurality of first switches havmg respective dissimilar bits withm the first groupmg of bits.

5 The communication network as recited m claim 3, wherem a decoder withm the first module is adapted to compare an address of the data with the first groupmg of bits if said address ongmates from a module within a hierarchical level dissimilar from the first hierarchical level.

6. The commumcation network as recited m claim 3, wherem the set of modules are interconnected across a portion of the commumcation network with a smgle control processor havmg a local bus extending between the control processor and each of the set of mtercoimected modules

7. The communication network as recited m claim 3, wherem the identification numbers associated with said mtercoimected modules internal to the communication network pomt the data through the commumcation network absent a look-up operation by any of the mtercoimected modules mtemal to the communication network

8 The communication network as recited m claim 3, wherem the identification numbers of each of the mtercoimected modules pomt the data absent a look-up operation by any of the plurality of modules mtemal to the network.

9. The communication network as recited m claim 1 , wherem the plurality of mtercoimected nodes compnses a set of mtercoimected modules, some of which compnse a traffic manager which dispatches a seπes of read operations to a memory coupled withm a data flow path, and wherem the memory mcludes a source address and a destination address of a pair of said plurality of mtercoimected nodes routably coupled withm the data flow path.

10. The commumcation network as recited m claim 1 , wherem the memory is adapted for temporanly storing said data which traverses the communication network, said data is temporanly stored m one section of the memory and control mformation applicable to said data is stored m another section of the memory.

1 1. The communication network as recited m claim 1 , further compnsmg a decoder placed withm a data flow path of the communication network for decodmg a destination address of a node at an exit pomt of the communication network.

12. The communication network as recited m claim 1, further compnsmg an entry module which compnses: a decoder coupled to receive an identification number of a node at an entry pomt mto the communication network; a storage device configured with a set of bits; and a compare u t coupled between the decoder and the storage device for comparmg the identification number with the set of bits to determme if the data will be transfeπed as secured data to the communication network.

13. The commumcation network as recited m claim 12, further compnsmg a counter coupled between the decoder and the storage device for counting the number of accesses upon the entry module by the entry device and blockmg further accesses if a count value resultmg from said counting exceeds a value stored within the register.

14. The communication network as recited m claim 12, further compnsmg a mappmg table coupled to map the identification number to a coπespondmg secuπty code.

15. The communication network as recited m claim 1, further compnsmg an exit module which compnses: an output port adapted to receive a packet of data havmg a header compnsmg a secunty code and an identification number, a storage device compnsmg at least one mappmg table; and a compare unit coupled between the output port and the storage device for comparmg the identification number with entnes withm the table to determine if the identification number deπves from a secured path that extends at least partially through the communication network.

16. The communication network as recited m claim 15, wherem the storage device compnses a plurality of mappmg tables, one of which is selected by the secunty code.

17. The communication network as recited m claim 1, further compnsmg. at least one intermediate module linked between an entry module and an exit module of the communication network; wherem said entry module is coupled to assign and transfer a secunty code and an identification number to a packet of data; and an exit compare unit withm the exit module coupled to compare the secunty code and the identification number before transferring the packet of data from the communication network

18. The communication network as recited m claim 1, further compnsmg a forwardmg module withm one or more of the plurality of mtercoimected nodes, said forwardmg module havmg means for assigning a pπoπty code withm a packet forwarded mto the network, and wherem the pπoπty code withm the mcommg packet is assigned to a packet which follows the mcommg packet for pπontizmg a packet which follows the mcommg packet.

19. The communication network as recited m claim 1, further compnsmg: a first forwardmg module coupled to receive a first packet of data and assign a first pπonty code to the first packet of data; a second forwardmg module coupled to receive a second packet of data and assign a second pnoπty code to the second packet of data; and an arbiter coupled to forward the first packet of data across a portion of the commumcation network mstead of the second packet of data if the first pnonty code is higher m pnonty than the second pnonty code.

20. The communication network as recited m claim 19, wherem the address from which the first and second packets of data is sent coπesponds to an identification code of a forwardmg module simated at a umque structural level withm the communication network

21. The communication network as recited m claim 1, further compnsmg a packet of mformation to be sent across a communication network, said packet compnsmg. data interposed between a header and a trailer; and a panty bit assigned to each word of the data, header and trailer, wherem the panty bit when added to a coπespondmg word of the header or trailer produces a value that is dissimilar from the panty bit when added to a coπespondmg word of the data.

22. The communication network as recited claim 1, further compnsmg: a transmission channel; a panty generator adapted to generate a panty bit coπespondmg to each word of a header, trailer and data withm a packet, wherem the panty bit, when added to coπespondmg words of the header or the trailer, produces a panty value dissimilar from the panty bit added to coπesponding words of the data; and a panty detector adapted to receive the header, trailer, data and generated panty forwarded from the panty generator across the transmission channel, said panty detector is coupled to detect the panty value and determine a coπespondmg demarcation between the header, trailer and data.

23. The communication network as recited m claim 1. further compnsmg a packet of information sent across the communication network, said packet compnsmg control bits and eπor identification bits that mdicate a transmission eπor-causmg dismrbance and where, withm the communication network, the transmission eπor- causmg disturbance exists.

24. The commumcation network as recited m claim 1, further compnsmg at least two transmission channels and a receivmg module connected to the transmission channels, said receivmg module havmg a loop- back conductor which operably connects the two transmission channels and retums a packet contammg eπor bits if one or more of the transmission channels downstream of the receivmg module is dismrbed.