US20010034798A1 - Multiple level minimum logic network - Google Patents
Multiple level minimum logic network Download PDFInfo
- Publication number
- US20010034798A1 US20010034798A1 US09/850,953 US85095301A US2001034798A1 US 20010034798 A1 US20010034798 A1 US 20010034798A1 US 85095301 A US85095301 A US 85095301A US 2001034798 A1 US2001034798 A1 US 2001034798A1
- Authority
- US
- United States
- Prior art keywords
- level
- node
- message
- nodes
- destination
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F15/00—Digital computers in general; Data processing equipment in general
- G06F15/16—Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F15/00—Digital computers in general; Data processing equipment in general
- G06F15/16—Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs
- G06F15/163—Interprocessor communication
- G06F15/173—Interprocessor communication using an interconnection network, e.g. matrix, shuffle, pyramid, star, snowflake
- G06F15/17356—Indirect interconnection networks
- G06F15/17368—Indirect interconnection networks non hierarchical topologies
- G06F15/17393—Indirect interconnection networks non hierarchical topologies having multistage networks, e.g. broadcasting scattering, gathering, hot spot contention, combining/decombining
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F15/00—Digital computers in general; Data processing equipment in general
- G06F15/16—Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs
- G06F15/163—Interprocessor communication
- G06F15/173—Interprocessor communication using an interconnection network, e.g. matrix, shuffle, pyramid, star, snowflake
- G06F15/17356—Indirect interconnection networks
- G06F15/17368—Indirect interconnection networks non hierarchical topologies
- G06F15/17375—One dimensional, e.g. linear array, ring
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F15/00—Digital computers in general; Data processing equipment in general
- G06F15/76—Architectures of general purpose stored program computers
- G06F15/80—Architectures of general purpose stored program computers comprising an array of processing units with common control, e.g. single instruction multiple data processors
- G06F15/8007—Architectures of general purpose stored program computers comprising an array of processing units with common control, e.g. single instruction multiple data processors single instruction multiple data [SIMD] multiprocessors
- G06F15/803—Three-dimensional arrays or hypercubes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/28—Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
- H04L12/42—Loop networks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L45/00—Routing or path finding of packets in data switching networks
- H04L45/02—Topology update or discovery
- H04L45/06—Deflection routing, e.g. hot-potato routing
Abstract
A network or interconnect structure utilizes a data flow technique that is based on timing and positioning of messages communicating through the interconnect structure. Switching control is distributed throughout multiple nodes in the structure so that a supervisory controller providing a global control function and complex logic structures are avoided. The interconnect structure operates as a “deflection” or “hot potato” system in which processing and storage overhead at each node is minimized. Elimination of a global controller and buffering at the nodes greatly reduces the amount of control and logic structures in the interconnect structure, simplifying overall control components and network interconnect components and improving speed performance of message communication.
Description
- The present invention relates to interconnection structures for computing and communication systems. More specifically, the present invention relates to multiple level interconnection structures in which control and logic circuits are minimized.
- Many advanced computing systems, including supercomputers for example, utilize multiple computational units to improve performance in what is called a parallel system. The system of interconnections among parallel computational units is an important characteristic for determining performance. One technique for interconnecting parallel computational units involves construction of a communication network similar to a telephone network in which groups of network elements are connected to switching systems. The switching systems are interconnected in a hierarchical manner so that any switching station manages a workable number of connections.
- One disadvantage of a network connection is an increase in the latency of access to another computational unit since transmission of a message traverses several stages of a network. Typically, periods of peak activity occur in which the network is saturated with numerous messages so that many messages simultaneously contend for the use of a switching station. Various network types have been devised with goals of reducing congestion, improving transmission speed and achieving a reasonable cost. These goals are typically attained by rapidly communicating between nodes and minimizing the number of interconnections that a node must support.
- One conventional interconnection scheme is a ring of nodes with each node connected to two other nodes so that the line of interconnections forms a circle. The definition of a ring, in accordance with a standard definition of a ring network in the art of computing (IBM Dictionary of Computing, McDaniel G. ed., McGraw-Hill, Inc., 1994, p. 584) is a network configuration in which devices are connected by unidirectional transmission links to form a closed path. Another simple conventional scheme is a mesh in which each node is connected to its four nearest neighbors. The ring and mesh techniques advantageously limit the number of interconnections supported by a node. Unfortunately, the ring and mesh networks typically are plagued by lengthy delays in message communication since the number of nodes traversed in sending a message from one node to another may be quite large. These lengthy delays commonly cause a computational unit to remain idle awaiting a message in transit to the unit.
- The earliest networks, generally beginning with telephone networks, utilize circuit switching in which each message is routed through the network along a dedicated path that is reserved for the duration of the communication analogous to a direct connection via a single circuit between the communicating parties. Circuit switching disadvantageously requires a lengthy setup time. Such delays are intolerable during the short and quick exchanges that take place between different computational units. Furthermore, a dedicated pathway is very wasteful of system bandwidth. One technique for solving the problems arising using circuit switching is called packet switching in which messages sent from one computational unit to another does not travel in a continuous stream to a dedicated circuit. Instead, each computational unit is connected to a node that subdivides messages into a sequence of data packets. A message contains an arbitrary sequence of binary digits that are preceded by addressing information. The length of the entire message is limited to a defined maximum length. A “header” containing at least the destination address and a sequence number is attached to each packet, and the packets are sent across the network. Addresses are read and packets are delivered within a fraction of a second. No circuit setup delay is imposed because no circuit is set up. System bandwidth is not wasted since there is no individual connection between two computational units. However, a small portion of the communication capacity is used for routing information, headers and other control information. When communication advances in isolated, short bursts, packet switching more efficiently utilizes network capacity. Because no transmission capacity is specifically reserved for an individual computational unit, time gaps between packets are filled with packets from other users. Packet switching implements a type of distributed multiplexing system by enabling all users to share lines on the, network continuously.
- Advances in technology result in improvement in computer system performance. However, the manner in which these technological advances are implemented will greatly determine the extent of improvement in performance. For example, performance improvements arising from completely optical computing strongly depend on an interconnection scheme that best exploits the advantages of optical technology.
- In accordance with the present invention, a multiple level minimum logic network interconnect structure has a very high bandwidth and low latency. Control of interconnect structure switching is distributed throughout multiple nodes in the structure so that a supervisory controller providing a global control function is not necessary. A global control function is eliminated and complex logic structures are avoided by a novel data flow technique that is based on timing and positioning of messages communicating through the interconnect structure. Furthermore, the interconnect structure implements a “deflection” or “hot potato” design in which processing and storage overhead at each node is minimized by routing a message packet through an additional output port rather than holding the packet until a desired output port is available. Accordingly, the usage of buffers at the nodes is eliminated. Elimination of a global controller and buffering at the nodes greatly reduces the amount of control and logic structures in the interconnect structure, simplifying overall control components and network interconnect components, improving speed performance of message communication and potentially reducing interconnection costs substantially. Implementation of the interconnect structure is highly flexible so that fully electronic, fully optical and mixed electronic-optical embodiments are achieved. An implementation using all optical switches is facilitated by nodes exploiting uniquely simple logic and elimination of buffering at the nodes.
- The multiple level minimum logic network interconnect architecture is used for various purposes. For example, in some embodiments the architecture is used as an interconnect structure for a massively parallel computer such as a supercomputer. In other exemplary embodiments, the architecture forms an interconnect structure linking a group of workstations, computers, terminals, ATM machines, elements of a national flight control system and the like. Another usage is an interconnect structure in various telecommunications applications or an interconnect structure for numerous schedulers operating in a business main frame.
- In accordance with one aspect of the present invention, an interconnect apparatus includes a plurality of nodes and a plurality of interconnect lines selectively connecting the nodes in a multiple level structure in which the levels include a richly interconnected collection of rings. The multiple level structure includes a plurality of J+1 levels in a hierarchy of levels and a plurality of 2JK nodes at each level. If integer K is an odd number, the nodes on a level M are situated on 2J-M rings with each ring including 2MK nodes. Message data leaves the interconnect structure from nodes on a level zero. Each node has multiple communication terminals. Some are message data input and output terminals. Others are control input and output terminals. For example, a node A on
level 0, the innermost level, receives message data from a node B onlevel 0 and also receives message data from a node C onlevel 1. Node A sends message data to a node D onlevel 0 and also sends message data to a device E that is typically outside the interconnect structure. One example of a device E is an input buffer of a computational unit. Node A receives a control input signal from a device F which is commonly outside the interconnect structure. An example of a device F is an output buffer of a computational unit. Node A sends a control signal to a node G onlevel 1. - All message data enters the interconnect structure on an outermost level J. For example, a node A on level J, the outermost level, receives message data from a node B on level J and also receives message data from a device C that is outside the interconnect structure. One example of device C is an output buffer of a computational unit. Node A sends message data to a node D on level J and also sends message data to a node E on level J−1. Node A receives a control input signal from a node F on level J−1. Node A sends a control signal to a device G that is typically outside the interconnect structure. An example of a device G is an output buffer of a computational unit.
- Nodes between the
innermost level 0 and the outermost level J communicate message data and control signals among other nodes. For example, a node A on a level T that is neitherlevel 0 or level J receives message data from a node B on level T and also receives message data from a node C onlevel T+ 1. Node A sends message data to a node D on level T and also sends message data to a node E on level T−1. Node A receives a control input signal from a node F on level T−1. Node A sends a control signal to a node G onlevel T+ 1. - Level M has 2J-M rings, each containing 2MK nodes for a total of 2JK nodes on level M. Specifically:
-
Level 0 has 2J rings, each containing 20K=K nodes for a total of 2JK nodes onlevel 0. -
Level 1 has 2J−1 rings, each containing 21K=2K nodes for a total of 2JK nodes onlevel 1. -
Level 2 has 2J−2 rings, each containing 22K=4K nodes for a total of 2JK nodes on level M. - Level J−2 has 2J-(J−2)=4 rings, each containing 2(J−2)K nodes for a total of 2JK nodes on level J−2.
- Level J−1 has 2(J-(J−1)=2 rings, each containing 2(J−1)K nodes for a total of 2JK nodes on level J−1.
- Level J has 2J-J32 1 ring containing 2(J−1)K nodes for a total of 2JK nodes on level J.
- For a ring RT on a level T which is not the outermost level J, then one ring RT+1 on level T+1 exists such that each node A on ring RT receives data from a node B on ring RT and a node C on ring RT+1. For a ring RT on a level T which is not the
innermost level 0, then there exist exactly two rings R1 T−1 and R2 T−1 on level T−1 such that a node A on ring RT sends message data to a node D on ring RT and a node E on either ring R1 T−1 or ring R2 T−1. A message on any level M of the interconnect structure can travel to two of the rings on level M−1 and is eventually able to travel to 2M of the rings onlevel 0. - In the following discussion a “predecessor” of a node sends message data to that node. An “timmediate predecessor” sends message data to a node on the same ring. A “successor” of a node receives message data from that node. An “immediate successor” receives message data to a node on the same ring.
- For a node ART on ring RT on level T, there are nodes BRT and DRT on ring RT of level T such that node BRT is an immediate predecessor of node ART and node DRT is an immediate successor of node ART. Node ART receives message data from node BRT and sends message data to node DRT. Node ART receives message data from a device C that is not on the ring RT and sends data to a device E that is not on ring RT. If the level is not the
innermost level 0, then device E is a node on level T−1 and there is an immediate predecessor node F on the same ring as device E. Node ART receives control information from device F. If node ART is on node T equal to zero, then device E is outside the interconnect structure and device E sends control information to node ART. For example, if device E is an input buffer of a computational unit, then the control information from device E to node ART indicates to node ART whether device E is ready to receive message data from node ART. Node DRT receives message data from a device G that is not on ring RT. Node ART sends a control signal to device G. - Control information is conveyed to resolve data transmission conflicts in the interconnect structure. Each node is a successor to a node on the adjacent outer level and an immediate successor to a node on the same level. Message data from the immediate successor has priority. Control information is send from nodes on a level to nodes on the adjacent outer level to warn of impending conflicts.
- When the levels are evenly spaced and the nodes on each ring and each level are evenly spaced, the interconnect structure forms a three-dimensional cylindrical structure. The interconnect structure is fully defined by designating the interconnections for each node A of each level T to devices or nodes B, C, D, E, F and G. Each node or device has a location designated in three-dimensional cylindrical coordinates (r, θ, z) where radius r is an integer which specifies the cylinder number from 0 to J, angle θ is an integer multiple of 2π/K, which specifies the spacing of nodes around the circular cross-section of a cylinder from 0 to K−1, and height z is a binary integer which specifies distance along the z-axis from 0 to 2J−1. Height z is expressed as a binary number because the interconnection between nodes in the z-dimension is most easily described as a binary digit manipulation. On the
innermost level 0, one ring is spanned in one pass through the angles θ from 0 to K−1 and each height z designates a ring. Onlevel 1, one ring is spanned in two passes through the angles θ and two heights z are used to designate one ring. The ring structure proceeds in this manner through the outermost ring J in which one ring is spanned in all 2J heights along the z-axis. - Node A on a ring R receives message data from a node B, which is an immediate predecessor of node A on ring R. For a node A located at a node position N(r,θ,z), node B is positioned at N(r,(θ−1)mod K,H,(z)) on level r. (θ−1)mod K is equal K when θ is equal to 0 and equal to θ−1 otherwise. The conversion of z to Hr(z) on a level r is described for z=[
z J−1, zJ−2, . . , zr, zr−1, . . . , z2, z1, z0] by reversing the order of low-order z bits from zr−1 to z0] into the form z=[z J−1, zJ−2, . . . , z1, z0, z1, z2, . . . , zr−1], subtracting one (modulus 2 1) and reversing back the modified low-order z bits. - Node A also receives message data from a device C which is not on level r. If node A is positioned on the outermost level r=J, then device C is outside of the interconnect structure. If node A is not positioned on the outermost level, then device C is a node located at position N(r+1,(θ−1)mod K,z) on
level r+ 1. - Node A sends message data to a node D, which is an immediate successor to node A on ring R. Node D is located at node position N(r,(θ+1)mod K,h(z)) on level r. (θ+1)mod K is equal 0 when θ is equal to K−1 and equal to θ+1 otherwise. The conversion of z to hr(z) on a level r is described for z=[zJ−1, zJ−2, . . . , z1, zr−1, . . . z2, z1, z0] by reversing the order of low-order z bits from zr−1 to z0] into the form z=[zJ−1, zJ−2, . . . , zr, z0, z1, z2, . . . , zr−1], adding one (modulus 21) and reversing back the low-order z bits.
- Node A also sends message data to a device E that is not on the same level r as node A. If node A is on the innermost level r=0, node A(r,θ,z) is interconnected with a device (e.g. a computational unit) outside of the interconnect structure. Otherwise, node A is interconnected to send message data to device E, which is a node located at node position N(r−1, (θ+1)mod K, z) on level r−1.
- Node A receives control information from a device F. If node A is on the innermost level r=0, the device F is the same as device E. If node A is not on the innermost level, device F is a node which is distinct from the device E. Node F is located at node position N(r−1,θ,Hr−1(z)) on level r−1.
- Node A sends control information to a device G. If node A is on the outermost level r=J, then device G is positioned outside of the interconnect structure. Device G is a device, for example a computational unit, that sends message data to node D. If node A is not positioned on level r=J, then device G is a node which is located at node position N(r+1,θ,hr+1(z)) on level r+1 and device G sends message data to node D.
- In accordance with a second aspect of the present invention, a method is shown of transmitting a message from a node N to a target destination in a first, a second and a third dimension of three dimensions in an interconnect structure arranged as a plurality of nodes in a topology of the three dimensions. The method includes the steps of determining whether a node en route to the target destination in the first and second dimensions and advancing one level toward the destination level of the third dimension is blocked by another message, advancing the message one level toward the destination level of the third dimension when the en route node is not blocked and moving the message in the first and second dimensions along a constant level in the third dimension otherwise. This method further includes the step of specifying the third dimension to describe a plurality of levels and specifying the first and second dimensions to described a plurality of nodes on each level. A control signal is sent from the node en route to the node N on a level q in the third dimension, the control signal specifying whether the node en route is blocked. Transmission of a message is timed using a global clock specifying timing intervals to keep integral time modulus the number of nodes at a particular cylindrical height, the global clock time interval being equal to the second time interval and the first time interval being smaller than the global time interval. A first time interval a is set for moving the message in only the first and second dimensions. A second time interval α-β is set for advancing the message one level toward the destination level. A third time interval is set for sending the control signal from the node en route to the node N, the third time interval being equal to β.
- In accordance with a third aspect of the present invention, a method is shown of transmitting a message from an input device to an output device through an interconnect structure. The message travels through the interconnect structure connecting a plurality of nodes in a three dimensional structure. The message has a target destination corresponding to a target ring on
level 0 of the interconnect structure. A message M at a node N on level T en route to a target ring onlevel 0 advances to a node N′ on level T−1 so long as the target ring is accessible from node N′ and no other higher priority message is progressing to node N′ to block the progress of message M. Whether the target ring is accessible from node N′ is typically efficiently determined by testing a single bit of a binary code designating the target ring. Whether a higher priority message is blocking the progress of message M is efficiently determined using timed control signals. If a message is blocked at a time t, the message is in position to progress to the next level attime t+ 2. If a message is blocked by a message M′ on level T−1, then a limited time duration will transpire before the message M′ is able to block message M again. - A global clock controls traffic flow in the interconnect structure. Data flow follows rules that allow much of the control information to be “hidden” in system timing so that, rather than encoding all control information in a message packet header, timing considerations convey some information. For example, the target ring is encoded in the message packet header but, in some embodiments of the interconnect structure, designation of the target computational unit is determined by the timing of arrival of a message with respect to time on the global clock.
- The disclosed multiple level interconnect structure has many advantages. One advantage is that the structure is simple, highly ordered and achieves fast and efficient communication for systems having a wide range of sizes, from small systems to enormous systems.
- In addition, the interconnect structure is highly advantageous for many reasons. The interconnect structure resolves contention among messages directed toward the same node and ensures that a message that is blocked makes a complete tour of the messages at a given angle on a level before the blocking message is in position to block again. In this manner, a message inherently moves to cover all possible paths to the next level. A blocking message typically proceeds to subsequent levels so that overlying messages are not blocked for long.
- The features of the invention believed to be novel are specifically set forth in the appended claims. However, the invention itself, both as to its structure and method of operation, may best be understood by referring to the following description and accompanying drawings.
- FIGS. 1A, 1B,1C and 1D are abstract three-dimensional pictorial illustrations of the structure of an embodiment of a multiple level minimum logic interconnect apparatus.
- FIG. 2 is a schematic diagram of a node, node terminals and interconnection lines connected to the terminals.
- FIGS. 3A, 3B and3C are schematic block diagrams that illustrate interconnections of nodes on various levels of the interconnect structure.
- FIG. 4 is an abstract schematic pictorial diagram showing the topology of levels of an interconnect structure.
- FIG. 5 is an abstract schematic pictorial diagram showing the topology of nodes of an interconnect structure.
- FIG. 6 is an abstract schematic pictorial diagram which illustrates the manner in which nodes of the rings on a particular cylindrical level are interconnected.
- FIG. 7 illustrates interconnections of a node on level zero.
- FIG. 8 depicts interconnections of a node on level one.
- FIG. 9 depicts interconnections of a node on level two.
- FIG. 10 depicts interconnections of a node on level three.
- FIG. 11 is an abstract schematic pictorial diagram which illustrates interconnections between devices and nodes of a ring on the low level cylinder.
- FIG. 12 is an abstract schematic pictorial diagram which illustrates interconnections among nodes of two adjacent cylindrical levels.
- FIG. 13 is an abstract schematic pictorial diagram showing interconnections of nodes on cylindrical level one.
- FIG. 14 is an abstract schematic pictorial diagram showing interconnections of nodes on cylindrical level two.
- FIG. 15 is an abstract schematic pictorial diagram showing interconnections of nodes on cylindrical level three.
- FIG. 16 is an abstract schematic pictorial diagram illustrating the interaction of messages on adjacent levels of an embodiment of the interconnection structure.
- FIG. 17 is a timing diagram which illustrates timing of message communication in the described interconnect structure.
- FIG. 18 is a pictorial representation illustrating the format of a message packet including a header and payload.
- FIG. 19 is a pictorial diagram which illustrates the operation of a lithium niobate node, a first exemplary node structure.
- FIG. 20 is a pictorial diagram which illustrates the operation of a nonlinear optical loop mirror (NOLM), a second exemplary node structure.
- FIG. 21 is a pictorial diagram which illustrates the operation of a terahertz optical asymmetrical demultiplexer (TOAD) switch, a third exemplary node structure.
- FIG. 22 is a pictorial diagram showing the operation of a regenerator utilizing a lithium niobate gate.
- FIG. 23 is an abstract schematic pictorial diagram illustrating an alternative embodiment of an interconnect structure in which devices issue message packets to multiple nodes.
- FIG. 24 is an abstract schematic pictorial diagram illustrating an alternative embodiment of an interconnect structure in which devices receive message packets from multiple nodes.
- FIG. 25 is an abstract schematic pictorial diagram illustrating an alternative embodiment of an interconnect structure in which devices issue message packets to nodes at various interconnect levels.
- Referring to FIGS. 1A, 1B,1C and 1D, an embodiment of a multiple level minimum
logic interconnect apparatus 100 includesmultiple nodes 102 which are connected in a multiple level interconnect structure by interconnect lines 104. The multiple level interconnect structure is shown illustratively as a three-dimensional structure to facilitate understanding. - The
nodes 102 in the multiple level interconnect structure are arranged to includemultiple levels 110, eachlevel 110 having a hierarchical significance so that, after a message is initiated in the structure, the messages generally move from aninitial level 112 to afinal level 114 in the direction of levels of a previoushierarchical significance 116 to levels of a subsequenthierarchical significance 118. Illustratively, eachlevel 110 includes multiple structures which are calledrings 120. Eachring 120 includesmultiple nodes 102. The term “rings” is used merely to facilitate understanding of the structure of a network in the abstract in which visualization of the structure as a collection of concentriccylindrical levels 110 is useful. - The different FIGS. 1A, 1B,1C and 1D are included to more easily visualize and understand the interconnections between nodes. FIG. 1A illustrates message data transmission interconnections between
nodes 102 on the variouscylindrical levels 110. FIG. 1B adds a depiction of message data transmission interconnections betweennodes 102 anddevices 130 to the interconnections illustrated in FIG. 1A. FIG. 1C further shows message data interconnections betweennodes 102 on different levels. FIG. 1D cumulatively shows the interconnections shown in FIGS. 1A, 1B and 1C in addition to control interconnections between thenodes 102. - The actual physical geometry of an interconnect structure is not to be limited to a cylindrical structure. What is important is that multiple nodes are arranged in a first class of groups and the first class of groups are arranged into a second class of groups. Reference to the first class of groups as rings and the second class of groups as levels is meant to be instructive but not limiting.
- The
illustrative interconnect apparatus 100 has a structure which includes a plurality of J+1levels 110. Eachlevel 110 includes a plurality of 2JK nodes 102. Each level M contains 2J-Mrings 120, each containing 2MK nodes 102. The total number ofnodes 102 in the entire structure is (J+1)2JK. Theinterconnect apparatus 100 also includes a plurality 2JK devices 130. In the illustrative embodiment, each device of the 2JK devices 130 is connected to a data output port of each of theK nodes 102 in each ring of the 2 rings of thefinal level 114. Typically, in an interconnect structure of a computer adevice 130 is a computational unit such as a processor-memory unit or a cluster of processor-memory units and input and output buffers. - Referring to FIG. 2, an
interconnect structure 200 of anode 102 has three input terminals and three output terminals. The input terminals include a firstdata input terminal 210, a seconddata input terminal 212 and acontrol input terminal 214. The output terminals include a firstdata output terminal 220, a seconddata output terminal 222 and acontrol output terminal 224. The data input and output terminals of a node communicate message data with other nodes. The control terminals communicate control bits with other nodes for controlling transmission of message data. The number of control bits for controlling message transmission is efficiently reduced since much of the logic throughout theinterconnect structure 200 is determined by timing of the receipt of control bits and message data in a manner to be detailed hereinafter. Only one control bit enters a node and only one control bit leaves at a given time step. Messages are communicated by generating a clock signal for timing time units. Message transmission is controlled so that, during one time unit, anynode 102 receives message data from only one input terminal of thedata input terminals - Referring to FIGS. 3 through 16, the topology of an
interconnect structure 300 is illustrated. To facilitate understanding, thestructure 300 is illustrated as a collection of concentric cylinders in three dimensions r, θ and z. Each node or device has a location designated (r, θ, z) which relates to a position (r, 2πθ/K, z) in three-dimensional cylindrical coordinates where radius r is an integer which specifies the cylinder number from 0 to J, angle θ is an integer which specifies the spacing of nodes around the circular cross-section of a cylinder from 0 to K−1, and height z is a binary integer which specifies distance along the z-axis from 0 to 2J−1. Height z is expressed as a binary number because the interconnection between nodes in the z-dimension is most easily described as a manipulation of binary digits. Accordingly, aninterconnect structure 300 is defined with respect to two design parameters J and K. - FIGS. 3A, 3B and3C are schematic block diagrams that show interconnections of nodes on various levels of the interconnect structure. FIG. 3A shows a
node A RJ 320 on a ring R of outermost level J and the interconnections ofnode A RJ 320 tonode B RJ 322,device C 324,node D RJ 326,node E R(J−1) 328,node F R(J−1) 330 anddevice G 332. FIG. 3B shows anode A RT 340 on a ring R of a level T and the interconnections ofnode A RT 340 tonode B RT 342,node C R(T+1) 344,node D RT 346,node E R(T−1) 348,node F R(T−1) 350 andnode G R(T+1) 352. FIG. 3C shows anode A R0 360 on a ring R ofinnermost level 0 and the interconnections ofnode A R0 360 tonode B R0 362,node C R1 364,node D R0 366,device E 368 andnode G R1 372. - In FIGS. 3A, 3B and3C interconnections are shown with solid lines with arrows indicating the direction of message data flow and dashed lines with arrows indicating the direction of control message flow. In summary, for nodes A, B and D and nodes or devices C, E, F, G:
- (1) A is on level T.
- (2) B and C send data to A.
- (3) D and E receive data from A.
- (4) F sends a control signal to A.
- (5) G receives a control signal from A.
- (6) B and D are on level T.
- (7) B is the immediate predecessor of A.
- (8) D is the immediate successor to A.
- (9) C, E, F and G are not on level T.
- The positions in three-dimensional cylindrical notation of the various nodes and devices is as follows:
- (10) A is positioned at node N(r, θ, z).
- (11) B is positioned at node N(r, θ−1, HT(z)).
- (12) C is either positioned at node N(r+1, θ−1, z) or is outside the interconnect structure.
- (13) D is positioned at node N(r, θ+1, hT(z)).
- (14) E is either positioned at node N(r−1, θ+1, z) or is outside the interconnect structure and the same as device F.
- (15) F is either positioned at node N(r−1, θ, HT−1(z)) or is outside the interconnect structure and the same as device E.
- (16) G is either positioned at node N(r+1, θ, hT(z)) or is outside the interconnect structure.
- In this notation, (θ−1)mod K is equal K when θ is equal to 0 and equal to θ−1 otherwise. The conversion of z to Hr(z) on a level r is described for z=[zJ−1, zJ−2, . . . , zr, zr−1, . . . , z2, z1, z0] by reversing the order of low-order z bits from zr−1 to z0] into the form z=[zJ−1, zJ−2, . . . , zr, z0, z1, z2, . . . , zr−1], subtracting (modulus 2r) and reversing back the low-order z bits. Similarly, (θ+1)mod K is equal 0 when θ is equal to K−1 and equal to θ+1 otherwise. The conversion of z to hr(z) on a level r is described for z=[zJ−1, zJ−2, . . . , zr, zr−1, . . . , z2, z1, z0] by reversing the order of low-order z bits from
z r−1 to z0 into the form z=[zJ−1, zJ−2, . . . , zr, z0, z1, z2, . . . , zr−1], adding (modulus 2r) and reversing back the low-order z bits. - Referring to FIG. 4, concentric cylindrical levels zero310, one 312, two 314 and three 316 are shown for a J=3
interconnect structure 300 wherelevel 0 refers to the innermost cylindrical level, progressing outward and numerically to the outermostcylindrical level 3. Anode 102 on a level T is called a level T node. - An interconnect structure has J+1 levels and 2JK nodes on each level. Referring to FIG. 5, the design parameter K is set equal to 5 so that the
interconnect structure 300 has four levels (J+1=3+1=4) with 40 (2JK=(23)5=40) nodes on each level. - Referring to FIG. 6, the interconnect structure is fully defined by designating the interconnections for each
node A 530 of each level T to devices ornodes B 532,C 534,D 536,E 538, F 540 andG 542. - Node A(r,θ,z)530 is interconnected with an immediate predecessor node B(r,(θ−1)mod K,Hr(z)) 532 on level r. If node A(r,θ,z) 530 is on the outermost level r=J, node A(r,θ,z) 530 is interconnected with a device (e.g. a computational unit of a computer) outside of the interconnect structure. Otherwise, node A(r,θ,z) 530 is interconnected with a predecessor node C(r+1,(θ−1)mod K,z) 534 on
level r+ 1. - Node A(r,θ,z)530 is interconnected with an immediate successor node D(r,(θ+1)mod K,hr(z)) 536 on level r. If node A(r,θ,z) 530 is on the innermost level r=0, node A(r,θ,z) 530 is interconnected with a device (e.g. a computational unit) outside of the interconnect structure. Otherwise, node A(r,θ,z) 530 is interconnected with a successor node E(r−1, (θ+1)mod K,z) 538 on level r−1 to send message data.
- If node A(r,θ,z)530 is on the innermost level r=0, node A(r,θ,z) 530 is interconnected with a device (e.g. a computational unit) outside of the interconnect structure. Otherwise, node A(r,θ,z) 530 is interconnected with a node F(r−1,θ,Hr−1(z)) 540 on level r−1 which supplies a control input signal to node A(r,θ,z) 530.
- If node A(r,θ,z)530 is on the outermost level r=J, node A(r,θ,z) 530 is interconnected with a device (e.g. a computational unit) outside of the interconnect structure. Otherwise, node A(r,θ,z) 530 is interconnected with a node G(r+1, θ,hr+1(z)) 542 on level r+1 which receives a control output signal from A(r,θ,z) 530.
- Specifically, the interconnections of a node A for the example of an interconnect structure with interconnect design parameters J=3 and K=5 are defined for all nodes on a ring. Every ring is unidirectional and forms a closed curve so that the entire structure is defined by designating for each node A, a node D that receives data from node A.
- Referring to FIG. 7 in conjunction with FIG. 6, interconnections of a node A on level zero are shown. Node A(0,θ,z)) 530 is interconnected to receive message data from immediate predecessor node B(0,(θ−1)
mod 5,z) 532 onlevel 0 and to send message data to immediate successor node D(0,(θ+1)mod 5,z) 536 onlevel 0. Although the interconnection term in the second dimension for nodes B and D is previously defined as Hr(z) and kr(z), respectively, on level zero, Hr(z) and hr(z) are equal to z. Node A(0,θ,z) 530 is also interconnected to receive message data from predecessor node C(1,(θ−1)mod 5,z) 534 onlevel 1 and to send message data to a device E(θ,z) 538. Node A(0,θ,z)) 530 is interconnected to receive a control input signal from a device F((θ−1)mod 5,z) 540 and to send a control output signal to node G(1,θ,h1(z)) 542 onlevel 1. - Referring to FIG. 8 in conjunction with FIG. 6, interconnections of a node A on level one are shown. Node A(1,θ,z) 530 is interconnected to receive message data from immediate predecessor node B(1,(θ−1)
mod 5,H1(z)) 532 onlevel 1 and to send message data to immediate successor node D(1,(θ+1)mod 5,h1(z)) 536 onlevel 1. Height z is expressed as a binary number (base 2) having the form [z2,z1,z0]. For level one, when z is [z2,z1,0] then h1(z) and H1(z) are both [z2,z1,1]. When z is [z2,z1,1] then h1(z) and H1(z) are both [z2,z1,0]. Node A(1,θ,z) 530 is also interconnected to receive message data from predecessor node C(2,(θ−1)mod 5,z) 534 onlevel 2 and to send message data to successor node E(0,(θ+1)mod 5,z) 538 onlevel 0. Node A(1,θ,z) 530 is interconnected to receive a control input signal from a node F(0,θ,H1(z)) 540 on level zero and to send a control output signal to node G(2,θ,h2(z)) 542 onlevel 2. - Referring to FIG. 9 in conjunction with FIG. 6, interconnections of a node A on level two are shown. Node A(2,θ,z) 530 is interconnected to receive message data from immediate predecessor node B(2,(θ−1)
mod 5,H2(z)) 532 onlevel 2 and to send message data to immediate successor node D(2,(θ+1)mod 5,h2(z)) 536 onlevel 2. Height z is expressed as a binary number (base 2) having the form [z2,z1,z0]. For level two, when z is [z2,0,0] then h2(z) is [z2,1,0] and H2(z) is [z2,1,1]. When z is [z2,0,1] then h2(z) is [z2,1,1] and H2(z) is [z2,1,0]. When z is [z2,1,0] then h2(z) is [z2,0,1] and H2(z) is [z2,0,0.] When z is [z2,1,1] then h2(z) is [z2,0,0] and H2(z) is [z 20,1]. Node A(2,0,z) 530 is also interconnected to receive message data from predecessor node C(3,(θ−1)mod 5,z) 534 onlevel 3 and to send message data to successor node E(1,(θ+1)mod 5,z) 538 onlevel 1. Node A(2,θ,z) 530 is interconnected to receive a control input signal from a node F(1,θ,H2(z)) 540 onlevel 1 and to send a control output signal to node G(3,θ,h3(z)) 542 onlevel 3. - Referring to FIG. 10 in conjunction with FIG. 6, interconnections of a node A on level three are shown. Node A(3,θ,z) 530 is interconnected to receive message data from immediate predecessor node B(3,(θ−1)
mod 5,H3(z)) 532 onlevel 3 and to send message data to immediate successor node D(3,(θ+1)mod 5,h3(z)) 536 onlevel 3. For level three, when z is [0,0,0] then h3(z) is [1,0,0] and H3(z) is [1,1,1]. When z is [0,0,1] then h3(z) is [1,0,1] and H3(z) is [1,1,0]. When z is [0,1,0] then h3(z) is [1,1,0] and H3(z) is [1,0,0]. When z is [0,1,1] then h3(z) is [1,1,1] and H3(z) is [1,0,1]. When z is [1,0,0] then h3(z) is [0,1,0] and H3(z) is [0,0,0]. When z is [1,0,1] then h3(z) is [0,1,1] and H3(z) is [0,0,1]. When z is [1,1,0] then h3(z) is [0,0,1] and H3(z) is [0,1,0]. When z is [1,1,1] then h3(z) is [0,0,0] and H3(z) is [0,1,1]. Node A(3,θ,z) 530 is also interconnected to receive message data from predecessor node C(4,(θ−1)mod 5,z) 534 onlevel 4 and to send message data to successor node E(2,(θ+1)mod 5,z) 538 onlevel 2. Node A(3,θ,z) 530 is interconnected to receive a control input signal from a node F(2,θ,H3(z)) 540 onlevel 2 and to send a control output signal to node G(4,θ,h4(z)) 542 onlevel 4. - FIG. 11 illustrates interconnections between
devices 130 andnodes 102 of aring 120 on the cylindrical level zero 110. In accordance with the description of theinterconnect structure 200 of anode 102 discussed with respect to FIG. 2, anode 102 has three input terminals and three output terminals, including two data input terminals and one control input terminal and two data output terminals and one control output terminal. In a simple embodiment, adevice 130 has onedata input terminal 402, one control bitinput terminal 404, onedata output terminal 406 and one controlbit output terminal 408. - Referring to FIG. 11,
nodes 102 at the lowestcylindrical level 110, specifically nodes N(0,θ,z), are connected to devices CU(θ,z). In particular, thedata input terminal 402 of devices CU(θ,z) are connected to the seconddata output terminal 222 of nodes N(0,θ,z). The controlbit output terminal 408 of devices CU(θ,z) are connected to thecontrol input terminal 214 of nodes N(0,θ,z). - The devices CU(0,z) are also connected to nodes N(J,θ,z) at the outermost cylinder level. In particular, the
data output terminal 406 of devices CU(θ,z) are connected to the seconddata input terminal 212 of nodes N(J,θ,z). The control bitinput terminal 404 of devices CU(θ,z) are connected to thecontrol output terminal 224 of nodes N(0,θ,z)). Messages are communicated from devices CU(θ,z) to nodes N(J,θ,z) at the outermost cylindrical level J. Then messages move sequentially inward from the outermost cylindrical level J to level J−1, level J−2 and so forth unit the messages reachlevel 0 and then enter a device. Messages on the outermost cylinder J can reach any of the 2J rings at level zero. Generally, messages on any cylindrical level T can reach a node on 2T rings on level zero. - FIG. 12 illustrates interconnections among
nodes 102 of two adjacentcylindrical levels 110. Referring to FIG. 12 in conjunction with FIG. 2,nodes 102 at the Tcylindrical level 110, specifically nodes N(T,θ,z) 450, have terminals connected to nodes on the T level, the T+1 level and the T−1 level. These connections are such that the nodes N(T,θ,z) 450 have one data input terminal connected to a node on the same level T and one data input terminal connected to another source, usually a node on the next outer level T+1 but for nodes on the outermost level J, a device is a source. In particular, nodes N(T,θ,z) 450 have a firstdata input terminal 210 which is connected to a firstdata output terminal 220 of nodes N(T+1,θ−1,z) 452. Also, nodes N(T,θ,z) 450 have a firstdata output terminal 220 which is connected to a firstdata input terminal 210 of nodes N(T−1,θ+1,z) 454. - The nodes N(T,θ,z)450 also have a second
data input terminal 212 and a seconddata output terminal 222 which are connected tonodes 102 on the same level T. The seconddata input terminal 212 of nodes N(T,θ,z) 450 are connected to the seconddata output terminal 222 of nodes N(T,θ−1,hT(z)) 456. The seconddata output terminal 222 of nodes N(T,θ,z) 450 are connected to the seconddata input terminal 212 of nodes N(T,θ+1,HT(z)) 458. The cylinder height designation HT(z) is determined using an inverse operation of the technique for determining height designation hT(z). The interconnection of nodes from cylindrical height to height (height z to height HT(z) and height hT(z) to height z) on the same level T is precisely defined according to a height transformation technique and depends on the particular level T within which messages are communicated. Specifically in accordance with the height transformation technique, the height position z is put into binary form where z=z J−12J−1+z J−12J−2+ . . . +z T2T+z T−12T−1+ . . . +z 121+z 020. A next height position hT(z) is determined using a process including three steps. First, binary coefficients starting with coefficient z, up to and but not including coefficient zT are reversed in order while coefficients zT and above are kept the same. Thus, after the first step the height position becomesz JH−12J−1+z J−22J−2+ . . . +z T2T+z 020+z 121+ . . . +z T−22T−2+z T−12T−1. Second, an odd number modulus 2T, for example one, is added to the height position after inversion. Third, circularity of the height position is enforced by limiting the inverted and incremented height position bymodulus 2T. Fourth, the first step is repeated, again inverting the binary coefficients below the zJ coefficient of the previously inverted, incremented and limited height position. The inverse operation for deriving height descriptor HT(z) is determined in the same manner except that, rather than adding the odd number modulus 2T to the order-inverted bit string, the same odd number modulus 2T is added to the order-inverted bit string. - The interconnection between
nodes 102 on the same level is notable and highly advantageous for many reasons. For example, the interconnection structure resolves contention among messages directed toward the same node. Also, the interconnection structure ensures that a message on a particular level that is blocked by messages on the next level makes a complete tour of the messages on that level before any message is in position to block again. Thus a message inherently moves to cover all possible paths to the next level. Furthermore, a blocking message must cycle through all rings of a level to block a message twice. Consequently, every message is diverted to avoid continuously blocking other messages. In addition, blocking messages typically proceed to subsequent levels so that overlying messages are not blocked for long. - When messages are sent from second
data output terminal 222 of a node N(T,θ,z) 450 to a seconddata input terminal 212 of a node N(T,θ+1,hT(z)), a control code is also sent from acontrol output terminal 224 of the node N(,θ,z) 450 to acontrol input terminal 214 of a node N(T+1,θ,hT+1(z)), the node on level T+1 that has a data output terminal connected to a data input terminal of node N(T,θ+1,hT(z)). This control code prohibits node N(T+1,θ,hT+1(z)) from sending a message to node N(T,θ+1,hT+1(z)) at the time node N(T,θ,z) 450 is sending a message to node N(T,θ+1,hT+1(z)). When node N(T+1,θ,hT+1(z)) is blocked from sending a message to node N(T,θ+1,hT+1(z)), the message is deflected to a node onlevel T+ 1. Thus, messages communicated on the same level have priority over messages communicated from another level. - The second
data output terminal 222 of nodes N(T,θ−1,HTz)) are connected to a seconddata input terminal 212 of nodes N(T,θ,z) 450 so that nodes N(T,θ,z) 450 receive messages from nodes N(T,θ−1,HT(z)) that are blocked from transmission to nodes N(T−1,θ,HT(z)). Also, thecontrol output terminal 224 of nodes N(T−1,θ,HT(z)) to thecontrol input terminal 214 of nodes N(T,θ,z) 450 to warn of a blocked node and to inform nodes N(T,θ,z) 450 not to send data at this time since no node receives data from two sources at the same time. - Referring to FIG. 13, interconnections of
nodes 102 on cylindrical level one exemplify the described interconnections and demonstrate characteristics and advantages that arise from the general interconnection technique. In this example, the number of nodes K at a cylindrical height is five and the number ofheights 2J is 22, or 4, for a three level (J+1)interconnect structure 500. Nodes N(1,θ,z) 510 have: (1) a firstdata input terminal 210 connected to a firstdata output terminal 220 of nodes N(2,θ−1,z) 512, (2) acontrol output terminal 224 connected to controlinput terminal 214 of nodes N(2,θ,h2(z)) 512, (3) a firstdata output terminal 220 connected to a firstdata input terminal 210 of nodes N((0,θ+1,z) 516, (4) acontrol input terminal 214 connected to acontrol output terminal 224 of nodes N(0,θ,H,(z)) 516, (5) a seconddata input terminal 212 connected to the seconddata output terminal 222 of nodes N(1,θ−1,H1(z)) 520, and (6) a seconddata output terminal 222 connected to the seconddata input terminal 212 of nodes N(1,θ+1,h1(z)) 522. For nodes N(1,θ,z) 510 on level one, height z differs from height h1(z) and height H1(z) only in the final bit position. - Messages are communicated through the
interconnect structure 500 in discrete time steps. A global clock (not shown) generates timing signals in discrete time steps modulus the number of nodes K at a cylindrical height z of a cylindrical level r. When messages traverse theinterconnect structure 500 on the same level (for example, level one) because nodes on an inner level are blocked, messages are communicated from node to node in the discrete time steps. For theinterconnect structure 500 with an odd number (K=5) of nodes at a cylindrical level, if data traverses level one for 2K time steps, then the message packet visits 2K different nodes. On time step 2K +1, message packets will begin repeating nodes following the sequential order of the first node traversal. Because the global clock generates the discrete time steps integral time modulus K, if a message packet on level one is over the target ring of that packet at a time T=0 (modulus K) and is deflected by a message on level zero, the message will be over the target ring also at a time T=0 (modulus K) to make another attempt to enter the target ring. In various embodiments, this timing characteristic is consistent throughout the interconnect structure so that, if a message packet is in a position to descend to the next level at a time T=0 (modulus K), the packet will once again be in a position to descend at a subsequent time T=0 (modulus K). - Referring to FIG. 14 in conjunction with FIG. 13, interconnections of
nodes 102 on cylindrical level two further exemplify described interconnections. In FIG. 14, a level twomessage path 620 is shown overlying thepaths 610 and 612 of messages moving on level one. The number of nodes K at a cylindrical level is five and the number oflevels 2J is 22, or 4, for a three level (J+1)interconnect structure 500. Same-level interconnections of nodes N(2,θ,z) include: (1) a seconddata input terminal 212 connected to the seconddata output terminal 222 of nodes N(2,θ−1,h2(z)) and (2) a seconddata output terminal 222 connected to the seconddata input terminal 212 of nodes N(2,θ+1,H2(z)). For nodes N(2,θ,z) on level two, height z differs from height h2(z) and height H2(z) only in the final two bit positions. Generally stated in binary form for any suitable number of nodes K at a height and number ofheights 2J in a level, bits z and h2(z) on cylindrical level two are related as follows: - [zJ−1, zJ−2, . . . , z2, 0,0]′=[zJ−1, zJ−2, . . . , z2, 1,0];
- [zJ−1, zJ−2, . . . , z2, 1,0]′=[zJ−1, zJ−2, . . . , z2, 0,1];
- [zJ−1, zJ−2, . . . , z2, 0,1]′=[zJ−1, zJ−2, . . . , z2, 1,1] and
- [zJ−1, zJ−2, . . . , z2, 1,1]′=[zJ−1, zJ−2, . . . , z2, 0,0]. PCT
- A second advantage of this interconnection technique for nodes on the same level is that blocked messages are directed to avoid subsequent blocking. FIG. 14 illustrates a message blocking condition and its resolution. On level one, a
message m 0 610 is shown at node N001 and a message m1 612 at node N011.A message M 620 on level two at node N002 is targeted for ring zero. At a time zero,message M 620 is blocked and deflected by message m1 612 to node N122 at time one. Assuming that messages m0 and m1 are also deflected and traversing level one, at a time onemessage m 0 610 is at node N111 and message m1 612 at node N101. At a time two,message M 620 moves to node N212, message m0 610 to node N201 and message m1 612 to node N211. Thus, at time two,message M 620 is deflected bymessage m 0 610. At time four,message M 620 is again blocked by message m1 612. This alternating blocking ofmessage M 620 by messages m0 610 and m1 612 continues indefinitely as long as messages m0 610 and m1 612 are also blocked. This characteristic is pervasive throughout the interconnect structure so that a single message on an inner level cannot continue to block a message on an outer level. Because a single message packet cannot block another packet and blocking packets continually proceed through the levels, blocking does not persist. - Referring to FIG. 15, interconnections of
nodes 102 on cylindrical level three show additional examples of previously described interconnections. A level threemessage path 720 is shown overlying thepaths heights 2J is 23 (8), for a four level (J+1) interconnect structure. Same-level interconnections of nodes N(3,θ,z) include: (1) a seconddata input terminal 212 connected to the seconddata output terminal 222 of nodes N(3,θ−1,h3(z)) and (2) a seconddata output terminal 222 connected to the seconddata input terminal 212 of nodes N(3,θ+1,H3(z)). For nodes N(3,θ,z) on level three, height z differs from height h3(z) and height H3(z) only in the final three bit positions. Generally stated in binary form for any suitable number of nodes K at a cylindrical height and number ofheights 2J in a level, bits z and h3(z) on cylindrical level three are related as follows: - [zJ−1, zJ−2, . . . z3, 0, 0, 0]′=[zJ−1, zJ−2, . . . , z3, 1, 0, 0];
- [zJ−1, zJ−2, . . . z3, 1, 0, 0]′=[zJ−1, zJ−2, . . . , z3, 0, 1, 0];
- [zJ−1, zJ−2, . . . z3, 0, 1, 0]′=[zJ−1, zJ−2, . . . , z3, 1, 1, 0];
- [zJ−1, zJ−2, . . . z3, 1, 1, 0]′=[zJ−1, zJ−2, . . . , z3, 0, 0, 1];
- [zJ−1, zJ−2, . . . z3, 0, 0, 1]′=[zJ−1, zJ−2, . . . , z3, 1, 0, 1];
- [zJ−1, zJ−2, . . . z3, 1, 0, 1]′=[zJ−1, zJ−2, . . . , z3, 0, 1, 1];
- [zJ−1, zJ−2, . . . z3, 0, 1, 1]′=[zJ−1, zJ−2, . . . , z3, 1, 1, 1];
- [zJ−1, zJ−2, . . . z3, 1, 1, 1]′=[zJ−1, zJ−2, . . . , z3, 0, 0, 0];
- FIG. 15 illustrates another example of a message blocking condition and its resolution. On level two, a
message m 0 710 is shown at node N002, a message m1 712 at node N012, amessage m 2 714 at node N022 and amessage m 3 716 at node N032.A message M 720 on level three at node N303 is targeted for ring zero. At a time zero,message M 720 is blocked and deflected bymessage m 3 716 to node N173 at time one. Assuming that messages m0, m1, m2 and m3 are also deflected and traversing level two, at a time onemessage m 0 710 is at node N132, message m1 712 at node N122, message m2 714 at node N102 and message m3 716 at node N112. At a time two,message M 720 moves to node N233, message m0 710 to node N212, message m1 712 to node N202, message m2 714 to node N232 and message m3 716 to node N222. Thus, at time two,message M 720 is deflected by message m1 712. At time four,message M 720 is blocked bymessage m 2 714. At time six,message M 720 is blocked bymessage m 0 710. At time eight,message M 720 is again blocked bymessage m 3 716. This alternating blocking ofmessage M 720 by messages m0 710, m1 712, m2 714 andm 3 716 continues indefinitely as long as messages m0 710, m1 712, m2 714 andm 3 716 are also blocked. - This analysis illustrates the facility by which the described interconnect structure avoids blocking at any level. Thus, “hot spots” of congestion in the structure are minimized. This characteristic is maintained at all levels in the structure.
- The described interconnect structure provides that every node N(0,θ,z) on level zero is accessible by any node N(J,θ,z) on outermost level J. However, only half of the nodes N(0,θ,z) on level zero are accessible by a node N(J−1 ,θ,z) on the level once removed from the outermost level. Data at a node N(1,θ,z) on level one can access any node N(0,θ,z) on level zero so long as the binary representation of height z of level one and the binary representation of ring r of level zero differ only in the last bit. Similarly, data at a node N(2,θ,z) on level two can access any node N(0,θ,z) on level zero so long as the binary representation of height z of level two and the binary representation of ring r of level zero differ only in the last two bits. A general rule is that, data at a node N(T,θ,z) on level T can access any node N(0,θ,z) on level zero so long as the binary representation of height z of level T and the binary representation of ring r of level zero differ only in the last T bits. Accordingly, moving from the outermost level J to level J−1 fixes the most significant bit of the address of the target ring. Moving from level J−1 to level J−2 fixes the next most significant bit of the address of the target ring and so forth. At level zero, no bits are left to be fixed so that no header bit is tested and a message is always passed to a device. In some embodiments, an additional header bit is included and tested at a level zero node. This final bit may be used for various purposes, such as for directing message data to a particular buffer of a device when the device accepts the message data. An advantage of including an additional bit in the header and performing a bit test at the final node is that all the nodes at all levels of the interconnect structure operate consistently.
- In some embodiments of an interconnect structure, an additional header bit is included in a message packet. This bit indicates that a message packet is being transmitted. Another purpose for such an additional bit in the header is to identify which bit in the header is the control bit.
- A message packet moves from a level T to the next inner level T−1 so long as two conditions are met, as follows: (1) the target ring of the message packet is accessible from level T−1, and (2) the message packet is not blocked by a message on the level T−1.
- One significant aspect of this structure is that any message packet at a node N(T,θ,z) on a level T that can access its target ring can also access the target ring from a node N(T−1,θ+1,z) only if the bit T−1 of the address ring is the same as bit T−1 of the target ring. Therefore, analysis of only a single bit yields all information for determining a correct routing decision.
- Referring to FIG. 16, a general relationship between message packets on two adjacent levels T and T+1 is described. In this example, a message packet M at a node N450 on level four, which is targeted for ring zero, is potentially blocked by eight message packets m0 810,
m 1 811,m 2 812,m 3 813,m 4 814,m 5 815, m6 816 andm 7 817 at nodes N3i0 residing on each of theheights 0 to 7 on level three. Although the behavior of the interconnect structure is analyzed with respect to levels three and four for purposes of illustration, the analysis is applicable to any arbitrary adjacent levels. At an arbitrary time step, illustratively called time step zero, the message M moves from node N450 on level four to node N351 on level three unless a control code is send to node N450 from a level three node having a data output terminal connected to node N351. In this example, node N310 has a data output terminal connected to node N351 and, at time step zero,message m 7 817 resides at node N310. Accordingly, node N310 sends a control code, in this example a single bit code, to node N450, causing deflection of message M to node N4D1 (where D is a hexadecimal designation of 13) on an interconnection line. A bit line illustratively shows the control connection from node N310 to node N450. At a time step one, message M moves from node N4D1 to node N432 on interconnection line regardless of whether a node N328 is blocked because ring zero is not accessible from node N328. At a time step two, message M moves from node N432 to node N334 unless a control blocking code is sent from node N352 to node N432 where node N352 is the node on level three that has a data output terminal connected to a data input terminal of node N334. However, the message M is blocked from accessing node N334 because message m6 currently resides at node N352 at time step two. A deflection control code is sent from node N352 to node N432 on control bit line 822. Furthermore, assuming that none of the message packets t progresses to level two and beyond, at time step four, message M is blocked by message m2 via a control code sent on control bit line. At time six, message M is blocked by message m4 though a blocking control code on control bit line. - This example illustrates various advantages of the disclosed interconnection structure. First, deflections of the message M completely tour all of the heights on a level T if messages mj on level T−1 continue to block progression to the level T−1 for all levels T. Accordingly, a message M on a level T is blocked for a complete tour of the heights only if 2T−1 messages are in position on level T−1 to 15 block message M. In general, a message mj on a level T−1 must remain on the level T−1 for 2T+1 time steps to block the same message M on level T twice.
- The description exemplifies an interconnect structure in which messages descend from an outer level to devices at a core inner layer by advancing one level when the height dimension matches the destination ring location and traversing the rings when the ring location does not match the height designation. In other embodiments, the messages may move from an inner level to an outer level. In some embodiments, the heights may be traversed as the level changes and the height held constant as the level remains stationary. In these embodiments, the progression of messages through nodes is substantially equivalent to the disclosed interconnect structure. However, the advantage of the disclosed network that avoids blocking of messages is negated.
- Referring to FIG. 17, a timing diagram illustrates timing of message communication in the described interconnect structure. In various embodiments of the interconnect structure, control of message communication is determined by timing of message arrival at a node. A message packet, such as a packet900 shown in FIG. 18, includes a header 910 and a
payload 920. The header 910 includes a series ofbits 912 designating the target ring in a binary form. When a source device CU(θ1,z1) at an angle θ1 and height z, sends a message packet M to a destination device CU(θ2,z2) at an angle θ2 and height z2, thebits 912 of header 910 are set to the binary representation of height z2. - A global clock servicing an entire interconnect structure keeps integral time modulus K where, again, K designates the number of nodes n at a cylinder height z. There are two constants α and β such that the duration of α exceeds the duration of β and the following five conditions are met. First, the amount of time for a message M to exit a node N(T,θ+1,hT(z)) on level T after exiting a node N(T,θ,z) also on level T is α. Second, the amount of time for a message M to exit a node N(T−1,θ+1,z) on level T−1 after exiting a node N(T,θ,z) on level T is α−β. Third, the amount of time for a message to travel from a device CU to a node N(r,θ,z) is α−β. Fourth, when a message M moves from a node N(r,θ,z) to a node N(r,θ+1, hr(z)) in time duration (a, the message M also causes a control code to be sent from node N(r,θ,z) to a node N(r+1,θ+1,hr(z)) to deflect messages on the outer
level r+ 1. The time that elapses from the time that message M enters node N(r,θ,z) until the control bit arrives at node N(r+1,θ+1,hr+1(z)) is time duration β. The aforementioned fourth condition also is applicable when a message M moves from a node N(J,θ,z) to a node N(J,θ+1,hJ(z)) at the outermost level J so that the message M also causes a control code to be sent from node N(J,θ,z) to a device CU(θ,z). The time that elapses from the time that message M enters node N(r,θ,z) until the control bit arrives at device CU(θ,z) is time duration β. Fifth, the global clock generates timing pulses at a rate of α. - When the source device CU(θ1,z1) sends a message packet M to the destination device CU(θ2,z2), the message packet M is sent from a data output terminal of device CU(θ1,z1) to a data input terminal of node N(J,θ1,z1) at the outermost level J. Message packets and control bits enter nodes N(T,θ,z) on a level T at times having the form nα+Lβ where n is a positive integer. The message M from device CU(θ1,z1) is sent to the data input terminal of node N(J,θ1,z1) at a time t0−β and is inserted into the data input terminal of node N(J,θ1,z1) at time t0 so long as the node N(J,θ1,z1) is not blocked by a control bit resulting from a message traversing on the level J. Time to has the form (θ2−θ1)α+Jβ. Similarly, there is a time of the form (θ2−θ1)α+Jβ at which a data input terminal of node N(J,θ1,z1) is receptive to a message packet from device CU(θ1,z1).
- Nodes N(r,θ,z) include logic that controls routing of messages based on the target address of a message packet M and timing signals from other nodes. A first logic switch (not shown) of node N(r,θ,z) determines whether the message packet M is to proceed to a node N(T−1,θ+1,z) on the next level T−1 or whether the node N(T−1,θ+1,z) is blocked. The first logic switch of node N(r,θ,z) is set according to whether a single-bit blocking control code sent from node N(T−1,θ,HT(z)) arrives at node N(r,θ,z) at a time to. For example, in some embodiments the first logic switch takes a
logic 1 value when a node N(T−1,θ+1,z) is blocked and alogic 0 value otherwise. A second logic switch (not shown) of node N(r,θ,z) determines whether the message packet M is to proceed to a node N(T−1,θ+1,z) on the next level T−1 or whether the node N(T−1,θ+1,z) is not in a suitable path for accessing the destination device CU(θ2,z2) of the message packet M. The message packet M includes the binary representation of destination height z2 (z2(J), z2(J−1), . . . , z2(T), . . . , z2(1), z2(0). The node N(T,θ,z) on level T includes a single-bit designation zT of the height designation z (zJ, zJ−1, . . . , zT, . . . , z1, z0). In this embodiment, when the first logic switch has alogic 0 value and the bit designation z2(T) of the destination height is equal to the height designation zT, then the message packet M proceeds to the next level at node N(T−1,θ+1,z) and the destination height bit z2(T) is stripped from the header of message packet M. Otherwise, the message packet M traverses on the same level T to node N(T,θ+1,hT(z)). If message packet M proceeds to node N(T−1,θ+1,z), then message packet M arrives at a time t0+(α−β) which is equal to a time (z2−z1+1)α+(J−1)β. If message packet M traverses to node N(T,θ+1,hT(z)), then message packet M arrives at a time t0+α, which is equal to a time (z2−z1+1)α+Jβ. As message packet M is sent from node N(r,θ,z) to node N(T,θ+1,hT(z)), a single-bit control code is sent to node N(T+1,θ+1,HT+1(z)) (or device CU(θ,z) which arrives at time t0+β. This timing scheme is continued throughout the interconnect structure, maintaining synchrony as message packets are advanced and deflected. - The message packet M reaches level zero at the designated destination height z2. Furthermore, the message packet M reaches the targeted destination device CU(θ2,z2) at a time zero modulus K (the number of nodes at a height z). If the targeted destination device CU(θ2,z2) is ready to accept the message packet M, an input port is activated at time zero modulus K to accept the packet. Advantageously, all routing control operations are achieved by comparing two bits, without ever comparing two multiple-bit values. Further advantageously, at the exit point of the interconnect structure as message packets proceed from the nodes to the devices, there is no comparison logic. If a device is prepared to accept a message, the message enters the device via a clock-controlled gate.
- Many advantages arise as a consequence of the disclosed timing and interconnect scheme. In an optical implementation, rather than an electronic implementation, of the interconnect structure, signals that encode bits of the header typical have a longer duration than bits that encode the payload. Header bits are extended in duration because, as messages communicate through the interconnect structure, timing becomes slightly skewed. Longer duration header bits allow for accurate reading of the bits even when the message is skewed. In contrast, payload bits encode data that is not read during communication through the interconnect structure. The disclosed timing scheme is advantageous because the number of header bits in a message is greatly reduced. Furthermore, in some embodiments the number of header bits is decremented as bits are used for control purposes at each level then discarded while messages pass from level to level in the interconnect structure. In embodiments that discard a control bit for each level of the interconnect structure, logic at each node is simplified since the control bit at each level is located at the same position throughout the interconnect structure.
- That messages communicated on the same level have priority over messages communicated from another level is similarly advantageous because message contention is resolved without carrying priority information in the message header. Message contention is otherwise typically resolved by giving priority to messages that have been in an interconnect structure the longest or to predetermined prioritization. These techniques use information stored in the header to resolve contention.
- Although it is advantageous that the interconnect structure and message communication method determines message transmission routing using self-routing decision-making which is local to the nodes and depends on message timing, in some embodiments of the control structure, both local and global communication control is employed. For example, one embodiment of an interconnect structure uses local control which is based on timing to control transmission of message packets in a first transmission mode and alternatively uses global control via a scheduler to administer communication of lengthy strings of message data in a second mode. In the global mode, the usage of a scheduler makes the usage of control bit input and output terminals unnecessary.
- One consequence of self-routing of message packets is that the ordering of message packet receipt at a target device may be variable. In some embodiments, the correct order of message segments is ordered by sending ordering information in the message header. Other embodiments employ an optical sorter to order message packets.
- Although many advantages are realized through a control structure and communication method which utilizes timing characteristics, rather than control bits in the header, to control message routing, some interconnect node technologies more suitably operate in a routing system utilizing no timing component. Thus in these technologies, instead of introducing a message at predetermined time so that the message arrives at a preset destination at a designated, routing information is contained in additional header bits. Accordingly, a designated target device position is included in header bits, for example bits following the designated target ring position.
- In one embodiment, the label of a target device is represented as a single logic one in a string of logic zeros. Thus, when a message arrives at a device N, the device samples the Nth bit of the device element of the header (as distinguished from the ring element) and accepts the message if the Nth bit is a logic one. This technique is highly suitable for optical node implementations.
- Nodes
- The nodes N(r,θ,z) have been described in generic terms to refer to various data communication switches for directing data to alternative data paths. Node structures which are presently available include electronic nodes, optical nodes and mixed optical/electronic nodes. What is claimed include, for example, interconnect and timing methods, an interconnect apparatus and an interconnect topology. These methods and apparati involve nodes in a generic sense. Thus, the scope of the claims is not limited by the particular type of node described herein and is to extend to any node known now or in the future, which performs the function of the nodes described herein.
- One example of a
node 1300 is shown, referring to FIG. 19, which includes a lithium niobate (LiNbO3)gate 1302. Thelithium niobate gate 1302 has twodata input terminals data output terminals control input terminal 1330.Various control circuitry 1340 is added to thelithium niobate gate 1302 to form acontrol output terminal 1332 of thenode 1300.Node 1300 also includes optical toelectronic converters lithium niobate gate 1302 is forms a 2×2 crossbar.Data paths 1342 and 1344 are optical and the control of thenode 1300 is electronic. Thelithium niobate gate 1302 is combined with aphotodetectors node 1300 for various embodiments of an interconnect structure. - In operation, as a
message packet 1360 approaches thenode 1300, part of themessage packet signal 1360 is split off and an appropriate bit of the message packet header (not shown) designating a bit of the binary representation of destination ring in accordance with the discussion hereinbefore, is read by thephotodetector 1350. This bit is converted from optical form to an electronic signal. This bit, a bit designating the cylinder height upon which thenode 1300 lies and a bit designating whether a destination node on the next level is blocked are processed electronically and a result of the logical tests of these bits is directed to thecontrol input terminal 1330 of thelithium niobate gate 1302. In a first type of lithium niobate gate technology, if the result signal is a logic zero, the gate switches in the cross state. In a second type of lithium niobate gate technology, a logic zero result signal switches the gate in a bar (straight through) state. - Referring to FIG. 20, an additional example of a
node 1400 is shown.Node 1400 uses a nonlinear optical loop mirror (NOLM) 1410 to perform a switching function. A nonlinear optical loop mirror is a device that makes use of the refractive index of a material to form a completely optical switch that is extremely fast. One example of a NOLM switch includes adata input terminal 1412 and acontrol input terminal 1414. Depending upon the signal at thecontrol input terminal 1414, data either leaves theNOLM 1410 through the samedata input terminal 1412 from which the data entered (hence the term mirror) or the data exits through adata output terminal 1416. Data is polarized and split into two signal “halves” of equal intensity. In the absence of a control pulse, the two halves of the signal recombine and leave theNOLM 1410 through thedata input terminal 1414. When a control pulse is applied to thecontrol input terminal 1414, the control pulse is polarized at right angles to the data pulse and inserted into theNOLM 1410 so that the control pulse travels with one half of the data pulse. The control pulse is more intense than the data pulse and the combined first half of the data pulse and the control pulse quickly pass the second half of the data pulse so that the second half of the data pulse is only minimally accelerated. Thus, the two halves of the data pulse travel with slightly different velocities and are 180° out of phase when the two halves are recombined. This phase difference causes the combined data pulse signal to pass through thedata output terminal 1416. One disadvantage of theNOLM 1410 is that switching is operational only when a long optical transmission loop is employed, thus latency is a problem. - Referring to FIG. 21, another example of a
node 1500 is shown which uses a terahertz optical asymmetrical demultiplexer (TOAD)switch 1510. TheTOAD switch 1510 is a variation of theNOLM switch 1410. TheTOAD 1510 includes anoptical fiber loop 1512 and asemiconductor element 1514, a nonlinear element (NLE) or a semiconductor optical amplifier for example. TheTOAD switch 1510 has aninput data terminal 1520 which also serves as an output data port under some conditions. TheTOAD switch 1510 also has a separate secondoutput data terminal 1522. Thesemiconductor element 1514 is placed asymmetrically with respect to thecenter 1516 of thefiber optic loop 1512. Adistance 1518 from thesemiconductor element 1514 to thecenter 1516 of thefiber optic loop 1512 is the distance to transmit one bit of data. TheTOAD 1510 functions by removing a single bit from a signal having a high data rate. TheTOAD 1510 is switched by passing a constant electrical current through thesemiconductor element 1514. An optical signal entering the semiconductor material causes the index of refraction of the material to immediately change. After the optical signal terminates, the index of refraction slowly (a time span of several bits) drifts back to the level previous to application of the optical signal. A control pulse is an optical signal having an intensity higher than that of an optical data signal and polarization at right angles to the optical data signal. An optical data input signal is polarized and split into two signal “halves” of equal intensity. The control pulse is injected in a manner to move through thefiber optic loop 1512 directly over the bit that is to be removed. Because thedistance 1518 is exactly one bit long, one half of the split optical data signal corresponding to a bit leaves thesemiconductor element 1514 just as the other half of the bit enters thesemiconductor element 1514. The control pulse only combines with one half of the optical data signal bit so that the velocity of the two halves differs. The combined data and control signal bit exits theTOAD 1510 at theinput data terminal 1520. Thus, this first bit is removed from the data path. A next optical signal bit is split and a first half and second half, moving in opposite directions, are delayed approximately the same amount as the index of refraction of thesemiconductor element 1514 gradually changes so that this bit is not removed. After a few bits have passes through thesemiconductor element 1514, the semiconductor material relaxes and another bit is ready to be multiplexed from the optical data signal. Advantageously, theTOAD 1510 has a very shortoptical transmission loop 1512. - Regenerators
- It is a characteristic of certain nodes that messages lose strength and pick up noise as they propagate through the nodes. Using various other nodes, message signals do not lose strength but noise accumulates during message transmission. Accordingly, in various embodiments of the interconnect structure, signal regenerators or amplifiers are used to improve message signal fidelity after messages have passed through a number of nodes.
- Referring to FIG. 22, one embodiment of a
regenerator 1600 is shown which is constructed using alithium niobate gate 1602. Alithium niobate gate 1602 regenerates message data having a transmission speed of the order of 2.5 gigabits. Thelithium niobate gate 1602 detects and converts an optical message signal to an electronic signal which drives anelectronic input port 1604 of thelithium niobate gate 1602. Thelithium niobate gate 1602 is clocked using a clock signal which is applied to one of twooptical data ports gate 1602. The clock signal is switched by the electronic control pulses and a high fidelity regenerated signal is emitted from thelithium niobate gate 1602. - Typically, an interconnect structure utilizing a
lithium niobate gate 1602 in aregenerator 1600 also uses lithium niobate gates to construct nodes. One large power laser (not shown) supplies high fidelity timing pulses to all of the regenerators in an interconnect structure. Theillustrative regenerator 1600 andnode 1650 combination includes anoptical coupler 1620 which has a first data input connection to a node on the same level C as thenode 1650 and a second data input connection to a node on the overlyinglevel C+ 1. Theillustrative regenerator 1600 also includes aphotodetector 1622 connected to an output terminal of theoptical coupler 1620, optical toelectronic converter 1624 which has an input terminal connected to theoptical coupler 1620 through thephotodetector 1622 and an output terminal which is connected to thelithium niobate gate 1602. An output terminal of thelithium niobate gate 1602 is connected to a second lithium niobate gate (not shown) of a node (not shown). Two signal lines of the lithium niobate gate (not shown) are combined, regenerated and switched. - When regenerators or amplifiers are incorporated to improve signal fidelity and if the time expended by a regenerator or amplifier to recondition a message signal exceeds the time α−β, then the regenerator or amplifier is placed prior to the input terminal of the node and timing is modified to accommodate the delay.
- Other Embodiments
- The interconnect structure shown in FIGS. 1 through 16 is a simplified structure, meant to easily convey understanding of the principles of the invention. Numerous variations to the basic structure are possible. Various examples of alternative interconnect structures are discussed hereinafter, along with advantages achieved by these alternative structures.
- Referring to FIG. 23, an alternative embodiment of an interconnect structure1000 includes
devices 1030 which issue message packets tomultiple nodes 1002 of the outermost level J. In theinterconnect apparatus 100 shown in FIGS. 1 through 16, a device CU(θ,z) initiates a message transmission operation by sending a message packet to a node N(J,θ,z). In the alternative interconnect structure 1000, the device CU(θ,z) initiates a message transmission operation by sending a message packet to node N(J,θ,z) but, in addition, also includes interconnections to additional multiple nodes N(J,θ,z) where z designates cylinder heights selected fromheights 0 to 2J of the outermost level J and θ designates node angles selected fromangles 0 to K of the heights z. In the case that a device sends messages to more than one node in the outermost level, the disclosed timing scheme maintains the characteristic that messages arrive at the target node at time zero modulus K. - Devices are connected to many nodes in the outermost level J to avoid congestion upon entry into the interconnect structure caused by multiple devices sending a series of messages at a high rate to nodes having converging data paths. In some embodiments, the nodes to which a device is connected are selected at random. In other embodiments, the multiple interconnection of a device to several nodes is selected in a predetermined manner. An additional advantage arising from the connection of a device to several nodes increases the input bandwidth of a communication network.
- Referring to FIG. 24, an alternative embodiment of an interconnect structure1100 includes
devices 1130 which receive message packets frommultiple nodes 1102 of theinnermost level 0. In this example, the number of nodes K at a particular height z is nine and eachdevice 1130 is connected to receive message from three nodes on level zero. The interconnect structure 1100 is advantageous for improving network exit bandwidth when the number of nodes K on at a particular height is large. - In the example in which the number of nodes K on a height z is nine and each device receives messages from three nodes on level zero, each node on ring zero is connected to a buffer that has three levels. At
time 0, message data is injected into the level zero buffer. At time three, data is injected into the level one buffer. At time 6, data is injected into the level two buffer. A device CU(θ,0) reads from the level zero buffer at node N(0,θ,0), from the level one buffer at node N(0,(θ+3)mod 9,0), and from the level two buffer at node N(0,(θ+6)mod 9,0). This reading of message data is accomplished in a synchronous or nonsynchronous manner. If in the synchronous mode, a time t is expended to transfer data from the buffer to the device. In this case, the device CU(θ,0) reads from the level zero buffer at time t, reads from the level three buffer attime 3+t, and reads from the level six buffer at time 6+t. In an asynchronous mode, device CU(θ,O) interconnects to the three buffers as described hereinbefore and reads message data whenever a buffer signals that data is available. - Referring to FIG. 25, an alternative embodiment of an
interconnect structure 1200 includesdevices 1230 which issue message packets tomultiple nodes 1202, not only in the outermost level J but also in other levels. In thealternative interconnect structure 1200, the device CU(θ,z) initiates a message transmission operation by sending a message packet to node N(J,θ,z) but, in addition, also includes interconnections to additional multiple nodes N(T,θ,z) where T designates levels of theinterconnect structure 1200, z designates cylinder heights selected fromheights 0 to 2J of the outermost level J and θ designates node angles selected from angles θ to K of the heights z. In the case that a device sends messages to nodes in more than level, message communication is controlled according to a priority, as follows. First, messages entering a node N(r,θ,z) from the same level T have a first priority. Second, messages entering a node N(r,θ,z) from a higher level T+1 have a second priority. Messages entering a node N(r,θ,z) from a device CU(θ,z) have last priority. The alternative embodiment ofinterconnect structure 1200 allows a device to send messages to neighboring devices more rapidly. The disclosed timing scheme maintains the characteristic that messages arrive at the node designated in the message header at time zero modulus K. - In these various embodiments, devices accept data from level zero nodes using one of various predetermined techniques. Some embodiments rely exclusively on timing to determine when the devices accept data so that devices accept data at time zero modulus K. Some embodiments include devices that accept message data at various predetermined times with respect to modulus K timing. Still other embodiments have devices that accept data whenever a buffer is ready to accept data.
- Wave Division Multiplexing Embodiment
- In another embodiment of an interconnect structure, message signal bandwidth is increased using wave division multiplexing. A plurality of K colors are defined and generated in a message signal that is transmitted using an interconnect structure having K devices at a cylinder height. Accordingly, each device is assigned a particular color. Message packets travel to a preselected target ring in the manner described hereinbefore for a single wavelength interconnect system. Message packets pass from level zero to the appropriate device depending on the color assigned to the message packet.
- A message includes a header and a payload. The header and payload are distinguished by having different colors. Similarly, the payload is multiplexed using different colors, which are also different from the color of the header. Message bandwidth is also increased by combining different messages of different colors for simultaneous transmission of the messages. Furthermore, different messages of different colors are bound on a same target ring, combined an transmitted simultaneously. All messages are not demultiplexed at all of the nodes but, rather, are demultiplexed at input buffers to the devices.
- Variable Base ij Height Structure Embodiment
- In a farther additional embodiment, an interconnect structure has iJ cylindrical heights on a level for each of J+1 levels, where i is a suitable integer number such as 2 (the previously described embodiment), 3, 4 or more. As was described previously, each height contains K nodes, and each node has two data input terminals, two data output terminals, one control input terminal and one control output terminal.
- For example, an interconnect structure may have 3J heights per level. On level one, message data is communicated to one of three level zero heights. On level two, message data is communicated to one of nine level zero heights and so forth. This result is achieved as follows. First, the two output data terminals of a node N(r,θ,z) are connected to input data terminals of a node N(T−1,θ+1,z) and a node N(T,θ+1,hT(z)), in the manner previously discussed. However in this further embodiment, a third height transformation hThT(hT(z))) rather than a second height transformation hT(hT(z)) is equal to the original height designation z. With the nodes interconnected in this manner, the target ring is accessible to message data on every third step on level one. In an interconnect structure having this form, although nodes having two output data terminals are suitable, advantages are gained by increasing the number of output data terminals to three. Thus, one data output terminal of a node on a given level is connected to two nodes on that level and to one node on a successive level. Accordingly, each level has 3J heights and a message packet and a message can descend to a lower level every other step.
- In this manner, many different interconnect structures are formed by utilizing iJ heights per level for various numbers i. Where i is equal to 4, the fourth height transformation hT(hT(hT(hT(z)))) is equal to the original height designation z. If i is 5, the fifth height transformation hT(hT(hT(hT(hT(z))))) is the same as the original height z, and so forth.
- In the variable base iJ height structure embodiment, whether the target ring is accessible from a particular node is determined by testing a more than one bit 10 of a code designating the target ring.
- Variable Base Transformation Technique Embodiment
- In a still further embodiment of an interconnect structure, the height transformation technique outlined hereinbefore is modified as follows. In this embodiment, a base three notation height transform technique is utilized rather than the binary height transformation technique discussed previously. In the base three transformation technique, a target ring is designated by a sequence of base three numbers. Thus, one a level n, the n low-order base three numbers of the height designation are reversed in order, the low-order-bit-reversed height designation is incremented by one, and the n low-order base three numbers are reversed back again. An exemplary interconnect structure has four levels (J=3 plus one), nine heights (3J=33) per level and five nodes (K=5) per height. In accordance with the base three height transformation technique, node N2(201)3 on
level 2 has a first data input terminal connected to a data output terminal of node N3(220)2 onlevel 3, a second data input terminal connected to a data output terminal of node N2(220)2 on level two. Node N2(201)3 also has a first data output terminal connected to a data input terminal of node N1(201)4 on level one and a second data output terminal connected to a data input terminal of node N2(211)4. Node N2(201)3 also has a control input bit connected to a control output bit of node N1(200)3 and a control output bit connected to a control input bit of node N3(211)3. In this embodiment, the header includes a synch bit followed by the address of a target ring in base three. For example, the base three numbers are symbolized in binary form as 00, 01 and 10 or using three bits in theform - Further additional height transformation techniques are possible using various numeric bases, such as
base 5 or base 7 arithmetic, and employing the number reversal, increment and reversal back method discussed previously. - Multiple Level Step Embodiment
- In another embodiment, an interconnect structure of the nodes have ten terminals, including five input terminals and five output terminals. The input terminals include three data input terminals and two control input terminals. The output terminals include three data output terminals and two control output terminals. In this interconnect structure, nodes are generally connected among five adjacent cylindrical levels. Specifically, nodes N(T,θ,z) at the T cylindrical level have terminals connected to nodes on the T, T+1, T+2, T−1 and T−2 levels. These connections are such that the nodes N(T,θ,z) have data input terminals connected to nodes on the same level T, the next outer level T+1 and the previous outer
level T+ 2. In particular, nodes N(T,θ,z) have data input terminals connected to data output terminals of nodes N(T,θ−1,hT(z)), N(T+1,θ−1,z) and N(T+2,θ−1,z). Nodes N(T,θ,z) also have control output terminals, which correspond to data input terminals, connected to nodes on the next outer level T+1 and the previous outerlevel T+ 2. Nodes N(T,θ,z) have control output terminals connected to control input terminals of nodes N(T+1,θ−1,z) and N(T+2,θ−1,z). Nodes N(T,θ,z) also have data output terminals connected to nodes on the same level T, the next inner level T−1 and the subsequent inner level T−2. In particular, nodes N(T,θ,z) have data output terminals connected to data output terminals of nodes N(T,θ+1,HT(z)), N(T−1,θ+1,z) and N(T−2,θ+1,z). Nodes N(T,θ,z) also have control input terminals, which correspond to data output terminals, connected to nodes on the next inner level T−1 and the subsequent inner level T−2. Nodes N(T,θ,z) have control input terminals connected to control output terminals of nodes N(T−1,θ+1,z) and N(T−2,θ+1,z). - This ten-terminal structure applies only to nodes at the intermediate levels to J−2 since nodes at the outer levels J and J−1 and at the
inner levels - This ten-terminal structure allows messages to skip past levels when possible and thereby pass through fewer nodes at the cost of increasing logic at the nodes. Only one message is allowed to enter a node at one time. The priority of message access to a node is that a message on the same level has top priority, a message from a node one level removed has second priority and a message from a node two levels away has last priority. Messages descend two levels whenever possible. The timing rules for an interconnect structure using the six-terminal nodes. Advantages of the ten-terminal node interconnect structure are that messages pass more quickly through the levels.
- Other interconnect structure embodiments include nodes having more than ten terminals so that data and control terminals are connected to additional nodes on additional levels. For example, various nodes N(T,θ,z) also have associated control input terminals and data output terminals, which are connected to nodes on inner levels T−3, T−4 and so on. In other examples, various nodes N(T,θ,z) also have associated control output terminals and data input terminals, which are connected to nodes on outer levels T+3, T+4 and so on. In various interconnect structure embodiments, nodes may be connected among all levels or selected levels.
- Multiple Interconnections to the Same Level Embodiment
- Additional interconnect structure embodiments utilize additional interconnections among nodes on the same level. Specifically, nodes N(T,θ,z) on the level T have interconnections in addition to the connections of (1) an output data terminal connected to an input data terminal of nodes N(T,θ+1,hT(z)) and (2) an input data terminal connected to an output data terminal of nodes N(T,θ−1,HT(z)). Thus nodes N(T,θ,z) on the level T have interconnections including a connection of (1) an output data terminal connected to an input data terminal of nodes N(T,θ+1,gT(z)) and (2) an input data terminal connected to an output data terminal of nodes N(T,θ−1,hr(z)). Like cylinder height hT(z), height gT(z) is on the half of the interconnect structure of level T that is opposite to the position of height z (meaning bit T of the binary code describing height hT(z) and gT(z) is complementary to bit T of height z).
- Multiple Interconnections to a Next Level Embodiment
- A multiple interconnections to a next level embodiment is similar to the multiple interconnections to the same level embodiment except that node N(T,θ,z) has one output data terminal connected to one node N(T,θ+1,hT(z)) on level T and two output data terminals connected to two nodes N(T−1,θ+1,z) and N(T−1,θ+1,gT−1(z)) on level T−1. Thus one data output interconnection traverses the same level, a second interconnection progresses one level and a third interconnection both progresses one level and traverses. Like height hT(z), height gT(z) is on the half of the interconnect structure of level T that is opposite to the position of height z. Conflicts between node access are resolved by applying a first priority to messages moving on the same level, a second priority to messages progressing one level and a third priority to messages both progressing one level and traversing.
- The description of certain embodiments of this invention is intended to be illustrative and not limiting. Numerous other embodiments will be apparent to those skilled in the art, all of which are included within the broad scope of this invention. For example, many different types of devices may be connected using the interconnect structure including, but not limited to, workstations, computers, terminals, ATM machines, elements of a national flight control system and the like. Also, other interconnection transformations other than hT and HT may be implemented to describe the interconnections between nodes.
- The description and claims occasionally make reference to an interconnect structure which is arranged in multiple dimensions. This reference to dimensions is useful for understanding the interconnect structure topology. However, these dimensions are not limited to spatial dimensions but generally refer to groups of nodes which are interconnected in a particular manner.
Claims (38)
1. An interconnect apparatus, comprising:
a plurality of nodes; and
a plurality of interconnect lines selectively coupling the nodes in a multiple level structure, the multiple level structure being arranged to include:
a plurality of J+1 levels in a hierarchy of levels T arranged from a level T equal to 0 to a level T equal to J;
a plurality of 2J-T rings in each level T; and
a plurality of 2TK nodes in a ring.
2. An apparatus according to wherein a node A on a level T greater than 0 and less than J has a plurality of interconnections including:
claim 1
an input interconnection from a node B on the level T;
an input interconnection from a node C on a level T +1;
an output interconnection to a node D on the level T; and
an output interconnection to a node E on a level T−1.
3. An apparatus according to wherein a node A on a level T greater than 0 and less than J has a plurality of interconnections including:
claim 2
a control input interconnection from the node F on the level T−1; and
a control output interconnection to the node G on the level T+1.
4. An apparatus according to wherein a node A on a level T greater than zero and less than J has a plurality of interconnections further including:
claim 2
an input interconnection from a node H on a level T−2; and
an output interconnection to a node I on a level T+2.
5. An apparatus according to wherein a node A on a level T greater than zero and less than J has a plurality of interconnections further including:
claim 4
a control input interconnection from a node J on a level T+2; and
a control output interconnection to a node K on a level T−2.
6. An apparatus according to wherein at most one input interconnection of input connections B and C is active at one time.
claim 2
7. An apparatus according to wherein at most one output interconnection of output connections D and E is active at one time.
claim 2
8. An apparatus according to wherein messages communicated on the input interconnection from the node B on the level T have a higher priority than messages communicated on the input interconnection from the node C on the level T+1.
claim 2
9. An apparatus according to wherein:
claim 2
a series of 2TK sequential node A to node D interconnections on the level T traverses each of 2TK nodes on one ring once.
10. An apparatus according to wherein the multiple level structure has a three-dimensional cylindrical topology in which each node has a location designated in three-dimensional cylindrical coordinates (r, θ, z) where radius r is an integer which specifies the cylinder number from 0 to J, θ is an integer which specifies the 2πθ/K spacing of nodes around the circular cross-section of a cylinder from 0 to K−1, and height z is a binary integer which specifies distance along the z-axis from 0 to 2J−1.
claim 1
11. An apparatus according to wherein:
claim 10
a node A(r,θ,z) is interconnected with an immediate predecessor node B(r,(θ−1)mod K,Hr(z)) on level r for receiving message data;
node A(r,θ,z) is interconnected with a predecessor node C(r+1,(θ−1)mod K,z) on level r+1 for receiving message data;
node A(r,θ,z) is interconnected with an immediate successor node D(r,(θ+1)mod K,hr(z)) on level r for sending message data;
node A(r,θ,z) is interconnected with a successor node E(r−1, (θ+1)mod K,z) on level r−1 for sending message data;
node A(r,θ,z) is interconnected with a node F(r−1, ,Hr(z)) on level r−1 for receiving a control input signal; and
node A(r,θ,z) is interconnected with a node G(r+1,θ,hr+1(z)) on level r+1 for sending a control output signal.
12. An apparatus according to wherein:
claim 11
height z=[zJ−1, zJ−2, . . . , zr, zr−1, . . . , z2, z1, z0] is converted to hr(z) on the level r by
reversing the order of low-order z bits from zr−1 to z0] into the form adding 1 (modulus 2r); and
reversing back the low-order z bits; and
height z is converted to Hr(z) on the level r by
reversing the order of low-order z bits from zr−1 top z0] into z0] into the form z=[zJ−1, zJ−2, . . . , zr, z0, z1, z2, . . . zr−1];
subtracting 1 (modulus 2r); and
reversing back the low-order z bits.
13. An apparatus according to wherein:
claim 11
height z=[zJ−1, zJ−2, . . . , zr, zr−1, . . . , z2, z1, z0] is converted to hr(z) on the level r by
reversing the order of low-order z bits from zr−1 to z0] into the form z=[zJ−1, zJ−2, . . . , zr, z0, z1, z2, . . . zr−1];
Adding J (modulus 2r) in which J is an odd integer; and
reversing back the low-order z bits; and
height z is converted to Hr(z) on the level r by
reversing the order of low-order z bits from zr−1 to z0] into the form z=[zJ−1, zJ−2, . . . , zr, z0, z1, z2, . . . , zr−1];
subtracting J (modulus 2r); and
reversing back the low-order z bits.
14. An apparatus according to wherein a node A(J,θ,z) on an outermost level J includes:
claim 10
a first interconnection with a device outside of the multiple level structure for receiving message data; and
a second interconnection with a device outside of the multiple level structure for sending a control output signal.
15. An apparatus according to wherein a node A(0,θ,z)) on an innermost level 0 includes:
claim 10
a first interconnection with a device outside of the multiple level structure for sending message data; and
a second interconnection with a device outside of the multiple level structure for receiving a control output signal.
16. An apparatus according to wherein: on a level T, one ring is spanned in 2T passes through the angles θ from 0 to K−1 so that 2T heights z designate one ring.
claim 10
17. An apparatus according to , further comprising:
claim 1
a plurality of devices coupled to the nodes of a level.
18. An apparatus according to , further comprising:
claim 1
a plurality of devices coupled to the nodes of level 0; and
a plurality of interconnect lines coupling the plurality of devices to respective nodes in the level J.
19. An apparatus according to , wherein a device is coupled to a plurality of nodes in the level J.
claim 18
20. An apparatus according to , wherein:
claim 1
WT rings are interconnected on a level T;
WT−1 rings are interconnected on a level T−1; and
the WT−1 rings on level T−1 are divided into WT mutually exclusive collections (C1, C2, . . . . , CWT) such that each of the rings in collection Cn of level T−1 receive messages from ring RM of level T.
21. A method of transmitting a message from a node N to a target destination in a first, a second and a third dimension of three dimensions in an interconnect structure arranged as a plurality of nodes in a topology of the three dimensions, the method comprising the steps of:
determining whether a node en route to the target destination in the first and second dimensions and advancing one level toward the destination level of the third dimension is blocked by another message;
advancing the message one level toward the destination level of the third dimension when the en route node is not blocked; and
moving the message in the first and second dimensions along a constant level in the third dimension otherwise.
22. A method according to , further comprising the steps of:
claim 21
specifying the first dimension to describe a plurality of levels, the second dimension to describe a plurality of nodes spanning a cross-section of a level, and the third dimension to describe a plurality of nodes in the cross-section of a level;
sending a control signal from the node en route to the node N on a level q in the first dimension, the control signal specifying whether the node en route is blocked;
timing transmission of a message using a global clock specifying timing intervals to keep integral time modulus the number of nodes in a cross-section of a level, the global clock time interval being equal to the second time interval and the first time interval being smaller than the global time interval;
setting a first time interval a for moving the message in the second and third dimensions;
setting a second time interval α−β for advancing the message one level toward the destination level; and
setting a third time interval for sending the control signal from the node en route to the node N, the third time interval being equal to β.
23. A method according to , further comprising the steps of:
claim 22
timing the message moving and advancing steps so that the messages enter node N on level q at times having the form nα+qβ; and
timing the control signal sending step so that the control signals enter node N on level q at times having the form nα+qβ so long as the node en route is not blocked.
24. A method according to , further comprising the steps of:
claim 21
timing transmission of a message using a global clock;
setting a first time interval for moving the message in the second and third dimensions; and
setting a second time interval for advancing the message one level toward the destination level.
25. A method according to , further comprising the steps of:
claim 24
specifying the first dimension to describe a plurality of levels, the second dimension to describe a plurality of nodes spanning a cross-section of a level, and the third dimension to describe a plurality of nodes in the cross-section of a level;
specifying timing interval of the global clock to keep integral time modulus the number of nodes in a cross-section of a level, the global clock time interval being equal to the second time interval and the first time interval being smaller than the global time interval.
26. A method according to further comprising the steps of:
claim 21
defining a header and a payload in the message;
encoding the destination in the second dimension in the header;
determining whether a potentially en route node is en route to the target destination including the steps of:
comparing the encoded destination in the second dimension to an encoded position of the potentially en route node;
resolving that the potentially en route node is en route when the encoded destination is the same as the encoded position of the potentially en route node.
27. A method according to wherein:
claim 26
the destination in the third dimension in the header is encoded in a plurality of single-bit codes, each single-bit code relating to a level of the third dimension;
the position of the potentially en route node is encoded in a single-bit code; and
the comparing step is a single-bit comparison of the level-specific, single-bit destination code and the single-bit position code.
28. A method according to further comprising the step of:
claim 27
discarding the level-specific, single-bit destination code in the as the message advances one level.
29. A method according to further comprising the step of:
claim 21
on a level T, one ring is spanned in 2T passes through the nodes in the second dimension so that 2T nodes in the third dimension designate one ring.
interconnecting the three dimensional interconnect structure so that advancing of levels from a start level to the destination level furnishes access to all nodes in a ring.
30. A method according to wherein a message injected into the interconnect structure at a node N(J, θ1, z1) and targeted to exit the interconnect structure at a node N(O, θ2, z2) and injected at a time (θ2−θ1)modK *α+Jβ causes the message to arrive at node N(O, θ2, z2) at time 0.
claim 21
31. A communication interconnect structure for transmitting messages, comprising:
a plurality of nodes arranged in a structure including:
a hierarchy of levels from a source level to a destination level;
a plurality of nodes spanning a cross-section of a level; and
a plurality of nodes in a cross-section span;
a plurality of interconnect lines coupling the nodes in the structure including for a node N on a level L:
a message input interconnect line coupled to a node on a previous level L+1;
a message input interconnect line coupled to a node on the level L;
a message output interconnect line coupled to a node on a subsequent level L−1; and
a message output interconnect line coupled to a node on a subsequent level L−1.
32. An interconnect structure according to , further comprising:
claim 31
a control input interconnect line coupled to the node on the subsequent level L−1 which is coupled to the message output interconnect line; and
means for receiving a message on the control input interconnect line and, in accordance with the message, selectively transmitting a message on the message output interconnect line coupled to the subsequent level L−1 node or on the message output interconnect line coupled to the level L.
33. An interconnect structure according to , further comprising:
claim 32
a control output interconnect line coupled to the node on the previous level L+1 which is coupled to the message input interconnect line;
means for determining that a message is blocking the node N; and
means for communicating via the control input interconnect line informing whether the node N is blocked.
34. An interconnect structure according to , further comprising:
claim 33
means for timing a message transmission time of a message transmitted from a level to a subsequent level and for timing a control signal transmission time of a control signal from a subsequent level to a level so that the control signal arrives first at a node.
35. An interconnect structure according to , further comprising:
claim 34
a control output interconnect line coupled to the node on the previous level L+1 which is coupled to the message input interconnect line;
means for determining that a message is blocking the node N; and
means for communicating via the control input interconnect line informing whether the node N is blocked.
36. A method of communicating messages in an interconnect structure comprising the steps of:
arranging a plurality of nodes in a structure including a plurality of hierarchical levels from a source level to a destination level, a plurality of nodes spanning a cross-section of a level and a plurality of nodes in a cross-section span, the nodes having an input connection on the same level, an input connection on a previous level, an output connection on the same level and an output connection on a subsequent level;
specifying a destination node in the destination level for receiving a message;
originating the message at a node in the source level;
communicating a message from node to node including the steps of:
determining at a node whether a node on a subsequent level is directed toward the destination node;
determining at a node whether the node on the subsequent level is blocked by another message;
advancing the message to the node on the subsequent level when the node is directed toward the destination node and a node is unblocked; and
otherwise traversing the message to a node on the same level.
37. A method according to wherein the step of determining whether a node on a subsequent level is directed toward the destination node further comprises the steps of:
claim 36
encoding the destination node in a message in the header field;
encoding a designation of node position for the nodes at each level; and
determining that the node on the subsequent level is directed toward the destination node when the destination node encoding and the node position designation encoding correspond.
38. A communication interconnect structure comprising:
a plurality of nodes; and
a plurality of interconnect lines coupling the nodes, a node X of the plurality of nodes having:
a message input interconnect line coupled to a node A; and
a message input interconnect line coupled to a node B, the node X accepting a message input from the node A and a message input from the node B with a control interconnect line being coupled between the node A and the node B for communicating a control signal determining a priority between conflicting messages.
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US09/850,953 US20010034798A1 (en) | 1995-07-21 | 2001-05-07 | Multiple level minimum logic network |
US10/773,693 US7426214B2 (en) | 1995-07-21 | 2004-06-16 | Multiple level minimum logic network |
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US08/505,513 US5996020A (en) | 1995-07-21 | 1995-07-21 | Multiple level minimum logic network |
US09/397,333 US6272141B1 (en) | 1995-07-21 | 1999-09-14 | Multiple level minimum logic network |
US09/850,953 US20010034798A1 (en) | 1995-07-21 | 2001-05-07 | Multiple level minimum logic network |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/397,333 Division US6272141B1 (en) | 1995-07-21 | 1999-09-14 | Multiple level minimum logic network |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/852,009 Division US7068671B2 (en) | 1995-07-21 | 2001-05-07 | Multiple level minimum logic network |
Publications (1)
Publication Number | Publication Date |
---|---|
US20010034798A1 true US20010034798A1 (en) | 2001-10-25 |
Family
ID=24010615
Family Applications (5)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US08/505,513 Expired - Lifetime US5996020A (en) | 1995-07-21 | 1995-07-21 | Multiple level minimum logic network |
US09/397,333 Expired - Lifetime US6272141B1 (en) | 1995-07-21 | 1999-09-14 | Multiple level minimum logic network |
US09/852,009 Expired - Fee Related US7068671B2 (en) | 1995-07-21 | 2001-05-07 | Multiple level minimum logic network |
US09/850,953 Abandoned US20010034798A1 (en) | 1995-07-21 | 2001-05-07 | Multiple level minimum logic network |
US10/773,693 Expired - Fee Related US7426214B2 (en) | 1995-07-21 | 2004-06-16 | Multiple level minimum logic network |
Family Applications Before (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US08/505,513 Expired - Lifetime US5996020A (en) | 1995-07-21 | 1995-07-21 | Multiple level minimum logic network |
US09/397,333 Expired - Lifetime US6272141B1 (en) | 1995-07-21 | 1999-09-14 | Multiple level minimum logic network |
US09/852,009 Expired - Fee Related US7068671B2 (en) | 1995-07-21 | 2001-05-07 | Multiple level minimum logic network |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/773,693 Expired - Fee Related US7426214B2 (en) | 1995-07-21 | 2004-06-16 | Multiple level minimum logic network |
Country Status (14)
Country | Link |
---|---|
US (5) | US5996020A (en) |
EP (3) | EP0842473B1 (en) |
JP (1) | JP3594198B2 (en) |
KR (1) | KR100401682B1 (en) |
AT (1) | ATE327537T1 (en) |
AU (1) | AU725826B2 (en) |
CA (1) | CA2227271C (en) |
DE (2) | DE69636165D1 (en) |
ES (1) | ES2160556T1 (en) |
GR (1) | GR20010300054T1 (en) |
HK (1) | HK1010926A1 (en) |
MX (1) | MX9800626A (en) |
NZ (2) | NZ503094A (en) |
WO (1) | WO1997004399A2 (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20050132089A1 (en) * | 2003-12-12 | 2005-06-16 | Octigabay Systems Corporation | Directly connected low latency network and interface |
Families Citing this family (26)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5996020A (en) | 1995-07-21 | 1999-11-30 | National Security Agency | Multiple level minimum logic network |
US6289021B1 (en) * | 1997-01-24 | 2001-09-11 | Interactic Holdings, Llc | Scaleable low-latency switch for usage in an interconnect structure |
US6754207B1 (en) * | 1998-01-20 | 2004-06-22 | Interactic Holdings, Llc | Multiple-path wormhole interconnect |
US6801551B1 (en) | 1998-05-15 | 2004-10-05 | University Of Central Florida | Programmable multiwavelength modelocked laser |
US6661816B2 (en) | 1998-06-11 | 2003-12-09 | University Of Central Florida | Multiwavelength modelocked lasers |
US6314115B1 (en) | 1998-05-15 | 2001-11-06 | University Of Central Florida | Hybrid WDM-TDM optical communication and data link |
US6247061B1 (en) * | 1998-06-09 | 2001-06-12 | Microsoft Corporation | Method and computer program product for scheduling network communication packets originating from different flows having unique service requirements |
US6807667B1 (en) * | 1998-09-21 | 2004-10-19 | Microsoft Corporation | Method and system of an application program interface for abstracting network traffic control components to application programs |
US6842788B1 (en) * | 2000-10-11 | 2005-01-11 | Nortel Networks Limited | Computing and using resource colors for composite links |
US7016363B1 (en) * | 2000-10-19 | 2006-03-21 | Interactic Holdings, Llc | Scaleable interconnect structure utilizing quality-of-service handling |
US7221677B1 (en) * | 2000-10-19 | 2007-05-22 | Interactic Holdings, Llc | Scalable apparatus and method for increasing throughput in multiple level minimum logic networks using a plurality of control lines |
US20030035371A1 (en) * | 2001-07-31 | 2003-02-20 | Coke Reed | Means and apparatus for a scaleable congestion free switching system with intelligent control |
KR20070007769A (en) * | 2003-10-29 | 2007-01-16 | 인터랙틱 홀딩스 엘엘시 | Highly parallel switching systems utilizing error correction |
WO2005086791A2 (en) * | 2004-03-08 | 2005-09-22 | Interactic Holdings, Llc | Highly parallel switching systems utilizing error correction ii |
US20060171386A1 (en) * | 2004-09-01 | 2006-08-03 | Interactic Holdings, Llc | Means and apparatus for a scaleable congestion free switching system with intelligent control III |
WO2006069197A2 (en) * | 2004-12-20 | 2006-06-29 | Interactic Holdings, Llc | Scaleable controlled interconnect with optical and wireless applications |
US8559443B2 (en) | 2005-07-22 | 2013-10-15 | Marvell International Ltd. | Efficient message switching in a switching apparatus |
US20070076761A1 (en) * | 2005-09-15 | 2007-04-05 | Coke Reed | Apparatus for interconnecting multiple devices to a synchronous device |
US7895560B2 (en) * | 2006-10-02 | 2011-02-22 | William Stuart Lovell | Continuous flow instant logic binary circuitry actively structured by code-generated pass transistor interconnects |
WO2008052181A2 (en) | 2006-10-26 | 2008-05-02 | Reed Coke S | A network interface card for use in parallel computing systems |
US7957400B2 (en) * | 2009-03-26 | 2011-06-07 | Terascale Supercomputing Inc. | Hierarchical network topology |
US20100250784A1 (en) * | 2009-03-26 | 2010-09-30 | Terascale Supercomputing Inc. | Addressing Scheme and Message Routing for a Networked Device |
US7957385B2 (en) * | 2009-03-26 | 2011-06-07 | Terascale Supercomputing Inc. | Method and apparatus for packet routing |
US9930117B2 (en) | 2014-09-30 | 2018-03-27 | Interactic Holdings, Llc | Matrix vector multiply techniques |
EP3204868A1 (en) | 2014-10-08 | 2017-08-16 | Interactic Holdings LLC | Fast fourier transform using a distributed computing system |
KR20210096280A (en) | 2018-12-12 | 2021-08-04 | 인터랙틱 홀딩스, 엘엘씨 | Method and apparatus for improved data transfer between processor cores |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5224100A (en) * | 1991-05-09 | 1993-06-29 | David Sarnoff Research Center, Inc. | Routing technique for a hierarchical interprocessor-communication network between massively-parallel processors |
US5521910A (en) * | 1994-01-28 | 1996-05-28 | Cabletron Systems, Inc. | Method for determining a best path between two nodes |
US5600641A (en) * | 1994-07-07 | 1997-02-04 | International Business Machines Corporation | Voice circuit emulation system in a packet switching network |
US5694393A (en) * | 1993-11-11 | 1997-12-02 | U.S. Philips Corporation | Communications system |
US5781551A (en) * | 1994-09-15 | 1998-07-14 | Texas Instruments Incorporated | Computer communications system with tree architecture and communications method |
Family Cites Families (22)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4814980A (en) * | 1986-04-01 | 1989-03-21 | California Institute Of Technology | Concurrent hypercube system with improved message passing |
US4933836A (en) * | 1986-10-29 | 1990-06-12 | United Technologies Corporation | n-Dimensional modular multiprocessor lattice architecture |
JP2509947B2 (en) * | 1987-08-19 | 1996-06-26 | 富士通株式会社 | Network control system |
US5339396A (en) * | 1987-11-18 | 1994-08-16 | Hitachi, Ltd. | Interconnection network and crossbar switch for the same |
US5181017A (en) * | 1989-07-27 | 1993-01-19 | Ibm Corporation | Adaptive routing in a parallel computing system |
EP0419201B1 (en) * | 1989-09-18 | 1996-11-27 | Fujitsu Limited | Communication control system between parallel computers |
US5313590A (en) * | 1990-01-05 | 1994-05-17 | Maspar Computer Corporation | System having fixedly priorized and grouped by positions I/O lines for interconnecting router elements in plurality of stages within parrallel computer |
US5253248A (en) * | 1990-07-03 | 1993-10-12 | At&T Bell Laboratories | Congestion control for connectionless traffic in data networks via alternate routing |
US5471623A (en) * | 1991-02-26 | 1995-11-28 | Napolitano, Jr.; Leonard M. | Lambda network having 2m-1 nodes in each of m stages with each node coupled to four other nodes for bidirectional routing of data packets between nodes |
US5524082A (en) * | 1991-06-28 | 1996-06-04 | International Business Machines Corporation | Redundancy removal using quasi-algebraic methods |
JPH0581216A (en) * | 1991-09-20 | 1993-04-02 | Hitachi Ltd | Parallel processor |
US5533198A (en) * | 1992-11-30 | 1996-07-02 | Cray Research, Inc. | Direction order priority routing of packets between nodes in a networked system |
DE4307174A1 (en) * | 1993-03-08 | 1994-09-15 | Philips Patentverwaltung | Local network |
US5617413A (en) * | 1993-08-18 | 1997-04-01 | The United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration | Scalable wrap-around shuffle exchange network with deflection routing |
US5546596A (en) * | 1993-08-24 | 1996-08-13 | Intel Corporation | Method and apparatus for integrated local and express routing in a multiprocessor |
US5583990A (en) * | 1993-12-10 | 1996-12-10 | Cray Research, Inc. | System for allocating messages between virtual channels to avoid deadlock and to optimize the amount of message traffic on each type of virtual channel |
US5544163A (en) * | 1994-03-08 | 1996-08-06 | Excel, Inc. | Expandable telecommunications system |
US5606551A (en) * | 1994-12-21 | 1997-02-25 | Lucent Technologies Inc. | Bidirectional mesh network |
US5684959A (en) * | 1995-04-19 | 1997-11-04 | Hewlett-Packard Company | Method for determining topology of a network |
US5577029A (en) * | 1995-05-04 | 1996-11-19 | Interwave Communications | Cellular communication network having intelligent switching nodes |
US6578010B1 (en) * | 1995-06-05 | 2003-06-10 | George A. Teacherson | Multi-node network marketing computer system |
US5996020A (en) * | 1995-07-21 | 1999-11-30 | National Security Agency | Multiple level minimum logic network |
-
1995
- 1995-07-21 US US08/505,513 patent/US5996020A/en not_active Expired - Lifetime
-
1996
- 1996-07-19 NZ NZ503094A patent/NZ503094A/en not_active IP Right Cessation
- 1996-07-19 KR KR10-1998-0700415A patent/KR100401682B1/en not_active IP Right Cessation
- 1996-07-19 AT AT96924569T patent/ATE327537T1/en not_active IP Right Cessation
- 1996-07-19 WO PCT/US1996/011828 patent/WO1997004399A2/en active IP Right Grant
- 1996-07-19 EP EP96924569A patent/EP0842473B1/en not_active Expired - Lifetime
- 1996-07-19 EP EP00120188A patent/EP1058195A3/en not_active Withdrawn
- 1996-07-19 ES ES00120188T patent/ES2160556T1/en active Pending
- 1996-07-19 DE DE69636165T patent/DE69636165D1/en not_active Expired - Fee Related
- 1996-07-19 AU AU64988/96A patent/AU725826B2/en not_active Ceased
- 1996-07-19 DE DE1058195T patent/DE1058195T1/en active Pending
- 1996-07-19 NZ NZ313016A patent/NZ313016A/en not_active IP Right Cessation
- 1996-07-19 EP EP00120146A patent/EP1058194A3/en not_active Withdrawn
- 1996-07-19 JP JP50682297A patent/JP3594198B2/en not_active Expired - Fee Related
- 1996-07-19 CA CA002227271A patent/CA2227271C/en not_active Expired - Fee Related
-
1998
- 1998-01-21 MX MX9800626A patent/MX9800626A/en not_active IP Right Cessation
- 1998-11-20 HK HK98112191A patent/HK1010926A1/en not_active IP Right Cessation
-
1999
- 1999-09-14 US US09/397,333 patent/US6272141B1/en not_active Expired - Lifetime
-
2001
- 2001-05-07 US US09/852,009 patent/US7068671B2/en not_active Expired - Fee Related
- 2001-05-07 US US09/850,953 patent/US20010034798A1/en not_active Abandoned
- 2001-10-31 GR GR20010300054T patent/GR20010300054T1/en unknown
-
2004
- 2004-06-16 US US10/773,693 patent/US7426214B2/en not_active Expired - Fee Related
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5224100A (en) * | 1991-05-09 | 1993-06-29 | David Sarnoff Research Center, Inc. | Routing technique for a hierarchical interprocessor-communication network between massively-parallel processors |
US5694393A (en) * | 1993-11-11 | 1997-12-02 | U.S. Philips Corporation | Communications system |
US5521910A (en) * | 1994-01-28 | 1996-05-28 | Cabletron Systems, Inc. | Method for determining a best path between two nodes |
US5600641A (en) * | 1994-07-07 | 1997-02-04 | International Business Machines Corporation | Voice circuit emulation system in a packet switching network |
US5781551A (en) * | 1994-09-15 | 1998-07-14 | Texas Instruments Incorporated | Computer communications system with tree architecture and communications method |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20050132089A1 (en) * | 2003-12-12 | 2005-06-16 | Octigabay Systems Corporation | Directly connected low latency network and interface |
Also Published As
Publication number | Publication date |
---|---|
WO1997004399A2 (en) | 1997-02-06 |
US20070186008A1 (en) | 2007-08-09 |
EP0842473A2 (en) | 1998-05-20 |
DE1058195T1 (en) | 2001-10-25 |
KR19990035759A (en) | 1999-05-25 |
CA2227271A1 (en) | 1997-02-06 |
US7068671B2 (en) | 2006-06-27 |
EP1058194A2 (en) | 2000-12-06 |
ATE327537T1 (en) | 2006-06-15 |
NZ503094A (en) | 2001-11-30 |
ES2160556T1 (en) | 2001-11-16 |
EP0842473B1 (en) | 2006-05-24 |
JP2000500253A (en) | 2000-01-11 |
EP1058195A2 (en) | 2000-12-06 |
US6272141B1 (en) | 2001-08-07 |
EP1058195A3 (en) | 2010-03-10 |
AU6498896A (en) | 1997-02-18 |
CA2227271C (en) | 2002-12-31 |
EP1058194A3 (en) | 2010-03-10 |
KR100401682B1 (en) | 2004-02-05 |
US20010021192A1 (en) | 2001-09-13 |
DE69636165D1 (en) | 2006-06-29 |
AU725826B2 (en) | 2000-10-19 |
JP3594198B2 (en) | 2004-11-24 |
HK1010926A1 (en) | 1999-07-02 |
NZ313016A (en) | 2000-05-26 |
MX9800626A (en) | 1998-11-30 |
GR20010300054T1 (en) | 2001-10-31 |
US7426214B2 (en) | 2008-09-16 |
US5996020A (en) | 1999-11-30 |
WO1997004399A3 (en) | 1997-04-10 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US6272141B1 (en) | Multiple level minimum logic network | |
US6289021B1 (en) | Scaleable low-latency switch for usage in an interconnect structure | |
JP2595025B2 (en) | High-speed packet switching equipment using space division type switches | |
US5377182A (en) | Non-blocking crossbar permutation engine with constant routing latency | |
US7397808B2 (en) | Parallel switching architecture for multiple input/output | |
US5617413A (en) | Scalable wrap-around shuffle exchange network with deflection routing | |
EP0366938B1 (en) | High speed switch as for an optical communication system | |
US9529775B2 (en) | Network topology of hierarchical ring with gray code and binary code | |
WO2000076256A1 (en) | Photonic switch using time-slot interchange | |
WO2002093843A1 (en) | Router | |
Callegati et al. | Design of a WDM optical packet switch for IP traffic | |
AU768042B2 (en) | Multiple level minimum logic network | |
AU2004200799A1 (en) | Multiple level minimum logic network | |
Cotter et al. | Ultrafast all-optical signal processing for packet switching | |
Liu et al. | Time division multiplexed de Bruijn network and ShuffleNet for optical communications | |
SU1499367A1 (en) | Routing device | |
Wang et al. | Contention resolution in the loop-augmented shufflenet multihop lightwave network | |
JPH0427243A (en) | Atm cell concentrating system | |
JPH02288439A (en) | Cell switch | |
Sue et al. | Synchronous flow control in wormhole routed optical networks | |
MXPA99006982A (en) | A scalable low-latency switch for usage in an interconnect structure |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |