WO1998017043A2 - Massively parallel processor networks with optical express channels - Google Patents
Massively parallel processor networks with optical express channels Download PDFInfo
- Publication number
- WO1998017043A2 WO1998017043A2 PCT/US1997/017297 US9717297W WO9817043A2 WO 1998017043 A2 WO1998017043 A2 WO 1998017043A2 US 9717297 W US9717297 W US 9717297W WO 9817043 A2 WO9817043 A2 WO 9817043A2
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- light
- data
- wavelength
- fiber optic
- destination
- Prior art date
Links
- 230000003287 optical effect Effects 0.000 title claims description 55
- 239000000835 fiber Substances 0.000 claims abstract description 43
- 238000004891 communication Methods 0.000 claims description 19
- 238000012545 processing Methods 0.000 claims description 13
- 239000000470 constituent Substances 0.000 claims description 3
- 239000000969 carrier Substances 0.000 abstract description 3
- 239000011159 matrix material Substances 0.000 abstract description 2
- 238000000034 method Methods 0.000 description 11
- 230000005540 biological transmission Effects 0.000 description 10
- 230000001934 delay Effects 0.000 description 9
- 238000013459 approach Methods 0.000 description 7
- 230000008901 benefit Effects 0.000 description 6
- 230000003595 spectral effect Effects 0.000 description 6
- 238000013461 design Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 230000008520 organization Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 241001481833 Coryphaena hippurus Species 0.000 description 2
- 238000003491 array Methods 0.000 description 2
- 230000015556 catabolic process Effects 0.000 description 2
- 230000001427 coherent effect Effects 0.000 description 2
- 238000006731 degradation reaction Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000004744 fabric Substances 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 241001522296 Erithacus rubecula Species 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000004888 barrier function Effects 0.000 description 1
- 230000003139 buffering effect Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000004100 electronic packaging Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000007667 floating Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 235000003642 hunger Nutrition 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000002184 metal Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000005693 optoelectronics Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 229920006395 saturated elastomer Polymers 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000037351 starvation Effects 0.000 description 1
- 238000010897 surface acoustic wave method Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04J—MULTIPLEX COMMUNICATION
- H04J14/00—Optical multiplex systems
- H04J14/02—Wavelength-division multiplex systems
- H04J14/0201—Add-and-drop multiplexing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04J—MULTIPLEX COMMUNICATION
- H04J14/00—Optical multiplex systems
- H04J14/02—Wavelength-division multiplex systems
- H04J14/0201—Add-and-drop multiplexing
- H04J14/0202—Arrangements therefor
- H04J14/0206—Express channels arrangements
Definitions
- the United States Government has rights in this invention pursuant to Contract No. W-7405-ENG-48 between the United States Department of Energy and the University of California for the operation of Lawrence Livermore National Laboratory. BACKGROUND OF THE INVENTION Field of the Invention
- the present invention relates to computer systems and fiber optics communication devices and more particularly to wavelength or frequency division multiple access of node interconnection in massively parallel processor networks according to local or express data traffic routing.
- a key barrier to higher performance levels in massively parallel processors (MPPs) is the communication limits that exist amongst the individual processors and between the processors and main memory. Such communication limits include message latencies that could be reduced, e.g., by increasing bandwidth.
- the time delays between initial message transmission and reception stem from the use of information packets that are relayed many times, e.g., in a bucket- brigade fashion from node-to-node within a communication fabric.
- the packet address header is read to route each message packet appropriately to its intended destination. If this occurs more than once, unnecessary latency in the delivery of the message packet is added and can stall processors waiting for the data. Performance suffers when the processors are starved of needed data. Too narrow a data bandwidth can also degrade performance by forcing the dataneeded by the processor to be broken into more than one packet. The processor cannot continue until all the required packets are received.
- Multiprocessing is of great current interest for both general HPC applications, massively parallel processing, and integrated sensor /processor systems. Increases in system node count, computing power per node, and /or sensor-generated data rate increase the communication required to maintain a balanced system that fully utilizes available computing power and sensor data.
- Traditional electronic solutions are not keeping pace with advances in processor performance and sensor complexity and have increasing difficulty providing sufficient communication bandwidth.
- shared memory away from message passing
- multiprocessors places additional stress on inter-processor communications due to the short messages and rapid memory access associated with cache-to-cache coherence traffic.
- Remote hyper-spectral sensing allows potential threats to be assessed with spectral information third axis imaging techniques.
- Three-dimensional data sets comprise layers of two-dimensional image pixels, with each layer representing a different spectral window. If each image layer contains lO ⁇ xlO ⁇ pixels each 12 bits deep and there are 10 ⁇ layers, one for each spectral bin, each cube would represent 1.2x10 ⁇ 0 bits of data.
- hyper-spectral sensing can generate one data cube per second, or a sensor data flow of 1.2x10 ⁇ 0 bits per second, enough to overwhelm communications, storage, and analysts' resources.
- the information flow can be reduced by down selecting data with artificial intelligence processors at each stage of a surveillance operation, e.g., using on-platform mathematical image transform filtering.
- Such "smart" techniques require sophisticated data processing capabilities at the remote sensor platform.
- N 1000
- the communication traffic generated e.g., sensor/processor, processor/processor, and processor/memory, would be about 60 gigabytes /s in a balanced computing environment. This is far beyond the capabilities of prior art electronic busses.
- Dally describes an interconnection network of an array of nodes, where each node in the array is capable of routing messages.
- Immediately adjacent nodes are connected to each other by local channels.
- Messages traveling from a source node to a destination node travel through local channels and through intermediate nodes interconnected by local channels between the nodes.
- the local channels may comprise duplex pairs of unidirectional channels with a separate unidirectional channel for carrying messages to a given node, as well as a separate unidirectional channel for carrying messages from the given node.
- "Express channels” are included that run in parallel with the local and intermediate channels. Such provide an alternative message path between the source nodes and the destination nodes.
- Each express channel provides a path between pairs of more separated nodes that bypass the local traffic in the intermediate nodes.
- the interconnection network further includes interchanges for interfacing the local channels with the express channels so that messages may travel over either the local channel or the express channel.
- Such an interconnection network is particularly well suited for a "k-ary, n-cube" topology.
- the interconnection network nodes may comprise processors as well as memory, and the processors may include private memory.
- the interchange points are situated periodically throughout the interconnection network.
- a hierarchical interchange organization is supposedly well suited for use with multiple express channels.
- a first interchange interfaces a first of the express channels with the local channels
- a second interchange interfaces the second of the express channels with the local channels.
- Other hierarchical interchange configurations include more than two levels of express channels. Additional interchanges may be included to interface the multiple express channels with each other.
- Hierarchical interchanges may be positioned in a stepwise fashion so that messages can bubble up to a top level express channel and then descend back down to a bottom local channel level, e.g., to maximize efficiency.
- the benefit of such hierarchical organization is that the distance component of latency only increases logarithmically with increasing distance.
- the express channels may be provided in multiple dimensions. For instance, express channels may be provided for linear arrays of nodes oriented in each of the multiple dimensions.
- low-dimensional k-ary n-cube interconnection networks have node delays that dominate their wire delays. For any message sent from a starting node to a destination node, the total delay the messages experience is primarily due to the delays incurred by traveling through intermediate nodes, compared to the delays incurred by traveling over wire channels. An ideal network could transfer messages at close to the speed of light.
- k-ary n-cube interconnection networks in real systems have a distance-related component of latency that is more than an order of magnitude less than the speed of light.
- Low- dimensional k-ary n-cube interconnection networks also have channel widths that are limited by the node pin count, rather than being limited by wire density. The channel width of such networks can be limited by the wire density, but as a practical matter, the pin density and pin count primarily limit the channel width.
- the ratio of node delay to wire delay and the ratio of pin density to wire density cannot generally be balanced in ordinary k-ary n-cube networks.
- the wire length and wire density can be adjusted to be independent of the choice of radix (k), dimension (n) and channel width (w), e.g., a so-called "express cube".
- the wire length of the express channels are increased to the point where the wire delays dominate the node delays and the latency approaches its optimal limit.
- the number of express channels is adjusted to increase throughput until the available wiring media is saturated.
- Wavelength division multiplexing has conventionally been used to increase network capacity and bandwidth, to allocate bandwidth, and even to route information.
- Wavelength division multiplexing and free-space optics have been used to interconnect circuit boards within a computer.
- Some researchers have proposed a wavelength reuse scheme that enables larger asynchronous transfer mode (ATM) networks. None of such previous approaches has reduced the data latencies and increased message bandwidth.
- ATM asynchronous transfer mode
- Reducing the power received by each destination will reduce the data transmission rates that are possible, which slows down communication and the message throughput.
- Increasing the optical power is not always practical, because the devices used have maximum power limits or "eye-safe" laser powers that may be exceeded.
- Reducing the system size to have fewer nodes is incongruous when the object is to build large, parallel processing machines in which computational performance scales with number of nodes.
- Architectures that use n-to- n broadcast, n-to-n star couplers, or n-to-1 combining in the optical domain suffer from the power inefficiencies of 1/n.
- Wavelength tunable lasers presently provide, at most, sixty-four different wavelengths. Multiple sources, each at a fixed wavelength, are needed to generate large numbers of wavelengths. But such multiplicity necessitates complex electronics and associated electronic packaging to build the transmitters involved. A related issue is the number of wavelengths used in the system. Conventional telecom systems typically use only four wavelengths and are expected to grow to as many as thirty-two over time. If the system requires many wavelengths to support many nodes, it is unattractive because the system size, e.g., node count, will be limited by wavelength division multiplexing technology.
- each node must receive all system wavelengths, and there are many system wavelengths, the cost and size of the opto-electronic receivers becomes a problem.
- interfacing the electronic output of all receivers to the node is a costly, complicated electronic problem for more than four receivers.
- centralized controllers are needed to establish communication paths, e.g., to make sure two messages don't interfere with one another, the operation of the controller is unacceptably slow.
- Centralized controllers require information about transmissions occurring non-locally in the interconnect; it doesn't have to be a physically centralized device. Slow speed results from the need to gather information about transmission requests from all nodes, process this information, and then redistribute it to the nodes.
- Centralized control is slow because it takes time to set up the circuits. Such control is also complex and adds cost.
- Some schemes require that all messages be launched at the same time, to make sure that certain kinds of messages never coexist simultaneously on the same fiber and wavelength. Such prevents any two messages from interferring with one another.
- Global synchronization is difficult due to having to maintain accurate timing across a large system. Delays tend to vary, resulting in desynchronization.
- Sasayama et al. describe in United States Patent, 5,506,712, a time-slotted, synchronized wavelength division multiplexing approach to connect each of m inputs to some number of outputs. It requires one system wavelength for every input port, e.g., each tunable frequency converter means on each input highway assigns mutually different frequency channels to the optical signals on each highway. This is undesirable, because the number of frequency channels is likely to be limited by practical constraints; for example, the stability of source and multiplexer components. Such, in turn, limit the number of inputs to the system. Sasayama, et al., requires an m-wavelength tunable source at each of the m switch inputs. Such components are currently research curiosities and are not commercially available.
- n:l combiner For large numbers of nodes, it is difficult to build an n:l combiner. It would be better to guide each optical output to a single destination to make the best use of the optical power, e.g., using simple 2:1 combining and dividing wavelength-selective elements.
- Requiring n wavelength sources at each of the n system transmitters means a very large number of sources (n x n) are needed in a system. This adds cost, reduces reliability, and requires substantial electronic circuitry to address each wavelength source independently. Sotom describes in United States Patent 5,485,297, an optical switch that uses tunable wavelength division multiplexing sources, and optical switch matrices plus star couplers to route wavelength division multiplexing transmissions to a particular destination.
- the purpose of the switches is to minimize the size of the star coupler to improve optical power utilization and minimize the number of system wavelengths required by routing messages on the same wavelength to different star couplers.
- the disadvantage of this approach is the need for a centralized control that analyzes the traffic pattern for the inputs and then sets all the switches to make sure two signals on the same wavelength never go to the same star. This kind of centralized control is slow, complex, and costly.
- Sharony et al. describes in United States Patent 5,495,356, a time-slotted approach that requires global synchronization.
- Optical space switches e.g., photonic switches in Fig. 4, or wavelength switching is used for wavelength selective switching. Centralized control is needed to operate such switches.
- Sharony et al. also uses l:n splitting which is power inefficient and has limited switch tuning times.
- H. Obara and Y. Hamazumi in "Star coupler based wavelength division multiplexer switch employing tunable devices with reduced tunability range", Electronics Letters, June 18, 1992; Vol. 28, No. 13, pp. 1268-1270, describe a star coupler broadcast that is power inefficient.
- a set of tunable laser diodes are used, corresponding to one tunable laser per switch input. Fewer tunable components, about one per every four nodes, is preferred for improved cost and reliability.
- the architecture described requires centralized control and can be rather complex.
- M. Kavehrad and M. Tabiani describe in "Selective broadcast optical passive star coupler design for dense wavelength division multiplexer networks", IEEE Photonics Letters, vol. 3, no. 5, May 1991, pp.
- the splitting loss power inefficiency by selective broadcast optical star coupler to limit broadcasts to only a few nodes.
- the proposed device appears complicated to build and attempts to tradeoff splitting losses against the number of system wavelengths used.
- the number of system wavelengths equals the number of nodes. This is unattractive, because the total number of system wavelengths is likely to be technologically limited and limits the size of the system by limiting the number of nodes.
- Darcie et al. describes in United States Patent 5,483,369, systems based on multiplexing of RF signals on carrier frequencies up to a few GHz, or 10 ⁇ Hz, and using surface-acoustic-wave devices. Such does not translate well to the multiplexing of optical signals on carrier frequencies on the order of 10 ⁇ Hz.
- the system Darcie proposes uses carrier frequencies that are so low that only a very few high speed (GHz) channels can be multiplexed, since the desired channel modulation/transmission rate must always be significantly less than the total spectral extent.
- An object of the present invention is to provide a scalable parallel processor system.
- a further object of the present invention is to provide an interconnect system for networks with large numbers of nodes using express channels to improve performance.
- a still further object of the present invention is to provide an optical method for separating and routing local and express channel data.
- an optical method for separating and routing local and express channel data comprises interconnecting the nodes in a network with fiber optic cables.
- a single fiber optic cable carries both express channel traffic and local channel traffic.
- Express channel traffic is placed on, or filtered from, the fiber optic cable at a light frequency or a color different from that of the local channel traffic.
- the express channel traffic is thus placed on a light carrier that skips over the local intermediate nodes one-by-one by reflecting off of selective mirrors placed at each local node.
- the local-channel-traffic light carriers pass through the selective mirrors and are not reflected.
- a single fiber optic cable can thus be threaded throughout a three-dimensional matrix of nodes with the x,y,z directions of propagation encoded by the color of the respective light carriers for both local and express channel traffic.
- frequency division multiple access is used to separate the local and express channels in a hierarchy.
- An advantage of the present invention is a massively parallel processor system is provided that can exceed a thousand nodes without substantial performance degradation due to conventional latency and bandwidth limitations.
- Another advantage of the present invention is that an interconnect system for networks with large numbers of nodes is provided that reduces the cost and bulk to effect the wiring between nodes.
- a still further advantage of the present invention is an optical method for separating and routing local and express channel data is provided that is inherently free of significant latency and bandwidth problems.
- Another advantage of the present invention is an optical method for separating and routing local and express channel data is provided in which each transmission on a particular wavelength is destined for only one destination, so maximally efficient use is made of optical power.
- An advantage of the present invention is an optical method for separating and routing local and express channel data is provided that uses a single wavelength at most nodes. No wavelength tuning is required at such nodes. Multiple wavelength sources are thus needed at only a few nodes.
- FIG. 1 is a diagram of a network computing system embodiment of the present invention showing the use of two wavelengths and a single fiber optic cable daisy chained to support overlapping express channels;
- Fig. 2 is a block diagram of a printed circuit board embodiment of the present invention showing the optical tap of a node for two wavelengths;
- Fig. 3 is a block diagram of a network of the present invention showing a hierarchical ring topology and express channels realizable with the PC board of Fig. 2.
- DETAILED DESCRIPTION OF THE INVENTION Fig. 1 illustrates a network computing system of the present invention for parallel processing, and referred to herein by the general reference numeral 10. While only four network nodes (a-d) are shown in a ring topology, such are intended to represent the case of a thousand or more nodes connected together and in other topologies including bus and star types.
- the system 10 comprises a first node (a) 12 that includes an add/ drop light interference filter 14 for a light wavelength ⁇ 2 , a laser diode transmitter 16 tuned to the light wavelength ⁇ 2 , a photo-detector 18 and a node processor (P) 20.
- a multi-mode fiber (MMF) 22 carries any light transmitted by the laser diode transmitter 16 that passed through the add/drop light interference filter 14 and any non- ⁇ 2 that reflected.
- a second node (b) 24 includes an add /drop light interference filter 26 for a light wavelength ⁇ i, a laser diode transmitter 28 tuned to the light wavelength ⁇ i, a photo-detector 30 and a node processor (P) 32.
- Another multi-mode fiber (MMF) 34 carries any light transmitted by the laser diode transmitter 28 that passed through the add/ drop light interference filter 26 and any non- ⁇ that was reflected from MMF 22.
- Such non- ⁇ i light includes the ⁇ 2 light emitted by laser diode transmitter 16.
- the system 10 further comprises a third node (c) 36 that includes an add /drop light interference filter 38 for the light wavelength ⁇ 2 , a laser diode transmitter 40 tuned to the light wavelength ⁇ 2 , a photo-detector 42 and a node processor (P) 44.
- a multi-mode fiber (MMF) 46 carries any light transmitted by the laser diode transmitter 40 that passed through the add/drop light interference filter 38 and any non- ⁇ 2 that reflected.
- Such non- ⁇ 2 light includes the ⁇ i light emitted by laser diode transmitter 28 and is ultimately received by the photo-detector 54.
- a fourth node (d) 48 includes an add /drop light interference filter 50 for the light wavelength ⁇ i, a laser diode transmitter 52 tuned to the light wavelength ⁇ i, a photo-detector 54 and a node processor (P)
- MMF multi-mode fiber
- Such non- ⁇ i light includes the ⁇ 2 light emitted by laser diode transmitter 40 and is ultimately received by the photo-detector 18.
- the forwarding of the ⁇ i light from the laser diode transmitter 28 at node (b) 24 by node (c) 36 to node (d) 48 incurs none of the usual latency caused by conventional systems.
- the forwarding of the ⁇ light from the laser diode transmitter 40 at node (c) 36 by node (d) 48 to node (a) 12 also did not incur any of the usual latency common to conventional systems.
- Fig. 2 illustrates a printed circuit (PC) board embodiment of the present invention, referred to herein by the general reference numeral 100.
- a fiber optic cable 102 introduces several discrete beams of light represented by beams 104-106 from another node in a network that include wavelengths ⁇ i, ⁇ 2 and ⁇ n .
- An add/drop mirror 108 tuned to wavelength ⁇ i removes signals having wavelength ⁇ i in the beams
- a photo-emitter 112 adds new signals having wavelength ⁇ i into the beams 104-106 before passing them on to a add/drop mirror 114 tuned to wavelength ⁇ 2 - Incoming signals from the fiber optic cable 102 having the wavelength ⁇ 2 and ⁇ n reflect from the add/drop mirror 108.
- a photo-emitter 118 adds new signals having wavelength ⁇ 2 into the beams 104-106 before passing them on to a fiber optic cable 120 which connects to a next node in the network.
- light signals are represented by dashed lines and electrical signals are represented by solid lines.
- Light signals having wavelengths other than ⁇ i or ⁇ 2 are passed through the PC board 100 without incurring a processing overhead. Therefore, signals not intended for delivery to the PC board
- the PC board 100 would preferably not use wavelengths ⁇ i or ⁇ 2 .
- the PC board 100 would also be incapable of interfering with light signals other than those with wavelengths ⁇ i or ⁇ 2 .
- the fiber optic cables 102 and 120 comprise a parallel optical interconnect (POI) ribbon fiber cable of n-number of constituent fiber optic cables that each carry a bit of information.
- POI parallel optical interconnect
- POI ribbon thus provides for word-wide communication n-bits in width.
- Planar microlens arrays are used to focus each constituent fiber optic cable to its corresponding element in a photo-emitter and photo- detector array.
- a set of three scalable coherent interface (SCI) interfaces are provided by SCI node chips 122-124.
- the SCI is defined by IEEE/ ANSI standard number 1596, and is implemented in commercially available integrated circuits such as sold by LSI Logic (Milpitas, CA) as type L64601.
- the SCI uses a collection of fast point-to-point unidirectional links to provide data throughput that exceeds traditional computer-bus services for high-performance multiprocessor systems.
- the SCI supports distributed, shared memory with optional cache coherence for tightly coupled systems, and message-passing for loosely coupled systems.
- Initial SCI links are defined at one gigabyte/s (16-bit parallel) and one gigabits/s (serial).
- the internal transactions are implemented by packets and protocols defined by computer programs.
- SCI supports efficient multiprocessor lock and the usual read-and-write transactions.
- Distributed cache-coherence protocols are included in the commercially marketed devices and can recover from an arbitrary number of transmission failures. Multiprocessor conflicts cannot cause deadlocks or starvation because the SCI protocols will ensure forward progress.
- the transceivers, protocol logic and FIFO-registers all fit into a single chip and package.
- IBM reported at the International Solid State Circuits Conference (ISSCC) '95 that their BiCMOS SCI-link chip receiver and transmitter could operate at 1000 megabytes per second while communicating with user application logic at the same time.
- the IBM SCI-link chip uses thirty-six signal pins for each of the two high speed SCI links. No analog phase-locked-loop technology is used, which makes such parts easy to manufacture in quantity.
- the standard deskewing protocol in the SCI standard allowed IBM to implement dynamic compensation for cable skew. Thus, inexpensive cables can be used. Advanced CMOS processes were expected to be fast enough in 1995 that the BiCMOS process could be replaced.
- the Dolphin 1.6 gigabit/sec SCI LINKCONTROLLERTM is a CMOS implementation of the SCI link interface standard used for high throughput, low latency interconnection. It has two 1.6 Gigabit/sec SCI links, a 400 megabytes/sec local bus TTL interface, a 208-pin metal quad flat pack, and operates with twisted pair or fiber optic cables.
- the LINKCONTROLLER interface chip implements the transport layer protocols of the SCI standard. SCI protocols are pseudo bus protocols that allow distributed systems to be interconnected and operated at backplane speeds. The link chip connects to other node-local components through a unique local bus.
- the B-Link local bus is implemented as a low-cost multi-master bus connection.
- a high throughput built-in buffer management system supports SCI transactions and coherent cache line operations.
- the chip operates with 18-bit parallel twisted pair cables or with serial/parallel fiber optic links.
- the LinkControllerTM chip is a general purpose SCI interface chip and is part of Dolphin's architecture implementation of multiprocessor SCI interfaces.
- the LINKCONTROLLER chip can connect to an external CACHE DIRECTORY CONTROLLER (CDC) or a CACHE MEMORY CONTROLLER (CMC) through the B-Link local bus.
- the CDC/CMC chip implements the SCI protocols and send packets to the LINKCONTROLLER for transmission.
- LINKCONTROLLER chips can be connected to a node, thus accesses can be interleaved or duplicate links can be used in fault tolerant systems.
- the high throughput operation is suited for high performance bus bridge applications.
- the LINKCONTROLLER chip has built-in routing support so it can be used to design high throughput, low latency SCI switches. Queue and routing functions are implemented with a pair of interconnected devices 126 and 128. These provide two-way data communication to a host node via a host interface 130.
- Fig. 3 represents a massively parallel processing (MPP) network embodiment of the present invention, referred to herein by the general reference numeral 200.
- a hierarchical set of ring communication networks, represented by rings 201-203 are provided.
- a plurality of network nodes, represented by nodes 204-211 are each connected to the ring 201 and communicate round robin on light carrier wavelength ⁇ i.
- the rings 202 and 203 provide first and second express channels.
- the ring 202 "stops" at the nodes 204, 206, 208, and
- the nodes 205, 207, 209, and 211 are not participants in the network supported by the ring 202.
- the ring 203 is even more express than the ring 202, and stops only at nodes 204 and 208.
- the nodes 205- 207, and 209-211 are not participants in the network supported by the ring 203.
- the ring 201 could be thought of as a local area network (LAN) with clients at nodes 205, 207, 209, and 211 that are connected by a bridge located at any of nodes 204, 206, 208, and 210 to wider area networks on the rings 202 and 203.
- the network 200 may be implemented at each node with the PC board 100 or similar embodiments.
- the three rings 201-203 illustrated in Fig. 3 are, in actual practice, carried in and out of every PC board 100 by fiber optic cables 102 and 120.
- the wavelengths that simply reflect through from fiber optic cable 102 to fiber optic cable 120, without being trapped by falling through the add/drop mirrors 108 or 114, are represented in Fig. 3 as not passing through particular nodes 205-207 and 209-211.
- the wavelength ⁇ 3 on the ring 203 skips past every one of nodes 205-207 and 209-211.
- the wavelengths represented by wavelengths ⁇ i, 2 , ⁇ n in Fig. 2 for PC board 100 may not be the same wavelengths from one PC board 100 to the next, in order to suit the topology and hierarchy illustrated in Fig. 3 and other topologies and hierarchies.
- Embodiments of the present invention include rapidly reconfigurable, high-bandwidth crossbar switches.
- Higher complexity communication networks of arbitrary topology can be constructed from multiple units, to interconnect many nodes with exceptionally low latency and congestion.
- Hierarchical express channels suitable for both massively parallel processing and cluster computing are also made possible.
Abstract
Description
Claims
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
AU46528/97A AU4652897A (en) | 1996-09-30 | 1997-09-26 | Massively parallel processor networks with optical express channels |
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US2706296P | 1996-09-30 | 1996-09-30 | |
US60/027,062 | 1996-09-30 | ||
US08/900,692 US5943150A (en) | 1996-09-30 | 1997-07-25 | Massively parallel processor networks with optical express channels |
US08/900,692 | 1997-07-25 |
Publications (3)
Publication Number | Publication Date |
---|---|
WO1998017043A2 true WO1998017043A2 (en) | 1998-04-23 |
WO1998017043A3 WO1998017043A3 (en) | 1998-07-09 |
WO1998017043A9 WO1998017043A9 (en) | 1998-08-13 |
Family
ID=26701991
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US1997/017297 WO1998017043A2 (en) | 1996-09-30 | 1997-09-26 | Massively parallel processor networks with optical express channels |
Country Status (3)
Country | Link |
---|---|
US (1) | US5943150A (en) |
AU (1) | AU4652897A (en) |
WO (1) | WO1998017043A2 (en) |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10088643B1 (en) | 2017-06-28 | 2018-10-02 | International Business Machines Corporation | Multidimensional torus shuffle box |
US10169048B1 (en) | 2017-06-28 | 2019-01-01 | International Business Machines Corporation | Preparing computer nodes to boot in a multidimensional torus fabric network |
US10356008B2 (en) | 2017-06-28 | 2019-07-16 | International Business Machines Corporation | Large scale fabric attached architecture |
US10571983B2 (en) | 2017-06-28 | 2020-02-25 | International Business Machines Corporation | Continuously available power control system |
Families Citing this family (47)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6373605B1 (en) * | 1996-05-02 | 2002-04-16 | Interdigital Technology Corporation | Optical interconnect for high speed processor |
US5515194A (en) * | 1993-05-28 | 1996-05-07 | Interdigital Technology Corporation | Optical interconnect for high speed processors |
JPH10229386A (en) * | 1997-02-17 | 1998-08-25 | Nec Corp | Optical fiber amplifier and optical communication using the same |
US6631018B1 (en) | 1997-08-27 | 2003-10-07 | Nortel Networks Limited | WDM optical network with passive pass-through at each node |
US6665496B1 (en) * | 1998-04-27 | 2003-12-16 | Ciena Corporation | WDM ring transmission system |
US7272321B1 (en) | 1999-05-10 | 2007-09-18 | Alloptic, Inc. | Passive optical network |
US6674971B1 (en) | 1999-09-03 | 2004-01-06 | Teraconnect, Inc. | Optical communication network with receiver reserved channel |
US7386236B1 (en) | 1999-09-27 | 2008-06-10 | Alloptic, Inc. | Multiple wavelength TDMA optical network |
US6721315B1 (en) * | 1999-09-30 | 2004-04-13 | Alcatel | Control architecture in optical burst-switched networks |
US6810528B1 (en) * | 1999-12-03 | 2004-10-26 | Sony Computer Entertainment America Inc. | System and method for providing an on-line gaming experience through a CATV broadband network |
AUPQ617500A0 (en) * | 2000-03-10 | 2000-04-06 | Telstra R & D Management Pty Ltd | A communications network architecture |
US6671733B1 (en) | 2000-03-24 | 2003-12-30 | International Business Machines Corporation | Internal parallel system channel |
US6580845B1 (en) | 2000-08-11 | 2003-06-17 | General Nutronics, Inc. | Method and device for switching wavelength division multiplexed optical signals using emitter arrays |
JP2002057739A (en) * | 2000-08-14 | 2002-02-22 | Fujitsu Ltd | Transmitter provided with interface conversion function |
US6826619B1 (en) | 2000-08-21 | 2004-11-30 | Intel Corporation | Method and apparatus for preventing starvation in a multi-node architecture |
US6538818B2 (en) * | 2000-09-20 | 2003-03-25 | General Nutronics, Inc. | Method and device for wavelength switching, wavelength division multiplexing, and time division multiplexing |
US6487643B1 (en) | 2000-09-29 | 2002-11-26 | Intel Corporation | Method and apparatus for preventing starvation in a multi-node architecture |
US6772298B2 (en) | 2000-12-20 | 2004-08-03 | Intel Corporation | Method and apparatus for invalidating a cache line without data return in a multi-node architecture |
US6791412B2 (en) * | 2000-12-28 | 2004-09-14 | Intel Corporation | Differential amplifier output stage |
US7234029B2 (en) | 2000-12-28 | 2007-06-19 | Intel Corporation | Method and apparatus for reducing memory latency in a cache coherent multi-node architecture |
US6721918B2 (en) | 2000-12-29 | 2004-04-13 | Intel Corporation | Method and apparatus for encoding a bus to minimize simultaneous switching outputs effect |
US6594045B2 (en) * | 2001-05-23 | 2003-07-15 | Tropic Networks, Inc. | Flexible optical network architecture and optical add/drop multiplexer/demultiplexer therefor |
US6971098B2 (en) | 2001-06-27 | 2005-11-29 | Intel Corporation | Method and apparatus for managing transaction requests in a multi-node architecture |
US20030131167A1 (en) * | 2001-12-20 | 2003-07-10 | Rankin Linda J. | Node ID discovery |
US20040052530A1 (en) * | 2002-09-17 | 2004-03-18 | Cechan Tian | Optical network with distributed sub-band rejections |
US7552192B2 (en) * | 2002-12-18 | 2009-06-23 | Ronnie Gerome Carmichael | Massively parallel computer network-utilizing MPACT and multipoint parallel server (MPAS) technologies |
US7613900B2 (en) * | 2003-03-31 | 2009-11-03 | Stretch, Inc. | Systems and methods for selecting input/output configuration in an integrated circuit |
US7581081B2 (en) | 2003-03-31 | 2009-08-25 | Stretch, Inc. | Systems and methods for software extensible multi-processing |
US8001266B1 (en) * | 2003-03-31 | 2011-08-16 | Stretch, Inc. | Configuring a multi-processor system |
US7590829B2 (en) * | 2003-03-31 | 2009-09-15 | Stretch, Inc. | Extension adapter |
US7483636B2 (en) * | 2003-07-28 | 2009-01-27 | Fujitsu Limited | Optical network with sub-band rejection and bypass |
US7418575B2 (en) * | 2003-07-29 | 2008-08-26 | Stretch, Inc. | Long instruction word processing with instruction extensions |
US7373642B2 (en) * | 2003-07-29 | 2008-05-13 | Stretch, Inc. | Defining instruction extensions in a standard programming language |
US20050095001A1 (en) * | 2003-10-29 | 2005-05-05 | Fujitsu Limited | Method and system for increasing network capacity in an optical network |
US7416352B2 (en) * | 2005-09-08 | 2008-08-26 | Northrop Grumman Corporation | Optical multi-channel free space interconnect |
US8280251B2 (en) * | 2009-04-20 | 2012-10-02 | Oracle America, Inc. | Data transmission using direct and indirect optical paths |
US8348523B1 (en) * | 2009-10-09 | 2013-01-08 | Lockheed Martin Corporation | Network topology module and system |
US8824496B2 (en) * | 2009-10-30 | 2014-09-02 | Oracle America, Inc. | Two-phase arbitration mechanism for shared optical links |
US8473659B2 (en) * | 2010-01-15 | 2013-06-25 | Oracle America, Inc. | Time division multiplexing based arbitration for shared optical links |
US8285140B2 (en) * | 2010-02-17 | 2012-10-09 | Oracle International Corporation | Shared-source-row optical data channel organization for a switched arbitrated on-chip optical network |
US8406623B2 (en) * | 2010-02-17 | 2013-03-26 | Oracle International Corporation | Data channel organization for a switched arbitrated on-chip optical network |
US8565608B2 (en) | 2011-07-11 | 2013-10-22 | Oracle International Corporation | Optical network with switchable drop filters |
US8606113B2 (en) | 2011-07-11 | 2013-12-10 | Oracle International Corporation | Optical network with tunable optical light sources |
US8655120B2 (en) | 2011-07-11 | 2014-02-18 | Oracle International Corporation | Arbitrated optical network using tunable drop filters |
US11109122B1 (en) * | 2020-06-30 | 2021-08-31 | Microsoft Technology Licensing, Llc | Using free-space optics to interconnect a plurality of computing nodes |
US11832033B2 (en) | 2020-11-03 | 2023-11-28 | Microsoft Technology Licensing, Llc | Efficiently interconnecting computing nodes to enable use of high-radix network switches |
US11539453B2 (en) | 2020-11-03 | 2022-12-27 | Microsoft Technology Licensing, Llc | Efficiently interconnecting a plurality of computing nodes to form a circuit-switched network |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5576875A (en) * | 1994-04-13 | 1996-11-19 | France Telecom | Telecommunications network organized in reconfigurable wavelength-division-multiplexed optical loops |
US5625478A (en) * | 1995-09-14 | 1997-04-29 | Lucent Technologies Inc. | Optically restorable WDM ring network using simple add/drop circuitry |
US5760934A (en) * | 1995-05-17 | 1998-06-02 | France Telecom | Ring network for transmitting wavelength-multiplexed informations |
US5764821A (en) * | 1994-02-06 | 1998-06-09 | Lucent Technologies Inc. | Large capacity local access network |
Family Cites Families (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4914648A (en) * | 1987-03-26 | 1990-04-03 | American Telephone And Telegraph Company | Multichannel, multihop lightwave communication system |
US5367642A (en) * | 1990-09-28 | 1994-11-22 | Massachusetts Institute Of Technology | System of express channels in an interconnection network that automatically bypasses local channel addressable nodes |
SE469149B (en) * | 1990-12-07 | 1993-05-17 | Ellemtel Utvecklings Ab | OPTICAL SELECTOR, OPTICAL CROSS COUPLER, AND MOVE TO RETURN GROUPS OF OPTICAL SIGNALS |
US5475518A (en) * | 1991-05-10 | 1995-12-12 | Olympus Optical Co., Ltd. | Apparatus for optically transmitting data |
US5446572A (en) * | 1992-02-14 | 1995-08-29 | The Mitre Corporation | Optical interconnects for high speed backplanes using spectral slicing |
US5485297A (en) * | 1992-10-09 | 1996-01-16 | Alcatel N.V. | Optical switching matrix |
US5533188A (en) * | 1992-10-19 | 1996-07-02 | The United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration | Fault-tolerant processing system |
US5495356A (en) * | 1993-04-30 | 1996-02-27 | Nec Research Institute, Inc. | Multidimensional switching networks |
DE69434263T2 (en) * | 1993-07-14 | 2006-01-12 | Nippon Telegraph And Telephone Corp. | Photonic coupling field with frequency routing for time division multiplex links |
US5483369A (en) * | 1993-11-03 | 1996-01-09 | At&T Corp. | Communication systems using passive RF routing |
US5550818A (en) * | 1994-09-19 | 1996-08-27 | Bell Communications Research, Inc. | System for wavelength division multiplexing/asynchronous transfer mode switching for network communication |
US5504608A (en) * | 1995-05-25 | 1996-04-02 | At&T Corp. | Adjustable filter for tuning multimode optical signals |
-
1997
- 1997-07-25 US US08/900,692 patent/US5943150A/en not_active Expired - Fee Related
- 1997-09-26 WO PCT/US1997/017297 patent/WO1998017043A2/en active Application Filing
- 1997-09-26 AU AU46528/97A patent/AU4652897A/en not_active Abandoned
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5764821A (en) * | 1994-02-06 | 1998-06-09 | Lucent Technologies Inc. | Large capacity local access network |
US5576875A (en) * | 1994-04-13 | 1996-11-19 | France Telecom | Telecommunications network organized in reconfigurable wavelength-division-multiplexed optical loops |
US5760934A (en) * | 1995-05-17 | 1998-06-02 | France Telecom | Ring network for transmitting wavelength-multiplexed informations |
US5625478A (en) * | 1995-09-14 | 1997-04-29 | Lucent Technologies Inc. | Optically restorable WDM ring network using simple add/drop circuitry |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10088643B1 (en) | 2017-06-28 | 2018-10-02 | International Business Machines Corporation | Multidimensional torus shuffle box |
US10169048B1 (en) | 2017-06-28 | 2019-01-01 | International Business Machines Corporation | Preparing computer nodes to boot in a multidimensional torus fabric network |
US10356008B2 (en) | 2017-06-28 | 2019-07-16 | International Business Machines Corporation | Large scale fabric attached architecture |
US10571983B2 (en) | 2017-06-28 | 2020-02-25 | International Business Machines Corporation | Continuously available power control system |
US10616141B2 (en) | 2017-06-28 | 2020-04-07 | International Business Machines Corporation | Large scale fabric attached architecture |
US11029739B2 (en) | 2017-06-28 | 2021-06-08 | International Business Machines Corporation | Continuously available power control system |
Also Published As
Publication number | Publication date |
---|---|
AU4652897A (en) | 1998-05-11 |
US5943150A (en) | 1999-08-24 |
WO1998017043A3 (en) | 1998-07-09 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US5943150A (en) | Massively parallel processor networks with optical express channels | |
WO1998017043A9 (en) | Massively parallel processor networks with optical express channels | |
Yuan et al. | Distributed path reservation algorithms for multiplexed all-optical interconnection networks | |
Yin et al. | LIONS: An AWGR-based low-latency optical switch for high-performance computing and data centers | |
US6016211A (en) | Optoelectronic smart pixel array for a reconfigurable intelligent optical interconnect | |
Szymanski et al. | Reconfigurable intelligent optical backplane for parallel computing and communications | |
Collet et al. | Architectural approach to the role of optics in monoprocessor and multiprocessor machines | |
JPH05323387A (en) | Programmable optical-crossbar-switch | |
US6674971B1 (en) | Optical communication network with receiver reserved channel | |
US8285140B2 (en) | Shared-source-row optical data channel organization for a switched arbitrated on-chip optical network | |
JP2918772B2 (en) | Packet switching arrangement | |
JPH06268683A (en) | Packet distribution network | |
US6411418B1 (en) | High-performance parallel processors based on star-coupled wavelength division multiplexing optical interconnects | |
Louri et al. | A spanning multichannel linked hypercube: a gradually scalable optical interconnection network for massively parallel computing | |
JPH05323408A (en) | Compact programmable parallel processing system | |
Webb et al. | A class of highly scalable optical crossbar-connected interconnection networks (SOCNs) for parallel computing systems | |
Feehrer et al. | Design and implementation of a prototype optical deflection network | |
EP1340162B1 (en) | Optical signal processing arrangement | |
Jonsson | High performance fiber-optic interconnection networks for real-time computing systems | |
Sauer et al. | Photonic interconnects for gigabit multicomputer communications | |
Jonsson et al. | A fiber-optic interconnection concept for scaleable massively parallel computing | |
WO2020014464A1 (en) | Spatial spectral mesh | |
Jonsson | Optical interconnections in parallel radar signal processing systems | |
Jonsson | Fiber-optic interconnection networks for signal processing applications | |
JPH05273628A (en) | Compact programmable processing module |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AK | Designated states |
Kind code of ref document: A2 Designated state(s): AL AM AT AU AZ BA BB BG BR BY CA CH CN CU CZ DE DK EE ES FI GB GE GH HU IL IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MD MG MK MN MW MX NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT UA UG UZ VN YU ZW AM AZ BY KG KZ MD RU TJ TM |
|
AL | Designated countries for regional patents |
Kind code of ref document: A2 Designated state(s): GH KE LS MW SD SZ UG ZW AT BE CH DE DK ES FI FR GB GR IE IT LU MC NL PT |
|
AK | Designated states |
Kind code of ref document: A3 Designated state(s): AL AM AT AU AZ BA BB BG BR BY CA CH CN CU CZ DE DK EE ES FI GB GE GH HU IL IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MD MG MK MN MW MX NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT UA UG UZ VN YU ZW AM AZ BY KG KZ MD RU TJ TM |
|
AL | Designated countries for regional patents |
Kind code of ref document: A3 Designated state(s): GH KE LS MW SD SZ UG ZW AT BE CH DE DK ES FI FR GB GR IE IT LU MC NL PT |
|
DFPE | Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101) | ||
COP | Corrected version of pamphlet |
Free format text: PAGES 1/3-3/3, DRAWINGS, REPLACED BY NEW PAGES 1/3-3/3; DUE TO LATE TRANSMITTAL BY THE RECEIVING OFFICE |
|
NENP | Non-entry into the national phase |
Ref country code: JP Ref document number: 1998518370 Format of ref document f/p: F |
|
REG | Reference to national code |
Ref country code: DE Ref legal event code: 8642 |
|
122 | Ep: pct application non-entry in european phase | ||
NENP | Non-entry into the national phase |
Ref country code: CA |