US20050117562A1 - Method and apparatus for distributing traffic over multiple switched fibre channel routes - Google Patents
Method and apparatus for distributing traffic over multiple switched fibre channel routes Download PDFInfo
- Publication number
- US20050117562A1 US20050117562A1 US11/032,346 US3234605A US2005117562A1 US 20050117562 A1 US20050117562 A1 US 20050117562A1 US 3234605 A US3234605 A US 3234605A US 2005117562 A1 US2005117562 A1 US 2005117562A1
- Authority
- US
- United States
- Prior art keywords
- port
- identifier
- address
- paths
- exchange
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L45/00—Routing or path finding of packets in data switching networks
- H04L45/24—Multipath
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
- H04L47/12—Avoiding congestion; Recovering from congestion
- H04L47/125—Avoiding congestion; Recovering from congestion by balancing the load, e.g. traffic engineering
Definitions
- the invention relates to the field of computer networks.
- the invention relates to distributing network traffic between a pair of networked machines over multiple available routes through a network interconnecting the machines.
- Most modern computer networks including switched Fibre Channel networks, are packet oriented. In these networks, data transmitted between machines is divided into chunks of size no greater than a predetermined maximum. Each chunk is typically packaged with a header and a trailer into a packet for transmission. In Fibre Channel networks, packets are known as Frames.
- Packets encounter delay while being routed through a network.
- Many networks have switches or routers that receive packets, store them, and forward the packets on towards their destinations when communications resources become available; storing and forwarding of packets introduces delay. Additional delay may be caused by propagation delay in the network interconnect between machines or switches of the network.
- a Sequence is a group of one or more frames, forming part of an exchange, transmitted in a single direction over the network.
- a sequence may contain data, status, or control information.
- Each exchange may contain one or more sequences, and may contain data sequences of multiple frames with control and acknowledgment sequences that are often single frames.
- a Fibre Channel network having at least one switch is a switched Fibre Channel fabric.
- a Fibre Channel switch is a routing device generally capable of receiving frames, storing them, decoding destination information from headers, and forwarding them to their destination or to another switch further along a path toward their destination.
- a network interface of a switch for connection of the switch to a machine is known as an F_Port.
- An F_Port having the ability to connect to a Fibre Channel Arbitrated Loop is known as an FL_Port.
- An E_Port is a network interface of a switch for connection of that switch to another switch of a fabric.
- a G_Port is a port having the ability to operate as either an F_Port or an E_Port; and a GL_Port further has the ability to connect to a Fibre Channel Arbitrated Loop.
- F_Port includes any port of a switch that connects through a link to a machine, whether it be an F_Port, G_Port, GL_Port, or an FL_Port.
- an E_Port includes any port of a switch that connects through a link to another switch, regardless of whether it be an E_Port, GL_Port, or G_Port.
- the term switch port includes any port of a switch, whether it be an E_port or F_port as defined herein.
- N_Port A network interface for connection of a machine to a Fibre Channel fabric is known as an N_Port, and a machine attached to a Fibre Channel network is known as a node.
- An L_Port is a network interface for connection of a machine to a Fibre Channel Arbitrated Loop
- an NL_Port is an N_Port also having the ability to connect to a Fibre Channel Arbitrated Loop.
- N_Port includes both N_Ports and NL_Ports.
- Nodes attached to a Fibre Channel network may be computers, or may be storage devices such as RAID systems, disk drives, or other storage servers.
- a Fibre Channel exchange operates between an originator N_Port and a responder N_Port.
- an originator N_Port may request an I/O operation such as a disk write; the machine attached to the responder N_Port performs the operation.
- N Ports may be originators for some exchanges, and responders for others.
- Each Fibre Channel N_Port is assigned identification for use as a destination address for frames intended for it, this identification is unique to the specific Fibre Channel network at a given time.
- Each Fibre Channel N_Port participating in an exchange assigns exchange identification to that exchange, that exchange identification being unique among the exchanges in progress on that N_Port but not necessarily unique across the network.
- a link is the data transmission and reception hardware and any associated firmware that form a connection between an N_Port and an F_Port of a switch, or between E_Ports of two switches, of a Fibre Channel fabric.
- a link may incorporate a Fibre Channel Arbitrated Loop.
- a computer network there may be more than one possible path, or sequence of links, switches, hubs, routers, etc. that may be traversed by a frame, between two machines attached to the network.
- Multiple paths may be intentional, providing extra capacity or redundant paths to protect against switch, node, or line failures, or may be unintentional consequences of network topology.
- Multiple paths between a pair of N_Ports may exist if there are two or more switches in the network.
- Machines transmitting data on modern high-speed networks usually do not wait for each frame to be acknowledged before transmitting following frames—multiple frames of a single Fibre Channel sequence may exist in a Fibre Channel fabric at the same time. Further, frames of multiple sequences of a single exchange may also exist simultaneously in a Fibre Channel fabric, as may frames of multiple exchanges originated by any given N_Port.
- an early-transmitted frame suffering long delay on one path may arrive at its destination after a late-transmitted frame that suffers little delay on another path.
- Frames transmitted on different paths thus may arrive at the destination N_Port out-of-order, meaning that they are received in a different order than they were transmitted by their originating machine.
- Frames received out-of-order may, and often do, require collection and sorting into correct order before they can be fully processed by the receiving machine.
- Some network protocols including the TCP Internet protocol, presume out-of-order delivery and require that receiving machines collect and re-order frames before executing any command associated with them.
- Other order-dependent protocols including the FCP protocol for encapsulating the SCSI storage interface protocol over Fibre Channel, assume that frames arrive in correct order—requiring that the Fibre Channel fabric deliver frames in-order.
- Some order-dependent protocols detect, and permit retry of, out-of-order frames even if they do not require that destinations perform resequencing.
- Fibre Channel frame headers include a sequence count field with which out-of-order frames may be detected within a sequence.
- Fibre Channel fabrics support a variety of order-dependent and order-independent protocols running on top of their low-level Fibre Channel mechanism.
- Links of an active path may be shared with traffic between other N_Ports, including N_Ports of other machines.
- N_Ports including N_Ports of other machines.
- loads and network configurations change, it is possible for a statically routed active path to become a bottleneck while alternate paths may have unused capacity. It is desirable to make use of any available, otherwise unused, capacity of these alternate paths to provide improved network throughput.
- a RAID system may queue several read or write commands, received from one or more machines. Once queued, these commands are executed from the queue to or from cache, or to or from disk, in an order depending on availability of data in cache, disk availability and disk rotation. With proper interlocks, execution may often be in an order different from that in which the commands were received.
- Commands that may be queued in these devices may include commands from multiple processes, or threads, running on a single machine having one or more processors.
- a transaction-processing system may have several processes running, each process requiring access to a different record of a database on a RAID system, all requesting access to the database at about the same time.
- Each process may then create read, write, lock, or unlock commands for the database. Queuing and execution of each of these commands requires that a exchange of frames be transmitted between the machine and the device.
- Fibre channel frame headers have a D_ID field that encodes identification of the destination N_Port of the frame. They also have an S_ID field that encodes identification of the originating port of the frame. There is also an OX_ID field that encodes the exchange identifier assigned by the originating N_Port, and an RX_ID field that encodes the exchange identifier assigned by the receiving N_Port of the exchange. Since the receiving N_Port does not assign RX_ID until the exchange has begun and a frame is sent in response to other frames of the exchange, the RX_ID field of early frames of an exchange, including the first frame sent by the originating N_Port, may not match the RX_ID of late frames of the exchange.
- a network such as a Fibre Channel fabric, having two or more machines attached, each attached to the fabric through at least one N_Port, has a first and a second path between an N_Port of a first machine and an N_Port of a second machine.
- the first machine originates several commands for execution on the second machine and embeds those commands and associated data in frames. Frames belonging to a first command are recognized and transmitted between the first and second machines over the first path, while frames belonging to a second command are transmitted between the first and second machines over the second path.
- Frames belonging to an individual exchange are recognized through the OX_ID field of the frame headers.
- frames belonging to an individual exchange are recognized through a combination of the OX_ID and the S_ID fields of the frame headers.
- These fields, together with the destination address (D_ID) of the frame are input to a function whose output is used by routing and distributing tasks of one or more switches to index routing tables at a switch of the network fabric.
- These routing tables contain information determining the link over which each frame will be sent through the fabric from that switch towards the destination. In this way, the routing tables determine paths, from what may be a multiplicity of possible paths, that each frame will follow through the network.
- frames relating to the same exchange therefore follow the same path through the network, and therefore arrive in-order.
- Frames of simultaneous, but different, exchanges may be routed over different paths thus distributing traffic between the available paths.
- the routing tables are updated to reflect valid paths through the network and desired frame distribution among them. If more than one valid path appears in the table for any given destination, commands to that destination will tend to be distributed between the paths according to the frequency with which each path appears in the table.
- FIG. 1 is an illustration of a Fibre Channel network having several machines and several paths between some of these nodes;
- FIG. 1A an illustration of multiple processes causing overlapping exchanges on an N_Port
- FIG. 2A an example of frames for a simple write exchange
- FIG. 2B an example of frames for a simple read exchange
- FIG. 3 an illustration of a Fibre Channel frame, as known in the art, detailing header information associated with the frame;
- FIG. 3A an illustration of a prior-art routing table for routing frames based upon D_ID
- FIG. 3B an illustration of a prior-art routing table for routing frames based upon S_ID and D_ID;
- FIG. 3C an illustration of a routing table of the present invention for routing frames based upon D_ID and OX_ID;
- FIG. 3D an illustration of a routing table of the present invention for routing frames based upon D_ID, OX_ID, and S_ID;
- FIG. 4A an illustration of a routing table system incorporating separate D_ID and OX_ID hash functions ahead of a routing table
- FIG. 4B an illustration of a routing table system incorporating separate D_ID and OX_ID hash functions ahead of, and a level of indirection after, a base routing table;
- a switched Fibre Channel network ( FIG. 1 ) has at least two machines, with a switched Fibre Channel fabric 100 interconnecting them.
- the fabric may incorporate two or more switches.
- Machines of the network may include computers 102 104 , and 120 , and RAID or other storage systems 106 each having at least one N_Port 108 , 112 , 114 , 118 , and 122 , for interconnection to the fabric.
- Each N_Port 108 , 112 , 114 , 118 , and 122 connects through a link 130 , 134 , 136 , 140 , and 142 to a switch of the switches 150 , 152 , and 154 of the fabric 100 .
- Switches 150 , 152 , and 154 of the fabric may further be interconnected by additional links 160 , 162 , and 164 . Switches of the fabric may be joined by multiple links, switch 152 connects to switch 154 by a redundant link 165 .
- computer 120 may communicate to RAID system 106 through a first path comprising N_Port 122 , link 142 , switch 150 , link 162 , switch 154 , link 140 , and N_Port 118 ; or through a second path comprising N_Port 122 , link 142 , switch 150 , link 160 , switch 152 , link 164 , switch 154 , link 140 , and N_Port 118 .
- a third path may also exist similar to the second path but using the redundant link 165 from switch 152 to switch 154 , comprising N_Port 122 , link 142 , switch 150 , link 160 , switch 152 , link 165 , switch 154 , link 140 , and N_Port 118 .
- computer 102 may communicate with computer 104 through a path comprising N_Port 108 , link 130 , switch 150 , link 162 , switch 154 , link 136 and N_Port 114 , or through an alternative path comprising N_Port 108 , link 130 , snitch 150 , link 160 , switch 152 , link 164 , switch 154 , link 136 , and N_Port 114 .
- the active path may include one or more elements that become overloaded, or become a bottleneck for these communications. For example, if the active path from N_Port 108 of computer 102 to N_Port 114 of computer 104 is through link 162 and the active path from N_Port 122 of computer 120 to N_Port 118 of RAID system 106 is also through link 162 , it is possible for link 162 to have a heavy load while link 160 is idle.
- Each of these processes 200 and 202 may generate an I/O request 204 and 206 as known in the art, each of which in turn is performed through an exchange 208 and 210 as known in the art.
- These exchanges may overlap in time as they are transferred by the N_Port 122 to and from the fabric; overlapping I/O operations may result from multiple concurrent processes on a machine and many other known causes. For example but not by way of limitation, a disk write operation and a disk read operation may overlap.
- a disk-write command may be packetized as a write exchange FIG. 2A comprising a write command frame 250 sent from the originating N_Port 251 to a receiving N_Port 252 , and a write-data sequence 254 sent after a transfer ready frame 255 is received by the originating N_Port 251 .
- a response status frame 256 is returned to the originating N_Port 251 . Additional acknowledgment and control frames may also be used.
- a disk-read I/O command becomes a read exchange, FIG.
- FIG. 2B which operates through transmission of at least a read command frame 260 from the originating N_Port 251 to a receiving N_Port 252 , which may be associated with a RAID system or other storage device.
- the receiving N_Port 252 returns a data sequence 264 and status 266 frames to the originating N_Port 251 , which may be associated with a computer.
- the write exchange of FIG. 2A may overlap the read exchange of FIG. 2B .
- the originating port read command 260 may be transmitted by the originating port 251 after the write command frame 250 is transmitted and before the transfer ready frame 255 is received by the originating port 251 .
- Each frame, or packet, transmitted over a Fibre Channel network has structure as illustrated in FIG. 3 .
- the frame contains a header, an optional payload, and a trailer.
- the header includes several fields, including a Destination Identification (D_ID) field 300 , a Source Identification (S_ID) field 302 , an Originator Exchange Identifier (OX_ID) 304 , and a Responder Exchange Identifier (RX_ID) 306 .
- the RX_ID 306 may change during an exchange because it is assigned by the responder node after the first frames of an exchange are received by that node; the OX_ID 304 is stable within an exchange. It is possible for a switch to nearly-simultaneously receive frames having identical D_ID 300 and OX_ID 304 fields from different sources, having different S_ID fields 302 .
- a switch of a Switched Fibre Channel Fabric receives frames having the format of FIG. 3 , and typically has multiple switch ports, such as E_Ports 170 and 178 ( FIG. 1 ), and F_Ports 174 and 176 of switch 150 .
- E_Ports 170 and 178 FIG. 1
- F_Ports 174 and 176 of switch 150 .
- the switch 150 receives a frame on an incoming switch port it is expected to forward that frame on a selected outgoing port of the switch.
- the selected outgoing port is a switch port, other than the incoming switch port, on a path from the originating N_Port to the receiving N_Port.
- a routing table 330 may be used to generate an outgoing port selector for controlling the outgoing switch port on which frames are forwarded by the switch.
- the D_ID 300 field is transformed by a hash-function 332 to an address 334 , the address locating a table entry in the table 330 .
- Each entry has an outgoing port selector 336 that controls the switch port on which the frame is forwarded by the switch.
- some switches input the S_ID field 302 ( FIG. 3B ) of the frame, or an incoming switch port number on which the frame was received, to a hash function 342 in addition to the D_ID field 300 .
- the hash function 342 generates an address 344 that locates a table entry in a routing table 346 .
- the table entry then provides an outgoing port selector 348 . This permits the switch to route traffic to a given destination from two different sources over two different routes.
- a routing table 350 is indexed by an address 354 generated by a hash function 352 of the D_ID field 300 and the OX_ID field 304 of each frame header.
- An outgoing port selector 356 is derived from a table entry of the routing table 350 located in the table by the address 354 . The outgoing port selector 356 is used to control the switch port on which frames are transmitted.
- the S_ID field 302 is used by a hash function 360 ( FIG. 3D ) to generate an address 362 .
- Address 362 is then used to generate an outgoing port selector 364 by reading a table entry from a routing table 366 .
- This embodiment provides opportunity to independently control frame distribution between available paths for each source.
- Frames belonging to the same exchange have the same OX_ID field; therefore these frames follow the same route through the network and tend to arrive in-order within that exchange. Frames may, however, arrive out-of-order with respect to frames of other exchanges.
- OX_ID field 304 may be considered by a routing table to distribute frames between a preferred and a less preferred path. For example, if three bits of OX_ID are considered by a routing table of switch 150 , eight table entries may be addressed for the same D_ID. If three of these have an outgoing port selector specifying E_Port 170 , while five specify E_Port 178 , about three-eighths of frames will tend to follow the path through switches 150 and 154 while five-eighths of frames will tend to follow the path through switches 150 , 152 , and 154 . If more than one valid path appears in the table for any given destination, exchanges directed to that destination are thus distributed between the paths according to the frequency with which each path appears in the table.
- routing tables are updated to reflect valid paths through the network and the desired frame distribution among them.
- the is routing tables are also adjusted as a load-balancer task, which may run on any compute-capable machine or switch of the network, adjusts demand on elements of the network. For example, should the link 162 attached to E_Port 170 of switch 150 fail, those routing table entries specifying this port may be replaced by entries specifying E_Port 178 so that frames may reach their intended destination.
- hash function 340 it is expected that significant distribution of traffic among multiple routes can be achieved by considering as few as one or several low bits of the OX_ID field.
- a hash function 400 ( FIG. 4A ) of the D_ID field 300 generates an address-X 402 for a two-dimensional routing table 404 .
- a second hash function 406 generates an address-Y 408 for the routing table 404 from the OX_ID field 304 and may also consider the S_ID field 302 .
- the routing table generates a outgoing port selector 410 as previously described.
- the routing table 404 therefore has a predetermined, number of port entries for each valid D_ID, each entry of which is readily locatable.
- the set of port entries for a particular D_ID are referenced as a line of the routing table.
- FIG. 4A is advantageous because only one line of the routing table need be rewritten to alter the distribution of frames between paths to an individual N_Port. Further, this embodiment lends itself to control of frame distribution among paths because the number of entries associated with each destination is constant and these entries are readily located in the table.
- routing table of the present invention has been described as producing an outgoing port selector from a hash function of the D_ID and OX_ID fields 300 and 304 , that operation may be either direct or indirect.
- a level of indirection is used such that paths may be taken in or out of service quickly, without need to rewrite many of the outgoing port selectors in the routing table.
- a hash function 420 of the D_ID field 300 generates an address-X 422 .
- a second hash function 424 of at least one bit of the OX_ID field 304 , and, optionally, the S_ID field 302 produces an address-Y 426 .
- the address-X 422 and the address-Y are combined to address a routing table 428 .
- the routing table 428 thereupon produces a path code 430 .
- Path code 430 is then translated by a portmap table 432 into the outgoing port selector 434 .
- Path code 430 may have more bits than outgoing port selector 434 .
- the routing table 428 may be adjusted to balance the load. Alternatively, if path code 430 has more bits than the outgoing port selector 434 , it may not be necessary to rewrite the routing table 428 .
- Routing tables of the present invention may be implemented in firmware or hardware of the switch. It is known that implementation of routing tables in hardware provides advantage for switches having heavy load and large numbers of switch ports.
- routing table 350 of FIG. 3C, 366 of FIG. 3D, 404 of FIG. 4A , or 428 of FIG. 4B may be implemented with a static RAM, and the portmap table 432 with a second static RAM.
- the routing table address inputs are multiplexed so it can be written by a processor of the switch such that the processor can maintain the routing table.
- the routing table is thereby addressable by either the address generated by the hash function or functions, or by an address generated by the processor.
- the hash function used for addressing the routing table may be any of many hash functions known in the art of computer science.
- This function may also comprise concatenation of a preselected group of bits of each input to the hash function; such as concatenation of one or more low-order OX_ID bits with several bits of the D_ID field to produce an index to the routing table.
- This function may also comprise concatenation of functions of bits from each field, or concatenation of bits of results of a function applied to each field.
- a computer program product is a machine-readable memory having recorded on it a program for performing a particular function; this may be a read-only memory or may be an erasable and rewritable memory such as RAM, CD-RW, tape, flash memory, or magnetic disk. It is anticipated that routing control software for controlling routing tables as herein described may be distributed or operated as a computer program product. Similarly, a switch containing firmware for constructing and utilizing the routing table of the present invention in routing frames is expected to contain memory having that firmware, and therefore contains a computer program product.
- the invention is not limited to a pair of paths.
- the invention is applicable to any reasonable number of concurrently available paths between nodes of a network.
Abstract
A computer network has two or more switches and a plurality of links. A first machine and a second machine are interconnected by the network in such a way that there exist multiple paths through the network from an N_Port of the first machine to an N_Port of the second machine. Network traffic from the N_Port of the first machine to the N_Port of the second machine is distributed between the multiple paths such that frames related to any single exchange traverse the same path yet frames of a first exchange need not traverse the same path as frames of a second exchange. Frames of each exchange therefore tend to be received by their destination in order with respect to other frames of that exchange, while they are not necessarily received in-order with respect to frames of other exchanges.
Description
- The invention relates to the field of computer networks. In particular, the invention relates to distributing network traffic between a pair of networked machines over multiple available routes through a network interconnecting the machines.
- Most modern computer networks, including switched Fibre Channel networks, are packet oriented. In these networks, data transmitted between machines is divided into chunks of size no greater than a predetermined maximum. Each chunk is typically packaged with a header and a trailer into a packet for transmission. In Fibre Channel networks, packets are known as Frames.
- Packets encounter delay while being routed through a network. Many networks have switches or routers that receive packets, store them, and forward the packets on towards their destinations when communications resources become available; storing and forwarding of packets introduces delay. Additional delay may be caused by propagation delay in the network interconnect between machines or switches of the network.
- The multiple packets, or frames, associated with a single Fibre Channel operation are known as a exchange. A Sequence is a group of one or more frames, forming part of an exchange, transmitted in a single direction over the network. A sequence may contain data, status, or control information. Each exchange may contain one or more sequences, and may contain data sequences of multiple frames with control and acknowledgment sequences that are often single frames. A Fibre Channel network having at least one switch is a switched Fibre Channel fabric. A Fibre Channel switch is a routing device generally capable of receiving frames, storing them, decoding destination information from headers, and forwarding them to their destination or to another switch further along a path toward their destination.
- A network interface of a switch for connection of the switch to a machine is known as an F_Port. An F_Port having the ability to connect to a Fibre Channel Arbitrated Loop is known as an FL_Port. An E_Port is a network interface of a switch for connection of that switch to another switch of a fabric. A G_Port is a port having the ability to operate as either an F_Port or an E_Port; and a GL_Port further has the ability to connect to a Fibre Channel Arbitrated Loop. For purposes of this patent F_Port includes any port of a switch that connects through a link to a machine, whether it be an F_Port, G_Port, GL_Port, or an FL_Port. Further, for purposes of this patent, an E_Port includes any port of a switch that connects through a link to another switch, regardless of whether it be an E_Port, GL_Port, or G_Port. Further, for purposes of this patent, the term switch port includes any port of a switch, whether it be an E_port or F_port as defined herein.
- A network interface for connection of a machine to a Fibre Channel fabric is known as an N_Port, and a machine attached to a Fibre Channel network is known as a node. An L_Port is a network interface for connection of a machine to a Fibre Channel Arbitrated Loop, and an NL_Port is an N_Port also having the ability to connect to a Fibre Channel Arbitrated Loop. For purposes of this patent, the term N_Port includes both N_Ports and NL_Ports.
- Machines, or “Nodes”, attached to a Fibre Channel network may be computers, or may be storage devices such as RAID systems, disk drives, or other storage servers.
- A Fibre Channel exchange operates between an originator N_Port and a responder N_Port. For example, an originator N_Port may request an I/O operation such as a disk write; the machine attached to the responder N_Port performs the operation. N Ports may be originators for some exchanges, and responders for others. Each Fibre Channel N_Port is assigned identification for use as a destination address for frames intended for it, this identification is unique to the specific Fibre Channel network at a given time. Each Fibre Channel N_Port participating in an exchange assigns exchange identification to that exchange, that exchange identification being unique among the exchanges in progress on that N_Port but not necessarily unique across the network.
- For purposes of this application, a link is the data transmission and reception hardware and any associated firmware that form a connection between an N_Port and an F_Port of a switch, or between E_Ports of two switches, of a Fibre Channel fabric. A link may incorporate a Fibre Channel Arbitrated Loop.
- In a computer network, there may be more than one possible path, or sequence of links, switches, hubs, routers, etc. that may be traversed by a frame, between two machines attached to the network. Multiple paths may be intentional, providing extra capacity or redundant paths to protect against switch, node, or line failures, or may be unintentional consequences of network topology. Multiple paths between a pair of N_Ports may exist if there are two or more switches in the network.
- It is known that frames routed on different paths through a network may suffer different delays. Further, delay on each path varies with traffic on each link of the path, the arbitration sequence of each arbitrated loop forming part of a link, flow control delays like those often injected to avoid buffer overflow, and switch loading.
- Machines transmitting data on modern high-speed networks usually do not wait for each frame to be acknowledged before transmitting following frames—multiple frames of a single Fibre Channel sequence may exist in a Fibre Channel fabric at the same time. Further, frames of multiple sequences of a single exchange may also exist simultaneously in a Fibre Channel fabric, as may frames of multiple exchanges originated by any given N_Port.
- If frames of a sequence are transmitted on different paths through a fabric, an early-transmitted frame suffering long delay on one path may arrive at its destination after a late-transmitted frame that suffers little delay on another path. Frames transmitted on different paths thus may arrive at the destination N_Port out-of-order, meaning that they are received in a different order than they were transmitted by their originating machine.
- Frames received out-of-order may, and often do, require collection and sorting into correct order before they can be fully processed by the receiving machine. Some network protocols, including the TCP Internet protocol, presume out-of-order delivery and require that receiving machines collect and re-order frames before executing any command associated with them. Other order-dependent protocols, including the FCP protocol for encapsulating the SCSI storage interface protocol over Fibre Channel, assume that frames arrive in correct order—requiring that the Fibre Channel fabric deliver frames in-order. Some order-dependent protocols detect, and permit retry of, out-of-order frames even if they do not require that destinations perform resequencing. Fibre Channel frame headers include a sequence count field with which out-of-order frames may be detected within a sequence.
- Fibre Channel fabrics support a variety of order-dependent and order-independent protocols running on top of their low-level Fibre Channel mechanism.
- Since frames transmitted over the same path through a network tend to arrive in order, many Fibre Channel systems permitting order-dependent protocols restrict communication between any two N_Ports to transmission over one active path in each direction. Any other path between the N_Ports may be usable as an alternate path should an active path fail, but may remain little used until that failure occurs. Networks that failover from an active path to an alternate path are known in the art of Fibre Channel networks. Frame routing of this type is known herein as static routing with alternate paths.
- Links of an active path, especially links between switches, may be shared with traffic between other N_Ports, including N_Ports of other machines. As loads and network configurations change, it is possible for a statically routed active path to become a bottleneck while alternate paths may have unused capacity. It is desirable to make use of any available, otherwise unused, capacity of these alternate paths to provide improved network throughput.
- It is known that many machines, including RAID storage subsystems, have the ability to queue multiple commands for execution. For example, a RAID system may queue several read or write commands, received from one or more machines. Once queued, these commands are executed from the queue to or from cache, or to or from disk, in an order depending on availability of data in cache, disk availability and disk rotation. With proper interlocks, execution may often be in an order different from that in which the commands were received.
- Commands that may be queued in these devices may include commands from multiple processes, or threads, running on a single machine having one or more processors. For example, a transaction-processing system may have several processes running, each process requiring access to a different record of a database on a RAID system, all requesting access to the database at about the same time. Each process may then create read, write, lock, or unlock commands for the database. Queuing and execution of each of these commands requires that a exchange of frames be transmitted between the machine and the device.
- Fibre channel frame headers have a D_ID field that encodes identification of the destination N_Port of the frame. They also have an S_ID field that encodes identification of the originating port of the frame. There is also an OX_ID field that encodes the exchange identifier assigned by the originating N_Port, and an RX_ID field that encodes the exchange identifier assigned by the receiving N_Port of the exchange. Since the receiving N_Port does not assign RX_ID until the exchange has begun and a frame is sent in response to other frames of the exchange, the RX_ID field of early frames of an exchange, including the first frame sent by the originating N_Port, may not match the RX_ID of late frames of the exchange.
- A network, such as a Fibre Channel fabric, having two or more machines attached, each attached to the fabric through at least one N_Port, has a first and a second path between an N_Port of a first machine and an N_Port of a second machine. The first machine originates several commands for execution on the second machine and embeds those commands and associated data in frames. Frames belonging to a first command are recognized and transmitted between the first and second machines over the first path, while frames belonging to a second command are transmitted between the first and second machines over the second path.
- Frames belonging to an individual exchange are recognized through the OX_ID field of the frame headers. In an alternative embodiment, frames belonging to an individual exchange are recognized through a combination of the OX_ID and the S_ID fields of the frame headers. These fields, together with the destination address (D_ID) of the frame, are input to a function whose output is used by routing and distributing tasks of one or more switches to index routing tables at a switch of the network fabric. These routing tables contain information determining the link over which each frame will be sent through the fabric from that switch towards the destination. In this way, the routing tables determine paths, from what may be a multiplicity of possible paths, that each frame will follow through the network.
- Except when routing tables are being updated, frames relating to the same exchange therefore follow the same path through the network, and therefore arrive in-order. Frames of simultaneous, but different, exchanges may be routed over different paths thus distributing traffic between the available paths.
- As nodes, switches, and links are added to or removed from the network, and as a load-balancer adjusts demand on elements of the network, the routing tables are updated to reflect valid paths through the network and desired frame distribution among them. If more than one valid path appears in the table for any given destination, commands to that destination will tend to be distributed between the paths according to the frequency with which each path appears in the table.
-
FIG. 1 is an illustration of a Fibre Channel network having several machines and several paths between some of these nodes; -
FIG. 1A , an illustration of multiple processes causing overlapping exchanges on an N_Port; -
FIG. 2A , an example of frames for a simple write exchange; -
- n
-
FIG. 2B , an example of frames for a simple read exchange; -
FIG. 3 , an illustration of a Fibre Channel frame, as known in the art, detailing header information associated with the frame; -
FIG. 3A , an illustration of a prior-art routing table for routing frames based upon D_ID; -
FIG. 3B , an illustration of a prior-art routing table for routing frames based upon S_ID and D_ID; -
FIG. 3C , an illustration of a routing table of the present invention for routing frames based upon D_ID and OX_ID; -
FIG. 3D , an illustration of a routing table of the present invention for routing frames based upon D_ID, OX_ID, and S_ID; -
FIG. 4A , an illustration of a routing table system incorporating separate D_ID and OX_ID hash functions ahead of a routing table; and -
FIG. 4B , an illustration of a routing table system incorporating separate D_ID and OX_ID hash functions ahead of, and a level of indirection after, a base routing table; - A switched Fibre Channel network (
FIG. 1 ) has at least two machines, with a switchedFibre Channel fabric 100 interconnecting them. The fabric may incorporate two or more switches. - Machines of the network may include
computers 102 104, and 120, and RAID orother storage systems 106 each having at least oneN_Port N_Port link switches fabric 100.Switches additional links switch 152 connects to switch 154 by aredundant link 165. - There may be, and preferably are, more than one path between a first and a second machine of the network. There are frequently also more than one possible path from a first N_Port to a second N_Port. For example,
computer 120 may communicate toRAID system 106 through a firstpath comprising N_Port 122, link 142,switch 150, link 162,switch 154, link 140, andN_Port 118; or through a secondpath comprising N_Port 122, link 142,switch 150, link 160,switch 152, link 164,switch 154, link 140, andN_Port 118. A third path may also exist similar to the second path but using theredundant link 165 fromswitch 152 to switch 154, comprisingN_Port 122, link 142,switch 150, link 160,switch 152, link 165,switch 154, link 140, andN_Port 118. Similarly,computer 102 may communicate withcomputer 104 through apath comprising N_Port 108, link 130,switch 150, link 162,switch 154, link 136 andN_Port 114, or through an alternativepath comprising N_Port 108, link 130,snitch 150, link 160,switch 152, link 164,switch 154, link 136, andN_Port 114. - Consider the first and second path described above between
computer 120 andRAID system 106. In a network utilizing static routing, only one of these paths is active at a given time. The active path may include one or more elements that become overloaded, or become a bottleneck for these communications. For example, if the active path fromN_Port 108 ofcomputer 102 toN_Port 114 ofcomputer 104 is throughlink 162 and the active path fromN_Port 122 ofcomputer 120 toN_Port 118 ofRAID system 106 is also throughlink 162, it is possible forlink 162 to have a heavy load whilelink 160 is idle. - There may be multiple processes simultaneously executing on
computer 120. Each of theseprocesses 200 and 202 (FIG. 1A ) may generate an I/O request exchange N_Port 122 to and from the fabric; overlapping I/O operations may result from multiple concurrent processes on a machine and many other known causes. For example but not by way of limitation, a disk write operation and a disk read operation may overlap. - A disk-write command may be packetized as a write exchange
FIG. 2A comprising awrite command frame 250 sent from the originatingN_Port 251 to a receivingN_Port 252, and a write-data sequence 254 sent after a transferready frame 255 is received by the originatingN_Port 251. When writing to cache or disk has been completed by the receiving N_Port's machine, a response status frame 256 is returned to the originatingN_Port 251. Additional acknowledgment and control frames may also be used. Similarly, a disk-read I/O command becomes a read exchange,FIG. 2B , which operates through transmission of at least aread command frame 260 from the originatingN_Port 251 to a receivingN_Port 252, which may be associated with a RAID system or other storage device. When data associated with the read operation is ready, the receivingN_Port 252 returns adata sequence 264 and status 266 frames to the originatingN_Port 251, which may be associated with a computer. The write exchange ofFIG. 2A may overlap the read exchange ofFIG. 2B . For example and not by way of limitation, it is possible that the originating port readcommand 260 may be transmitted by the originatingport 251 after thewrite command frame 250 is transmitted and before the transferready frame 255 is received by the originatingport 251. - Each frame, or packet, transmitted over a Fibre Channel network has structure as illustrated in
FIG. 3 . The frame contains a header, an optional payload, and a trailer. The header includes several fields, including a Destination Identification (D_ID)field 300, a Source Identification (S_ID)field 302, an Originator Exchange Identifier (OX_ID) 304, and a Responder Exchange Identifier (RX_ID) 306. TheRX_ID 306 may change during an exchange because it is assigned by the responder node after the first frames of an exchange are received by that node; theOX_ID 304 is stable within an exchange. It is possible for a switch to nearly-simultaneously receive frames havingidentical D_ID 300 andOX_ID 304 fields from different sources, having different S_ID fields 302. - A switch of a Switched Fibre Channel Fabric receives frames having the format of
FIG. 3 , and typically has multiple switch ports, such asE_Ports 170 and 178 (FIG. 1 ), and F_Ports 174 and 176 ofswitch 150. Once theswitch 150 receives a frame on an incoming switch port it is expected to forward that frame on a selected outgoing port of the switch. The selected outgoing port is a switch port, other than the incoming switch port, on a path from the originating N_Port to the receiving N_Port. - It is known that a routing table 330,
FIG. 3A indexed by ahash function 332 of theD_ID 300 field of a frame header, may be used to generate an outgoing port selector for controlling the outgoing switch port on which frames are forwarded by the switch. TheD_ID 300 field is transformed by a hash-function 332 to anaddress 334, the address locating a table entry in the table 330. Each entry has anoutgoing port selector 336 that controls the switch port on which the frame is forwarded by the switch. - In an effort to improve the ability of network management software to optimize traffic flow on a network, some switches input the S_ID field 302 (
FIG. 3B ) of the frame, or an incoming switch port number on which the frame was received, to ahash function 342 in addition to theD_ID field 300. As in the routing system ofFIG. 3A , thehash function 342 generates anaddress 344 that locates a table entry in a routing table 346. The table entry then provides anoutgoing port selector 348. This permits the switch to route traffic to a given destination from two different sources over two different routes. - In a switch of the present invention, a routing table 350,
FIG. 3C , is indexed by anaddress 354 generated by ahash function 352 of theD_ID field 300 and theOX_ID field 304 of each frame header. Anoutgoing port selector 356 is derived from a table entry of the routing table 350 located in the table by theaddress 354. Theoutgoing port selector 356 is used to control the switch port on which frames are transmitted. - In an alternative embodiment of a switch of the present invention, the
S_ID field 302, as well as theD_ID field 300 and theOX_ID field 304, of each frame header is used by a hash function 360 (FIG. 3D ) to generate anaddress 362.Address 362 is then used to generate anoutgoing port selector 364 by reading a table entry from a routing table 366. This embodiment provides opportunity to independently control frame distribution between available paths for each source. - Consider frames received by a
switch 150 of the present invention fromcomputer 120 and intended forRAID system 106N_Port 118. The headers of each of these frames are decoded byswitch 150. In the network as illustrated, frames havingD_ID field 300 corresponding to a destination ofN_Port 118 may reach that destination through a path throughswitches switch 154 directly. A hash function of theD_ID field 300 and at least one bit of theOX_ID field 304 of the header are therefore used to index routing table 180 to select the outgoing switch port. The routing table 180 has the structure illustrated inFIG. 4C or 4D. The hash function is selected such that all entries of the routing table 180 that may be selected by avalid D_ID field 300 correspond to a valid outgoing port on a path to the N_Port identified by D_ID that is distinct from the incoming switch port. - Frames belonging to the same exchange have the same OX_ID field; therefore these frames follow the same route through the network and tend to arrive in-order within that exchange. Frames may, however, arrive out-of-order with respect to frames of other exchanges.
- In a Fibre Channel network, there may be paths between two ports that are “better” in some way than others. Multiple bits of the
OX_ID field 304 may be considered by a routing table to distribute frames between a preferred and a less preferred path. For example, if three bits of OX_ID are considered by a routing table ofswitch 150, eight table entries may be addressed for the same D_ID. If three of these have an outgoing portselector specifying E_Port 170, while five specifyE_Port 178, about three-eighths of frames will tend to follow the path throughswitches switches - As machines, switches, and links are added to or removed from the network the routing tables are updated to reflect valid paths through the network and the desired frame distribution among them. The is routing tables are also adjusted as a load-balancer task, which may run on any compute-capable machine or switch of the network, adjusts demand on elements of the network. For example, should the link 162 attached to
E_Port 170 ofswitch 150 fail, those routing table entries specifying this port may be replaced byentries specifying E_Port 178 so that frames may reach their intended destination. - It is not necessary that the hash function 340 consider all bits of the OX_ID field, it is expected that significant distribution of traffic among multiple routes can be achieved by considering as few as one or several low bits of the OX_ID field.
- In an alternative embodiment of the present invention, a hash function 400 (
FIG. 4A ) of theD_ID field 300 generates an address-X 402 for a two-dimensional routing table 404. Asecond hash function 406 generates an address-Y 408 for the routing table 404 from theOX_ID field 304 and may also consider theS_ID field 302. The routing table generates aoutgoing port selector 410 as previously described. The routing table 404 therefore has a predetermined, number of port entries for each valid D_ID, each entry of which is readily locatable. The set of port entries for a particular D_ID are referenced as a line of the routing table. - The embodiment of
FIG. 4A is advantageous because only one line of the routing table need be rewritten to alter the distribution of frames between paths to an individual N_Port. Further, this embodiment lends itself to control of frame distribution among paths because the number of entries associated with each destination is constant and these entries are readily located in the table. - While the routing table of the present invention has been described as producing an outgoing port selector from a hash function of the D_ID and
OX_ID fields FIG. 4B . In this embodiment, ahash function 420 of theD_ID field 300 generates an address-X 422. Asecond hash function 424 of at least one bit of theOX_ID field 304, and, optionally, theS_ID field 302, produces an address-Y 426. The address-X 422 and the address-Y are combined to address a routing table 428. The routing table 428 thereupon produces apath code 430.Path code 430 is then translated by a portmap table 432 into theoutgoing port selector 434.Path code 430 may have more bits thanoutgoing port selector 434. - In this embodiment, should a link fail it may be possible to rewrite the portmap table 432 to reroute all frames onto a functioning link (if one exists) in less time than it would take to restructure the routing table 428. Once the frames are rerouted onto a functioning link by rewriting the portmap table 432, the routing table 428 may be adjusted to balance the load. Alternatively, if
path code 430 has more bits than theoutgoing port selector 434, it may not be necessary to rewrite the routing table 428. - Routing tables of the present invention may be implemented in firmware or hardware of the switch. It is known that implementation of routing tables in hardware provides advantage for switches having heavy load and large numbers of switch ports. In a hardware implementation, routing table 350 of
FIG. 3C, 366 ofFIG. 3D, 404 ofFIG. 4A , or 428 ofFIG. 4B , may be implemented with a static RAM, and the portmap table 432 with a second static RAM. In such an embodiment, the routing table address inputs are multiplexed so it can be written by a processor of the switch such that the processor can maintain the routing table. The routing table is thereby addressable by either the address generated by the hash function or functions, or by an address generated by the processor. - The hash function used for addressing the routing table may be any of many hash functions known in the art of computer science. This function may also comprise concatenation of a preselected group of bits of each input to the hash function; such as concatenation of one or more low-order OX_ID bits with several bits of the D_ID field to produce an index to the routing table. This function may also comprise concatenation of functions of bits from each field, or concatenation of bits of results of a function applied to each field.
- A computer program product is a machine-readable memory having recorded on it a program for performing a particular function; this may be a read-only memory or may be an erasable and rewritable memory such as RAM, CD-RW, tape, flash memory, or magnetic disk. It is anticipated that routing control software for controlling routing tables as herein described may be distributed or operated as a computer program product. Similarly, a switch containing firmware for constructing and utilizing the routing table of the present invention in routing frames is expected to contain memory having that firmware, and therefore contains a computer program product.
- While much reference has been made to a first and second path through the network, the invention is not limited to a pair of paths. The invention is applicable to any reasonable number of concurrently available paths between nodes of a network.
- While the invention has been particularly shown and described with reference to a preferred embodiment thereof, it will be understood by those skilled in the art that various other changes in the form and details may be made without departing from the spirit and scope of the invention.
Claims (18)
1-17. (canceled)
18. A method of routing between or among a plurality of paths in a network fabric between a first N_Port and a second N_Port, comprising:
receiving a packet containing a destination identifier and an originator exchange identifier;
accessing an entry of a routing table using an address generated based on at least portions of each of the destination identifier and originator exchange identifier; and
selecting one of the plurality of paths between the first N_Port and second N_Port based on the accessed entry of the routing table.
19. The method of claim 18 , wherein the accessed entry of the routing table contains an outgoing port selector, and wherein selecting one of the plurality of paths comprises selecting one of a plurality of outgoing ports of a switch based on the outgoing port selector.
20. The method of claim 18 , further comprising generating the address by applying a hash function on the at least portions of each of the destination identifier and originator exchange identifier.
21. The method of claim 20 , wherein the received packet further includes a source identifier, and wherein generating the address comprises generating the address by applying the hash function on the at least portions of each of the destination identifier and originator exchange identifier and at least a portion of the source identifier.
22. The method of claim 18 , wherein routing between or among the plurality of paths in a network fabric includes routing between or among the plurality of paths in a Fibre Channel fabric.
23. The method of claim 18 , further comprising:
selecting a first one of the paths between the first N_Port and second N_Port in response to a first value of the destination identifier and a first value of at least a portion of the originator exchange identifier; and
selecting a second one of the paths between the first N_Port and second N_Port in response to the first value of the destination identifier and a second value of the at least the portion of the originator exchange identifier.
24. The method of claim 18 , wherein the packet further contains a source identifier, and wherein the packet is received at an incoming port of a switch,
the method further comprising applying the at least portions of the destination identifier and originator exchange identifier to a hash function, the hash function to generate the address,
the hash function to further receive as an input a field selected from the group consisting of the source identifier and a port number of the incoming port of the switch.
25. The method of claim 18 , wherein receiving the packet comprises receiving the packet sent by a computer running plural processes, the originator exchange identifier to identify an exchange for performing an input/output operation associated with one of the processes in the computer.
26. A program product comprising computer-readable code to distribute data between or among paths between a first N_Port and a second N_Port, the computer-readable code executable to cause a switch to:
maintain a routing table having plural entries selectable by an address;
receive a packet containing a destination identifier and an exchange identifier;
generate the address based on at least portions of each of the destination identifier and the exchange identifier; and
route the packet on one of the paths based on the entry of the routing table selected by the generated address.
27. The program product of claim 26 , wherein each entry of the routing table includes an outgoing port selector,
wherein routing the packet on one of the paths comprises routing the packet through one of plural outgoing ports of the switch based on the outgoing port selector contained in the entry selected by the address.
28. The program product of claim 26 , wherein generating the address comprises generating the address by applying a hash function on the at least portions of each the destination identifier and exchange identifier.
29. The program product of claim 28 , wherein the received packet further includes a source identifier, and wherein generating the address comprises generating the address by applying the hash function on the at least portions of each of the destination identifier and exchange identifier and at least a portion of the source identifier.
30. The program product of claim 26 , wherein routing the packet on one of the paths comprises routing the packet on one of the paths of a Fibre Channel fabric.
31. The program product of claim 26 , wherein the exchange identifier comprises an originator exchange identifier, and wherein receiving the packet comprises receiving a packet containing the destination identifier and the originator exchange identifier, and wherein generating the address is based on at least portions of each of the destination identifier and originator exchange identifier.
32. A switch operable in a network having a plurality of paths, comprising:
a plurality of ports including a first port to receive a frame over the network, and additional ports;
a storage to store a routing table having entries for identifying corresponding additional ports for routing a frame received at the first port, wherein the received frame has a destination identifier and an exchange identifier; and
a generator to generate an address based on at least portions of each of the destination identifier and exchange identifier, the address to select one of the entries of the routing table; and
routing logic to route the received frame through one of the additional ports based on the entry selected by the address.
33. The switch of claim 32 , wherein the additional ports are part of respective plurality of paths between a first N_Port and a second N_Port.
34. The switch of claim 33 , wherein the plurality of paths are part of a Fibre Channel fabric.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/032,346 US20050117562A1 (en) | 2000-09-26 | 2005-01-10 | Method and apparatus for distributing traffic over multiple switched fibre channel routes |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US09/669,396 US6847647B1 (en) | 2000-09-26 | 2000-09-26 | Method and apparatus for distributing traffic over multiple switched fiber channel routes |
US11/032,346 US20050117562A1 (en) | 2000-09-26 | 2005-01-10 | Method and apparatus for distributing traffic over multiple switched fibre channel routes |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/669,396 Continuation US6847647B1 (en) | 2000-09-26 | 2000-09-26 | Method and apparatus for distributing traffic over multiple switched fiber channel routes |
Publications (1)
Publication Number | Publication Date |
---|---|
US20050117562A1 true US20050117562A1 (en) | 2005-06-02 |
Family
ID=34063531
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/669,396 Expired - Lifetime US6847647B1 (en) | 2000-09-26 | 2000-09-26 | Method and apparatus for distributing traffic over multiple switched fiber channel routes |
US11/032,346 Abandoned US20050117562A1 (en) | 2000-09-26 | 2005-01-10 | Method and apparatus for distributing traffic over multiple switched fibre channel routes |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/669,396 Expired - Lifetime US6847647B1 (en) | 2000-09-26 | 2000-09-26 | Method and apparatus for distributing traffic over multiple switched fiber channel routes |
Country Status (1)
Country | Link |
---|---|
US (2) | US6847647B1 (en) |
Cited By (41)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20030056000A1 (en) * | 2001-07-26 | 2003-03-20 | Nishan Systems, Inc. | Transfer ready frame reordering |
US20030081608A1 (en) * | 2001-10-08 | 2003-05-01 | Alcatel | Method for distributing load over multiple shared resources in a communication network and network applying such a method |
US20030118053A1 (en) * | 2001-12-26 | 2003-06-26 | Andiamo Systems, Inc. | Methods and apparatus for encapsulating a frame for transmission in a storage area network |
US20040100910A1 (en) * | 2002-11-27 | 2004-05-27 | Andiamo Systems, Inc. | Methods and devices for exchanging peer parameters between network devices |
US20050036499A1 (en) * | 2001-12-26 | 2005-02-17 | Andiamo Systems, Inc., A Delaware Corporation | Fibre Channel Switch that enables end devices in different fabrics to communicate with one another while retaining their unique Fibre Channel Domain_IDs |
US20050117522A1 (en) * | 2003-12-01 | 2005-06-02 | Andiamo Systems, Inc. | Apparatus and method for performing fast fibre channel write operations over relatively high latency networks |
US20050169188A1 (en) * | 2002-06-12 | 2005-08-04 | Andiamo Systems, Inc. | Methods and apparatus for characterizing a route in a fibre channel fabric |
US20050243830A1 (en) * | 2004-04-30 | 2005-11-03 | Wrenn Richard F | System for determining network route quality using sequence numbers |
US20050243817A1 (en) * | 2004-04-30 | 2005-11-03 | Wrenn Richard F | System and method for message routing in a network |
US20060087963A1 (en) * | 2004-10-25 | 2006-04-27 | Cisco Technology, Inc. | Graceful port shutdown protocol for fibre channel interfaces |
US20060153186A1 (en) * | 2004-12-29 | 2006-07-13 | Cisco Technology, Inc. | In-order fibre channel packet delivery |
US20060245361A1 (en) * | 2005-04-27 | 2006-11-02 | Cisco Technology, Inc | Load balancing technique implemented in a storage area network |
US20070283119A1 (en) * | 2006-05-31 | 2007-12-06 | International Business Machines Corporation | System and Method for Providing Automated Storage Provisioning |
US20080063149A1 (en) * | 2006-05-26 | 2008-03-13 | Whaleback Systems Corporation | Call Quality Monitoring |
US7406034B1 (en) | 2002-04-01 | 2008-07-29 | Cisco Technology, Inc. | Methods and apparatus for fibre channel frame delivery |
US20090052892A1 (en) * | 2007-01-19 | 2009-02-26 | Perkins Drew D | Communication Network with Co-Routed Multi-Channel Traffic |
US20100008375A1 (en) * | 2002-04-01 | 2010-01-14 | Cisco Technology, Inc. | Label switching in fibre channel networks |
US20100034100A1 (en) * | 2008-08-06 | 2010-02-11 | Finisar Corporation | Network load tester with real-time detection and recording |
US7706255B1 (en) * | 2007-01-29 | 2010-04-27 | Solace Systems, Inc. | Communications apparatus with redundant switching or backpressure mechanism |
US7916628B2 (en) | 2004-11-01 | 2011-03-29 | Cisco Technology, Inc. | Trunking for fabric ports in fibre channel switches and attached devices |
US20110216769A1 (en) * | 2010-03-08 | 2011-09-08 | Brocade Communications Systems, Inc. | Dynamic Path Selection |
US8805918B1 (en) * | 2002-09-11 | 2014-08-12 | Cisco Technology, Inc. | Methods and apparatus for implementing exchange management for virtualization of storage within a storage area network |
US20140328351A1 (en) * | 2001-04-24 | 2014-11-06 | Brocade Communications Systems, Inc. | Topology for Large Port Count Switch |
US10097612B2 (en) | 2006-05-26 | 2018-10-09 | Fuze, Inc. | Selecting routes through a network |
US10140172B2 (en) | 2016-05-18 | 2018-11-27 | Cisco Technology, Inc. | Network-aware storage repairs |
US10222986B2 (en) | 2015-05-15 | 2019-03-05 | Cisco Technology, Inc. | Tenant-level sharding of disks with tenant-specific storage modules to enable policies per tenant in a distributed storage system |
US10243823B1 (en) | 2017-02-24 | 2019-03-26 | Cisco Technology, Inc. | Techniques for using frame deep loopback capabilities for extended link diagnostics in fibre channel storage area networks |
US10243826B2 (en) | 2015-01-10 | 2019-03-26 | Cisco Technology, Inc. | Diagnosis and throughput measurement of fibre channel ports in a storage area network environment |
US10254991B2 (en) | 2017-03-06 | 2019-04-09 | Cisco Technology, Inc. | Storage area network based extended I/O metrics computation for deep insight into application performance |
US10303534B2 (en) | 2017-07-20 | 2019-05-28 | Cisco Technology, Inc. | System and method for self-healing of application centric infrastructure fabric memory |
US10404596B2 (en) | 2017-10-03 | 2019-09-03 | Cisco Technology, Inc. | Dynamic route profile storage in a hardware trie routing table |
US10545914B2 (en) | 2017-01-17 | 2020-01-28 | Cisco Technology, Inc. | Distributed object storage |
US10585830B2 (en) | 2015-12-10 | 2020-03-10 | Cisco Technology, Inc. | Policy-driven storage in a microserver computing environment |
US10664169B2 (en) | 2016-06-24 | 2020-05-26 | Cisco Technology, Inc. | Performance of object storage system by reconfiguring storage devices based on latency that includes identifying a number of fragments that has a particular storage device as its primary storage device and another number of fragments that has said particular storage device as its replica storage device |
US10713203B2 (en) | 2017-02-28 | 2020-07-14 | Cisco Technology, Inc. | Dynamic partition of PCIe disk arrays based on software configuration / policy distribution |
US10778765B2 (en) | 2015-07-15 | 2020-09-15 | Cisco Technology, Inc. | Bid/ask protocol in scale-out NVMe storage |
US10826829B2 (en) | 2015-03-26 | 2020-11-03 | Cisco Technology, Inc. | Scalable handling of BGP route information in VXLAN with EVPN control plane |
US10872056B2 (en) | 2016-06-06 | 2020-12-22 | Cisco Technology, Inc. | Remote memory access using memory mapped addressing among multiple compute nodes |
US10942666B2 (en) | 2017-10-13 | 2021-03-09 | Cisco Technology, Inc. | Using network device replication in distributed storage clusters |
US11563695B2 (en) | 2016-08-29 | 2023-01-24 | Cisco Technology, Inc. | Queue protection using a shared global memory reserve |
US11588783B2 (en) | 2015-06-10 | 2023-02-21 | Cisco Technology, Inc. | Techniques for implementing IPV6-based distributed storage space |
Families Citing this family (27)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7050392B2 (en) * | 2001-03-30 | 2006-05-23 | Brocade Communications Systems, Inc. | In-order delivery of frames during topology change |
US20020156918A1 (en) * | 2001-04-23 | 2002-10-24 | Brocade Communications Systems, Inc. | Dynamic path selection with in-order delivery within sequence in a communication network |
US7239641B1 (en) * | 2001-04-24 | 2007-07-03 | Brocade Communications Systems, Inc. | Quality of service using virtual channel translation |
US7480796B2 (en) * | 2001-06-07 | 2009-01-20 | Kwan Sofware Engineering, Inc. | System and method for authenticating data using incompatible digest functions |
US20030084219A1 (en) * | 2001-10-26 | 2003-05-01 | Maxxan Systems, Inc. | System, apparatus and method for address forwarding for a computer network |
US7548975B2 (en) * | 2002-01-09 | 2009-06-16 | Cisco Technology, Inc. | Methods and apparatus for implementing virtualization of storage within a storage area network through a virtual enclosure |
US7145914B2 (en) | 2001-12-31 | 2006-12-05 | Maxxan Systems, Incorporated | System and method for controlling data paths of a network processor subsystem |
US20030185221A1 (en) * | 2002-03-29 | 2003-10-02 | Alan Deikman | Network controller with shadowing of software routing tables to hardware routing tables |
US7379970B1 (en) | 2002-04-05 | 2008-05-27 | Ciphermax, Inc. | Method and system for reduced distributed event handling in a network environment |
US7295561B1 (en) | 2002-04-05 | 2007-11-13 | Ciphermax, Inc. | Fibre channel implementation using network processors |
US7406038B1 (en) | 2002-04-05 | 2008-07-29 | Ciphermax, Incorporated | System and method for expansion of computer network switching system without disruption thereof |
US7307995B1 (en) * | 2002-04-05 | 2007-12-11 | Ciphermax, Inc. | System and method for linking a plurality of network switches |
US20040017769A1 (en) * | 2002-04-11 | 2004-01-29 | International Business Machines Corporation | Method of establishing a route redundancy in a data transmission system using static routes |
US20030202510A1 (en) * | 2002-04-26 | 2003-10-30 | Maxxan Systems, Inc. | System and method for scalable switch fabric for computer network |
US8798043B2 (en) * | 2002-06-28 | 2014-08-05 | Brocade Communications Systems, Inc. | Apparatus and method for routing traffic in multi-link switch |
US7619974B2 (en) | 2003-10-31 | 2009-11-17 | Brocade Communication Systems, Inc. | Frame traffic balancing across trunk groups |
US7443799B2 (en) | 2003-10-31 | 2008-10-28 | Brocade Communication Systems, Inc. | Load balancing in core-edge configurations |
US20050187898A1 (en) * | 2004-02-05 | 2005-08-25 | Nec Laboratories America, Inc. | Data Lookup architecture |
US7907626B2 (en) * | 2004-08-12 | 2011-03-15 | Broadcom Corporation | Method and system to allocate exchange identifications for Fibre Channel N-Port aggregation |
US9137212B2 (en) * | 2006-12-04 | 2015-09-15 | Oracle America, Inc. | Communication method and apparatus using changing destination and return destination ID's |
US20080298246A1 (en) * | 2007-06-01 | 2008-12-04 | Hughes William A | Multiple Link Traffic Distribution |
CN102055657B (en) * | 2009-11-02 | 2013-03-13 | 杭州华三通信技术有限公司 | Load distributing method of fiber channel (FC) data message and FC exchanger system |
US9665924B2 (en) * | 2015-04-01 | 2017-05-30 | Microsoft Technology Licensing, Llc | Prioritized requesting of mapping layers |
US20180376448A1 (en) * | 2017-06-23 | 2018-12-27 | Amazon Technologies, Inc. | Wi-fi protocol enhancement techniques for low power networking for the internet-of-things |
US10644994B1 (en) * | 2018-01-16 | 2020-05-05 | Amazon Technologies, Inc. | Ensuring separate network paths between pairs of source and destination devices |
US10630579B1 (en) * | 2018-01-16 | 2020-04-21 | Amazon Technologies, Inc. | Ensuring separate paths for network traffic between source devices and a destination device |
US11233730B2 (en) * | 2019-07-09 | 2022-01-25 | Nvidia Corporation | Network routing using aggregated links |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5414704A (en) * | 1992-10-22 | 1995-05-09 | Digital Equipment Corporation | Address lookup in packet data communications link, using hashing and content-addressable memory |
US5592472A (en) * | 1994-10-27 | 1997-01-07 | Hewlett-Packard Company | High performance path allocation system and method for a fiber optic switch for a fiber optic network |
US6118776A (en) * | 1997-02-18 | 2000-09-12 | Vixel Corporation | Methods and apparatus for fiber channel interconnection of private loop devices |
US6310881B1 (en) * | 1998-10-20 | 2001-10-30 | Terabeam Corporation | Method and apparatus for network control |
US6401128B1 (en) * | 1998-08-07 | 2002-06-04 | Brocade Communiations Systems, Inc. | System and method for sending and receiving frames between a public device and a private device |
-
2000
- 2000-09-26 US US09/669,396 patent/US6847647B1/en not_active Expired - Lifetime
-
2005
- 2005-01-10 US US11/032,346 patent/US20050117562A1/en not_active Abandoned
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5414704A (en) * | 1992-10-22 | 1995-05-09 | Digital Equipment Corporation | Address lookup in packet data communications link, using hashing and content-addressable memory |
US5592472A (en) * | 1994-10-27 | 1997-01-07 | Hewlett-Packard Company | High performance path allocation system and method for a fiber optic switch for a fiber optic network |
US6118776A (en) * | 1997-02-18 | 2000-09-12 | Vixel Corporation | Methods and apparatus for fiber channel interconnection of private loop devices |
US6401128B1 (en) * | 1998-08-07 | 2002-06-04 | Brocade Communiations Systems, Inc. | System and method for sending and receiving frames between a public device and a private device |
US6310881B1 (en) * | 1998-10-20 | 2001-10-30 | Terabeam Corporation | Method and apparatus for network control |
Cited By (76)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9094328B2 (en) * | 2001-04-24 | 2015-07-28 | Brocade Communications Systems, Inc. | Topology for large port count switch |
US20140328351A1 (en) * | 2001-04-24 | 2014-11-06 | Brocade Communications Systems, Inc. | Topology for Large Port Count Switch |
US20030056000A1 (en) * | 2001-07-26 | 2003-03-20 | Nishan Systems, Inc. | Transfer ready frame reordering |
US8675655B2 (en) * | 2001-10-08 | 2014-03-18 | Alcatel Lucent | Method for distributing load over multiple shared resources in a communication network and network applying such a method |
US20030081608A1 (en) * | 2001-10-08 | 2003-05-01 | Alcatel | Method for distributing load over multiple shared resources in a communication network and network applying such a method |
US20030118053A1 (en) * | 2001-12-26 | 2003-06-26 | Andiamo Systems, Inc. | Methods and apparatus for encapsulating a frame for transmission in a storage area network |
US20050036499A1 (en) * | 2001-12-26 | 2005-02-17 | Andiamo Systems, Inc., A Delaware Corporation | Fibre Channel Switch that enables end devices in different fabrics to communicate with one another while retaining their unique Fibre Channel Domain_IDs |
US7499410B2 (en) | 2001-12-26 | 2009-03-03 | Cisco Technology, Inc. | Fibre channel switch that enables end devices in different fabrics to communicate with one another while retaining their unique fibre channel domain—IDs |
US20100008375A1 (en) * | 2002-04-01 | 2010-01-14 | Cisco Technology, Inc. | Label switching in fibre channel networks |
US9350653B2 (en) | 2002-04-01 | 2016-05-24 | Cisco Technology, Inc. | Label switching in fibre channel networks |
US8462790B2 (en) | 2002-04-01 | 2013-06-11 | Cisco Technology, Inc. | Label switching in fibre channel networks |
US7406034B1 (en) | 2002-04-01 | 2008-07-29 | Cisco Technology, Inc. | Methods and apparatus for fibre channel frame delivery |
US7206288B2 (en) * | 2002-06-12 | 2007-04-17 | Cisco Technology, Inc. | Methods and apparatus for characterizing a route in fibre channel fabric |
US20070153816A1 (en) * | 2002-06-12 | 2007-07-05 | Cisco Technology, Inc. | Methods and apparatus for characterizing a route in a fibre channel fabric |
US7830809B2 (en) * | 2002-06-12 | 2010-11-09 | Cisco Technology, Inc. | Methods and apparatus for characterizing a route in a fibre channel fabric |
US20050169188A1 (en) * | 2002-06-12 | 2005-08-04 | Andiamo Systems, Inc. | Methods and apparatus for characterizing a route in a fibre channel fabric |
US8805918B1 (en) * | 2002-09-11 | 2014-08-12 | Cisco Technology, Inc. | Methods and apparatus for implementing exchange management for virtualization of storage within a storage area network |
US9733868B2 (en) | 2002-09-11 | 2017-08-15 | Cisco Technology, Inc. | Methods and apparatus for implementing exchange management for virtualization of storage within a storage area network |
US20040100910A1 (en) * | 2002-11-27 | 2004-05-27 | Andiamo Systems, Inc. | Methods and devices for exchanging peer parameters between network devices |
US8605624B2 (en) | 2002-11-27 | 2013-12-10 | Cisco Technology, Inc. | Methods and devices for exchanging peer parameters between network devices |
US20110090816A1 (en) * | 2003-06-26 | 2011-04-21 | Cisco Technology, Inc. | FIBRE CHANNEL SWITCH THAT ENABLES END DEVICES IN DIFFERENT FABRICS TO COMMUNICATE WITH ONE ANOTHER WHILE RETAINING THEIR UNIQUE FIBRE CHANNEL DOMAIN_IDs |
US8625460B2 (en) | 2003-06-26 | 2014-01-07 | Cisco Technology, Inc. | Fibre channel switch that enables end devices in different fabrics to communicate with one another while retaining their unique fibre channel domain—IDs |
US7876711B2 (en) | 2003-06-26 | 2011-01-25 | Cisco Technology, Inc. | Fibre channel switch that enables end devices in different fabrics to communicate with one another while retaining their unique fibre channel domain—IDs |
US7934023B2 (en) | 2003-12-01 | 2011-04-26 | Cisco Technology, Inc. | Apparatus and method for performing fast fibre channel write operations over relatively high latency networks |
US20050117522A1 (en) * | 2003-12-01 | 2005-06-02 | Andiamo Systems, Inc. | Apparatus and method for performing fast fibre channel write operations over relatively high latency networks |
US20050243817A1 (en) * | 2004-04-30 | 2005-11-03 | Wrenn Richard F | System and method for message routing in a network |
US9838297B2 (en) | 2004-04-30 | 2017-12-05 | Hewlett Packard Enterprise Development Lp | System and method for message routing in a network |
US7440458B2 (en) * | 2004-04-30 | 2008-10-21 | Hewlett-Packard Development Company, L.P. | System for determining network route quality using sequence numbers |
US9210073B2 (en) | 2004-04-30 | 2015-12-08 | Hewlett-Packard Development Company, L.P. | System and method for message routing in a network |
US20050243830A1 (en) * | 2004-04-30 | 2005-11-03 | Wrenn Richard F | System for determining network route quality using sequence numbers |
US20060087963A1 (en) * | 2004-10-25 | 2006-04-27 | Cisco Technology, Inc. | Graceful port shutdown protocol for fibre channel interfaces |
US7916628B2 (en) | 2004-11-01 | 2011-03-29 | Cisco Technology, Inc. | Trunking for fabric ports in fibre channel switches and attached devices |
US8750094B2 (en) | 2004-11-01 | 2014-06-10 | Cisco Technology, Inc. | Trunking for fabric ports in Fibre channel switches and attached devices |
US20110141906A1 (en) * | 2004-11-01 | 2011-06-16 | Cisco Technology, Inc. | Trunking for fabric ports in fibre channel switches and attached devices |
US7649844B2 (en) | 2004-12-29 | 2010-01-19 | Cisco Technology, Inc. | In-order fibre channel packet delivery |
US20060153186A1 (en) * | 2004-12-29 | 2006-07-13 | Cisco Technology, Inc. | In-order fibre channel packet delivery |
US7881325B2 (en) * | 2005-04-27 | 2011-02-01 | Cisco Technology, Inc. | Load balancing technique implemented in a storage area network |
US20060245361A1 (en) * | 2005-04-27 | 2006-11-02 | Cisco Technology, Inc | Load balancing technique implemented in a storage area network |
US8457000B2 (en) | 2006-05-26 | 2013-06-04 | Communications Acquistions, LLC | Call quality monitoring |
US10097612B2 (en) | 2006-05-26 | 2018-10-09 | Fuze, Inc. | Selecting routes through a network |
US10554720B1 (en) | 2006-05-26 | 2020-02-04 | Fuze, Inc. | Selecting routes through a network |
US20080063149A1 (en) * | 2006-05-26 | 2008-03-13 | Whaleback Systems Corporation | Call Quality Monitoring |
US11070606B1 (en) | 2006-05-26 | 2021-07-20 | Fuze, Inc. | Selecting routes through a network |
US8913506B2 (en) | 2006-05-26 | 2014-12-16 | Thinking Phone Networks, Inc. | Call quality monitoring |
US7587570B2 (en) * | 2006-05-31 | 2009-09-08 | International Business Machines Corporation | System and method for providing automated storage provisioning |
US20070283119A1 (en) * | 2006-05-31 | 2007-12-06 | International Business Machines Corporation | System and Method for Providing Automated Storage Provisioning |
US8111685B2 (en) * | 2007-01-19 | 2012-02-07 | Infinera Corporation | Communication network with co-routed multi-channel traffic |
US20090052892A1 (en) * | 2007-01-19 | 2009-02-26 | Perkins Drew D | Communication Network with Co-Routed Multi-Channel Traffic |
US7706255B1 (en) * | 2007-01-29 | 2010-04-27 | Solace Systems, Inc. | Communications apparatus with redundant switching or backpressure mechanism |
US8144602B2 (en) * | 2008-08-06 | 2012-03-27 | Jds Uniphase Corporation | Network load tester with real-time detection and recording |
US20100034100A1 (en) * | 2008-08-06 | 2010-02-11 | Finisar Corporation | Network load tester with real-time detection and recording |
US20110216769A1 (en) * | 2010-03-08 | 2011-09-08 | Brocade Communications Systems, Inc. | Dynamic Path Selection |
US10243826B2 (en) | 2015-01-10 | 2019-03-26 | Cisco Technology, Inc. | Diagnosis and throughput measurement of fibre channel ports in a storage area network environment |
US10826829B2 (en) | 2015-03-26 | 2020-11-03 | Cisco Technology, Inc. | Scalable handling of BGP route information in VXLAN with EVPN control plane |
US11354039B2 (en) | 2015-05-15 | 2022-06-07 | Cisco Technology, Inc. | Tenant-level sharding of disks with tenant-specific storage modules to enable policies per tenant in a distributed storage system |
US10222986B2 (en) | 2015-05-15 | 2019-03-05 | Cisco Technology, Inc. | Tenant-level sharding of disks with tenant-specific storage modules to enable policies per tenant in a distributed storage system |
US10671289B2 (en) | 2015-05-15 | 2020-06-02 | Cisco Technology, Inc. | Tenant-level sharding of disks with tenant-specific storage modules to enable policies per tenant in a distributed storage system |
US11588783B2 (en) | 2015-06-10 | 2023-02-21 | Cisco Technology, Inc. | Techniques for implementing IPV6-based distributed storage space |
US10778765B2 (en) | 2015-07-15 | 2020-09-15 | Cisco Technology, Inc. | Bid/ask protocol in scale-out NVMe storage |
US10585830B2 (en) | 2015-12-10 | 2020-03-10 | Cisco Technology, Inc. | Policy-driven storage in a microserver computing environment |
US10949370B2 (en) | 2015-12-10 | 2021-03-16 | Cisco Technology, Inc. | Policy-driven storage in a microserver computing environment |
US10140172B2 (en) | 2016-05-18 | 2018-11-27 | Cisco Technology, Inc. | Network-aware storage repairs |
US10872056B2 (en) | 2016-06-06 | 2020-12-22 | Cisco Technology, Inc. | Remote memory access using memory mapped addressing among multiple compute nodes |
US10664169B2 (en) | 2016-06-24 | 2020-05-26 | Cisco Technology, Inc. | Performance of object storage system by reconfiguring storage devices based on latency that includes identifying a number of fragments that has a particular storage device as its primary storage device and another number of fragments that has said particular storage device as its replica storage device |
US11563695B2 (en) | 2016-08-29 | 2023-01-24 | Cisco Technology, Inc. | Queue protection using a shared global memory reserve |
US10545914B2 (en) | 2017-01-17 | 2020-01-28 | Cisco Technology, Inc. | Distributed object storage |
US10243823B1 (en) | 2017-02-24 | 2019-03-26 | Cisco Technology, Inc. | Techniques for using frame deep loopback capabilities for extended link diagnostics in fibre channel storage area networks |
US11252067B2 (en) | 2017-02-24 | 2022-02-15 | Cisco Technology, Inc. | Techniques for using frame deep loopback capabilities for extended link diagnostics in fibre channel storage area networks |
US10713203B2 (en) | 2017-02-28 | 2020-07-14 | Cisco Technology, Inc. | Dynamic partition of PCIe disk arrays based on software configuration / policy distribution |
US10254991B2 (en) | 2017-03-06 | 2019-04-09 | Cisco Technology, Inc. | Storage area network based extended I/O metrics computation for deep insight into application performance |
US11055159B2 (en) | 2017-07-20 | 2021-07-06 | Cisco Technology, Inc. | System and method for self-healing of application centric infrastructure fabric memory |
US10303534B2 (en) | 2017-07-20 | 2019-05-28 | Cisco Technology, Inc. | System and method for self-healing of application centric infrastructure fabric memory |
US10999199B2 (en) | 2017-10-03 | 2021-05-04 | Cisco Technology, Inc. | Dynamic route profile storage in a hardware trie routing table |
US10404596B2 (en) | 2017-10-03 | 2019-09-03 | Cisco Technology, Inc. | Dynamic route profile storage in a hardware trie routing table |
US11570105B2 (en) | 2017-10-03 | 2023-01-31 | Cisco Technology, Inc. | Dynamic route profile storage in a hardware trie routing table |
US10942666B2 (en) | 2017-10-13 | 2021-03-09 | Cisco Technology, Inc. | Using network device replication in distributed storage clusters |
Also Published As
Publication number | Publication date |
---|---|
US6847647B1 (en) | 2005-01-25 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US6847647B1 (en) | Method and apparatus for distributing traffic over multiple switched fiber channel routes | |
US6876656B2 (en) | Switch assisted frame aliasing for storage virtualization | |
US11640362B2 (en) | Procedures for improving efficiency of an interconnect fabric on a system on chip | |
JP5376371B2 (en) | Network interface card used for parallel computing systems | |
US7349399B1 (en) | Method and apparatus for out-of-order processing of packets using linked lists | |
US8223778B2 (en) | Routing table architecture | |
US6246684B1 (en) | Method and apparatus for re-ordering data packets in a network environment | |
US5898826A (en) | Method and apparatus for deadlock-free routing around an unusable routing component in an N-dimensional network | |
US5602839A (en) | Adaptive and dynamic message routing system for multinode wormhole networks | |
US8078903B1 (en) | Automatic load-balancing and seamless failover of data flows in storage media encryption (SME) | |
US7230914B2 (en) | Method and system for synchronizing a standby route distributor in a distributed routing platform | |
JP3809873B2 (en) | Method and system for ordered dynamic distribution of packet flows by network processing means | |
EP0744851A1 (en) | Distributed data processing system | |
US10826823B2 (en) | Centralized label-based software defined network | |
KR19980070065A (en) | System and method for managing the processing of relatively large data objects in a communication stack | |
US8566833B1 (en) | Combined network and application processing in a multiprocessing environment | |
US6983334B2 (en) | Method and system of tracking missing packets in a multicast TFTP environment | |
JPH02178755A (en) | Multiple processor load sharing apparatus | |
WO2021083375A1 (en) | Method and apparatus for detecting link states | |
CN108337116B (en) | Message order-preserving method and device | |
US20060031622A1 (en) | Software transparent expansion of the number of fabrics coupling multiple processsing nodes of a computer system | |
JPH10333836A (en) | Disk array controller | |
US20220103450A1 (en) | Port-to-port network routing using a storage device | |
US7944842B2 (en) | Opportunistic queueing injection strategy for network load balancing | |
EP1576773A1 (en) | Return path derivation in packet-switched networks |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |