WO2016149905A1

WO2016149905A1 - Field programmable gate array memory allocation

Info

Publication number: WO2016149905A1
Application number: PCT/CN2015/074945
Authority: WO
Inventors: Qunyang LIN; Junqing Xie; Xunteng XU; Xiaofeng Yu; Shuai WANG
Original assignee: Hewlett-Packard Development Company, L.P.
Priority date: 2015-03-24
Filing date: 2015-03-24
Publication date: 2016-09-29

Abstract

A system includes a field programmable gate array (FPGA) including a first memory divided into a number of blocks, a processor, and a second memory. The second memory stores machine readable instructions to cause the processor to allocate the on-chip memory of the FPGA based on a number of nodes at each height of an Internet Protocol (IP) address routing table binary trie and to flash shape graphs for nodes of the binary trie onto the on-chip memory of the FPGA according to the allocated on-chip memory for pipeline processing of lookup of the routing table.

Description

FIELD PROGRAMMABLE GATE ARRAY MEMORY ALLOCATION

BACKGROUND

Computer networks can be used to allow physical or virtual networked devices， such as personal computers， servers， and data storage devices， to exchange data. In some networks， routers can be used to forward data (e.g.， in the form of Internet Protocol (IP) data packets) from one network device to another network device. Such routers can include one or more routing tables that contain information to allow the router to forward data to different network devices. For example， a router may use such a routing table to perform an IP lookup procedure for an incoming IP packet in order to forward the packet to a next router along the route to the packet’s network destination.

BRIEF DESCRIPTION OF THE DRAWINGS

For a detailed description of various examples， reference will now be made to the accompanying drawings in which:

FIG. 1 is a diagram of a router， according to an example.

FIG. 2 is a flowchart for a method， according to an example.

FIG. 3 is a diagram of a binary trie， according to an example.

FIG. 4 is a diagram of an IP lookup engine， according to an example.

FIG. 5 (a) is a diagram of a routing table， according to an example.

FIG. 5 (b) is a diagram of a binary trie， according to an example.

FIG. 5 (c) is a diagram of a shape graph， according to an example.

FIG. 5 (d) is a diagram illustrating stages of pipeline processing， according to an example.

FIG. 5 (e) is a diagram illustrating on-chip memory allocation， according to an example.

FIG. 6 is a diagram of a system， according to an example.

FIG. 7 is a diagram of a machine-readable storage medium， according to an example.

DETAILED DESCRIPTION

The following discussion is directed to various examples of the disclosure. Although one or more of these examples may be preferred， the examples disclosed herein should not be interpreted， or otherwise used， as limiting the scope of the disclosure， including the claims. In addition， the following description has broad application， and the discussion of any example is meant only to be descriptive of that example， and not intended to intimate that the scope of the disclosure， including the claims， is limited to that example. Throughout the present disclosure， the terms "a" and "an" are intended to denote at least one of a particular element. In addition， as used herein， the term "includes" means includes but not limited to， the term "including" means including but not limited to. The term "based on" means based at least in part on.

As provided above， a router or other network routing device can include one or more routing tables that contain information to allow the router to forward data to one or more network destinations. For example， some IP routing techniques rely on a routing table to determine an output port corresponding to an appropriate network destination for a packet via an IP lookup procedure. This may be accomplished by determining a prefix for the IP packet， wherein the prefix is the longest matching prefix corresponding to the network destination IP address of the IP packet. The router may then refer to the routing table to identify the output port for the network destination based on next hop information (NHI) ， the process of which is described in further detail below. In many cases， such a routing table may include a large number of prefixes， which may significantly increase lookup time and reduce the throughput for the IP lookup procedure. Moreover， such a large routing table may unduly consume a large amount of memory space when locally stored on the router. In view of the above， compression of the routing table may be desirable， especially for storage of a routing table on a Field Programmable Gate Array (FPGA) ， which can have limited on-chip storage capabilities.

Certain implementations of the present disclosure are directed to techniques for compressing information within a routing table and for resilient management and allocation of on-chip router memory for IP lookup. For example， in some implementations and as described in further detail herein， such a technique can include: (1) splitting a binary trie representing a routing table of a router into a plurality of layers of a hierarchy， wherein each layer comprises a plurality of nodes， (2) determining a plurality of groups of isomorphic nodes for each layer， (3) generating a shape graph by merging the isomorphic nodes within each of the plurality of groups for each layer， (4) allocating on-chip memory of an FPGA based on the number of nodes in each layer， and (5) flashing the shape graph onto the on-chip memory of the FPGA for pipeline processing of lookup of the routing table. In some implementations， such memory allocation can provide for more resilient memory management that can， for example: (1) reduce a risk of running out of memory at a given pipeline stage， (2) accommodate more routing entries given a constraint amount of on-chip memory， and (3) adapt to routing tables of different distributions. Other advantages of implementations presented herein will be apparent upon review of the description and figures.

FIG. 1 is a block diagram of a router 100 that can， for example， be used to route a packet 102 from a network source 104 to a network destination 106 within a network. Such a network can， for example， be in the form of a Wide Area Network (WAN) that can， for example， be communicatively coupled to a number of other networks via a number of routers. In some implementations， the WAN may be the Internet， and the data packets may be IP packets. Other networks within such a networking environment can， for example， include Local Area Networks (LANs) ， Metropolitan Area Networks (MANs) ， Virtual Private Networks (VPNs) ， other WANs， etc. Network source 104 can， for example， be in the form of a source computing device， such as a desktop computer， laptop computer， or server， from which packet 102 originates， or a network node， such as another router， that is along the packet’s route to a destination computing device. Likewise， network destination 106 may be the destination computing device， such as a desktop computer， laptop computer， or server， to which packet 102 is to be routed， or may be a network node， such as another router， that is along the packet’s route to the destination computing device.

It is appreciated that such a network can include any suitable number of computing devices and routers. Moreover， although the term “router” is used throughout this description (e.g.， with respect to the description of router 100) ， it is appreciated that this term can refer broadly to other suitable devices capable of network data forwarding. For example， a general purpose computer can include suitable hardware and machine-readable instructions that allow the computer to function as router 100 or other network data path element. It is appreciated that the term “router” as used herein can include other network data path elements in the form of suitable switches， gateways and other devices that can provide router-like functionality. It is further appreciated that in some implementations， the functionality of router 100 may be virtualized， for example in the form of a virtual router operating in a virtualized network environment. In some implementations， router 100 can be in the form of a multilayer router that operates at multiple layers of the OSI model (e.g.， the data link and network layers) .

In some situations， a single router 100 may be used to route packet 102 from network source 104 to network destination 106 within a network or between networks. In other situations， a number of routers 100 may be used to route packet 102 from network source 104 to network destination 106. For example， each router that is used to transfer a packet to network destination 106 may determine a next router， e.g.， next hop， to which the packet is to be sent during the routing procedure. As used herein， the term “hop” can， for example， correspond to each step along a forwarding path of data en route to network destination 106.

The implementation of router 100 depicted in FIG. 1 includes an input switch 108 configured to receive packet 102 from network source 104， and an output switch 112 configured to send packet 102 to network destination 106. Input switch 108 may include a number of input ports 110， and each input port 110 may be connected to a different network node， such as another router or a computing device. As shown for example in FIG. 1， network source 104 may be connected to one of a plurality of input ports 110. In addition， output switch 112 may include a number of output ports 114， and each output port 114 may be connected to a different network node， such as another router or another computing device. As shown for example in FIG. 1， network destination 106 may be connected to one of the input ports 110. Router 100 may include an equal number of input ports 110 and output ports 114， as shown for example in FIG. 1. In some implementations， router 100 may include a greater number of input ports 110 than output ports 114 or vice versa.

Router 100 may include a memory 116 that is adapted to store packet 102 received from network source 104 until the appropriate network destination 106 has been determined. Memory 116 may include random access memory (RAM) ， read only memory (ROM) ， flash memory， or any other suitable memory systems. Memory 116 can， for example， be connected to input switch 108 and output switch 112 via a data bus 118. Data bus 118 can， for example， enable the transfer of data packet 102 between input switch 108 and output switch 112.

The implementation of router 100 illustrated in FIG. 1 further includes a controller 120 that is adapted to direct the routing of packet 102. Controller 120 can， for example， be connected to input switch 108 and output switch 112 via data bus 118. Thus， controller 120 may be configured to determine routing information relating to data packet 102 as it is received at input switch 108. Such routing information can， for example， be used by controller 120 to perform a shape graph lookup procedure (as described in further detail below for example with respect to the method of FIG. 2) via a shape graph lookup module 122. According to examples described in further detail below， shape graph lookup module 122 may be any type of application or program that is adapted to perform a shape graph lookup procedure. Such a shape graph lookup procedure can， for example， include searching a shape graph 124 that is stored in on-chip FPGA memory 126 (e.g.， FPGA memory that is stored within controller 120) . In some examples， FPGA memory 126 may include any other suitable type of memory， such as Block RAM (BRAM) ， Distributed RAM， or the like. Shape graph 124 may be searched to determine next hop information based on a longest matching prefix for the IP address of network destination 106 referred to in the packet’s routing information. Such next hop information may further include an output port index for routing packet 102.

Once the output port index has been determined， shape graph lookup module 122 may determine a corresponding output port 114 for routing packet 102 to network destination 106. This may be accomplished by searching an output port indexing array 130 stored within off-chip RAM 128 (e.g.， RAM that is stored remotely with respect to controller 120) . RAM 128 can， for example， be accessible by shape graph lookup module 122 via data bus 118. RAM 128 may include， for example， dynamic RAM (DRAM) ， static RAM (SRAM) ， or the like. Further， in some examples， RAM 128 may include any other suitable type of memory， such as ROM， flash memory， or the like. It is appreciated that in some implementations， RAM 128 is stored remotely to router 100 and accessible through a suitable wired or wireless data communication path.

Controller 120 may then send the information relating to network destination 106 to output switch 112. Output switch 112 may retrieve packet 102 from memory 116， and forward packet 102 to network destination 106. It is to be understood the block diagram of FIG. 1 is not intended to indicate that router 100 is to include all of the components shown in FIG. 1 in every case. Further， any number of suitable additional or alternative components may be included within router 100， depending on the details of a specific implementation.

FIG. 2 is a flowchart for an example method 132 according to the present disclosure. Certain techniques described herein relate to the generation of a compressed routing table， referred to herein as a “shape graph， ” by compressing a binary trie that represents a routing table for a given router. Further details regarding the process for generating such shape graphs is provided in WO 2014/047863 titled “Generating a shape graph for a routing table， ” the entire disclosure of which is incorporated by reference. This shape graph may be used to determine next hop information for a packet received by the router. The next hop information may relate to a next location to which a packet is to be routed on its way to a particular network destination. The next hop information for a specific packet can， for example， be determined by identifying a node corresponding to a longest matching prefix for the packet’s network destination IP address within the shape graph. Further， indicators for nodes of the shape graph may be referred to herein as “colors， ” and the process of assigning indicators to the nodes may be referred to herein as “coloring” the nodes. Although execution of method 132 and other methods described herein make reference to systems， mediums， and other aspects of the disclosure described herein， other suitable devices for execution of these methods will be apparent to those of skill in the art. Method 132 illustrated in the flowchart of FIG. 2 as well as the methods described in the other figures can， for example， be implemented in the form of executable instructions stored on a memory of a computer system， executable instructions stored on a storage medium， in the form of electronic circuitry， or another suitable form.

Method 132 includes a step 134 of splitting a binary trie representing a routing table of a router into a plurality of layers of a hierarchy， wherein each layer comprises a plurality of nodes. In some implementations， such a routing table can， for example， store IP addresses or another suitable networking addressing scheme. Such IP addresses can， for example， be in the form of a 32-bit number corresponding to Internet Protocol Version 4 (IPv4) addresses. In some implementations， addresses stored in the routing table can， for example， be in the form of a 128-bit number corresponding to Internet Protocol Version 6 (IPv6) addresses. In some implementations， the routing table can include a combination of IPv4 and IPv6 addresses and/or other forms of network addresses (e.g.， network addresses that are not in the form of IP addresses) . In some implementations， step 134 includes determining a number of layers， as well as a height of each layer， based on the size of the binary trie， such as， for example， the number of bits included within the binary trie.

FIG. 3 illustrates an example of such a binary trie 136 and is described in further detail below. Binary trie 136 may be a binary trie with back-edges that may be used to organize prefixes and next hop information within a routing table. Binary trie 136 may include a root node 138 located at the top of binary trie 136， any number of intermediary nodes 140 branching off root node 138， and any number of leaf nodes 142 located at the bottom of binary trie 136 and branching off intermediary nodes 140 or root node 138. Construction of binary trie 136 may be performed by merging prefixes within a routing table such that root node 138 corresponds to the starting point of the IP lookup procedure for the IP packet， and each leaf node 142 corresponds to specific next hop information relating to a longest matching prefix obtained by walking from root node 138 to leaf node 142 based on the network destination IP address of the IP packet. It is appreciated that an uncompressed binary trie such as that shown in FIG. 3 may occupy a large amount of memory space within router 100. Thus， it may be desirable to compress binary trie 136 to generate one or more shape graphs， as discussed further below.

Method 132 includes a step 144 of determining a plurality of groups of isomorphic nodes for each layer. This may be accomplished by traversing all

nodes

138， 140， and 142 within binary trie 136 starting from the bottom of binary trie 136， and assigning an indicator to each node. The indicator may be used to merge the nodes into a number of node groups， referred to herein as “sub-tries， ” wherein each sub-trie includes identical， or isomorphic， nodes. Each sub-trie may include any number of nodes， or， in some cases， may include only one node. Further， according to the example shown in FIG. 3， the indicator can be represented by a real number in {1， 2， ...， n} ， wherein n denotes the total number of sub-tries. However， it is to be understood that any suitable type of indicator may be used. As discussed above， traversal of the nodes may be performed starting from the bottom of the binary trie 136， i.e.， starting with leaf nodes 142. All leaf nodes 142 may be assigned a same indicator， e.g.， the indicator “1， ” as shown in FIG. 3. Then， moving upwards from leaf nodes 142， the left and right intermediary nodes 140 may be examined， and groups of isomorphic intermediary nodes 140 may be assigned a same indicator. Once root node 138 is reached， e.g.， once all of the nodes have been traversed， all the nodes of the same indicator， e.g.， all the sub-tries， may be merged to obtain the shape graph.

Step 144 can， for example， include sorting nodes within a binary trie according to height. The binary trie of FIG. 3 includes indicators for the shape ID of each node in the form of “AA-BB” where AA is a unique indicator for each node (e.g.， A-Z) and BB indicates the mapping shape ID of each node. Sorting of the binary trie based on height of the nodes may be accomplished through two traversals of the binary trie. For example， a post-order traversal of the binary trie can， for example， be used to calculate the height of every node， and another traversal of the binary trie can， for example， be used to perform a counting sort over all nodes according to their calculated heights. In some implementations， separate traversals of the binary trie may be avoided by partitioning nodes on-the-fly. For example， let S_i be the set of nodes with height i. Initially， all the leaf nodes may be inserted into S₀. During the node coloring process， when a node v is assigned a color i， the node v’s sibling node u may be checked. If the sibling node u has been colored， then node v and node u’s parent node p may be added into the set S_i+1. This partitioning procedure can be performed at the same time as the coloring procedure with one auxiliary array， and it may be performed within a constant time for every node in the trie.

In some implementations of step 144， nodes within the binary trie are colored to form isomorphic sub-tries. Every node within the binary trie may be represented as a triple， including the value (has-entry， left-color， right-color) ， which specifies whether the node corresponds to a prefix， the color of the node branching to the left， and the color of the node branching to the right， respectively. The isomorphic sub-tries may be determined by performing three partitioning procedures. The first partitioning procedure may involve partitioning nodes into two groups by their has-entry value， which may be equal to 0 or 1. The second partitioning procedure may involve partitioning nodes within each group according to their left colors， and the third partitioning procedure may involve partitioning nodes with the same has-entry and left color by their right colors. The second and third partitioning procedures， which involve grouping nodes with the same left color and right color pairs together and assigning a distinct color to the group of nodes， is described further below.

The colors of the nodes may be represented by real numbers in {1， 2， ...， n} . An array H with n entries may be constructed， where entry i is a pointer to a linked list of color pairs for color i. A linked list L may also be constructed to record the occupied entries by the left colors in H. For the second partitioning procedure， all pairs of colors may be inserted into the array in turn. When a color pair (a， b) is inserted， the entry a in the array H may be checked. If the entry a is associated with an empty linked list， the pair may be assigned as the first item of the linked list， pointed to from entry a in H， and a may be inserted into L. Otherwise， the color pair may be inserted into the linked list from entry a in H， and L may remain unchanged. After all the pairs may been inserted into H， a linked list L of all the left colors is obtained， as well as an array H storing all the pairs. For every item i in L， the linked list of pairs from entry i in H may be examined， and the same process may be repeated according to the right colors in order to obtain a linked list R of right colors. From the array H and the linked list R， the pairs that have the same left and right colors may be collected into groups， and a distinct color may be assigned to each group of pairs， or sub-trie.

Method 132 includes a step 146 of generating a shape graph by merging the isomorphic nodes within each of the plurality of groups for each layer. In some implementations， for each layer， isomorphic nodes determined in step 144 within each group are merged to generate a shape graph. As provided herein， the shape graph for each layer may include a number of prefixes relating to IP addresses of network destinations within a networking environment of the router. As used herein， references to merging isomorphic sub-tries to form a shape graph may refer to merging nodes of the same color. According to the example described herein， has-entry nodes， i.e.， nodes within the shape graph that relate to next hop information， may be referred to as “black nodes， ” and the other nodes may be referred to as “white nodes. ”

In some implementations， a separate shape graph may be determined in accordance with for each layer of the multi-layer binary trie according to the method 132. As one example (e.g.， for IPv4 applications) ， a 32-bit binary trie may be split into five layers， including a 9-bit layer， a 6-bit layer， a 4-bit layer， a 3-bit layer， and a 10-bit layer. As another example (e.g.， for IPv6 applications) ， a 64-bit binary trie may be split into five layers， including three 12-bit layer and two 14-bit layers.

Method 132 includes a step 148 of allocating on-chip memory of a field programmable gate array (FPGA) based on the number of graph nodes in each layer. In some implementations， on-chip memory of the FPGA is divided into a number of blocks and each block is associated with a pipeline stage corresponding to a layer of the hierarchy. In some implementations， order to guarantee line speed processing， each block is exclusively read and/or written by its associated pipeline stage. In some implementations， each block is associated to a pipeline stage containing a computation unit for the implementation of lookup instructions. FIG. 4 is a diagram that illustrates an IP lookup engine’s on-chip memory layout and FPGA pipeline. Lookup data structures (graph nodes) are stored in on-chip memory of the FPGA which is divided into a number of blocks. Each block is associated to a pipeline stage containing some computational units (e.g.， register， LUT (Look-Up-Table) ) for the implementation of lookup instructions. Each pipeline stage retrieves a graph node from its associated memory block and completes the computation task in a hardware clock cycle. For an input packet， it is output after a fixed time of delay caused by the processing of all pipeline stages. As described in further detail below， in some implementations， on-chip memory of the FPGA is distributed based on the height distribution of graph nodes for each layer-for example， in some implementations， more on-chip memory of the FPGA is distributed for a layer in which the height distribution of graph nodes is more dense.

FIGs. 5 (a) - (e) illustrate an example memory allocation for on-chip memory of an FPGA. In this example， various aspects and stages of the present disclosure are illustrated. For example， FIG. 5 (a) depicts an example routing table with various prefixes and next hop information， FIG. 5 (b) depicts a binary trie corresponding to the routing table of FIG. 5 (a) ， FIG. 5 (c) depicts a shape graph for the binary trie of (b) ， FIG. 5 (d) depicts pipeline processing stages for each height of the binary trie and shape graph) ， and FIG. 5 (e) depicts on-chip memory allocations for the shape graph.

As provided above， FIG. 5 (c) is a schematic of a shape graph that is generated from the binary trie of FIG. 5 (b) via one or more methods described herein. In some cases， only one shape graph may be generated from the binary trie. However， in other cases， binary trie 136 may be split into a number of layers， and multiple shape graphs 5 (c) may be generated from multi-layer binary trie 136. The shape graph may include a number of black nodes and white nodes. Each of the white nodes may represent one or more isomorphic sub-tries within the binary trie. The black nodes may represent leaf nodes within the binary trie. In addition， each of the black nodes may include one or more output port indexes， wherein each output port index corresponds to a particular prefix. The output port index may point to specific next hop information that is stored within an output port indexing array (see FIG. 5 (a) ) . Such next hop information may include the output port number of a particular network destination. Thus， the shape graph may be used to identify an output port number corresponding to the longest matching prefix for the network destination IP address of an IP packet. The output port number may be determined according to the shape graph lookup procedure， as discussed in further detail below.

As depicted in FIG. 5 (e) ， on-chip memory of the FPGA can， for example， be allocated substantially in proportion to the number of nodes at each height of the generated shape graph. This can， for example， provide for a resilient memory management approach that can significantly reduce the risk of out of memory at each pipeline stage. In the example depicted in FIG. 5 (e) on-chip memory is allocated for each stage depending on the number of nodes at each level of the shape graph relative to the total number of nodes in the shape graph. That is， the entire shape graph of FIG. 5 (c) includes 5 nodes， with level 1 including a single node (i.e.， 20％) . Therefore， 20％of the on-chip memory of the FPGA is allocated for level 1. Likewise，

levels

2 and 4 also include a single node and therefore are also each allocated 20％of the on-chip memory of the FPGA. Level 3 includes 2 nodes and is therefore allocated 40％ of the on-chip memory of the FPGA.

The example of FIGs. 5 (a) - (e) illustrate a proportional relationship between number of graph nodes and memory allocation. However， it is appreciated that other relationships may be used. For example， in some implementations， a given level may be allocated a greater or less than proportional amount of on-chip memory based on one or more factors. In some implementations， method 132 includes a step of permitting nodes of a lower layer to use on-chip memory of the FPGA allocated to nodes of a higher layer. This can， for example， be inherent within the memory structure of on FPGA’s on-chip memory or may be initiated by a separate activation step.

In accordance with the present disclosure， a mapping graph node of a trie node can be put in a memory block ahead of the graph nodes mapped to its children. The mapping procedure between trie nodes and graph nodes can， for example， be performed bottom up. The memory allocation is shown below in pseudo-code in Algorithm 1， which can， for example given the graph node， select a block and assign a slot for it:

With reference to the above algorithm， assuming that there are N stages of pipeline， each stage is associated to a block of on-chip memory； the graph node mapped to leaf nodes in the trie is put into the last block (Block #N) . For a trie node μ， suppose the mapping graph nodes of μ’s left and right children are put in Block #i and Block #j respectively， then the first choice is to put the graph node of μ in Block # [min (i， j) -1] . If the Block # [min (i， j) -1] is full， then it can be attempted to be placed in Block # [min (i， j) -2] . The operation can be done iteratively， when encountering a full block， until finding a non-full one. Via this method， an overflow risk (i.e.， the number of graph nodes of a specific height is more than the preset accommodations of the corresponding stage) can be significantly reduced， as the memory of higher nodes can， in some implementations， be shared with lower nodes. To facilitate the above allocation method， the value of N can， for example， be selected to be at least as large as the height of trie. After this step， a majority of memory can be distributed evenly in stages where the corresponding height distribution of graph nodes is dense.

As the number of pipeline stages is larger than the number of bits of the IP address， some stages may just do nothing (NOP) during the lookup operation in pipelines. The pseudo-code of lookup operation of each stage is shown in Algorithm 2 below. In Algorithm 2 given an input IP address ip [] ， a position of bit and an address of a graph node， the computation task is performed using a bit (indicating by the position) of the IP address， the index for the target NHI (stored in the NHI array) is calculated. In this implementation， the index is initialized as 0 in the first stage. The address of a graph node is outputted for the next stage. Example pseudo-code for Algorithm 2 is provided below:

With reference to algorithm 2 above， in each stage， the address of the current graph node is compared with the maximum/minimum address of memory block associated to the current stage. If the address of current node resides in the current stage， the index for the target NHI is calculated and shifted to the next graph node. Otherwise， no operation is performed.

Method 132 includes a step 150 of flashing the shape graph generated in step 146 onto the on-chip memory of the FPGA for pipeline processing of lookup of the routing table. It is appreciated that in some implementations， the shape graph may be stored in on-chip memory within a controller of the router (e.g.， with reference to the implementation of router 100 of FIG. 1， within controller 120) ， while other information， such as the next hop information for the various nodes， may be stored in off-chip memory locally housed within the router or otherwise in data communication with the router. As but one example， next hop information， such as port number information， for each black node may be stored in an array in off-chip memory， such as RAM. In addition， in some cases， each node of the shape graph may store the number of black nodes of the original sub-trie that is rooted at the node. On-chip memory may rely on fast FPGA memory， while the off-chip memory may rely on slower RAM. In view of the above， techniques described herein may be used to improve the routing procedure for router 100 by allowing for fast searching of the shape graph within on-chip memory of a router， and provide efficient storage of the next hop information in the off-chip memory of router 100.

It is appreciated that certain networks (e.g.， the Internet) can be rather dynamic， with network nodes going down or up. For example， when a network is unstable， for example， during Border Gateway Patrol (BGP) flapping， thousands of updates may occur in a second. To account for such fluctuations， routing tables are often updated to reflect the current status of a given network. The methods， systems， and mediums of the present disclosure can be used to address at least three types of updates to network routing: (1) modification of next hop information； (2) insertion of a new prefix； and (3) deletion of a prefix from a routing table. In accordance with certain implementations of the present disclosure， the data structure can be updated without blocking the lookup operation. For such a situation (and other situations) an incremental update algorithm and a batch update algorithm can be provided， both of which can， in some implementations， be performed on-the-fly.

As provided above， a shape graph representation of a routing table can be constructed within a router (e.g.， within controller 120 of router 100 or another suitable computing device) and then is flashed into the lookup engine. After a number of updates， a new shape can be constructed to reflect the changes in the routing table. To update the data structure in the on-chip memory， in some implementations， method 132 can include steps relating to modifying shape graphs stored on the on-chip FPGA memory 126. In some implementations， method 132 can include a step of flashing an updated shape graph onto the on-chip FPGA memory 126， wherein the updated shape graph partially overlaps an outdated shape graph previously flashed onto on-chip FPGA memory 126. Following this step， method 132 can include a step of switching entry of the outdated shape graph to the updated shape graph. Following this step， method 132 can include a step of removing the portion of the outdated shape graph that does not overlap the updated shape graph. This update procedure can， for example， be performed “bottom up” to provide that parent nodes are allocated in blocks ahead of their children nodes.

A shape graph lookup procedure can， for example， involve walking through the shape graph according to the network destination IP address of the IP packet while keeping track of a counter value， beginning at the root node with a counter value of 0. When a right node is encountered， i.e.， when an input of 1 is received， the counter value may be increased by the number of black nodes associated with the same source node. In addition， the counter value may be increased by 1 each time a black node is encountered， independently of whether the black node is a right node or a left node， e.g.， whether an input of 1 or 0 is received. The counter value at the last visited black node may be continuously updated and stored. This counter value may represent the output port index in the output port indexing array that corresponds to the appropriate output port number for the IP packet. Alternatively， if the walk ends at a black node， the final counter value may be used as the output port index.

As provided above， the generated shape graph can be used to route data to a given network destination. For example， with reference to router 100 and IP packet 102 of FIG. 1， IP packet 102 may be received at router 100 from network source 104， and a longest matching prefix corresponding to an IP address of network destination 106 for IP packet 102 may be determined within any of the shape graphs. An output port index corresponding to the longest matching prefix may be identified， and an output port number for network destination 106 may be identified based on the output port index. IP packet 102 may then be forwarded to network destination 106 via the identified output port number of router 100. Further， in some cases， multiple IP packets 102 may be routed at the same time according to a pipeline processing procedure.

It is to be understood that the process flow diagram of FIG. 2 is not intended to indicate that the steps of the method 132 are to be executed in any particular order， or that all of the steps of the method 132 are to be included in every case. Further， any number of additional steps not shown in FIG. 2 may be included within the method 132， depending on the details of the specific implementation.

FIG. 6 illustrates a diagram of a system 152 that can be used in accordance with the present disclosure. For illustration， the description of system 152 provided herein makes reference to various aspects of router 100 and other implementations of the disclosure. However， it is appreciated that system 152 may include alternative or additional functional aspects， components， etc. and is not to be limited by the above description of router 100. As described in further detail below， system 152 includes an FPGA 154， processor 156 and a memory 158 that stores machine-readable instructions that when executed by processor 156 are to allocate the on-chip memory of the FPGA based on a number of graph nodes at each height of an IP address routing table binary trie (instructions 160) and to flash shape graphs for nodes of the binary trie onto the on-chip memory of the FPGA according to the allocated on-chip memory for pipeline processing of lookup of the routing table (instructions 162) .

Instructions

160 and 162 can incorporate one or more aspects of one or more steps of method 132 (or other implementations of the present disclosure) described above. It is appreciated that additional or alternative instructions may be provided for additional or alternative functionality for any aspect of the present disclosure. The various aspects of system 152 including processor 156， memory 158， and

instructions

160 and 162 will be described in further detail below.

FPGA 154 includes a memory 164 divided into a number of blocks. Memory 164 is in the form of on-chip memory of FPGA 154. FPGA 154 can， for example， be in the form of an integrated circuit designed to be configured by a customer or a designer after manufacturing. The configuration of FPGA 154 can， for example， be specified using a hardware description language (HDL) ， similar to that used for an application-specific integrated circuit (ASIC) . FPGA 154 can， for example， contain an array of programmable logic blocks， and a hierarchy of reconfigurable interconnects that allow the blocks to be "wired together. " Such logic blocks can be configured to perform complex combinational functions as well as other logic gate functionality such as AND and XOR functionality. FPGA 154 may incorporate one or more aspects of router 100 described above， such as for example aspects of controller 120， FPGA 154， etc. in order to provide the functionality described herein.

Processor 156 of system 152 can， for example， be in the form of a central processing unit (CPU) ， a semiconductor-based microprocessor， a digital signal processor (DSP) such as a digital image processing unit， other hardware devices or processing elements suitable to retrieve and execute instructions stored in memory 158， or suitable combinations thereof. Processor 156 can， for example， include single or multiple cores on a chip， multiple cores across multiple chips， multiple cores across multiple devices， or suitable combinations thereof. Processor 156 can be functional to fetch， decode， and execute instructions as described herein. As an alternative or in addition to retrieving and executing instructions， processor 156 can， for example， include at least one integrated circuit (IC) ， other control logic， other electronic circuits， or suitable combination thereof that include a number of electronic components for performing the functionality of instructions stored on memory 158. Processor 156 can， for example， be implemented across multiple processing units and instructions may be implemented by different processing units in different areas of system 152.

Memory 158 of system 152 can， for example， be in the form of a non-transitory machine-readable storage medium， such as a suitable electronic， magnetic， optical， or other physical storage apparatus to contain or store information such as machine-

readable instructions

160 and 162. Such instructions can be operative to perform one or more functions described herein， such as those described herein with respect to the method of FIG. 2 or other methods described herein. Memory 158 can， for example， be housed within the same housing as processor 156 for system 152， such as within a computing tower case for system 152. In some implementations， memory 158 and processor 156 are housed in different housings. As used herein， the term “machine-readable storage medium” can， for example， include Random Access Memory (RAM) ， flash memory， a storage drive (e.g.， a hard disk) ， any type of storage disc (e.g.， a Compact Disc Read Only Memory (CD-ROM) ， any other type of compact disc， a DVD， etc. ) ， and the like， or a combination thereof. In some implementations， memory 158 can correspond to a memory including a main memory， such as a Random Access Memory (RAM) ， where software may reside during runtime， and a secondary memory. The secondary memory can， for example， include a nonvolatile memory where a copy of machine-readable instructions are stored. It is appreciated that both machine-readable instructions as well as related data can be stored on memory mediums and that multiple mediums can be treated as a single medium for purposes of description.

Memory 158 can be in communication with processor 156 via a communication link 166. Communication link 166 can be local or remote to a machine (e.g.， a computing device) associated with processor 156. Examples of a local communication link 166 can include an electronic bus internal to a machine (e.g.， a computing device) where memory 158 is one of volatile， non-volatile， fixed， and/or removable storage medium in communication with processor 156 via the electronic bus.

In some implementations， one or more aspects of system 152 can be in the form of functional modules that can， for example， be operative to execute one or more processes of

instructions

160 and 162 or other functions and/or instructions described herein relating to other implementations of the disclosure. As used herein， the term “module” refers to a combination of hardware (e.g.， a processor such as an integrated circuit or other circuitry) and software (e.g.， machine-or processor-executable instructions， commands， or code such as firmware， programming， or object code) . A combination of hardware and software can include hardware only (i.e.， a hardware element with no software elements) ， software hosted at hardware (e.g.， software that is stored at a memory and executed or interpreted at a processor) ， or at hardware and software hosted at hardware. It is further appreciated that the term “module” is additionally intended to refer to one or more modules or a combination of modules. Each module of a system 152 can， for example， include one or more machine-readable storage mediums and one or more computer processors.

In view of the above， it is appreciated that the various instructions of system 152 described above can correspond to separate and/or combined functional modules. For example， instructions 160 can correspond to an “allocation module” to allocate on-chip memory of the FPGA based on a number of nodes at each height of an IP address routing table binary trie segment a datastream into a plurality of blocks and instructions 162 can correspond to a “flashing module” to flash shape graphs for the nodes of the binary trie onto the on-chip memory of the FPGA according to the allocated on-chip memory for pipeline processing of lookup of the routing table. It is further appreciated that a given module can be used for multiple related functions. As but one example， in some implementations， a single module can be used to both allocate on-chip memory (e.g.， corresponding to the process of instructions 160) and to flash shape graphs onto the on-chip memory (e.g.， corresponding to the process of instructions 162) .

FIG. 7 illustrates a machine-readable storage medium 168 including various instructions stored therein that can be executed by a processor to cause processor 156 to generate shape graphs based on the nodes of the binary trie (instructions 170) ， allocate on-chip memory of a field programmable gate array (FPGA) based on a number of nodes at each height of an Internet Protocol (IP) address routing table binary trie (instructions 172) ， and flash the generated shape graphs onto the on-chip memory of the FPGA according to the allocated on-chip memory (instructions 174) .

Instructions

170， 172， and 174 can incorporate one or more aspects of one or more steps of method 132 (or other implementations of the present disclosure) described above. It is appreciated that additional or alternative instructions may be provided for additional or alternative functionality for any aspect of the present disclosure.

For illustration， the description of machine-readable storage medium 168 provided herein makes reference to various aspects of system 152 (e.g.， processor 156) ， router 100， and other implementations of the disclosure. Although one or more aspects of system 152 (as well as its corresponding

instructions

170， 172， and 174) can be applied or otherwise incorporated with medium 168， it is appreciated that in some implementations， medium 168 may be stored or housed separately from such a system. For example， in some implementations， medium 168 can be in the form of Random Access Memory (RAM) ， flash memory， a storage drive (e.g.， a hard disk) ， any type of storage disc (e.g.， a Compact Disc Read Only Memory (CD-ROM) ， any other type of compact disc， a DVD， etc. ) ， and the like， or a combination thereof.

While certain implementations have been shown and described above， various changes in form and details may be made. For example， some features that have been described in relation to one implementation and/or process can be related to other implementations. In other words， processes， features， components， and/or properties described in relation to one implementation can be useful in other implementations. Furthermore， it should be appreciated that the systems and methods described herein can include various combinations and/or sub-combinations of the components and/or features of the different implementations described. Thus， features described with reference to one or more implementations can be combined with other implementations described herein.

As used herein， “logic” is an alternative or additional processing resource to perform a particular action and/or function， etc.， described herein， which includes hardware， e.g.， various forms of transistor logic， application specific integrated circuits (ASICs) ， etc.， as opposed to machine executable instructions， e.g.， software firmware， etc.， stored in memory and executable by a processor. Further， as used herein， “a” or “anumber of” something can refer to one or more such things. For example， “anumber of widgets” can refer to one or more widgets. Also， as used herein， “a plurality of” something can refer to more than one of such things.

Claims

A method comprising:

splitting a binary trie representing a routing table of a router into a plurality of layers of a hierarchy, wherein each layer of the plurality of layers comprises a plurality of nodes；

determining a plurality of groups of isomorphic nodes for each layer；

generating a shape graph by merging the isomorphic nodes within each of the plurality of groups for each layer；

allocating on-chip memory of a field programmable gate array (FPGA) based on the number of graph nodes in each layer； and

flashing the shape graph onto the on-chip memory of the FPGA for pipeline processing of lookup of the routing table.
The method of claim 1, wherein on-chip memory of the FPGA is distributed based on a height distribution of graph nodes for each layer.
The method of claim 2, wherein more on-chip memory of the FPGA is distributed for a layer in which the height distribution of graph nodes is more dense.
The method of claim 1, further comprising:

permitting nodes of a lower layer to use on-chip memory of the FPGA allocated to nodes of a higher layer.
The method of claim 1, wherein the routing table stores Internet Protocol (IP) addresses, and wherein the shape graph is flashed onto the on-chip memory of the FPGA to allow pipeline processing of IP address lookups.
The method of claim 1, wherein the on-chip memory of the FPGA is divided into a number of blocks and each block is associated with a pipeline stage corresponding to a layer of the hierarchy.
The method of claim 6, wherein each block contains a computation unit for the implementation of lookup instructions.
The method of claim 7, wherein the computational unit is a Look-Up-Table (LUT) .
The method of claim 1, wherein flashing the shape graph onto the on-chip memory of the FPGA includes mapping a first graph node in a memory block ahead of graph nodes mapped to children of the first graph node.
The method of claim 1, further comprising:

flashing an updated shape graph onto the on-chip memory of the FPGA, wherein the updated shape graph partially overlaps an outdated shape graph previously flashed onto the on-chip memory of the FPGA；

switching entry of the outdated shape graph to the updated shape graph； and

removing the portion of the outdated shape graph that does not overlap the updated shape graph.
A non-transitory machine-readable storage medium having stored thereon machine readable instructions to cause a computer processor to:

generate shape graphs based on nodes of the binary trie；

allocate on-chip memory of a field programmable gate array (FPGA) based on a number of nodes at each height of an Internet Protocol (IP) address routing table binary trie； and

flash the generated shape graphs onto the on-chip memory of the FPGA according to the allocated on-chip memory.
The medium of claim 11, wherein on-chip memory of the FPGA is allocated substantially in proportion to the number of nodes at each height of the generated shape graph.
A system comprising:

a field programmable gate array (FPGA) including a first memory divided into a number of blocks, wherein the first memory is on-chip memory of the FPGA；

a processor； and

a second memory storing machine readable instructions to cause the processor to:

allocate the on-chip memory of the FPGA based on a number of nodes at each height of an Internet Protocol (IP) address routing table binary trie； and

flash shape graphs for nodes of the binary trie onto the on-chip memory of the FPGA according to the allocated on-chip memory for pipeline processing of lookup of the routing table.
The system of claim 13, wherein the second memory is in the form of computer data storage.
The system of claim 13, wherein each block of the first memory is associated to a pipeline stage containing computational units for the implementation of lookup instructions.