CN102724078A - End-to-end network flow reconstruction method based on compression sensing in dynamic network - Google Patents

Abstract

The invention discloses an end-to-end network flow reconstruction method based on compression sensing in a dynamic network. In a large-scale IP backbone network, the OD flow is selected by a random walk method; the flow value of partial OD flow acquired by a router subset is described by constructing a sparse flow matrix; a compressing sensing reconstruction module is established by adopting a main information analysis method; a relationship between the OD flow generated by the router sublet and all the end-to-end OD flow in the IP backbone network is descried by using the module, and further all the end-to-end OD flow of the whole IP backbone network is determined. The method can more accurately acquire end-to-end network flow detail characteristic, cannot consume mass hardware resources, can track the dynamic changes of the OD flow in a real time manner, and the reconstruction error is less.

Description

Under a kind of dynamic network based on the end to end network flow reconstructing method of compressed sensing

Technical field

The present invention belongs to that the dynamic network down-off is measured and analysis field, is specifically related under a kind of dynamic network the end to end network flow reconstructing method based on compressed sensing.

Background technology

In recent years, along with rapid development of Internet, though the service that the network application of being on the increase provides users with the convenient.But also make network become complicated day by day.For Virtual network operator, Network Management with control also more and more difficult.Traffic matrix is a most important input parameter in the network traffic engineering; All end-to-end fluxes in the expression network; And the distribution of flow has intactly been described, to network manager current network state is provided, but in practical application, is difficult to accurately obtain traffic matrix.

Even traffic matrix is extremely important for network operator, but the estimated value that obtains traffic matrix exactly is very difficult.Therefore, traffic matrix estimates that becoming one has challenging research topic.In recent years, emerged a large amount of achievements in research.Vardi proposes to use the network tomography method to solve end-to-end flux reconstruct problem, and this subsequently method is found broad application, and is used to study the IP network internal feature.For the network tomography method, end-to-end flux can still can not be caught the correlation of the room and time of end-to-end flux through Poisson distribution and Gaussian distribution reconstruct; People such as P.Conti utilize maximal possibility estimation algorithm computation Hurst value to come the estimated flow matrix.People such as Nucci add weight to the routing configuration that derives from a plurality of routing iinformations, obtain little traffic matrix evaluated error with this.Y.Zhang etc. have proposed to describe with gravity field model the characteristic of present end-to-end flux, through obtaining extra constraint information, to overcome the highly problem of morbid state; A.Lakhina etc. have proposed PCA and have directly measured and made up the end-to-end flux reconstruction model; A.Soule etc. propose iteration Bayes inversion algorithm and come the reconstruct end-to-end flux based on the independent same distribution Poisson model hypothesis of end-to-end flux; G.Liang etc. have proposed a kind of pseudo-likelihood reconstructing method, use improved EM algorithm with PROBLEM DECOMPOSITION for several comprise a subproblem that OD is right, the error of estimation precision is decreased.

Though the statistical model correlation technique can obtain the estimated value of traffic matrix, and is widely used in the middle of the reality, these methods are estimated flow matrix exactly still.In fact directly measure end-to-end network traffics specific discharge and estimate more accurately, and equipment manufacturers provide a large amount of measuring technique and equipment, and this is wherein most typical to be exactly Netflow.Netflow can be through extracting and analyze source, the purpose IP address of packet, and source, destination slogan and protocol number obtain network flow value end to end.But it is unpractical being to use NetFlow to measure each bar OD stream, or even can not realize.This is because the port of router is when moving the NetFlow program, the hardware resource that consumption is a large amount of, and directly measurement will cause extra communication overhead.So just reduced the storing and forwarding efficiency of router, so increased network delay and caused network congestion easily.

Summary of the invention

To the deficiency of prior art, the present invention proposes under a kind of dynamic network the end to end network flow reconstructing method based on compressed sensing, to reach the minimizing hardware resource, the dynamic change of real-time tracking OD stream, the purpose of less reconstructed error.

Based on the end to end network flow reconstructing method of compressed sensing, may further comprise the steps under a kind of dynamic network:

Step 1, the system that is provided with flow length, the random walk number of times of random walk to OD;

Step 2, in the large-scale ip backbone network, adopt the mode of random walk to select OD stream, adopt the method that makes up boolean's sparseness measuring matrix to describe the opening and closing state of whole routers according to selected OD stream;

The flow value that step 3, the part of O D that adopts the method for constructing the rarefied flow moment matrix to describe the collection of router subclass flow; Described router subclass is the router of whole openings, and calculates measured value according to boolean's sparseness measuring matrix and the rarefied flow moment matrix of being constructed;

Step 4, according to the sparseness measuring matrix of being constructed in the step 3 with calculate the gained measured value; Adopt the principal component analysis method to make up the compressed sensing reconstruction model; With this model the relation of whole end-to-end OD streams in OD stream that the router subclass produces and the large-scale ip backbone network is described, and then whole OD stream flow end to end in definite entire I P backbone network.

Structure boolean sparseness measuring matrix described in the step 2 may further comprise the steps:

Step 2-1, an OD stream of structure complete graph comprise summit and limit in the complete graph, the number on summit is equal to the number of OD stream;

Step 2-2: random walk is set measures the number of times initial value;

Step 2-3: select a summit as unified starting point;

Step 2-4: complete graph is carried out equally distributed random walk, and according to the point that random walk is found, the router state that this point is corresponding then is an opening, otherwise is closed condition;

Step 2-5: less than initialization random walk measurement number of times is set if number of times is measured in random walk, then returns step 2-3; If random walk measurement number of times is greater than or equal to initialization random walk measurement number of times, then execution in step 2-6 is set;

Step 2-6: preserve boolean's sparseness measuring matrix.

Boolean's sparseness measuring matrix described in the step 1, this matrix meet constraint equidistance criterion; Line number in boolean's sparseness measuring matrix is represented the number of times of random walk large-scale ip backbone network, and the columns in boolean's sparseness measuring matrix is represented the sum of OD stream in the large-scale ip backbone network.

Following formula is satisfied in the calculating of the measured value described in the step 3:

Measured value=boolean's sparseness measuring matrix * rarefied flow moment matrix

Advantage of the present invention:

Based on the end to end network flow reconstructing method of compressed sensing, realize large-scale IP backbone is carried out network traffics reconstruct end to end under a kind of dynamic network of the present invention through the direct metering system of part.Can obtain network traffics details characteristic end to end more accurately, can't consume a large amount of hardware resources, can real-time tracking OD the dynamic change of stream, less reconstructed error.

Description of drawings

Fig. 1 is based on the end to end network flow reconstructing method flow chart of compressed sensing under the dynamic network of an embodiment of the present invention;

Fig. 2 is an embodiment of the present invention flow awareness reconstruct framework sketch map;

Fig. 3 makes up boolean's sparseness measuring matrix flow chart for an embodiment of the present invention;

Fig. 4 makes up boolean's sparseness measuring matrix sketch map for an embodiment of the present invention random walk;

Wherein, 4-1 is a complete graph;

Fig. 5 is the order variation sketch map of an embodiment of the present invention NMAE with traffic matrix;

Fig. 6 is that an embodiment of the present invention NMAE is with random walk length variations sketch map;

Fig. 7 needs the directly OD flow amount sketch map of measurement for an embodiment of the present invention;

Fig. 8 is that an embodiment of the present invention is estimated sketch map to the 30th, the 60th OD stream traffic matrix; A) be that the 30th OD stream traffic matrix estimated sketch map; B) be that the 60th OD stream traffic matrix estimated sketch map;

Fig. 9 is an embodiment of the present invention space relative error and time relative error sketch map; A) be space relative error sketch map; B) be time relative error sketch map;

Figure 10 is an embodiment of the present invention space relative error and time error cumulative distribution function sketch map; A) be space relative error cumulative distribution function sketch map; B) be time relative error cumulative distribution function sketch map.

Embodiment

Below in conjunction with accompanying drawing practical implementation of the present invention is further specified.

The embodiment of the invention at first makes up boolean's sparseness measuring matrix according to random walk, obeys RIP (constraint equidistance) criterion; Corresponding then boolean's sparseness measuring matrix is collected the part end-to-end flux with less router port, calculates measured value according to the linear relationship between boolean's sparseness measuring matrix, rarefied flow moment matrix and the measured value; Utilize PCA (principal component is analysed) model that traffic matrix is carried out singular value decomposition to satisfy the rarefaction condition, based on all OD stream traffic matrixs of compressed sensing reconstruct at last.

Use the real traffic data of Abilene backbone network in the embodiment of the invention, it has 12 nodes, 30 interior links, and 24 outer links, 144 end-to-end fluxes, emulated data adopts the 5min time interval, amounts to 2016 moment.

Traffic matrix in the Abilene backbone network can be shown in formula (1),

$M=\left[\begin{array}{cccc}{m}_1\left(1\right)& m_1\left(2\right)& ...& m_1\left(T\right)\\ m_2\left(1\right)& m_2\left(2\right)& ...& m_2\left(T\right)\\ .& .& & .\\ .& .& & .\\ .& .& & .\\ m_N\left(1\right)& m_N\left(2\right)& ...& m_N\left(T\right)\end{array}\right]---\left(1\right)$

M in the formula (1) represents traffic matrix, and source node is to the flow total amount of destination node in the extensive Abilene backbone network of row representative of matrix M.

Wherein, number of network node is n, OD fluxion N=n ²Therefore, M is the traffic matrix of a N * T.

m _j(t): the t moment of representing j bar OD stream;

M (t): the t row of representing M;

j∈{1,2,…,N}；

t∈{1,2,…,T}。

Compressed sensing is theoretical must to satisfy two characteristics: the one, and measure matrix and satisfy RIP (constraint equidistance) criterion; The one, traffic matrix is a sparse matrix.The requirement that regards to down in the compressed sensing theory for matrix combines formula (2) to describe.

Because the redundancy of signal supposes that vectorial s can be expressed as:

Wherein,

S is discrete signal N dimension space R ^NIn a N * 1 rank column vector;

x _iBe every column element of s, i=1,2,3 ..., N;

Be base vector,

Be R ^NOne group of orthogonal basis.

And if only if, and signal is by K (K<<n) rank base vector

During the linear combination that constitutes signal s be only sparse, i.e. { x in formula (2) _iTo have only K element be non-zero.Signal s is considered to compressible in the compressed sensing theory so.K also becomes the degree of rarefication of signal.

The length that random walk is set in the embodiment of the invention is r; The measurement number of times is p; Random walk number of times p=19 in

embodiment of the invention wherein; Duration of random walk r=11; The order q=5 of traffic matrix (q=K), boolean's sparseness measuring matrix that the method makes up is obeyed the RIP criterion.

Fig. 1 be under the dynamic network of the embodiment of the invention based on the end to end network flow reconstructing method flow chart of compressed sensing, may further comprise the steps:

Step 1, the order of the length, random walk number of times and the traffic matrix that are directed to the router random walk in the system is set;

The order of described traffic matrix is the degree of rarefication that satisfies the theoretical matrix of compressed sensing, the order q=5 of traffic matrix in the present embodiment;

Step 2, in the large-scale ip backbone network, adopt the mode of random walk to select OD stream, adopt the method that makes up boolean's sparseness measuring matrix to describe the opening and closing state of whole routers according to selected OD stream;

Adopt the mode of random walk to select router in the embodiment of the invention; The router that is selected is formed the router subclass; Adopt the method that makes up boolean's sparseness measuring matrix to describe the router subclass that is selected and open or closed condition, make up boolean's sparseness measuring matrix A of a p * N according to p random walk.Fig. 2 is a flow awareness reconstruct framework sketch map of the present invention; The network operator utilizes NetFlow to collect the end to end network data on flows on the router that partly moves through control network devices (comprising hardware and software); Be the unlatching of network management center control router or close, and collect data on flows from the NetFlow that each router interface moves.After collecting part end to end network flow, recover whole OD stream traffic matrixs at network management center.Fig. 3 specifically may further comprise the steps for the embodiment of the invention makes up boolean's sparseness measuring matrix flow chart:

Step 2-1, structure OD flow complete graph G, and (wherein, V and E distinguish the set on representative of graphics summit and limit for V, E) (like complete graph 4-1 among Fig. 4).The number on summit is equal to the number N of OD stream;

Step 2-2: be provided with and measure the number of times parameter c _Iteration=1;

Step 2-3: select a summit as unified starting point;

Step 2-4: sharp (V E) carries out equally distributed random walk, and according to the point that random walk is found, the router state that this point is corresponding then is opening (being 1), otherwise is closed condition (being 0) to G;

Fig. 4 makes up boolean's sparseness measuring matrix sketch map for random walk of the present invention, and wherein, N bar OD stream is represented on the summit of black, and random walk is carried out on the summit.The summit 10,15 in the embodiment of the invention, and 18,70 and 72 is summits that random walk is found, and use NetFlow to measure the 10th, 15,18,70 and 72 OD streams.a _iThe row of the boolean's sparseness measuring matrix that makes up for random walk through the inferior random walk of p (p=19), obtains boolean's sparseness measuring matrix A of a p * N (19 * 144), and is as follows:

$A=\left[\begin{array}{c}{a}_1\\ a_2\\ .\\ .\\ .\\ a_p\end{array}\right]---\left(3\right)$

a _iBe that a capable vector is:

a _i=[a _i1,a _i2,a _i3,…,a _iN],i∈{1,2,3,…,p}. (4)

When a summit was found by random walk, its corresponding matrix element value was 1.Among Fig. 3, a _iThe 10th, 15,18,70,72 element values are " 1 ", and all the other are " 0 ", and a promptly is set _I10, a _I15, a _I18, a _I70And a _I72" 1 ".

Step 2-5: if c _Iteration<p then is provided with c _Iteration=c _Iteration+ 1, carry out and return step 204; If c _Iteration>P, then execution in step 207;

Step 2-6: preserve boolean's sparseness measuring matrix.

The flow value that step 3, the part of O D that adopts the method for constructing the rarefied flow moment matrix to describe the collection of router subclass flow; Described router subclass is the router of whole openings, and calculates measured value according to boolean's sparseness measuring matrix and the rarefied flow moment matrix of being constructed;

In the embodiment of the invention, according to formula (5)～(7) all network traffics are carried out zero-mean and handle.

In the embodiment of the invention, one section of the historical data on flows of intercepting as prior information, uses M _HisExpression.The zero-mean processing procedure is expressed as:

M'=M _his-M ^mean (5)

Wherein, M ' is the rarefied flow moment matrix of zero-mean after handling;

M _HisBe historical data on flows matrix;

$M^{\mathrm{mean}}=\left[\begin{array}{c}{\mathrm{<figure-callout\; id="1"\; label="m"\; filenames="HDA00001834071600042.png,HDA00001834071600051.png"\; state="\{\{state\}\}">m</figure-callout>}}_1^{\mathrm{mean}},{\mathrm{<figure-callout\; id="1"\; label="m"\; filenames="HDA00001834071600042.png,HDA00001834071600051.png"\; state="\{\{state\}\}">m</figure-callout>}}_1^{\mathrm{mean}},{\mathrm{<figure-callout\; id="1"\; label="m"\; filenames="HDA00001834071600042.png,HDA00001834071600051.png"\; state="\{\{state\}\}">m</figure-callout>}}_1^{\mathrm{mean}},...,{\mathrm{<figure-callout\; id="1"\; label="m"\; filenames="HDA00001834071600042.png,HDA00001834071600051.png"\; state="\{\{state\}\}">m</figure-callout>}}_1^{\mathrm{mean}}\\ .\\ .\\ .\\ m_N^{\mathrm{mean}},m_N^{\mathrm{mean}},m_N^{\mathrm{mean}},...,m_N^{\mathrm{mean}}\end{array}\right]---\left(6\right)$

Formula (6) is the matrix of a N * T, and its element is just like giving a definition:

$m_j^{\mathrm{mean}}=\frac{1}{T}\underset{t=1}{\overset{T}{\mathrm{\&Sigma;}}}{m}_j\left(t\right)---\left(7\right)$

Wherein: T is the length of the historical flow of intercepting;

In the embodiment of the invention, utilize historical traffic matrix to calculate measured value Y according to formula (8)～(10).

T measured value y (t) constantly.

$\left[\begin{array}{c}{y}_1\left(t\right)\\ y_2\left(t\right)\\ .\\ .\\ .\\ y_p\left(t\right)\end{array}\right]=\left[\begin{array}{c}{a}_1\\ a_2\\ .\\ .\\ .\\ a_p\end{array}\right]\&times;\left[\begin{array}{c}{m}_1\left(t\right)\\ m_2\left(t\right)\\ .\\ .\\ .\\ m_N\left(t\right)\end{array}\right].---\left(8\right)$

Formula (8) is equivalent to,

y(t)=A×m(t) (9)

To all constantly, following formula is arranged:

$Y=A\&times;M^{\&prime;}$

$=\left[\begin{array}{c}{a}_1\\ a_2\\ .\\ .\\ .\\ a_p\end{array}\right]\&times;\left[\begin{array}{cccc}{m}_1\left(1\right)& m_1\left(2\right)& ...& m_1\left(T\right)\\ m_2\left(1\right)& m_2\left(2\right)& ...& m_2\left(T\right)\\ .& .& & .\\ .& .& & .\\ .& .& & .\\ m_N\left(1\right)& m_N\left(2\right)& ...& m_N\left(T\right)\end{array}\right]---\left(10\right)$

Wherein, measured value Y, boolean's sparseness measuring matrix A and rarefied flow moment matrix M ' they are respectively p * T, the matrix of p * N and N * T.

Because A is a sparse random matrix, so we only need know that element value corresponding with A in the M ' matrix in the formula (10) can obtain measured value Y.

In the embodiment of the invention, obtain the union of the OD adfluxion of all random walks measurements through formula (11).

Suppose that

representative needs the OD adfluxion of measuring through the i time random walk, all essential OD streams are expressed as:

$\left\{\begin{array}{c}{S}^f={\&cup;}_{i=1}^p{S}_i^f\\ p\&le;N\end{array}\right.---\left(11\right)$

Wherein: S ^fRepresentative is asked union to the OD stream that generates from the random walk first time to the p time random walk;

representative needs the OD adfluxion through the i time random walk measurement;

Therefore, the number that generates the corresponding essential OD stream of all essential OD streams

is traffic matrix so for

$\stackrel{\stackrel{\&OverBar;}{\&OverBar;}}{M}=\left[\begin{array}{c}{m}_1\left(t\right)\\ m_2\left(t\right)\\ .\\ .\\ .\\ m_N\left(t\right)\end{array}\right]---\left(12\right)$

Wherein:

is for comprising the sparse matrix of essential OD stream;

each row is represented an OD stream time series; Extraction needs

the bar OD that directly measures to flow, the behavior 0 of corresponding

matrixes of all the other OD streams.In formula (10) since A be boolean's sparseness measuring matrix, so just can obtain Y as long as know the respective columns among the M '.That is to say that Y=A * M ' is equivalent to

Specify the method for selecting directly to measure OD stream subclass below for example.Shown in formula (13):

$\begin{array}{c}\begin{array}{ccc}Y\left(t\right)& A_{(5\&times;10)}& m\left(t\right)\end{array}\\ \left[\begin{array}{c}{y}_1\left(t\right)\\ y_2\left(t\right)\\ y_3\left(t\right)\\ y_4\left(t\right)\\ y_5\left(t\right)\end{array}\right]=\left[\begin{array}{c}0001010100\\ 0001100000\\ 1000010000\\ 1000110000\\ 0001000000\end{array}\right]\&times;\left[\begin{array}{c}{\mathrm{<figure-callout\; id="1"\; label="OD"\; filenames="HDA00001834071600042.png,HDA00001834071600051.png"\; state="\{\{state\}\}">OD</figure-callout>}}_1\\ 0\\ 0\\ {\mathrm{<figure-callout\; id="4"\; label="OD"\; filenames="HDA00001834071600041.png,HDA00001834071600051.png"\; state="\{\{state\}\}">OD</figure-callout>}}_4\\ {\mathrm{<figure-callout\; id="5"\; label="OD"\; filenames="HDA00001834071600061.png"\; state="\{\{state\}\}">OD</figure-callout>}}_5\\ {\mathrm{<figure-callout\; id="6"\; label="OD"\; filenames="HDA00001834071600041.png,HDA00001834071600061.png"\; state="\{\{state\}\}">OD</figure-callout>}}_6\\ 0\\ {\mathrm{<figure-callout\; id="8"\; label="OD"\; filenames="HDA00001834071600041.png"\; state="\{\{state\}\}">OD</figure-callout>}}_8\\ 0\\ 0\end{array}\right]\end{array}---\left(13\right)$

Boolean's sparseness measuring matrix A is the sparse matrix that " 1 " or " 0 " is formed in the formula, and on behalf of the number of times of random walk, columns, the line number of A represent the OD stream of each random walk traversal.T at any time, the OD stream that utilizes " 1 " index in the matrix A column element to measure.A among Fig. 4 (5 row, 10 row) boolean's sparseness measuring matrix so, i.e. random walk 5 times, the OD fluxion that travel through is 10.Utilize for the first time column element among the A " 1 " manipulative indexing to the OD flow point be OD4, OD6, OD8; The OD stream that indexes for the second time is OD4, OD5; By that analogy, the OD stream that obtains altogether measuring through 5 random walks is respectively: OD1, OD4, OD5, OD6, OD8.

Step 4, according to the sparseness measuring matrix of being constructed in the step 3 with calculate the gained measured value; Adopt PCA (principal component is analysed) method to make up the compressed sensing reconstruction model; With this model the relation of whole end-to-end OD streams in OD stream that the router subclass produces and the large-scale ip backbone network is described, and then whole OD stream flow end to end in definite entire I P backbone network.

For satisfying the requirement of compressed sensing traffic matrix rarefaction characteristic, use pca model that historical traffic matrix

is carried out singular value decomposition in the embodiment of the invention.

Step 4-1: measured value Y is carried out singular value decomposition;

Below in conjunction with formula (14)～(17) principle of singular value decomposition is further specified.

The main thought of principal component analysis (PCA) is that high dimensional data is projected to than lower dimensional space.M in the pca model * n rank traffic matrix L is expressed as

L=U∑V ^T， (14)

Wherein: U is an orthogonal matrix;

UU ^TOr U ^TU is a unit matrix;

∑ is the singular value of the diagonal matrix of L for the diagonal element class value;

V is L ^TThe characteristic vector of L;

L ^TTransposition for L.

In fact, formula (14) is that L utilizes orthogonal basis U ∑ to space projection.The L matrix column is base vector σ _ku _kLinear combination, the normalization base vector is expressed as:

$u_k=\frac{{\mathrm{Lv}}_k}{{\mathrm{\&sigma;}}_k},k=\mathrm{1,2},...,\min (m,n),---\left(15\right)$

Wherein: u _k, v _kRepresent the k row of U and V respectively;

(m n) represents the order of matrix L to min;

σ _kIt is the energy feature that the singular value of matrix L is represented L.

Understand the low-rank approximate procedure for ease, formula (14) can be expressed as:

$L={\mathrm{U\&Sigma;V}}^T=\underset{k=1}{\overset{\min (m,n)}{\mathrm{\&Sigma;}}}{\mathrm{\&sigma;}}_k{u}_k{v}_k^T,---\left(16\right)$

Therefore, can select big singular value to remove approximate ewal matrix, this process can be explained with mathematical formulae:

$\hat{L}=\underset{k=1}{\overset{q}{\mathrm{\&Sigma;}}}{\mathrm{\&sigma;}}_k{u}_k{v}_k^T.---\left(17\right)$

Wherein: constant q < min (m, n);

σ _kBe q maximum singular value;

Q is the order of

, and it equals degree of rarefication.

In the embodiment of the invention, suppose historical traffic matrix M _HisChang Dud>N.According to formula (14),

can be decomposed into:

$M_{\mathrm{his}}^T=U_{\mathrm{his}}{\mathrm{\&Sigma;}}_{\mathrm{his}}{V}_{\mathrm{his}}^T,---\left(18\right)$

Wherein, U _His, ∑ _HisWith

Be respectively d * N, the matrix of N * N and N * N.

Step 4-2: according to historical traffic matrix and singular value decomposition method, calculating does further analysis.

Can calculate V according to formula (18) _His∑ _HisBecause every OD stream in the traffic matrix all has long correlation property, principal component remains unchanged basically.Therefore, in conjunction with formula (18), formula (10) can be similar to and be converted into:

Y=A×V _his∑ _hisU ^T=ΘU ^T, (19)

Wherein, AV _His∑ _His=Θ.

According to formula (19), the preceding q row of intercepting Θ have new matrix Θ _ColExpression;

According to formula (16), Y can approximate representation be:

$Y=\mathrm{\&Theta;}{U}^T=\mathrm{\&Theta;}\left[\begin{array}{c}{u}_{\mathrm{PC}}^T\\ u^T\end{array}\right]\&ap;\mathrm{\&Theta;}\left[\begin{array}{c}{\mathrm{<figure-callout\; id="0"\; label="u\; PC\; T"\; filenames="HDA00001834071600041.png,HDA00001834071600042.png"\; state="\{\{state\}\}">u</figure-callout>}}_{\mathrm{PC}T}^{}\\ 0\end{array}\right]=\mathrm{\&Theta;}{U}_{\mathrm{PC}}^T,---\left(20\right)$

Wherein, And u ^TBe respectively U ^TQ * T (q<p) and (N-q) * T rank submatrix;

is a sparse matrix that degree of rarefication is q.

Step 4-3: realize the matrix rarefaction after the singular value decomposition through protruding optimization method:

${\hat{U}}_{\mathrm{PC}}^T=\arg \min {\left|\right|U_{\mathrm{PC}}^T\left|\right|}_{1}s.t.\mathrm{\&Theta;}{U}_{\mathrm{PC}}^T=Y,---\left(21\right)$

Where:

such that take the minimum time value;

So, formula (20) can be transformed to:

$Y={\mathrm{\&Theta;}}_{\mathrm{col}}{u}_{\mathrm{PC}}^T,---\left(22\right)$

Wherein, Θ _ColBe the q of Θ<the p row.Therefore, l ₂The norm minimization problem can be used for finding the solution u _PC, that is,

${\hat{u}}_{\mathrm{PC}}^T=\arg \min {\left|\right|u_{\mathrm{PC}}^T\left|\right|}_{2}s.t.{\mathrm{\&Theta;}}_{\mathrm{col}}{u}_{\mathrm{PC}}^T=Y,---\left(23\right)$

With this, formula (22) is an overdetermined problem, is easy to find optimization to solve scheme.

From formula (22), can know Θ _ColAnd u _PCBe the row non-singular matrix.Therefore, can obtain following formula,

$Y=u_y{\mathrm{\&Sigma;}}_y{v}_y^T.---\left(24\right)$

Wherein, matrix u _ySingular value decomposition through Y obtains;

Matrix Be Y ^TThe characteristic vector of Y;

Because Θ _ColAnd u _PCAll be row non-singular matrix, then Θ _ColOrder equal u _yOrder (rank (Θ _Col)=rank (u _y)) and with Θ _ColThe space of launching for base equals with u _PCSpace (span (Θ for the base expansion _Col)=span (u _y)).

Step 4-4: a nearest step of matrix is analyzed according to orthogonal transform.

Calculate R according to formula (25);

Orthogonal transform matrix is obeyed following constraints, exists matrix R to satisfy following formula,

u _y=Θ _colR. (25)

According to formula (26); (27), calculate

Hypothesis matrix W satisfies

With W Θ _Col=I, I are unit matrix.

Therefore,

$\mathrm{WY}=W{\mathrm{\&Theta;}}_{\mathrm{col}}{u}_{\mathrm{PC}}^T=u_{\mathrm{PC}}^T.---\left(26\right)$

In addition,

Wu _y=WΘ _colR=R. (27)

According to (28), calculate

Can obtain according to formula (26) and (27):

$\left\{\begin{array}{c}\mathrm{WY}=u_{\mathrm{PC}}^T\\ {\mathrm{Wu}}_y=R\end{array}\right..---\left(28\right)$

$u_{\mathrm{PC}}^T=\left({\mathrm{Ru}}_y^T\right)Y---\left(29\right)$

Obtain

according to above step

Through calculating

Obtain traffic matrix M according to formula (18) _HisEstimated value.

Through above step, we can obtain the estimated result of network traffics matrix according to directly measuring part network traffics end to end.This matrix description all flow sizes end to end in the large scale IP backbone.

The Performance Evaluation result:

Fig. 5 is the order variation sketch map of embodiment of the invention NMAE with traffic matrix, and (NMAE) assesses performance of the present invention with the normalization mean absolute error, and the value of order q is respectively 1,2 in the embodiment of the invention, 3,4,5,6,7,8 (as shown in Figure 6).

The NMAE computing formula is:

$\mathrm{NMAE}=\frac{\underset{j,t}{\mathrm{\&Sigma;}}|{\hat{m}}_j\left(t\right)-m_j\left(t\right)|}{\underset{j,t}{\mathrm{\&Sigma;}}\left|m_j\left(t\right)\right|}.---\left(29\right)$

Wherein, represents t j bar OD flow estimated value constantly;

m _j(t) represent t j bar OD flow actual value constantly.

Simulation result shows along with q changes, not too big variation of algorithm performance of the present invention.Yet q receives some constraints, and at first order q is equivalent to degree of rarefication.That is, in formula (20), when order is q, U ^TCan by

It is approximate,

Row be that q is sparse.Boolean's sparseness measuring matrix A is the matrix of a p * N, p=O (Klog (N/K)).In the emulation, K=q, therefore, p=O (qlog (N/q)).Measuring number of times p directly influences the total amount of the OD stream that needs measurement.In the embodiment of the invention, the total amount that q equals OD stream is set according to p.In Fig. 5, a fluctuation appears in NMAE, is because p this moment is tending towards flowing border qlog (N/q).In the case, p-qlog (N/q) is more little, and the reconstruct failed probability is big more.Fig. 6 be embodiment of the invention NMAE with random walk length variations sketch map, NMAE is along with the length of random walk increases on a declining curve basically.If r is enough little, robustness will descend so; If r is too big, can produce a part of OD that we have to measure stream again.

Shown in Figure 7, explain that OD stream chooses the number Normal Distribution, on behalf of OD, the intercept of straight line flow the average of choosing number, and slope is represented standard deviation.Simulation result shows: the about 105-120 bar of our needs OD diffluence recovers traffic matrix.In the flow compression reconfiguration, adopt the PCA method in the embodiment of the invention, and do not carried out other preliminary treatment, therefore, the embodiment of the invention needs 60% link operation NetFlow.

In Fig. 8, preceding 500 points are used to calculate V as prior information in algorithm _His∑ _HisThe 30th and the 60th OD stream of embodiment of the invention selection is at random tested.We find that three kinds of methods can track the dynamic change of flow in time.Yet in Fig. 8 (a), bigger fluctuation appears in SRSVD (sparse reconstruct singular value decomposition) and TomoG (Gravity Models), and FSR (flow sensing reconstructing) can reach and obtain the 30th OD stream exactly; Fig. 8 (b) clearly finds out accurately 60 OD streams of reconstruct of SRSVD and TomoG, and FSR can obtain the trend of changes in flow rate more accurately.Therefore, the better network traffics of estimating peer-to-peer of our method of experimental result proof.

In order to assess the advantage of algorithm of the present invention more accurately, the embodiment of the invention contrasts this three kinds of methods with the another one standard.Because network traffics demonstrate time variation and temporal correlation, the embodiment of the invention goes to analyze the characteristic of FSR with reference to space relative error (SREs) and time relative error (TREs).

SREs and TREs are expressed as:

$\mathrm{SRE}\left(n\right)=\frac{{\left|\right|{\hat{x}}_T\left(n\right)-x_T\left(n\right)\left|\right|}_2}{{\left|\right|x_T\left(n\right)\left|\right|}_2},n=\mathrm{1,2},...,N---\left(29\right)$

Wherein:

expression T n bar OD flow estimated value constantly;

x _T(n) expression T n bar OD flow actual value constantly.

$\mathrm{TRE}\left(t\right)=\frac{{\left|\right|{\hat{x}}_N\left(t\right)-x_N\left(t\right)\left|\right|}_2}{{\left|\right|x_N\left(t\right)\left|\right|}_2},t=\mathrm{1,2},...,T---\left(30\right)$

Wherein: expression N bar OD flow is in t estimated value constantly;

x _N(t) expression N bar OD flow is at t actual value constantly.

Here nonnegative integer N and T represent OD stream sum respectively and measure constantly.|| || ₂Expression l ₂Norm.The space error that SERs has embodied different OD streams distributes.TREs has represented the time error distribution simultaneously.

Among Fig. 9 (a), SRSVD and TomoG method indicate that the 10th and the 40th OD stream have very high fluctuation.Because preceding half segment data of this OD stream is very little, the energy that in whole network traffics, accounts for is especially little.So be difficult to estimate exactly these OD stream.And then obtain FSR, and SRSVD, the mean space correlation branch error of TomoG is not 0.47,0.77,0.68.But the FSR method of the embodiment of the invention has minimum space correlation error preceding half section of OD stream.This explanation has very stable and effective evaluation characteristic to the flow FSR of small scale.In Fig. 9 (b), all methods all have sudden change on time-domain.But FSR has very low sudden change in the little time interval.Simultaneously, FSR, SRSVD, correlation error average time of TomoG is respectively 0.20,0.29,0.26, so FSR has minimum time correlation error.

Remove to obtain the traffic matrix evaluation characteristic through the space relative error of three kinds of methods and the cumulative distribution function of time relative error in the embodiment of the invention.Figure 10 (a) shows FSR, SRSVD, and three kinds of methods of TomoG, the space relative error that about 92%, 73%, 84% OD stream obtains is 0.83.And, the embodiment of the invention to about 80% estimation constantly, Figure 10 (b) shows that the time relative error that obtains is 0.25,0.31,0.30.Therefore this measurement result has been represented the identical estimated result of three kinds of methods, and wherein the evaluated error of the FSR method of embodiment of the invention employing is minimum.

In sum, explain that the traffic matrix that obtains through the reconstruct of FSR method is more accurate.

Claims (4)

1. Under the dynamic network based on the end to end network flow reconstructing method of compressed sensing, it is characterized in that: may further comprise the steps:

   Step 1, the system that is provided with flow length, the random walk number of times of random walk to OD;

   Step 2, in the large-scale ip backbone network, adopt the mode of random walk to select OD stream, adopt the method that makes up boolean's sparseness measuring matrix to describe the opening and closing state of whole routers according to selected OD stream;

   The flow value that step 3, the part of O D that adopts the method for constructing the rarefied flow moment matrix to describe the collection of router subclass flow; Described router subclass is the router of whole openings, and calculates measured value according to boolean's sparseness measuring matrix and the rarefied flow moment matrix of being constructed;

   Step 4, according to the sparseness measuring matrix of being constructed in the step 3 with calculate the gained measured value; Adopt the principal component analysis method to make up the compressed sensing reconstruction model; With this model the relation of whole end-to-end OD streams in OD stream that the router subclass produces and the large-scale ip backbone network is described, and then whole OD stream flow end to end in definite entire I P backbone network.
2. 2. based on the end to end network flow reconstructing method of compressed sensing, it is characterized in that under the dynamic network according to claim 1: the structure boolean sparseness measuring matrix described in the step 2 may further comprise the steps:

   Step 2-1, an OD stream of structure complete graph comprise summit and limit in the complete graph, the number on summit is equal to the number of OD stream;

   Step 2-2: random walk is set measures the number of times initial value;

   Step 2-3: select a summit as unified starting point;

   Step 2-4: complete graph is carried out equally distributed random walk, and according to the point that random walk is found, the router state that this point is corresponding then is an opening, otherwise is closed condition;

   Step 2-5: less than initialization random walk measurement number of times is set if number of times is measured in random walk, then returns step 2-3; If random walk measurement number of times is greater than or equal to initialization random walk measurement number of times, then execution in step 2-6 is set;

   Step 2-6: preserve boolean's sparseness measuring matrix.
3. 3. based on the end to end network flow reconstructing method of compressed sensing, it is characterized in that under the dynamic network according to claim 1: the boolean's sparseness measuring matrix described in the step 1, this matrix meet constraint equidistance criterion; Line number in boolean's sparseness measuring matrix is represented the number of times of random walk large-scale ip backbone network, and the columns in boolean's sparseness measuring matrix is represented the sum of OD stream in the large-scale ip backbone network.
4. 4. based on the end to end network flow reconstructing method of compressed sensing, it is characterized in that: following formula is satisfied in the calculating of the measured value described in the step 3 under the dynamic network according to claim 1:

   Measured value=boolean's sparseness measuring matrix * rarefied flow moment matrix.

Images