WO2001091375A2

WO2001091375A2 - Method and system for measuring one-way delay variation

Info

Publication number: WO2001091375A2
Application number: PCT/US2001/015741
Authority: WO
Inventors: Michael Brown
Original assignee: Williams Communications Llc
Priority date: 2000-05-19
Filing date: 2001-05-15
Publication date: 2001-11-29
Also published as: WO2001091375A3; MXPA02011438A; EP1282956A2; AU2001263161A1; JP2003534715A; CA2409001A1

Abstract

A method and a system for measuring one-way delay variation in a network transmits probe packets with a transmission time indication and a sequence number. The network receives a first probe packet and a second probe packet and notes their arrival times. If the two probe packets are sequential, the one-way delay variation is calculated between the two probe packets. The delay variation is equal to the difference in arrival time between the first and second probe packets minus the difference in the transmission time between the first and second probe packets.

Description

METHOD AND SYSTEM FOR MEASURING ONE-WAY DELAY

VARIATION

INVENTORS Michael Brown

BACKGROUND

Field of Invention

The present invention relates generally to network performance measurement, and more particularly, to measuring the one-way delay variation in a network using a non-uniform data transmission rate.

Background of the Invention

In certain situations, it is desirable to send data over a network with a high priority and with a guaranteed maximum transit time. For example, certain data may be needed in real time or may be of high importance. Currently, certain conventional network protocols (such as the Asynchronous Transfer Mode (ATM) protocol and the Frame Relay protocol) contain provisions for indicting a "level of service" that particular transmitted data is to receive. Users pay premiums to obtain higher levels of service in an ATM network. It would be desirable to send data over the Internet with the same type of guarantees.

Many business customers have declined to migrate their strategic network systems from Frame Relay, ATM and private lines to the much more cost-effective public IP Internet because the Internet cannot provide the performance and service guarantees they require. This lack of willingness to send data via the Internet has slowed the acceptance of Internet Virtual Private Networks (VPNs). If this lack of quality and confidence is left unchecked, it will slow Internet market segment growth into the B-2-B commerce market, which is estimated to exceed $7.3 trillion in B2B e-commerce transactions in the coming years.

The potential exists even in today's network to completely revolutionize B2B communications using the Public IP Internet if certain performance requirements are met. Business customers need assurances from Internet carriers in the form of end-to-end Service Level Agreements (SLAs) that have teeth and that cover multiple carriers. Guarantees for availability, packet loss, delay, and throughput must be met. When these requirements are met, a majority of the traffic currently in private B2B networks can migrate to the public IP Internet. Once this occurs, Internet carriers will generate more incremental revenue and business customers will be able to consolidate communications and reduce cost.

An important feature in an Internet network capable of providing quality of service guarantees to customers is the ability to prove that the promised quality of service is being met, by measuring actual network performance. One measure of network performance is the latency, or delay, measurement, which measures how much time it takes for a packet to get from one point to another on a network. A related measure of performance is the delay variation, which measures the rate of change in the latency measurement. Jitter is the amount that the transmission rate actually varies from the mean during a current time period. Jitter is a critical parameter that affects the quality of network transmissions that are highly delay- sensitive, such as audio and video transmissions.

Network delay can be measured either for a packet round-trip, or in a single direction. The round-trip delay time may be measured using both sending and receiving equipment in the same location, and is therefore convenient to measure. One existing method involves measuring the round-trip delay time by using a network "ping" feature. Ping is a common network debugging tool that places a timestamp in each packet, which is echoed back and can be used to compute the time required for each round trip packet exchange. However, the round trip delay variation time may not accurately reflect individual one-way delay variation. Frequently, the sending and receiving paths may not be symmetrical, and therefore differences will exist between the sending and receiving direction delay variations. More precise measurements of individual paths will be achieved by measuring one-way delay variation.

Existing methods of measuring one-way delay variation have disadvantages. One existing method for measuring one-way delay variation relies on an absolute clock to synchronize the time between two different monitoring points on the network. However, an absolute clock, such as a Global Positioning System (GPS) time receiver, is necessary in order to accurately correlate a "sent" time with a "received" time. It would be preferable to be able to measure one-way delay variation without requiring an absolute clock. The ability to measure the one-way delay without an absolute clock will allow the one-way delay variation measurement to be much more widely deployed than at present. An additional existing method of measuring delay variation relies on sending packets across a network at a known, pre-determined rate. Given this known sending rate, variations from the known rate can be measured. However, a more flexible method for measuring oneway delay variation is desirable, which would allow the granularity of measurements to change as network performance changes.

SUMMARY OF THE INVENTION

The present invention allows the one-way delay variation across a network to be measured in a flexible manner. No absolute clock is required to perform the measurements. Additionally, packets used for measurement are sent at a non-uniform rate, wherein the rate used is adjusted based upon operational parameters. For example, the measurement frequency rate may be increased when the one-way delay variation is changing most rapidly. This allows for increased measurement granularity when network conditions appear to be fluctuating. Additionally, the non-uniform sending rate allows a system administrator to vary the bandwidth put on the system by one-way delay variation measurements. This ability to vary bandwidth is desirable, for example, if existing network bandwidth is currently overloaded, because the system administrator can decrease the delay variation measuring rate to avoid further bandwidth stress.

A transmitter on a network transmits probe packets with a transmission time indication

^• and a sequence number. A receiver on the network receives a first probe packet and a second probe packet. The one-way delay variation between the two probe packets is calculated. The delay variation is equal to the difference in arrival time between the first and second probe packets minus the difference in the transmission time indication between the first and second probe packets.

Advantages of the invention will be set forth in part in the description which follows and in part will be apparent from the description or may be learned by practice of the invention. The objects and advantages of the invention will be realized and attained by means

_* of the elements and combinations particularly pointed out in the appended claims and equivalents.

BRIEF DESCRIPTION OF THE DRAWINGS

Fig. 1 A is a diagram of a network system illustrating the use of a transmitter and a receiver to measure network delay parameters in an embodiment of the present invention. Fig. IB is a diagram of a probe packet used in measuring one-way delay variation in an embodiment of the present invention.

Fig. 2 is a flowchart of a method for sending packets for measuring one-way delay variation in an embodiment of the present invention.

Fig. 3 is a flowchart of a method for receiving packets and calculating the one-way delay variation in a network in an embodiment of the present invention.

The figures depict a preferred embodiment of the present invention for purposes of illustration only. One skilled in the art will readily recognize from the following discussion that alternative embodiments of the structures and methods illustrated herein may be employed without departing from the principles of the invention described herein.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Reference will now be made in detail to several embodiments of the present invention, examples of which are illustrated in the accompanying drawings. Wherever practicable, the same reference numbers will be used throughout the drawings to refer to the same or like parts.

A network system includes a monitoring system to measure the one-way delay variation between different points in the network. The one-way delay variation is the change in the network delay, or latency, over time. Network latency is the time it takes for a signal to travel between different points on the network. One embodiment of a signal is an Internet Protocol (IP) packet. However, it will be evident to one of skill in the art that the present invention may be used with many other types of network signals.

The one-way delay variation of a network path is calculated based on the sending and arrival times of packets traveling the network path. The arrival time A of a packet is equal to the sending, or transmission time Eplus the network delay D:

A = T + D (1)

However, if the arrival time A and transmission time T are not measured with synchronized clocks, the absolute value of the network delay D cannot be measured accurately as the difference between A measured at the arrival clock and T measured at the transmission clock. However, the delay variation in the network may be accurately calculated by comparing differences in arrival and transmission times between two separate packets sent on the network.

The network delay D is equal to a fixed network delay Df_ιx plus the network delay variation D_var that occurred during transmission of the particular packet at issue:

D = D_fa +D_var (2)

The fixed delay D_βx is assumed to be constant for all packets traveling on the same network path or route. This fixed delay is composed of the propagation delays and the minimum processing time of each piece of switching and transmission equipment. The delay variation D_var is the portion of the delay D that varies during the transmission of different packets. This variable delay is created by a variety of factors, including queuing delays and variations in the processing time of each piece of switching and transmission equipment.

From equation 1 above, the arrival time difference between two separate packets (i) and (t+1) will equal the transmission time difference between the two packets plus the delay time difference between the two packets:

A{i) - A(i + 1) = [T(ι) - T(i + 1)] + [D(i) - D(i + 1)] (3)

which may be expressed as :

ΔA = ΔT + ΔD (4)

Based upon equation 2, the difference in the network delay between packets (i) and (i+1) may be expressed as:

ΔD=[i⁾„(/)-_JD_[ϊ(t+l)]+[E»_var( -i⁾ _var(t+l)] (5)

Since the average delay D_fιx is a constant, D_fα(i)-D_^ (i-W) =0, fix and therefore:

AD=D_{v r}(i)-D_vaι(i+ϊ) (6) Therefore the one-way delay variation for a network path is calculated as:

Δ _var = A4 - ΔE (7)

The method of equation 7 is implemented in a network monitoring system to measure delay variation in one or more routes of a network (or networks). The network monitoring system includes a transmitter and a receiver capable of sending packets between themselves. In one embodiment, there is one transmitter for each receiver. In another embodiment, a single transmitter is adapted to transmit signals to a plurality of receivers, using, for example, a multicast IP packet type. In another embodiment, a first number of transmitters is smaller than a second number of receivers. In another embodiment, a first number of transmitters is greater than a second number of receivers.

Fig. 1 A is an illustration of an embodiment of a network system for measuring oneway delay variation. A network 101 includes a set of routers 120A-G and a transmitter 100. A network 102 includes a set of routers 130A-D and a receiver 110. Networks 101 and 102 are interconnected at a set of connections 140A, 140B and 140C. Transmitter 100 sends probe packets to receiver 110 via a route 180, which includes routers 120E, 130C and 130D. The probe packets are used to measure the one-way delay variation on route 180 between transmitter 100 and receiver 110.

Fig. 1A additionally illustrates why measuring the round-trip delay and delay variation is not equivalent to doubling the one-way delay variation. In the example shown, a return route 190 is used to route packets from receiver 110 to transmitter 100. In this case, a round-trip delay measurement would combine measurements of the delay on route 180 and route 190. Thus the round-trip delay variation is not specific to route 180.

Fig. IB illustrates an embodiment of a probe packet suitable for use in measuring the one-way delay variation on a network. A probe packet 10 includes a sequence number 12 and a transmission time 14. Successive probe packets are given successive sequence numbers. The transmission time 14 is implemented as a time stamp in one embodiment. It will be evident to one of skill in the art that different types of probe packets may be used for measuring the one-way delay variation in a network. For example, in one embodiment, the information included in the probe packet may be appended to normal data-carrying packets traveling between the transmitter and receiver locations.

In one embodiment, the transmitter and the receiver are both computers connected to the network and adapted to perform the transmitter or receiver functions. In another embodiment, the functionality of the transmitter is embedded in an application specific integrated circuit (ASIC), and the functionality of the receiver is embedded in a separate ASIC. In yet another embodiment, the transmitter and receiver are software modules adapted to perform the transmitter or receiver functions.

Fig. 2 is a flowchart of the functions performed by the transmitter. The transmitter initializes 210 a sequence of probe packets by setting the sequence number equal to /. The transmitter then sends 220 a probe packet with a transmission time indication and a sequence number. In one embodiment, the transmission time indication is a timestamp, which indicates the current time at the transmitter. The transmitter waits 230 for a period of time INTERVAL, and then increments 240 the sequence number by one and returns to step 220 to send another probe packet.

The sequence numbers do not have to be increased by an integer amount. It will be evident to one of skill in the art that the sequence numbers may be set to increase in any logical sequence, as long as the receiver is able to distinguish between sequential versus nonsequential probe packets.

The INTERVAL time period may be set by a system user and may be changed as desired during the probe packet transmitting process. For example, in one embodiment a typical INTERVAL time is five seconds. However, if the system user notes that the one-way delay variation measurements appear to be changing rapidly, the INTERVAL time may be decreased to increase the granularity of measurement for the one-way delay variation. Additionally, if the system user notes that network traffic appears to be approaching network bandwidth limitations, the system user may increase the INTERVAL time in order to decrease the amount of bandwidth added to the network by the probe packets. In one embodiment, the transmitter automatically varies the INTERVAL time parameter in response to changes in the one-way delay variation measurements or changes in network bandwidth.

Fig. 3 is a flowchart of the functions performed by a receiver. The receiver waits 310 to receive a probe packet 305. Upon receipt of probe packet 305, the receiver stores 320 the time probe packet 305 arrived T(t), and stores the sequence number of probe packet 305 as the last sequence number SEQ(t).

The receiver then waits 330 to receive a next probe packet 325 with a sequence number SEQ(t+l). Upon receipt of next probe packet 325, the receiver checks 340 to determine if probe packets 305 and 325 are sequential. Assuming monotomcally increasing sequence numbers:

Is SEQ(M) = SEQ(z) + 1 ?

If probe packets 305 and 325 are not sequential, the receiver returns to step 320 and stores the sequence number and arrival time for probe packet 325 as SEQ(z^') and T(t), respectively. In one embodiment, the receiver will not calculate the one-way delay variation on probe packets that are not sequential. Non-sequential packets indicate that probe packets have been lost, and the system will note this problem. It will be evident to one of skill in the art that if the possible loss of probe packets is not a concern, the timestamping procedure allows the one-way delay variation measurement to proceed using non-sequential packets.

If probe packets 305 and 325 are sequential, the current one-way delay variation is calculated 350 as the difference between the two packets' arrival times minus the difference between the two packets' timestamp times (representing the two packets' transmission times). The one-way delay variation may be positive or negative, indicating that the network one-way delay for the measured path is either increasing or decreasing, respectively. The receiver then returns to waiting 310 for the next probe packet.

The receiver stores successive one-way delay variation measurements. Periodically, the receiver transfers the stored measurements back to a central data collection system. The data collection system compiles statistics from multiple receivers, and prepares a statistical summary of one-way delay variation times throughout the network. The statistical summary provides msight into the workings of the network. For example, areas of the network experiencing wide fluctuations in delay variation are identified. The effect of available network bandwidth increases and decreases will also be identified by variations in the one-way delay variation measurement in addition to queuing changes on the transmission and switching equipment. The accuracy of the one-way delay variation measurements is dependent on the INTERVAL time and the stability of the transmitter and receiver internal clocks. Both the transmitter and the receiver contain internal clocks for noting the time at which a signal is sent and received, respectively. It is not necessary for the transmitter and the receiver to have synchronized absolute clocks. The transmitter and receiver internal clocks need only possess reasonable internal stability, which is achieved by the types of clocks found on existing computers. For example, a Pentium personal computer has an internal clock that is stable to at least 10 seconds/day, or approximately 1 part in 10,000. Assuming that the clock drift is uniform, an INTERVAL time between sending probe packets of 10 seconds allows the one- way delay variation to be measured to within one millisecond. If measurements with a longer INTERVAL time or a greater precision are desired, the internal clock stability of the transmitter and receiver is increased.

Although the invention has been described in considerable detail with reference to certain embodiments, other embodiments are possible. As will be understood by those of skill in the art, the invention may be embodied in other specific forms without departing from the essential characteristics thereof. For example, the transmitter and the receiver may be implemented as hardware or software modules. Additionally, different numbers of transmitters and receivers may be combined to perform multiple simultaneous one-way delay variation measurements on different network routes. Accordingly, the present invention is intended to embrace all such alternatives, modifications and variations as fall within the spirit and scope of the appended claims and equivalents.

Claims

We claim:

1. A method for measuring one-way delay variation in a network, comprising: receiving a first signal including a first transmission time indication at a first time; receiving a second signal including a second transmission time indication at a second time; and calculating a delay variation as the difference between the first time and the second time, minus the difference between the first transmission time indication and the second transmission time indication.

2. The method of claim 1, wherein the signal is an Internet Protocol packet.

3. The method of claim 1 , wherein the signal is a multicast Internet Protocol packet.

4. The method of claim 1, wherein each transmission time indication is a timestamp that indicates a time at which each associated signal was sent.

5. The method of claim 1, wherein the first and second signals include, respectively, a first and second sequence number.

6. A method for measuring one-way delay variation in a network, comprising: sending a sequential series of packets, wherein a transmission time is indicated on each packet; receiving the sequential series of packets, wherein a receiving time is associated with each packet; and calculating a delay variation as the difference in receiving times between two sequential packets minus the difference in transmission times between two sequential packets.

7. A system for measuring one-way delay variation in a network, comprising: a transmitter adapted to send a first packet with a first timestamp, and further adapted to send a second packet with a second timestamp; and a receiver adapted to receive the first packet at a first time and to receive the second packet at a second time, wherein the receiver calculates a delay variation time as the difference between the first and second times minus the difference between the first and second timestamps.

8. The system of claim 7, wherein the transmitter is further adapted to append a first sequence number to the first packet and append a second sequence number to the second packet.

9. A computer program product for measuring the one-way delay variation in a network, the computer program product comprising: a computer readable medium that stores program code including: program code that receives a first signal with a first transmission time indication and a first sequence number at a first time; program code that receives a second signal with a second transmission time indication and a second sequence number at a second time; and program code that calculates a delay variation as the difference between the first time and the second time, minus the difference between the first transmission time indication and the second transmission time indication.