WO2003013109A1

WO2003013109A1 - Improved crosstalk identification for spectrum management in broadband telecommunications systems

Info

Publication number: WO2003013109A1
Application number: PCT/US2002/024187
Authority: WO
Inventors: Robert Hausman; Stefano Galli; Kenneth Kerpez
Original assignee: Telcordia Technologies, Inc.
Priority date: 2001-07-31
Filing date: 2002-07-31
Publication date: 2003-02-13
Also published as: EP1413121A1; JP2004537926A; CA2454239A1; JP4498737B2; EP1413121A4

Abstract

Crosstalk between subscriber loops used to transmit different broadband services through the same bundled telephone cable is a significant limitation to providing digital subscriber line services. A method for estimating the crosstalk, identifying the sources of the crosstalk and predicting additional sources of crosstalk are disclosed. The crosstalk sources are identified in the frequency domain by maximizing the correlation (530) with a 'basis set' of received crosstalk PSDs which consist of the cascade of a finite set of known transmit PSDs types times a representative set of crosstalk couplings. Multiple crosstalk types are identified with a technique of successive spectral subtraction. Once a type is identified the crosstalk disturber is compared against all other members of the complete set for that type. Additionally, Multiple Regression (MR) techniques and a Matching Pursuit (MP) alogrithm 535) are used to increase the ability of the system and method to identify various crosstalk disturbers.

Description

IMPROVED CROSSTALK IDENTIFICATION FOR SPECTRUM

MANAGEMENT IN BROADBAND TELECOMMUNICATIONS

SYSTEMS

RELATED APPLICATIONS

The present application claims the benefit of U.S. Provisional Application No.

60/309,075 filed 7/31/01 and entitled "Method Of Identifying The Quantity And Type Of

Crosstalkers Disturbing A Subscriber Loop" and is a continuation-in-part of U.S. Patent

Application No. 09/922,064 filed August 3, 2001 and entitled "Crosstalk Identification For Spectrum Management In Broadband Telecommunication Systems."

FIELD OF THE INVENTION

This invention is related to the identification of crosstalk between subscriber loops transmitting high bandwidth telecommunication services through bundled telephone lines or cables, particularly to identification of crosstalk in digital subscriber line ("DSL") service loops. Identification of the crosstalk enables minimization of its effect through crosstalk cancellation, and as part of the spectrum management of the local plant in a telecommunications system. Identification of the crosstalk disturbers is accomplished through a variety of methods including Multiple Regression (MR) analysis and an application of a Best Basis Set (BBS) problem with a sparse solution including use of a Matching Pursuit (MP) algorithm.

BACKGROUND

The mainstay of the telephone company local network is the local subscriber loop. The local subscriber loop is now being used to provide broadband digital telecommunication services such as DSL service. Such broadband DSL services include integrated services digital subscriber network ("ISDN"), high-rate digital subscriber line

("HDSL"), asymmetrical digital subscriber lines ("ADSL") and very high rate digital l subscriber lines ("VDSL") technology. DSL services allow residential and business customers to send and/or receive digital data at higher rates of speed than were previously possible using analog modem technology.

DSL technologies are engineered to operate over a class of subscriber loops, such as nonloaded loops (18 kft) or Carrier Serving Area (CSA) loops (9 to 12 kft). Digital

Subscriber Line (DSL) technology exploits the existing, ubiquitous, copper telephone loop plant to provide megabit per second (Mbps) high-speed Internet access and other services.

The great majority of residential customers and many business customers are served by such metallic (copper) twisted pair cables connected from a local switch in the central office ("CO") to the subscriber's land-line telephones. For each subscriber, telephone and DSL signals travel on a single twisted pair from a central office (CO) to the subscriber. Many, sometimes thousands, of twisted pairs are wrapped together in a single cable. The twisting of the pairs keeps the average amount of electromagnetic coupling between the balanced circuits on each pair to a low level, resulting in low crosstalk coupling between different circuits at voice frequencies. Twisted-pair cabling was designed and built to carry voice services, requiring only a very low probability of intelligible crosstalk at voice frequencies, generally up to a few kHz.

Crosstalk generally increases, however, with increasing frequency, and because DSL frequencies extend into the MegaHertz ("MHz") range, crosstalk becomes the major limitation to high-speed DSL. Because a number of individual twisted pairs are wrapped together in a binder, and a number of binders are in each telephone cable (there are typically 12, 13, 25, 50 or 100 pairs in a binder) the crosstalk between pairs in the same binder is much higher than it is between pairs in different binders. Telephone cabling can be thought of as a multi-input, multi-output interference channel with crosstalk in-between from 12 to 100 channels, although there are typically a smaller number of high-power crosstalkers in each pair. DSL technology is still fairly new. The general approach to date has been to treat each signal on each twisted pair entirely separately, and to assume that crosstalk is simply additive Gaussian noise of which nothing is known. With this approach, subscribers can enjoy relatively high-speed service, far faster than voice-band modems, up to multiple Megabits per second ("Mbps"). This is the current static approach to DSL spectrum management.

As time progresses it is expected that there will be many more DSL users each demanding higher speed service. This will result in more crosstalk and higher-bandwidth services that are more vulnerable to crosstalk. Without accurate systems and methods for identifying crosstalk, canceling such crosstalk and thereby carefully managing the available frequency spectrum in local loop plants there may be no way to avoid problems that will ultimately result in customer disappointment, delay and higher costs.

The American National Standards Institute (ANSI) Working Group T1 E1.4, being responsible for DSL standards on ISDN Basic Access, HDSL and ADSL, initiated a project in 1998 to develop relevant standards for spectrum management. ANSI Standard T1.417- 2001 , Spectrum Management for Loop Transmission Systems, was approved and issued in January 2001. This ANSI standard applies relatively rigid rules uniformly across the entire loop plant. There have been a number of proposals for new DSL systems, for example the class of symmetric DSL ("SDSL"). Unfortunately, this creativity complicates the problem of spectrum management, as is apparent from Table 1 , which shows the nine currently agreed classes of DSL systems.

Table 1. Spectrum Management Classes

FIG. 8 shows currently adopted power spectral density ("PSD") templates for each of these spectrum management classes except Class 6 with line 110 representing Class 1 and line 180 representing Class 9 with the others represented respectively in between. To comply with the ANSI standard, the PSD of the signal transmitted by a piece of equipment must fall under the template at all frequencies. This will generally be verified during manufacture, but if problems arise in the field the local operator will need to resolve incompatibilities and complaints, possibly including identification of systems that are transmitting in violation of the standard. Current rules for DSL spectrum management assume static worst case crosstalk types and crosstalk couplings. These rules do not take into account the individual types of crosstalk sources or crosstalk couplings related to a particular pair in a cable, which may be considerably different than the near worst-case couplings that are assumed in the ANSI standard. Dynamic spectrum management would take into account the individual types of crosstalk sources and crosstalk couplings of each particular cable, and could greatly increase the number of customers that can be provided DSL service and their bit rates. Thus, a system that can characterize crosstalk on a loop-by-loop basis has the potential to yield a better crosstalk characterization of the plant. Furthermore, the use of such crosstalk data in a loop spectrum management database, in turn has the potential to be mined, correlated and exploited to provide more optimal performances for individual subscriber loops.

It would be desirable to have a technique that could characterize the crosstalk environment on a loop-by-loop basis, in a mechanized and highly accurate manner without the need for special equipment or intervention at the subscriber's location.

Further it would be desirable to use the automated system for the identification of crosstalk for the spectrum management of DSL systems.

Therefore, it is desirable to have a means for identifying crosstalk between local loops that will enable a system operator to identify actual or potential crosstalk disturbers depending on the configuration of the DSL services being provided to a plurality of customers.

SUMMARY

In accordance with the present invention a method and system for the identification of crosstalk in high-bandwidth DSL systems are used to ensure spectral compatibility between a number of DSL lines/subscribers in a loop plant providing a means for spectrum management in a local loop plant.

A novel method for crosstalk identification in the frequency domain computes the correlation coefficient between the measured spectrum and a set of pre-defined PSDs that may be viewed as a crosstalk basis set. The size of the basis set may be managed through the use of a singular value decomposition of the complete basis set in accordance with the type of crosstalk disturber.

In the method of the present invention the measured power spectral density of a subscriber loop is correlated with a predetermined set of power spectral densities for a group of possible crosstalk disturbers and the crosstalk disturber having the most closely correlated power spectral density is selected as the identified disturber. Once identified, the power spectral density for the selected crosstalk disturber is subtracted from the measured power spectral density of said subscriber loop using spectral subtraction and the resulting power spectral density is again correlated with the predetermined set of power spectral densities for the group of possible crosstalk disturbers to identify additional disturbers.

The method may also employ Multiple Regression (MR) techniques or the Matching Pursuit (MP) algorithm for finding an optimal sparse representation of a vector from an overcomplete set of vectors. These techniques can be employed to increase the accuracy of the crosstalk disturber identification method using correlation or can be used in place of the correlation method.

The method may be applied to a DMT receiver capable of receiving a signal, η(k)

and capable of generating a set of decisions

, from decision variables Q,{n), and

channel estimates H(n) . A data sheer extracts the set of decisions D_l (ή) , from decision

variables Q,{n). A crosstalk PSD estimator estimates the power spectral density of the

crosstalk disturber based on the set of decisions D_τ (ή) , the decision variables Q,(n) and

the channel estimates H(ή) and a crosstalk identifier identifies the type of the crosstalk

disturber by correlating the estimated power spectral density of the crosstalk disturber with a set of possible disturbers and selecting the disturber exhibiting the maximum correlation. Furthermore, a crosstalk predictor may be added to predict the p-th order linear MMSE prediction of the crosstalk disturber. The prediction may be used to cancel the effect of the crosstalk disturber in the DMT receiver. The identity of crosstalk disturbers can also be sent to an Operations Support

System (OSS) for the management of the broadband spectrum available for a plurality of subscriber loops. The information could also be used to cancel the effect of the crosstalk disturber in a DSL receiver.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 depicts a DSL subscriber loop system having two DSL subscriber loops connecting to a central office;

FIG. 2 depicts an illustrative embodiment of a crosstalk identification system according to the present invention; FIG. 3 depicts a cable binder having N twisted pairs of copper wire;

FIG. 4 depicts the power sum of NEXT loss in dB for a binder having 25 twisted pairs of copper wire;

FIG. 5 depicts examples of measured pair-to-pair NEXT loss;

FIG. 6 depicts an example of a fourteen member canonical set of pair-to-pair NEXT couplings;

FIG. 7 depicts an example of a fourteen member basis set of T1 NEXT couplings;

FIG. 8 depicts the power spectral density ("PSD") templates for each of the spectrum management classes;

FIG. 9 depicts the NEXT PSD for a first unknown disturber; FIG. 10 depicts the NEXT PSD for a second unknown disturber;

FIG. 11 is a flow diagram of the crosstalk identification and subtraction algorithms of the present invention.

FIG. 12 is a diagram depicting measured mixed crosstalk containing a mixture of crosstalk from SDSL @1040 kbps and HDSL. FIG. 13 is a flow diagram of the crosstalk identification algorithm using Multiple

Regression (MR) and the BBS sparse solution technique with both a reduced and original crosstalk basis set.

FIG. 14 is a flow diagram of the crosstalk identification algorithm incorporating the use of Multiple Regression (MR) or the BBS sparse solution techniques such as Matching Pursuit.

DETAILED DESCRIPTION FIG. 1 depicts crosstalk in an example of a local loop environment. DSL loops 18 comprising twisted pair copper wire connect two subscribers 20 to a central office ("CO") 12 or remote terminal ("RT"). Near-end crosstalk ("NEXT") 30 between the two loops may be present at the end of loops where the disturber's transmitter is located (either at the CO or the RT) while far-end crosstalk ("FEXT") 40 may be present at the end of the loop 18 opposite to where the disturber's transmitter is located.

Twisted-pair wire 18 can be considered a multi-user channel by expressing the received signal vector as

Y(f) = R(f) + N(f) + H(f)D(f) (1 ) where R(f) is the received message vector, N(f) is the vector of independent background noise, D(f) is the vector of transmitted signals creating crosstalk on nearby pairs, and H(f) is the matrix of crosstalk couplings. In the case of near-end crosstalk (NEXT), R(f) is transmitted from one end of the cable and D(f) is transmitted from the other end. The received crosstalk H(f)D(f) can be canceled, minimized or otherwise treated so as to be less interfering than white noise.

The twisted-pair multi-user channel is generally time invariant. Crosstalk couplings may vary as the temperature of the cable changes, but only very slightly. The crosstalk sources may turn on and off, but not very often because the main application of broadband service is "always on" Internet access. Time invariance allows the possibility of very accurate estimates of crosstalk coupling using large sample sizes. Joint statistics can be obtained by a "third-party" operating a third party system, which is connected to all lines through communications links to the modems, and the knowledge of all the transmitted data could be used to estimate the crosstalk couplings. A "third-party" or operations center having access to multi-user crosstalk statistics would be far more capable of diagnosing crosstalk problems than an uninformed technician.

Received crosstalk can be measured by temporarily connecting measurement equipment to a single pair while crosstalk couplings could be measured by connecting measurement equipment to multiple pairs. The accuracy of estimates depends strongly on which elements of the multi-pair cable may be accessed, for example crosstalk couplings can be estimated accurately by accessing data from both transmitters and receivers. Crosstalk parameter estimation can be performed off-line using very many samples of data. Knowledge of crosstalk parameters can be useful for spectrum management and maintenance purposes and can also be used at run time to enhance modem performance. There are many vagaries in the telephone outside plant, some older cables have poor crosstalk performance. Unanticipated problems from crosstalk are likely to occur and may be extremely difficult to diagnose. The method of the present invention can be performed in any single receiver. The first step is identification of the type of DSL line that is generating the crosstalk. To accomplish this a receiver estimates which varieties of DSL are generating crosstalk. This can be accomplished because there is actually a small number of common crosstalker types at high frequencies: T1 lines, integrated services digital subscriber lines (ISDN), high bit-rate digital subscriber lines (HDSL), single-pair high bit-rate digital subscriber lines (HDSL2), asymmetric digital subscriber lines (ADSL), and symmetric digital subscriber lines (SDSL). Single-pair high-speed digital subscriber lines (G.shdsl) and very-high rate digital subscriber lines (VDSL) are also likely to become possible sources of crosstalk soon. . Each of these crosstalker types has a unique crosstalk power spectral density (PSD), and this PSD is estimated with classic matched-filter correlation techniques. The basic implementation of the method is described by steps 310, 320 and 330 of FIG. 1 1 , which also shows the additional steps in the iterative process described below.

Referring to FIG. 2, a crosstalk identification system according to the present invention provides for a computer 200 having a processor 210 in communication with a memory 220 (which can comprise either primary memory, such as RAM, or secondary memory, such as a magnetic disk drive or other storage media or a combination thereof) and input/output (I/O) unit 230. I/O unit 230 is adapted to receive data indicative of the power spectral density of one or more subscriber loops 18. Display 260 may be used to display graphical and/or textual information related to the PSD measurements and crosstalk identification. Other input/output devices (not shown) may be used in conjunction with the computer 200 such as a keyboard, mouse, touchpad, trackball etc. in order to provide a user interface.

Processor 210 executes the method steps described herein as stored in memory 220 in order to identify the crosstalk on one or more subscriber lines 18. Additional software may also be executed to generate the PSD from the measurements input through I/O unit 230. Alternatively, a separate test measurement device capable of measuring the PSD for the various subscriber loops may be attached to I/O unit 230.

The identification algorithm is extended to the case of multiple crosstalkers by a process of successive estimation and cancellation using spectral subtraction described in greater detail below.

Knowledge of the type of crosstalkers could be used for crosstalk cancellation in a DSL receiver. Once the type of the crosstalk source is determined, then the correlations of the source of the crosstalk are known, assuming correct decisions. These correlations are used to construct an optimal set of weightings for a type of predictive filtering that minimizes the received crosstalk power before detection in the receiver. The coefficients of the predictive filter are predetermined based on the known properties of each of the types of crosstalk.

Sources of crosstalk are often called "disturbers." There are two types of crosstalk: near-end crosstalk (NEXT) and far-end crosstalk (FEXT). NEXT is more powerful than FEXT, particularly below about 1 MHz where many DSLs use overlapping spectra. If there is one crosstalker, then the received crosstalk PSD is the product of a PSD transmitted on a nearby pair times the crosstalk coupling transfer function. With multiple crosstalkers the received crosstalk PSD is the linear sum of each component. FIG. 3 depicts a cable binder 100 having N twisted pairs of copper wire 18. Lines 40 of FIG. 3 represent the NEXT between the first two twisted pairs of copper wire 18 while lines 41 represent the NEXT between the first and last pairs of copper wire 18.

Crosstalk can be characterized in terms of power-sums. For a given pair in a binder group, the power-sum NEXT is formed from the sum of the pair-to-pair NEXT coupling powers of the other pairs in the binder group into this given pair. For a typical 25- pair binder group, the 25 power sums are shown in FIG. 4. The power-sum is usually displayed as a power-sum "loss" and the lower the loss the higher the NEXT coupling. The NEXT power-sum loss is approximately linear with frequency on the log-log scale. The NEXT model often used for studies in the industry is stated as expected 1 % worst case power-sum crosstalk loss as a function of frequency. This means that on average, 1 % of the pairs tested are subjected to power-sum crosstalk loss less than that assumed by the model at the given frequency. Such a model is a smooth curve versus frequency, in which the loss decreases at about 15 dB per decade of frequency. This statistical model ensures that the 1 % worst case NEXT power sum level follows the curve over frequency but this noise level may appear on different pairs at different frequencies. The 24-disturber NEXT power level for one specific pair is not a smooth curve if plotted over frequency and the deviation from the statistical model increases as the number of interferers used in the power sum is reduced. The power-sum loss for less than a full binder group depends on the distribution of the pairs on which the crosstalking signal appears. For a 25-pair binder group there are 300 distinct pair-to-pair couplings (assuming symmetry). Examples of six measured pair-to-pair NEXT couplings from a binder group of twisted pair wires 18 are depicted in FIG. 5. It should be noted that there is wide variation of coupling strength for each pair-to-pair combination with frequency. The power summing process smoothes and averages some of this variation. The received signal on pair number k can be expressed as

Y_k (/) = R_k (/) + N_κ (/) + ∑ "_{= ≠k} H_Λ (f)D, (/) (2) where R_k(f) is the received message signal, N_k(f) is independent background noise, D_t(f) is the signal transmitted on pair /, H_ik(f) is the crosstalk coupling between pair /and pair k, and the crosstalk received on pair k from a system transmitting on pair / is H_ik (f)D_t (/) .

Crosstalk has a number of elements which may be identified or estimated: the crosstalk coupling H_l (f) between each pair, the crosstalkers' transmitted spectrum D,(f), sequences of sampled received crosstalk, etc. DSL modems can measure crosstalk samples.

Assuming correct decisions, the received sampled sum of crosstalk and noise is easily

obtained in each receiver by subtracting R_k(f) = R_k(f) from Y (f). These samples and their

statistics can be exploited to improve the performance of the receiver. Often a process of successive estimation and cancellation is performed where the most powerful crosstalk disturber is first estimated and subtracted, then the next most powerful and so on.

If crosstalk is measured while a DSL system is running, what is measured or estimated is the crosstalk in the bandwidth of the considered DSL system. For this reason, other DSL services running on adjacent pairs may not be detected if their bandwidth is not significantly overlapping, with the bandwidth of the disturbed system. A typical example would be ISDN crosstalk into non-overlapped downstream ADSL. If this is the case, there is little performance degradation for the considered DSL service but, from a spectrum management point of view, it may be important to have an accurate map of all the services that generate crosstalk into a given pair. Moreover, it may also be important to identify the services that are generating crosstalk on a pair that may not even be carrying DSL services. An accurate map of all the services that generate crosstalk into a given pair is an impossible goal to achieve via modem-based methods of system identification if the considered pair is not in use. In this case, the way to achieve such a complete description of the crosstalkers is to perform a preliminary PSD measurement, e.g. via a spectrum analyzer or a selective voltage meter.

An important aspect of the crosstalk identification algorithm is a calculation of crosstalk correlation in the frequency domain. The correlation receiver is known to be an optimum detector in the case of equally likely transmitted signals with known waveforms transmitted over an AWGN channel. It is mathematically equivalent to the matched-filter receiver which can be shown to maximize the output signal to noise ratio by using Schwarz's inequality. The basic structure of a correlation receiver is used here, except that it is adapted with known transmitted signals but unknown crosstalk couplings

The correlation coefficient p_{x γ} between two data arrays X = {x₁,....,

Y = {y₁,...., y_N} with standard deviations σ and σ_γ and means μ_x and μ_γ , is defined

as

„ =**^ , ⁽3⁾ ^σx * ^σγ

1 ^N where Cov(X,Y) = — ∑ χ_j -μ_x ) »(y_j -μ_γ) and -l ≤ p_xγ < 1 .

^N 7=1

The correlation coefficient is a measure of how well the two data sets move together (positive values) or move apart from one another (negative values).

The data array Y = {y_l,...., y_N } is the measured crosstalk power spectrum density

(PSD) of the unknown DSL disturber. The data array X = {x_λ,...., x_N } is a reference or

basis crosstalk PSD profile caused by a known DSL disturber. Each basis set of crosstalk PSDs is generated from a single canonical set of measured strong pair-to-pair crosstalk couplings and a specific type of transmitted DSL PSD. Each type of transmitted DSL PSD is multiplied by all of the canonical crosstalk couplings to generate a basis set for that DSL. The assumption is that a member of this basis crosstalk set is highly correlated with any crosstalk that is strong enough to be troublesome or measurable. An example of a canonical set of pair-to-pair couplings is shown in FIG. 6. This canonical set comprises the fourteen out of 300 possible pair-to-pair NEXT couplings with the smallest crosstalk loss sum. The crosstalk loss sum is obtained by dB summing over 401 equally spaced frequency samples in the 10 kHz - 2 MHz band. The basis set of T1 disturber NEXT PSDs for this example is shown in FIG. 7. Basis sets for ISDN, HDSL, ADSL, 2B1Q SDSL and HDSL2 can be generated similarly. Increasing the number of members belonging to the canonical set of crosstalk transfer functions would increase the accuracy of the identification algorithm. Another example of a canonical set of pair-to-pair couplings could be all 300 measured independent pair-to-pair couplings. A particularly useful NEXT basis set is found to comprise the ISDN, HDSL, ADSL, 2B1 Q SDSL and HDSL2 NEXT sets that have power exceeding or equaling the 49^th disturber power for a given DSL type obtained from the ANSI 1% NEXT model as defined in the Spectrum Management Standard, T1.417. There are several ways of determining the canonical crosstalk PSD basis set. The

basis set Ξ ≡ |x⁽¹⁾ X⁽²⁾ ••• X^(P) J is the full rank NxP matrix containing all the

canonical crosstalk PSDs. Vector X^il) ≡

is the h crosstalk PSD

belonging to the canonical basis and

is its sample at frequency f,. Vector

a ≡

e2 ••^■ e_N is an

error vector. The presence of the error vector E is due to the fact that the measured crosstalk Y is due to a pair-to-pair coupling function that may be close, but not equal to, a coupling in the canonical basis.

Identification of the crosstalk disturbers is generally improved with increasing dictionary size. Large dictionaries, however, imply searches with high computational complexity. The chosen size for the canonical set is a trade-off between accuracy and computational complexity. As described above, a canonical PSD basis set can be formed by using all the available measured pair-to-pair crosstalk couplings and combining them with all the DSL types under consideration. This approach, however, yields a very large basis set. To reduce the size of this large basis set, the crosstalk power within each DSL type can be ranked with only the first B vectors with the smallest crosstalk loss sum being chosen. The crosstalk loss sum is obtained by dB summing over the frequency samples in the band of interest.

The similarity of crosstalkers of the same type (but different pair-to-pair couplings) is much greater than the similarity of crosstalkers of different types. The present invention exploits this observation by applying the following method. For each type of crosstalkers, the dominant dimensions from the space spanned by all the PSD's for that type are extracted. Vectors corresponding to these dimensions are then used in lieu of the complete set of crosstalkers for that type. After the type of crosstalker is identified, the search for pair-to-pair couplings in the basis that is closed to the actual one is done within the subset of the original (and large) basis pertaining to the identified crosstalker type.

To obtain the above-described vectors, the set or matrices of all possible PSD couplings Ξ is partitioned into A sub-matrices pertaining to each type of crosstalker or disturber, j. Performing a Singular value Decomposition (SVD) on each of these sub- matrices results in the following equation: _Ξ ⁽J⁾ _{= u} ⁽J⁾ _D ⁽J⁾ _v ⁽J⁾'_t I ≤ J ≤ A , (4)

where (β^} and V^ are orthogonal and D^a> is a diagonal matrix containing the singular values. Given the number of dimensions q,- that are desired to be used to approximate Ξ^ϋ>, the columns of U ⁾ are extracted which correspond to the q,- largest singular values. These columns are then used as the basis for the crosstalkers of the given type. Note that this

^A new basis has P = ∑qj vectors and is much smaller than the original one (i.e., P «P).

Moreover, all vectors in each iβ⁾ are orthogonal, thus mitigating any multicollinearity issues associated with the old basis.

This new and reduced-size basis enables us to identify efficiently which type(s) of crosstalk exist in the sample, but not which particular pair-to-pair couplings are associated with the crosstalk. However, after identifying the crosstalk types, we can rerun the analysis using the subset of the original crosstalk basis associated with that type of crosstalker. Practically, we have decomposed the original search problem among P=A B vectors, into

two search problems, the first one among P «B vectors, the second one among B vectors In the method of the present invention for the case of a single disturber, the measured crosstalk PSD is correlated as described in the last subsection with each of the basis PSDs, and the identification is simply the type of crosstalker which has the highest correlation. In the examples set forth herein, most common DSL types are simulated: ISDN Basic Rate Interface (BRI), HDSL, T1 , ADSL, 400 kbps SDSL, 1040 kbps SDSL, 1552 kbps SDSL, and HDSL2. All transmit system PSDs are as defined in the Spectrum Management Standard, T1.417. ADSL and HDSL2 have different PSDs upstream and downstream.

An unknown disturber's NEXT PSD is shown in FIG. 9. The calculated correlation with each member of each DSL crosstalk basis set (for the example of a fourteen member canonical crosstalk coupling set ) is given in Table 2 set forth below. The DSL with the highest correlation and thus the identity of the unknown disturber is correctly identified as 1552 kbps SDSL.

Table 2.

Another unknown disturber's NEXT PSD is shown in FIG. 10. The calculated correlation with each member of each DSL crosstalk basis set (for the example of a fourteen member canonical crosstalk coupling set) is given in Table 3. The DSL with the highest correlation and thus the identity of this unknown disturber is correctly identified as downstream ADSL.

Table 3.

In realistic cases it has been found that the DSL type of a single disturber with significant power is identified correctly nearly every time.

The case of mixed crosstalk arises when at least two different services are present in a binder. If the measured crosstalk PSD consists of the contribution of different kinds of disturbers, the identification algorithm described above would not identify the whole set of crosstalkers all at once but would identify the strongest candidate only.

In order to overcome this limitation in cases of mixed crosstalk, the method of the present invention uses a frequency-domain onion peeling technique based on spectral subtraction methods as set forth in FIG. 11. Spectral subtraction is a method originally proposed in speech and music processing for the restoration of the power spectrum of a signal observed in additive noise through subtraction of an estimate of the average noise spectrum from the noisy signal spectrum.

At the start 300 of the method the iteration counter is set to zero 305 and at step

310 a measurement of the PSD of a subscriber loop 18 is taken. Next at step 320 the first estimated PSD is set to the measured PSD as set forth below:

(5)

At step 325, the iteration counter is set at one and the estimated PSD of the received crosstalk is correlated at step 330 with the basis set to identify the first source of crosstalk, i.e., the one having the highest correlation in accordance with the procedure described above.

The onion-peeling technique used to identify additional crosstalk disturbers can be explained by focusing on the crosstalk received on a single pair. The subscript k in

equation (2) is dropped and y(m) = r(m) + n(m) + . h_t (m)d_i (m) is the noisy received signal

in the sampled-time domain, where r(m) is the received message signal, n(m) is additive background noise, and y(m) is the noisy observation, respectively. To simplify further, define each crosstalk component as

= h;(m)d;(m) so that the composite received

crosstalk is c(m) = .c_i(m) - .h_i(m)d_i(m) . Then ignoring the background noise and

transforming to the frequency domain, Y(f) = R(f) + C(f) . Power spectrum subtraction is

defined as in the following:

² ] |2 where R(f) is an estimate of the signal PSD, \Y(f)\ is the instantaneous PSD of the

noisy observation, b is a coefficient chosen to minimize the residual error and \C(f)\ is the

time-averaged noise spectra. The basic assumption is that the noise is a slowly varying process so that averaging would not smear the noise spectrum. Let c(m) = c-ι(m) + c₂(m) be a composite crosstalk constituted of two crosstalk terms pertaining to two different DSL systems. Assuming that the identification algorithm (described above)identifies the strongest disturber that generated crosstalk c^m), an

Λ 2 estimate of its PSD C_λ(f) can be obtained. At step 340 a decision is made as to whether

the maximum correlation is greater than a set threshold. If it is not, then the algorithm has identified all of the crosstalk components. If it is, the identified component must be subtracted for further processing. Basically, the composite crosstalk c(m) can be viewed as a noisy observation of the useful signal c₂(m) embedded in the noise c-ι(m). Because the output of the crosstalk identification algorithm is a true PSD obtained on the basis of a PSD mask and an estimated pair-to-pair coupling function, there is no need to perform an

averaging operation on the estimated PSD of the strongest crosstalker (noise) C_λ(f)

Moreover, the instantaneous PSD of the noisy observation \Y(f)\ (necessary in the

original formulation of the problem because of the non-stationary nature of speech and

music signals) can be replaced in our case by the measured PSD \C( f )\ . Therefore, the

power spectrum subtraction for crosstalk identification can be defined as

c₂( f )² = \c( f )\² -lc f ) (7)

yielding the residual PSD C₂(f)\ at step 350 of FIG. 11.

Due to the great variations in the pair-to-pair coupling functions, spectral subtraction can result in negative estimates of the power spectrum. Power spectra are non-negative functions of frequency, and any negative estimate of these variables should be mapped into a non-negative value. To avoid negative estimates, the power spectrum is post processed in steps 360 and 370 using a mapping function T[ ]of the form:

Empirically a good value of β ranged from approximately 0.0 to approximately 0.1. The

simplest choice for the function Φ[-] is Φ[|C(/)| ] = noise floor = -140 dBm/Hz . Another

possible choice could be Φ[|C( )|²] = β

. A flow chart for implementing the crosstalk identification algorithm is given in FIG.

11. The residual PSD is fed to the crosstalk identification algorithm that is now

able to unveil the nature of the second disturber and estimate its PSD C₂(f) Steps 325

and 330 are reiterated until all disturbers are identified, i.e., the iteration ceases at step 380 when the maximum correlation is less than or equal to the threshold at step 340. It has been found by simulations that overall accuracy is enhanced by retaining and using identified crosstalkers only if the maximum crosstalk correlation is greater than a certain threshold that is empirically determined. The threshold is preferably between approximately 0.7 and 0.99 and is more preferably approximately 0.9. An example of the technique follows.

Consider the measured mixed crosstalk given in Fig. 12 that happens to be a mixture of crosstalk from SDSL ©1040 kbps and HDSL. The crosstalk identification algorithm yields the results given in Table 4.

Table 4

The crosstalk identification method of the present invention as set forth in FIG. 11 first identifies the strongest disturber as SDSL @1040 kbps. On the next iteration it identifies HDSL as the second strongest disturber. Finally on the third iteration, the maximum correlation of the remainder NEXT is less than 0.90, the set threshold.

The use of spectral subtraction may cause deterioration in the quality and the information content of the residual crosstalk PSD. In the literature, there are a number of variants of spectral subtraction that aim to provide consistent performance improvement across a range of SNRs. The spectral subtraction equation can be expressed as the product of the noisy signal spectrum and the frequency response of a spectral subtraction filter as:

assuming for now that b=0, where the frequency response of the spectral subtraction filter is defined as:

The spectral subtraction filter H(ή is a zero-phase filter, with its magnitude response in the range 0 ≤ H(/) < 1. The filter acts as an SNR-dependent attenuator. The attenuation at

each frequency increases with the decreasing SNR, and conversely decreases with increasing SNR.

The least mean squared error linear filter for noise removal is the Wiener filter. The implementation of the Wiener filter requires the power spectra (or equivalent^ the correlation functions) of the signal and the noise process. The Wiener filter is based on ensemble average spectra of the signal and the noise, and the averaging operations are taken across the ensemble of different realizations of the signal and noise processes. On the other hand, the spectral subtraction filter, uses instantaneous spectra of the noisy observation and the time averaged spectra of the noise. This is necessary since in spectral subtraction only one realization of the noise process is available.

Spectral subtraction is computationally efficient because it only requires an estimate of the noise power spectrum. Because it does not utilize the statistics of the signal process, however, it may use too little a priori information. However, for an ergodic process, the time-averaged spectrum approaches the ensemble averaged spectrum, so that the spectral subtraction filter asymptotically approaches the Wiener filter. This property does not hold when processing speech and music signals because they are intrinsically non-stationary (and, therefore, non-ergodic). When spectral subtraction is applied to the present case of crosstalk identification, however, the quantities involved are true PSDs and not instantaneous or time-averaged ones. Therefore, the implementation of spectral subtraction may be viewed as a sort of optimal Wiener filtering. This interpretation may explain why spectral subtraction seems to perform very well when applied to successive crosstalk identification. The major limitation encountered is the non-linear processing distortion due to the necessity of mapping negative values into non-negative values. This distortion increases at every iteration when equation (8) is implemented and limits the accuracy of the method to the identification of up to a small number of different crosstalker types. From a practical point of view, this does not look as a major limitation because it is very unlikely that more than a few interferers cause performance degradation in any one DSL system.

There are methods other than correlation that may be used to identify the source of crosstalk including the use of multiple regression techniques and Matching Pursuit (MP) algorithms. Such methods would replace the identification of crosstalk disturber by correlation or could be used to refine the correlation method. Two other approaches to the problem of multiple disturber identification as described below, enhance the identification accuracy, thus by-passing the non-linear distortion problems due to the spectral subtraction technique. Linear multiple regression approach

Let Y = {y_j}ϋ^~ι be the set of frequency samples of the measured crosstalk PSD

caused by an unknown DSL disturber. Similarly, let X =

be the set of frequency

samples of the basis crosstalk PSD profile caused by a known disturber. Let P be the number of all the basis crosstalk PSD profiles, so that the set of frequency samples of the

/ -th basis crosstalk PSD profile is X^{( k )} =

, where 1 < k ≤ P . The problem is to

find the single disturber that generates crosstalk V given the set of all the crosstalk PSD profiles χ(^k) - { }•_, , i-e. to determine the relationship between the two variables X^

and Y. This is a classical statistical problem that is solved with linear regression. In particular, the linear regression model is:

_yi=a^{k) +b^(k)x_j ^(k) +e^{k} X≤i≤N , l≤k≤P (11)

where the regression coefficients a^(fc) and b^(k> are determined by the condition that the sum

S<^k> = ∑(e = ∑(y,- -a^(k) -b^ik)x^{k)) of the squared residuals is minimum. The i=X =l fitting of the regression coefficients is done over the N frequency samples. In so doing, P

residuals are obtained and the DSL with the lowest S' ' is identified as the disturber.

It is easy to show that the sum of squared residuals S' ' can be expressed in

terms of the correlation coefficient in (3). In fact, plugging into S^ ' the expression of the optimal regression coefficients, after some manipulation it can be shown that:

N

Now, since the term ∑(y_/-μy) is constant, the search for the minimum sum of i=l

squared residuals S^ is equivalent to the search of the highest correlation coefficient

p_v(k)_v- Therefore, the correlation method described for the single disturber case is

equivalent to solving a single linear regression problem.

This interesting result justifies the possibility of addressing the problem of identification of multiple disturbers as a multiple regression problem. In fact, the linear multiple regression model is ( 1 < i ≤ N , 1 < kj ≤ P , 1 < j ≤ M ):

_yi =_fl0(*ⁱ--*^if ) _+fll(*ι.~.*^Jk )*(*^!> +... + a_M ^ikl'-"'^kM)x^kM) +el^kl'^"-'^kM) , (13) where M is the hypothesized number of disturbing DSL types. Standard techniques are available to solve this problem.

While standard techniques are available to solve such multiple regression problems the application of multiple regression techniques requires the use of several non-standard limitations. All non-zero coefficients must be positive, i.e. the power contribution to the spectral density by each disturber is positive. Moreover, there are many more potential disturber spectra than frequencies at which the power is evaluated, whereas standard regression techniques require that the number of independent variables be strictly less than the number of observations. Independent variables (the disturbers) may be disqualified from the model in pre-specified groups based upon a group's total power contribution to the modeled spectra. The problem of having more independent variables than observations can be solved by using the Singular Value Decomposition method described in the present invention that allows a dramatic reduction of the size of the crosstalk basis set. A way of checking that the coefficients remain positive may be efficiently addressed by computing the so called t- or f- statistics for every coefficient, so that we can eliminate a certain number of columns at every iteration on the basis of the variance of their estimation. These quantities cal also be used for the stopping criterion. One of the main advantages of using multiple regression techniques is the fact that no intermediate residual needs to be computed, thus alleviating the non-linear distortion effects due to the calculation of the residuals.

Matching Pursuit Approach

Rewriting equation (13) with M=P, i.e. considering all the PSD profiles of the basis,

and by posing AQ ¹"^"' ^M - 0 it can be seen that:

Passing to a vector notation, equation (14) can be expressed as: P

Y = a_lX^{( 1)} + a₂X^{( 2 )} + ... + a_PX^{( P )} + E = ∑a_jX^{( j )} + E = Ξ + E , (15)

7=1

where Y ≡ [y_j y₂ ^{■ ■}• y^ ] *^{s tne} vector containing the measured crosstalk PSD from

unknown disturbers across all frequencies, Ξ ≡ [z⁽¹⁾ X^{(2) • • •} Z^(P) J is the full rank

Λ/xP matrix containing all the PSD profiles, a ≡

a₂ ^{• • •} a_PY is the vector

containing the weighting coefficients, and E ≡ [ei e₂ ^{■ ■ ■} e_N ] is the vector containing

the residuals over all the frequency points, respectively.

In the majority of the cases of practical interest for crosstalk identification, it is the case that Λ/<P. For example, the inventors have previously considered cases for which

Λ/=401 and P ranges between 80 and 2,560 (obviously, higher values of P yield lower misidentification rates). Since Λ/<P, matrix Ξ constitutes an overcomplete set of vectors for

Si^N . Moreover, in practice, only combinations of at most 10 crosstalkers can be considered measurable (or harmful) within a binder. Therefore, among the P weighting coefficients ay, 1 ≤ j ≤ P , only few should be expected to be non-zero.

On the basis of the previous considerations, the problem of identification of multiple crosstalkers can be considered equivalent to the problem of finding an optimal sparse representation of a vector from an overcomplete set of vectors. Unfortunately, it has been recently shown that the optimal solution to this problem is an NP-complete problem and requires a combinatorial search of prohibitive cost. Thus, more practical but suboptimal vector selection algorithms have been developed. Among these algorithms, those suitable for the problem of multiple crosstalker identification are the Matching Pursuit (MP) algorithm and its variations, and the FOCal Undetermined System Solver (FOCUSS). Given a signal vector Y and an overcomplete basis Ξ, these algorithms solve the problem of finding the most compact representation of V within a given tolerance using the basis vectors in the dictionary Ξ. In order to apply these techniques to our problem, several modifications to the original algorithms must be implemented. In particular, in addition to the non-standard limitations mentioned above, some or all of the following modifications should be considered. At each iteration, the difference between the measured data and its projection along the direction of the chosen columns must be carried out. This difference must always be positive. When a column is chosen, it is added to the dictionary Ξ and no residual is computed. When a column is chosen, the whole dictionary is projected along that direction and no residual is computed.

In FIG. 13 the use of the MR or BBS techniques in a method of identifying an crosstalk disturber is depicted. In FIG. 13 at step 410 the power spectral density (PSD) for a given line is measured. At step 420 the basis set of possible crosstalk disturbers is reduced using single value decomposition. The reduced crosstalk basis set is then used to identify, at step 430, the of the identity of the crosstalk disturbers using either the MR or BBS technique described above. At step 440 the actual pair-to-pair coupling of the crosstalk disturber is then estimated using the MR or BBS techniques but utilizing the original complete basis set pertaining to the identified disturber.

It is also possible to combine the correlation method with the two methods of crosstalk identification previously described. This can be done by using the correlation method for an initial estimate of the first crosstalker, and then refining such estimate using the MR and BBS techniques. The correlation method would identify the type of crosstalker, and the MR or BBS algorithms would exploit this information to refine the estimate of the crosstalker. This combination of methods has proven to be very effective in the construction of good residuals so that the overall accuracy of the identification is enhanced. An example on how to combine these methods is given in FIG. 14. In FIG. 14 at step 510 the power spectral density (PSD) for a given line is measured. At step 520 an iteration counter is set and at step 530 the identity of the crosstalk disturber is first estimated using the correlation technique described above. At step 535 the estimated crosstalk is refined using the MR or BBS techniques. At step 540 the stopping criterion is applied and is the identified crosstalk disturber falls within the stopping criterion then the method ends at step 580. If the stopping criterion has not been met the residual is computed using spectral subtraction at step 550 as described above and the iteration counter is increased by a count of one at step 525, The next most likely crosstalk disturber is then estimated in the next iteration which process continues until the stopping criterion is met.

The above description has been presented only to illustrate and describe the invention. It is not intended to be exhaustive or to limit the invention to any precise form disclosed. Many modifications and variations are possible in light of the above teaching. The applications described were chosen and described in order to best explain the principles of the invention and its practical application to enable others skilled in the art to best utilize the invention on various applications and with various modifications as are suited to the particular use contemplated. For example, it is known that the measurement of the power spectral density on a subscriber line could either be a direct measurement taken through a network analyzer or other equivalent device but could also include estimated measurements based on information provided to the system from a modem.

Claims

CLAIMSWe claim:

1. A method for identifying the source of crosstalk disturbance in a subscriber loop comprising the steps of: measuring the power spectral density of the noise present on a subscriber loop; correlating the power spectral density for said subscriber loop with a predetermined set of power spectral densities for a group of possible crosstalk disturbers; and, selecting the crosstalk disturber having the most closely correlated power spectral density; and, refining the selection of the crosstalk disturber using an additional mathematical technique.

2. The method of claim 1 further comprising the steps of: subtracting the power spectral density for the identified crosstalk disturber from the measured power spectral density of said subscriber loop using spectral subtraction to generate a residual power spectral density; correlating the residual power spectral density with the predetermined set of power spectral densities for the group of possible crosstalk disturbers; and, selecting the crosstalk disturber having the most closely collated power spectral density to the residual power spectral density, and refining the selection of the crosstalk disturber using an additional mathematical technique.

3. The method of claim 2 wherein the steps of subtracting, correlating, selecting and identifying are iteratively performed until a stopping criterion is reached.

4. The method of claim 3 wherein the stopping criterion is a predetermined correlation threshold.

5. The methods of claims 1 and 2 wherein the step of refining in each comprises the application of multiple regression techniques.

6. The methods of claims 1 and 2 wherein the step of refining comprises the use of a matching pursuit (MP) algorithm.

7. The methods of claims 1 and 2 wherein the step of refining comprises the use of the FOCal Undetermined Problem Solver (FOCUSS).

8. The method of claim 2 wherein negative power spectral densities resulting from said subtracting step are mapped into a non-negative value.

9. The method of claim 1 further comprising the step of sending the identity of the selected crosstalk disturber to a system for cancellation of the crosstalk disturber.

10. The method of claim 1 further comprising the step of sending the estimated pair-to-pair coupling of the selected crosstalk disturber to a system for cancellation of the crosstalk disturber.

11. A method for identifying crosstalk disturbers on a subscriber loop comprising the steps of: measuring the power spectral density of the noise present on a subscriber loop; matching the power spectral density for said subscriber loop with a predetermined set of power spectral densities for a group of possible crosstalk disturbers using multiple regression to identify crosstalk disturbers;

12. The method of claim 11 wherein all non-zero coefficients in the multiple regression must be positive.

13. The method of claim 11 wherein in the application of the multiple regression technique selected crosstalk disturbers are disqualified from the model in pre-specified groups based upon a group's total power contribution to the modeled spectra.

14. A method for identifying crosstalk disturbers on a subscriber loop comprising the steps of: measuring the power spectral density of the noise present on a subscriber loop; matching the power spectral density for said subscriber loop with a predetermined set of power spectral densities for a group of possible crosstalk disturbers using a best basis selection technique to identify the set of crosstalk disturbers;

15. The method of claim 14 wherein the best basis selection technique uses a Matching Pursuit (MP) algorithm.

16. The method of claim 14 wherein the best basis selection technique uses the FOCal Undetermined Problem Solver (FOCUSS).

17. The method of claim 14 wherein the difference between the residual power spectral density and its projection along the direction of the chosen columns must always be positive.

18. The method of claim 14 wherein when a column is chosen, the column is added to the dictionary Ξ and no residual is computed.

19. In a system for identifying the source of crosstalk disturbance in a subscriber loop wherein the measured power spectral density of the noise on the subscriber loop is compared with a predetermined set of power spectral densities for a group of possible disturbers, the method of determining the predetermined set of power spectral densities for a group of possible disturbers comprising the steps of: transmitting a signal over a test loop in a bundle of test loops; measuring the pair-to-pair crosstalk couplings to generate a set of canonical crosstalk couplings; repeating the transmitting and measuring steps for each test loop; multiplying the power spectral density for each type of DSL by the set of canonical crosstalk couplings to generate the set of power spectral densities for a possible disturber;

20. The method of claim 19 wherein the matrix containing the crosstalk basis set is partitioned into sub-matrices pertaining to each type of possible crosstalk disturber, and a singular value decomposition is performed on each sub-matrix to produce a reduced crosstalk basis set .

21. The methods of claims 1 , 2, 11 and 14 wherein the disturber type is identified using a reduced crosstalk basis set wherein the matrix containing the crosstalk basis set is partitioned into sub-matrices pertaining to each type of possible crosstalk disturber, and a singular value decomposition is performed on each sub-matrix to produce a reduced crosstalk basis set.

22. A method for identifying crosstalk disturbers on a subscriber loop based on the availability of a crosstalk basis set comprising the steps of: measuring the power spectral density of the noise present on a subscriber loop; performing a singular value decomposition on the matrix of the crosstalk basis set to reduce its size; identifying the disturbers for said subscriber loop with said reduced size crosstalk basis set using a first mathematical technique; identifying the closest pair-to-pair coupling pertaining to the disturber using a second mathematical technique on the portion of the original crosstalk basis set pertaining to said identified disturber types

23. The method of claim 22 wherein the first mathematical technique is a correlation technique, a multiple regression technique or a best basis selection method.

24. The method of claim 22 wherein the second mathematical technique is a correlation technique, a multiple regression technique or a best basis selection method.

25. The methods of claims 23 or 24 wherein the best basis selection method is a matching pursuit (MP) algorithm.

26. The methods of claims 23 or 24 wherein the best basis selection method is the FOCal Undetermined Problem Solver (FOCUSS).