DE102004022660B4 - Apparatus and method for analyzing an information signal - Google Patents

Abstract

In order to analyse an information signal, a significant short-time spectrum is extracted from the information signal. The extraction device (16) is embodied in such a way as to extract the short-time spectra which come closer to a specific characteristic than other short-time spectra of the information signal. The extracted short-time spectra are then decomposed (18) into component signals, by ICA analysis, a component signal spectrum representing a profile spectrum of a sound source which generates a sound corresponding to the required characteristic. An amplitude envelope is calculated (20) for each profile spectrum from a series of short-time spectra of the information signal and from the determined profile spectra, said envelope indicating how the profile spectrum of a sound source generally varies over time. The profile spectra and associated amplitude envelopes describe the information signal that can be further evaluated, e.g. for the purposes of a transcription in the case of a music signal.

Description

The The present invention relates to the analysis of information signals, such as audio signals and in particular the analysis of information signals resulting from a superposition of sub-signals consist of a partial signal from a single source or a Group of individual sources.

The progressive development of digital distribution media for multimedia Contents leads to a big one Diversity of offered data. For the human user is the limit of the manageable long exceeded. Thus, the content description of the data by metadata increasingly gains in importance. in principle the goal is not only text files but also z. Music files, Video files or other information signal files searchable with the same comfort as usual text databases becomes. An approach for this is the well-known MPEG 7 standard.

Especially in the analysis of audio signals, ie signals, the music and / or Language is the extraction of fingerprints from greater Importance.

The aim is further "enriching" audio data with metadata in order, for example, to a piece of music to recover metadata based on a fingerprint. The "fingerprint" should on the one hand meaningful be, and on the other hand as possible short and concise be. "Fingerprint" means thus a compressed information signal generated from a music signal, which does not contain the metadata but for referencing to the metadata e.g. by searching in a database, e.g. in a system for the identification of Audio material ("AudioID").

Usually consist of music data from the overlay of sub-signals from single sources. While listening to a pop music typically there are relatively few individual sources, namely the Singers who Guitar, bass guitar, drums and a keyboard, so can the number of sources for an orchestral piece get very tall. An orchestral piece and a pop piece of music For example, consist of a superposition of the individual Instruments emitted tones. An orchestral piece or any piece of music thus provides a superposition of Partial signals from individual sources, the sub-signals the from the individual instruments of the orchestra or pop music ensemble generated sounds and the individual instruments are single sources.

alternative can also groups of original ones Sources are understood as single sources, giving a signal at least two individual sources can be assigned.

A Analysis of a general information signal will be hereinafter merely exemplified by an orchestral signal. The analysis of a Orchestra signal can be performed in many ways. So there may be a desire to recognize the individual instruments and to extract the individual signals of the instruments from the total signal and if necessary to convert it into a musical notation, the musical notation would act as "metadata" Possibilities of Analysis is to extract a dominant rhythm being a rhythm extraction based on the percussion instruments better than on the basis of the more sound-giving instruments, which are also referred to as harmonic-sustained or "harmonic sustained" instruments become. While Percussion instruments typically timpani, drums, rattles or otherwise Including percussion instruments are among the most harmoniously endured All other instruments, such as violins, wind instruments, Etc.

Farther Become the percussion instruments all those acoustic or synthetic sound generators counted which contribute to the rhythm section due to their sound characteristics (e.g., rhythm guitar).

So would it be for example, for rhythm extraction of a piece of music desirable from the entire piece of music only to extract percussive parts and a rhythm detection then to carry out on the basis of these percussive shares, without that the rhythm recognition by signals of the harmoniously endured Instruments "disturbed" is.

on the other hand any analysis for the purpose of extracting metadata, the only information of harmonic sustained instruments (e.g., a harmonic or melodic analysis) benefit from an upstream separation and a further processing of the harmoniously held shares.

In recently, Past has been related to the use of the technique in this regard Blind Source Separation (BSS) and Independent Component Analysis (ICA) for signal processing and signal analysis. applications can be found in particular in biomedical technology, communication technology, the artificial one Intelligence and image processing.

Generally The term BSS includes techniques for separating signals a mix of signals with a minimum of prior knowledge of the Nature of the signals and the mixing process. The ICA is a procedure which makes use of the assumption that the sources underlying a mix at least to some extent statistically independent of each other are. Furthermore, the mixing process is considered to be fixed in time and the number of observed mixed signals not smaller than the number assumed the mixture underlying the source signals.

• [1] M.A. Casey and A. Westner, "Separation of Mixed Audio Sources by Independent Subspace Analysis", in Proc. of the International Computer Music Conference, Berlin, 2000
• [2] I.F.O. Orife, "Riddim: A rhythm analysis and decomposition tool based on independent subspace analysis", Master thesis, Darthmouth College, Hanover, New Hampshire, 2001
• [3] C. Uhle, C. Dittmar and T. Sporer, "Extraction of Drum Tracks from polyphonic Music using Independent Subspace Analysis", in Proc. of the Fourth International Symposium on Independent Component Analysis, Nara, Japan, 2003
• [4] D. Fitzgerald, B. Lawlor and E. Coyle, "Prior Subspace Analysis for Drum Transcription", in Proc. Of the 114th AES Convention, Amsterdam, 2003
• [5] D. Fitzgerald, B. Lawlor and E. Coyle, "Drum Transcription in the presence of pitched instruments using Prior Subspace Analysis", in Proc. of the ISSC, Limerick, Ireland, 2003
• [6] M. Plumbley, "Algorithms for Non-Negative Independent Component Analysis", in IEEE Transactions on Neural Networks, 14 (3), pp 534–543, May 2003

An extension of the ICA is the Independent Subspace Analysis (ISA). Here, the components are subdivided into independent subspaces or subspaces whose components need not be statistically independent. By transforming the music signal, a multi-dimensional representation of the mixed signal is determined and the last assumption for the ICA is met. Various methods for calculating the independent components have been developed in recent years. Relevant references, some of which also deal with the analysis of audio signals, are the following:

• [1] MA Casey and A. Westner, "Separation of Mixed Audio Sources by Independent Subspace Analysis," in Proc. of the International Computer Music Conference, Berlin, 2000
• [2] IFO Orife, "Riddim: A rhythm analysis and decomposition tool based on independent subspace analysis", Master thesis, Dartmouth College, Hanover, New Hampshire, 2001
• [3] C. Uhle, C. Dittmar and T. Sporer, "Extraction of Drum Tracks from Polyphonic Music Using Independent Subspace Analysis", in Proc. of the Fourth International Symposium on Independent Component Analysis, Nara, Japan, 2003
• [4] D. Fitzgerald, B. Lawlor and E. Coyle, Prior Subspace Analysis for Drum Transcription, in Proc. Of the 114th AES Convention, Amsterdam, 2003
• [5] D. Fitzgerald, B. Lawlor and E. Coyle, "Drum Transcription in the Presence of Pitched Instruments Using Prior Subspace Analysis", in Proc. of the ISSC, Limerick, Ireland, 2003
• [6] M. Plumbley, "Algorithms for Non-Negative Independent Component Analysis," in IEEE Transactions on Neural Networks, 14 (3), pp 534-543, May 2003

In [1] is a method of separating single sources from mono audio signals shown. In [2] is an application for separation into single tracks and subsequently given the rhythm analysis. In [3], a component analysis is performed to a separation into percussive and non-percussive sounds of a polyphonic piece to reach. In [4] the Independent Component Analysis (ICA) applied to amplitude bases, by means of generally calculated Frequency bases from a spectrogram representation of a drum track be won. This is done for the purpose of transcription. In [5] this method is extended to polyphonic music pieces.

The first aforementioned publication by Casey is illustrated below by way of example for the prior art. This publication describes a method for separating mixed audio sources by the Technique of the independent Subspace analysis. This is done using BSS techniques an audio signal split into individual component signals. To the Determining which of the individual component signals belong to a multicomponent subspace is a grouping done to that the similarity of the components are represented by a so-called Ixegramm becomes. The Ixegram is called a cross-entropy matrix of independent components referred to each other. It is calculated by all individual Component signals examined in pairs in a correlation calculation to find a measure of how similar two Components are. about All component signals therefore become an exhaustive pairwise similarity calculation carried out, so that there is a similarity matrix yields, when along a y-axis all component signals are applied, and further along the x-axis also all component signals are plotted. This two-dimensional array provides a similarity measure for each component signal another component signal. The Ixegram, so the two-dimensional Matrix, is now used to perform a clustering, with this grouping using a clustering algorithm the basis of diadic data. To be optimal Partitioning of the Ixegram in k classes is performed defines a cost function that reduces the compactness within a Clusters measures and homogeneity between clusters. The cost function is minimized, so that ultimately has an association of individual components to individual subspaces results. Applied to a signal that is a speaker in context represents a continuous waterfall noise, arises as Subspace of the speaker, wherein the reconstructed information signal of the speaker subspace a significant attenuation of the waterfall noise shows.

A disadvantage of the concepts described is the fact that the case occurs very likely that the signal components of a source come to lie on different component signals. This is the reason that, as stated above, a complex and computation-intensive similarity calculation among all the component signals is performed to obtain the two-dimensional similarity matrix the basis of which is then carried out by means of a cost function to be minimized, a division of component signals into subspaces.

Of Another disadvantage is that in the case where several individual Sources exist, so where the output signal is not known a priori is, after a long time Invoice a similarity distribution but that the similarity distribution exists even no actual Insight into the actual Audio scene delivers. So knows the Viewers merely that certain component signals to each other are similar in terms of minimized cost function. He knows, however not what information these ultimately obtained subspaces carry or which original one Single source or which group of single sources through a subspace being represented.

The Independent Subspace Analysis (ISA) can thus be used to a time-frequency representation, e.g. a spectrogram, an audio signal into independent component spectra disassemble. Set the previously described previous methods either to a calculation-intensive determination of frequency and amplitude bases from the entire spectrogram or a priori defined frequency bases. Such a priori defined frequency bases or profile spectra consist, for example, in saying that that in one piece very likely a trumpet occurs, and then to signal analysis an example spectrum a trumpet is used.

This Procedure has the disadvantage that you from the outset all occurring Instruments must know what the automated processing already runs counter to this principle. Another disadvantage is that it works exactly when not just trumpets, for example, but many different types of trumpets, all in their timbre and thus differentiate in their spectrum. Would you proceed in such a way that Now you use all kinds of sample spectra for component analysis, so the process is again very complex and gets a lot high redundancy, because typically not all conceivable different Trumpets occur in one piece, but only trumpets of a single species, so with a single Profile spectrum, or maybe with a few different ones Tones, so few profile spectra. Becomes even more problematic it with different notes of a trumpet, especially every tone depending on pitch has a spread / compressed profile spectrum. To this to take into account the computational effort is also immense.

on the other hand then the decomposition is extraordinary due to ISA concepts computationally expensive and prone to failure, though the entire spectrogram is used. It should be noted that a spectrogram typically from a sequence of single spectra exists, wherein between the individual spectra a hopping period is defined, and where a spectrum is a certain number of Represents samples, so that a spectrum has a certain length of time, ie a block of samples of the signal is assigned. typically, is the duration that passes through the block of samples from the Spectrum is calculated, rep presented becomes, much larger than the hopping time to be in order for the required frequency resolution and in view of the required time resolution a satisfactory To get spectrogram. On the other hand, however, it can be seen that these Spectrogram representation extraordinary is redundant. For example, consider the case that a Hopping time is 10 ms, and that a spectrum is a block of samples with a temporal length of For example, based on 100 ms, each sample enters 10 consecutive spectra. The resulting redundancy especially if a larger number of instruments which drives computing time requirements to astronomical heights.

Of another is the approach, based on the entire spectrogram to work for such cases disadvantageous in which not all of a signal contained Sources are to be extracted, but only for example sources of a certain type, that is, sources that have a certain characteristic to have. Such a characteristic can be percussive sources, ie Percussion instruments, or so-called pitched instruments, which also as harmonic-sustained instruments which typical melody instruments, such as trumpet, Violin, etc. are. A procedure based on all this Working sources is too time-consuming and ultimately too too little robust, if for example only some sources, namely the Sources that are to fulfill a specific characteristic are extracted should. In this case, namely Single spectra of the spectrogram in which such sources are not or only very slightly, falsify the overall result or "washed out", since these spectra of course, the spectrogram also go into the final component analysis like the significant spectra.

WO 01/88900 A2 discloses a concept for identifying an audio content. A set of frequency subbands of the audio signal is selected. Then the subband energy is calculated for each subband. An energy flow signal is then generated for each subband. On the basis of this information, a fingerprint is then formed, which is compared to fingerprints in a database to identify an audio file.

The GB 2363227 A discloses a concept for analyzing music to determine a characteristic section. For this purpose, data is divided into segments according to a recorded structure. The sound information is then analyzed to detect an envelope, and then divided into segments based on a tendency of increasing and decreasing variations in the envelope. By comparing the segments, a characteristic area such as a bridge portion of the music piece can be identified and extracted.

The US patent US 6,140,568 A discloses a system and method for automatically detecting and identifying a plurality of frequencies simultaneously present in an audio signal as well as the durations, amplitudes and phases of those frequencies. Then, harmonic components are filtered to determine which frequencies are fundamental frequencies.

The Object of the present invention is to provide a robust and computationally efficient concept for analyzing an information signal create.

These The object is achieved by a device for analyzing an information signal according to claim 1, a method for analyzing an information signal according to claim 24 or a computer program according to claim 25 solved.

Of the The present invention is based on the finding that a This achieves robust and efficient information signal analysis will that first significant short-term spectra or significant short-term spectra derived short-term spectra, such as difference spectra etc. from the entire information signal or from the spectrogram of the information signal, such short-term spectra which come closer to a specific characteristic as other short-term spectra of the information signal.

Preferably extract short-term spectra that have percussive components, and thus short-term spectra are not extracted, the harmonic Shares have. In this case, the specific characteristic is a percussive or percussion characteristic.

The extracted short-term spectra or from the extracted short-term spectra derived short-term spectra then become a disassembly facility the short-term spectra supplied in component signal spectra, wherein a component signal spectrum is a profile spectrum of a sound source represents, which produces a tone of the desired characteristic corresponds, and wherein another component signal spectrum a another profile spectrum of a sound source that represents a sound generated, which also corresponds to the characteristic sought.

Finally will on the basis of the profile spectra of the sound sources an amplitude envelope over the Time is calculated for the calculation of the amplitude envelope over the Time the determined profile spectra as well as the original ones Short-term spectra can be used, so that for each time, to the one Short-term spectrum was decreased, also obtained an amplitude value becomes.

The information thus obtained, namely different Profile spectra and amplitude envelopes for the profile spectra a full description the music or information signal with respect to the specified Characteristic, after which has been extracted, so this information already sufficient, to make a transcription, so with concepts of feature extraction and segmentation first, which instrument "belongs" to the profile spectrum, and what rhythm exists, so what increases and waste events are present on indicate played notes of this instrument at certain times.

The The present invention is advantageous in that for the calculation the component analysis, ie the decomposition, not the entire spectrogram is used, but only extracted short-term spectra, that is the calculation of Independent Subspace Analysis (ISA) based only a subset of all spectra takes place, so the computational requirements be lowered. Furthermore, the robustness in terms of the Detecting certain sources, sometimes other short-term spectra, that do not meet the specified characteristic in component analysis are not present and thus no disturbance or "blurring" of the actual Represent spectra.

Furthermore is the inventive concept to the effect that the profile spectra directly from the Signal can be determined without the problem of prefabricated profile spectra which in turn results in either inaccurate results or too an increased Calculation work lead would.

Preferably, for the detection and classification of percussive, non-harmonic Instru The concept according to the invention is used in polyphonic audio signals in order to obtain profile spectra as well as amplitude envelopes for the individual profile spectra.

preferred embodiments The present invention will be described below with reference to FIGS attached drawings explained in detail. Show it:

1 a block diagram of the inventive apparatus for analyzing an information signal;

2 a block diagram of a preferred embodiment of the inventive apparatus for analyzing an information signal;

3a an example of an amplitude envelope for a percussive source;

3b an example of a profile spectrum for a percussive source;

4a an example of an amplitude envelope for a harmonically sustained instrument; and

4b an example of a profile spectrum for a harmoniously sustained instrument.

1 shows a preferred embodiment of a device according to the invention for analyzing an information signal via an input line 10 a facility 12 for providing a sequence of short-term spectra representing the information signal. As it is by a detour line 14 in 1 is shown, which is drawn by dashed lines, the information signal can also be, for example, in temporal form a device 16 for extracting significant short-term spectra or short-term spectra-derived short-term spectra from the information signal, wherein the means for extracting is adapted to extract such short-term spectra which are closer to a specific characteristic than other short-term spectra of the information signal.

The extracted spectra, ie the original short-term spectra or the short-term spectra derived from the original short-term spectra, for example by differentiating, differentiating and rectifying or by other operations, become a device 18 for decomposing the extracted short-term spectra into component signal spectra, wherein a component signal spectrum represents a profile spectrum of a sound source producing a sound corresponding to the sought characteristic, and wherein another profile spectrum represents another sound source producing a sound which is the same sought characteristic also corresponds.

The profile spectra eventually become a device 20 for calculating an amplitude envelope for the one audio source, the amplitude envelope indicating how the profile spectra of a sound source change over time, and in particular how the intensity or weighting of a profile spectrum changes over time. The device 20 is designed to work on the basis of the sequence of short - term spectra on the one hand, and on the other hand, based on the profile spectra, on the other hand 1 evident. On the output side, the device delivers 20 to calculate amplitude envelopes for the sources while setting up 18 Provides profile spectra for the sound sources. The profile spectra and the associated amplitude envelopes provide a complete description of the portion of the information signal that corresponds to the specific characteristic. Preferably, this portion is the percussive portion of a piece of music. Alternatively, however, this share could also be the harmonic component. In this case, the means for extracting significant short-term spectra would be designed differently, as in the case where the specific characteristic is a percussive characteristic.

Hereinafter, referring to 2 a preferred embodiment of the present invention is shown. Preferably, with the profile spectra F and the amplitude envelopes E, a detection and classification percussive, non-harmonic instruments is performed, as well as by a block 22 in 2 is shown. This will be discussed later.

Like it out 2 is apparent, is the device 12 for providing a sequence of short-term spectra to generate an amplitude spectrogram X by means of an appropriate time-frequency transformation. The time / frequency setup 12 is preferably a means for performing a short-time Fourier transform with a certain hopping period, or comprises filter banks. Optionally, a phase spectrogram is also obtained as an additional source of information, as in 2 through a phase arrow 13 is shown. Then, by differentiating along the time extent of each individual spectrogram line, ie each individual frequency bin, a difference spectrogram X is obtained. won, as by the differentiator 16a is shown. The negative components resulting from differentiation are set to zero or - alternatively - made positive. This results in a non-negative difference spectrogram X ^. This non-negative difference spectrogram is a maximum searcher 16c , which is designed to be after the times t, that is, after the indices of the corresponding spectrogram columns, the occurrence of local maxima in a detection function e, before the maximum seeker 16c is calculated to search. As will be explained later, the detection function can be obtained, for example, by summing over all rows of X 1 and then smoothing.

Optionally, it is preferred to include the phase information transmitted via the phase line 13 from the block 12 to the block 16c be used as an indicator of the reliability of the found maxima. The spectra for which the maximum seeker detects a maximum in the detection function are used as X ^ _t and represent the extracted short-term spectra.

In block 18a a Principle Component Analysis (PCA) is performed. Here, a sought number of components d is first set. Then, the PCA is performed by a suitable method such as Singular Value Decomposition or Eigenvalue Decomposition over the columns of the matrix X _t . X ~ = X ^ t * T

The transformation matrix T effects a dimensional reduction on X ~, which results in a reduction in the number of columns of this matrix. Furthermore, a decorrelation and variance normalization is achieved. In block 18b then a non-negative Independent Component Analysis is executed. Here, the method of non-negative Independent Component Analysis shown in [6] is performed on X ~ to calculate a separation matrix A. According to the following equation, X ~ is decomposed into independent components. F = A × X ~

Independent components F are interpreted as static spectral profiles or profile spectra of the occurring sound sources. In a block 20 Then the amplitude base or the amplitude envelope E is extracted according to the following equation for the individual sound sources. E = F × X

The Amplitude base is called a set of time-varying amplitude envelopes interpreted the corresponding spectral profiles.

According to the invention spectral profile obtained from the music signal itself. This will the computational complexity across from the previous method is reduced, and it is a higher robustness across from stationary Signal components, ie signal components due to Harmonic Sustained Reached instruments.

In a block 22 Then, a feature extraction and a classification operation are performed. In particular, the components are divided into two subsets, namely first in a subset with the properties not percussive, so quasi-harmonic, and in another percussive subset .. In addition, the components with the property percussive / dissonant further classified in various instrument classes.

to Classification into the two subsets will be the characteristics of percussivity or spectral Dissonance used.

The following characteristics are used for instrument classification:
smoothed version of the spectral profiles as a search pattern in a training database with profiles of individual instruments, spectral centroid, spectral propagation, spectral skew, center frequencies, intensities, extent, skewness of the clearest partial lines, ...

For example, the following instrument classes can be classified:
Kick Drum, Snare Drum, Hi-Hat, Cymbal, Tom, Bongo, Conga, Woodblock, Cowbell, Timbale, Shaker, Tabla, Tambourine, Triangle, Daburka, Castagnets, Handclaps.

In a block 24 can then be carried out to further increase the robustness of the inventive concept, a decision for drum inserts or an acceptance or acceptance of percussive maxima. Thus, maxima with a transient rise in the amplitude envelope above a variable threshold are assumed to be a percussive event, while maxima having a transient rise below the variable threshold are discarded or recognized as an artifact and ignored. The variable threshold preferably varies with the total amplitude over a larger range around the maximum. The output is in an appropriate form that associates with the time of percussive events an instrument class, an intensity, and possibly other information, such as note or rhythm information in MIDI format.

At this point it should be noted that the device 16 for extracting significant short-term spectra in order to perform this extraction on the basis of actual short-term spectra, as for example in a Short-term Fourier transform can be obtained. Particularly, in the example of application of the present invention in which the specific characteristic is the percussion characteristic, it is preferable to extract not actual short-term spectra but short-term spectra from a differentiated spectrogram, that is, differential spectra. The differentiation, as in block 16a in 2 is shown, the sequence of short-term spectra results in a sequence of derived or differentiated spectra, each (differentiated) short-term spectrum now containing the changes between an original spectrum and the next spectrum. This stationary and stable components in a signal, so for example signal components due to Harmonic Sustained instruments are robustly and reliably eliminated. This is because the differentiation emphasizes changes in the signal and suppresses equal proportions. Percussive instruments, however, are characterized by the fact that the sounds produced by these instruments are highly transient in terms of their time course.

In addition, it is preferred the PCA 18a and the non-negative ICA 18b , more generally, the decomposition operation for decomposing the extracted short-term spectra in the block 18 from 1 not with the original short-term spectra but with the derived short-term spectra. In this case, the effect is exploited that for strongly transient signals the differentiated signal is very similar to the original signal before differentiation, which is the case in particular when there are very rapid changes in a signal. This applies to percussive instruments.

It should also be noted that the device 18 for disassembling a PCA 18a with subsequent non-negative ICA ( 18b ), in any case to determine a profile spectrum performs a weighted linear compensation of the extracted spectra, which are supplied by the device performs. This means that the extracted spectra as a whole are subjected to specific weighting factors calculated by the individual methods and are combined linearly, that is to say by subtraction or addition. Therefore, at least in part, the effect is observed that the device 18 for storing the extracted short-term spectra can have a functionality counteracting the differentiation, so that the profile spectra, which are determined for the sound sources, are not differentiated profile spectra but the actual profile spectra. In any case, it has been found that the use of differentiated spectra, ie difference spectra from a difference spectrogram in conjunction with a decomposition algorithm in the device 18 based on a weighted linear combination of the individual extracted spectra, leading to profile spectra for the individual high quality and high selectivity sound sources.

Would against it only stationary Shares processed further, so is the specific characteristic not a percussive but a harmonious characteristic, thus it is preferred to preprocess the spectrogram Integration, that is to say by summation, to reach the stationary parts across from increase the transient shares. Also in this case will it prefers, using the sum spectra, so the integrated Spectrograms the profile spectra for the individual - then harmonic - sound sources to calculate.

Below, individual functionalities of the inventive concept are shown in more detail. However, in a preferred embodiment of the present invention, typical digital audio signals are still first pre-processed by the device 8th preprocessed. Furthermore, it is preferred, as a PCM audio signal, in the preprocessing device 8th is input, mono-files with a width of 16 bits per sample at a sampling frequency of 44.1 Hz. These audio signals, that is, this stream of audio samples, which may also be a stream of video samples and generally a stream of information samples, become the pre-processor 8th to perform time-domain preprocessing using software-based emulation of an acoustic effects device, often referred to as an "exciter." In this concept, the pre-processing stage amplifies 8th the high-frequency part of the audio signal. This is accomplished by performing a non-linear distortion on a high-pass-filtered version of the signal and adding the result of the distortion to the original signal. It turns out that this preprocessing is particularly favorable when hi-hats are to be judged, or similarly high-sounding low-intensity idiophones. Their energetic weight relative to the overall music signal is increased by this step, while most harmonically-endured and lower-pitched percussion instruments are not affected.

One Another positive side effect is the fact that MP3 encoded and again decoded files inherently low-pass filtered by this process were again received high frequency information.

A spectral representation of the preprocessed time signal is then made using the time / frequency means 12 receive, the preferential Example, a short-time Fourier transform (STFT) performs STFT = Short Time Fourier Transform).

to Implementation of the time / frequency facility will be a relative one size Block size of preferably 4096 values and a high overlap prefers. First will have a good spectral resolution for the lower frequency range, ie for the lower spectral coefficient needed. Further becomes the temporal resolution to a desired Increased accuracy, by a small hop size, so get a small hop interval between adjacent blocks becomes. In the preferred embodiment be how it's done 4096 samples per block of a short-term Fourier transform subjected, which corresponds to a time block length of 92 ms. When Hop size will be one Value of 10 ms used. This means that each sample is over 9 times occurs consecutively in a short-term spectra.

The device 12 is designed to obtain an amplitude spectrum X. The phase information can also be calculated and, as will be explained later, in the extreme value or maximum searcher 16c be used.

The magnitude spectrum X now has n frequency bins or frequency coefficients and m columns or frames, ie individual short-time spectra. The time-variant changes of each spectral coefficient are differentiated over all frames or individual spectra, by the differentiator 16a to decimate the influence of harmonic-sustaining sound sources, and to facilitate the subsequent detection of transients. The differentiation, which preferably has a difference between two short-term spectra of the sequence, may also have certain normalizations.

It should be noted that the differentiation can lead to negative values, so that in a block 16b a half-wave rectification is performed to remove this effect. Alternatively, however, the negative signs could simply be reversed, which is not preferred in view of the later component decomposition.

Due to the rectifier 16b Thus, a non-negative difference spectrogram is obtained, which is the maximum finder 16c is supplied.

The maximum viewfinder 16c performs an event detection, which will be discussed below. The detection of multiple local extremes and preferably local maxima associated with transient deployment events in the music signal is performed by first defining a time tolerance that separates two consecutive percussion inserts. In the preferred embodiment, a time of 68 ms is used as a constant value derived from the time resolution and knowledge about the music signal. In particular, this value determines the number of individual spectra or differentiated individual spectra which must occur at least between two consecutive inserts. The use of this minimum distance is also supported by the observation that a sixteenth note takes 60 ms at an upper tempo limit of a very high tempo of 250 bpm.

Around to be able to perform an automated maximum search, is differentiated from the rectified spectrum, ie of the sequence of rectified (different) short-term spectra derived a detection function, based on which the maximum search can be performed. To everyone Time to get a value of this function is simply a Sum over all frequency coefficients or all spectral bins determined. to smoothing this then resulting one - dimensional function over the Time will be a convolution of the obtained function with a suitable Hann window carried out, so that a relatively smooth function e is obtained. To the positions t of the maxima, a sliding window of the tolerance length over the entire way e "pushed" to the ability to reach a maximum per step.

The reliability The Maximasuche is improved by the fact that preferably only the Maxima are kept in a window for more than a date, since they are most likely the ones interested Peaks are. So it is preferable to use the maxima that have one predetermined threshold of times, for example three times, represent a maximum, the threshold ultimately being determined by the ratio of block length will depend on hop size. from that It can be seen that a maximum, if it really is a significant Maximum is, a certain number of times, so ultimately a certain number of overlapping ones Spectra actually must be a maximum when it is thought that in the before numerical values shown each sample in at least 9 consecutive Short-term spectra "mixed in".

In the preferred embodiment of the present invention, as represented by the phase arrow, the unwrapped phase information of the original spectrogram is used as the reliability function. It has been found that in the phase information a significantly positive-directed Phase jump must occur in addition to an estimated insertion time t, thereby avoiding that small ripples are mistaken for "onsets".

According to the invention will now a small section of the difference spectrogram, namely a through Differentiation emerged short-term spectrum extracted and the fed to subsequent decomposition device.

Below is on the functionality of the device 18a to perform a principal component analysis. From the steps described in the previous section, the information about the time of occurrence t and the spectral compositions of the inserts, ie the extracted short-term spectra X _t , are derived. For real music signals, one typically finds a large number of transient events within the duration of the piece of music. Even with a simple example of a piece at a speed of 120 beats per minute (bpm), it turns out that in a four-minute segment, 480 events can be set, assuming that only quarter notes occur. With regard to the goal of finding only a few significant subspaces or profile spectra, the principal component analysis (PCA) on X ~ _t is thus applied to the extracted short-term spectra or to short-term spectra derived from the extracted short-term spectra.

Using this known technique, it is possible to reduce the entire set of collected short-term spectra to a limited number of decorrelated principal components, resulting in a good representation of the original data with little reconstruction error. For this purpose, an eigenvalue decomposition (EVD) of the covariance matrix of the data set is calculated. From the set of eigenvectors, the eigenvectors having the d largest eigenvalues are selected to provide the coefficients for the linear combination of the original vectors according to the following equation X ~ = X ^ t * T

Therefore, T describes a transformation matrix that is actually a subset of the manifold of eigenvectors. In addition, the reciprocal values of the eigenvalues are used as scaling factors, which not only results in a decorrelation, but also provides a variance normalization, which in turn results in a whitening effect. Alternatively, a singular value decomposition (SVD) of X ~ _t may be used. It has been found that the SVD is equivalent to the PCA with EVD. The whitened components X ~ are subsequently placed in the ICA stage 18b fed into, which will be discussed below.

Generally said Independent Component Analysis (ICA) is a technique which is used to make a set of linear mixed signals in theirs original To decompose sources or component signals. A requirement for optimal behavior of the algorithm is the statistical independence of the sources. Preferably becomes a non-negative ICA uses the intuitive concept of optimizing a Builds cost function that describes the non-negativity of the components. This cost function is related to a reconstruction error, by axis pair rotations of two or more variables in the positive quadrant of the common probability density function (PDF) introduced becomes. The assumptions for This model implies that the original source signals are positive are at zero and have a nonzero PDF, and that they're up to a certain extent are linearly independent. The first concept is always fulfilled, since the vectors subjected to the ICA are differentiated and halfwave-balanced version X ^ of the original Spectrogram X results, which thus never values smaller as zero, but certainly values equal to zero. The second Limitation is taken into account if the spectra collected at operating times are the linear combinations a small set of original ones Source spectra are considered, the instruments considered characterize. This of course means a pretty rough approximation, However, in the large number of cases it turns out to be sufficiently good.

Furthermore, it is preferably used that the spectra having inserts, and in particular the spectra of actual percussion instruments, have no invariant structures, but are not subject to any changes in their spectral composition here. Nevertheless, it can be assumed that there are characteristic properties characteristic of spectral profiles of percussion sounds, thus allowing the whitened components X - to be separated into their potential source or profile spectra F according to the following equation. F = A × X ~

A denotes a d × d demixing matrix which is determined by the ICA process which actually separates the individual components X ~. The sources F are also called profile spectra in this document. Each profile spectrum has n frequency bins in the same way as a spectrum of the original spectrogram, but for all times except the amplitude normalization, it is the Amplitude envelope - identical. This means that such a profile spectrum contains only the spectral information related to an onset spectrum of an instrument. In order to preferably circumvent any scaling of the components introduced by PCA and ICA, a transformation matrix R is used according to the following equation: R = T · A T

The normalization of R with its absolute maximum value results in weighting coefficients in a range of -1 to +1, so that spectral profiles extracted using the equation below F = X ~ t · R

values in the range of the original spectrogram to have. Another standardization is made by dividing each spectral profile achieved through its L2 standard.

As it already executed before has been the adoption of independence and acceptance of the Invariance for given short-term spectra is not always 100% fulfilled. So it is no surprise that the spectral profiles obtained after demixing are still certain dependencies can have. However, this should not be considered a faulty behavior. Have tests with spectral profiles of individual drum sounds demonstrated that the spectral profiles also have a strong dependence between the use spectra of different percussive instruments to have. A way to measure the degree of overlap and similarity Along the frequency axis is to perform crosstalk measurements. For illustrative purposes, the Spectral profiles obtained from the ICA process as a transfer function considered by highly frequency selective parts in a filter bank being overlapping Passbands can lead to crosstalk in the output of the filter bank channels. The crosstalk measure between two spectral profiles is calculated according to the following Equation calculated.

In From the above equation, i ranges from 1 to d, j ranges from 1 to d and holds that j is not equal to i. In fact, this value is based on the known cross-correlation coefficients, the same however, uses a different standardization.

Based on the particular profile spectra is now in the block 20 from 2 an amplitude envelope determination performed. This is the original spectrogram, ie the result of eg by the device 12 from 1 or in time / frequency / converter 12 from 2 obtained short-term spectra used. The following equation applies: E = F × X

As a second information source, the differentiated version of the amplitude envelopes can also be determined from the difference spectrogram according to the following equation: E ^ = F · X ^

Essential At this concept is that no further ICA calculation with the amplitude envelopes carried out becomes. Instead, the concept according to the invention makes highly specialized spectral profiles that are very close to the spectra of the instruments, actually appear in the signal. Nevertheless, the extracted amplitude envelopes are only in certain cases nice detection functions with sharp points, for example, for dance-oriented music with very dominant percussive rhythm parts. Often they contain amplitude envelopes smaller peaks and plateaus, from the crosstalk effects mentioned above resulting can.

Below is a closer implementation of the device 22 for feature extraction and classification. It is known that the actual number of components for real music signals is initially unknown. "Components" in this context mean both the spectral profiles and the corresponding amplitude envelopes.If the number d of components extracted is too low, artefacts of the disregarded components will most likely occur in other components, whereas if too many components are extracted, the Unfortunately, this split can occur even with the right number of components and sometimes make capturing the real components harder.

To overcome this problem, a maximum number d of components in the PCA or ICA process is specified. Subsequently, the extracted components are classified using a set of spectral-based and time-based features. The classification should provide two pieces of information. First, the components are to be eliminated from the further process, which with high certainty as non-percussive be recognized. Furthermore, the remaining components are to be assigned to predefined instrument classes.

A suitable measure of the discrimination of the amplitude envelopes is given by the percussivity, which is mentioned in the third technical publication. Here, a modified version is used, using the correlation coefficient between corresponding amplitude envelopes in E ^ and E. The degree of correlation between both vectors tends to be small when the characteristic plateaus, which are related to harmonically sustained tones, appear in the non-differentiated amplitude envelopes E. These probably disappear in the differentiated version E ^. Both vectors are much more similar in the case of transient amplitude envelopes derived from percussive tones. For this purpose is on 3a and 4a directed. In 3a is shown a very fast and very high amplitude envelope for a percussive source, while in 4a an amplitude envelope for a harmonically sustained instrument is shown. 3a is an amplitude envelope for a kick drum while 4a is an amplitude envelope for a trumpet. The amplitude envelope for the trumpet shows a relatively rapid rise, and then a relatively slow decay, as is typical of harmonic sustained instruments. In contrast, the amplitude envelope for a percussive element increases, as in 3a is shown to be very fast and very strong, but also drops again just as fast and steeply, as a drum sound typically does not fade away very long due to the nature of the generation of that sound.

The amplitude envelopes can thus be used for classification or feature extraction just as well as the profile spectra explained below, which in the case of a percussive source ( 3b ; Hi-hat) and 4b clearly distinguish in the case of a harmonically sustained instrument (guitar). For example, the harmonically sustained instrument shows a clear manifestation of the harmonics, whereas the percussive source has a rather noisy spectrum, which does not have pronounced harmonics, but overall has an area in which energy is concentrated Energy is concentrated, is very broadband.

It is therefore preferably a spectrally-based measure, ie a measure of the profile spectra (eg 3b and 4b ) is used to separate spectra of harmonic sustained tones from spectra related to percussive sounds. Again, in the preferred embodiment, a modified version of the calculation of this measure is used, showing tolerance to spectral lag phenomena, all harmonics dissonance and proper normalization. A higher degree of computational efficiency is achieved by replacing an original dissonance function with a frequency pair weighting matrix.

The Assignment of spectral profiles to a-prioridefined classes of percussive instruments is replaced by a simple classifier Classify the k next Neighbors with spectral profiles of individual instruments as a training database created. The distance function becomes at least one correlation coefficient calculated between a query profile and a database profile. To the classification in cases low reliability, so at low correlation coefficients, verify, or to verify a multiple occurrence of the same instruments, will be additional Features that provide detailed information about the shape of the spectral profile deliver, extracted. These include those already mentioned above individual characteristics.

Below will continue on the functionality of the decider 24 in 2 received. Drum-type inserts are used in the amplitude envelopes, such as in the amplitude envelope in 3a , using conventional peak selection techniques, also referred to as peak picking. Only peaks in a tolerance range in addition to the original times t, ie the times in which the maximum seeker 16c a result, are considered as candidates for missions. Remaining peaks extracted from the amplitude envelopes are first stored for further consideration. The value of the amount of the amplitude envelope is assigned to each insert candidate at its position. If this value does not exceed a predetermined dynamic threshold, then the mission is not accepted. The threshold varies over the amount of energy in a larger time range surrounding the inserts. Most of the crosstalk influence of harmonically sustained instruments as well as simultaneously playing percussive instruments can be reduced in this step. Further, it is preferred to distinguish whether concurrent inserts of different percussive instruments actually exist or only exist due to crosstalk effects. A solution to this problem is preferably to accept these further occurrence events whose value is relatively high compared to the value of the strongest instrument at the time of use.

According to the invention thus an automatic detection and preferably also an automatic classification of non-pitched percussion sive instruments in real polyphonic music signals, the starting point for this being the profile spectra on the one hand and the amplitude envelope on the other hand. From the percussive instruments, the rhythmic information of a piece of music can be well extracted, which in turn should lead to a favorable note-to-note transcription.

Depending on the circumstances of the inventive method for analyzing an information signal implemented in hardware or in software become. Implementation can be performed on a digital storage medium, in particular a floppy disk or CD with electronically readable Control signals are made that way with a programmable computer system can work together that the procedure is carried out becomes. Generally, the invention thus also consists in a computer program product with a program code stored on a machine-readable carrier for execution of the procedure when the computer program product runs on a computer. In other In words Thus, the invention can be thought of as a computer program with a program code to carry out the process can be realized when the computer program is up a computer expires.

Claims (25)

Device for analyzing an information signal, comprising: a device ( 16 ) for extracting significant short-term spectra or short-term spectra of the information signal derived significant short-term spectra from the information signal, the device ( 16 ) for extracting to extract such short-term spectra which come closer to a specific characteristic than other short-term spectra of the information signal; a facility ( 18 ) for decomposing the extracted short-term spectra into component signal spectra, wherein a component signal spectrum represents a profile spectrum of a sound source producing a sound corresponding to the sought characteristic, and wherein another component signal spectrum represents a profile spectrum of another sound source producing a sound of the sought characteristic corresponds; and a facility ( 20 ) for calculating an amplitude envelope for the sound sources, wherein an amplitude envelope for a sound source indicates how a profile spectrum of the sound source changes over time, using the profile spectra and a sequence of short-term spectra representing the information signal. Device according to claim 1, in which the device ( 16 ) for extracting to preprocess the information signal ( 8th ) that signal components in the information signal at higher frequencies are emphasized over signal components in the information signal at lower frequencies in the information signal. Device according to Claim 2, in which the device ( 16 ) is designed to extract in order to 8th ) high-pass filtering the information signal, non-linearly distorting the high-pass-filtered version of the information signal, and adding the non-linearly distorted signal to the original information signal. Device according to one of the preceding claims, in which the device ( 16 ) for extracting the information signal of a time domain frequency domain conversion ( 12 ) to obtain a sequence of short-term spectra, wherein two temporally adjacent short-term spectra relate to sections of the information signal which overlap except for one hopping interval. Apparatus according to claim 4, wherein each short-term spectrum has a sequence of spectral coefficients, and in which the device ( 16 ) is designed to extract in order to differentiate the sequence of short-term spectra in terms of time ( 16a ) to obtain a sequence of differentiated short-term spectra, where, in a differentiated short-term spectrum, information about changes in a short-term spectrum to a temporally preceding or subsequent short-term spectrum is detected. Device according to claim 5, in which the device ( 16 ) for extracting to obtain a differentiated short-term spectrum by forming for each spectral coefficient a difference of the spectral coefficient in a current short-term spectrum and a preceding or succeeding short-term spectrum. Device according to claim 5 or 6, in which the device ( 16 ) for extracting to rectify the differentiated short-term spectra ( 16b ), so that a rectified differentiated short-term spectrum has no negative values. Device according to one of Claims 5 to 7, in which the device ( 16 ) for extracting to determine significant short-term spectra based on the differentiated short-term spectra. Device according to claim 8, wherein the device ( 16 ) is designed to extract for sum each differentiated short-term spectrum spectral coefficients or values derived from spectral coefficients from the differentiated short-term spectrum ( 16c ) in order to obtain a summation value for a short-term spectrum, so that a detection function results over time. Device according to Claim 9, in which the device ( 16 ) for extracting to smooth the detection function over time. Device according to claim 9 or 10, in which the device ( 16 ) for extracting to find maxima in the detection function at a time ( 16c ), and to use a differentiated short-term spectrum or a short-time spectrum as a significant spectrum, which is assigned a point in time at which the detection function has a maximum. Device according to one of Claims 9 to 11, in which the device ( 16 ) is designed to be extractable in order to regard as significant only the maximums of the detection function which are spaced apart from each other by more than a predefined period of time. Device according to one of Claims 4 to 12, in which the device ( 16 ) is designed for extracting in order to determine magnitude spectra as a consequence of short-time spectra and to use phase information of the short-term spectra in the extraction of the significant short-term spectra. Device according to one of the preceding claims, in which the device ( 18 ) for decomposing to add the extracted short-term spectra weighted ( 18a ) to obtain a reduced number of extracted short-term spectra. Device according to one of Claims 1 to 14, in which the device ( 18 ) is designed for disassembly to perform a principal component analysis for dimensional reduction ( 18a ) to obtain processed short-term spectra. Device according to one of the preceding claims, in which the device ( 18 ) is designed to decompose to provide independent component analysis ( 18b ) to generate a plurality of component signals, wherein a component signal is associated with an information source contributing to the information signal. Device according to one of the preceding claims, in which the device ( 20 ) for calculating the amplitude envelope is adapted to multiply a matrix comprising the profile spectra and a matrix comprising a sequence of short-term spectra of the information signal to obtain the amplitude envelopes for the sound sources. Device according to one of the preceding claims, in which the device ( 20 ) m calculating the amplitude envelope is adapted to further determine a differentiated amplitude envelope using the profile spectra for the sound sources and using the difference spectrogram. Device according to one of the preceding claims, further comprising a device ( 22 ) for classifying the component signals into percussive component signals and non-percussive component signals. Device according to Claim 19, in which the device ( 22 ) is arranged to classify on the basis of the profile spectra and / or the amplitude envelopes. Device according to Claim 19 or 20, in which the device ( 20 ) for classifying to extract a feature from the profile spectra or the amplitude envelopes and to compare them with features of known sources in a database. Device according to one of the preceding claims, further comprising a device ( 24 ) for examining the amplitude envelopes for a sound source to then accept a maximum in the amplitude envelope as a use of a signal from the sound source when the device ( 16 ) had extracted a significant short-term spectrum to extract at a time point similar to a threshold. Device according to one of the preceding claims, in which the device ( 20 ) for computing the amplitude envelope to calculate the amplitude envelope for a sound source such that the amplitude envelope indicates how an intensity or weighting of a profile spectrum of the sound source changes over time. A method of analyzing an information signal, comprising the steps of: extracting ( 16 ) from significant short-term spectra or short-term spectra of the information signal derived significant short-term spectra from the information signal, such short-term spectra are extracted, which come closer to a specific characteristic than other short-term spectra of the information signal; Disassemble ( 18 ) of the extracted short-term spectra in component signal spectra, wherein a component signal spectrum represents a profile spectrum of a sound source producing a sound which is the one sought Corresponds to a characteristic, and wherein another component signal spectrum represents a profile spectrum of another sound source producing a sound corresponding to the sought characteristic; and calculating ( 20 ) of an amplitude envelope for the sound sources, wherein an amplitude envelope for a sound source indicates how a profile spectrum of the sound source changes over time, using the profile spectra and a sequence of short-term spectra representing the information signal. Computer program with a program code to carry out the Method for analyzing an information signal according to claim 24, when the computer program runs on a computer.